2024-09-21T04:14:25.999 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T04:14:26.024 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T04:14:26.135 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701 branch: wip-guits-main-2024-09-20-2310 description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: gabrioux@ibm.com first_in_suite: false job_id: '7913701' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/squid ' name: gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-20-2310 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: mon pg warn min per osd: 0 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-guits-main-2024-09-20-2310 sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3363 sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-20-2310 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrStPiFXw1jiNX/TdWjyorrqd7w90UNmAcPuTWVWvl3zfrcyrSGG7IBFh5bUy+32oJTtMevbOgWy24Za46TlGI= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCPctkQqvhSPs/vEv+dCFwEe429fhJ8fSgOazsyiLB7BsExMivEKCRn2N17CNuI3nmgPaXC+CzvJj3YD71lhTN8= tasks: - install: branch: squid exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: squid conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:squid roleless: true - print: '**** done end installing squid cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_03:44:43 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-09-21T04:14:26.136 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa; will attempt to use it 2024-09-21T04:14:26.137 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks 2024-09-21T04:14:26.137 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T04:14:26.138 INFO:teuthology.task.internal:Checking packages... 2024-09-21T04:14:26.445 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2' 2024-09-21T04:14:26.445 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:14:26.445 INFO:teuthology.packaging:ref: None 2024-09-21T04:14:26.445 INFO:teuthology.packaging:tag: None 2024-09-21T04:14:26.445 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-20-2310 2024-09-21T04:14:26.445 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:14:26.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-20-2310 2024-09-21T04:14:26.587 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5103.gf36283e7 2024-09-21T04:14:26.589 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T04:14:26.598 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T04:14:26.598 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T04:14:26.604 INFO:teuthology.task.internal:Saving configuration 2024-09-21T04:14:26.624 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T04:14:26.631 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T04:14:26.655 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 04:09:22.628683', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrStPiFXw1jiNX/TdWjyorrqd7w90UNmAcPuTWVWvl3zfrcyrSGG7IBFh5bUy+32oJTtMevbOgWy24Za46TlGI='} 2024-09-21T04:14:26.677 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 04:09:22.630314', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCPctkQqvhSPs/vEv+dCFwEe429fhJ8fSgOazsyiLB7BsExMivEKCRn2N17CNuI3nmgPaXC+CzvJj3YD71lhTN8='} 2024-09-21T04:14:26.677 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T04:14:26.683 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-09-21T04:14:26.683 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-09-21T04:14:26.683 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T04:14:26.735 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd5ef3fc280>, signals=[15]) 2024-09-21T04:14:26.736 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T04:14:26.741 INFO:teuthology.task.internal:Opening connections... 2024-09-21T04:14:26.742 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:14:26.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:14:26.830 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T04:14:26.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:14:26.921 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T04:14:26.928 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-21T04:14:26.946 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-21T04:14:26.946 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-21T04:14:27.004 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-09-21T04:14:27.004 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-09-21T04:14:27.004 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T04:14:27.005 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T04:14:27.006 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-21T04:14:27.029 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-09-21T04:14:27.048 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-09-21T04:14:27.048 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-09-21T04:14:27.105 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T04:14:27.106 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T04:14:27.107 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T04:14:27.107 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-09-21T04:14:27.130 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T04:14:27.139 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T04:14:27.144 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T04:14:27.144 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-21T04:14:27.146 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-09-21T04:14:27.162 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T04:14:27.188 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T04:14:27.188 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-21T04:14:27.204 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-09-21T04:14:27.233 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T04:14:27.290 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T04:14:27.304 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T04:14:27.304 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T04:14:27.305 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T04:14:27.305 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T04:14:27.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:14:27.322 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-21T04:14:27.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:14:27.323 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-21T04:14:27.379 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-503.el9.x86_64 2024-09-21T04:14:27.380 INFO:teuthology.task.kernel:Running kernel on smithi053: 5.14.0-503.el9.x86_64 2024-09-21T04:14:27.380 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum install -y kernel 2024-09-21T04:14:27.381 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-503.el9.x86_64 2024-09-21T04:14:27.381 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-503.el9.x86_64 2024-09-21T04:14:27.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-09-21T04:14:29.602 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-21T04:14:31.151 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.3 MB 00:01 2024-09-21T04:14:32.696 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-21T04:14:38.537 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-21T04:14:40.213 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 19 kB 00:00 2024-09-21T04:14:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-21T04:14:41.275 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-21T04:14:44.643 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-21T04:14:46.335 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-21T04:14:46.847 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T04:14:47.373 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-21T04:14:48.829 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T04:14:48.829 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T04:14:48.930 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T04:14:48.935 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T04:14:48.936 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:14:48.936 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:14:48.936 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:14:48.936 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-09-21T04:14:48.936 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:14:48.936 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 79 M 2024-09-21T04:14:48.936 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 126 M 2024-09-21T04:14:48.936 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:14:49.341 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 119 kB/s | 36 kB 00:00 2024-09-21T04:14:52.945 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-21T04:14:54.902 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T04:14:54.902 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T04:14:54.995 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:14:55.000 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:14:55.000 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-21T04:14:55.001 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:14:55.002 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 79 M 2024-09-21T04:14:55.002 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 126 M 2024-09-21T04:14:55.002 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:14:55.555 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 119 kB/s | 36 kB 00:00 2024-09-21T04:15:04.731 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-09-21T04:15:06.279 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 1.4 MB/s | 16 MB 00:11 2024-09-21T04:15:14.061 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 1.8 MB/s | 35 MB 00:18 2024-09-21T04:15:22.421 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 885 kB/s | 29 MB 00:33 2024-09-21T04:15:25.014 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 983 kB/s | 35 MB 00:35 2024-09-21T04:15:25.014 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:15:25.015 INFO:teuthology.orchestra.run.smithi053.stdout:Total 2.2 MB/s | 79 MB 00:36 2024-09-21T04:15:25.723 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:15:26.100 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:15:26.100 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:15:27.067 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:15:27.068 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:15:29.635 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:15:30.325 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T04:15:30.422 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T04:15:31.084 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 824 kB/s | 29 MB 00:35 2024-09-21T04:15:31.084 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:15:31.085 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.2 MB/s | 79 MB 00:36 2024-09-21T04:15:31.793 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:15:32.177 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:15:32.177 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:15:33.183 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T04:15:33.211 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:15:33.211 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:15:33.318 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T04:15:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:15:36.463 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T04:15:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T04:15:38.439 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T04:15:38.477 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:15:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T04:15:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T04:15:43.621 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:15:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T04:15:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:15:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:22.456 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:22.806 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T04:16:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T04:16:22.807 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T04:16:23.663 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:23.663 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:16:23.664 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:16:23.664 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T04:16:23.664 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T04:16:23.664 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T04:16:23.664 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T04:16:23.664 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:16:23.664 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:16:24.007 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T04:16:24.089 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T04:16:24.090 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T04:16:24.090 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T04:16:24.090 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T04:16:24.090 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T04:16:24.090 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-09-21T04:16:24.091 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T04:16:24.091 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T04:16:24.148 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T04:16:24.149 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T04:16:24.149 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2024-09-21T04:16:24.180 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2024-09-21T04:16:24.180 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2024-09-21T04:16:24.180 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.06 2024-09-21T04:16:24.180 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 82.el9 2024-09-21T04:16:24.180 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2024-09-21T04:16:24.180 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Group : Unspecified 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 8274670 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : builder@centos.org 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:16:24.181 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T04:16:24.182 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T04:16:24.182 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T04:16:24.182 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2024-09-21T04:16:24.182 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:16:24.182 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T04:16:24.183 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T04:16:24.183 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 5.14.0-508.el9.x86_64 2024-09-21T04:16:24.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T04:16:24.955 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-09-21T04:16:26.322 INFO:teuthology.orchestra.run.smithi053.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T04:16:26.357 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-09-21T04:16:26.363 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-21T04:16:26.378 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.QjKdhTq6DE 2024-09-21T04:16:26.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QjKdhTq6DE 2024-09-21T04:16:26.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.QjKdhTq6DE 2024-09-21T04:16:26.594 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.QjKdhTq6DE is 6KB 2024-09-21T04:16:26.642 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.QjKdhTq6DE 2024-09-21T04:16:26.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T04:16:26.726 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T04:16:26.727 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T04:16:26.727 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T04:16:26.727 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T04:16:26.728 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T04:16:26.872 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-09-21T04:16:28.711 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:28.753 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T04:16:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T04:16:29.107 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:16:29.934 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:16:30.284 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T04:16:30.372 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T04:16:30.373 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T04:16:30.373 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T04:16:30.373 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T04:16:30.373 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T04:16:30.373 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-09-21T04:16:30.373 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T04:16:30.373 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T04:16:30.431 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T04:16:30.431 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T04:16:30.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-09-21T04:16:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-09-21T04:16:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-09-21T04:16:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-09-21T04:16:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 82.el9 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8274670 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:16:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T04:16:30.464 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T04:16:30.464 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T04:16:30.464 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-09-21T04:16:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:16:30.464 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T04:16:30.464 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T04:16:30.465 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-508.el9.x86_64 2024-09-21T04:16:30.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T04:16:31.210 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-21T04:16:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T04:16:32.630 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-21T04:16:32.631 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-21T04:16:32.646 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.mPpPwbsk6k 2024-09-21T04:16:32.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mPpPwbsk6k 2024-09-21T04:16:32.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.mPpPwbsk6k 2024-09-21T04:16:32.850 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.mPpPwbsk6k is 6KB 2024-09-21T04:16:32.898 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.mPpPwbsk6k 2024-09-21T04:16:32.913 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T04:16:32.975 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T04:16:32.975 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T04:16:32.975 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T04:16:32.975 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T04:16:32.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T04:16:33.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-21T04:16:56.899 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-09-21T04:16:56.899 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T04:16:56.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:17:03.135 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-09-21T04:17:03.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-21T04:17:03.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:17:15.247 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-09-21T04:17:21.651 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-21T04:17:24.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T04:17:24.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:17:30.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-21T04:17:30.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:17:42.639 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-09-21T04:17:54.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T04:17:54.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:17:55.066 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-21T04:17:55.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T04:17:55.489 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T04:17:55.489 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-21T04:17:55.504 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-508.el9.x86_64 2024-09-21T04:17:55.505 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T04:17:55.505 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T04:17:55.505 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T04:17:56.506 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-09-21T04:17:56.506 DEBUG:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T04:17:56.600 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2024-09-21T04:17:56.619 DEBUG:teuthology.parallel:result is None 2024-09-21T04:18:30.661 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T04:18:42.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-21T04:18:42.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:18:43.046 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-21T04:18:43.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-21T04:18:43.427 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T04:18:43.428 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-21T04:18:43.445 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-508.el9.x86_64 2024-09-21T04:18:43.445 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T04:18:43.445 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T04:18:43.445 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T04:18:44.446 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-09-21T04:18:44.446 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T04:18:44.518 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-09-21T04:18:44.541 DEBUG:teuthology.parallel:result is None 2024-09-21T04:18:44.541 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T04:18:44.548 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T04:18:44.549 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T04:18:44.551 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T04:18:44.572 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T04:18:44.584 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T04:18:44.595 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T04:18:44.595 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T04:18:44.610 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T04:18:44.643 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T04:18:44.650 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T04:18:44.651 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T04:18:44.679 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T04:18:44.712 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T04:18:44.714 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T04:18:44.723 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T04:18:44.726 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T04:18:44.728 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T04:18:44.734 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T04:18:44.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T04:18:44.768 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T04:18:44.797 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T04:18:44.805 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T04:18:44.805 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T04:18:44.837 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T04:18:44.854 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T04:18:44.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T04:18:44.984 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T04:18:45.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T04:18:45.113 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:18:45.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T04:18:45.182 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T04:18:45.228 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T04:18:45.284 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T04:18:45.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T04:18:45.414 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:18:45.414 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T04:18:45.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-21T04:18:45.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-09-21T04:18:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:18:45.577 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:18:45.986 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T04:18:45.999 INFO:teuthology.task.internal:Starting timer... 2024-09-21T04:18:45.999 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T04:18:46.008 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T04:18:46.016 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T04:18:46.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-09-21T04:18:46.058 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-09-21T04:18:46.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-09-21T04:18:46.137 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-09-21T04:18:46.139 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T04:18:46.139 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-09-21T04:18:46.168 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-09-21T04:18:46.168 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-09-21T04:18:46.211 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-09-21T04:18:46.211 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive'} 2024-09-21T04:18:46.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:18:46.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:18:46.240 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:18:46.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:18:46.281 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T04:18:46.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-09-21T04:18:46.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-09-21T04:18:46.355 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T04:18:46.364 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T04:18:46.373 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T04:18:46.374 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi053.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T04:22:35.110 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com')] 2024-09-21T04:22:35.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-21T04:22:35.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:22:35.190 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-21T04:22:35.276 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-21T04:22:35.276 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T04:22:35.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T04:22:35.355 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-21T04:22:35.440 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-21T04:22:35.440 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T04:22:35.451 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T04:22:35.451 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T04:22:35.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:22:35.454 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T04:22:35.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:22:35.485 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T04:22:35.501 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T04:22:35.513 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T04:22:35.527 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-09-21T04:22:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T04:22:35.540 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-09-21T04:22:35.555 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T04:22:35.555 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-09-21T04:22:35.567 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-09-21T04:22:35.570 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T04:22:35.582 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T04:22:35.597 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T04:22:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:22:35.639 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:22:35.639 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-21T04:22:35.639 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:22:35.639 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:22:35.639 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:22:35.640 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:22:35.661 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:22:35.665 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:22:35.665 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-21T04:22:35.665 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:22:35.665 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:22:35.665 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:22:35.665 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:22:35.666 INFO:teuthology.run_tasks:Running task install... 2024-09-21T04:22:35.679 DEBUG:teuthology.task.install:project ceph 2024-09-21T04:22:35.679 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'}} 2024-09-21T04:22:35.679 DEBUG:teuthology.task.install:config {'branch': 'squid', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'} 2024-09-21T04:22:35.679 INFO:teuthology.task.install:Using flavor: default 2024-09-21T04:22:35.686 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T04:22:35.686 INFO:teuthology.task.install:extra packages: [] 2024-09-21T04:22:35.687 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'squid', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2', 'tag': None, 'wait_for_package': False} 2024-09-21T04:22:35.687 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:22:35.687 INFO:teuthology.packaging:ref: None 2024-09-21T04:22:35.687 INFO:teuthology.packaging:tag: None 2024-09-21T04:22:35.687 INFO:teuthology.packaging:branch: squid 2024-09-21T04:22:35.687 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:22:35.687 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-21T04:22:35.689 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'squid', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2', 'tag': None, 'wait_for_package': False} 2024-09-21T04:22:35.689 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:22:35.689 INFO:teuthology.packaging:ref: None 2024-09-21T04:22:35.689 INFO:teuthology.packaging:tag: None 2024-09-21T04:22:35.689 INFO:teuthology.packaging:branch: squid 2024-09-21T04:22:35.689 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:22:35.689 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-21T04:22:35.890 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/ 2024-09-21T04:22:35.890 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-21T04:22:35.926 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/ 2024-09-21T04:22:35.926 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-21T04:22:36.000 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T04:22:36.000 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:22:36.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T04:22:36.027 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T04:22:36.027 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:22:36.027 INFO:teuthology.packaging:ref: None 2024-09-21T04:22:36.027 INFO:teuthology.packaging:tag: None 2024-09-21T04:22:36.027 INFO:teuthology.packaging:branch: squid 2024-09-21T04:22:36.028 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:22:36.028 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/squid/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T04:22:36.040 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T04:22:36.040 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:22:36.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T04:22:36.068 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T04:22:36.069 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:22:36.069 INFO:teuthology.packaging:ref: None 2024-09-21T04:22:36.069 INFO:teuthology.packaging:tag: None 2024-09-21T04:22:36.069 INFO:teuthology.packaging:branch: squid 2024-09-21T04:22:36.069 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:22:36.069 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/squid/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T04:22:36.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T04:22:36.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T04:22:36.172 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T04:22:36.214 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T04:22:36.239 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-09-21T04:22:36.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-09-21T04:22:36.280 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-09-21T04:22:36.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-09-21T04:22:36.481 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-09-21T04:22:36.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw 2024-09-21T04:22:36.523 INFO:teuthology.orchestra.run.smithi053.stdout:45 files removed 2024-09-21T04:22:36.550 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw 2024-09-21T04:22:37.076 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-21T04:22:37.116 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-21T04:22:37.361 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-21T04:22:37.409 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-21T04:22:37.636 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T04:22:37.684 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T04:22:38.303 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T04:22:39.242 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.3 MB 00:01 2024-09-21T04:22:40.121 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-21T04:22:40.952 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-21T04:22:44.513 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-21T04:22:46.256 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-21T04:22:46.328 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-21T04:22:47.160 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T04:22:47.937 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-21T04:22:49.133 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-21T04:22:52.724 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-21T04:22:54.880 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T04:22:55.062 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:22:55.065 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:22:55.065 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:22:55.065 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 11 M 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.4 M 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 21 M 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T04:22:55.066 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 751 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 507 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.3 M 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 162 k 2024-09-21T04:22:55.067 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 300 k 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout:Install 30 Packages 2024-09-21T04:22:55.068 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 2 Packages 2024-09-21T04:22:55.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:22:55.069 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 60 M 2024-09-21T04:22:55.069 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:22:56.025 INFO:teuthology.orchestra.run.smithi007.stdout:(1/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 10 MB/s | 5.4 MB 00:00 2024-09-21T04:22:56.075 INFO:teuthology.orchestra.run.smithi007.stdout:(2/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 506 kB/s | 25 kB 00:00 2024-09-21T04:22:56.176 INFO:teuthology.orchestra.run.smithi007.stdout:(3/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 7.3 MB/s | 751 kB 00:00 2024-09-21T04:22:56.276 INFO:teuthology.orchestra.run.smithi007.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 5.0 MB/s | 507 kB 00:00 2024-09-21T04:22:56.485 INFO:teuthology.orchestra.run.smithi007.stdout:(5/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 11 MB/s | 11 MB 00:00 2024-09-21T04:22:56.527 INFO:teuthology.orchestra.run.smithi007.stdout:(6/32): python3-ceph-argparse-19.1.1-373.g4e45e 1.1 MB/s | 45 kB 00:00 2024-09-21T04:22:56.593 INFO:teuthology.orchestra.run.smithi007.stdout:(7/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_64 17 MB/s | 5.3 MB 00:00 2024-09-21T04:22:56.619 INFO:teuthology.orchestra.run.smithi007.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 1.4 MB/s | 136 kB 00:00 2024-09-21T04:22:56.652 INFO:teuthology.orchestra.run.smithi007.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 2.7 MB/s | 162 kB 00:00 2024-09-21T04:22:56.678 INFO:teuthology.orchestra.run.smithi007.stdout:(10/32): python3-rados-19.1.1-373.g4e45e6b4.el9 5.3 MB/s | 321 kB 00:00 2024-09-21T04:22:56.703 INFO:teuthology.orchestra.run.smithi007.stdout:(11/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 5.9 MB/s | 300 kB 00:00 2024-09-21T04:22:56.728 INFO:teuthology.orchestra.run.smithi007.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-21T04:22:56.954 INFO:teuthology.orchestra.run.smithi007.stdout:(13/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x 15 MB/s | 21 MB 00:01 2024-09-21T04:22:57.012 INFO:teuthology.orchestra.run.smithi007.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 346 kB/s | 107 kB 00:00 2024-09-21T04:22:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 518 kB/s | 160 kB 00:00 2024-09-21T04:22:57.063 INFO:teuthology.orchestra.run.smithi007.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-21T04:22:57.104 INFO:teuthology.orchestra.run.smithi007.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-21T04:22:57.130 INFO:teuthology.orchestra.run.smithi007.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 633 kB/s | 42 kB 00:00 2024-09-21T04:22:57.163 INFO:teuthology.orchestra.run.smithi007.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-21T04:22:57.189 INFO:teuthology.orchestra.run.smithi007.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-21T04:22:57.214 INFO:teuthology.orchestra.run.smithi007.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 427 kB/s | 36 kB 00:00 2024-09-21T04:22:57.239 INFO:teuthology.orchestra.run.smithi007.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-21T04:22:57.243 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:22:57.253 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:22:57.253 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 11 M 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.4 M 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 21 M 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T04:22:57.254 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 751 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 507 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.3 M 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-21T04:22:57.255 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 162 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 300 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:22:57.256 INFO:teuthology.orchestra.run.smithi053.stdout:Install 30 Packages 2024-09-21T04:22:57.257 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 2 Packages 2024-09-21T04:22:57.257 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:22:57.257 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 60 M 2024-09-21T04:22:57.257 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:22:57.307 INFO:teuthology.orchestra.run.smithi007.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-21T04:22:57.332 INFO:teuthology.orchestra.run.smithi007.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 349 kB/s | 26 kB 00:00 2024-09-21T04:22:57.357 INFO:teuthology.orchestra.run.smithi007.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-21T04:22:57.382 INFO:teuthology.orchestra.run.smithi007.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T04:22:57.416 INFO:teuthology.orchestra.run.smithi007.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-21T04:22:57.466 INFO:teuthology.orchestra.run.smithi007.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-21T04:22:57.550 INFO:teuthology.orchestra.run.smithi007.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T04:22:57.625 INFO:teuthology.orchestra.run.smithi007.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T04:22:57.733 INFO:teuthology.orchestra.run.smithi007.stdout:(31/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 13 MB/s | 3.4 MB 00:00 2024-09-21T04:22:57.842 INFO:teuthology.orchestra.run.smithi007.stdout:(32/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 11 MB/s | 3.2 MB 00:00 2024-09-21T04:22:57.848 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:22:57.848 INFO:teuthology.orchestra.run.smithi007.stdout:Total 22 MB/s | 60 MB 00:02 2024-09-21T04:22:57.956 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:22:57.990 INFO:teuthology.orchestra.run.smithi053.stdout:(1/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 8.6 MB/s | 5.4 MB 00:00 2024-09-21T04:22:57.997 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:22:57.997 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:22:58.040 INFO:teuthology.orchestra.run.smithi053.stdout:(2/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 505 kB/s | 25 kB 00:00 2024-09-21T04:22:58.149 INFO:teuthology.orchestra.run.smithi053.stdout:(3/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 6.8 MB/s | 751 kB 00:00 2024-09-21T04:22:58.233 INFO:teuthology.orchestra.run.smithi053.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 6.0 MB/s | 507 kB 00:00 2024-09-21T04:22:58.374 INFO:teuthology.orchestra.run.smithi053.stdout:(5/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 10 MB/s | 11 MB 00:01 2024-09-21T04:22:58.424 INFO:teuthology.orchestra.run.smithi053.stdout:(6/32): python3-ceph-argparse-19.1.1-373.g4e45e 906 kB/s | 45 kB 00:00 2024-09-21T04:22:58.658 INFO:teuthology.orchestra.run.smithi053.stdout:(7/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x8 16 MB/s | 21 MB 00:01 2024-09-21T04:22:58.683 INFO:teuthology.orchestra.run.smithi053.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 527 kB/s | 136 kB 00:00 2024-09-21T04:22:58.698 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:22:58.698 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:22:58.709 INFO:teuthology.orchestra.run.smithi053.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 3.1 MB/s | 162 kB 00:00 2024-09-21T04:22:58.734 INFO:teuthology.orchestra.run.smithi053.stdout:(10/32): python3-rados-19.1.1-373.g4e45e6b4.el9 6.2 MB/s | 321 kB 00:00 2024-09-21T04:22:58.759 INFO:teuthology.orchestra.run.smithi053.stdout:(11/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 5.9 MB/s | 300 kB 00:00 2024-09-21T04:22:58.784 INFO:teuthology.orchestra.run.smithi053.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-21T04:22:58.868 INFO:teuthology.orchestra.run.smithi053.stdout:(13/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_6 8.4 MB/s | 5.3 MB 00:00 2024-09-21T04:22:58.927 INFO:teuthology.orchestra.run.smithi053.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 640 kB/s | 107 kB 00:00 2024-09-21T04:22:58.977 INFO:teuthology.orchestra.run.smithi053.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-21T04:22:59.002 INFO:teuthology.orchestra.run.smithi053.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 737 kB/s | 160 kB 00:00 2024-09-21T04:22:59.052 INFO:teuthology.orchestra.run.smithi053.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 846 kB/s | 42 kB 00:00 2024-09-21T04:22:59.111 INFO:teuthology.orchestra.run.smithi053.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-21T04:22:59.162 INFO:teuthology.orchestra.run.smithi053.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-09-21T04:22:59.187 INFO:teuthology.orchestra.run.smithi053.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-21T04:22:59.212 INFO:teuthology.orchestra.run.smithi053.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 358 kB/s | 36 kB 00:00 2024-09-21T04:22:59.254 INFO:teuthology.orchestra.run.smithi053.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-21T04:22:59.296 INFO:teuthology.orchestra.run.smithi053.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-21T04:22:59.321 INFO:teuthology.orchestra.run.smithi053.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-21T04:22:59.346 INFO:teuthology.orchestra.run.smithi053.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 288 kB/s | 26 kB 00:00 2024-09-21T04:22:59.371 INFO:teuthology.orchestra.run.smithi053.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T04:22:59.405 INFO:teuthology.orchestra.run.smithi053.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-21T04:22:59.447 INFO:teuthology.orchestra.run.smithi053.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-21T04:22:59.522 INFO:teuthology.orchestra.run.smithi053.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T04:22:59.614 INFO:teuthology.orchestra.run.smithi053.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T04:22:59.723 INFO:teuthology.orchestra.run.smithi053.stdout:(31/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 12 MB/s | 3.4 MB 00:00 2024-09-21T04:22:59.790 INFO:teuthology.orchestra.run.smithi053.stdout:(32/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 12 MB/s | 3.2 MB 00:00 2024-09-21T04:22:59.795 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:22:59.795 INFO:teuthology.orchestra.run.smithi053.stdout:Total 24 MB/s | 60 MB 00:02 2024-09-21T04:22:59.901 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:22:59.941 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:22:59.942 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:23:00.260 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:23:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-21T04:23:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-21T04:23:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-21T04:23:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T04:23:00.438 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T04:23:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T04:23:00.634 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:23:00.634 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:23:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T04:23:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-21T04:23:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T04:23:01.234 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T04:23:01.324 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 8/34 2024-09-21T04:23:01.359 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T04:23:01.405 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T04:23:01.435 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-21T04:23:01.521 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 11/34 2024-09-21T04:23:01.591 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T04:23:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T04:23:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T04:23:01.727 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-21T04:23:01.837 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-21T04:23:01.869 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-21T04:23:01.920 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-21T04:23:01.951 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-21T04:23:01.980 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-21T04:23:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-21T04:23:02.189 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:23:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-21T04:23:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-21T04:23:02.320 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-21T04:23:02.343 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T04:23:02.351 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-21T04:23:02.367 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T04:23:02.401 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T04:23:02.483 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T04:23:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-21T04:23:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-21T04:23:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T04:23:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T04:23:02.967 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-21T04:23:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-21T04:23:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T04:23:03.095 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T04:23:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T04:23:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T04:23:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-21T04:23:03.289 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-21T04:23:03.421 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 8/34 2024-09-21T04:23:03.456 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T04:23:03.519 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T04:23:03.548 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-21T04:23:03.598 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 11/34 2024-09-21T04:23:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T04:23:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T04:23:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T04:23:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-21T04:23:03.907 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-21T04:23:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-21T04:23:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-21T04:23:04.031 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-21T04:23:04.060 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-21T04:23:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-21T04:23:04.419 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-21T04:23:04.464 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T04:23:04.555 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T04:23:04.586 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-21T04:23:04.630 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-21T04:23:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-21T04:23:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T04:23:05.246 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T04:23:05.335 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-21T04:23:05.368 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-21T04:23:05.738 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T04:23:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T04:23:06.390 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T04:23:06.439 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T04:23:06.531 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T04:23:06.531 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T04:23:06.531 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T04:23:06.531 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:06.565 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T04:23:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T04:23:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T04:23:08.446 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T04:23:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T04:23:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T04:23:08.585 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T04:23:08.585 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T04:23:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:08.626 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T04:23:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T04:23:17.284 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-09-21T04:23:17.284 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-09-21T04:23:17.285 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-09-21T04:23:17.285 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-09-21T04:23:17.285 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-09-21T04:23:17.285 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-09-21T04:23:17.285 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-09-21T04:23:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:18.303 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T04:23:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T04:23:18.330 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:18.330 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T04:23:18.330 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T04:23:18.330 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T04:23:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T04:23:18.422 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T04:23:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T04:23:19.254 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T04:23:19.255 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-09-21T04:23:19.255 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-09-21T04:23:19.255 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-09-21T04:23:19.255 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-09-21T04:23:19.255 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-09-21T04:23:19.255 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-09-21T04:23:19.255 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-09-21T04:23:19.255 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T04:23:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-21T04:23:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-21T04:23:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-21T04:23:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-21T04:23:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-21T04:23:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-21T04:23:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-21T04:23:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-21T04:23:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T04:23:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-21T04:23:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-21T04:23:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T04:23:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T04:23:20.294 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:20.294 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T04:23:20.294 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T04:23:20.294 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T04:23:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T04:23:20.386 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T04:23:20.386 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T04:23:20.704 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T04:23:20.704 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:20.704 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-09-21T04:23:20.704 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.704 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.704 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T04:23:20.705 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.706 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:20.707 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T04:23:20.707 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T04:23:20.707 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T04:23:20.707 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T04:23:20.707 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T04:23:20.707 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:20.707 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:23:21.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-test 2024-09-21T04:23:21.643 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:29 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:23:21.726 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T04:23:21.726 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-21T04:23:21.726 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T04:23:21.727 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-21T04:23:21.728 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-21T04:23:21.729 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-21T04:23:21.729 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-21T04:23:21.729 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-21T04:23:21.729 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T04:23:21.729 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-21T04:23:21.729 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-21T04:23:21.779 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 49 M 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 163 k 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:23:21.780 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:21.781 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-21T04:23:21.781 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:21.781 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 49 M 2024-09-21T04:23:21.781 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 209 M 2024-09-21T04:23:21.782 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:23:21.944 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.6 MB/s | 304 kB 00:00 2024-09-21T04:23:21.986 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T04:23:22.053 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 732 kB/s | 163 kB 00:00 2024-09-21T04:23:22.786 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T04:23:22.786 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:22.786 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-09-21T04:23:22.786 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.786 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.786 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T04:23:22.787 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T04:23:22.788 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T04:23:22.789 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:22.789 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:23:23.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-test 2024-09-21T04:23:23.361 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-21T04:23:23.363 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:23.365 INFO:teuthology.orchestra.run.smithi007.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-21T04:23:23.392 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:23:23.415 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:23:23.415 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:23:23.701 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:29 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:23:23.805 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:23:23.805 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:23:23.838 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:23:23.839 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:23.839 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:23:23.839 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:23.839 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:23:23.839 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 49 M 2024-09-21T04:23:23.839 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:23:23.839 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 163 k 2024-09-21T04:23:23.839 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T04:23:23.840 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T04:23:23.840 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:23.840 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:23:23.840 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:23.840 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-09-21T04:23:23.840 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:23.840 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 49 M 2024-09-21T04:23:23.840 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 209 M 2024-09-21T04:23:23.841 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:23:24.341 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 745 kB/s | 163 kB 00:00 2024-09-21T04:23:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:23:24.423 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T04:23:24.461 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T04:23:24.488 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T04:23:24.717 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-21T04:23:24.758 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 478 kB/s | 304 kB 00:00 2024-09-21T04:23:25.634 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-21T04:23:25.636 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:25.637 INFO:teuthology.orchestra.run.smithi053.stdout:Total 27 MB/s | 49 MB 00:01 2024-09-21T04:23:25.665 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:23:25.688 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:23:25.688 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:23:26.081 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:23:26.081 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:23:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:23:26.765 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T04:23:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T04:23:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T04:23:29.374 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T04:23:29.448 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T04:23:31.714 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T04:23:31.786 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T04:23:31.964 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T04:23:31.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/4 2024-09-21T04:23:31.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/4 2024-09-21T04:23:31.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:32.261 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:23:32.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph 2024-09-21T04:23:33.222 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:41 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:23:33.364 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.4 M 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-21T04:23:33.367 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.7 M 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 17 M 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T04:23:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T04:23:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T04:23:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T04:23:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T04:23:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T04:23:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:33.370 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:23:33.370 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:33.370 INFO:teuthology.orchestra.run.smithi007.stdout:Install 37 Packages 2024-09-21T04:23:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:33.371 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 30 M 2024-09-21T04:23:33.371 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 108 M 2024-09-21T04:23:33.371 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:23:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:(1/37): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 49 kB/s | 6.6 kB 00:00 2024-09-21T04:23:33.905 INFO:teuthology.orchestra.run.smithi007.stdout:(2/37): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 5.1 MB/s | 1.5 MB 00:00 2024-09-21T04:23:33.972 INFO:teuthology.orchestra.run.smithi007.stdout:(3/37): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 6.5 MB/s | 2.4 MB 00:00 2024-09-21T04:23:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:(4/37): ceph-mgr-modules-core-19.1.1-373.g4e45e 2.9 MB/s | 249 kB 00:00 2024-09-21T04:23:34.148 INFO:teuthology.orchestra.run.smithi007.stdout:(5/37): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-21T04:23:34.248 INFO:teuthology.orchestra.run.smithi007.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-21T04:23:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T04:23:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/4 2024-09-21T04:23:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/4 2024-09-21T04:23:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T04:23:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T04:23:34.465 INFO:teuthology.orchestra.run.smithi007.stdout:(8/37): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-21T04:23:34.491 INFO:teuthology.orchestra.run.smithi007.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 704 kB/s | 135 kB 00:00 2024-09-21T04:23:34.549 INFO:teuthology.orchestra.run.smithi007.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 3.1 MB/s | 1.2 MB 00:00 2024-09-21T04:23:34.575 INFO:teuthology.orchestra.run.smithi007.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-21T04:23:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T04:23:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:34.596 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:23:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T04:23:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T04:23:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:34.597 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:23:34.608 INFO:teuthology.orchestra.run.smithi007.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 877 kB/s | 125 kB 00:00 2024-09-21T04:23:34.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph 2024-09-21T04:23:35.159 INFO:teuthology.orchestra.run.smithi007.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 63 kB/s | 35 kB 00:00 2024-09-21T04:23:35.317 INFO:teuthology.orchestra.run.smithi007.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 232 kB/s | 172 kB 00:00 2024-09-21T04:23:35.342 INFO:teuthology.orchestra.run.smithi007.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 238 kB/s | 188 kB 00:00 2024-09-21T04:23:35.489 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:41 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:23:35.629 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:23:35.631 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:35.631 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:23:35.631 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.4 M 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.7 M 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 17 M 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T04:23:35.632 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T04:23:35.633 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T04:23:35.634 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T04:23:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:35.635 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:23:35.635 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:35.635 INFO:teuthology.orchestra.run.smithi053.stdout:Install 37 Packages 2024-09-21T04:23:35.635 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:35.638 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 30 M 2024-09-21T04:23:35.638 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 108 M 2024-09-21T04:23:35.638 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:23:35.859 INFO:teuthology.orchestra.run.smithi007.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 110 kB/s | 77 kB 00:00 2024-09-21T04:23:35.909 INFO:teuthology.orchestra.run.smithi007.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 86 kB/s | 51 kB 00:00 2024-09-21T04:23:35.935 INFO:teuthology.orchestra.run.smithi007.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 298 kB/s | 22 kB 00:00 2024-09-21T04:23:35.960 INFO:teuthology.orchestra.run.smithi007.stdout:(19/37): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-09-21T04:23:36.019 INFO:teuthology.orchestra.run.smithi007.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 526 kB/s | 43 kB 00:00 2024-09-21T04:23:36.039 INFO:teuthology.orchestra.run.smithi053.stdout:(1/37): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 45 kB/s | 6.6 kB 00:00 2024-09-21T04:23:36.061 INFO:teuthology.orchestra.run.smithi007.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-09-21T04:23:36.086 INFO:teuthology.orchestra.run.smithi007.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 432 kB/s | 11 kB 00:00 2024-09-21T04:23:36.111 INFO:teuthology.orchestra.run.smithi007.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 709 kB/s | 18 kB 00:00 2024-09-21T04:23:36.136 INFO:teuthology.orchestra.run.smithi007.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-21T04:23:36.162 INFO:teuthology.orchestra.run.smithi007.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-21T04:23:36.187 INFO:teuthology.orchestra.run.smithi007.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-21T04:23:36.213 INFO:teuthology.orchestra.run.smithi007.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-21T04:23:36.222 INFO:teuthology.orchestra.run.smithi053.stdout:(2/37): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 7.2 MB/s | 2.4 MB 00:00 2024-09-21T04:23:36.238 INFO:teuthology.orchestra.run.smithi007.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-21T04:23:36.263 INFO:teuthology.orchestra.run.smithi007.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-21T04:23:36.264 INFO:teuthology.orchestra.run.smithi053.stdout:(3/37): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 4.1 MB/s | 1.5 MB 00:00 2024-09-21T04:23:36.280 INFO:teuthology.orchestra.run.smithi007.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-09-21T04:23:36.306 INFO:teuthology.orchestra.run.smithi007.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 392 kB/s | 16 kB 00:00 2024-09-21T04:23:36.331 INFO:teuthology.orchestra.run.smithi007.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-21T04:23:36.348 INFO:teuthology.orchestra.run.smithi007.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 850 kB/s | 36 kB 00:00 2024-09-21T04:23:36.348 INFO:teuthology.orchestra.run.smithi053.stdout:(4/37): ceph-mgr-modules-core-19.1.1-373.g4e45e 2.9 MB/s | 249 kB 00:00 2024-09-21T04:23:36.381 INFO:teuthology.orchestra.run.smithi007.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-21T04:23:36.398 INFO:teuthology.orchestra.run.smithi007.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-21T04:23:36.423 INFO:teuthology.orchestra.run.smithi007.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-21T04:23:36.449 INFO:teuthology.orchestra.run.smithi007.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-21T04:23:36.453 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:36.453 INFO:teuthology.orchestra.run.smithi007.stdout:Total 9.8 MB/s | 30 MB 00:03 2024-09-21T04:23:36.515 INFO:teuthology.orchestra.run.smithi053.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-21T04:23:36.610 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:23:36.655 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:23:36.655 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:23:36.715 INFO:teuthology.orchestra.run.smithi053.stdout:(6/37): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_6 34 MB/s | 17 MB 00:00 2024-09-21T04:23:36.774 INFO:teuthology.orchestra.run.smithi053.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 4.7 MB/s | 1.2 MB 00:00 2024-09-21T04:23:36.832 INFO:teuthology.orchestra.run.smithi053.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-09-21T04:23:36.858 INFO:teuthology.orchestra.run.smithi053.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 750 kB/s | 106 kB 00:00 2024-09-21T04:23:36.892 INFO:teuthology.orchestra.run.smithi053.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.1 MB/s | 125 kB 00:00 2024-09-21T04:23:36.942 INFO:teuthology.orchestra.run.smithi053.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-21T04:23:37.008 INFO:teuthology.orchestra.run.smithi053.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T04:23:37.067 INFO:teuthology.orchestra.run.smithi053.stdout:(13/37): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_ 4.6 MB/s | 4.7 MB 00:01 2024-09-21T04:23:37.092 INFO:teuthology.orchestra.run.smithi053.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 415 kB/s | 35 kB 00:00 2024-09-21T04:23:37.126 INFO:teuthology.orchestra.run.smithi053.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 934 kB/s | 172 kB 00:00 2024-09-21T04:23:37.144 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:23:37.145 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:23:37.169 INFO:teuthology.orchestra.run.smithi053.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-09-21T04:23:37.202 INFO:teuthology.orchestra.run.smithi053.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 576 kB/s | 77 kB 00:00 2024-09-21T04:23:37.227 INFO:teuthology.orchestra.run.smithi053.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 414 kB/s | 42 kB 00:00 2024-09-21T04:23:37.269 INFO:teuthology.orchestra.run.smithi053.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-21T04:23:37.294 INFO:teuthology.orchestra.run.smithi053.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-21T04:23:37.319 INFO:teuthology.orchestra.run.smithi053.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 471 kB/s | 43 kB 00:00 2024-09-21T04:23:37.353 INFO:teuthology.orchestra.run.smithi053.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-21T04:23:37.378 INFO:teuthology.orchestra.run.smithi053.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 209 kB/s | 11 kB 00:00 2024-09-21T04:23:37.403 INFO:teuthology.orchestra.run.smithi053.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 367 kB/s | 18 kB 00:00 2024-09-21T04:23:37.429 INFO:teuthology.orchestra.run.smithi053.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-21T04:23:37.489 INFO:teuthology.orchestra.run.smithi053.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 228 kB/s | 19 kB 00:00 2024-09-21T04:23:37.554 INFO:teuthology.orchestra.run.smithi053.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 157 kB/s | 20 kB 00:00 2024-09-21T04:23:37.580 INFO:teuthology.orchestra.run.smithi053.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 514 kB/s | 46 kB 00:00 2024-09-21T04:23:37.613 INFO:teuthology.orchestra.run.smithi053.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-21T04:23:37.638 INFO:teuthology.orchestra.run.smithi053.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 658 kB/s | 16 kB 00:00 2024-09-21T04:23:37.664 INFO:teuthology.orchestra.run.smithi053.stdout:(31/37): python3-cherrypy-18.6.1-2.el9.noarch.r 971 kB/s | 358 kB 00:00 2024-09-21T04:23:37.689 INFO:teuthology.orchestra.run.smithi053.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-21T04:23:37.714 INFO:teuthology.orchestra.run.smithi053.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-21T04:23:37.748 INFO:teuthology.orchestra.run.smithi053.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-21T04:23:37.765 INFO:teuthology.orchestra.run.smithi053.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-21T04:23:37.790 INFO:teuthology.orchestra.run.smithi053.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T04:23:37.815 INFO:teuthology.orchestra.run.smithi053.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-21T04:23:37.820 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:37.820 INFO:teuthology.orchestra.run.smithi053.stdout:Total 14 MB/s | 30 MB 00:02 2024-09-21T04:23:37.979 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:23:38.024 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:23:38.024 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:23:38.557 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:23:38.558 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:23:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:23:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T04:23:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T04:23:38.699 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T04:23:38.775 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T04:23:38.816 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T04:23:38.848 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T04:23:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T04:23:39.031 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T04:23:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T04:23:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T04:23:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T04:23:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T04:23:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T04:23:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T04:23:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T04:23:39.352 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T04:23:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T04:23:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T04:23:39.517 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T04:23:39.670 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T04:23:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T04:23:39.812 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T04:23:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T04:23:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T04:23:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:23:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T04:23:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T04:23:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T04:23:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T04:23:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T04:23:40.188 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T04:23:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T04:23:40.244 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T04:23:40.277 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T04:23:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T04:23:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T04:23:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T04:23:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T04:23:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T04:23:40.505 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T04:23:40.548 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T04:23:40.563 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T04:23:40.578 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T04:23:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T04:23:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T04:23:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T04:23:40.728 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 32/37 2024-09-21T04:23:40.729 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T04:23:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T04:23:40.763 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T04:23:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T04:23:40.787 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:40.787 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T04:23:40.787 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T04:23:40.787 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T04:23:40.787 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:40.813 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T04:23:40.853 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T04:23:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T04:23:41.108 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T04:23:41.189 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T04:23:41.240 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T04:23:41.277 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T04:23:41.336 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T04:23:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T04:23:41.613 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T04:23:41.656 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T04:23:41.737 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T04:23:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T04:23:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T04:23:41.997 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T04:23:42.167 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 32/37 2024-09-21T04:23:42.198 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T04:23:42.226 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T04:23:42.226 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:42.226 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T04:23:42.226 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T04:23:42.227 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T04:23:42.227 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:42.318 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T04:23:42.347 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T04:23:42.347 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:42.348 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T04:23:42.348 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T04:23:42.348 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T04:23:42.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:42.802 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T04:23:42.827 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T04:23:42.828 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:42.828 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T04:23:42.828 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T04:23:42.828 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T04:23:42.828 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:43.078 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T04:23:43.103 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T04:23:43.103 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:43.103 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T04:23:43.103 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T04:23:43.103 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T04:23:43.103 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:43.169 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T04:23:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T04:23:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T04:23:43.778 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:43.778 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T04:23:43.778 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T04:23:43.778 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T04:23:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:44.241 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T04:23:44.266 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T04:23:44.267 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:44.267 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T04:23:44.267 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T04:23:44.267 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T04:23:44.267 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:44.516 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T04:23:44.541 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T04:23:44.541 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:44.541 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T04:23:44.542 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T04:23:44.542 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T04:23:44.542 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:44.609 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T04:23:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T04:23:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T04:23:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T04:23:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T04:23:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T04:23:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T04:23:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T04:23:46.023 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T04:23:46.024 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T04:23:46.025 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T04:23:46.026 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T04:23:46.027 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:46.027 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T04:23:46.300 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T04:23:46.301 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T04:23:46.302 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T04:23:46.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-base 2024-09-21T04:23:47.027 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:55 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:23:47.119 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:23:47.164 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:23:47.165 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:23:47.165 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:23:47.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install cephadm 2024-09-21T04:23:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T04:23:47.280 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:47.280 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T04:23:47.281 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T04:23:47.282 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:47.283 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:23:47.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-base 2024-09-21T04:23:47.843 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:55 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:23:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:23:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:47.980 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:23:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 765 k 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 772 k 2024-09-21T04:23:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:23:48.242 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:23:48.253 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-21T04:23:48.253 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:48.254 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-21T04:23:48.254 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:23:48.260 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:23:48.260 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:23:48.266 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:23:48.266 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:23:48.335 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:23:48.346 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:23:48.379 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:23:48.380 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:23:48.380 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:23:48.574 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install cephadm 2024-09-21T04:23:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T04:23:48.884 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T04:23:49.176 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:55 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:23:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T04:23:49.314 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-21T04:23:49.315 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:49.316 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 765 k 2024-09-21T04:23:49.316 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 772 k 2024-09-21T04:23:49.316 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:23:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T04:23:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:49.555 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:23:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:23:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:49.555 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:23:49.592 INFO:teuthology.orchestra.run.smithi053.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-21T04:23:49.592 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:49.592 INFO:teuthology.orchestra.run.smithi053.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-21T04:23:49.593 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:23:49.598 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:23:49.599 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:23:49.604 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:23:49.605 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:23:49.681 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:23:49.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T04:23:50.183 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T04:23:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T04:23:50.390 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:58 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:23:50.530 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:23:50.531 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-21T04:23:50.531 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:50.531 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:23:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 145 k 2024-09-21T04:23:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:50.532 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:23:50.532 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:50.532 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-21T04:23:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:50.532 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 145 k 2024-09-21T04:23:50.532 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 435 k 2024-09-21T04:23:50.532 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:23:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T04:23:50.743 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 688 kB/s | 145 kB 00:00 2024-09-21T04:23:50.743 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:50.744 INFO:teuthology.orchestra.run.smithi007.stdout:Total 684 kB/s | 145 kB 00:00 2024-09-21T04:23:50.744 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:23:50.752 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:23:50.752 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:23:50.814 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:23:50.814 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:23:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T04:23:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:50.912 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:23:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:23:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:50.912 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:23:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:23:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T04:23:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T04:23:51.054 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:51.054 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T04:23:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:51.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T04:23:51.747 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:23:51.884 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 145 k 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 145 k 2024-09-21T04:23:51.885 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 435 k 2024-09-21T04:23:51.886 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:23:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T04:23:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:51.970 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:23:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:51.971 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:23:52.097 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 685 kB/s | 145 kB 00:00 2024-09-21T04:23:52.098 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:52.098 INFO:teuthology.orchestra.run.smithi053.stdout:Total 680 kB/s | 145 kB 00:00 2024-09-21T04:23:52.098 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:23:52.107 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:23:52.107 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:23:52.167 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:23:52.168 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:23:52.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr 2024-09-21T04:23:52.320 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:23:52.407 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T04:23:52.433 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T04:23:52.434 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:23:52.434 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T04:23:52.434 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:52.812 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:00 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:23:52.905 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:23:52.949 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:23:52.950 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:23:52.950 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:23:53.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T04:23:53.361 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T04:23:53.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:53.361 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:23:53.361 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:23:53.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:53.362 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:23:53.584 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr 2024-09-21T04:23:53.630 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:01 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:23:53.767 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:23:53.768 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:53.768 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:23:53.768 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:53.768 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:23:53.768 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-21T04:23:53.768 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:23:53.768 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T04:23:53.768 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-09-21T04:23:53.769 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:53.770 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 8.7 M 2024-09-21T04:23:53.770 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 105 M 2024-09-21T04:23:53.770 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:23:54.192 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:00 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:23:54.285 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:23:54.330 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:23:54.331 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:23:54.331 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:23:54.347 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): ceph-grafana-dashboards-19.1.1-373.g4e4 215 kB/s | 32 kB 00:00 2024-09-21T04:23:54.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T04:23:54.405 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): ceph-prometheus-alerts-19.1.1-373.g4e45 80 kB/s | 16 kB 00:00 2024-09-21T04:23:54.531 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-21T04:23:54.581 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-09-21T04:23:54.656 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 7.8 MB/s | 3.6 MB 00:00 2024-09-21T04:23:54.706 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.3 MB/s | 548 kB 00:00 2024-09-21T04:23:54.732 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 266 kB/s | 19 kB 00:00 2024-09-21T04:23:54.765 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-21T04:23:54.791 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-21T04:23:54.816 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-21T04:23:54.866 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 13 MB/s | 2.0 MB 00:00 2024-09-21T04:23:54.967 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.9 MB/s | 863 kB 00:00 2024-09-21T04:23:54.970 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:54.971 INFO:teuthology.orchestra.run.smithi007.stdout:Total 7.2 MB/s | 8.7 MB 00:01 2024-09-21T04:23:55.006 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:01 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:23:55.021 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:23:55.040 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:23:55.040 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:23:55.143 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:23:55.144 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T04:23:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout:Install 12 Packages 2024-09-21T04:23:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:23:55.147 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 8.7 M 2024-09-21T04:23:55.147 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 105 M 2024-09-21T04:23:55.147 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:23:55.271 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:23:55.271 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:23:55.692 INFO:teuthology.orchestra.run.smithi053.stdout:(1/12): ceph-prometheus-alerts-19.1.1-373.g4e45 103 kB/s | 16 kB 00:00 2024-09-21T04:23:55.717 INFO:teuthology.orchestra.run.smithi053.stdout:(2/12): ceph-grafana-dashboards-19.1.1-373.g4e4 172 kB/s | 32 kB 00:00 2024-09-21T04:23:55.813 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:23:55.851 INFO:teuthology.orchestra.run.smithi053.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-21T04:23:55.894 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T04:23:55.910 INFO:teuthology.orchestra.run.smithi053.stdout:(4/12): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 9.5 MB/s | 3.6 MB 00:00 2024-09-21T04:23:55.937 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T04:23:55.943 INFO:teuthology.orchestra.run.smithi053.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-09-21T04:23:55.984 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T04:23:55.993 INFO:teuthology.orchestra.run.smithi053.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-09-21T04:23:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T04:23:56.027 INFO:teuthology.orchestra.run.smithi053.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.6 MB/s | 548 kB 00:00 2024-09-21T04:23:56.052 INFO:teuthology.orchestra.run.smithi053.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-21T04:23:56.078 INFO:teuthology.orchestra.run.smithi053.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-21T04:23:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T04:23:56.120 INFO:teuthology.orchestra.run.smithi053.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 3.1 MB/s | 863 kB 00:00 2024-09-21T04:23:56.145 INFO:teuthology.orchestra.run.smithi053.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-21T04:23:56.212 INFO:teuthology.orchestra.run.smithi053.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 9.3 MB/s | 2.0 MB 00:00 2024-09-21T04:23:56.215 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:23:56.216 INFO:teuthology.orchestra.run.smithi053.stdout:Total 8.1 MB/s | 8.7 MB 00:01 2024-09-21T04:23:56.254 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T04:23:56.266 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:23:56.284 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:23:56.285 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:23:56.345 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T04:23:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T04:23:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T04:23:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 10/12 2024-09-21T04:23:56.519 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:23:56.519 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:23:57.057 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:23:57.138 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T04:23:57.182 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T04:23:57.229 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T04:23:57.266 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T04:23:57.377 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T04:23:57.515 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T04:23:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 11/12 2024-09-21T04:23:57.615 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T04:23:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T04:23:57.664 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T04:23:57.699 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T04:23:57.731 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 10/12 2024-09-21T04:23:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T04:23:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/12 2024-09-21T04:23:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/12 2024-09-21T04:23:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/12 2024-09-21T04:23:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T04:23:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T04:23:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T04:23:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T04:23:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T04:23:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T04:23:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T04:23:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T04:23:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 11/12 2024-09-21T04:23:58.890 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T04:23:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T04:23:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T04:23:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T04:23:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T04:23:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T04:23:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:23:59.070 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:23:59.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T04:23:59.907 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:07 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:00.047 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T04:24:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-09-21T04:24:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:00.050 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 38 M 2024-09-21T04:24:00.050 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 212 M 2024-09-21T04:24:00.050 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:00.311 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T04:24:00.311 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/12 2024-09-21T04:24:00.311 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T04:24:00.312 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T04:24:00.434 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-21T04:24:00.500 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-21T04:24:00.525 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 357 kB/s | 33 kB 00:00 2024-09-21T04:24:00.568 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 437 kB/s | 18 kB 00:00 2024-09-21T04:24:00.626 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-21T04:24:00.654 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T04:24:00.654 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T04:24:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:00.656 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:00.768 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-37 16 MB/s | 7.4 MB 00:00 2024-09-21T04:24:00.852 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.5 MB/s | 3.0 MB 00:00 2024-09-21T04:24:00.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T04:24:00.969 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 14 MB/s | 4.9 MB 00:00 2024-09-21T04:24:01.052 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-09-21T04:24:01.078 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 792 kB/s | 245 kB 00:00 2024-09-21T04:24:01.103 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.8 MB/s | 250 kB 00:00 2024-09-21T04:24:01.361 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 53 MB/s | 16 MB 00:00 2024-09-21T04:24:01.362 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:01.363 INFO:teuthology.orchestra.run.smithi007.stdout:Total 29 MB/s | 38 MB 00:01 2024-09-21T04:24:01.509 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:07 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:01.648 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T04:24:01.649 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout:Install 12 Packages 2024-09-21T04:24:01.650 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:01.651 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 38 M 2024-09-21T04:24:01.651 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 212 M 2024-09-21T04:24:01.651 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:01.726 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:01.753 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:01.753 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:02.094 INFO:teuthology.orchestra.run.smithi053.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-21T04:24:02.161 INFO:teuthology.orchestra.run.smithi053.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-21T04:24:02.216 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:02.217 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:02.428 INFO:teuthology.orchestra.run.smithi053.stdout:(3/12): ceph-mgr-diskprediction-local-19.1.1-37 16 MB/s | 7.4 MB 00:00 2024-09-21T04:24:02.453 INFO:teuthology.orchestra.run.smithi053.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 91 kB/s | 33 kB 00:00 2024-09-21T04:24:02.654 INFO:teuthology.orchestra.run.smithi053.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-21T04:24:02.712 INFO:teuthology.orchestra.run.smithi053.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-21T04:24:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:03.087 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T04:24:03.127 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T04:24:03.179 INFO:teuthology.orchestra.run.smithi053.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 524 kB/s | 245 kB 00:00 2024-09-21T04:24:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T04:24:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T04:24:03.371 INFO:teuthology.orchestra.run.smithi053.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-09-21T04:24:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T04:24:03.488 INFO:teuthology.orchestra.run.smithi053.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-21T04:24:03.494 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T04:24:03.555 INFO:teuthology.orchestra.run.smithi053.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 5.5 MB/s | 4.9 MB 00:00 2024-09-21T04:24:03.583 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T04:24:04.014 INFO:teuthology.orchestra.run.smithi053.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.1 MB/s | 5.1 MB 00:00 2024-09-21T04:24:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T04:24:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T04:24:05.723 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T04:24:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T04:24:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T04:24:08.463 INFO:teuthology.orchestra.run.smithi053.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.3 MB/s | 16 MB 00:04 2024-09-21T04:24:08.464 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:08.465 INFO:teuthology.orchestra.run.smithi053.stdout:Total 5.6 MB/s | 38 MB 00:06 2024-09-21T04:24:08.838 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:08.865 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:08.865 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:09.328 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:09.328 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T04:24:09.500 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T04:24:09.501 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T04:24:09.954 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T04:24:09.955 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T04:24:09.955 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T04:24:09.955 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T04:24:09.955 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T04:24:09.955 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T04:24:09.955 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:09.955 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:10.205 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T04:24:10.245 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T04:24:10.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-rook 2024-09-21T04:24:10.349 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T04:24:10.388 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T04:24:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T04:24:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T04:24:10.707 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T04:24:10.937 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:18 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:11.078 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:11.079 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:11.079 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:24:11.079 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:11.079 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:11.079 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-21T04:24:11.079 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:11.080 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:11.081 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:11.081 INFO:teuthology.orchestra.run.smithi007.stdout:Install 13 Packages 2024-09-21T04:24:11.081 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:11.081 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 2.2 M 2024-09-21T04:24:11.081 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 26 M 2024-09-21T04:24:11.081 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:11.422 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T04:24:11.537 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T04:24:11.545 INFO:teuthology.orchestra.run.smithi007.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 192 kB/s | 19 kB 00:00 2024-09-21T04:24:11.587 INFO:teuthology.orchestra.run.smithi007.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 181 kB/s | 26 kB 00:00 2024-09-21T04:24:11.612 INFO:teuthology.orchestra.run.smithi007.stdout:(3/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 295 kB/s | 49 kB 00:00 2024-09-21T04:24:11.663 INFO:teuthology.orchestra.run.smithi007.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-21T04:24:11.705 INFO:teuthology.orchestra.run.smithi007.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-21T04:24:11.738 INFO:teuthology.orchestra.run.smithi007.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 714 kB/s | 54 kB 00:00 2024-09-21T04:24:11.772 INFO:teuthology.orchestra.run.smithi007.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-21T04:24:11.797 INFO:teuthology.orchestra.run.smithi007.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-21T04:24:11.822 INFO:teuthology.orchestra.run.smithi007.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 172 kB/s | 14 kB 00:00 2024-09-21T04:24:11.856 INFO:teuthology.orchestra.run.smithi007.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T04:24:11.881 INFO:teuthology.orchestra.run.smithi007.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-21T04:24:11.907 INFO:teuthology.orchestra.run.smithi007.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-21T04:24:11.940 INFO:teuthology.orchestra.run.smithi007.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-21T04:24:11.943 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:11.943 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-09-21T04:24:12.011 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:12.028 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:12.028 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:12.126 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:12.127 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T04:24:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T04:24:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T04:24:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T04:24:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T04:24:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T04:24:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T04:24:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T04:24:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T04:24:13.936 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T04:24:13.960 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T04:24:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T04:24:14.309 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T04:24:14.343 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T04:24:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T04:24:14.420 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T04:24:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T04:24:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-21T04:24:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T04:24:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T04:24:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T04:24:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:15.568 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:15.569 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:15.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T04:24:16.457 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:24 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:16.502 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T04:24:16.502 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-21T04:24:16.502 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T04:24:16.502 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T04:24:16.502 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T04:24:16.502 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T04:24:16.502 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T04:24:16.502 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T04:24:16.503 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T04:24:16.503 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T04:24:16.503 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T04:24:16.503 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T04:24:16.598 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:16.598 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:16.599 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:16.600 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:16.600 INFO:teuthology.orchestra.run.smithi007.stdout:Install 6 Packages 2024-09-21T04:24:16.600 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:16.600 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 7.0 M 2024-09-21T04:24:16.600 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 33 M 2024-09-21T04:24:16.600 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T04:24:17.061 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T04:24:17.062 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T04:24:17.062 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T04:24:17.062 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T04:24:17.062 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T04:24:17.062 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T04:24:17.062 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:17.062 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:17.202 INFO:teuthology.orchestra.run.smithi007.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-21T04:24:17.252 INFO:teuthology.orchestra.run.smithi007.stdout:(2/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 620 kB/s | 146 kB 00:00 2024-09-21T04:24:17.303 INFO:teuthology.orchestra.run.smithi007.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T04:24:17.336 INFO:teuthology.orchestra.run.smithi007.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.0 MB/s | 548 kB 00:00 2024-09-21T04:24:17.361 INFO:teuthology.orchestra.run.smithi007.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.3 MB/s | 74 kB 00:00 2024-09-21T04:24:17.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-rook 2024-09-21T04:24:17.512 INFO:teuthology.orchestra.run.smithi007.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-21T04:24:17.515 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:17.515 INFO:teuthology.orchestra.run.smithi007.stdout:Total 7.7 MB/s | 7.0 MB 00:00 2024-09-21T04:24:17.591 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:17.603 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:17.604 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:17.710 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:17.710 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:18.019 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:24 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:18.156 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:18.158 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:24:18.158 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:18.158 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-21T04:24:18.158 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:24:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T04:24:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:18.159 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:18.160 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:18.160 INFO:teuthology.orchestra.run.smithi053.stdout:Install 13 Packages 2024-09-21T04:24:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:18.160 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 2.2 M 2024-09-21T04:24:18.160 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 26 M 2024-09-21T04:24:18.160 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:18.320 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T04:24:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T04:24:18.377 INFO:teuthology.orchestra.run.smithi053.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 238 kB/s | 26 kB 00:00 2024-09-21T04:24:18.402 INFO:teuthology.orchestra.run.smithi053.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 144 kB/s | 19 kB 00:00 2024-09-21T04:24:18.427 INFO:teuthology.orchestra.run.smithi053.stdout:(3/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 311 kB/s | 49 kB 00:00 2024-09-21T04:24:18.503 INFO:teuthology.orchestra.run.smithi053.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-21T04:24:18.528 INFO:teuthology.orchestra.run.smithi053.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-21T04:24:18.561 INFO:teuthology.orchestra.run.smithi053.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 918 kB/s | 54 kB 00:00 2024-09-21T04:24:18.595 INFO:teuthology.orchestra.run.smithi053.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-21T04:24:18.620 INFO:teuthology.orchestra.run.smithi053.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 249 kB/s | 14 kB 00:00 2024-09-21T04:24:18.646 INFO:teuthology.orchestra.run.smithi053.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.2 MB/s | 279 kB 00:00 2024-09-21T04:24:18.679 INFO:teuthology.orchestra.run.smithi053.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-21T04:24:18.713 INFO:teuthology.orchestra.run.smithi053.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.8 MB/s | 90 kB 00:00 2024-09-21T04:24:18.768 INFO:teuthology.orchestra.run.smithi053.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 486 kB/s | 59 kB 00:00 2024-09-21T04:24:18.805 INFO:teuthology.orchestra.run.smithi053.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.5 MB/s | 1.0 MB 00:00 2024-09-21T04:24:18.807 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:18.807 INFO:teuthology.orchestra.run.smithi053.stdout:Total 3.4 MB/s | 2.2 MB 00:00 2024-09-21T04:24:18.874 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:18.890 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:18.890 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:18.928 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T04:24:18.990 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:18.991 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T04:24:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T04:24:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T04:24:19.517 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:19.693 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T04:24:19.750 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T04:24:19.805 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T04:24:19.840 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T04:24:19.883 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T04:24:19.918 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T04:24:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-21T04:24:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T04:24:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T04:24:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T04:24:19.919 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T04:24:19.997 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T04:24:20.108 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T04:24:20.160 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T04:24:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T04:24:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:20.192 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T04:24:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T04:24:20.192 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T04:24:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T04:24:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T04:24:20.193 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:20.193 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:20.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-fuse 2024-09-21T04:24:20.933 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T04:24:20.989 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T04:24:21.022 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T04:24:21.061 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:29 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:21.071 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T04:24:21.100 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T04:24:21.199 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:21.200 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:21.200 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-21T04:24:21.200 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:21.200 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:21.200 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 897 k 2024-09-21T04:24:21.200 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-21T04:24:21.200 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T04:24:21.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:21.201 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:21.201 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:21.201 INFO:teuthology.orchestra.run.smithi007.stdout:Install 2 Packages 2024-09-21T04:24:21.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:21.201 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 976 k 2024-09-21T04:24:21.202 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 2.9 M 2024-09-21T04:24:21.202 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:21.489 INFO:teuthology.orchestra.run.smithi007.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 499 kB/s | 80 kB 00:00 2024-09-21T04:24:21.614 INFO:teuthology.orchestra.run.smithi007.stdout:(2/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 3.1 MB/s | 897 kB 00:00 2024-09-21T04:24:21.614 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:21.614 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.3 MB/s | 976 kB 00:00 2024-09-21T04:24:21.626 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:21.649 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:21.650 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:21.736 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:21.737 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T04:24:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T04:24:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T04:24:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:22.079 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T04:24:22.158 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T04:24:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T04:24:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:22.233 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T04:24:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T04:24:22.233 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:22.234 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:22.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T04:24:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T04:24:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-21T04:24:23.109 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:29 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T04:24:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:23.118 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T04:24:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:23.118 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:23.248 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:23.248 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:23.248 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:24:23.248 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout:Install 6 Packages 2024-09-21T04:24:23.249 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:23.250 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 7.0 M 2024-09-21T04:24:23.250 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 33 M 2024-09-21T04:24:23.250 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:23.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados-devel 2024-09-21T04:24:23.514 INFO:teuthology.orchestra.run.smithi053.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-21T04:24:23.539 INFO:teuthology.orchestra.run.smithi053.stdout:(2/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 622 kB/s | 146 kB 00:00 2024-09-21T04:24:23.590 INFO:teuthology.orchestra.run.smithi053.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-21T04:24:23.615 INFO:teuthology.orchestra.run.smithi053.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 768 kB/s | 58 kB 00:00 2024-09-21T04:24:23.640 INFO:teuthology.orchestra.run.smithi053.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-21T04:24:23.840 INFO:teuthology.orchestra.run.smithi053.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-21T04:24:23.843 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:23.843 INFO:teuthology.orchestra.run.smithi053.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-21T04:24:23.916 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:23.928 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:23.928 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:24.031 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:32 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:24.036 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:24.036 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:24.170 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 127 k 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 456 k 2024-09-21T04:24:24.171 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:24.369 INFO:teuthology.orchestra.run.smithi007.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 642 kB/s | 127 kB 00:00 2024-09-21T04:24:24.369 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:24.370 INFO:teuthology.orchestra.run.smithi007.stdout:Total 637 kB/s | 127 kB 00:00 2024-09-21T04:24:24.370 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:24.376 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:24.376 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:24.443 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:24.443 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:24.655 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T04:24:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:24.710 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T04:24:25.104 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T04:24:25.365 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T04:24:25.374 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:25.374 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:25.374 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:25.374 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:25.374 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:25.374 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:25.445 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T04:24:25.479 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T04:24:25.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs2 2024-09-21T04:24:26.212 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:34 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T04:24:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-21T04:24:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T04:24:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T04:24:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T04:24:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T04:24:26.305 INFO:teuthology.orchestra.run.smithi007.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:26.351 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:26.352 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:24:26.352 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:26.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs-devel 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T04:24:26.508 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:26.509 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:26.758 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-fuse 2024-09-21T04:24:27.028 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:35 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-21T04:24:27.168 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 32 k 2024-09-21T04:24:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 142 k 2024-09-21T04:24:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:27.331 INFO:teuthology.orchestra.run.smithi007.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 196 kB/s | 32 kB 00:00 2024-09-21T04:24:27.332 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:27.332 INFO:teuthology.orchestra.run.smithi007.stdout:Total 194 kB/s | 32 kB 00:00 2024-09-21T04:24:27.332 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:27.335 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:27.336 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:27.363 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:33 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:27.450 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:27.501 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:27.501 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:27.501 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 897 k 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout:Install 2 Packages 2024-09-21T04:24:27.502 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:27.503 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 976 k 2024-09-21T04:24:27.503 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 2.9 M 2024-09-21T04:24:27.503 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:27.709 INFO:teuthology.orchestra.run.smithi053.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 436 kB/s | 80 kB 00:00 2024-09-21T04:24:27.801 INFO:teuthology.orchestra.run.smithi053.stdout:(2/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 3.2 MB/s | 897 kB 00:00 2024-09-21T04:24:27.801 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:27.801 INFO:teuthology.orchestra.run.smithi053.stdout:Total 3.2 MB/s | 976 kB 00:00 2024-09-21T04:24:27.813 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:27.836 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:27.836 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:27.922 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:27.922 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:28.110 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:28.172 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:28.172 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:28.257 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T04:24:28.336 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T04:24:28.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados2 2024-09-21T04:24:28.958 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:36 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T04:24:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-21T04:24:29.050 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:29.095 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:29.096 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:24:29.097 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:29.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librbd1 2024-09-21T04:24:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T04:24:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:29.308 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T04:24:29.308 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:29.308 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:29.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados-devel 2024-09-21T04:24:29.766 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:29.858 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:29.904 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:29.905 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:24:29.905 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:29.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rados 2024-09-21T04:24:30.216 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:36 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:30.358 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:30.358 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:30.358 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 127 k 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 456 k 2024-09-21T04:24:30.359 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:30.547 INFO:teuthology.orchestra.run.smithi053.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 673 kB/s | 127 kB 00:00 2024-09-21T04:24:30.547 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:30.547 INFO:teuthology.orchestra.run.smithi053.stdout:Total 668 kB/s | 127 kB 00:00 2024-09-21T04:24:30.547 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:30.553 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:30.553 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:30.582 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:38 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:30.621 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:30.673 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:30.719 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:30.720 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:24:30.720 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:30.775 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:30.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rgw 2024-09-21T04:24:30.889 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:31.331 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:31.395 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:39 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:31.485 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:31.531 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:31.532 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:24:31.532 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:31.589 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:31.589 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:31.590 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:31.590 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:31.590 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:31.590 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:31.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-cephfs 2024-09-21T04:24:31.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs2 2024-09-21T04:24:32.218 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:32.312 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:32.357 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:32.358 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:24:32.359 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:32.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rbd 2024-09-21T04:24:32.480 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:38 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:32.574 INFO:teuthology.orchestra.run.smithi053.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:32.619 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:32.620 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:24:32.620 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:32.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs-devel 2024-09-21T04:24:33.035 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:41 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:33.127 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:33.173 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:33.174 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-21T04:24:33.174 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:33.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-fuse 2024-09-21T04:24:33.292 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:39 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:33.432 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:33.433 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:33.433 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-21T04:24:33.433 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:33.433 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:33.433 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-21T04:24:33.433 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:33.433 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:33.433 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:33.434 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-21T04:24:33.434 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:33.434 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 32 k 2024-09-21T04:24:33.434 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 142 k 2024-09-21T04:24:33.434 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:33.591 INFO:teuthology.orchestra.run.smithi053.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 202 kB/s | 32 kB 00:00 2024-09-21T04:24:33.591 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:33.592 INFO:teuthology.orchestra.run.smithi053.stdout:Total 200 kB/s | 32 kB 00:00 2024-09-21T04:24:33.592 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:33.595 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:33.595 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:33.607 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:33.608 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:33.905 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:41 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:34.047 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-21T04:24:34.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:34.049 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 85 k 2024-09-21T04:24:34.049 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 231 k 2024-09-21T04:24:34.049 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:34.243 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 442 kB/s | 85 kB 00:00 2024-09-21T04:24:34.243 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:34.244 INFO:teuthology.orchestra.run.smithi007.stdout:Total 438 kB/s | 85 kB 00:00 2024-09-21T04:24:34.244 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:34.322 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:34.548 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:34.548 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:34.548 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:34.548 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:34.548 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:34.548 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:34.576 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:34.741 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados2 2024-09-21T04:24:34.968 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:35.222 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:35.222 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:35.344 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:41 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:35.438 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:35.485 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:35.486 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:24:35.486 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:35.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-mirror 2024-09-21T04:24:35.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librbd1 2024-09-21T04:24:36.126 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:44 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:36.153 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:42 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:36.246 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:36.266 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:36.266 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-21T04:24:36.266 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:36.266 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-21T04:24:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:36.267 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:36.267 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:36.267 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-21T04:24:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:36.267 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 3.1 M 2024-09-21T04:24:36.267 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 13 M 2024-09-21T04:24:36.268 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:36.292 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:36.293 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:24:36.293 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:36.366 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rados 2024-09-21T04:24:36.605 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 9.2 MB/s | 3.1 MB 00:00 2024-09-21T04:24:36.605 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:36.606 INFO:teuthology.orchestra.run.smithi007.stdout:Total 9.2 MB/s | 3.1 MB 00:00 2024-09-21T04:24:36.606 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:36.616 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:36.616 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:36.701 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:36.702 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:36.968 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:42 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:37.061 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:37.107 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:37.108 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:24:37.108 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:37.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rgw 2024-09-21T04:24:37.214 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:37.242 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:37.242 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:24:37.242 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T04:24:37.242 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T04:24:37.242 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T04:24:37.243 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:37.779 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:37.871 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:37.917 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:37.918 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:24:37.918 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:37.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-cephfs 2024-09-21T04:24:38.391 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:38.392 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:38.392 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:38.392 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:38.589 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:44 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:38.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-nbd 2024-09-21T04:24:38.683 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:38.729 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:38.730 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:24:38.730 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:38.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rbd 2024-09-21T04:24:39.273 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:47 ago on Sat 21 Sep 2024 04:22:52 AM UTC. 2024-09-21T04:24:39.406 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:45 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:39.412 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-21T04:24:39.412 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-21T04:24:39.412 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:39.412 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-21T04:24:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 170 k 2024-09-21T04:24:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:39.413 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-21T04:24:39.413 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-21T04:24:39.413 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-21T04:24:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:39.413 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 170 k 2024-09-21T04:24:39.413 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 494 k 2024-09-21T04:24:39.413 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-21T04:24:39.498 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T04:24:39.543 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:39.545 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-21T04:24:39.545 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:39.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-fuse 2024-09-21T04:24:39.642 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 746 kB/s | 170 kB 00:00 2024-09-21T04:24:39.642 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:39.643 INFO:teuthology.orchestra.run.smithi007.stdout:Total 741 kB/s | 170 kB 00:00 2024-09-21T04:24:39.643 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-21T04:24:39.651 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-21T04:24:39.651 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-21T04:24:39.719 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-21T04:24:39.719 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-21T04:24:39.882 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-21T04:24:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:40.221 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:46 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:40.360 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:40.360 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-21T04:24:40.360 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:40.360 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-21T04:24:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:40.361 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:40.361 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:40.361 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-21T04:24:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:40.361 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 85 k 2024-09-21T04:24:40.361 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 231 k 2024-09-21T04:24:40.362 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:40.364 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:40.561 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 429 kB/s | 85 kB 00:00 2024-09-21T04:24:40.561 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:40.561 INFO:teuthology.orchestra.run.smithi053.stdout:Total 426 kB/s | 85 kB 00:00 2024-09-21T04:24:40.562 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:40.569 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:40.569 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:40.622 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:40.622 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:40.622 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-21T04:24:40.622 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:40.622 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:24:40.623 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-21T04:24:40.637 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:40.637 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:40.792 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:40.856 DEBUG:teuthology.parallel:result is None 2024-09-21T04:24:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:41.277 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:41.522 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:41.522 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:41.522 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:41.522 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:41.522 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:41.522 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:41.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-mirror 2024-09-21T04:24:42.369 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:48 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:42.507 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:42.508 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-21T04:24:42.508 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:42.508 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-21T04:24:42.508 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:42.509 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:42.509 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:42.509 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-21T04:24:42.509 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:42.510 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 3.1 M 2024-09-21T04:24:42.510 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 13 M 2024-09-21T04:24:42.510 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 9.0 MB/s | 3.1 MB 00:00 2024-09-21T04:24:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:Total 8.9 MB/s | 3.1 MB 00:00 2024-09-21T04:24:42.856 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:42.866 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:43.403 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:43.505 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:43.532 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:43.532 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T04:24:43.532 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T04:24:43.532 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T04:24:43.533 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T04:24:43.533 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:44.620 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:44.620 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:44.621 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:44.621 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:44.621 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:44.621 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:44.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-nbd 2024-09-21T04:24:45.503 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:51 ago on Sat 21 Sep 2024 04:22:54 AM UTC. 2024-09-21T04:24:45.646 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 170 k 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-21T04:24:45.647 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-21T04:24:45.648 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:45.648 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 170 k 2024-09-21T04:24:45.648 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 494 k 2024-09-21T04:24:45.648 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-21T04:24:45.858 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 811 kB/s | 170 kB 00:00 2024-09-21T04:24:45.859 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:24:45.859 INFO:teuthology.orchestra.run.smithi053.stdout:Total 804 kB/s | 170 kB 00:00 2024-09-21T04:24:45.860 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-21T04:24:45.868 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-21T04:24:45.868 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-21T04:24:45.935 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-21T04:24:45.936 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-21T04:24:46.095 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-21T04:24:46.209 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:46.653 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T04:24:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:46.909 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-21T04:24:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T04:24:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:24:46.909 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-21T04:24:47.172 DEBUG:teuthology.parallel:result is None 2024-09-21T04:24:47.172 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:24:47.172 INFO:teuthology.packaging:ref: None 2024-09-21T04:24:47.172 INFO:teuthology.packaging:tag: None 2024-09-21T04:24:47.172 INFO:teuthology.packaging:branch: squid 2024-09-21T04:24:47.172 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:24:47.173 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-21T04:24:47.393 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T04:24:47.419 INFO:teuthology.orchestra.run.smithi007.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-21T04:24:47.419 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-21T04:24:47.419 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-21T04:24:47.421 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T04:24:47.421 INFO:teuthology.packaging:ref: None 2024-09-21T04:24:47.421 INFO:teuthology.packaging:tag: None 2024-09-21T04:24:47.421 INFO:teuthology.packaging:branch: squid 2024-09-21T04:24:47.421 INFO:teuthology.packaging:sha1: f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:24:47.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-21T04:24:47.633 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T04:24:47.658 INFO:teuthology.orchestra.run.smithi053.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-21T04:24:47.658 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-21T04:24:47.658 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-21T04:24:47.661 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T04:24:47.661 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:24:47.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T04:24:47.692 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:24:47.692 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T04:24:47.728 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T04:24:47.728 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:24:47.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T04:24:47.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T04:24:47.830 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:24:47.830 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T04:24:47.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T04:24:47.928 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T04:24:47.928 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:24:47.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T04:24:47.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T04:24:48.024 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:24:48.025 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T04:24:48.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T04:24:48.122 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T04:24:48.123 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:24:48.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T04:24:48.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T04:24:48.218 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:24:48.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T04:24:48.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T04:24:48.313 INFO:teuthology.run_tasks:Running task print... 2024-09-21T04:24:48.321 INFO:teuthology.task.print:**** done install task... 2024-09-21T04:24:48.321 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T04:24:48.408 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'squid', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'global': {'mon pg warn min per osd': 0}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}}, 'image': 'quay.ceph.io/ceph-ci/ceph:squid', 'roleless': True, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'} 2024-09-21T04:24:48.408 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:squid 2024-09-21T04:24:48.408 INFO:tasks.cephadm:Cluster fsid is 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:24:48.408 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T04:24:48.409 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-09-21T04:24:48.409 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi007': '172.21.15.7', 'mon.smithi053': '172.21.15.53'} 2024-09-21T04:24:48.409 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T04:24:48.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-09-21T04:24:48.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-09-21T04:24:48.468 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for squid 2024-09-21T04:24:48.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:24:48.612 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83198/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5103-gf36283e7', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5103.gf36283e7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-20-2310/f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:53:49.938247', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-20-2310', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-20-2310/f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T04:24:48.812 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 4e45e6b490e6cbe082cdf155fea780438ec53fc7 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=squid 2024-09-21T04:24:48.813 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/x86_64/flavors/default/cephadm 2024-09-21T04:24:48.813 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/x86_64/flavors/default/cephadm 2024-09-21T04:24:48.813 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T04:24:49.520 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Sep 21 04:24 /home/ubuntu/cephtest/cephadm 2024-09-21T04:24:49.520 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T04:24:50.139 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-r--r--. 1 ubuntu ubuntu 784608 Sep 21 04:24 /home/ubuntu/cephtest/cephadm 2024-09-21T04:24:50.140 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T04:24:50.159 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T04:24:50.189 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:squid on all hosts... 2024-09-21T04:24:50.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid pull 2024-09-21T04:24:50.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid pull 2024-09-21T04:24:51.870 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2024-09-21T04:24:51.947 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2024-09-21T04:25:23.060 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T04:25:23.060 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)", 2024-09-21T04:25:23.060 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "164465d2334a36ec69b5e540adc75ef73c92a8a0546664850c2173107383597c", 2024-09-21T04:25:23.060 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-09-21T04:25:23.060 INFO:teuthology.orchestra.run.smithi053.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64" 2024-09-21T04:25:23.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-21T04:25:23.060 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T04:25:23.474 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:25:23.475 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)", 2024-09-21T04:25:23.475 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "164465d2334a36ec69b5e540adc75ef73c92a8a0546664850c2173107383597c", 2024-09-21T04:25:23.475 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-09-21T04:25:23.475 INFO:teuthology.orchestra.run.smithi007.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64" 2024-09-21T04:25:23.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-21T04:25:23.475 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:25:23.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-09-21T04:25:23.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-09-21T04:25:23.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-09-21T04:25:23.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-09-21T04:25:23.642 INFO:tasks.cephadm:Writing seed config... 2024-09-21T04:25:23.643 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-09-21T04:25:23.643 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-09-21T04:25:23.643 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-21T04:25:23.643 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-21T04:25:23.643 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-21T04:25:23.643 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T04:25:23.643 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T04:25:23.644 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T04:25:23.645 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T04:25:23.645 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T04:25:23.645 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T04:25:23.645 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T04:25:23.645 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:25:23.645 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T04:25:23.661 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 70154924-77d1-11ef-baf5-efdc52797490 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T04:25:23.661 DEBUG:teuthology.orchestra.run.smithi007:mon.smithi007> sudo journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service 2024-09-21T04:25:23.703 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T04:25:23.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid -v bootstrap --fsid 70154924-77d1-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:25:23.914 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-21T04:25:23.914 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:squid', '-v', 'bootstrap', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-09-21T04:25:23.914 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T04:25:23.914 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-09-21T04:25:23.939 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 5.2.2 2024-09-21T04:25:23.940 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-09-21T04:25:23.940 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-09-21T04:25:23.948 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T04:25:23.948 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T04:25:23.954 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T04:25:23.954 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-21T04:25:23.961 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-09-21T04:25:23.968 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-09-21T04:25:23.968 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-09-21T04:25:23.968 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-09-21T04:25:23.994 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 5.2.2 2024-09-21T04:25:23.994 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T04:25:23.994 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-09-21T04:25:23.994 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-09-21T04:25:24.001 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T04:25:24.001 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T04:25:24.007 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T04:25:24.007 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-21T04:25:24.014 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-09-21T04:25:24.020 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-09-21T04:25:24.021 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-09-21T04:25:24.021 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-09-21T04:25:24.021 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:24.021 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 139942680414048 on /run/cephadm/70154924-77d1-11ef-baf5-efdc52797490.lock 2024-09-21T04:25:24.021 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 139942680414048 acquired on /run/cephadm/70154924-77d1-11ef-baf5-efdc52797490.lock 2024-09-21T04:25:24.021 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-09-21T04:25:24.022 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-09-21T04:25:24.022 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-09-21T04:25:24.025 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-09-21T04:25:24.025 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-09-21T04:25:24.028 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T04:25:24.028 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T04:25:24.028 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1791sec hoplimit 64 pref medium 2024-09-21T04:25:24.031 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T04:25:24.031 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T04:25:24.031 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T04:25:24.031 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T04:25:24.031 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-21T04:25:24.031 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T04:25:24.032 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-09-21T04:25:24.032 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-09-21T04:25:24.032 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T04:25:24.032 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T04:25:24.033 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2024-09-21T04:25:24.528 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 164465d2334a36ec69b5e540adc75ef73c92a8a0546664850c2173107383597c 2024-09-21T04:25:24.528 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:squid... 2024-09-21T04:25:24.528 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T04:25:24.528 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:66069d7a5240ff24cb47e5b84cff1b0dd6013e59abb2a061101c2af76e3da9e1 2024-09-21T04:25:24.528 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T04:25:24.528 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:164465d2334a36ec69b5e540adc75ef73c92a8a0546664850c2173107383597c 2024-09-21T04:25:24.528 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T04:25:25.516 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable) 2024-09-21T04:25:25.516 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable) 2024-09-21T04:25:25.516 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T04:25:26.446 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-09-21T04:25:26.447 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-09-21T04:25:27.385 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQC2Su5mNfJ/LxAAUX1J1qtb7LW9iArgVB26Og== 2024-09-21T04:25:28.332 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQC3Su5mPVo8KhAAYNHaZ+ANqKIsVJZxYRXWAg== 2024-09-21T04:25:29.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQC4Su5mLU8UJhAAo/bnkXaeJUeNy8Izcuwilw== 2024-09-21T04:25:29.213 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-09-21T04:25:30.143 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T04:25:30.143 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T04:25:30.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:30.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T04:25:30.144 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for smithi007 [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T04:25:30.144 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-09-21T04:25:30.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:30.144 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T04:25:30.144 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:25:30.144 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-09-21T04:25:31.041 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.smithi007 on 2024-09-21T04:25:31.260 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T04:25:31.417 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T04:25:31.620 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-70154924-77d1-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-70154924-77d1-11ef-baf5-efdc52797490.target. 2024-09-21T04:25:31.620 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-70154924-77d1-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-70154924-77d1-11ef-baf5-efdc52797490.target. 2024-09-21T04:25:31.873 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007 2024-09-21T04:25:31.873 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service: Unit ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service not loaded. 2024-09-21T04:25:32.052 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-70154924-77d1-11ef-baf5-efdc52797490.target.wants/ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service → /etc/systemd/system/ceph-70154924-77d1-11ef-baf5-efdc52797490@.service. 2024-09-21T04:25:32.113 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 systemd[1]: Starting Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:25:32.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 podman[27570]: 2024-09-21 04:25:32.198599467 +0000 UTC m=+0.016719036 image pull 164465d2334a36ec69b5e540adc75ef73c92a8a0546664850c2173107383597c quay.ceph.io/ceph-ci/ceph:squid 2024-09-21T04:25:32.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 podman[27570]: 2024-09-21 04:25:32.305541911 +0000 UTC m=+0.123661479 container create 402d6f48794a8a3d03b809ecc5f86ef9864ffd60b25342053f3dfdd8a7e32875 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-4e45e6b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T04:25:32.607 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T04:25:32.607 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-09-21T04:25:32.615 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T04:25:32.615 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-21T04:25:32.615 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-09-21T04:25:32.615 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T04:25:32.615 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-09-21T04:25:32.615 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-09-21T04:25:32.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 podman[27570]: 2024-09-21 04:25:32.516037966 +0000 UTC m=+0.334157535 container init 402d6f48794a8a3d03b809ecc5f86ef9864ffd60b25342053f3dfdd8a7e32875 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-4e45e6b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T04:25:32.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 podman[27570]: 2024-09-21 04:25:32.519909355 +0000 UTC m=+0.338028914 container start 402d6f48794a8a3d03b809ecc5f86ef9864ffd60b25342053f3dfdd8a7e32875 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-4e45e6b) 2024-09-21T04:25:32.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 ceph-mon[27588]: mkfs 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:32.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 ceph-mon[27588]: mon.smithi007 is new leader, mons smithi007 in quorum (ranks 0) 2024-09-21T04:25:32.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 bash[27570]: 402d6f48794a8a3d03b809ecc5f86ef9864ffd60b25342053f3dfdd8a7e32875 2024-09-21T04:25:32.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:32 smithi007 systemd[1]: Started Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:25:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: mon.smithi007 is new leader, mons smithi007 in quorum (ranks 0) 2024-09-21T04:25:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: monmap epoch 1 2024-09-21T04:25:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: last_changed 2024-09-21T04:25:29.508493+0000 2024-09-21T04:25:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: created 2024-09-21T04:25:29.508493+0000 2024-09-21T04:25:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: min_mon_release 19 (squid) 2024-09-21T04:25:33.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: election_strategy: 1 2024-09-21T04:25:33.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-09-21T04:25:33.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: fsmap 2024-09-21T04:25:33.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T04:25:33.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: mgrmap e1: no daemons active 2024-09-21T04:25:33.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:33 smithi007 ceph-mon[27588]: from='client.? 172.21.15.7:0/1422037036' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-21T04:25:33.736 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi007 (age 0.587156s) 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T04:25:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T04:25:33.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T04:25:33.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T04:25:33.739 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:33.739 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-09-21T04:25:33.739 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T04:25:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:34.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T04:25:34.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T04:25:34.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T04:25:34.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T04:25:34.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T04:25:34.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T04:25:34.770 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-09-21T04:25:34.882 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:34 smithi007 ceph-mon[27588]: from='client.? 172.21.15.7:0/2906344270' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-21T04:25:34.882 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:34 smithi007 ceph-mon[27588]: from='client.? 172.21.15.7:0/2906344270' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T04:25:35.731 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-09-21T04:25:35.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:35 smithi007 ceph-mon[27588]: from='client.? 172.21.15.7:0/2991910164' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:25:35.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:35 smithi007 systemd[1]: Stopping Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:25:36.440 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:36 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[27584]: 2024-09-21T04:25:36.029+0000 7fcb4a4b1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi007 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:25:36.440 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:36 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[27584]: 2024-09-21T04:25:36.029+0000 7fcb4a4b1640 -1 mon.smithi007@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T04:25:36.440 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:36 smithi007 podman[27829]: 2024-09-21 04:25:36.255315566 +0000 UTC m=+0.345813291 container died 402d6f48794a8a3d03b809ecc5f86ef9864ffd60b25342053f3dfdd8a7e32875 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-4e45e6b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T04:25:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:36 smithi007 podman[27829]: 2024-09-21 04:25:36.554321035 +0000 UTC m=+0.644818759 container cleanup 402d6f48794a8a3d03b809ecc5f86ef9864ffd60b25342053f3dfdd8a7e32875 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-4e45e6b) 2024-09-21T04:25:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:36 smithi007 bash[27829]: ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007 2024-09-21T04:25:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:36 smithi007 podman[27842]: 2024-09-21 04:25:36.788696288 +0000 UTC m=+0.536605873 container remove 402d6f48794a8a3d03b809ecc5f86ef9864ffd60b25342053f3dfdd8a7e32875 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-4e45e6b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T04:25:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service: Deactivated successfully. 2024-09-21T04:25:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 systemd[1]: Stopped Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:25:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 systemd[1]: Starting Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:25:37.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 podman[27935]: 2024-09-21 04:25:37.148726204 +0000 UTC m=+0.016183856 image pull 164465d2334a36ec69b5e540adc75ef73c92a8a0546664850c2173107383597c quay.ceph.io/ceph-ci/ceph:squid 2024-09-21T04:25:37.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 podman[27935]: 2024-09-21 04:25:37.256136421 +0000 UTC m=+0.123594096 container create 1fc3cec404f2f7bde6b5e049958af9b1b1ed36ce79fd506ebec4da41abb9a88b (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, RELEASE=squid-4e45e6b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T04:25:37.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 podman[27935]: 2024-09-21 04:25:37.475106009 +0000 UTC m=+0.342563643 container init 1fc3cec404f2f7bde6b5e049958af9b1b1ed36ce79fd506ebec4da41abb9a88b (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-4e45e6b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T04:25:37.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 podman[27935]: 2024-09-21 04:25:37.478340975 +0000 UTC m=+0.345798609 container start 1fc3cec404f2f7bde6b5e049958af9b1b1ed36ce79fd506ebec4da41abb9a88b (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-4e45e6b, maintainer=Guillaume Abrioux ) 2024-09-21T04:25:37.565 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T04:25:37.797 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable), process ceph-mon, pid 2 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: pidfile_write: ignore empty --pid-file 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: load: jerasure load: lrc 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: RocksDB version: 7.9.2 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Git sha 0 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Compile date 2024-09-20 20:39:57 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: DB SUMMARY 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: DB Session ID: VEHPDHDY211Y3991BDOX 2024-09-21T04:25:37.798 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: CURRENT file: CURRENT 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi007/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi007/store.db: 000009.log size: 95809 ; 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.error_if_exists: 0 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.create_if_missing: 0 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.paranoid_checks: 1 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T04:25:37.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.env: 0x555b19628940 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.info_log: 0x555b1a38d8c0 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.statistics: (nil) 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.use_fsync: 0 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_log_file_size: 0 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.allow_fallocate: 1 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T04:25:37.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.use_direct_reads: 0 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.db_log_dir: 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.wal_dir: 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T04:25:37.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.write_buffer_manager: 0x555b1a39cdc0 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.unordered_write: 0 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T04:25:37.802 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.row_cache: None 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.wal_filter: None 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.two_write_queues: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.wal_compression: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.atomic_flush: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T04:25:37.803 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.log_readahead_size: 0 2024-09-21T04:25:37.804 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T04:25:37.804 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T04:25:37.804 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T04:25:37.804 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T04:25:37.804 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T04:25:37.804 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T04:25:37.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T04:25:37.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_background_jobs: 2 2024-09-21T04:25:37.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_background_compactions: -1 2024-09-21T04:25:37.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_subcompactions: 1 2024-09-21T04:25:37.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T04:25:37.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T04:25:37.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T04:25:37.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_open_files: -1 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_background_flushes: -1 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Compression algorithms supported: 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: kZSTD supported: 0 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: kXpressCompression supported: 0 2024-09-21T04:25:37.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: kBZip2Compression supported: 0 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: kLZ4Compression supported: 1 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: kZlibCompression supported: 1 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: kSnappyCompression supported: 1 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi007/store.db/MANIFEST-000010 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.merge_operator: 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_filter: None 2024-09-21T04:25:37.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555b1a38d520) 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_type: 0 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_block_index_type: 0 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_shortening: 1 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: checksum: 4 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: no_block_cache: 0 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache: 0x555b1a3811f0 2024-09-21T04:25:37.808 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_options: 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: capacity : 536870912 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: num_shard_bits : 4 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: strict_capacity_limit : 0 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_compressed: (nil) 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: persistent_cache: (nil) 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_size: 4096 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_size_deviation: 10 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_restart_interval: 16 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_block_restart_interval: 1 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: metadata_block_size: 4096 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: partition_filters: 0 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: use_delta_encoding: 1 2024-09-21T04:25:37.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: filter_policy: bloomfilter 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: whole_key_filtering: 1 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: verify_compression: 0 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: format_version: 5 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: enable_index_compression: 1 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_align: 0 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_auto_readahead_size: 262144 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: prepopulate_block_cache: 0 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: initial_auto_readahead_size: 8192 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression: NoCompression 2024-09-21T04:25:37.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.num_levels: 7 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:25:37.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T04:25:37.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T04:25:37.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T04:25:37.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.inplace_update_support: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.bloom_locality: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.max_successive_merges: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T04:25:37.815 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.ttl: 2592000 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.enable_blob_files: false 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.min_blob_size: 0 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T04:25:37.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi007/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f38080b9-bf82-48bb-b516-5795a3936218 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892737505128, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892737506065, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91810, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 288, "table_properties": {"data_size": 89851, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11912, "raw_average_key_size": 48, "raw_value_size": 83219, "raw_average_value_size": 336, "num_data_blocks": 12, "num_entries": 247, "num_filter_entries": 247, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726892737, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f38080b9-bf82-48bb-b516-5795a3936218", "db_session_id": "VEHPDHDY211Y3991BDOX", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726892737506161, "job": 1, "event": "recovery_finished"} 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi007/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555b1a3aee00 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: DB pointer 0x555b1a4aa000 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T04:25:37.817 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** DB Stats ** 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: L0 2/0 91.56 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Sum 2/0 91.56 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:25:37.818 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T04:25:37.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative compaction: 0.00 GB write, 27.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval compaction: 0.00 GB write, 27.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Block cache BinnedLRUCache@0x555b1a3811f0#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: starting mon.smithi007 rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi007 fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007@-1(???) e1 preinit fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007@-1(???).mds e1 new map 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007@-1(???).mds e1 print_map 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: e1 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: btime 2024-09-21T04:25:32:556900+0000 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:25:37.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: legacy client fscid: -1 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: No filesystems configured 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 bash[27935]: 1fc3cec404f2f7bde6b5e049958af9b1b1ed36ce79fd506ebec4da41abb9a88b 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 systemd[1]: Started Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mon.smithi007 is new leader, mons smithi007 in quorum (ranks 0) 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: monmap epoch 1 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: last_changed 2024-09-21T04:25:29.508493+0000 2024-09-21T04:25:37.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: created 2024-09-21T04:25:29.508493+0000 2024-09-21T04:25:37.822 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: min_mon_release 19 (squid) 2024-09-21T04:25:37.822 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: election_strategy: 1 2024-09-21T04:25:37.822 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-09-21T04:25:37.822 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: fsmap 2024-09-21T04:25:37.822 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T04:25:37.822 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:37 smithi007 ceph-mon[27954]: mgrmap e1: no daemons active 2024-09-21T04:25:38.578 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T04:25:38.591 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:25:38.591 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-09-21T04:25:38.592 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T04:25:38.592 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T04:25:38.592 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8443 ... 2024-09-21T04:25:38.824 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-70154924-77d1-11ef-baf5-efdc52797490@mgr.smithi007.huohfl 2024-09-21T04:25:38.824 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-70154924-77d1-11ef-baf5-efdc52797490@mgr.smithi007.huohfl.service: Unit ceph-70154924-77d1-11ef-baf5-efdc52797490@mgr.smithi007.huohfl.service not loaded. 2024-09-21T04:25:38.983 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-70154924-77d1-11ef-baf5-efdc52797490.target.wants/ceph-70154924-77d1-11ef-baf5-efdc52797490@mgr.smithi007.huohfl.service → /etc/systemd/system/ceph-70154924-77d1-11ef-baf5-efdc52797490@.service. 2024-09-21T04:25:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:39 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3808488171' entity='client.admin' 2024-09-21T04:25:39.510 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T04:25:39.511 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-09-21T04:25:39.518 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T04:25:39.518 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-21T04:25:39.518 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-09-21T04:25:39.519 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T04:25:39.527 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T04:25:39.528 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-09-21T04:25:39.535 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T04:25:39.536 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-21T04:25:39.536 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-09-21T04:25:39.536 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-09-21T04:25:39.536 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-09-21T04:25:39.536 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-09-21T04:25:40.423 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:40 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2606444037' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-21T04:25:40.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:40.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:25:40.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "70154924-77d1-11ef-baf5-efdc52797490", 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T04:25:40.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "smithi007" 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T04:25:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T04:25:40.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T04:25:40.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T04:25:40.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:40.663 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T04:25:32:556900+0000", 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:40.664 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T04:25:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T04:25:32.557534+0000", 2024-09-21T04:25:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:25:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T04:25:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:25:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-09-21T04:25:43.384 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:43 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1111065077' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "70154924-77d1-11ef-baf5-efdc52797490", 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T04:25:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "smithi007" 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T04:25:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T04:25:43.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T04:25:43.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T04:25:43.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T04:25:43.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T04:25:32:556900+0000", 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T04:25:43.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T04:25:32.557534+0000", 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:25:43.771 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-09-21T04:25:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: Activating manager daemon smithi007.huohfl 2024-09-21T04:25:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: mgrmap e2: smithi007.huohfl(active, starting, since 0.00291826s) 2024-09-21T04:25:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:25:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:25:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:25:44.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:25:44.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi007.huohfl", "id": "smithi007.huohfl"}]: dispatch 2024-09-21T04:25:44.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: Manager daemon smithi007.huohfl is now available 2024-09-21T04:25:44.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:25:44.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' 2024-09-21T04:25:44.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/trash_purge_schedule"}]: dispatch 2024-09-21T04:25:44.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' 2024-09-21T04:25:44.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:44 smithi007 ceph-mon[27954]: from='mgr.14100 172.21.15.7:0/3336651887' entity='mgr.smithi007.huohfl' 2024-09-21T04:25:46.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:46 smithi007 ceph-mon[27954]: mgrmap e3: smithi007.huohfl(active, since 1.00763s) 2024-09-21T04:25:46.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:46 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/4185465763' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-21T04:25:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:25:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "70154924-77d1-11ef-baf5-efdc52797490", 2024-09-21T04:25:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T04:25:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "smithi007" 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T04:25:46.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T04:25:46.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T04:25:47.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:47.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T04:25:47.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T04:25:47.000 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T04:25:32:556900+0000", 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T04:25:47.001 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T04:25:32.557534+0000", 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T04:25:47.002 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:25:47.003 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-09-21T04:25:47.904 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:47 smithi007 ceph-mon[27954]: mgrmap e4: smithi007.huohfl(active, since 2s) 2024-09-21T04:25:48.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:48 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2746914902' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-21T04:25:50.067 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T04:25:50.068 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T04:25:50.069 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T04:25:50.069 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T04:25:50.069 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T04:25:50.069 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T04:25:50.069 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-09-21T04:25:53.603 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:53 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2023215444' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-21T04:25:54.561 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:54 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2023215444' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T04:25:54.562 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:54 smithi007 ceph-mon[27954]: mgrmap e5: smithi007.huohfl(active, since 8s) 2024-09-21T04:25:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: Active manager daemon smithi007.huohfl restarted 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: Activating manager daemon smithi007.huohfl 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: mgrmap e6: smithi007.huohfl(active, starting, since 0.00334413s) 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi007.huohfl", "id": "smithi007.huohfl"}]: dispatch 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: Manager daemon smithi007.huohfl is now available 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:25:58.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:25:58.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:25:58.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:25:58.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:58 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/trash_purge_schedule"}]: dispatch 2024-09-21T04:25:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:59 smithi007 ceph-mon[27954]: Found migration_current of "None". Setting to last migration. 2024-09-21T04:25:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:25:59 smithi007 ceph-mon[27954]: mgrmap e7: smithi007.huohfl(active, since 1.00488s) 2024-09-21T04:26:01.649 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:01 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:01.649 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:01 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:01.649 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:01 smithi007 ceph-mon[27954]: mgrmap e8: smithi007.huohfl(active, since 2s) 2024-09-21T04:26:01.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:26:01.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-21T04:26:01.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T04:26:01.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "smithi007.huohfl", 2024-09-21T04:26:01.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T04:26:01.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:26:01.937 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-09-21T04:26:01.937 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 8... 2024-09-21T04:26:02.614 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:02 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1707832438' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-21T04:26:02.884 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:26:02.884 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-21T04:26:02.884 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T04:26:02.884 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:26:02.885 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 8 is available 2024-09-21T04:26:02.885 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-09-21T04:26:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:03 smithi007 ceph-mon[27954]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T04:26:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:03 smithi007 ceph-mon[27954]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T04:26:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:03 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:02] ENGINE Bus STARTING 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:03 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:02] ENGINE Serving on http://172.21.15.7:8765 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:03 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:03] ENGINE Serving on https://172.21.15.7:7150 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:03 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:03] ENGINE Bus STARTED 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:03 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:03 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:03] ENGINE Error in HTTPServer.serve 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Traceback (most recent call last): 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._run(expiration_interval) 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:26:03.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:26:03.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s = self.context.wrap_socket( 2024-09-21T04:26:03.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:26:03.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: return self.sslsocket_class._create( 2024-09-21T04:26:03.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:26:03.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self.do_handshake() 2024-09-21T04:26:03.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:26:03.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._sslobj.do_handshake() 2024-09-21T04:26:03.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:26:04.634 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:04 smithi007 ceph-mon[27954]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:04.635 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:04 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:04.635 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:04 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:26:05.157 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T04:26:05.157 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-09-21T04:26:05.556 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:05 smithi007 ceph-mon[27954]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:07.146 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:07 smithi007 ceph-mon[27954]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:07.147 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:07 smithi007 ceph-mon[27954]: Generating ssh key... 2024-09-21T04:26:07.147 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:07 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:07.147 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:07 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:07.911 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4rWqPADZMBCYjw9MPPul0Y39v83H09/oY8Cmz11c7vIoGiIuvohFh8ebszB3oetb/HEZ8j9mJazjhPA5T5N9I6NfBZo+QZ66TNUL+f6+Yg5+3NKtMZMnKw+fSXxNUiD/AJMbjLqutRGOWIY+QEzpkFxUkd+fONXy5W5g86fjLaDgBz/ajRKzfugIx6BklOUn0h5YIMMxtcSaLVcalAeteBnL86tcMvvwLIHiqdp06NtKmMnhSnoGtrdiGvigsCVW0RtRo7H4OMxsEnJDHaErHOnob//PDZOAVeaGJQOWL/rOxPu+KJVOtBfvgCUb2t2EtxqC5uY/kNXxUppgbVhwMQNdVYRlZKWo0Z2q0CMpptOs3i5L5ojEn/ddq8YfA58f+/TmvnYeOnqrKE2fLE3SSSS+b+RE0Ehg8GCpkhmYHrffhGy5uScJAawBDLfY9dYq8MO6I/sYr9R5cVaAQnC8IaaLS3gZrCiki2STY7xhGJEFKj6u2G+9+h4FFF5FApj8= ceph-70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:26:07.912 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T04:26:07.912 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T04:26:07.915 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-09-21T04:26:08.675 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:08 smithi007 ceph-mon[27954]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:09.509 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:09 smithi007 ceph-mon[27954]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:10.422 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:10 smithi007 ceph-mon[27954]: Deploying cephadm binary to smithi007 2024-09-21T04:26:13.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-09-21T04:26:13.012 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying mon service with default placement... 2024-09-21T04:26:13.670 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:13 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:13.670 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:13 smithi007 ceph-mon[27954]: Added host smithi007 2024-09-21T04:26:13.670 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:13 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:26:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T04:26:14.758 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying mgr service with default placement... 2024-09-21T04:26:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:14 smithi007 ceph-mon[27954]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:14 smithi007 ceph-mon[27954]: Saving service mon spec with placement count:5 2024-09-21T04:26:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:14 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:16.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T04:26:16.166 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying crash service with default placement... 2024-09-21T04:26:16.550 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:16 smithi007 ceph-mon[27954]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:16.551 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:16 smithi007 ceph-mon[27954]: Saving service mgr spec with placement count:2 2024-09-21T04:26:16.551 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:16 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:16.551 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:16 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:16.551 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:16 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:17.883 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:17 smithi007 ceph-mon[27954]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:17.883 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:17 smithi007 ceph-mon[27954]: Saving service crash spec with placement * 2024-09-21T04:26:17.883 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:17 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:17.883 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:17 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:17.885 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-09-21T04:26:17.885 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying ceph-exporter service with default placement... 2024-09-21T04:26:19.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-09-21T04:26:19.059 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying prometheus service with default placement... 2024-09-21T04:26:19.666 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:19 smithi007 ceph-mon[27954]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:26:19.667 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:19 smithi007 ceph-mon[27954]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:19.667 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:19 smithi007 ceph-mon[27954]: Saving service ceph-exporter spec with placement * 2024-09-21T04:26:19.667 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:19 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:20.174 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-09-21T04:26:20.174 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying grafana service with default placement... 2024-09-21T04:26:20.557 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:20 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:20.558 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:20 smithi007 ceph-mon[27954]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:20.558 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:20 smithi007 ceph-mon[27954]: Saving service prometheus spec with placement count:1 2024-09-21T04:26:20.558 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:20 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:21.383 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-09-21T04:26:21.383 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying node-exporter service with default placement... 2024-09-21T04:26:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:21 smithi007 ceph-mon[27954]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:26:22.022 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:21 smithi007 ceph-mon[27954]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:22.022 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:21 smithi007 ceph-mon[27954]: Saving service grafana spec with placement count:1 2024-09-21T04:26:22.022 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:21 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:23.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-09-21T04:26:23.185 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying alertmanager service with default placement... 2024-09-21T04:26:23.300 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:23 smithi007 ceph-mon[27954]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:23.300 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:23 smithi007 ceph-mon[27954]: Saving service node-exporter spec with placement * 2024-09-21T04:26:23.300 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:23 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:24.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:24 smithi007 ceph-mon[27954]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:26:24.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:24 smithi007 ceph-mon[27954]: from='mgr.14118 172.21.15.7:0/2379530199' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:24.417 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-09-21T04:26:25.295 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:25 smithi007 ceph-mon[27954]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:25.296 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:25 smithi007 ceph-mon[27954]: Saving service alertmanager spec with placement count:1 2024-09-21T04:26:25.296 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:25 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2641321990' entity='client.admin' 2024-09-21T04:26:26.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:26 smithi007 ceph-mon[27954]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:26:26.797 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-09-21T04:26:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:27 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3799613437' entity='client.admin' 2024-09-21T04:26:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:28 smithi007 ceph-mon[27954]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:26:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:28 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1047541824' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-21T04:26:29.268 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:29 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1047541824' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T04:26:29.269 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:29 smithi007 ceph-mon[27954]: mgrmap e9: smithi007.huohfl(active, since 29s) 2024-09-21T04:26:29.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:26:29.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T04:26:29.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T04:26:29.928 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "smithi007.huohfl", 2024-09-21T04:26:29.929 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T04:26:29.929 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:26:29.929 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-09-21T04:26:29.929 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-09-21T04:26:30.180 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:30 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1159657960' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-21T04:26:33.451 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: Active manager daemon smithi007.huohfl restarted 2024-09-21T04:26:33.451 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: Activating manager daemon smithi007.huohfl 2024-09-21T04:26:33.451 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T04:26:33.451 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: mgrmap e10: smithi007.huohfl(active, starting, since 0.00326793s) 2024-09-21T04:26:33.451 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:26:33.452 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi007.huohfl", "id": "smithi007.huohfl"}]: dispatch 2024-09-21T04:26:33.452 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:26:33.452 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:26:33.452 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:26:33.452 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: Manager daemon smithi007.huohfl is now available 2024-09-21T04:26:33.452 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:26:33.452 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:26:33.452 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:33 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/trash_purge_schedule"}]: dispatch 2024-09-21T04:26:35.063 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-21T04:26:35.063 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T04:26:35.063 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T04:26:35.063 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-21T04:26:35.063 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-09-21T04:26:35.063 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-09-21T04:26:35.396 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:35 smithi007 ceph-mon[27954]: mgrmap e11: smithi007.huohfl(active, since 1.00573s) 2024-09-21T04:26:35.396 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:35 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:34] ENGINE Bus STARTING 2024-09-21T04:26:35.396 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:35 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:34] ENGINE Serving on https://172.21.15.7:7150 2024-09-21T04:26:35.396 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:35 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:34] ENGINE Error in HTTPServer.serve 2024-09-21T04:26:35.396 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Traceback (most recent call last): 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._run(expiration_interval) 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s = self.context.wrap_socket( 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: return self.sslsocket_class._create( 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:26:35.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self.do_handshake() 2024-09-21T04:26:35.398 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:26:35.398 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._sslobj.do_handshake() 2024-09-21T04:26:35.398 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:26:35.398 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:35 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:34] ENGINE Serving on http://172.21.15.7:8765 2024-09-21T04:26:35.398 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:35 smithi007 ceph-mon[27954]: [21/Sep/2024:04:26:34] ENGINE Bus STARTED 2024-09-21T04:26:36.310 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-21T04:26:36.311 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-09-21T04:26:36.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:36 smithi007 ceph-mon[27954]: mgrmap e12: smithi007.huohfl(active, since 2s) 2024-09-21T04:26:36.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:36 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:36.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:36 smithi007 ceph-mon[27954]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:36.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:36 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:36.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:36 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:37 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:37 smithi007 ceph-mon[27954]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:37 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:38.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$GkfyyPLNohOT.r7CI.PNrObtV.AqOvgGyAjgHhXV2YxqR7rYu8OI.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726892797, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T04:26:38.600 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-09-21T04:26:39.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:39 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1712233459' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-21T04:26:39.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:39 smithi007 ceph-mon[27954]: mgrmap e13: smithi007.huohfl(active, since 6s) 2024-09-21T04:26:39.756 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T04:26:39.767 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T04:26:39.767 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-09-21T04:26:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T04:26:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-21T04:26:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-09-21T04:26:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T04:26:39.778 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-09-21T04:26:39.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:39.778 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-09-21T04:26:39.778 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-09-21T04:26:39.778 INFO:teuthology.orchestra.run.smithi007.stdout: Password: 11nk8voj0p 2024-09-21T04:26:39.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:39.778 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config directory 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:41.189 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T04:26:41.190 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:41.190 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-09-21T04:26:41.190 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:41.190 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-09-21T04:26:41.190 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:41.190 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T04:26:41.190 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:41.190 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-09-21T04:26:41.237 INFO:tasks.cephadm:Fetching config... 2024-09-21T04:26:41.237 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:26:41.237 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T04:26:41.256 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T04:26:41.256 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:26:41.256 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T04:26:41.314 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T04:26:41.314 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:26:41.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/keyring of=/dev/stdout 2024-09-21T04:26:41.383 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T04:26:41.383 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:26:41.383 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T04:26:41.439 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T04:26:41.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4rWqPADZMBCYjw9MPPul0Y39v83H09/oY8Cmz11c7vIoGiIuvohFh8ebszB3oetb/HEZ8j9mJazjhPA5T5N9I6NfBZo+QZ66TNUL+f6+Yg5+3NKtMZMnKw+fSXxNUiD/AJMbjLqutRGOWIY+QEzpkFxUkd+fONXy5W5g86fjLaDgBz/ajRKzfugIx6BklOUn0h5YIMMxtcSaLVcalAeteBnL86tcMvvwLIHiqdp06NtKmMnhSnoGtrdiGvigsCVW0RtRo7H4OMxsEnJDHaErHOnob//PDZOAVeaGJQOWL/rOxPu+KJVOtBfvgCUb2t2EtxqC5uY/kNXxUppgbVhwMQNdVYRlZKWo0Z2q0CMpptOs3i5L5ojEn/ddq8YfA58f+/TmvnYeOnqrKE2fLE3SSSS+b+RE0Ehg8GCpkhmYHrffhGy5uScJAawBDLfY9dYq8MO6I/sYr9R5cVaAQnC8IaaLS3gZrCiki2STY7xhGJEFKj6u2G+9+h4FFF5FApj8= ceph-70154924-77d1-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T04:26:41.524 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4rWqPADZMBCYjw9MPPul0Y39v83H09/oY8Cmz11c7vIoGiIuvohFh8ebszB3oetb/HEZ8j9mJazjhPA5T5N9I6NfBZo+QZ66TNUL+f6+Yg5+3NKtMZMnKw+fSXxNUiD/AJMbjLqutRGOWIY+QEzpkFxUkd+fONXy5W5g86fjLaDgBz/ajRKzfugIx6BklOUn0h5YIMMxtcSaLVcalAeteBnL86tcMvvwLIHiqdp06NtKmMnhSnoGtrdiGvigsCVW0RtRo7H4OMxsEnJDHaErHOnob//PDZOAVeaGJQOWL/rOxPu+KJVOtBfvgCUb2t2EtxqC5uY/kNXxUppgbVhwMQNdVYRlZKWo0Z2q0CMpptOs3i5L5ojEn/ddq8YfA58f+/TmvnYeOnqrKE2fLE3SSSS+b+RE0Ehg8GCpkhmYHrffhGy5uScJAawBDLfY9dYq8MO6I/sYr9R5cVaAQnC8IaaLS3gZrCiki2STY7xhGJEFKj6u2G+9+h4FFF5FApj8= ceph-70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:26:41.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4rWqPADZMBCYjw9MPPul0Y39v83H09/oY8Cmz11c7vIoGiIuvohFh8ebszB3oetb/HEZ8j9mJazjhPA5T5N9I6NfBZo+QZ66TNUL+f6+Yg5+3NKtMZMnKw+fSXxNUiD/AJMbjLqutRGOWIY+QEzpkFxUkd+fONXy5W5g86fjLaDgBz/ajRKzfugIx6BklOUn0h5YIMMxtcSaLVcalAeteBnL86tcMvvwLIHiqdp06NtKmMnhSnoGtrdiGvigsCVW0RtRo7H4OMxsEnJDHaErHOnob//PDZOAVeaGJQOWL/rOxPu+KJVOtBfvgCUb2t2EtxqC5uY/kNXxUppgbVhwMQNdVYRlZKWo0Z2q0CMpptOs3i5L5ojEn/ddq8YfA58f+/TmvnYeOnqrKE2fLE3SSSS+b+RE0Ehg8GCpkhmYHrffhGy5uScJAawBDLfY9dYq8MO6I/sYr9R5cVaAQnC8IaaLS3gZrCiki2STY7xhGJEFKj6u2G+9+h4FFF5FApj8= ceph-70154924-77d1-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T04:26:41.586 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4rWqPADZMBCYjw9MPPul0Y39v83H09/oY8Cmz11c7vIoGiIuvohFh8ebszB3oetb/HEZ8j9mJazjhPA5T5N9I6NfBZo+QZ66TNUL+f6+Yg5+3NKtMZMnKw+fSXxNUiD/AJMbjLqutRGOWIY+QEzpkFxUkd+fONXy5W5g86fjLaDgBz/ajRKzfugIx6BklOUn0h5YIMMxtcSaLVcalAeteBnL86tcMvvwLIHiqdp06NtKmMnhSnoGtrdiGvigsCVW0RtRo7H4OMxsEnJDHaErHOnob//PDZOAVeaGJQOWL/rOxPu+KJVOtBfvgCUb2t2EtxqC5uY/kNXxUppgbVhwMQNdVYRlZKWo0Z2q0CMpptOs3i5L5ojEn/ddq8YfA58f+/TmvnYeOnqrKE2fLE3SSSS+b+RE0Ehg8GCpkhmYHrffhGy5uScJAawBDLfY9dYq8MO6I/sYr9R5cVaAQnC8IaaLS3gZrCiki2STY7xhGJEFKj6u2G+9+h4FFF5FApj8= ceph-70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:26:41.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T04:26:41.780 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:41 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1881322124' entity='client.admin' 2024-09-21T04:26:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:26:43.326 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T04:26:43.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T04:26:43.812 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:26:43.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:43 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/626535947' entity='client.admin' 2024-09-21T04:26:45.388 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2024-09-21T04:26:45.389 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:26:45.389 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2024-09-21T04:26:45.410 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:26:45.410 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:26:45.469 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2024-09-21T04:26:45.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch host add smithi053 2024-09-21T04:26:45.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:45 smithi007 ceph-mon[27954]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:45.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:45 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:46.159 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:26:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:26:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:26:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:26:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-21T04:26:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:26:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:26:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:47 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:26:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:48 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:26:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:48 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:26:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:48 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:26:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:48 smithi007 ceph-mon[27954]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:48 smithi007 ceph-mon[27954]: Deploying daemon ceph-exporter.smithi007 on smithi007 2024-09-21T04:26:49.181 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:49 smithi007 ceph-mon[27954]: Deploying cephadm binary to smithi053 2024-09-21T04:26:50.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:50 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:50.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:50 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:50.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:50 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:50.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:50 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:50.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:50 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:26:50.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:50 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-21T04:26:50.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:50 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:26:50.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:50 smithi007 ceph-mon[27954]: Deploying daemon crash.smithi007 on smithi007 2024-09-21T04:26:50.981 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi053' with addr '172.21.15.53' 2024-09-21T04:26:51.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T04:26:52.089 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:51 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:52.089 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:51 smithi007 ceph-mon[27954]: Added host smithi053 2024-09-21T04:26:52.125 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:26:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:26:52.835 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2024-09-21T04:26:53.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:53 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:53.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:53 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:53.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:53 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:53.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:53 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:53.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:53 smithi007 ceph-mon[27954]: Deploying daemon node-exporter.smithi007 on smithi007 2024-09-21T04:26:53.563 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T04:26:53.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T04:26:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:26:54.247 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:54 smithi007 ceph-mon[27954]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:26:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:54 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:55.145 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-21T04:26:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:55 smithi007 ceph-mon[27954]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:26:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:55 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/848715645' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-21T04:26:56.385 INFO:tasks.cephadm:Adding mon.smithi007 on smithi007 2024-09-21T04:26:56.386 INFO:tasks.cephadm:Adding mon.smithi053 on smithi053 2024-09-21T04:26:56.386 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch apply mon '2;smithi007:172.21.15.7=smithi007;smithi053:172.21.15.53=smithi053' 2024-09-21T04:26:56.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:56 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/848715645' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T04:26:56.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:56 smithi007 ceph-mon[27954]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T04:26:56.656 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:26:56.702 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:26:57.287 INFO:teuthology.orchestra.run.smithi053.stdout:Scheduled mon update... 2024-09-21T04:26:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:57 smithi007 ceph-mon[27954]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:26:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:57 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:57 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:57 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:57 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:57.818 DEBUG:teuthology.orchestra.run.smithi053:mon.smithi053> sudo journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi053.service 2024-09-21T04:26:57.820 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:26:57.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:26:58.110 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:26:58.155 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:26:58.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:58 smithi007 ceph-mon[27954]: Deploying daemon alertmanager.smithi007 on smithi007 2024-09-21T04:26:58.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:58 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:26:58.824 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:26:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:26:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:59 smithi007 ceph-mon[27954]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:26:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:59 smithi007 ceph-mon[27954]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi007:172.21.15.7=smithi007;smithi053:172.21.15.53=smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:26:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:59 smithi007 ceph-mon[27954]: Saving service mon spec with placement smithi007:172.21.15.7=smithi007;smithi053:172.21.15.53=smithi053;count:2 2024-09-21T04:26:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:59 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:26:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:26:59 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/11343256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:00.450 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:00.450 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:00.703 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:00.749 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:01.408 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:01.408 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:01.408 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:01.431 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:01 smithi007 ceph-mon[27954]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:02.420 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:02 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/2320337010' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:02.997 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:02.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:03.252 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:03.308 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:03.434 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:03 smithi007 ceph-mon[27954]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:03.999 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:04.000 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:04.399 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:04 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/2298524655' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T04:27:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:05 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:05.620 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:05.620 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:05.875 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:05.920 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:06 smithi007 ceph-mon[27954]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T04:27:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:06 smithi007 ceph-mon[27954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T04:27:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:06 smithi007 ceph-mon[27954]: Deploying daemon grafana.smithi007 on smithi007 2024-09-21T04:27:06.656 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:06.656 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:06.656 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:07 smithi007 ceph-mon[27954]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:07 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3036264629' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:08.200 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:08.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:08.457 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:08.503 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:09.149 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:09.149 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:09.149 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:09 smithi007 ceph-mon[27954]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:09 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:09 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/2030800098' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:10.680 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:10.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:10.981 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:11.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:11 smithi007 ceph-mon[27954]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:11.640 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:11.640 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:11.641 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:12.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:12 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/469023172' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:13.238 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:13.238 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:13 smithi007 ceph-mon[27954]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:13.495 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:13.540 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:14.221 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:14.221 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/1561218263' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:15.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:15 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:15.837 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:15.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:16.091 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:16.137 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:16 smithi007 ceph-mon[27954]: Deploying daemon prometheus.smithi007 on smithi007 2024-09-21T04:27:16.773 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:16.773 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:16.773 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:17 smithi007 ceph-mon[27954]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:17 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3981735813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:18.275 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:18.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:18.532 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:18.578 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:19.220 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:19.220 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:19.221 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:19 smithi007 ceph-mon[27954]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:19 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:20.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:20 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/2652177290' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:20.859 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:20.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:21.121 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:21.166 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:21.443 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:21 smithi007 ceph-mon[27954]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:21.797 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:21.797 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:22.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:22 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/1194219489' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:23.389 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:23.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:23.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:23 smithi007 ceph-mon[27954]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:23.653 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:24.345 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:24.345 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:24.345 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:25 smithi007 ceph-mon[27954]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:25 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/945026787' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:25 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:25 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:25 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:25.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:25 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-21T04:27:25.875 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:25.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:26.137 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:26.183 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:26.829 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:26.829 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:26 smithi007 ceph-mon[27954]: from='mgr.14162 172.21.15.7:0/4088508629' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T04:27:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:26 smithi007 ceph-mon[27954]: mgrmap e14: smithi007.huohfl(active, since 52s) 2024-09-21T04:27:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:26 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3195017347' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:28.436 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:28.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:28.694 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:28.740 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:29.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:29.361 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:29.362 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:29.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:29 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/869637408' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:31.003 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:31.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:31.258 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:31.304 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: Active manager daemon smithi007.huohfl restarted 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: Activating manager daemon smithi007.huohfl 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: mgrmap e15: smithi007.huohfl(active, starting, since 0.00322072s) 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi007.huohfl", "id": "smithi007.huohfl"}]: dispatch 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:27:31.407 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: Manager daemon smithi007.huohfl is now available 2024-09-21T04:27:31.408 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:27:31.408 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:31.408 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:27:31.408 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:27:31.985 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:31.986 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:31.986 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:32.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:32 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/trash_purge_schedule"}]: dispatch 2024-09-21T04:27:32.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:32 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:32.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:32 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3329779258' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:32.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:32 smithi007 ceph-mon[27954]: mgrmap e16: smithi007.huohfl(active, since 1.00533s) 2024-09-21T04:27:33.532 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:33.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:33.781 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:33.842 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:33 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:33.842 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:33 smithi007 ceph-mon[27954]: [21/Sep/2024:04:27:32] ENGINE Bus STARTING 2024-09-21T04:27:33.842 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:33 smithi007 ceph-mon[27954]: [21/Sep/2024:04:27:32] ENGINE Serving on http://172.21.15.7:8765 2024-09-21T04:27:33.842 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:33 smithi007 ceph-mon[27954]: [21/Sep/2024:04:27:32] ENGINE Serving on https://172.21.15.7:7150 2024-09-21T04:27:33.842 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:33 smithi007 ceph-mon[27954]: [21/Sep/2024:04:27:32] ENGINE Bus STARTED 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:33 smithi007 ceph-mon[27954]: [21/Sep/2024:04:27:32] ENGINE Error in HTTPServer.serve 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Traceback (most recent call last): 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._run(expiration_interval) 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s = self.context.wrap_socket( 2024-09-21T04:27:33.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:27:33.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: return self.sslsocket_class._create( 2024-09-21T04:27:33.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:27:33.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self.do_handshake() 2024-09-21T04:27:33.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:27:33.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._sslobj.do_handshake() 2024-09-21T04:27:33.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:27:34.499 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:34.499 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:34.499 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:34.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:34 smithi007 ceph-mon[27954]: mgrmap e17: smithi007.huohfl(active, since 2s) 2024-09-21T04:27:34.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:34.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:34 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/508203515' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:36.180 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:36.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:36.439 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:36.493 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T04:27:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:27:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:37.174 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:37.174 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:37 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/1487689328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:38.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:27:38.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:27:38.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:27:38.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-21T04:27:38.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-21T04:27:38.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:27:38.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:38 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:27:39.315 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:39.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:39.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:27:39.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:27:39.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:27:39.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:39.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:39.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:27:39.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:39.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:39.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:39.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:27:39.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-21T04:27:39.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:27:39.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:39 smithi007 ceph-mon[27954]: Deploying daemon ceph-exporter.smithi053 on smithi053 2024-09-21T04:27:40.134 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:27:41.037 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:41.038 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:41 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/121542784' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:42.708 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:42.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:27:43.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-21T04:27:43.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:27:43.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:43 smithi007 ceph-mon[27954]: Deploying daemon crash.smithi053 on smithi053 2024-09-21T04:27:43.577 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:27:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:44.410 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:44.410 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:44 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/4280958026' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:46.156 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:46.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:46.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:45 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:46.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:45 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:46.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:45 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:46.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:45 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:46.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:45 smithi007 ceph-mon[27954]: Deploying daemon node-exporter.smithi053 on smithi053 2024-09-21T04:27:46.413 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:27:47.260 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:47.260 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:47.261 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:47 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:47 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:27:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:48 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3592469214' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:49.510 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:49.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:49.869 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:27:50.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:50.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:50.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:50.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:50.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:27:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T04:27:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:27:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:27:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:49 smithi007 ceph-mon[27954]: Deploying daemon mgr.smithi053.lnadao on smithi053 2024-09-21T04:27:51.078 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:51.078 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/2204385030' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:52.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:52.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:52.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:27:52.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:27:52.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:27:52.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:52 smithi007 ceph-mon[27954]: Deploying daemon mon.smithi053 on smithi053 2024-09-21T04:27:52.984 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:52.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:53.909 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:27:54.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:54 smithi007 ceph-mon[27954]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:27:54.699 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:27:54.700 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:25:29.508493Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T04:27:54.700 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2024-09-21T04:27:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:27:55 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3313822380' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:27:56.697 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-21T04:27:56.698 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T04:27:56.951 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:28:00.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:00 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: mon.smithi007 calling monitor election 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/2357701596' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/crt"}]: dispatch 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: mon.smithi053 calling monitor election 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:28:00.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: mon.smithi007 is new leader, mons smithi007,smithi053 in quorum (ranks 0,1) 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: monmap epoch 2 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: last_changed 2024-09-21T04:27:55.080724+0000 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: created 2024-09-21T04:25:29.508493+0000 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: min_mon_release 19 (squid) 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: election_strategy: 1 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: fsmap 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: mgrmap e17: smithi007.huohfl(active, since 29s) 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: overall HEALTH_OK 2024-09-21T04:28:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: Standby manager daemon smithi053.lnadao started 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/2357701596' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/2357701596' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/key"}]: dispatch 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/2357701596' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:00.657 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:28:00.658 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":2,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","modified":"2024-09-21T04:27:55.080724Z","created":"2024-09-21T04:25:29.508493Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi053","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T04:28:00.658 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 2 2024-09-21T04:28:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:01 smithi053 ceph-mon[35554]: mgrmap e18: smithi007.huohfl(active, since 29s), standbys: smithi053.lnadao 2024-09-21T04:28:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi053.lnadao", "id": "smithi053.lnadao"}]: dispatch 2024-09-21T04:28:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:28:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:01 smithi053 ceph-mon[35554]: from='client.? 172.21.15.53:0/1874089167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:28:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:28:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:28:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:01 smithi007 ceph-mon[27954]: mgrmap e18: smithi007.huohfl(active, since 29s), standbys: smithi053.lnadao 2024-09-21T04:28:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi053.lnadao", "id": "smithi053.lnadao"}]: dispatch 2024-09-21T04:28:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:28:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:01 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/1874089167' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-21T04:28:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:28:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:28:01.754 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T04:28:01.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T04:28:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:02.402 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:02 smithi007 ceph-mon[27954]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:02.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:02 smithi053 ceph-mon[35554]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:02.670 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:28:02.670 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-21T04:28:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:28:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] 2024-09-21T04:28:03.342 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T04:28:03.342 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:28:03.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T04:28:03.371 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:28:03.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:28:03.437 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:28:03.437 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T04:28:03.467 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:28:03.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:28:03.534 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T04:28:03.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T04:28:03.537 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T04:28:03.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T04:28:03.580 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T04:28:03.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T04:28:03.623 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T04:28:03.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T04:28:03.666 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T04:28:03.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/805113314' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:28:03.687 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:28:03.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:28:03.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:28:03.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:03 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:03.688 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T04:28:03.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T04:28:03.731 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T04:28:03.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T04:28:03.775 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T04:28:03.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T04:28:03.817 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T04:28:03.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T04:28:03.861 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T04:28:03.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T04:28:03.904 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T04:28:03.904 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/805113314' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:03.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:28:03.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:28:03.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:03 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:03.947 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T04:28:03.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T04:28:03.994 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T04:28:03.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T04:28:04.037 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T04:28:04.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T04:28:04.082 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T04:28:04.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T04:28:04.136 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T04:28:04.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T04:28:04.183 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-21T04:28:04.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-21T04:28:04.227 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T04:28:04.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T04:28:04.271 DEBUG:tasks.cephadm:set 18 configs 2024-09-21T04:28:04.271 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi007') 2024-09-21T04:28:04.720 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:04.735 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:04.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:04 smithi007 ceph-mon[27954]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:04.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:04 smithi007 ceph-mon[27954]: Reconfiguring mon.smithi007 (unknown last config time)... 2024-09-21T04:28:04.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:04 smithi007 ceph-mon[27954]: Reconfiguring daemon mon.smithi007 on smithi007 2024-09-21T04:28:04.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:04 smithi053 ceph-mon[35554]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:04.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:04 smithi053 ceph-mon[35554]: Reconfiguring mon.smithi007 (unknown last config time)... 2024-09-21T04:28:04.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:04 smithi053 ceph-mon[35554]: Reconfiguring daemon mon.smithi007 on smithi007 2024-09-21T04:28:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:05.104 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:05.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:05.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:05.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: Reconfiguring mgr.smithi007.huohfl (unknown last config time)... 2024-09-21T04:28:05.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.huohfl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:28:05.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:28:05.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:05.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: Reconfiguring daemon mgr.smithi007.huohfl on smithi007 2024-09-21T04:28:05.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:05.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:05 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1423764939' entity='client.admin' 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: Reconfiguring mgr.smithi007.huohfl (unknown last config time)... 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.huohfl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: Reconfiguring daemon mgr.smithi007.huohfl on smithi007 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:05 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1423764939' entity='client.admin' 2024-09-21T04:28:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:06.605 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:06.607 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:06.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:06 smithi007 ceph-mon[27954]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:06.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:06 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2511225951' entity='client.admin' 2024-09-21T04:28:06.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:06 smithi053 ceph-mon[35554]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:06.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:06 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2511225951' entity='client.admin' 2024-09-21T04:28:07.046 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:07.047 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:07.048 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:07.769 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:07 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/745271783' entity='client.admin' 2024-09-21T04:28:08.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:07 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/745271783' entity='client.admin' 2024-09-21T04:28:09.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:08 smithi053 ceph-mon[35554]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:08 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1621141816' entity='client.admin' 2024-09-21T04:28:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:08 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3649694722' entity='client.admin' 2024-09-21T04:28:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:08 smithi007 ceph-mon[27954]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:08 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1621141816' entity='client.admin' 2024-09-21T04:28:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:08 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3649694722' entity='client.admin' 2024-09-21T04:28:09.499 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:10.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:10 smithi007 ceph-mon[27954]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:10.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:10 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2277584569' entity='client.admin' 2024-09-21T04:28:10.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:10 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2955800966' entity='client.admin' 2024-09-21T04:28:11.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:10 smithi053 ceph-mon[35554]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:11.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:10 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2277584569' entity='client.admin' 2024-09-21T04:28:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:10 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2955800966' entity='client.admin' 2024-09-21T04:28:12.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:11 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1670188130' entity='client.admin' 2024-09-21T04:28:12.381 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:11 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1670188130' entity='client.admin' 2024-09-21T04:28:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:12 smithi007 ceph-mon[27954]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:12 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2548989772' entity='client.admin' 2024-09-21T04:28:13.385 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:12 smithi053 ceph-mon[35554]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:13.385 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:12 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2548989772' entity='client.admin' 2024-09-21T04:28:14.618 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:14 smithi007 ceph-mon[27954]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:14.618 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:14 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:14.618 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:14 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:14.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:14 smithi007 ceph-mon[27954]: Reconfiguring ceph-exporter.smithi007 (monmap changed)... 2024-09-21T04:28:14.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:14 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:28:14.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:14 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:14.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:14 smithi007 ceph-mon[27954]: Reconfiguring daemon ceph-exporter.smithi007 on smithi007 2024-09-21T04:28:14.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:14 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/4056087439' entity='client.admin' 2024-09-21T04:28:14.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:14 smithi053 ceph-mon[35554]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:14 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:14 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:14 smithi053 ceph-mon[35554]: Reconfiguring ceph-exporter.smithi007 (monmap changed)... 2024-09-21T04:28:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:14 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:28:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:14 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:14 smithi053 ceph-mon[35554]: Reconfiguring daemon ceph-exporter.smithi007 on smithi007 2024-09-21T04:28:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:14 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/4056087439' entity='client.admin' 2024-09-21T04:28:14.842 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:15.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 -- 172.21.15.7:0/548870987 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c0c06e280 msgr2=0x7f0c0c070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:15.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 --2- 172.21.15.7:0/548870987 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c0c06e280 0x7f0c0c070740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c14000c00 tx=0x7f0c14009560 comp rx=0 tx=0).stop 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 -- 172.21.15.7:0/548870987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c24102b60 msgr2=0x7f0c24197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 --2- 172.21.15.7:0/548870987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c24102b60 0x7f0c24197a40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800b4e0 tx=0x7f0c1800b9a0 comp rx=0 tx=0).stop 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 -- 172.21.15.7:0/548870987 shutdown_connections 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 --2- 172.21.15.7:0/548870987 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c0c06e280 0x7f0c0c070740 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 --2- 172.21.15.7:0/548870987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c24102b60 0x7f0c24197a40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 --2- 172.21.15.7:0/548870987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c24101960 0x7f0c24197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 -- 172.21.15.7:0/548870987 >> 172.21.15.7:0/548870987 conn(0x7f0c240fd0f0 msgr2=0x7f0c24106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.122+0000 7f0c2934a640 1 -- 172.21.15.7:0/548870987 shutdown_connections 2024-09-21T04:28:15.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.123+0000 7f0c2934a640 1 -- 172.21.15.7:0/548870987 wait complete. 2024-09-21T04:28:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.016+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3210946552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 msgr2=0x7f8cc0102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.016+0000 7f8cc5f92640 1 --2- 172.21.15.7:0/3210946552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 0x7f8cc0102fe0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009880 tx=0x7f8cb402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.016+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3210946552 shutdown_connections 2024-09-21T04:28:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.016+0000 7f8cc5f92640 1 --2- 172.21.15.7:0/3210946552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 0x7f8cc0102fe0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.016+0000 7f8cc5f92640 1 --2- 172.21.15.7:0/3210946552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc0101960 0x7f8cc0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.016+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3210946552 >> 172.21.15.7:0/3210946552 conn(0x7f8cc00fd0f0 msgr2=0x7f8cc00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.016+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3210946552 shutdown_connections 2024-09-21T04:28:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.016+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3210946552 wait complete. 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cc5f92640 1 Processor -- start 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cc5f92640 1 -- start start 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cc5f92640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc0101960 0x7f8cc0197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cc5f92640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 0x7f8cc01979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cc5f92640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc010a780 con 0x7f8cc0102b60 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cc5f92640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc010a600 con 0x7f8cc0101960 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 0x7f8cc01979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 0x7f8cc01979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48344/0 (socket says 172.21.15.7:48344) 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cbeffd640 1 -- 172.21.15.7:0/3047145716 learned_addr learned my addr 172.21.15.7:0/3047145716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.017+0000 7f8cbf7fe640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc0101960 0x7f8cc0197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cbeffd640 1 -- 172.21.15.7:0/3047145716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc0101960 msgr2=0x7f8cc0197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cbeffd640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc0101960 0x7f8cc0197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cbeffd640 1 -- 172.21.15.7:0/3047145716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc0198030 con 0x7f8cc0102b60 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cbeffd640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 0x7f8cc01979b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006010 tx=0x7f8cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cbcff9640 1 -- 172.21.15.7:0/3047145716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cb403f070 con 0x7f8cc0102b60 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc019ca10 con 0x7f8cc0102b60 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cbcff9640 1 -- 172.21.15.7:0/3047145716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cb4004ac0 con 0x7f8cc0102b60 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cbcff9640 1 -- 172.21.15.7:0/3047145716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cb4035030 con 0x7f8cc0102b60 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.018+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc019cfa0 con 0x7f8cc0102b60 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.019+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc0101d60 con 0x7f8cc0102b60 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.020+0000 7f8cbcff9640 1 -- 172.21.15.7:0/3047145716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7f8cb4042070 con 0x7f8cc0102b60 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.020+0000 7f8cbcff9640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e230 0x7f8ca80706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.020+0000 7f8cbcff9640 1 -- 172.21.15.7:0/3047145716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8cb40adc00 con 0x7f8cc0102b60 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.020+0000 7f8cbf7fe640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e230 0x7f8ca80706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.021+0000 7f8cbf7fe640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e230 0x7f8ca80706f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8cc01029c0 tx=0x7f8cb000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.024+0000 7f8cbcff9640 1 -- 172.21.15.7:0/3047145716 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8cb40b4080 con 0x7f8cc0102b60 2024-09-21T04:28:16.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.225+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f8cc0062df0 con 0x7f8cc0102b60 2024-09-21T04:28:16.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.228+0000 7f8cbcff9640 1 -- 172.21.15.7:0/3047145716 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f8cb40802f0 con 0x7f8cc0102b60 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.230+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e230 msgr2=0x7f8ca80706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.230+0000 7f8cc5f92640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e230 0x7f8ca80706f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8cc01029c0 tx=0x7f8cb000b390 comp rx=0 tx=0).stop 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.230+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 msgr2=0x7f8cc01979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.230+0000 7f8cc5f92640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 0x7f8cc01979b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006010 tx=0x7f8cb40046c0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.231+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 shutdown_connections 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.231+0000 7f8cc5f92640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e230 0x7f8ca80706f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.231+0000 7f8cc5f92640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc0102b60 0x7f8cc01979b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.231+0000 7f8cc5f92640 1 --2- 172.21.15.7:0/3047145716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc0101960 0x7f8cc0197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.231+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 >> 172.21.15.7:0/3047145716 conn(0x7f8cc00fd0f0 msgr2=0x7f8cc0106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.231+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 shutdown_connections 2024-09-21T04:28:16.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:16.231+0000 7f8cc5f92640 1 -- 172.21.15.7:0/3047145716 wait complete. 2024-09-21T04:28:16.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:16 smithi053 ceph-mon[35554]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:16.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:16 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/548870987' entity='client.admin' 2024-09-21T04:28:16.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:16 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3001863465' entity='client.admin' 2024-09-21T04:28:16.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:28:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:16 smithi007 ceph-mon[27954]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:16 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/548870987' entity='client.admin' 2024-09-21T04:28:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:16 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3001863465' entity='client.admin' 2024-09-21T04:28:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:28:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.600+0000 7efc6b48f640 1 -- 172.21.15.7:0/224271135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 msgr2=0x7efc64107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.600+0000 7efc6b48f640 1 --2- 172.21.15.7:0/224271135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 0x7efc64107110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efc540098b0 tx=0x7efc5402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.600+0000 7efc6b48f640 1 -- 172.21.15.7:0/224271135 shutdown_connections 2024-09-21T04:28:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.600+0000 7efc6b48f640 1 --2- 172.21.15.7:0/224271135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 0x7efc64107110 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.600+0000 7efc6b48f640 1 --2- 172.21.15.7:0/224271135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641023f0 0x7efc641047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.600+0000 7efc6b48f640 1 -- 172.21.15.7:0/224271135 >> 172.21.15.7:0/224271135 conn(0x7efc640fc010 msgr2=0x7efc640fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.600+0000 7efc6b48f640 1 -- 172.21.15.7:0/224271135 shutdown_connections 2024-09-21T04:28:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.600+0000 7efc6b48f640 1 -- 172.21.15.7:0/224271135 wait complete. 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc6b48f640 1 Processor -- start 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc6b48f640 1 -- start start 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc6b48f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641023f0 0x7efc64197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc6b48f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 0x7efc64197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc6b48f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc6410a7b0 con 0x7efc64104d20 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc6b48f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efc6410a630 con 0x7efc641023f0 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc68a2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 0x7efc64197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc68a2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 0x7efc64197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48320/0 (socket says 172.21.15.7:48320) 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc68a2a640 1 -- 172.21.15.7:0/3001863465 learned_addr learned my addr 172.21.15.7:0/3001863465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.601+0000 7efc6922b640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641023f0 0x7efc64197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc68a2a640 1 -- 172.21.15.7:0/3001863465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641023f0 msgr2=0x7efc64197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc68a2a640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641023f0 0x7efc64197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc68a2a640 1 -- 172.21.15.7:0/3001863465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc64198100 con 0x7efc64104d20 2024-09-21T04:28:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc68a2a640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 0x7efc64197a80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efc54009880 tx=0x7efc540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc5a7fc640 1 -- 172.21.15.7:0/3001863465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efc5403f070 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6419cae0 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc5a7fc640 1 -- 172.21.15.7:0/3001863465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc54004a50 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc5a7fc640 1 -- 172.21.15.7:0/3001863465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efc54035030 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.602+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6419d070 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.603+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc641027d0 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.604+0000 7efc5a7fc640 1 -- 172.21.15.7:0/3001863465 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7efc54042070 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.604+0000 7efc5a7fc640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7efc5006e240 0x7efc50070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.604+0000 7efc5a7fc640 1 -- 172.21.15.7:0/3001863465 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7efc540adaf0 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.604+0000 7efc6922b640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7efc5006e240 0x7efc50070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.604+0000 7efc6922b640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7efc5006e240 0x7efc50070700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efc60005fd0 tx=0x7efc6000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.608+0000 7efc5a7fc640 1 -- 172.21.15.7:0/3001863465 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc54004c00 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.787+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7efc6419d2a0 con 0x7efc64104d20 2024-09-21T04:28:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.790+0000 7efc5a7fc640 1 -- 172.21.15.7:0/3001863465 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v22)=0 v22) ==== 108+0+0 (secure 0 0 0) 0x7efc54080180 con 0x7efc64104d20 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7efc5006e240 msgr2=0x7efc50070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7efc5006e240 0x7efc50070700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efc60005fd0 tx=0x7efc6000b390 comp rx=0 tx=0).stop 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 msgr2=0x7efc64197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 0x7efc64197a80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efc54009880 tx=0x7efc540045e0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 shutdown_connections 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7efc5006e240 0x7efc50070700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc64104d20 0x7efc64197a80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 --2- 172.21.15.7:0/3001863465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc641023f0 0x7efc64197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 >> 172.21.15.7:0/3001863465 conn(0x7efc640fc010 msgr2=0x7efc640fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 shutdown_connections 2024-09-21T04:28:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:15.792+0000 7efc6b48f640 1 -- 172.21.15.7:0/3001863465 wait complete. 2024-09-21T04:28:17.317 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:17 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3047145716' entity='client.admin' 2024-09-21T04:28:17.353 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi007') 2024-09-21T04:28:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:17 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3047145716' entity='client.admin' 2024-09-21T04:28:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.529+0000 7f834b425640 1 -- 172.21.15.7:0/3555323443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344104d80 msgr2=0x7f8344107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.529+0000 7f834b425640 1 --2- 172.21.15.7:0/3555323443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344104d80 0x7f8344107170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8334009880 tx=0x7f833402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:28:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.529+0000 7f834b425640 1 -- 172.21.15.7:0/3555323443 shutdown_connections 2024-09-21T04:28:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.529+0000 7f834b425640 1 --2- 172.21.15.7:0/3555323443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344104d80 0x7f8344107170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.529+0000 7f834b425640 1 --2- 172.21.15.7:0/3555323443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8344102450 0x7f8344104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.529+0000 7f834b425640 1 -- 172.21.15.7:0/3555323443 >> 172.21.15.7:0/3555323443 conn(0x7f83440fc010 msgr2=0x7f83440fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.529+0000 7f834b425640 1 -- 172.21.15.7:0/3555323443 shutdown_connections 2024-09-21T04:28:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.529+0000 7f834b425640 1 -- 172.21.15.7:0/3555323443 wait complete. 2024-09-21T04:28:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f834b425640 1 Processor -- start 2024-09-21T04:28:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f834b425640 1 -- start start 2024-09-21T04:28:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f834b425640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344102450 0x7f83441974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f834b425640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8344104d80 0x7f83441979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f834b425640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f834410a740 con 0x7f8344102450 2024-09-21T04:28:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f834b425640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f834410a5c0 con 0x7f8344104d80 2024-09-21T04:28:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f83491c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344102450 0x7f83441974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f83491c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344102450 0x7f83441974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48364/0 (socket says 172.21.15.7:48364) 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f83491c1640 1 -- 172.21.15.7:0/3362711784 learned_addr learned my addr 172.21.15.7:0/3362711784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f83489c0640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8344104d80 0x7f83441979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f83491c1640 1 -- 172.21.15.7:0/3362711784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8344104d80 msgr2=0x7f83441979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f83491c1640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8344104d80 0x7f83441979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.530+0000 7f83491c1640 1 -- 172.21.15.7:0/3362711784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8344198060 con 0x7f8344102450 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.531+0000 7f83491c1640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344102450 0x7f83441974a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f834000d6f0 tx=0x7f834000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.531+0000 7f833a7fc640 1 -- 172.21.15.7:0/3362711784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8340017070 con 0x7f8344102450 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.531+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834419caa0 con 0x7f8344102450 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.531+0000 7f833a7fc640 1 -- 172.21.15.7:0/3362711784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f834000de00 con 0x7f8344102450 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.531+0000 7f833a7fc640 1 -- 172.21.15.7:0/3362711784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8340004bb0 con 0x7f8344102450 2024-09-21T04:28:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.531+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f834419d000 con 0x7f8344102450 2024-09-21T04:28:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.532+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f834410af20 con 0x7f8344102450 2024-09-21T04:28:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.533+0000 7f833a7fc640 1 -- 172.21.15.7:0/3362711784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7f8340004e60 con 0x7f8344102450 2024-09-21T04:28:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.533+0000 7f833a7fc640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f833006e240 0x7f8330070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.533+0000 7f83489c0640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f833006e240 0x7f8330070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.533+0000 7f833a7fc640 1 -- 172.21.15.7:0/3362711784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f834008cb90 con 0x7f8344102450 2024-09-21T04:28:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.534+0000 7f83489c0640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f833006e240 0x7f8330070700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8334006010 tx=0x7f8334005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.538+0000 7f833a7fc640 1 -- 172.21.15.7:0/3362711784 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f834005f280 con 0x7f8344102450 2024-09-21T04:28:17.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.715+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f8344102830 con 0x7f8344102450 2024-09-21T04:28:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.718+0000 7f833a7fc640 1 -- 172.21.15.7:0/3362711784 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f8340063ab0 con 0x7f8344102450 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f833006e240 msgr2=0x7f8330070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f833006e240 0x7f8330070700 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8334006010 tx=0x7f8334005a20 comp rx=0 tx=0).stop 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344102450 msgr2=0x7f83441974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344102450 0x7f83441974a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f834000d6f0 tx=0x7f834000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 shutdown_connections 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f833006e240 0x7f8330070700 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8344104d80 0x7f83441979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 --2- 172.21.15.7:0/3362711784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8344102450 0x7f83441974a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 >> 172.21.15.7:0/3362711784 conn(0x7f83440fc010 msgr2=0x7f83440fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 shutdown_connections 2024-09-21T04:28:17.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:17.721+0000 7f834b425640 1 -- 172.21.15.7:0/3362711784 wait complete. 2024-09-21T04:28:19.187 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:18 smithi007 ceph-mon[27954]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:19.187 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:18 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3362711784' entity='client.admin' 2024-09-21T04:28:19.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:18 smithi053 ceph-mon[35554]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:19.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:18 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3362711784' entity='client.admin' 2024-09-21T04:28:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:19.722 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:20.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.206+0000 7f023f598640 1 -- 172.21.15.7:0/2269621863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 msgr2=0x7f0238102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:20.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.206+0000 7f023f598640 1 --2- 172.21.15.7:0/2269621863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 0x7f0238102fe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0228009880 tx=0x7f022802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:28:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.206+0000 7f023f598640 1 -- 172.21.15.7:0/2269621863 shutdown_connections 2024-09-21T04:28:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.206+0000 7f023f598640 1 --2- 172.21.15.7:0/2269621863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 0x7f0238102fe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.206+0000 7f023f598640 1 --2- 172.21.15.7:0/2269621863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0238101960 0x7f0238101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.206+0000 7f023f598640 1 -- 172.21.15.7:0/2269621863 >> 172.21.15.7:0/2269621863 conn(0x7f02380fd0f0 msgr2=0x7f02380ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.206+0000 7f023f598640 1 -- 172.21.15.7:0/2269621863 shutdown_connections 2024-09-21T04:28:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.206+0000 7f023f598640 1 -- 172.21.15.7:0/2269621863 wait complete. 2024-09-21T04:28:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023f598640 1 Processor -- start 2024-09-21T04:28:20.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023f598640 1 -- start start 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023f598640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0238101960 0x7f0238075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023f598640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 0x7f02380745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023f598640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f023810a780 con 0x7f0238102b60 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023f598640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f023810a600 con 0x7f0238101960 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023cb33640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 0x7f02380745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023cb33640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 0x7f02380745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48394/0 (socket says 172.21.15.7:48394) 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023cb33640 1 -- 172.21.15.7:0/1067199185 learned_addr learned my addr 172.21.15.7:0/1067199185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.207+0000 7f023d334640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0238101960 0x7f0238075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f023cb33640 1 -- 172.21.15.7:0/1067199185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0238101960 msgr2=0x7f0238075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f023cb33640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0238101960 0x7f0238075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f023cb33640 1 -- 172.21.15.7:0/1067199185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0238074b20 con 0x7f0238102b60 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f023cb33640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 0x7f02380745e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0228009230 tx=0x7f02280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f022e7fc640 1 -- 172.21.15.7:0/1067199185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f022803f070 con 0x7f0238102b60 2024-09-21T04:28:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f022e7fc640 1 -- 172.21.15.7:0/1067199185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0228004b80 con 0x7f0238102b60 2024-09-21T04:28:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0238074db0 con 0x7f0238102b60 2024-09-21T04:28:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f022e7fc640 1 -- 172.21.15.7:0/1067199185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0228035030 con 0x7f0238102b60 2024-09-21T04:28:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.208+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02380752e0 con 0x7f0238102b60 2024-09-21T04:28:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.209+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0238101d60 con 0x7f0238102b60 2024-09-21T04:28:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.210+0000 7f022e7fc640 1 -- 172.21.15.7:0/1067199185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7f0228042070 con 0x7f0238102b60 2024-09-21T04:28:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.210+0000 7f022e7fc640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f022406e240 0x7f0224070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.210+0000 7f022e7fc640 1 -- 172.21.15.7:0/1067199185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f02280adc00 con 0x7f0238102b60 2024-09-21T04:28:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.210+0000 7f023d334640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f022406e240 0x7f0224070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.211+0000 7f023d334640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f022406e240 0x7f0224070700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f02381029c0 tx=0x7f023400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.215+0000 7f022e7fc640 1 -- 172.21.15.7:0/1067199185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f02280b4080 con 0x7f0238102b60 2024-09-21T04:28:20.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.394+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f0238062df0 con 0x7f0238102b60 2024-09-21T04:28:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.397+0000 7f022e7fc640 1 -- 172.21.15.7:0/1067199185 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v25)=0 v25) ==== 125+0+0 (secure 0 0 0) 0x7f02280802f0 con 0x7f0238102b60 2024-09-21T04:28:20.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f022406e240 msgr2=0x7f0224070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:20.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f022406e240 0x7f0224070700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f02381029c0 tx=0x7f023400b390 comp rx=0 tx=0).stop 2024-09-21T04:28:20.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 msgr2=0x7f02380745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:20.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 0x7f02380745e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0228009230 tx=0x7f02280046c0 comp rx=0 tx=0).stop 2024-09-21T04:28:20.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 shutdown_connections 2024-09-21T04:28:20.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f022406e240 0x7f0224070700 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:20.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0238102b60 0x7f02380745e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:20.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 --2- 172.21.15.7:0/1067199185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0238101960 0x7f0238075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:20.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.399+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 >> 172.21.15.7:0/1067199185 conn(0x7f02380fd0f0 msgr2=0x7f0238106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:20.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.400+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 shutdown_connections 2024-09-21T04:28:20.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:20.400+0000 7f023f598640 1 -- 172.21.15.7:0/1067199185 wait complete. 2024-09-21T04:28:21.142 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:20 smithi007 ceph-mon[27954]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:21.142 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:20 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1067199185' entity='client.admin' 2024-09-21T04:28:21.171 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi007') 2024-09-21T04:28:21.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:20 smithi053 ceph-mon[35554]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:20 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1067199185' entity='client.admin' 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 -- 172.21.15.7:0/702600176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 msgr2=0x7fcc70100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 --2- 172.21.15.7:0/702600176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 0x7fcc70100c60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009880 tx=0x7fcc6c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 -- 172.21.15.7:0/702600176 shutdown_connections 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 --2- 172.21.15.7:0/702600176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc70101a60 0x7fcc70101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 --2- 172.21.15.7:0/702600176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 0x7fcc70100c60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 -- 172.21.15.7:0/702600176 >> 172.21.15.7:0/702600176 conn(0x7fcc700fc010 msgr2=0x7fcc700fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 -- 172.21.15.7:0/702600176 shutdown_connections 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 -- 172.21.15.7:0/702600176 wait complete. 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 Processor -- start 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 -- start start 2024-09-21T04:28:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.353+0000 7fcc77eda640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 0x7fcc7006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc77eda640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc70101a60 0x7fcc7019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc77eda640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70109680 con 0x7fcc70100860 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc77eda640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70109500 con 0x7fcc70101a60 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc75c76640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 0x7fcc7006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc75c76640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 0x7fcc7006a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48420/0 (socket says 172.21.15.7:48420) 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc75c76640 1 -- 172.21.15.7:0/1203276227 learned_addr learned my addr 172.21.15.7:0/1203276227 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc75475640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc70101a60 0x7fcc7019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc75c76640 1 -- 172.21.15.7:0/1203276227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc70101a60 msgr2=0x7fcc7019c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc75c76640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc70101a60 0x7fcc7019c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc75c76640 1 -- 172.21.15.7:0/1203276227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7019c7b0 con 0x7fcc70100860 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc75c76640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 0x7fcc7006a7d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009230 tx=0x7fcc6c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc66ffd640 1 -- 172.21.15.7:0/1203276227 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcc6c03f070 con 0x7fcc70100860 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc66ffd640 1 -- 172.21.15.7:0/1203276227 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc6c004a50 con 0x7fcc70100860 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7019ca40 con 0x7fcc70100860 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc66ffd640 1 -- 172.21.15.7:0/1203276227 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcc6c035030 con 0x7fcc70100860 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.354+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7019cf70 con 0x7fcc70100860 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.355+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc70100c60 con 0x7fcc70100860 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.356+0000 7fcc66ffd640 1 -- 172.21.15.7:0/1203276227 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7fcc6c042070 con 0x7fcc70100860 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.356+0000 7fcc66ffd640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fcc5c06e280 0x7fcc5c070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.356+0000 7fcc66ffd640 1 -- 172.21.15.7:0/1203276227 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcc6c0adc00 con 0x7fcc70100860 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.356+0000 7fcc75475640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fcc5c06e280 0x7fcc5c070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.357+0000 7fcc75475640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fcc5c06e280 0x7fcc5c070740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcc60005fd0 tx=0x7fcc60005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.360+0000 7fcc66ffd640 1 -- 172.21.15.7:0/1203276227 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcc6c0b4080 con 0x7fcc70100860 2024-09-21T04:28:21.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.542+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fcc7019d2c0 con 0x7fcc70100860 2024-09-21T04:28:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.546+0000 7fcc66ffd640 1 -- 172.21.15.7:0/1203276227 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v26)=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fcc6c07c630 con 0x7fcc70100860 2024-09-21T04:28:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fcc5c06e280 msgr2=0x7fcc5c070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fcc5c06e280 0x7fcc5c070740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcc60005fd0 tx=0x7fcc60005850 comp rx=0 tx=0).stop 2024-09-21T04:28:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 msgr2=0x7fcc7006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 0x7fcc7006a7d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009230 tx=0x7fcc6c0045c0 comp rx=0 tx=0).stop 2024-09-21T04:28:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 shutdown_connections 2024-09-21T04:28:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fcc5c06e280 0x7fcc5c070740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc70101a60 0x7fcc7019c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:21.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 --2- 172.21.15.7:0/1203276227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc70100860 0x7fcc7006a7d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:21.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 >> 172.21.15.7:0/1203276227 conn(0x7fcc700fc010 msgr2=0x7fcc700fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:21.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 shutdown_connections 2024-09-21T04:28:21.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:21.548+0000 7fcc77eda640 1 -- 172.21.15.7:0/1203276227 wait complete. 2024-09-21T04:28:22.142 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi007') 2024-09-21T04:28:22.142 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi007') 2024-09-21T04:28:22.142 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi007') 2024-09-21T04:28:22.142 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi007') 2024-09-21T04:28:22.631 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:22 smithi007 ceph-mon[27954]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:22.631 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:22 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1203276227' entity='client.admin' 2024-09-21T04:28:22.868 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:22 smithi053 ceph-mon[35554]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:22.868 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:22 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1203276227' entity='client.admin' 2024-09-21T04:28:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.960+0000 7f25d9e69640 1 -- 172.21.15.7:0/3287334337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 msgr2=0x7f25d4107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.960+0000 7f25d9e69640 1 --2- 172.21.15.7:0/3287334337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 0x7f25d4107190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009880 tx=0x7f25c802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:28:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.960+0000 7f25d9e69640 1 -- 172.21.15.7:0/3287334337 shutdown_connections 2024-09-21T04:28:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.960+0000 7f25d9e69640 1 --2- 172.21.15.7:0/3287334337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 0x7f25d4107190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.960+0000 7f25d9e69640 1 --2- 172.21.15.7:0/3287334337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25d4102470 0x7f25d4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.960+0000 7f25d9e69640 1 -- 172.21.15.7:0/3287334337 >> 172.21.15.7:0/3287334337 conn(0x7f25d40fc010 msgr2=0x7f25d40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.960+0000 7f25d9e69640 1 -- 172.21.15.7:0/3287334337 shutdown_connections 2024-09-21T04:28:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.960+0000 7f25d9e69640 1 -- 172.21.15.7:0/3287334337 wait complete. 2024-09-21T04:28:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.961+0000 7f25d9e69640 1 Processor -- start 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.961+0000 7f25d9e69640 1 -- start start 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.961+0000 7f25d9e69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25d4102470 0x7f25d4199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.961+0000 7f25d9e69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 0x7f25d4199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.961+0000 7f25d9e69640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f25d410a780 con 0x7f25d4104da0 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.961+0000 7f25d9e69640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f25d410a600 con 0x7f25d4102470 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 0x7f25d4199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 0x7f25d4199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45590/0 (socket says 172.21.15.7:45590) 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d2ffd640 1 -- 172.21.15.7:0/3971254869 learned_addr learned my addr 172.21.15.7:0/3971254869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d37fe640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25d4102470 0x7f25d4199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d2ffd640 1 -- 172.21.15.7:0/3971254869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25d4102470 msgr2=0x7f25d4199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d2ffd640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25d4102470 0x7f25d4199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d2ffd640 1 -- 172.21.15.7:0/3971254869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d419a230 con 0x7f25d4104da0 2024-09-21T04:28:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d2ffd640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 0x7f25d4199c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009230 tx=0x7f25c80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d0ff9640 1 -- 172.21.15.7:0/3971254869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f25c803f070 con 0x7f25d4104da0 2024-09-21T04:28:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d0ff9640 1 -- 172.21.15.7:0/3971254869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25c80044d0 con 0x7f25d4104da0 2024-09-21T04:28:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d419ec40 con 0x7f25d4104da0 2024-09-21T04:28:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.962+0000 7f25d0ff9640 1 -- 172.21.15.7:0/3971254869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f25c8035030 con 0x7f25d4104da0 2024-09-21T04:28:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.963+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d419f140 con 0x7f25d4104da0 2024-09-21T04:28:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.964+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d4102850 con 0x7f25d4104da0 2024-09-21T04:28:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.964+0000 7f25d0ff9640 1 -- 172.21.15.7:0/3971254869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7f25c8042070 con 0x7f25d4104da0 2024-09-21T04:28:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.964+0000 7f25d0ff9640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f25bc06e240 0x7f25bc070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.965+0000 7f25d0ff9640 1 -- 172.21.15.7:0/3971254869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f25c80adc90 con 0x7f25d4104da0 2024-09-21T04:28:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.965+0000 7f25d37fe640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f25bc06e240 0x7f25bc070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:23.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.965+0000 7f25d37fe640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f25bc06e240 0x7f25bc070700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25c400abe0 tx=0x7f25c400a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:23.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:23.973+0000 7f25d0ff9640 1 -- 172.21.15.7:0/3971254869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f25c807c6d0 con 0x7f25d4104da0 2024-09-21T04:28:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.154+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f25d4062df0 con 0x7f25d4104da0 2024-09-21T04:28:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.157+0000 7f25d0ff9640 1 -- 172.21.15.7:0/3971254869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v27)=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f25c8080380 con 0x7f25d4104da0 2024-09-21T04:28:24.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f25bc06e240 msgr2=0x7f25bc070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:24.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f25bc06e240 0x7f25bc070700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25c400abe0 tx=0x7f25c400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:28:24.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 msgr2=0x7f25d4199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 0x7f25d4199c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009230 tx=0x7f25c80048f0 comp rx=0 tx=0).stop 2024-09-21T04:28:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 shutdown_connections 2024-09-21T04:28:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f25bc06e240 0x7f25bc070700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d4104da0 0x7f25d4199c40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 --2- 172.21.15.7:0/3971254869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25d4102470 0x7f25d4199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 >> 172.21.15.7:0/3971254869 conn(0x7f25d40fc010 msgr2=0x7f25d4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 shutdown_connections 2024-09-21T04:28:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:24.159+0000 7f25d9e69640 1 -- 172.21.15.7:0/3971254869 wait complete. 2024-09-21T04:28:24.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:24 smithi007 ceph-mon[27954]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:24.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:24 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3971254869' entity='client.admin' 2024-09-21T04:28:24.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:24.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:24.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:28:24.917 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:24.942 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi007') 2024-09-21T04:28:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:24 smithi053 ceph-mon[35554]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:24 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3971254869' entity='client.admin' 2024-09-21T04:28:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:28:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.200+0000 7f9367d83640 1 -- 172.21.15.7:0/2982308114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 msgr2=0x7f9360100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.200+0000 7f9367d83640 1 --2- 172.21.15.7:0/2982308114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 0x7f9360100600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f935c009880 tx=0x7f935c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:28:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.200+0000 7f9367d83640 1 -- 172.21.15.7:0/2982308114 shutdown_connections 2024-09-21T04:28:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.200+0000 7f9367d83640 1 --2- 172.21.15.7:0/2982308114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9360100b40 0x7f9360100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.200+0000 7f9367d83640 1 --2- 172.21.15.7:0/2982308114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 0x7f9360100600 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.200+0000 7f9367d83640 1 -- 172.21.15.7:0/2982308114 >> 172.21.15.7:0/2982308114 conn(0x7f93600fc030 msgr2=0x7f93600fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9367d83640 1 -- 172.21.15.7:0/2982308114 shutdown_connections 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9367d83640 1 -- 172.21.15.7:0/2982308114 wait complete. 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9367d83640 1 Processor -- start 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9367d83640 1 -- start start 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9367d83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 0x7f9360199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9367d83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9360100b40 0x7f9360199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9367d83640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f936010ca10 con 0x7f9360100200 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9367d83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f936010c890 con 0x7f9360100b40 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9365b1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 0x7f9360199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9365b1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 0x7f9360199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45602/0 (socket says 172.21.15.7:45602) 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.201+0000 7f9365b1f640 1 -- 172.21.15.7:0/3978473820 learned_addr learned my addr 172.21.15.7:0/3978473820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f936531e640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9360100b40 0x7f9360199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9365b1f640 1 -- 172.21.15.7:0/3978473820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9360100b40 msgr2=0x7f9360199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9365b1f640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9360100b40 0x7f9360199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9365b1f640 1 -- 172.21.15.7:0/3978473820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936019a340 con 0x7f9360100200 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f936531e640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9360100b40 0x7f9360199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9365b1f640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 0x7f9360199780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f935c009230 tx=0x7f935c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9356ffd640 1 -- 172.21.15.7:0/3978473820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f935c03f070 con 0x7f9360100200 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9356ffd640 1 -- 172.21.15.7:0/3978473820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f935c0044d0 con 0x7f9360100200 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936019ecd0 con 0x7f9360100200 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9356ffd640 1 -- 172.21.15.7:0/3978473820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f935c035030 con 0x7f9360100200 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.202+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936019f230 con 0x7f9360100200 2024-09-21T04:28:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.203+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9360100600 con 0x7f9360100200 2024-09-21T04:28:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.204+0000 7f9356ffd640 1 -- 172.21.15.7:0/3978473820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7f935c042070 con 0x7f9360100200 2024-09-21T04:28:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.204+0000 7f9356ffd640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f934c06e280 0x7f934c070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.204+0000 7f9356ffd640 1 -- 172.21.15.7:0/3978473820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f935c0ade20 con 0x7f9360100200 2024-09-21T04:28:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.204+0000 7f936531e640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f934c06e280 0x7f934c070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.205+0000 7f936531e640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f934c06e280 0x7f934c070740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9350006fd0 tx=0x7f9350008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.209+0000 7f9356ffd640 1 -- 172.21.15.7:0/3978473820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f935c07c8e0 con 0x7f9360100200 2024-09-21T04:28:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.396+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f936019fb60 con 0x7f9360100200 2024-09-21T04:28:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.399+0000 7f9356ffd640 1 -- 172.21.15.7:0/3978473820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v28)=0 v28) ==== 106+0+0 (secure 0 0 0) 0x7f935c080590 con 0x7f9360100200 2024-09-21T04:28:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f934c06e280 msgr2=0x7f934c070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f934c06e280 0x7f934c070740 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9350006fd0 tx=0x7f9350008040 comp rx=0 tx=0).stop 2024-09-21T04:28:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 msgr2=0x7f9360199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 0x7f9360199780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f935c009230 tx=0x7f935c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:28:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 shutdown_connections 2024-09-21T04:28:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f934c06e280 0x7f934c070740 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9360100b40 0x7f9360199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 --2- 172.21.15.7:0/3978473820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9360100200 0x7f9360199780 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 >> 172.21.15.7:0/3978473820 conn(0x7f93600fc030 msgr2=0x7f93600fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 shutdown_connections 2024-09-21T04:28:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:25.401+0000 7f9367d83640 1 -- 172.21.15.7:0/3978473820 wait complete. 2024-09-21T04:28:25.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:25 smithi007 ceph-mon[27954]: Reconfiguring crash.smithi007 (monmap changed)... 2024-09-21T04:28:25.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:25 smithi007 ceph-mon[27954]: Reconfiguring daemon crash.smithi007 on smithi007 2024-09-21T04:28:25.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:25 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3978473820' entity='client.admin' 2024-09-21T04:28:25.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:25 smithi053 ceph-mon[35554]: Reconfiguring crash.smithi007 (monmap changed)... 2024-09-21T04:28:25.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:25 smithi053 ceph-mon[35554]: Reconfiguring daemon crash.smithi007 on smithi007 2024-09-21T04:28:25.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:25 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3978473820' entity='client.admin' 2024-09-21T04:28:26.030 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.030 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.030 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.030 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.030 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.030 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.030 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.031 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.031 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.031 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '70154924-77d1-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi007') 2024-09-21T04:28:26.033 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T04:28:26.033 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:28:26.033 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T04:28:26.051 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T04:28:26.052 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-21T04:28:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T04:28:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:28:26.108 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T04:28:26.108 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:28:26.109 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:28:26.109 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-21 04:27:24.270504834 +0000 2024-09-21T04:28:26.109 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-21 04:22:15.094378104 +0000 2024-09-21T04:28:26.109 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-21 04:22:15.094378104 +0000 2024-09-21T04:28:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-21 04:22:15.094378104 +0000 2024-09-21T04:28:26.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T04:28:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-21T04:28:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-21T04:28:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00011893 s, 4.3 MB/s 2024-09-21T04:28:26.178 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T04:28:26.237 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-21T04:28:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T04:28:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:28:26.294 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T04:28:26.294 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:28:26.295 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:28:26.295 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-21 04:27:24.271504819 +0000 2024-09-21T04:28:26.295 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-21 04:22:15.405373468 +0000 2024-09-21T04:28:26.295 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-21 04:22:15.405373468 +0000 2024-09-21T04:28:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-21 04:22:15.405373468 +0000 2024-09-21T04:28:26.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T04:28:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-21T04:28:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-21T04:28:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000116116 s, 4.4 MB/s 2024-09-21T04:28:26.362 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T04:28:26.420 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-21T04:28:26.477 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T04:28:26.477 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:28:26.477 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T04:28:26.478 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:28:26.478 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:28:26.478 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-21 04:27:24.271504819 +0000 2024-09-21T04:28:26.478 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-21 04:22:15.730368623 +0000 2024-09-21T04:28:26.478 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-21 04:22:15.730368623 +0000 2024-09-21T04:28:26.478 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-21 04:22:15.730368623 +0000 2024-09-21T04:28:26.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T04:28:26.542 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-21T04:28:26.542 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-21T04:28:26.542 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.5758e-05 s, 5.3 MB/s 2024-09-21T04:28:26.544 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T04:28:26.601 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-21 04:27:24.271504819 +0000 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-21 04:22:16.062363674 +0000 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-21 04:22:16.062363674 +0000 2024-09-21T04:28:26.657 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-21 04:22:16.062363674 +0000 2024-09-21T04:28:26.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T04:28:26.682 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:26 smithi007 ceph-mon[27954]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:26.706 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-21T04:28:26.706 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-21T04:28:26.706 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000105955 s, 4.8 MB/s 2024-09-21T04:28:26.707 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T04:28:26.764 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:28:26.765 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T04:28:26.785 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T04:28:26.786 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-21 04:27:54.376969646 +0000 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 04:22:15.091622580 +0000 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 04:22:15.091622580 +0000 2024-09-21T04:28:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-21 04:22:15.091622580 +0000 2024-09-21T04:28:26.847 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T04:28:26.913 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:26 smithi053 ceph-mon[35554]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:26.916 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-21T04:28:26.917 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-21T04:28:26.917 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000129794 s, 3.9 MB/s 2024-09-21T04:28:26.918 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T04:28:26.977 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-09-21T04:28:27.035 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T04:28:27.035 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:28:27.036 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T04:28:27.036 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:28:27.036 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:28:27.036 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-21 04:27:54.376969646 +0000 2024-09-21T04:28:27.036 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 04:22:15.414616200 +0000 2024-09-21T04:28:27.036 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 04:22:15.414616200 +0000 2024-09-21T04:28:27.036 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-21 04:22:15.414616200 +0000 2024-09-21T04:28:27.036 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T04:28:27.103 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-21T04:28:27.103 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-21T04:28:27.103 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000122217 s, 4.2 MB/s 2024-09-21T04:28:27.104 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T04:28:27.162 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-09-21T04:28:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T04:28:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:28:27.219 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T04:28:27.220 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:28:27.220 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:28:27.220 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-21 04:27:54.377969626 +0000 2024-09-21T04:28:27.220 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 04:22:15.729609967 +0000 2024-09-21T04:28:27.220 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 04:22:15.729609967 +0000 2024-09-21T04:28:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-21 04:22:15.729609967 +0000 2024-09-21T04:28:27.220 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T04:28:27.284 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-21T04:28:27.284 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-21T04:28:27.284 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.9495e-05 s, 5.1 MB/s 2024-09-21T04:28:27.285 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T04:28:27.342 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-21 04:27:54.377969626 +0000 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 04:22:16.069603240 +0000 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 04:22:16.069603240 +0000 2024-09-21T04:28:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-21 04:22:16.069603240 +0000 2024-09-21T04:28:27.399 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T04:28:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-21T04:28:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-21T04:28:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000115073 s, 4.4 MB/s 2024-09-21T04:28:27.465 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T04:28:27.522 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-09-21T04:28:27.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T04:28:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:28 smithi053 ceph-mon[35554]: Reconfiguring alertmanager.smithi007 (dependencies changed)... 2024-09-21T04:28:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:28 smithi053 ceph-mon[35554]: Reconfiguring daemon alertmanager.smithi007 on smithi007 2024-09-21T04:28:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:28 smithi053 ceph-mon[35554]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:28 smithi007 ceph-mon[27954]: Reconfiguring alertmanager.smithi007 (dependencies changed)... 2024-09-21T04:28:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:28 smithi007 ceph-mon[27954]: Reconfiguring daemon alertmanager.smithi007 on smithi007 2024-09-21T04:28:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:28 smithi007 ceph-mon[27954]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:29.841 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:30.295 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:30 smithi007 ceph-mon[27954]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:30.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:30 smithi053 ceph-mon[35554]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:31.437 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:28:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:31 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:28:32.329 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:32 smithi007 ceph-mon[27954]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:32.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:32 smithi053 ceph-mon[35554]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:33.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:33 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:33.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:33 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:33.889 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:33 smithi007 ceph-mon[27954]: Reconfiguring grafana.smithi007 (dependencies changed)... 2024-09-21T04:28:33.889 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:33 smithi007 ceph-mon[27954]: Reconfiguring daemon grafana.smithi007 on smithi007 2024-09-21T04:28:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:28:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:33 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:33 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:33 smithi053 ceph-mon[35554]: Reconfiguring grafana.smithi007 (dependencies changed)... 2024-09-21T04:28:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:33 smithi053 ceph-mon[35554]: Reconfiguring daemon grafana.smithi007 on smithi007 2024-09-21T04:28:34.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-09-21T04:28:35.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:34 smithi053 ceph-mon[35554]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:34 smithi007 ceph-mon[27954]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:35.284 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.901+0000 7f32b051e640 1 -- 172.21.15.7:0/1068975220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8104de0 msgr2=0x7f32a81071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.901+0000 7f32b051e640 1 --2- 172.21.15.7:0/1068975220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8104de0 0x7f32a81071d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3298009880 tx=0x7f329802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.901+0000 7f32b051e640 1 -- 172.21.15.7:0/1068975220 shutdown_connections 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.901+0000 7f32b051e640 1 --2- 172.21.15.7:0/1068975220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8104de0 0x7f32a81071d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.901+0000 7f32b051e640 1 --2- 172.21.15.7:0/1068975220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a81024b0 0x7f32a81048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.901+0000 7f32b051e640 1 -- 172.21.15.7:0/1068975220 >> 172.21.15.7:0/1068975220 conn(0x7f32a80fc010 msgr2=0x7f32a80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.901+0000 7f32b051e640 1 -- 172.21.15.7:0/1068975220 shutdown_connections 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.901+0000 7f32b051e640 1 -- 172.21.15.7:0/1068975220 wait complete. 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32b051e640 1 Processor -- start 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32b051e640 1 -- start start 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32b051e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a81024b0 0x7f32a8101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32b051e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8104de0 0x7f32a8100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32b051e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810a7a0 con 0x7f32a81024b0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32b051e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810a620 con 0x7f32a8104de0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32ae2ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a81024b0 0x7f32a8101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32ae2ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a81024b0 0x7f32a8101da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52646/0 (socket says 172.21.15.7:52646) 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32ae2ba640 1 -- 172.21.15.7:0/1060413358 learned_addr learned my addr 172.21.15.7:0/1060413358 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32adab9640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8104de0 0x7f32a8100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32ae2ba640 1 -- 172.21.15.7:0/1060413358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8104de0 msgr2=0x7f32a8100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32ae2ba640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8104de0 0x7f32a8100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32ae2ba640 1 -- 172.21.15.7:0/1060413358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a81009a0 con 0x7f32a81024b0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.902+0000 7f32ae2ba640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a81024b0 0x7f32a8101da0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f32a400d8d0 tx=0x7f32a400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.903+0000 7f329f7fe640 1 -- 172.21.15.7:0/1060413358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f32a4017070 con 0x7f32a81024b0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.903+0000 7f329f7fe640 1 -- 172.21.15.7:0/1060413358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f32a4010460 con 0x7f32a81024b0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.903+0000 7f329f7fe640 1 -- 172.21.15.7:0/1060413358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f32a400f5f0 con 0x7f32a81024b0 2024-09-21T04:28:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.903+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a8100c90 con 0x7f32a81024b0 2024-09-21T04:28:35.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.903+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81011f0 con 0x7f32a81024b0 2024-09-21T04:28:35.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.904+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a8102890 con 0x7f32a81024b0 2024-09-21T04:28:35.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.904+0000 7f329f7fe640 1 -- 172.21.15.7:0/1060413358 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7f32a4009a30 con 0x7f32a81024b0 2024-09-21T04:28:35.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.904+0000 7f329f7fe640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f329406e230 0x7f32940706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:28:35.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.905+0000 7f329f7fe640 1 -- 172.21.15.7:0/1060413358 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f32a408cea0 con 0x7f32a81024b0 2024-09-21T04:28:35.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.905+0000 7f32adab9640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f329406e230 0x7f32940706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:28:35.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.905+0000 7f32adab9640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f329406e230 0x7f32940706f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3298009230 tx=0x7f32980058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:28:35.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:35.909+0000 7f329f7fe640 1 -- 172.21.15.7:0/1060413358 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32a405f590 con 0x7f32a81024b0 2024-09-21T04:28:36.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:28:36.096+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f32a8105680 con 0x7f329406e230 2024-09-21T04:28:37.095 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:36 smithi053 ceph-mon[35554]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:37.096 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:28:37.096 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:28:37.096 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:37.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:36 smithi007 ceph-mon[27954]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:37.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:28:37.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:28:37.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:37.971 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:37 smithi007 ceph-mon[27954]: from='client.14330 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:28:38.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:37 smithi053 ceph-mon[35554]: from='client.14330 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:28:39.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:38 smithi053 ceph-mon[35554]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:38 smithi007 ceph-mon[27954]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:40 smithi007 ceph-mon[27954]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:40 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:40 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:40 smithi007 ceph-mon[27954]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-09-21T04:28:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:40 smithi007 ceph-mon[27954]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-09-21T04:28:40.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:40 smithi053 ceph-mon[35554]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:40.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:40 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:40.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:40 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:40.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:40 smithi053 ceph-mon[35554]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-09-21T04:28:40.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:40 smithi053 ceph-mon[35554]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-09-21T04:28:41.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:41 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2584505677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7692119a-1db2-417f-a086-0181309481b9"}]: dispatch 2024-09-21T04:28:41.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:41 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2584505677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7692119a-1db2-417f-a086-0181309481b9"}]': finished 2024-09-21T04:28:41.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:41 smithi053 ceph-mon[35554]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T04:28:41.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:41 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:28:41.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:41 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3672835605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:28:41.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:41 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2584505677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7692119a-1db2-417f-a086-0181309481b9"}]: dispatch 2024-09-21T04:28:41.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:41 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2584505677' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7692119a-1db2-417f-a086-0181309481b9"}]': finished 2024-09-21T04:28:41.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:41 smithi007 ceph-mon[27954]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T04:28:41.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:41 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:28:41.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:41 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3672835605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:28:42.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:42 smithi007 ceph-mon[27954]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:42 smithi053 ceph-mon[35554]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:45.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:44 smithi053 ceph-mon[35554]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:44 smithi007 ceph-mon[27954]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:46.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:46 smithi007 ceph-mon[27954]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:46.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:28:47.096 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:46 smithi053 ceph-mon[35554]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:47.096 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:28:49.030 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:48 smithi007 ceph-mon[27954]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:49.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:48 smithi053 ceph-mon[35554]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:50.865 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:50 smithi007 ceph-mon[27954]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:51.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:50 smithi053 ceph-mon[35554]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:53.152 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:52 smithi007 ceph-mon[27954]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:53.152 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:53.152 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:53.153 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:52 smithi007 ceph-mon[27954]: Reconfiguring ceph-exporter.smithi053 (monmap changed)... 2024-09-21T04:28:53.153 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:28:53.153 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:52 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:53.153 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:52 smithi007 ceph-mon[27954]: Reconfiguring daemon ceph-exporter.smithi053 on smithi053 2024-09-21T04:28:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:52 smithi053 ceph-mon[35554]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:52 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:52 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:52 smithi053 ceph-mon[35554]: Reconfiguring ceph-exporter.smithi053 (monmap changed)... 2024-09-21T04:28:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:52 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:28:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:52 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:52 smithi053 ceph-mon[35554]: Reconfiguring daemon ceph-exporter.smithi053 on smithi053 2024-09-21T04:28:54.118 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-21T04:28:54.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:54.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:54.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:54.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:28:54.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-21T04:28:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:54.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-21T04:28:54.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: Deploying daemon osd.0 on smithi007 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: Reconfiguring crash.smithi053 (monmap changed)... 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: Reconfiguring daemon crash.smithi053 on smithi053 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:28:55.159 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:54 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: Deploying daemon osd.0 on smithi007 2024-09-21T04:28:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: Reconfiguring crash.smithi053 (monmap changed)... 2024-09-21T04:28:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: Reconfiguring daemon crash.smithi053 on smithi053 2024-09-21T04:28:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:28:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:28:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:54 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:56.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:55 smithi053 ceph-mon[35554]: Reconfiguring mgr.smithi053.lnadao (monmap changed)... 2024-09-21T04:28:56.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:55 smithi053 ceph-mon[35554]: Reconfiguring daemon mgr.smithi053.lnadao on smithi053 2024-09-21T04:28:56.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:55 smithi007 ceph-mon[27954]: Reconfiguring mgr.smithi053.lnadao (monmap changed)... 2024-09-21T04:28:56.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:55 smithi007 ceph-mon[27954]: Reconfiguring daemon mgr.smithi053.lnadao on smithi053 2024-09-21T04:28:57.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:56 smithi053 ceph-mon[35554]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:28:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:28:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:56 smithi007 ceph-mon[27954]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:28:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:28:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:28:57.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: Reconfiguring mon.smithi053 (monmap changed)... 2024-09-21T04:28:57.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: Reconfiguring daemon mon.smithi053 on smithi053 2024-09-21T04:28:57.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi007.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi007.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi007.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:57.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:28:58.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: Reconfiguring mon.smithi053 (monmap changed)... 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: Reconfiguring daemon mon.smithi053 on smithi053 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi007.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi007.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi007.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T04:28:58.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:28:58.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:28:59.144 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:58 smithi007 ceph-mon[27954]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:59.144 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:58 smithi007 ceph-mon[27954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:28:59.144 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:58 smithi007 ceph-mon[27954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi007.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T04:28:59.144 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:58 smithi007 ceph-mon[27954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:28:59.144 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:58 smithi007 ceph-mon[27954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi007.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T04:28:59.144 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:58 smithi007 ceph-mon[27954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:28:59.144 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:28:58 smithi007 ceph-mon[27954]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi007.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T04:28:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:58 smithi053 ceph-mon[35554]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:28:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:58 smithi053 ceph-mon[35554]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:28:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:58 smithi053 ceph-mon[35554]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi007.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T04:28:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:58 smithi053 ceph-mon[35554]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:28:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:58 smithi053 ceph-mon[35554]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi007.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T04:28:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:58 smithi053 ceph-mon[35554]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:28:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:28:58 smithi053 ceph-mon[35554]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi007.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T04:29:00.618 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:00 smithi007 ceph-mon[27954]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:00.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:00.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:00.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:00.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:00.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:00 smithi053 ceph-mon[35554]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:00.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:00 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:00.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:00 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:00.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:00 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:00.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:00 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:01.629 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:29:01.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:29:02.601 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:02 smithi007 ceph-mon[27954]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:02.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:02 smithi053 ceph-mon[35554]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:04.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:04 smithi053 ceph-mon[35554]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:04.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:04 smithi007 ceph-mon[27954]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:04.787 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-09-21T04:29:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.783+0000 7f329f7fe640 1 -- 172.21.15.7:0/1060413358 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f32a8105680 con 0x7f329406e230 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f329406e230 msgr2=0x7f32940706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f329406e230 0x7f32940706f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3298009230 tx=0x7f32980058d0 comp rx=0 tx=0).stop 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a81024b0 msgr2=0x7f32a8101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a81024b0 0x7f32a8101da0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f32a400d8d0 tx=0x7f32a400dd90 comp rx=0 tx=0).stop 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 shutdown_connections 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f329406e230 0x7f32940706f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8104de0 0x7f32a8100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 --2- 172.21.15.7:0/1060413358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a81024b0 0x7f32a8101da0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 >> 172.21.15.7:0/1060413358 conn(0x7f32a80fc010 msgr2=0x7f32a80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 shutdown_connections 2024-09-21T04:29:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:04.785+0000 7f32b051e640 1 -- 172.21.15.7:0/1060413358 wait complete. 2024-09-21T04:29:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:05.509 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T04:29:05.511 INFO:tasks.cephadm:Deploying osd.1 on smithi007 with /dev/vg_nvme/lv_3... 2024-09-21T04:29:05.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T04:29:05.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:05.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:05.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:05.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:06.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:29:06 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[48982]: 2024-09-21T04:29:06.035+0000 7f639ede3640 -1 osd.0 0 log_to_monitors true 2024-09-21T04:29:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:29:06.903 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:06 smithi007 ceph-mon[27954]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:06.904 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:06 smithi007 ceph-mon[27954]: from='osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-21T04:29:07.097 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:06 smithi053 ceph-mon[35554]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:07.098 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:06 smithi053 ceph-mon[35554]: from='osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-21T04:29:08.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:07 smithi053 ceph-mon[35554]: from='osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T04:29:08.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:07 smithi053 ceph-mon[35554]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T04:29:08.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:07 smithi053 ceph-mon[35554]: from='osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-21T04:29:08.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:07 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:08.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:07 smithi053 ceph-mon[35554]: from='osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-21T04:29:08.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:07 smithi007 ceph-mon[27954]: from='osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T04:29:08.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:07 smithi007 ceph-mon[27954]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T04:29:08.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:07 smithi007 ceph-mon[27954]: from='osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-21T04:29:08.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:07 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:08.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:07 smithi007 ceph-mon[27954]: from='osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-21T04:29:08.195 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:29:07 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[48982]: 2024-09-21T04:29:07.791+0000 7f639a543640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T04:29:08.195 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:29:07 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[48982]: 2024-09-21T04:29:07.795+0000 7f6396b7d640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:29:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:08 smithi007 ceph-mon[27954]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:08 smithi007 ceph-mon[27954]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T04:29:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:08 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:08 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:08 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:09.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:08 smithi053 ceph-mon[35554]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:09.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:08 smithi053 ceph-mon[35554]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T04:29:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:08 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:08 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:08 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: purged_snaps scrub starts 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: purged_snaps scrub ok 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] boot 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:29:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:10.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:29:10.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: purged_snaps scrub starts 2024-09-21T04:29:10.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: purged_snaps scrub ok 2024-09-21T04:29:10.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: osd.0 [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] boot 2024-09-21T04:29:10.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T04:29:10.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:29:10.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:10.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:10.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:29:10.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:10.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:10.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:29:10.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:10.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:29:10.545 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:29:10.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-09-21T04:29:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:29:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:10 smithi053 ceph-mon[35554]: Detected new or changed devices on smithi007 2024-09-21T04:29:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:10 smithi053 ceph-mon[35554]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:10 smithi053 ceph-mon[35554]: Adjusting osd_memory_target on smithi007 to 11844M 2024-09-21T04:29:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:10 smithi053 ceph-mon[35554]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T04:29:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:10 smithi007 ceph-mon[27954]: Detected new or changed devices on smithi007 2024-09-21T04:29:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:10 smithi007 ceph-mon[27954]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T04:29:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:10 smithi007 ceph-mon[27954]: Adjusting osd_memory_target on smithi007 to 11844M 2024-09-21T04:29:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:10 smithi007 ceph-mon[27954]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T04:29:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 -- 172.21.15.7:0/2818796245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c104d40 msgr2=0x7f5d6c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:29:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 --2- 172.21.15.7:0/2818796245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c104d40 0x7f5d6c107130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098b0 tx=0x7f5d5802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:29:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 -- 172.21.15.7:0/2818796245 shutdown_connections 2024-09-21T04:29:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 --2- 172.21.15.7:0/2818796245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c104d40 0x7f5d6c107130 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 --2- 172.21.15.7:0/2818796245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c102410 0x7f5d6c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 -- 172.21.15.7:0/2818796245 >> 172.21.15.7:0/2818796245 conn(0x7f5d6c0fc030 msgr2=0x7f5d6c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:29:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 -- 172.21.15.7:0/2818796245 shutdown_connections 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 -- 172.21.15.7:0/2818796245 wait complete. 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 Processor -- start 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.940+0000 7f5d7309e640 1 -- start start 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d7309e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102410 0x7f5d6c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d7309e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c104d40 0x7f5d6c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d7309e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10a830 con 0x7f5d6c102410 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d7309e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10a6b0 con 0x7f5d6c104d40 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d70e3a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102410 0x7f5d6c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d70e3a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102410 0x7f5d6c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53522/0 (socket says 172.21.15.7:53522) 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d70e3a640 1 -- 172.21.15.7:0/3470628999 learned_addr learned my addr 172.21.15.7:0/3470628999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d6bfff640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c104d40 0x7f5d6c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d70e3a640 1 -- 172.21.15.7:0/3470628999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c104d40 msgr2=0x7f5d6c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:29:11.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d70e3a640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c104d40 0x7f5d6c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d70e3a640 1 -- 172.21.15.7:0/3470628999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c19a3a0 con 0x7f5d6c102410 2024-09-21T04:29:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d6bfff640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c104d40 0x7f5d6c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:29:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d70e3a640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102410 0x7f5d6c1997e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000b420 tx=0x7f5d6000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:29:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d69ffb640 1 -- 172.21.15.7:0/3470628999 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d60011630 con 0x7f5d6c102410 2024-09-21T04:29:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c19ede0 con 0x7f5d6c102410 2024-09-21T04:29:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d69ffb640 1 -- 172.21.15.7:0/3470628999 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d60016070 con 0x7f5d6c102410 2024-09-21T04:29:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.941+0000 7f5d69ffb640 1 -- 172.21.15.7:0/3470628999 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d60010d40 con 0x7f5d6c102410 2024-09-21T04:29:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.942+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c19f310 con 0x7f5d6c102410 2024-09-21T04:29:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.943+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c1027f0 con 0x7f5d6c102410 2024-09-21T04:29:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.943+0000 7f5d69ffb640 1 -- 172.21.15.7:0/3470628999 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7f5d6001b460 con 0x7f5d6c102410 2024-09-21T04:29:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.943+0000 7f5d69ffb640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f5d5c06e230 0x7f5d5c0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:29:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.944+0000 7f5d69ffb640 1 -- 172.21.15.7:0/3470628999 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7f5d6008cb10 con 0x7f5d6c102410 2024-09-21T04:29:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.944+0000 7f5d6bfff640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f5d5c06e230 0x7f5d5c0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:29:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.944+0000 7f5d6bfff640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f5d5c06e230 0x7f5d5c0706f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d58006010 tx=0x7f5d58005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:29:11.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:11.948+0000 7f5d69ffb640 1 -- 172.21.15.7:0/3470628999 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5d6005b2f0 con 0x7f5d6c102410 2024-09-21T04:29:12.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:12.134+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5d6c0626f0 con 0x7f5d5c06e230 2024-09-21T04:29:12.138 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:12.138 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:12.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:12.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:12 smithi053 ceph-mon[35554]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:12 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:29:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:12 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:29:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:12 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:12 smithi007 ceph-mon[27954]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:12 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:29:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:12 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:29:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:12 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:14.079 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:13 smithi007 ceph-mon[27954]: from='client.14348 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:29:14.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:13 smithi053 ceph-mon[35554]: from='client.14348 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:29:15.184 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:14 smithi007 ceph-mon[27954]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:15.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:14 smithi053 ceph-mon[35554]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:15 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1639817742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09975c57-317c-4d48-9926-8cfa7786b7b7"}]: dispatch 2024-09-21T04:29:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:15 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1639817742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09975c57-317c-4d48-9926-8cfa7786b7b7"}]': finished 2024-09-21T04:29:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:15 smithi053 ceph-mon[35554]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T04:29:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:15 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:15 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1639817742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09975c57-317c-4d48-9926-8cfa7786b7b7"}]: dispatch 2024-09-21T04:29:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:15 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1639817742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09975c57-317c-4d48-9926-8cfa7786b7b7"}]': finished 2024-09-21T04:29:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:15 smithi007 ceph-mon[27954]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T04:29:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:15 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:16 smithi053 ceph-mon[35554]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:29:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:29:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:16 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2518977569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:29:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:16 smithi007 ceph-mon[27954]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:29:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:29:17.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:17.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:16 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2518977569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:29:19.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:18 smithi053 ceph-mon[35554]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:18 smithi007 ceph-mon[27954]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:21.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:20 smithi053 ceph-mon[35554]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:20 smithi007 ceph-mon[27954]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:23.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:22 smithi053 ceph-mon[35554]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:22 smithi007 ceph-mon[27954]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:25.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:24 smithi053 ceph-mon[35554]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-21T04:29:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:24 smithi007 ceph-mon[27954]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-21T04:29:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:26.124 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:25 smithi007 ceph-mon[27954]: Deploying daemon osd.1 on smithi007 2024-09-21T04:29:26.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:25 smithi053 ceph-mon[35554]: Deploying daemon osd.1 on smithi007 2024-09-21T04:29:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:26 smithi007 ceph-mon[27954]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:27.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:26 smithi053 ceph-mon[35554]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:29.238 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:28 smithi007 ceph-mon[27954]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:28 smithi053 ceph-mon[35554]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:31.328 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:30 smithi007 ceph-mon[27954]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:31.328 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:29:31.328 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:31.328 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:30 smithi053 ceph-mon[35554]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:29:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:32.274 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:29:32.312 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:31 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:29:33.296 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:32 smithi007 ceph-mon[27954]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:33.296 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:32 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:33.296 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:32 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:32 smithi053 ceph-mon[35554]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:32 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:32 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:34.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.981+0000 7f5d69ffb640 1 -- 172.21.15.7:0/3470628999 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5d6c0626f0 con 0x7f5d5c06e230 2024-09-21T04:29:34.984 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 1 on host 'smithi007' 2024-09-21T04:29:34.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.983+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f5d5c06e230 msgr2=0x7f5d5c0706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:29:34.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.983+0000 7f5d7309e640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f5d5c06e230 0x7f5d5c0706f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d58006010 tx=0x7f5d58005a20 comp rx=0 tx=0).stop 2024-09-21T04:29:34.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.983+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102410 msgr2=0x7f5d6c1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:29:34.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.983+0000 7f5d7309e640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102410 0x7f5d6c1997e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000b420 tx=0x7f5d6000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:29:34.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.984+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 shutdown_connections 2024-09-21T04:29:34.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.984+0000 7f5d7309e640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f5d5c06e230 0x7f5d5c0706f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:34.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.984+0000 7f5d7309e640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c104d40 0x7f5d6c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:34.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.984+0000 7f5d7309e640 1 --2- 172.21.15.7:0/3470628999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102410 0x7f5d6c1997e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:34.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.984+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 >> 172.21.15.7:0/3470628999 conn(0x7f5d6c0fc030 msgr2=0x7f5d6c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:29:34.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.984+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 shutdown_connections 2024-09-21T04:29:34.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:34.984+0000 7f5d7309e640 1 -- 172.21.15.7:0/3470628999 wait complete. 2024-09-21T04:29:35.341 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:34 smithi007 ceph-mon[27954]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:35.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:34 smithi053 ceph-mon[35554]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:35.669 DEBUG:teuthology.orchestra.run.smithi007:osd.1> sudo journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T04:29:35.672 INFO:tasks.cephadm:Deploying osd.2 on smithi007 with /dev/vg_nvme/lv_2... 2024-09-21T04:29:35.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T04:29:36.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:35 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:36.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:35 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:36.378 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:29:36.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:35 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:35 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:37.158 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:36 smithi007 ceph-mon[27954]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:36 smithi053 ceph-mon[35554]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:37.444 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:29:37 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[56123]: 2024-09-21T04:29:37.155+0000 7f5561778640 -1 osd.1 0 log_to_monitors true 2024-09-21T04:29:38.123 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:37 smithi007 ceph-mon[27954]: from='osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-21T04:29:38.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:37 smithi053 ceph-mon[35554]: from='osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-21T04:29:39.333 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:29:38 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[56123]: 2024-09-21T04:29:38.995+0000 7f555ced8640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T04:29:39.334 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:29:39 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[56123]: 2024-09-21T04:29:39.000+0000 7f5559512640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:29:39.334 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:38 smithi007 ceph-mon[27954]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:39.334 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:38 smithi007 ceph-mon[27954]: from='osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T04:29:39.334 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:38 smithi007 ceph-mon[27954]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T04:29:39.334 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:38 smithi007 ceph-mon[27954]: from='osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-21T04:29:39.334 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:38 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:38 smithi053 ceph-mon[35554]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:38 smithi053 ceph-mon[35554]: from='osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T04:29:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:38 smithi053 ceph-mon[35554]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T04:29:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:38 smithi053 ceph-mon[35554]: from='osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-21T04:29:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:38 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:39 smithi007 ceph-mon[27954]: from='osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-21T04:29:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:39 smithi007 ceph-mon[27954]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T04:29:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:40.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:29:40.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-09-21T04:29:40.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:39 smithi053 ceph-mon[35554]: from='osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-21T04:29:40.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:39 smithi053 ceph-mon[35554]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T04:29:40.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:39 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:40.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:39 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:40.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:39 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:40.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:39 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:40 smithi053 ceph-mon[35554]: purged_snaps scrub starts 2024-09-21T04:29:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:40 smithi053 ceph-mon[35554]: purged_snaps scrub ok 2024-09-21T04:29:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:40 smithi053 ceph-mon[35554]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:40 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:40 smithi053 ceph-mon[35554]: osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] boot 2024-09-21T04:29:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:40 smithi053 ceph-mon[35554]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T04:29:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:40 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:40 smithi007 ceph-mon[27954]: purged_snaps scrub starts 2024-09-21T04:29:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:40 smithi007 ceph-mon[27954]: purged_snaps scrub ok 2024-09-21T04:29:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:40 smithi007 ceph-mon[27954]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T04:29:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:40 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:40 smithi007 ceph-mon[27954]: osd.1 [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] boot 2024-09-21T04:29:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:40 smithi007 ceph-mon[27954]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T04:29:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:40 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:29:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:29:42.034 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:42 smithi007 ceph-mon[27954]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T04:29:42.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:42 smithi053 ceph-mon[35554]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T04:29:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.409+0000 7f9c21acf640 1 -- 172.21.15.7:0/1073257406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c101a80 msgr2=0x7f9c1c101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:29:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.409+0000 7f9c21acf640 1 --2- 172.21.15.7:0/1073257406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c101a80 0x7f9c1c101f00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009880 tx=0x7f9c0402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:29:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.409+0000 7f9c21acf640 1 -- 172.21.15.7:0/1073257406 shutdown_connections 2024-09-21T04:29:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.409+0000 7f9c21acf640 1 --2- 172.21.15.7:0/1073257406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c101a80 0x7f9c1c101f00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.409+0000 7f9c21acf640 1 --2- 172.21.15.7:0/1073257406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c1c100880 0x7f9c1c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.409+0000 7f9c21acf640 1 -- 172.21.15.7:0/1073257406 >> 172.21.15.7:0/1073257406 conn(0x7f9c1c0fc030 msgr2=0x7f9c1c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:29:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.409+0000 7f9c21acf640 1 -- 172.21.15.7:0/1073257406 shutdown_connections 2024-09-21T04:29:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.409+0000 7f9c21acf640 1 -- 172.21.15.7:0/1073257406 wait complete. 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c21acf640 1 Processor -- start 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c21acf640 1 -- start start 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c21acf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100880 0x7f9c1c06afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c21acf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c1c101a80 0x7f9c1c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c21acf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1096c0 con 0x7f9c1c100880 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c21acf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c109540 con 0x7f9c1c101a80 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c1b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100880 0x7f9c1c06afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c1b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100880 0x7f9c1c06afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41664/0 (socket says 172.21.15.7:41664) 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c1b7fe640 1 -- 172.21.15.7:0/1031567560 learned_addr learned my addr 172.21.15.7:0/1031567560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.410+0000 7f9c12ffd640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c1c101a80 0x7f9c1c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c1b7fe640 1 -- 172.21.15.7:0/1031567560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c1c101a80 msgr2=0x7f9c1c19c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:29:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c1b7fe640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c1c101a80 0x7f9c1c19c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c1b7fe640 1 -- 172.21.15.7:0/1031567560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c19c830 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c1b7fe640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100880 0x7f9c1c06afb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00d8d0 tx=0x7f9c0c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c197fa640 1 -- 172.21.15.7:0/1031567560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9c0c017070 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c19cb20 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c197fa640 1 -- 172.21.15.7:0/1031567560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c0c00be10 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c197fa640 1 -- 172.21.15.7:0/1031567560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9c0c00f3c0 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.411+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c19d080 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.412+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c100c80 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.413+0000 7f9c197fa640 1 -- 172.21.15.7:0/1031567560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7f9c0c01a070 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.414+0000 7f9c197fa640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f9c0806e230 0x7f9c080706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.414+0000 7f9c197fa640 1 -- 172.21.15.7:0/1031567560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2414+0+0 (secure 0 0 0) 0x7f9c0c08d040 con 0x7f9c1c100880 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.414+0000 7f9c12ffd640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f9c0806e230 0x7f9c080706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:29:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.414+0000 7f9c12ffd640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f9c0806e230 0x7f9c080706f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c04000c00 tx=0x7f9c040058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:29:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.420+0000 7f9c197fa640 1 -- 172.21.15.7:0/1031567560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c0c05f290 con 0x7f9c1c100880 2024-09-21T04:29:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:29:42.619+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9c1c106030 con 0x7f9c0806e230 2024-09-21T04:29:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:43 smithi053 ceph-mon[35554]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:43 smithi053 ceph-mon[35554]: from='client.14366 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:29:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:43 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:29:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:43 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:29:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:43 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:43 smithi007 ceph-mon[27954]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:43 smithi007 ceph-mon[27954]: from='client.14366 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:29:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:29:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:29:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: Detected new or changed devices on smithi007 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: Adjusting osd_memory_target on smithi007 to 5922M 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:44.851 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:44 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:29:44.897 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:44.897 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: Detected new or changed devices on smithi007 2024-09-21T04:29:44.897 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:44.897 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:44.897 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:29:44.897 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:29:44.897 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: Adjusting osd_memory_target on smithi007 to 5922M 2024-09-21T04:29:44.897 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:44.898 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:44.898 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:29:44.898 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:44.898 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:44 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:29:46.780 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:46 smithi007 ceph-mon[27954]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:46.780 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:46.780 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:46.780 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:46 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1276862704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63457a4d-975d-4ea1-b6ac-770bf49aead3"}]: dispatch 2024-09-21T04:29:46.781 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:46 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1276862704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63457a4d-975d-4ea1-b6ac-770bf49aead3"}]': finished 2024-09-21T04:29:46.781 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:46 smithi007 ceph-mon[27954]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T04:29:46.781 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:29:46.781 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:29:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:46 smithi053 ceph-mon[35554]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:46 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1276862704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63457a4d-975d-4ea1-b6ac-770bf49aead3"}]: dispatch 2024-09-21T04:29:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:46 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1276862704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63457a4d-975d-4ea1-b6ac-770bf49aead3"}]': finished 2024-09-21T04:29:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:46 smithi053 ceph-mon[35554]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T04:29:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:29:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:29:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:47 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3730856679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:29:47.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:47 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3730856679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:29:48.792 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:48 smithi007 ceph-mon[27954]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:48.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:48 smithi053 ceph-mon[35554]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:50.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:50 smithi053 ceph-mon[35554]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:50 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:50 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:50 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:50 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:29:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:50 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:50.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:50 smithi007 ceph-mon[27954]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:50.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:50 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:50.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:50 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:50.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:50 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:50.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:50 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:29:50.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:50 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:29:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:52 smithi053 ceph-mon[35554]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:52.533 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:52 smithi007 ceph-mon[27954]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:54.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:54 smithi053 ceph-mon[35554]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:54 smithi007 ceph-mon[27954]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:55.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:55 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-21T04:29:55.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:55 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:55.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:55 smithi007 ceph-mon[27954]: Deploying daemon osd.2 on smithi007 2024-09-21T04:29:55.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:55 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-21T04:29:55.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:55 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:29:55.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:55 smithi053 ceph-mon[35554]: Deploying daemon osd.2 on smithi007 2024-09-21T04:29:56.594 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:56 smithi007 ceph-mon[27954]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:56.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:56 smithi053 ceph-mon[35554]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:58.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:29:58 smithi053 ceph-mon[35554]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:29:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:29:58 smithi007 ceph-mon[27954]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:00.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:00 smithi053 ceph-mon[35554]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:00.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:00 smithi053 ceph-mon[35554]: overall HEALTH_OK 2024-09-21T04:30:00.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:00 smithi007 ceph-mon[27954]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:00.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:00 smithi007 ceph-mon[27954]: overall HEALTH_OK 2024-09-21T04:30:01.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:01.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:01.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:01.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:30:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:30:02.927 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:02 smithi007 ceph-mon[27954]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:02.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:02 smithi053 ceph-mon[35554]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.678+0000 7f9c197fa640 1 -- 172.21.15.7:0/1031567560 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9c1c106030 con 0x7f9c0806e230 2024-09-21T04:30:04.681 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 2 on host 'smithi007' 2024-09-21T04:30:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.680+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f9c0806e230 msgr2=0x7f9c080706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.680+0000 7f9c21acf640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f9c0806e230 0x7f9c080706f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c04000c00 tx=0x7f9c040058d0 comp rx=0 tx=0).stop 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.680+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100880 msgr2=0x7f9c1c06afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.680+0000 7f9c21acf640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100880 0x7f9c1c06afb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00d8d0 tx=0x7f9c0c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.681+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 shutdown_connections 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.681+0000 7f9c21acf640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f9c0806e230 0x7f9c080706f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.681+0000 7f9c21acf640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c1c101a80 0x7f9c1c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.681+0000 7f9c21acf640 1 --2- 172.21.15.7:0/1031567560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c1c100880 0x7f9c1c06afb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.681+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 >> 172.21.15.7:0/1031567560 conn(0x7f9c1c0fc030 msgr2=0x7f9c1c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.681+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 shutdown_connections 2024-09-21T04:30:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:30:04.681+0000 7f9c21acf640 1 -- 172.21.15.7:0/1031567560 wait complete. 2024-09-21T04:30:04.912 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:04 smithi007 ceph-mon[27954]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:04.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:04 smithi053 ceph-mon[35554]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:05.437 DEBUG:teuthology.orchestra.run.smithi007:osd.2> sudo journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T04:30:05.439 INFO:tasks.cephadm:Deploying osd.3 on smithi053 with /dev/vg_nvme/lv_4... 2024-09-21T04:30:05.439 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T04:30:05.713 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:30:05.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:05.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:05 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:05.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:05.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:05 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:06.877 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:06 smithi053 ceph-mon[35554]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:06.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:06 smithi007 ceph-mon[27954]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:06.944 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:06 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[63454]: 2024-09-21T04:30:06.870+0000 7fd815422640 -1 osd.2 0 log_to_monitors true 2024-09-21T04:30:07.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:07 smithi053 ceph-mon[35554]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-21T04:30:07.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:07 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:07.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:07 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:07.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:07 smithi007 ceph-mon[27954]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-21T04:30:07.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:07 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:07.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:07 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:08.356 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:30:08.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-09-21T04:30:08.648 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:30:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:08 smithi053 ceph-mon[35554]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:08 smithi053 ceph-mon[35554]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T04:30:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:08 smithi053 ceph-mon[35554]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T04:30:08.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:08 smithi053 ceph-mon[35554]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-21T04:30:08.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:08 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:08 smithi007 ceph-mon[27954]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:08 smithi007 ceph-mon[27954]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T04:30:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:08 smithi007 ceph-mon[27954]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T04:30:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:08 smithi007 ceph-mon[27954]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-21T04:30:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:08 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:09.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.063+0000 7fa3cc750640 1 -- 172.21.15.53:0/555367558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 msgr2=0x7fa3c4102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.063+0000 7fa3cc750640 1 --2- 172.21.15.53:0/555367558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 0x7fa3c4102fe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009880 tx=0x7fa3b402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.063+0000 7fa3cc750640 1 -- 172.21.15.53:0/555367558 shutdown_connections 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.063+0000 7fa3cc750640 1 --2- 172.21.15.53:0/555367558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 0x7fa3c4102fe0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.063+0000 7fa3cc750640 1 --2- 172.21.15.53:0/555367558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c4101960 0x7fa3c4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.063+0000 7fa3cc750640 1 -- 172.21.15.53:0/555367558 >> 172.21.15.53:0/555367558 conn(0x7fa3c40fd110 msgr2=0x7fa3c40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.063+0000 7fa3cc750640 1 -- 172.21.15.53:0/555367558 shutdown_connections 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3cc750640 1 -- 172.21.15.53:0/555367558 wait complete. 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3cc750640 1 Processor -- start 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3cc750640 1 -- start start 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3cc750640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c4101960 0x7fa3c41974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3cc750640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 0x7fa3c4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3cc750640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410a7a0 con 0x7fa3c4101960 2024-09-21T04:30:09.068 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3cc750640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410a620 con 0x7fa3c4102b60 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3c9ceb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 0x7fa3c4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.064+0000 7fa3c9ceb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 0x7fa3c4197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47440/0 (socket says 172.21.15.53:47440) 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3c9ceb640 1 -- 172.21.15.53:0/92783280 learned_addr learned my addr 172.21.15.53:0/92783280 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3ca4ec640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c4101960 0x7fa3c41974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3c9ceb640 1 -- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c4101960 msgr2=0x7fa3c41974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3c9ceb640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c4101960 0x7fa3c41974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3c9ceb640 1 -- 172.21.15.53:0/92783280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c4198080 con 0x7fa3c4102b60 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3ca4ec640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c4101960 0x7fa3c41974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:30:09.069 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3c9ceb640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 0x7fa3c4197a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009230 tx=0x7fa3b4004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:30:09.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3bb7fe640 1 -- 172.21.15.53:0/92783280 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa3b403f070 con 0x7fa3c4102b60 2024-09-21T04:30:09.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419ca60 con 0x7fa3c4102b60 2024-09-21T04:30:09.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3bb7fe640 1 -- 172.21.15.53:0/92783280 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3b4004a90 con 0x7fa3c4102b60 2024-09-21T04:30:09.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3bb7fe640 1 -- 172.21.15.53:0/92783280 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa3b4035030 con 0x7fa3c4102b60 2024-09-21T04:30:09.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.065+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c419cff0 con 0x7fa3c4102b60 2024-09-21T04:30:09.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.066+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c4101d60 con 0x7fa3c4102b60 2024-09-21T04:30:09.070 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.067+0000 7fa3bb7fe640 1 -- 172.21.15.53:0/92783280 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) ==== 92493+0+0 (secure 0 0 0) 0x7fa3b4042070 con 0x7fa3c4102b60 2024-09-21T04:30:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.068+0000 7fa3bb7fe640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa3b006e240 0x7fa3b0070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:30:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.068+0000 7fa3bb7fe640 1 -- 172.21.15.53:0/92783280 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2566+0+0 (secure 0 0 0) 0x7fa3b40ae090 con 0x7fa3c4102b60 2024-09-21T04:30:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.068+0000 7fa3ca4ec640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa3b006e240 0x7fa3b0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:30:09.071 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.069+0000 7fa3ca4ec640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa3b006e240 0x7fa3b0070700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3c41029c0 tx=0x7fa3c0005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:30:09.075 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.072+0000 7fa3bb7fe640 1 -- 172.21.15.53:0/92783280 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa3b4080230 con 0x7fa3c4102b60 2024-09-21T04:30:09.262 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:09.259+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa3c4062630 con 0x7fa3b006e240 2024-09-21T04:30:09.680 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[63454]: 2024-09-21T04:30:09.411+0000 7fd810b82640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T04:30:09.680 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[63454]: 2024-09-21T04:30:09.417+0000 7fd8091b4640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:30:09.935 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:09 smithi053 ceph-mon[35554]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-21T04:30:09.935 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:09 smithi053 ceph-mon[35554]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T04:30:09.935 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:09.935 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:09.935 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:30:09.935 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:30:09.935 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:09 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:09.935 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:09 smithi053 ceph-mon[35554]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' 2024-09-21T04:30:09.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-mon[27954]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-21T04:30:09.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-mon[27954]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T04:30:09.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:09.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:09.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:30:09.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:30:09.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:09.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:09 smithi007 ceph-mon[27954]: from='osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876]' entity='osd.2' 2024-09-21T04:30:10.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:10 smithi007 ceph-mon[27954]: purged_snaps scrub starts 2024-09-21T04:30:10.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:10 smithi007 ceph-mon[27954]: purged_snaps scrub ok 2024-09-21T04:30:10.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:10 smithi007 ceph-mon[27954]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:10.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:10 smithi007 ceph-mon[27954]: from='client.24183 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:30:10.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:10 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:10.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:10 smithi007 ceph-mon[27954]: osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] boot 2024-09-21T04:30:10.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:10 smithi007 ceph-mon[27954]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T04:30:10.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:10 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:10.959 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:10 smithi053 ceph-mon[35554]: purged_snaps scrub starts 2024-09-21T04:30:10.959 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:10 smithi053 ceph-mon[35554]: purged_snaps scrub ok 2024-09-21T04:30:10.959 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:10 smithi053 ceph-mon[35554]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T04:30:10.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:10 smithi053 ceph-mon[35554]: from='client.24183 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:30:10.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:10 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:10.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:10 smithi053 ceph-mon[35554]: osd.2 [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] boot 2024-09-21T04:30:10.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:10 smithi053 ceph-mon[35554]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T04:30:10.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:10 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='client.? 172.21.15.53:0/698147262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d758b507-1cd4-4c67-b7fb-917f65419e07"}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d758b507-1cd4-4c67-b7fb-917f65419e07"}]: dispatch 2024-09-21T04:30:11.948 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T04:30:11.949 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d758b507-1cd4-4c67-b7fb-917f65419e07"}]': finished 2024-09-21T04:30:11.949 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T04:30:11.949 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:11.949 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-21T04:30:11.949 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:11 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:12.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-21T04:30:12.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/698147262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d758b507-1cd4-4c67-b7fb-917f65419e07"}]: dispatch 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d758b507-1cd4-4c67-b7fb-917f65419e07"}]: dispatch 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d758b507-1cd4-4c67-b7fb-917f65419e07"}]': finished 2024-09-21T04:30:12.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: osdmap e20: 4 total, 3 up, 4 in 2024-09-21T04:30:12.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:12.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-21T04:30:12.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:11 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:12 smithi007 ceph-mon[27954]: pgmap v87: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:12 smithi007 ceph-mon[27954]: Detected new or changed devices on smithi007 2024-09-21T04:30:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:12 smithi007 ceph-mon[27954]: Adjusting osd_memory_target on smithi007 to 3948M 2024-09-21T04:30:12.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:12 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3363274527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:30:12.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:12 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T04:30:12.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:12 smithi007 ceph-mon[27954]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T04:30:12.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:12 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:12 smithi053 ceph-mon[35554]: pgmap v87: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:12 smithi053 ceph-mon[35554]: Detected new or changed devices on smithi007 2024-09-21T04:30:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:12 smithi053 ceph-mon[35554]: Adjusting osd_memory_target on smithi007 to 3948M 2024-09-21T04:30:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:12 smithi053 ceph-mon[35554]: from='client.? 172.21.15.53:0/3363274527' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:30:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:12 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T04:30:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:12 smithi053 ceph-mon[35554]: osdmap e21: 4 total, 3 up, 4 in 2024-09-21T04:30:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:12 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:14.069 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:30:13 smithi007 sudo[68209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:30:14.069 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:30:13 smithi007 sudo[68209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:14.069 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:30:13 smithi007 sudo[68209]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:14.069 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:30:13 smithi007 sudo[68209]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:14.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:30:14 smithi007 sudo[68244]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:30:14.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:30:14 smithi007 sudo[68244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:14.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:30:14 smithi007 sudo[68244]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:14.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:30:14 smithi007 sudo[68244]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:14.944 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:30:14 smithi007 sudo[68306]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:30:14.944 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:30:14 smithi007 sudo[68306]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:14.944 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:30:14 smithi007 sudo[68306]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:14.945 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:30:14 smithi007 sudo[68306]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:14 smithi007 ceph-mon[27954]: pgmap v90: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:14 smithi007 ceph-mon[27954]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T04:30:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:14 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:14 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:14 smithi053 ceph-mon[35554]: pgmap v90: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:14 smithi053 ceph-mon[35554]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T04:30:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:14 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:14 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:15.369 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:30:15 smithi007 sudo[68352]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:30:15.369 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:30:15 smithi007 sudo[68352]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:15.369 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:30:15 smithi007 sudo[68352]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:15.369 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:30:15 smithi007 sudo[68352]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:15.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:15 smithi007 sudo[68392]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:30:15.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:15 smithi007 sudo[68392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:15.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:15 smithi007 sudo[68392]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:15.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:15 smithi007 sudo[68392]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:15.708 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:15.706+0000 7fa3bb7fe640 1 -- 172.21.15.53:0/92783280 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fa3b4003c40 con 0x7fa3c4102b60 2024-09-21T04:30:16.444 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68481]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:30:16.444 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68481]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:16.444 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68481]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:16.444 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68481]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:16.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68484]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:30:16.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68484]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:16.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68484]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:16.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68484]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: pgmap v92: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: mgrmap e19: smithi007.huohfl(active, since 2m), standbys: smithi053.lnadao 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:30:17.252 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:16 smithi053 ceph-mon[35554]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T04:30:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68487]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:30:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68487]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68487]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 sudo[68487]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: pgmap v92: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: mgrmap e19: smithi007.huohfl(active, since 2m), standbys: smithi053.lnadao 2024-09-21T04:30:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:17.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:17.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:17.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:17.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:17.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:30:17.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:16 smithi007 ceph-mon[27954]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T04:30:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 sudo[40800]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T04:30:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 sudo[40800]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 sudo[40800]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 sudo[40800]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 sudo[40861]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 sudo[40861]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 sudo[40861]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 sudo[40861]: pam_unix(sudo:session): session closed for user root 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 ceph-mon[35554]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 ceph-mon[35554]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:30:18.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 ceph-mon[35554]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T04:30:18.023 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:17 smithi007 ceph-mon[27954]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T04:30:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:30:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:30:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:17 smithi007 ceph-mon[27954]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T04:30:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:30:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:30:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:17 smithi007 ceph-mon[27954]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T04:30:18.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:19.022 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:18 smithi053 ceph-mon[35554]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:18 smithi007 ceph-mon[27954]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:20.306 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:19 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-21T04:30:20.309 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:19 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:20.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:19 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-21T04:30:20.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:19 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:21.229 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:20 smithi053 ceph-mon[35554]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:21.229 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:20 smithi053 ceph-mon[35554]: Deploying daemon osd.3 on smithi053 2024-09-21T04:30:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:20 smithi007 ceph-mon[27954]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:20 smithi007 ceph-mon[27954]: Deploying daemon osd.3 on smithi053 2024-09-21T04:30:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:22 smithi053 ceph-mon[35554]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-09-21T04:30:23.223 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:22 smithi007 ceph-mon[27954]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-09-21T04:30:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:24 smithi053 ceph-mon[35554]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-21T04:30:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:24 smithi007 ceph-mon[27954]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-21T04:30:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:26 smithi053 ceph-mon[35554]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-21T04:30:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:26 smithi007 ceph-mon[27954]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-21T04:30:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:29.111 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:28 smithi053 ceph-mon[35554]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-21T04:30:29.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:28 smithi007 ceph-mon[27954]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-21T04:30:30.463 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.459+0000 7fa3bb7fe640 1 -- 172.21.15.53:0/92783280 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa3c4062630 con 0x7fa3b006e240 2024-09-21T04:30:30.463 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 3 on host 'smithi053' 2024-09-21T04:30:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa3b006e240 msgr2=0x7fa3b0070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa3b006e240 0x7fa3b0070700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3c41029c0 tx=0x7fa3c0005f90 comp rx=0 tx=0).stop 2024-09-21T04:30:30.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 msgr2=0x7fa3c4197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 0x7fa3c4197a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009230 tx=0x7fa3b4004570 comp rx=0 tx=0).stop 2024-09-21T04:30:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 shutdown_connections 2024-09-21T04:30:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa3b006e240 0x7fa3b0070700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3c4102b60 0x7fa3c4197a00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 --2- 172.21.15.53:0/92783280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c4101960 0x7fa3c41974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 >> 172.21.15.53:0/92783280 conn(0x7fa3c40fd110 msgr2=0x7fa3c4106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:30:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 shutdown_connections 2024-09-21T04:30:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:30.462+0000 7fa3cc750640 1 -- 172.21.15.53:0/92783280 wait complete. 2024-09-21T04:30:30.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-21T04:30:30.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:30.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:30.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:30.785 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.786 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-21T04:30:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:30.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:31.076 DEBUG:teuthology.orchestra.run.smithi053:osd.3> sudo journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T04:30:31.078 INFO:tasks.cephadm:Deploying osd.4 on smithi053 with /dev/vg_nvme/lv_3... 2024-09-21T04:30:31.078 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T04:30:31.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:31 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:30:31.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:30:32.511 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:30:32.887 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:32 smithi053 ceph-mon[35554]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-21T04:30:32.887 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:30:32 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[41479]: 2024-09-21T04:30:32.753+0000 7fe4689fa640 -1 osd.3 0 log_to_monitors true 2024-09-21T04:30:32.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:32 smithi007 ceph-mon[27954]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-21T04:30:33.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:33 smithi053 ceph-mon[35554]: from='osd.3 [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-21T04:30:33.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:33 smithi053 ceph-mon[35554]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-21T04:30:34.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:33 smithi007 ceph-mon[27954]: from='osd.3 [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-21T04:30:34.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:33 smithi007 ceph-mon[27954]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-21T04:30:35.121 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[41479]: 2024-09-21T04:30:34.702+0000 7fe46415a640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T04:30:35.121 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[41479]: 2024-09-21T04:30:34.711+0000 7fe460794640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:30:35.121 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='osd.3 [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:35.122 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:34 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:35.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T04:30:35.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T04:30:35.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='osd.3 [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:30:35.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:35.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:30:35.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:35.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:35.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:30:35.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:35.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:35.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:35.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:34 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:35.392 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:30:35.422 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-09-21T04:30:35.684 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:30:35.959 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:35 smithi053 ceph-mon[35554]: Detected new or changed devices on smithi053 2024-09-21T04:30:35.959 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:35 smithi053 ceph-mon[35554]: Adjusting osd_memory_target on smithi053 to 14916M 2024-09-21T04:30:35.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:35 smithi053 ceph-mon[35554]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-21T04:30:35.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:35 smithi053 ceph-mon[35554]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T04:30:35.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:35 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:35.960 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:35 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.131+0000 7f8af3f42640 1 -- 172.21.15.53:0/1801276585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 msgr2=0x7f8aec107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.131+0000 7f8af3f42640 1 --2- 172.21.15.53:0/1801276585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 0x7f8aec107110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009880 tx=0x7f8adc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:30:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 -- 172.21.15.53:0/1801276585 shutdown_connections 2024-09-21T04:30:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 --2- 172.21.15.53:0/1801276585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 0x7f8aec107110 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 --2- 172.21.15.53:0/1801276585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec1023f0 0x7f8aec1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 -- 172.21.15.53:0/1801276585 >> 172.21.15.53:0/1801276585 conn(0x7f8aec0fc010 msgr2=0x7f8aec0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 -- 172.21.15.53:0/1801276585 shutdown_connections 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 -- 172.21.15.53:0/1801276585 wait complete. 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 Processor -- start 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 -- start start 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec1023f0 0x7f8aec197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 0x7f8aec197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec10a810 con 0x7f8aec1023f0 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.132+0000 7f8af3f42640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec10a690 con 0x7f8aec104d20 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af14dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 0x7f8aec197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af14dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 0x7f8aec197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35742/0 (socket says 172.21.15.53:35742) 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af14dd640 1 -- 172.21.15.53:0/1702567223 learned_addr learned my addr 172.21.15.53:0/1702567223 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-21T04:30:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af1cde640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec1023f0 0x7f8aec197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af14dd640 1 -- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec1023f0 msgr2=0x7f8aec197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af14dd640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec1023f0 0x7f8aec197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af14dd640 1 -- 172.21.15.53:0/1702567223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec198130 con 0x7f8aec104d20 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af14dd640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 0x7f8aec197ab0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0048f0 tx=0x7f8adc004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/1702567223 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8adc03f070 con 0x7f8aec104d20 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/1702567223 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8adc003ab0 con 0x7f8aec104d20 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec19cb10 con 0x7f8aec104d20 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/1702567223 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8adc035030 con 0x7f8aec104d20 2024-09-21T04:30:36.139 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.133+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec19d120 con 0x7f8aec104d20 2024-09-21T04:30:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.135+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec1027d0 con 0x7f8aec104d20 2024-09-21T04:30:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.135+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/1702567223 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f8adc042070 con 0x7f8aec104d20 2024-09-21T04:30:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.135+0000 7f8ae2ffd640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ad806e310 0x7f8ad80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:30:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.136+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/1702567223 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3807+0+0 (secure 0 0 0) 0x7f8adc0ae6f0 con 0x7f8aec104d20 2024-09-21T04:30:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.136+0000 7f8af1cde640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ad806e310 0x7f8ad80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:30:36.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.136+0000 7f8af1cde640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ad806e310 0x7f8ad80707d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009890 tx=0x7f8ae8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:30:36.144 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.140+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/1702567223 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8adc07c750 con 0x7f8aec104d20 2024-09-21T04:30:36.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:35 smithi007 ceph-mon[27954]: Detected new or changed devices on smithi053 2024-09-21T04:30:36.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:35 smithi007 ceph-mon[27954]: Adjusting osd_memory_target on smithi053 to 14916M 2024-09-21T04:30:36.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:35 smithi007 ceph-mon[27954]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-21T04:30:36.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:35 smithi007 ceph-mon[27954]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T04:30:36.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:35 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:36.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:35 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:36.329 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:36.326+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8aec105700 con 0x7f8ad806e310 2024-09-21T04:30:36.977 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: purged_snaps scrub starts 2024-09-21T04:30:36.977 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: purged_snaps scrub ok 2024-09-21T04:30:36.978 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:36.978 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: osd.3 [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] boot 2024-09-21T04:30:36.978 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T04:30:36.978 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:36.978 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:30:36.978 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:30:36.978 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: purged_snaps scrub starts 2024-09-21T04:30:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: purged_snaps scrub ok 2024-09-21T04:30:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T04:30:37.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: osd.3 [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] boot 2024-09-21T04:30:37.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T04:30:37.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:30:37.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:30:37.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:30:37.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:37.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:37 smithi053 ceph-mon[35554]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:30:37.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:37 smithi053 ceph-mon[35554]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T04:30:38.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:37 smithi007 ceph-mon[27954]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:30:38.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:37 smithi007 ceph-mon[27954]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T04:30:38.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:38 smithi053 ceph-mon[35554]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:38.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:38 smithi053 ceph-mon[35554]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T04:30:38.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:38 smithi053 ceph-mon[35554]: from='client.? 172.21.15.53:0/2193990289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea39b5-2b2e-4ba0-8224-c8642be0d3c4"}]: dispatch 2024-09-21T04:30:38.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:38 smithi053 ceph-mon[35554]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea39b5-2b2e-4ba0-8224-c8642be0d3c4"}]: dispatch 2024-09-21T04:30:38.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:38 smithi053 ceph-mon[35554]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bea39b5-2b2e-4ba0-8224-c8642be0d3c4"}]': finished 2024-09-21T04:30:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:38 smithi007 ceph-mon[27954]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:38 smithi007 ceph-mon[27954]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T04:30:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:38 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/2193990289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea39b5-2b2e-4ba0-8224-c8642be0d3c4"}]: dispatch 2024-09-21T04:30:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:38 smithi007 ceph-mon[27954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea39b5-2b2e-4ba0-8224-c8642be0d3c4"}]: dispatch 2024-09-21T04:30:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:38 smithi007 ceph-mon[27954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bea39b5-2b2e-4ba0-8224-c8642be0d3c4"}]': finished 2024-09-21T04:30:39.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:39 smithi053 ceph-mon[35554]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T04:30:39.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:39 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:30:39.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:39 smithi053 ceph-mon[35554]: from='client.? 172.21.15.53:0/823744556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:30:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:39 smithi007 ceph-mon[27954]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T04:30:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:30:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:39 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/823744556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:30:41.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:40 smithi053 ceph-mon[35554]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:41.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:40 smithi053 ceph-mon[35554]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T04:30:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:40 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:30:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:40 smithi007 ceph-mon[27954]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:40 smithi007 ceph-mon[27954]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T04:30:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:40 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:30:42.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:42 smithi007 ceph-mon[27954]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-21T04:30:43.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:42 smithi053 ceph-mon[35554]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-21T04:30:44.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:44 smithi053 ceph-mon[35554]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T04:30:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:44 smithi007 ceph-mon[27954]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T04:30:47.108 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:46 smithi053 ceph-mon[35554]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T04:30:47.108 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:30:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:46 smithi007 ceph-mon[27954]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T04:30:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:30:47.880 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:47 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-21T04:30:47.880 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:47 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:48.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:47 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-21T04:30:48.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:47 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:48.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:48 smithi053 ceph-mon[35554]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T04:30:48.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:48 smithi053 ceph-mon[35554]: Deploying daemon osd.4 on smithi053 2024-09-21T04:30:49.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:48 smithi007 ceph-mon[27954]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T04:30:49.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:48 smithi007 ceph-mon[27954]: Deploying daemon osd.4 on smithi053 2024-09-21T04:30:50.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:50 smithi053 ceph-mon[35554]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T04:30:51.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:50 smithi007 ceph-mon[27954]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T04:30:52.884 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:52 smithi053 ceph-mon[35554]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T04:30:52.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:52 smithi007 ceph-mon[27954]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T04:30:53.921 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:53.921 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:53.921 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:53 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:53 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:55.087 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:54 smithi053 ceph-mon[35554]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:54 smithi007 ceph-mon[27954]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:56 smithi007 ceph-mon[27954]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:56 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:57.004 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:56 smithi053 ceph-mon[35554]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:57.004 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:57.004 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:57.004 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:30:57.004 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:30:57.005 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:56 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:57.043 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 4 on host 'smithi053' 2024-09-21T04:30:57.044 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.040+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/1702567223 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8aec105700 con 0x7f8ad806e310 2024-09-21T04:30:57.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ad806e310 msgr2=0x7f8ad80707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:57.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ad806e310 0x7f8ad80707d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009890 tx=0x7f8ae8008040 comp rx=0 tx=0).stop 2024-09-21T04:30:57.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 msgr2=0x7f8aec197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:30:57.045 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 0x7f8aec197ab0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0048f0 tx=0x7f8adc004920 comp rx=0 tx=0).stop 2024-09-21T04:30:57.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 shutdown_connections 2024-09-21T04:30:57.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ad806e310 0x7f8ad80707d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:57.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec104d20 0x7f8aec197ab0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:57.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 --2- 172.21.15.53:0/1702567223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec1023f0 0x7f8aec197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:30:57.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 >> 172.21.15.53:0/1702567223 conn(0x7f8aec0fc010 msgr2=0x7f8aec0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:30:57.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 shutdown_connections 2024-09-21T04:30:57.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:30:57.043+0000 7f8af3f42640 1 -- 172.21.15.53:0/1702567223 wait complete. 2024-09-21T04:30:57.584 DEBUG:teuthology.orchestra.run.smithi053:osd.4> sudo journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T04:30:57.586 INFO:tasks.cephadm:Deploying osd.5 on smithi053 with /dev/vg_nvme/lv_2... 2024-09-21T04:30:57.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T04:30:57.901 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:57.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:57.902 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:57 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:58.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:30:58.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:58.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:57 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:30:58.293 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:30:59.009 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:58 smithi053 ceph-mon[35554]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:58 smithi007 ceph-mon[27954]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:30:59.310 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:30:59 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[47937]: 2024-09-21T04:30:59.212+0000 7fb5a5701640 -1 osd.4 0 log_to_monitors true 2024-09-21T04:30:59.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:59 smithi053 ceph-mon[35554]: from='osd.4 [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-21T04:30:59.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:30:59 smithi053 ceph-mon[35554]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-21T04:31:00.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:59 smithi007 ceph-mon[27954]: from='osd.4 [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-21T04:31:00.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:30:59 smithi007 ceph-mon[27954]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-21T04:31:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:00 smithi053 ceph-mon[35554]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:31:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:00 smithi053 ceph-mon[35554]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T04:31:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:00 smithi053 ceph-mon[35554]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T04:31:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:00 smithi053 ceph-mon[35554]: from='osd.4 [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:31:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:00 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:31:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:00 smithi053 ceph-mon[35554]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:31:01.055 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:00 smithi007 ceph-mon[27954]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:31:01.055 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:00 smithi007 ceph-mon[27954]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T04:31:01.055 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:00 smithi007 ceph-mon[27954]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T04:31:01.056 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:00 smithi007 ceph-mon[27954]: from='osd.4 [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:31:01.056 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:00 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:31:01.056 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:00 smithi007 ceph-mon[27954]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:31:01.685 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[47937]: 2024-09-21T04:31:01.494+0000 7fb5a0e61640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T04:31:01.686 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[47937]: 2024-09-21T04:31:01.500+0000 7fb599493640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: Detected new or changed devices on smithi053 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: Adjusting osd_memory_target on smithi053 to 7458M 2024-09-21T04:31:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:01.945 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:01.945 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:31:01.945 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:01.945 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:31:01.945 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:01 smithi053 ceph-mon[35554]: from='osd.4 ' entity='osd.4' 2024-09-21T04:31:01.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-21T04:31:01.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T04:31:01.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:31:01.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:31:01.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: Detected new or changed devices on smithi053 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: Adjusting osd_memory_target on smithi053 to 7458M 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:31:01.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:01 smithi007 ceph-mon[27954]: from='osd.4 ' entity='osd.4' 2024-09-21T04:31:02.439 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-21T04:31:02.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-09-21T04:31:02.724 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:31:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:02 smithi007 ceph-mon[27954]: purged_snaps scrub starts 2024-09-21T04:31:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:02 smithi007 ceph-mon[27954]: purged_snaps scrub ok 2024-09-21T04:31:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:02 smithi007 ceph-mon[27954]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:31:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:02 smithi007 ceph-mon[27954]: osd.4 [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] boot 2024-09-21T04:31:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:02 smithi007 ceph-mon[27954]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T04:31:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:02 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:31:03.097 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:02 smithi053 ceph-mon[35554]: purged_snaps scrub starts 2024-09-21T04:31:03.097 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:02 smithi053 ceph-mon[35554]: purged_snaps scrub ok 2024-09-21T04:31:03.097 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:02 smithi053 ceph-mon[35554]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T04:31:03.097 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:02 smithi053 ceph-mon[35554]: osd.4 [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] boot 2024-09-21T04:31:03.097 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:02 smithi053 ceph-mon[35554]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T04:31:03.097 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:02 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:31:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.179+0000 7f3495141640 1 -- 172.21.15.53:0/2369656532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490102b60 msgr2=0x7f3490102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:03.182 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.179+0000 7f3495141640 1 --2- 172.21.15.53:0/2369656532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490102b60 0x7f3490102fe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3484009880 tx=0x7f348402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.179+0000 7f3495141640 1 -- 172.21.15.53:0/2369656532 shutdown_connections 2024-09-21T04:31:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.179+0000 7f3495141640 1 --2- 172.21.15.53:0/2369656532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490102b60 0x7f3490102fe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.179+0000 7f3495141640 1 --2- 172.21.15.53:0/2369656532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3490101960 0x7f3490101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.179+0000 7f3495141640 1 -- 172.21.15.53:0/2369656532 >> 172.21.15.53:0/2369656532 conn(0x7f34900fd0f0 msgr2=0x7f34900ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.179+0000 7f3495141640 1 -- 172.21.15.53:0/2369656532 shutdown_connections 2024-09-21T04:31:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.179+0000 7f3495141640 1 -- 172.21.15.53:0/2369656532 wait complete. 2024-09-21T04:31:03.183 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.180+0000 7f3495141640 1 Processor -- start 2024-09-21T04:31:03.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.180+0000 7f3495141640 1 -- start start 2024-09-21T04:31:03.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.180+0000 7f3495141640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101960 0x7f34901974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:03.184 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.180+0000 7f3495141640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3490102b60 0x7f34901979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.180+0000 7f3495141640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f349010a780 con 0x7f3490102b60 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.180+0000 7f3495141640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f349010a600 con 0x7f3490101960 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101960 0x7f34901974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101960 0x7f34901974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49116/0 (socket says 172.21.15.53:49116) 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348ed9d640 1 -- 172.21.15.53:0/89692925 learned_addr learned my addr 172.21.15.53:0/89692925 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348e59c640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3490102b60 0x7f34901979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348ed9d640 1 -- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3490102b60 msgr2=0x7f34901979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348ed9d640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3490102b60 0x7f34901979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348ed9d640 1 -- 172.21.15.53:0/89692925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3490197fe0 con 0x7f3490101960 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348e59c640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3490102b60 0x7f34901979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f348ed9d640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101960 0x7f34901974b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f348000c8f0 tx=0x7f348000cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f3477fff640 1 -- 172.21.15.53:0/89692925 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3480004020 con 0x7f3490101960 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349019ca50 con 0x7f3490101960 2024-09-21T04:31:03.185 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f3477fff640 1 -- 172.21.15.53:0/89692925 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3480007e10 con 0x7f3490101960 2024-09-21T04:31:03.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f3477fff640 1 -- 172.21.15.53:0/89692925 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3480003e10 con 0x7f3490101960 2024-09-21T04:31:03.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.181+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349019cfb0 con 0x7f3490101960 2024-09-21T04:31:03.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.183+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349010af60 con 0x7f3490101960 2024-09-21T04:31:03.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.183+0000 7f3477fff640 1 -- 172.21.15.53:0/89692925 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f34800041c0 con 0x7f3490101960 2024-09-21T04:31:03.186 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.183+0000 7f3477fff640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f347806e2b0 0x7f3478070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:03.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.184+0000 7f3477fff640 1 -- 172.21.15.53:0/89692925 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4206+0+0 (secure 0 0 0) 0x7f348008dea0 con 0x7f3490101960 2024-09-21T04:31:03.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.184+0000 7f348e59c640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f347806e2b0 0x7f3478070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:03.187 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.184+0000 7f348e59c640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f347806e2b0 0x7f3478070770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34840096c0 tx=0x7f3484009650 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:03.190 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.188+0000 7f3477fff640 1 -- 172.21.15.53:0/89692925 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3480015330 con 0x7f3490101960 2024-09-21T04:31:03.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:03.373+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f349002c640 con 0x7f347806e2b0 2024-09-21T04:31:04.116 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:04 smithi053 ceph-mon[35554]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T04:31:04.116 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:04 smithi053 ceph-mon[35554]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 551 MiB used, 446 GiB / 447 GiB avail 2024-09-21T04:31:04.116 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:04 smithi053 ceph-mon[35554]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:31:04.117 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:04 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:31:04.117 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:04 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:31:04.117 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:04 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:04 smithi007 ceph-mon[27954]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T04:31:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:04 smithi007 ceph-mon[27954]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 551 MiB used, 446 GiB / 447 GiB avail 2024-09-21T04:31:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:04 smithi007 ceph-mon[27954]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:31:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:04 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-21T04:31:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:04 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-21T04:31:04.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:04 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:06.340 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:06 smithi053 ceph-mon[35554]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 551 MiB used, 446 GiB / 447 GiB avail 2024-09-21T04:31:06.340 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:06 smithi053 ceph-mon[35554]: from='client.? 172.21.15.53:0/1439352517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af9dd7dd-6063-4e79-9571-ee72f9bea315"}]: dispatch 2024-09-21T04:31:06.340 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:06 smithi053 ceph-mon[35554]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af9dd7dd-6063-4e79-9571-ee72f9bea315"}]: dispatch 2024-09-21T04:31:06.340 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:06 smithi053 ceph-mon[35554]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af9dd7dd-6063-4e79-9571-ee72f9bea315"}]': finished 2024-09-21T04:31:06.340 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:06 smithi053 ceph-mon[35554]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T04:31:06.340 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:06 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:06 smithi007 ceph-mon[27954]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 551 MiB used, 446 GiB / 447 GiB avail 2024-09-21T04:31:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:06 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/1439352517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af9dd7dd-6063-4e79-9571-ee72f9bea315"}]: dispatch 2024-09-21T04:31:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:06 smithi007 ceph-mon[27954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af9dd7dd-6063-4e79-9571-ee72f9bea315"}]: dispatch 2024-09-21T04:31:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:06 smithi007 ceph-mon[27954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af9dd7dd-6063-4e79-9571-ee72f9bea315"}]': finished 2024-09-21T04:31:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:06 smithi007 ceph-mon[27954]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T04:31:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:06 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:07 smithi053 ceph-mon[35554]: from='client.? 172.21.15.53:0/3464408536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:31:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:07 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3464408536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-21T04:31:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:08 smithi053 ceph-mon[35554]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:08 smithi007 ceph-mon[27954]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:10 smithi053 ceph-mon[35554]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:10 smithi007 ceph-mon[27954]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:12.298 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:12 smithi053 ceph-mon[35554]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:12.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:12 smithi007 ceph-mon[27954]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:14.174 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:14 smithi053 ceph-mon[35554]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:14 smithi007 ceph-mon[27954]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:15 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-21T04:31:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:15 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:15 smithi053 ceph-mon[35554]: Deploying daemon osd.5 on smithi053 2024-09-21T04:31:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:15 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-21T04:31:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:15 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:15 smithi007 ceph-mon[27954]: Deploying daemon osd.5 on smithi053 2024-09-21T04:31:16.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:16 smithi053 ceph-mon[35554]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:16.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:31:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:16 smithi007 ceph-mon[27954]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:31:18.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:18 smithi053 ceph-mon[35554]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:18 smithi007 ceph-mon[27954]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:20.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:20 smithi053 ceph-mon[35554]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:20.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:20 smithi007 ceph-mon[27954]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:21.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:21 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:31:21.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:21 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:21 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:21 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:31:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:21 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:21 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:22.345 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:22 smithi007 ceph-mon[27954]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:22 smithi053 ceph-mon[35554]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:24.120 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.116+0000 7f3477fff640 1 -- 172.21.15.53:0/89692925 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f349002c640 con 0x7f347806e2b0 2024-09-21T04:31:24.120 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 5 on host 'smithi053' 2024-09-21T04:31:24.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.118+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f347806e2b0 msgr2=0x7f3478070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:24.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.118+0000 7f3495141640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f347806e2b0 0x7f3478070770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34840096c0 tx=0x7f3484009650 comp rx=0 tx=0).stop 2024-09-21T04:31:24.122 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.118+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101960 msgr2=0x7f34901974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.118+0000 7f3495141640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101960 0x7f34901974b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f348000c8f0 tx=0x7f348000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:31:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.118+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 shutdown_connections 2024-09-21T04:31:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.119+0000 7f3495141640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f347806e2b0 0x7f3478070770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.119+0000 7f3495141640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3490102b60 0x7f34901979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.119+0000 7f3495141640 1 --2- 172.21.15.53:0/89692925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3490101960 0x7f34901974b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.119+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 >> 172.21.15.53:0/89692925 conn(0x7f34900fd0f0 msgr2=0x7f34901060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.119+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 shutdown_connections 2024-09-21T04:31:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:24.119+0000 7f3495141640 1 -- 172.21.15.53:0/89692925 wait complete. 2024-09-21T04:31:24.420 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:24 smithi053 ceph-mon[35554]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:24.421 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:24.421 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:24 smithi007 ceph-mon[27954]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:24.706 DEBUG:teuthology.orchestra.run.smithi053:osd.5> sudo journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T04:31:24.708 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-09-21T04:31:24.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T04:31:24.968 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.451+0000 7f99cb18e640 1 -- 172.21.15.7:0/1178408609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4101960 msgr2=0x7f99c4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.451+0000 7f99cb18e640 1 --2- 172.21.15.7:0/1178408609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4101960 0x7f99c4101d60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009880 tx=0x7f99b402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.451+0000 7f99cb18e640 1 -- 172.21.15.7:0/1178408609 shutdown_connections 2024-09-21T04:31:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.451+0000 7f99cb18e640 1 --2- 172.21.15.7:0/1178408609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c4102b60 0x7f99c4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.451+0000 7f99cb18e640 1 --2- 172.21.15.7:0/1178408609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4101960 0x7f99c4101d60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.451+0000 7f99cb18e640 1 -- 172.21.15.7:0/1178408609 >> 172.21.15.7:0/1178408609 conn(0x7f99c40fd0f0 msgr2=0x7f99c40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.451+0000 7f99cb18e640 1 -- 172.21.15.7:0/1178408609 shutdown_connections 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.451+0000 7f99cb18e640 1 -- 172.21.15.7:0/1178408609 wait complete. 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.452+0000 7f99cb18e640 1 Processor -- start 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.452+0000 7f99cb18e640 1 -- start start 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.452+0000 7f99cb18e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c4101960 0x7f99c41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.452+0000 7f99cb18e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4102b60 0x7f99c4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.452+0000 7f99cb18e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99c410a7a0 con 0x7f99c4102b60 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.452+0000 7f99cb18e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f99c410a620 con 0x7f99c4101960 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4102b60 0x7f99c4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4102b60 0x7f99c4197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56458/0 (socket says 172.21.15.7:56458) 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c3fff640 1 -- 172.21.15.7:0/889122583 learned_addr learned my addr 172.21.15.7:0/889122583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c8f2a640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c4101960 0x7f99c41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c3fff640 1 -- 172.21.15.7:0/889122583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c4101960 msgr2=0x7f99c41974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c3fff640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c4101960 0x7f99c41974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c3fff640 1 -- 172.21.15.7:0/889122583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c4198090 con 0x7f99c4102b60 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c3fff640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4102b60 0x7f99c4197a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c8f0 tx=0x7f99b800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c1ffb640 1 -- 172.21.15.7:0/889122583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f99b80079b0 con 0x7f99c4102b60 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c1ffb640 1 -- 172.21.15.7:0/889122583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f99b80054e0 con 0x7f99c4102b60 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c419cad0 con 0x7f99c4102b60 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.453+0000 7f99c1ffb640 1 -- 172.21.15.7:0/889122583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f99b8004dd0 con 0x7f99c4102b60 2024-09-21T04:31:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.454+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c419d060 con 0x7f99c4102b60 2024-09-21T04:31:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.455+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c4101d60 con 0x7f99c4102b60 2024-09-21T04:31:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.455+0000 7f99c1ffb640 1 -- 172.21.15.7:0/889122583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f99b800f450 con 0x7f99c4102b60 2024-09-21T04:31:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.455+0000 7f99c1ffb640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f99ac06e2b0 0x7f99ac070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.455+0000 7f99c1ffb640 1 -- 172.21.15.7:0/889122583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4321+0+0 (secure 0 0 0) 0x7f99b808dc30 con 0x7f99c4102b60 2024-09-21T04:31:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.455+0000 7f99c8f2a640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f99ac06e2b0 0x7f99ac070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.456+0000 7f99c8f2a640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f99ac06e2b0 0x7f99ac070770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99b4000c00 tx=0x7f99b4005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.460+0000 7f99c1ffb640 1 -- 172.21.15.7:0/889122583 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f99b8014030 con 0x7f99c4102b60 2024-09-21T04:31:25.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.634+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f99c4062df0 con 0x7f99c4102b60 2024-09-21T04:31:25.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.635+0000 7f99c1ffb640 1 -- 172.21.15.7:0/889122583 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v34) ==== 74+0+130 (secure 0 0 0) 0x7f99b805ba80 con 0x7f99c4102b60 2024-09-21T04:31:25.639 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f99ac06e2b0 msgr2=0x7f99ac070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f99ac06e2b0 0x7f99ac070770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99b4000c00 tx=0x7f99b4005a20 comp rx=0 tx=0).stop 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4102b60 msgr2=0x7f99c4197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4102b60 0x7f99c4197a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c8f0 tx=0x7f99b800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 shutdown_connections 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f99ac06e2b0 0x7f99ac070770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99c4102b60 0x7f99c4197a10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 --2- 172.21.15.7:0/889122583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99c4101960 0x7f99c41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 >> 172.21.15.7:0/889122583 conn(0x7f99c40fd0f0 msgr2=0x7f99c4106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 shutdown_connections 2024-09-21T04:31:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:25.637+0000 7f99cb18e640 1 -- 172.21.15.7:0/889122583 wait complete. 2024-09-21T04:31:26.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:25 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:25 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:25 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/889122583' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:31:26.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:25 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:26.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:25 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:26.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:25 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/889122583' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:31:26.491 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"num_osds":6,"num_up_osds":5,"osd_up_since":1726893061,"num_in_osds":6,"osd_in_since":1726893065,"num_remapped_pgs":0} 2024-09-21T04:31:26.692 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:31:26 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[54429]: 2024-09-21T04:31:26.433+0000 7efede411640 -1 osd.5 0 log_to_monitors true 2024-09-21T04:31:27.031 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:26 smithi053 ceph-mon[35554]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:27.031 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:26 smithi053 ceph-mon[35554]: from='osd.5 [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-21T04:31:27.031 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:26 smithi053 ceph-mon[35554]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-21T04:31:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:26 smithi007 ceph-mon[27954]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:26 smithi007 ceph-mon[27954]: from='osd.5 [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-21T04:31:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:26 smithi007 ceph-mon[27954]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-21T04:31:27.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T04:31:27.768 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:27 smithi007 ceph-mon[27954]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T04:31:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:27 smithi007 ceph-mon[27954]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T04:31:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:27 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:27 smithi007 ceph-mon[27954]: from='osd.5 [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:31:28.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:27 smithi007 ceph-mon[27954]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:31:28.285 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:27 smithi053 ceph-mon[35554]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T04:31:28.285 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:27 smithi053 ceph-mon[35554]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T04:31:28.285 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:27 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:28.285 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:27 smithi053 ceph-mon[35554]: from='osd.5 [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:31:28.285 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:27 smithi053 ceph-mon[35554]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-21T04:31:28.285 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:31:27 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[54429]: 2024-09-21T04:31:27.944+0000 7efed9b71640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T04:31:28.285 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:31:27 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[54429]: 2024-09-21T04:31:27.951+0000 7efed61ab640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:31:28.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.299+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/66171855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58100af0 msgr2=0x7fbf58100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:28.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.299+0000 7fbf5fa6e640 1 --2- 172.21.15.7:0/66171855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58100af0 0x7fbf58100f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009b60 tx=0x7fbf4c02a770 comp rx=0 tx=0).stop 2024-09-21T04:31:28.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.299+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/66171855 shutdown_connections 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.299+0000 7fbf5fa6e640 1 --2- 172.21.15.7:0/66171855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58100af0 0x7fbf58100f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.299+0000 7fbf5fa6e640 1 --2- 172.21.15.7:0/66171855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf58101e40 0x7fbf58102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.299+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/66171855 >> 172.21.15.7:0/66171855 conn(0x7fbf580fb770 msgr2=0x7fbf580fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/66171855 shutdown_connections 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/66171855 wait complete. 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5fa6e640 1 Processor -- start 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5fa6e640 1 -- start start 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5fa6e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf58100af0 0x7fbf58100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5fa6e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58101e40 0x7fbf5819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5fa6e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581096a0 con 0x7fbf58101e40 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5fa6e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58109520 con 0x7fbf58100af0 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5d009640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58101e40 0x7fbf5819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5d009640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58101e40 0x7fbf5819c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56480/0 (socket says 172.21.15.7:56480) 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5d009640 1 -- 172.21.15.7:0/561491380 learned_addr learned my addr 172.21.15.7:0/561491380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.300+0000 7fbf5d80a640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf58100af0 0x7fbf58100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf5d009640 1 -- 172.21.15.7:0/561491380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf58100af0 msgr2=0x7fbf58100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf5d009640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf58100af0 0x7fbf58100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf5d009640 1 -- 172.21.15.7:0/561491380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5819c7b0 con 0x7fbf58101e40 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf5d009640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58101e40 0x7fbf5819c1c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c030750 tx=0x7fbf4c030b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf56ffd640 1 -- 172.21.15.7:0/561491380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbf4c040020 con 0x7fbf58101e40 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf56ffd640 1 -- 172.21.15.7:0/561491380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbf4c003970 con 0x7fbf58101e40 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5819ca40 con 0x7fbf58101e40 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf56ffd640 1 -- 172.21.15.7:0/561491380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbf4c036040 con 0x7fbf58101e40 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.301+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5819cff0 con 0x7fbf58101e40 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.302+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf58100f70 con 0x7fbf58101e40 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.303+0000 7fbf56ffd640 1 -- 172.21.15.7:0/561491380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fbf4c043020 con 0x7fbf58101e40 2024-09-21T04:31:28.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.303+0000 7fbf56ffd640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbf4006e310 0x7fbf400707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.303+0000 7fbf56ffd640 1 -- 172.21.15.7:0/561491380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4358+0+0 (secure 0 0 0) 0x7fbf4c0ae920 con 0x7fbf58101e40 2024-09-21T04:31:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.303+0000 7fbf5d80a640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbf4006e310 0x7fbf400707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.303+0000 7fbf5d80a640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbf4006e310 0x7fbf400707d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbf58069f70 tx=0x7fbf48009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.308+0000 7fbf56ffd640 1 -- 172.21.15.7:0/561491380 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbf4c04a020 con 0x7fbf58101e40 2024-09-21T04:31:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.511+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbf58062df0 con 0x7fbf58101e40 2024-09-21T04:31:28.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.511+0000 7fbf56ffd640 1 -- 172.21.15.7:0/561491380 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v36) ==== 74+0+130 (secure 0 0 0) 0x7fbf4c07c6c0 con 0x7fbf58101e40 2024-09-21T04:31:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:31:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbf4006e310 msgr2=0x7fbf400707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:28.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbf4006e310 0x7fbf400707d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbf58069f70 tx=0x7fbf48009260 comp rx=0 tx=0).stop 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58101e40 msgr2=0x7fbf5819c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58101e40 0x7fbf5819c1c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c030750 tx=0x7fbf4c030b90 comp rx=0 tx=0).stop 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 shutdown_connections 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbf4006e310 0x7fbf400707d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf58101e40 0x7fbf5819c1c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 --2- 172.21.15.7:0/561491380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf58100af0 0x7fbf58100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 >> 172.21.15.7:0/561491380 conn(0x7fbf580fb770 msgr2=0x7fbf58104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 shutdown_connections 2024-09-21T04:31:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:28.514+0000 7fbf5fa6e640 1 -- 172.21.15.7:0/561491380 wait complete. 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:29.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:28 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/561491380' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:31:29.370 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"num_osds":6,"num_up_osds":5,"osd_up_since":1726893061,"num_in_osds":6,"osd_in_since":1726893065,"num_remapped_pgs":0} 2024-09-21T04:31:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:31:29.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:31:29.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:31:29.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:31:29.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:28 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/561491380' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:31:30.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T04:31:30.400 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:29 smithi007 ceph-mon[27954]: purged_snaps scrub starts 2024-09-21T04:31:30.400 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:29 smithi007 ceph-mon[27954]: purged_snaps scrub ok 2024-09-21T04:31:30.400 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:29 smithi007 ceph-mon[27954]: Detected new or changed devices on smithi053 2024-09-21T04:31:30.401 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:29 smithi007 ceph-mon[27954]: Adjusting osd_memory_target on smithi053 to 4972M 2024-09-21T04:31:30.401 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:29 smithi007 ceph-mon[27954]: osd.5 [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] boot 2024-09-21T04:31:30.401 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:29 smithi007 ceph-mon[27954]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T04:31:30.401 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:29 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:30.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:29 smithi053 ceph-mon[35554]: purged_snaps scrub starts 2024-09-21T04:31:30.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:29 smithi053 ceph-mon[35554]: purged_snaps scrub ok 2024-09-21T04:31:30.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:29 smithi053 ceph-mon[35554]: Detected new or changed devices on smithi053 2024-09-21T04:31:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:29 smithi053 ceph-mon[35554]: Adjusting osd_memory_target on smithi053 to 4972M 2024-09-21T04:31:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:29 smithi053 ceph-mon[35554]: osd.5 [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] boot 2024-09-21T04:31:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:29 smithi053 ceph-mon[35554]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T04:31:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:29 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:31:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:31.053 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:30 smithi007 ceph-mon[27954]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:31.054 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:30 smithi007 ceph-mon[27954]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T04:31:31.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 -- 172.21.15.7:0/3360960802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 msgr2=0x7f80cc103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 --2- 172.21.15.7:0/3360960802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 0x7f80cc103080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009880 tx=0x7f80b802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 -- 172.21.15.7:0/3360960802 shutdown_connections 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 --2- 172.21.15.7:0/3360960802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 0x7f80cc103080 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 --2- 172.21.15.7:0/3360960802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc0ffc20 0x7f80cc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 -- 172.21.15.7:0/3360960802 >> 172.21.15.7:0/3360960802 conn(0x7f80cc0fb770 msgr2=0x7f80cc0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 -- 172.21.15.7:0/3360960802 shutdown_connections 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 -- 172.21.15.7:0/3360960802 wait complete. 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 Processor -- start 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.243+0000 7f80d2bbe640 1 -- start start 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80d2bbe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc0ffc20 0x7f80cc1024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80d2bbe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 0x7f80cc100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80d2bbe640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc10a780 con 0x7f80cc102c00 2024-09-21T04:31:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80d2bbe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc10a600 con 0x7f80cc0ffc20 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 0x7f80cc100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 0x7f80cc100bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56506/0 (socket says 172.21.15.7:56506) 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80cbfff640 1 -- 172.21.15.7:0/1934641492 learned_addr learned my addr 172.21.15.7:0/1934641492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80d095a640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc0ffc20 0x7f80cc1024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80cbfff640 1 -- 172.21.15.7:0/1934641492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc0ffc20 msgr2=0x7f80cc1024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80cbfff640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc0ffc20 0x7f80cc1024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80cbfff640 1 -- 172.21.15.7:0/1934641492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1010f0 con 0x7f80cc102c00 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.244+0000 7f80cbfff640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 0x7f80cc100bb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80b8006010 tx=0x7f80b80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.245+0000 7f80c9ffb640 1 -- 172.21.15.7:0/1934641492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f80b803f070 con 0x7f80cc102c00 2024-09-21T04:31:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.245+0000 7f80c9ffb640 1 -- 172.21.15.7:0/1934641492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80b80044d0 con 0x7f80cc102c00 2024-09-21T04:31:31.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.245+0000 7f80c9ffb640 1 -- 172.21.15.7:0/1934641492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f80b8035030 con 0x7f80cc102c00 2024-09-21T04:31:31.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.245+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc101380 con 0x7f80cc102c00 2024-09-21T04:31:31.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.246+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1018b0 con 0x7f80cc102c00 2024-09-21T04:31:31.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.247+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc100020 con 0x7f80cc102c00 2024-09-21T04:31:31.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.247+0000 7f80c9ffb640 1 -- 172.21.15.7:0/1934641492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f80b8042070 con 0x7f80cc102c00 2024-09-21T04:31:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.248+0000 7f80c9ffb640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f80bc06e310 0x7f80bc0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.248+0000 7f80c9ffb640 1 -- 172.21.15.7:0/1934641492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f80b80ae6e0 con 0x7f80cc102c00 2024-09-21T04:31:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.248+0000 7f80d095a640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f80bc06e310 0x7f80bc0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.248+0000 7f80d095a640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f80bc06e310 0x7f80bc0707d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80cc069fb0 tx=0x7f80c0005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.252+0000 7f80c9ffb640 1 -- 172.21.15.7:0/1934641492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f80b80800b0 con 0x7f80cc102c00 2024-09-21T04:31:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.431+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f80cc19f990 con 0x7f80cc102c00 2024-09-21T04:31:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.432+0000 7f80c9ffb640 1 -- 172.21.15.7:0/1934641492 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) ==== 74+0+130 (secure 0 0 0) 0x7f80b80848e0 con 0x7f80cc102c00 2024-09-21T04:31:31.435 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:31:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f80bc06e310 msgr2=0x7f80bc0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f80bc06e310 0x7f80bc0707d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80cc069fb0 tx=0x7f80c0005f90 comp rx=0 tx=0).stop 2024-09-21T04:31:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 msgr2=0x7f80cc100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 0x7f80cc100bb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80b8006010 tx=0x7f80b80048f0 comp rx=0 tx=0).stop 2024-09-21T04:31:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 shutdown_connections 2024-09-21T04:31:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f80bc06e310 0x7f80bc0707d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc102c00 0x7f80cc100bb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 --2- 172.21.15.7:0/1934641492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc0ffc20 0x7f80cc1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.434+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 >> 172.21.15.7:0/1934641492 conn(0x7f80cc0fb770 msgr2=0x7f80cc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.435+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 shutdown_connections 2024-09-21T04:31:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:31.435+0000 7f80d2bbe640 1 -- 172.21.15.7:0/1934641492 wait complete. 2024-09-21T04:31:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:30 smithi053 ceph-mon[35554]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T04:31:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:30 smithi053 ceph-mon[35554]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T04:31:32.101 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:31:32.101 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:31 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1934641492' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:31:32.132 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"num_osds":6,"num_up_osds":6,"osd_up_since":1726893088,"num_in_osds":6,"osd_in_since":1726893065,"num_remapped_pgs":0} 2024-09-21T04:31:32.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:31:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:32.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:31 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:31:32.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:31 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1934641492' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-21T04:31:32.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.858+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4053546602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 msgr2=0x7f54a8102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:32.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.858+0000 7f54ad9fe640 1 --2- 172.21.15.7:0/4053546602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 0x7f54a8102f50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5498009c20 tx=0x7f549802aa00 comp rx=0 tx=0).stop 2024-09-21T04:31:32.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.859+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4053546602 shutdown_connections 2024-09-21T04:31:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.859+0000 7f54ad9fe640 1 --2- 172.21.15.7:0/4053546602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a8103490 0x7f54a810b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.859+0000 7f54ad9fe640 1 --2- 172.21.15.7:0/4053546602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 0x7f54a8102f50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.859+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4053546602 >> 172.21.15.7:0/4053546602 conn(0x7f54a80fc550 msgr2=0x7f54a80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.859+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4053546602 shutdown_connections 2024-09-21T04:31:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.859+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4053546602 wait complete. 2024-09-21T04:31:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54ad9fe640 1 Processor -- start 2024-09-21T04:31:32.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54ad9fe640 1 -- start start 2024-09-21T04:31:32.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54ad9fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 0x7f54a819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54ad9fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a8103490 0x7f54a819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54ad9fe640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54a810ede0 con 0x7f54a8102b70 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54ad9fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f54a810ec60 con 0x7f54a8103490 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 0x7f54a819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 0x7f54a819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56530/0 (socket says 172.21.15.7:56530) 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54a6ffd640 1 -- 172.21.15.7:0/4182342233 learned_addr learned my addr 172.21.15.7:0/4182342233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.860+0000 7f54a67fc640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a8103490 0x7f54a819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54a6ffd640 1 -- 172.21.15.7:0/4182342233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a8103490 msgr2=0x7f54a819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54a6ffd640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a8103490 0x7f54a819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54a6ffd640 1 -- 172.21.15.7:0/4182342233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a81999e0 con 0x7f54a8102b70 2024-09-21T04:31:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54a6ffd640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 0x7f54a819f820 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5498006010 tx=0x7f549802ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:32.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54ac9fc640 1 -- 172.21.15.7:0/4182342233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5498047070 con 0x7f54a8102b70 2024-09-21T04:31:32.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a8199c70 con 0x7f54a8102b70 2024-09-21T04:31:32.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54ac9fc640 1 -- 172.21.15.7:0/4182342233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5498003e30 con 0x7f54a8102b70 2024-09-21T04:31:32.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54ac9fc640 1 -- 172.21.15.7:0/4182342233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5498033050 con 0x7f54a8102b70 2024-09-21T04:31:32.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.861+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a819a1a0 con 0x7f54a8102b70 2024-09-21T04:31:32.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.862+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a806a4c0 con 0x7f54a8102b70 2024-09-21T04:31:32.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.863+0000 7f54ac9fc640 1 -- 172.21.15.7:0/4182342233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f549804a070 con 0x7f54a8102b70 2024-09-21T04:31:32.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.863+0000 7f54ac9fc640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f549406e2b0 0x7f5494070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:32.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.864+0000 7f54ac9fc640 1 -- 172.21.15.7:0/4182342233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f549803e030 con 0x7f54a8102b70 2024-09-21T04:31:32.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.864+0000 7f54a67fc640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f549406e2b0 0x7f5494070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:32.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.864+0000 7f54a67fc640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f549406e2b0 0x7f5494070770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f54a80fef10 tx=0x7f549000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:32.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:32.868+0000 7f54ac9fc640 1 -- 172.21.15.7:0/4182342233 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f54980807e0 con 0x7f54a8102b70 2024-09-21T04:31:33.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.048+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54a810b3e0 con 0x7f54a8102b70 2024-09-21T04:31:33.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.049+0000 7f54ac9fc640 1 -- 172.21.15.7:0/4182342233 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+11321 (secure 0 0 0) 0x7f5498085010 con 0x7f54a8102b70 2024-09-21T04:31:33.051 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:31:33.051 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","created":"2024-09-21T04:25:32.557057+0000","modified":"2024-09-21T04:31:29.949365+0000","last_up_change":"2024-09-21T04:31:28.943860+0000","last_in_change":"2024-09-21T04:31:05.743550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T04:30:11.508639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7692119a-1db2-417f-a086-0181309481b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6803","nonce":2931271991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6805","nonce":2931271991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6809","nonce":2931271991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6807","nonce":2931271991}]},"public_addr":"172.21.15.7:6803/2931271991","cluster_addr":"172.21.15.7:6805/2931271991","heartbeat_back_addr":"172.21.15.7:6809/2931271991","heartbeat_front_addr":"172.21.15.7:6807/2931271991","state":["exists","up"]},{"osd":1,"uuid":"09975c57-317c-4d48-9926-8cfa7786b7b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6811","nonce":3959135592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6813","nonce":3959135592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6817","nonce":3959135592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6815","nonce":3959135592}]},"public_addr":"172.21.15.7:6811/3959135592","cluster_addr":"172.21.15.7:6813/3959135592","heartbeat_back_addr":"172.21.15.7:6817/3959135592","heartbeat_front_addr":"172.21.15.7:6815/3959135592","state":["exists","up"]},{"osd":2,"uuid":"63457a4d-975d-4ea1-b6ac-770bf49aead3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6819","nonce":2953002876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6821","nonce":2953002876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6825","nonce":2953002876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6823","nonce":2953002876}]},"public_addr":"172.21.15.7:6819/2953002876","cluster_addr":"172.21.15.7:6821/2953002876","heartbeat_back_addr":"172.21.15.7:6825/2953002876","heartbeat_front_addr":"172.21.15.7:6823/2953002876","state":["exists","up"]},{"osd":3,"uuid":"d758b507-1cd4-4c67-b7fb-917f65419e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6801","nonce":3034241373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6803","nonce":3034241373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6807","nonce":3034241373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6805","nonce":3034241373}]},"public_addr":"172.21.15.53:6801/3034241373","cluster_addr":"172.21.15.53:6803/3034241373","heartbeat_back_addr":"172.21.15.53:6807/3034241373","heartbeat_front_addr":"172.21.15.53:6805/3034241373","state":["exists","up"]},{"osd":4,"uuid":"8bea39b5-2b2e-4ba0-8224-c8642be0d3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6809","nonce":1120118794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6811","nonce":1120118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6815","nonce":1120118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6813","nonce":1120118794}]},"public_addr":"172.21.15.53:6809/1120118794","cluster_addr":"172.21.15.53:6811/1120118794","heartbeat_back_addr":"172.21.15.53:6815/1120118794","heartbeat_front_addr":"172.21.15.53:6813/1120118794","state":["exists","up"]},{"osd":5,"uuid":"af9dd7dd-6063-4e79-9571-ee72f9bea315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6817","nonce":4215104838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6819","nonce":4215104838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6823","nonce":4215104838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6821","nonce":4215104838}]},"public_addr":"172.21.15.53:6817/4215104838","cluster_addr":"172.21.15.53:6819/4215104838","heartbeat_back_addr":"172.21.15.53:6823/4215104838","heartbeat_front_addr":"172.21.15.53:6821/4215104838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:29:07.047481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:29:38.183480+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:30:07.825963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:30:33.713760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:31:00.254040+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:31:27.450032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6800/2645587315":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/2202116322":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/2211484432":"2024-09-22T04:25:58.338381+0000","172.21.15.7:6801/2645587315":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/1594571982":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/3304037039":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/554975665":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6801/3995670699":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/2851424446":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6800/3808261568":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6801/3808261568":"2024-09-22T04:26:33.198427+0000","172.21.15.7:0/4161500397":"2024-09-22T04:26:33.198427+0000","172.21.15.7:0/3340019345":"2024-09-22T04:27:31.004325+0000","172.21.15.7:6800/3995670699":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/3901749065":"2024-09-22T04:27:31.004325+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f549406e2b0 msgr2=0x7f5494070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f549406e2b0 0x7f5494070770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f54a80fef10 tx=0x7f549000b390 comp rx=0 tx=0).stop 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 msgr2=0x7f54a819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 0x7f54a819f820 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5498006010 tx=0x7f549802ed10 comp rx=0 tx=0).stop 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 shutdown_connections 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f549406e2b0 0x7f5494070770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54a8103490 0x7f54a819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 --2- 172.21.15.7:0/4182342233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a8102b70 0x7f54a819f820 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 >> 172.21.15.7:0/4182342233 conn(0x7f54a80fc550 msgr2=0x7f54a8100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 shutdown_connections 2024-09-21T04:31:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:33.051+0000 7f54ad9fe640 1 -- 172.21.15.7:0/4182342233 wait complete. 2024-09-21T04:31:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:32 smithi007 ceph-mon[27954]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:32 smithi053 ceph-mon[35554]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:33.654 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T04:30:11.508639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T04:31:33.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T04:31:33.911 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:34.286 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:33 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/4182342233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.368+0000 7f4be9b6c640 1 -- 172.21.15.7:0/2316225230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 msgr2=0x7f4be4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.368+0000 7f4be9b6c640 1 --2- 172.21.15.7:0/2316225230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 0x7f4be4107ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009880 tx=0x7f4bd402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.368+0000 7f4be9b6c640 1 -- 172.21.15.7:0/2316225230 shutdown_connections 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.368+0000 7f4be9b6c640 1 --2- 172.21.15.7:0/2316225230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4be4101900 0x7f4be4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.368+0000 7f4be9b6c640 1 --2- 172.21.15.7:0/2316225230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 0x7f4be4107ce0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.368+0000 7f4be9b6c640 1 -- 172.21.15.7:0/2316225230 >> 172.21.15.7:0/2316225230 conn(0x7f4be40fd600 msgr2=0x7f4be40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.368+0000 7f4be9b6c640 1 -- 172.21.15.7:0/2316225230 shutdown_connections 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.369+0000 7f4be9b6c640 1 -- 172.21.15.7:0/2316225230 wait complete. 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.369+0000 7f4be9b6c640 1 Processor -- start 2024-09-21T04:31:34.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.369+0000 7f4be9b6c640 1 -- start start 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.369+0000 7f4be9b6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4be4101900 0x7f4be419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.369+0000 7f4be9b6c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 0x7f4be419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.369+0000 7f4be9b6c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4be410ee30 con 0x7f4be4107900 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.369+0000 7f4be9b6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4be410ecb0 con 0x7f4be4101900 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.369+0000 7f4bdbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 0x7f4be419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4bdbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 0x7f4be419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47292/0 (socket says 172.21.15.7:47292) 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4bdbfff640 1 -- 172.21.15.7:0/1983413710 learned_addr learned my addr 172.21.15.7:0/1983413710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4be37fe640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4be4101900 0x7f4be419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4bdbfff640 1 -- 172.21.15.7:0/1983413710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4be4101900 msgr2=0x7f4be419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4bdbfff640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4be4101900 0x7f4be419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4bdbfff640 1 -- 172.21.15.7:0/1983413710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be4199a70 con 0x7f4be4107900 2024-09-21T04:31:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4bdbfff640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 0x7f4be419fdf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00b420 tx=0x7f4bcc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4be17fa640 1 -- 172.21.15.7:0/1983413710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4bcc011630 con 0x7f4be4107900 2024-09-21T04:31:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4be17fa640 1 -- 172.21.15.7:0/1983413710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4bcc016070 con 0x7f4be4107900 2024-09-21T04:31:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be4199d60 con 0x7f4be4107900 2024-09-21T04:31:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4be17fa640 1 -- 172.21.15.7:0/1983413710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4bcc010d40 con 0x7f4be4107900 2024-09-21T04:31:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.370+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be419a2f0 con 0x7f4be4107900 2024-09-21T04:31:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.371+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be4103040 con 0x7f4be4107900 2024-09-21T04:31:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.372+0000 7f4be17fa640 1 -- 172.21.15.7:0/1983413710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f4bcc01b460 con 0x7f4be4107900 2024-09-21T04:31:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.372+0000 7f4be17fa640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4bd006e310 0x7f4bd00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.372+0000 7f4be17fa640 1 -- 172.21.15.7:0/1983413710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f4bcc08d530 con 0x7f4be4107900 2024-09-21T04:31:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.372+0000 7f4be37fe640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4bd006e310 0x7f4bd00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:34.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.373+0000 7f4be37fe640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4bd006e310 0x7f4bd00707d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4000c00 tx=0x7f4bd4031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:34.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.377+0000 7f4be17fa640 1 -- 172.21.15.7:0/1983413710 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4bcc05b250 con 0x7f4be4107900 2024-09-21T04:31:34.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:33 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/4182342233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:31:34.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.552+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4be419b070 con 0x7f4be4107900 2024-09-21T04:31:34.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.552+0000 7f4be17fa640 1 -- 172.21.15.7:0/1983413710 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f4bcc08d350 con 0x7f4be4107900 2024-09-21T04:31:34.556 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-21T04:31:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.554+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4bd006e310 msgr2=0x7f4bd00707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.554+0000 7f4be9b6c640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4bd006e310 0x7f4bd00707d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4000c00 tx=0x7f4bd4031400 comp rx=0 tx=0).stop 2024-09-21T04:31:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.554+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 msgr2=0x7f4be419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.555+0000 7f4be9b6c640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 0x7f4be419fdf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00b420 tx=0x7f4bcc00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:31:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.555+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 shutdown_connections 2024-09-21T04:31:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.555+0000 7f4be9b6c640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4bd006e310 0x7f4bd00707d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.555+0000 7f4be9b6c640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4be4107900 0x7f4be419fdf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.555+0000 7f4be9b6c640 1 --2- 172.21.15.7:0/1983413710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4be4101900 0x7f4be419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.555+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 >> 172.21.15.7:0/1983413710 conn(0x7f4be40fd600 msgr2=0x7f4be4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.555+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 shutdown_connections 2024-09-21T04:31:34.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:34.555+0000 7f4be9b6c640 1 -- 172.21.15.7:0/1983413710 wait complete. 2024-09-21T04:31:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:34 smithi007 ceph-mon[27954]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:34 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1983413710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-21T04:31:35.276 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T04:31:35.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T04:31:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:34 smithi053 ceph-mon[35554]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:34 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1983413710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-21T04:31:35.532 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.073+0000 7fc957089640 1 -- 172.21.15.7:0/42530092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc950104d20 msgr2=0x7fc950107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.073+0000 7fc957089640 1 --2- 172.21.15.7:0/42530092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc950104d20 0x7fc950107110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0098b0 tx=0x7fc93c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.073+0000 7fc957089640 1 -- 172.21.15.7:0/42530092 shutdown_connections 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.073+0000 7fc957089640 1 --2- 172.21.15.7:0/42530092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc950104d20 0x7fc950107110 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.073+0000 7fc957089640 1 --2- 172.21.15.7:0/42530092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9501023f0 0x7fc9501047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.073+0000 7fc957089640 1 -- 172.21.15.7:0/42530092 >> 172.21.15.7:0/42530092 conn(0x7fc9500fc010 msgr2=0x7fc9500fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.073+0000 7fc957089640 1 -- 172.21.15.7:0/42530092 shutdown_connections 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.073+0000 7fc957089640 1 -- 172.21.15.7:0/42530092 wait complete. 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc957089640 1 Processor -- start 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc957089640 1 -- start start 2024-09-21T04:31:36.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc957089640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9501023f0 0x7fc9501996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc957089640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950104d20 0x7fc950199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc957089640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc95010a790 con 0x7fc9501023f0 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc957089640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc95010a610 con 0x7fc950104d20 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc954e25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9501023f0 0x7fc9501996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc954e25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9501023f0 0x7fc9501996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47318/0 (socket says 172.21.15.7:47318) 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc954e25640 1 -- 172.21.15.7:0/146450893 learned_addr learned my addr 172.21.15.7:0/146450893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc94ffff640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950104d20 0x7fc950199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc954e25640 1 -- 172.21.15.7:0/146450893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950104d20 msgr2=0x7fc950199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc954e25640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950104d20 0x7fc950199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc954e25640 1 -- 172.21.15.7:0/146450893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95019a2a0 con 0x7fc9501023f0 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.074+0000 7fc954e25640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9501023f0 0x7fc9501996e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c810 tx=0x7fc94400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.075+0000 7fc94dffb640 1 -- 172.21.15.7:0/146450893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc944004020 con 0x7fc9501023f0 2024-09-21T04:31:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.075+0000 7fc94dffb640 1 -- 172.21.15.7:0/146450893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc9440041c0 con 0x7fc9501023f0 2024-09-21T04:31:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.075+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95019ece0 con 0x7fc9501023f0 2024-09-21T04:31:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.075+0000 7fc94dffb640 1 -- 172.21.15.7:0/146450893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc9440047a0 con 0x7fc9501023f0 2024-09-21T04:31:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.075+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95019f190 con 0x7fc9501023f0 2024-09-21T04:31:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.076+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9501027d0 con 0x7fc9501023f0 2024-09-21T04:31:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.076+0000 7fc94dffb640 1 -- 172.21.15.7:0/146450893 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fc9440079b0 con 0x7fc9501023f0 2024-09-21T04:31:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.077+0000 7fc94dffb640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc94006e2b0 0x7fc940070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.077+0000 7fc94dffb640 1 -- 172.21.15.7:0/146450893 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fc94408dcc0 con 0x7fc9501023f0 2024-09-21T04:31:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.077+0000 7fc94ffff640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc94006e2b0 0x7fc940070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:36.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.077+0000 7fc94ffff640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc94006e2b0 0x7fc940070770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009880 tx=0x7fc93c0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:36.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.081+0000 7fc94dffb640 1 -- 172.21.15.7:0/146450893 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc944014030 con 0x7fc9501023f0 2024-09-21T04:31:36.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.322+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc9500fdb00 con 0x7fc9501023f0 2024-09-21T04:31:36.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.324+0000 7fc94dffb640 1 -- 172.21.15.7:0/146450893 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) ==== 170+0+59 (secure 0 0 0) 0x7fc94405f610 con 0x7fc9501023f0 2024-09-21T04:31:36.327 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-21T04:31:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAoTO5mg+lQExAA0KY+z+sb5kioU4bmFBgIcA== 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc94006e2b0 msgr2=0x7fc940070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc94006e2b0 0x7fc940070770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009880 tx=0x7fc93c0058d0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9501023f0 msgr2=0x7fc9501996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9501023f0 0x7fc9501996e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c810 tx=0x7fc94400ccd0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 shutdown_connections 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc94006e2b0 0x7fc940070770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc950104d20 0x7fc950199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 --2- 172.21.15.7:0/146450893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9501023f0 0x7fc9501996e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 >> 172.21.15.7:0/146450893 conn(0x7fc9500fc010 msgr2=0x7fc950101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 shutdown_connections 2024-09-21T04:31:36.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:36.327+0000 7fc957089640 1 -- 172.21.15.7:0/146450893 wait complete. 2024-09-21T04:31:36.960 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:31:36.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T04:31:36.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T04:31:37.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T04:31:37.274 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:31:37.396 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:36 smithi053 ceph-mon[35554]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:37.396 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:36 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/146450893' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:31:37.396 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:36 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/146450893' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:31:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:36 smithi007 ceph-mon[27954]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:36 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/146450893' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:31:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:36 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/146450893' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 -- 172.21.15.53:0/4109353277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4101a30 msgr2=0x7f52b4101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 --2- 172.21.15.53:0/4109353277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4101a30 0x7f52b4101eb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009880 tx=0x7f52a802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 -- 172.21.15.53:0/4109353277 shutdown_connections 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 --2- 172.21.15.53:0/4109353277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4101a30 0x7f52b4101eb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 --2- 172.21.15.53:0/4109353277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b4100830 0x7f52b4100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 -- 172.21.15.53:0/4109353277 >> 172.21.15.53:0/4109353277 conn(0x7f52b40fbfa0 msgr2=0x7f52b40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 -- 172.21.15.53:0/4109353277 shutdown_connections 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 -- 172.21.15.53:0/4109353277 wait complete. 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 Processor -- start 2024-09-21T04:31:37.756 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.753+0000 7f52ba9c8640 1 -- start start 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52ba9c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4100830 0x7f52b4104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52ba9c8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b4101a30 0x7f52b4103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52ba9c8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4109650 con 0x7f52b4101a30 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52ba9c8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41094d0 con 0x7f52b4100830 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4100830 0x7f52b4104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4100830 0x7f52b4104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47508/0 (socket says 172.21.15.53:47508) 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b3fff640 1 -- 172.21.15.53:0/3466005672 learned_addr learned my addr 172.21.15.53:0/3466005672 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b37fe640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b4101a30 0x7f52b4103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b3fff640 1 -- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b4101a30 msgr2=0x7f52b4103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b3fff640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b4101a30 0x7f52b4103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b3fff640 1 -- 172.21.15.53:0/3466005672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b41036a0 con 0x7f52b4100830 2024-09-21T04:31:37.758 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b3fff640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4100830 0x7f52b4104aa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c8f0 tx=0x7f52a400cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:37.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b17fa640 1 -- 172.21.15.53:0/3466005672 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52a4004020 con 0x7f52b4100830 2024-09-21T04:31:37.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b4103990 con 0x7f52b4100830 2024-09-21T04:31:37.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b17fa640 1 -- 172.21.15.53:0/3466005672 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52a4007e10 con 0x7f52b4100830 2024-09-21T04:31:37.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.754+0000 7f52b17fa640 1 -- 172.21.15.53:0/3466005672 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52a4003e10 con 0x7f52b4100830 2024-09-21T04:31:37.759 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.755+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b4103ef0 con 0x7f52b4100830 2024-09-21T04:31:37.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.756+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b4100c30 con 0x7f52b4100830 2024-09-21T04:31:37.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.756+0000 7f52b17fa640 1 -- 172.21.15.53:0/3466005672 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f52a40041c0 con 0x7f52b4100830 2024-09-21T04:31:37.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.756+0000 7f52b17fa640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f529c06e2b0 0x7f529c070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:37.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.757+0000 7f52b17fa640 1 -- 172.21.15.53:0/3466005672 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f52a408ded0 con 0x7f52b4100830 2024-09-21T04:31:37.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.757+0000 7f52b37fe640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f529c06e2b0 0x7f529c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:37.760 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.757+0000 7f52b37fe640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f529c06e2b0 0x7f529c070770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52a8000c00 tx=0x7f52a80058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:37.763 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:37.761+0000 7f52b17fa640 1 -- 172.21.15.53:0/3466005672 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f52a405bb70 con 0x7f52b4100830 2024-09-21T04:31:38.005 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.003+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f52b402db00 con 0x7f52b4100830 2024-09-21T04:31:38.011 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.007+0000 7f52b17fa640 1 -- 172.21.15.53:0/3466005672 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (secure 0 0 0) 0x7f52a405f820 con 0x7f52b4100830 2024-09-21T04:31:38.011 INFO:teuthology.orchestra.run.smithi053.stdout:[client.1] 2024-09-21T04:31:38.011 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAqTO5mYtJKABAA7L1jehfLhqwa+y1InU8ZLQ== 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f529c06e2b0 msgr2=0x7f529c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f529c06e2b0 0x7f529c070770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52a8000c00 tx=0x7f52a80058d0 comp rx=0 tx=0).stop 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4100830 msgr2=0x7f52b4104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4100830 0x7f52b4104aa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c8f0 tx=0x7f52a400cdb0 comp rx=0 tx=0).stop 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 shutdown_connections 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f529c06e2b0 0x7f529c070770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52b4101a30 0x7f52b4103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 --2- 172.21.15.53:0/3466005672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52b4100830 0x7f52b4104aa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 >> 172.21.15.53:0/3466005672 conn(0x7f52b40fbfa0 msgr2=0x7f52b40fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 shutdown_connections 2024-09-21T04:31:38.012 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:31:38.009+0000 7f52ba9c8640 1 -- 172.21.15.53:0/3466005672 wait complete. 2024-09-21T04:31:38.641 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:31:38.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-21T04:31:38.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T04:31:38.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T04:31:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:39.234 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:38 smithi007 ceph-mon[27954]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:39.234 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:38 smithi007 ceph-mon[27954]: from='client.? 172.21.15.53:0/3466005672' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:31:39.234 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:38 smithi007 ceph-mon[27954]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:31:39.235 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:38 smithi007 ceph-mon[27954]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:31:39.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 -- 172.21.15.7:0/2558324921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 msgr2=0x7f098010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:39.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 --2- 172.21.15.7:0/2558324921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 0x7f098010b8c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0970009880 tx=0x7f097002a530 comp rx=0 tx=0).stop 2024-09-21T04:31:39.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 -- 172.21.15.7:0/2558324921 shutdown_connections 2024-09-21T04:31:39.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 --2- 172.21.15.7:0/2558324921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 0x7f098010b8c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:39.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 --2- 172.21.15.7:0/2558324921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102ab0 0x7f0980102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:39.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 -- 172.21.15.7:0/2558324921 >> 172.21.15.7:0/2558324921 conn(0x7f09800fc550 msgr2=0x7f09800fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 -- 172.21.15.7:0/2558324921 shutdown_connections 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 -- 172.21.15.7:0/2558324921 wait complete. 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 Processor -- start 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.415+0000 7f09875c0640 1 -- start start 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09875c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102ab0 0x7f0980100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09875c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 0x7f0980100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09875c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f098010ee40 con 0x7f09801033d0 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09875c0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f098010ecc0 con 0x7f0980102ab0 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f0984b5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 0x7f0980100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f0984b5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 0x7f0980100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47338/0 (socket says 172.21.15.7:47338) 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f0984b5b640 1 -- 172.21.15.7:0/2409697369 learned_addr learned my addr 172.21.15.7:0/2409697369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f098535c640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102ab0 0x7f0980100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:39.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f0984b5b640 1 -- 172.21.15.7:0/2409697369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102ab0 msgr2=0x7f0980100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f0984b5b640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102ab0 0x7f0980100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f0984b5b640 1 -- 172.21.15.7:0/2409697369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0980101190 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f0984b5b640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 0x7f0980100c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0970031460 tx=0x7f09700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09767fc640 1 -- 172.21.15.7:0/2409697369 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f097003f070 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0980101420 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09767fc640 1 -- 172.21.15.7:0/2409697369 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0970004ac0 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09767fc640 1 -- 172.21.15.7:0/2409697369 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0970035030 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.416+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09801a3df0 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.417+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098006a580 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.418+0000 7f09767fc640 1 -- 172.21.15.7:0/2409697369 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f0970042070 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.418+0000 7f09767fc640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f096c06e2b0 0x7f096c070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.418+0000 7f09767fc640 1 -- 172.21.15.7:0/2409697369 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f09700ae860 con 0x7f09801033d0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.418+0000 7f098535c640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f096c06e2b0 0x7f096c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:39.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.419+0000 7f098535c640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f096c06e2b0 0x7f096c070770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f097c009890 tx=0x7f097c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:39.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.423+0000 7f09767fc640 1 -- 172.21.15.7:0/2409697369 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f097007c500 con 0x7f09801033d0 2024-09-21T04:31:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:38 smithi053 ceph-mon[35554]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:38 smithi053 ceph-mon[35554]: from='client.? 172.21.15.53:0/3466005672' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:31:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:38 smithi053 ceph-mon[35554]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-21T04:31:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:38 smithi053 ceph-mon[35554]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:31:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.598+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f0980101c40 con 0x7f09801033d0 2024-09-21T04:31:39.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.599+0000 7f09767fc640 1 -- 172.21.15.7:0/2409697369 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v39) ==== 86+0+187 (secure 0 0 0) 0x7f09700801b0 con 0x7f09801033d0 2024-09-21T04:31:39.601 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:31:39.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f096c06e2b0 msgr2=0x7f096c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f096c06e2b0 0x7f096c070770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f097c009890 tx=0x7f097c008040 comp rx=0 tx=0).stop 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 msgr2=0x7f0980100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 0x7f0980100c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0970031460 tx=0x7f09700046c0 comp rx=0 tx=0).stop 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 shutdown_connections 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f096c06e2b0 0x7f096c070770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09801033d0 0x7f0980100c50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 --2- 172.21.15.7:0/2409697369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0980102ab0 0x7f0980100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 >> 172.21.15.7:0/2409697369 conn(0x7f09800fc550 msgr2=0x7f0980108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 shutdown_connections 2024-09-21T04:31:39.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:39.601+0000 7f09875c0640 1 -- 172.21.15.7:0/2409697369 wait complete. 2024-09-21T04:31:40.321 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:39 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2409697369' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-21T04:31:40.352 INFO:teuthology.orchestra.run.smithi007.stdout:[{"version":39,"timestamp":"2024-09-21T04:31:28.323545+0000","name":"","changes":[{"name":"osd/host:smithi053/osd_memory_target","previous_value":"7820460851","new_value":"5213640567"}]}] 2024-09-21T04:31:40.352 INFO:tasks.ceph_manager:config epoch is 39 2024-09-21T04:31:40.352 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T04:31:40.352 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T04:31:40.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T04:31:40.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:39 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2409697369' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-21T04:31:40.615 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:40.986 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:40 smithi007 ceph-mon[27954]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.062+0000 7f77953bf640 1 -- 172.21.15.7:0/1083943777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 msgr2=0x7f7790100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.062+0000 7f77953bf640 1 --2- 172.21.15.7:0/1083943777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 0x7f7790100c60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7784009880 tx=0x7f778402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.062+0000 7f77953bf640 1 -- 172.21.15.7:0/1083943777 shutdown_connections 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.062+0000 7f77953bf640 1 --2- 172.21.15.7:0/1083943777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 0x7f7790100c60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.062+0000 7f77953bf640 1 --2- 172.21.15.7:0/1083943777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106800 0x7f7790106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.062+0000 7f77953bf640 1 -- 172.21.15.7:0/1083943777 >> 172.21.15.7:0/1083943777 conn(0x7f77900fc520 msgr2=0x7f77900fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.062+0000 7f77953bf640 1 -- 172.21.15.7:0/1083943777 shutdown_connections 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.062+0000 7f77953bf640 1 -- 172.21.15.7:0/1083943777 wait complete. 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f77953bf640 1 Processor -- start 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f77953bf640 1 -- start start 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f77953bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 0x7f779019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f77953bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106800 0x7f779019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f77953bf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f779010dd30 con 0x7f7790100800 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f77953bf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f779010dbb0 con 0x7f7790106800 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f778effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 0x7f779019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f778effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 0x7f779019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47370/0 (socket says 172.21.15.7:47370) 2024-09-21T04:31:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f778effd640 1 -- 172.21.15.7:0/703847040 learned_addr learned my addr 172.21.15.7:0/703847040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f778e7fc640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106800 0x7f779019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f778effd640 1 -- 172.21.15.7:0/703847040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106800 msgr2=0x7f779019dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f778effd640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106800 0x7f779019dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.063+0000 7f778effd640 1 -- 172.21.15.7:0/703847040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7790197830 con 0x7f7790100800 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.064+0000 7f778effd640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 0x7f779019d6b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f778000b420 tx=0x7f778000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.064+0000 7f7777fff640 1 -- 172.21.15.7:0/703847040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7780011630 con 0x7f7790100800 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.064+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7790197b20 con 0x7f7790100800 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.064+0000 7f7777fff640 1 -- 172.21.15.7:0/703847040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7780016070 con 0x7f7790100800 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.064+0000 7f7777fff640 1 -- 172.21.15.7:0/703847040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7780010d40 con 0x7f7790100800 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.064+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7790198080 con 0x7f7790100800 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.065+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7790101f40 con 0x7f7790100800 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.065+0000 7f7777fff640 1 -- 172.21.15.7:0/703847040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f778001b460 con 0x7f7790100800 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.066+0000 7f7777fff640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f777806e2b0 0x7f7778070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.066+0000 7f778e7fc640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f777806e2b0 0x7f7778070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.066+0000 7f7777fff640 1 -- 172.21.15.7:0/703847040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f778008d680 con 0x7f7790100800 2024-09-21T04:31:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.066+0000 7f778e7fc640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f777806e2b0 0x7f7778070770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7790198b80 tx=0x7f7784031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.071+0000 7f7777fff640 1 -- 172.21.15.7:0/703847040 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f778005b320 con 0x7f7790100800 2024-09-21T04:31:41.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.293+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7790109da0 con 0x7f7790100800 2024-09-21T04:31:41.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.296+0000 7f7777fff640 1 -- 172.21.15.7:0/703847040 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) ==== 74+0+176557 (secure 0 0 0) 0x7f778005efd0 con 0x7f7790100800 2024-09-21T04:31:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:31:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f777806e2b0 msgr2=0x7f7778070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f777806e2b0 0x7f7778070770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7790198b80 tx=0x7f7784031400 comp rx=0 tx=0).stop 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 msgr2=0x7f779019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 0x7f779019d6b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f778000b420 tx=0x7f778000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 shutdown_connections 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f777806e2b0 0x7f7778070770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7790106800 0x7f779019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 --2- 172.21.15.7:0/703847040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7790100800 0x7f779019d6b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 >> 172.21.15.7:0/703847040 conn(0x7f77900fc520 msgr2=0x7f779006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 shutdown_connections 2024-09-21T04:31:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:41.299+0000 7f77953bf640 1 -- 172.21.15.7:0/703847040 wait complete. 2024-09-21T04:31:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:40 smithi053 ceph-mon[35554]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:41.926 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"flags":0,"active_gid":14217,"active_name":"smithi007.huohfl","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":4030185662},{"type":"v1","addr":"172.21.15.7:6801","nonce":4030185662}]},"active_addr":"172.21.15.7:6801/4030185662","active_change":"2024-09-21T04:27:31.004431+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14244,"name":"smithi053.lnadao","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/","prometheus":"http://172.21.15.7:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2677828643}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1384976015}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2178763757}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":4286237328}]}]} 2024-09-21T04:31:41.931 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T04:31:41.931 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T04:31:41.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:31:42.193 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:41 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/703847040' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-21T04:31:42.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:41 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/703847040' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-21T04:31:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.678+0000 7fb4532c6640 1 -- 172.21.15.7:0/3919355503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c106800 msgr2=0x7fb44c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.678+0000 7fb4532c6640 1 --2- 172.21.15.7:0/3919355503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c106800 0x7fb44c106be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb448009880 tx=0x7fb44802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.678+0000 7fb4532c6640 1 -- 172.21.15.7:0/3919355503 shutdown_connections 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.678+0000 7fb4532c6640 1 --2- 172.21.15.7:0/3919355503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb44c100800 0x7fb44c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.678+0000 7fb4532c6640 1 --2- 172.21.15.7:0/3919355503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c106800 0x7fb44c106be0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.678+0000 7fb4532c6640 1 -- 172.21.15.7:0/3919355503 >> 172.21.15.7:0/3919355503 conn(0x7fb44c0fc520 msgr2=0x7fb44c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.678+0000 7fb4532c6640 1 -- 172.21.15.7:0/3919355503 shutdown_connections 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.678+0000 7fb4532c6640 1 -- 172.21.15.7:0/3919355503 wait complete. 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb4532c6640 1 Processor -- start 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb4532c6640 1 -- start start 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb4532c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c100800 0x7fb44c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb4532c6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb44c106800 0x7fb44c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb4532c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c10dcb0 con 0x7fb44c100800 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb4532c6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c10db30 con 0x7fb44c106800 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb451062640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c100800 0x7fb44c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb451062640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c100800 0x7fb44c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47392/0 (socket says 172.21.15.7:47392) 2024-09-21T04:31:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb451062640 1 -- 172.21.15.7:0/526599759 learned_addr learned my addr 172.21.15.7:0/526599759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb450861640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb44c106800 0x7fb44c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb451062640 1 -- 172.21.15.7:0/526599759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb44c106800 msgr2=0x7fb44c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb451062640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb44c106800 0x7fb44c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.679+0000 7fb451062640 1 -- 172.21.15.7:0/526599759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c197850 con 0x7fb44c100800 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.680+0000 7fb451062640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c100800 0x7fb44c19d6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb448031460 tx=0x7fb4480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.680+0000 7fb4427fc640 1 -- 172.21.15.7:0/526599759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb44803f070 con 0x7fb44c100800 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.680+0000 7fb4427fc640 1 -- 172.21.15.7:0/526599759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4480035f0 con 0x7fb44c100800 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.680+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c197ae0 con 0x7fb44c100800 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.680+0000 7fb4427fc640 1 -- 172.21.15.7:0/526599759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb448035030 con 0x7fb44c100800 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.680+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c198010 con 0x7fb44c100800 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.681+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c101f40 con 0x7fb44c100800 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.681+0000 7fb4427fc640 1 -- 172.21.15.7:0/526599759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fb448042070 con 0x7fb44c100800 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.682+0000 7fb4427fc640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb43806e2b0 0x7fb438070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.682+0000 7fb450861640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb43806e2b0 0x7fb438070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.682+0000 7fb4427fc640 1 -- 172.21.15.7:0/526599759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fb4480ae740 con 0x7fb44c100800 2024-09-21T04:31:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.682+0000 7fb450861640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb43806e2b0 0x7fb438070770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0ffbb0 tx=0x7fb43c00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.686+0000 7fb4427fc640 1 -- 172.21.15.7:0/526599759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb448080090 con 0x7fb44c100800 2024-09-21T04:31:42.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.863+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb44c198d80 con 0x7fb44c100800 2024-09-21T04:31:42.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.864+0000 7fb4427fc640 1 -- 172.21.15.7:0/526599759 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+11321 (secure 0 0 0) 0x7fb4480848c0 con 0x7fb44c100800 2024-09-21T04:31:42.867 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:31:42.867 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","created":"2024-09-21T04:25:32.557057+0000","modified":"2024-09-21T04:31:29.949365+0000","last_up_change":"2024-09-21T04:31:28.943860+0000","last_in_change":"2024-09-21T04:31:05.743550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T04:30:11.508639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7692119a-1db2-417f-a086-0181309481b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6803","nonce":2931271991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6805","nonce":2931271991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6809","nonce":2931271991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6807","nonce":2931271991}]},"public_addr":"172.21.15.7:6803/2931271991","cluster_addr":"172.21.15.7:6805/2931271991","heartbeat_back_addr":"172.21.15.7:6809/2931271991","heartbeat_front_addr":"172.21.15.7:6807/2931271991","state":["exists","up"]},{"osd":1,"uuid":"09975c57-317c-4d48-9926-8cfa7786b7b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6811","nonce":3959135592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6813","nonce":3959135592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6817","nonce":3959135592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6815","nonce":3959135592}]},"public_addr":"172.21.15.7:6811/3959135592","cluster_addr":"172.21.15.7:6813/3959135592","heartbeat_back_addr":"172.21.15.7:6817/3959135592","heartbeat_front_addr":"172.21.15.7:6815/3959135592","state":["exists","up"]},{"osd":2,"uuid":"63457a4d-975d-4ea1-b6ac-770bf49aead3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6819","nonce":2953002876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6821","nonce":2953002876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6825","nonce":2953002876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6823","nonce":2953002876}]},"public_addr":"172.21.15.7:6819/2953002876","cluster_addr":"172.21.15.7:6821/2953002876","heartbeat_back_addr":"172.21.15.7:6825/2953002876","heartbeat_front_addr":"172.21.15.7:6823/2953002876","state":["exists","up"]},{"osd":3,"uuid":"d758b507-1cd4-4c67-b7fb-917f65419e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6801","nonce":3034241373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6803","nonce":3034241373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6807","nonce":3034241373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6805","nonce":3034241373}]},"public_addr":"172.21.15.53:6801/3034241373","cluster_addr":"172.21.15.53:6803/3034241373","heartbeat_back_addr":"172.21.15.53:6807/3034241373","heartbeat_front_addr":"172.21.15.53:6805/3034241373","state":["exists","up"]},{"osd":4,"uuid":"8bea39b5-2b2e-4ba0-8224-c8642be0d3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6809","nonce":1120118794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6811","nonce":1120118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6815","nonce":1120118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6813","nonce":1120118794}]},"public_addr":"172.21.15.53:6809/1120118794","cluster_addr":"172.21.15.53:6811/1120118794","heartbeat_back_addr":"172.21.15.53:6815/1120118794","heartbeat_front_addr":"172.21.15.53:6813/1120118794","state":["exists","up"]},{"osd":5,"uuid":"af9dd7dd-6063-4e79-9571-ee72f9bea315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6817","nonce":4215104838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6819","nonce":4215104838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6823","nonce":4215104838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6821","nonce":4215104838}]},"public_addr":"172.21.15.53:6817/4215104838","cluster_addr":"172.21.15.53:6819/4215104838","heartbeat_back_addr":"172.21.15.53:6823/4215104838","heartbeat_front_addr":"172.21.15.53:6821/4215104838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:29:07.047481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:29:38.183480+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:30:07.825963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:30:33.713760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:31:00.254040+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:31:27.450032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6800/2645587315":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/2202116322":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/2211484432":"2024-09-22T04:25:58.338381+0000","172.21.15.7:6801/2645587315":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/1594571982":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/3304037039":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/554975665":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6801/3995670699":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/2851424446":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6800/3808261568":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6801/3808261568":"2024-09-22T04:26:33.198427+0000","172.21.15.7:0/4161500397":"2024-09-22T04:26:33.198427+0000","172.21.15.7:0/3340019345":"2024-09-22T04:27:31.004325+0000","172.21.15.7:6800/3995670699":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/3901749065":"2024-09-22T04:27:31.004325+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:31:42.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb43806e2b0 msgr2=0x7fb438070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:42.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb43806e2b0 0x7fb438070770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0ffbb0 tx=0x7fb43c00b410 comp rx=0 tx=0).stop 2024-09-21T04:31:42.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c100800 msgr2=0x7fb44c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:42.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c100800 0x7fb44c19d6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb448031460 tx=0x7fb4480045e0 comp rx=0 tx=0).stop 2024-09-21T04:31:42.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 shutdown_connections 2024-09-21T04:31:42.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb43806e2b0 0x7fb438070770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:42.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb44c106800 0x7fb44c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:42.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 --2- 172.21.15.7:0/526599759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c100800 0x7fb44c19d6d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 >> 172.21.15.7:0/526599759 conn(0x7fb44c0fc520 msgr2=0x7fb44c06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 shutdown_connections 2024-09-21T04:31:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:42.866+0000 7fb4532c6640 1 -- 172.21.15.7:0/526599759 wait complete. 2024-09-21T04:31:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:42 smithi007 ceph-mon[27954]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:42 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/526599759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:31:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:42 smithi053 ceph-mon[35554]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:42 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/526599759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:31:43.600 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T04:31:43.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:31:43.862 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:44.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.334+0000 7fabd8b50640 1 -- 172.21.15.7:0/887418375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 msgr2=0x7fabd410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.334+0000 7fabd8b50640 1 --2- 172.21.15.7:0/887418375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 0x7fabd410b8d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009880 tx=0x7fabc802a530 comp rx=0 tx=0).stop 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.334+0000 7fabd8b50640 1 -- 172.21.15.7:0/887418375 shutdown_connections 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.334+0000 7fabd8b50640 1 --2- 172.21.15.7:0/887418375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 0x7fabd410b8d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.334+0000 7fabd8b50640 1 --2- 172.21.15.7:0/887418375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabd4102ac0 0x7fabd4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.334+0000 7fabd8b50640 1 -- 172.21.15.7:0/887418375 >> 172.21.15.7:0/887418375 conn(0x7fabd40fc520 msgr2=0x7fabd40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.334+0000 7fabd8b50640 1 -- 172.21.15.7:0/887418375 shutdown_connections 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.334+0000 7fabd8b50640 1 -- 172.21.15.7:0/887418375 wait complete. 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd8b50640 1 Processor -- start 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd8b50640 1 -- start start 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd8b50640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabd4102ac0 0x7fabd419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd8b50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 0x7fabd419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:44.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd8b50640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabd410edd0 con 0x7fabd41033e0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd8b50640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fabd410ec50 con 0x7fabd4102ac0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 0x7fabd419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 0x7fabd419dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51954/0 (socket says 172.21.15.7:51954) 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd1d9b640 1 -- 172.21.15.7:0/3452326074 learned_addr learned my addr 172.21.15.7:0/3452326074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.335+0000 7fabd259c640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabd4102ac0 0x7fabd419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabd1d9b640 1 -- 172.21.15.7:0/3452326074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabd4102ac0 msgr2=0x7fabd419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabd1d9b640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabd4102ac0 0x7fabd419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabd1d9b640 1 -- 172.21.15.7:0/3452326074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd41977e0 con 0x7fabd41033e0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabd1d9b640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 0x7fabd419dba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fabc8031460 tx=0x7fabc8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabc37fe640 1 -- 172.21.15.7:0/3452326074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fabc803f070 con 0x7fabd41033e0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd4197a70 con 0x7fabd41033e0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabc37fe640 1 -- 172.21.15.7:0/3452326074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fabc80035f0 con 0x7fabd41033e0 2024-09-21T04:31:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabc37fe640 1 -- 172.21.15.7:0/3452326074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fabc8035030 con 0x7fabd41033e0 2024-09-21T04:31:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.336+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd4197fa0 con 0x7fabd41033e0 2024-09-21T04:31:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.337+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd406aa90 con 0x7fabd41033e0 2024-09-21T04:31:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.338+0000 7fabc37fe640 1 -- 172.21.15.7:0/3452326074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fabc8042070 con 0x7fabd41033e0 2024-09-21T04:31:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.338+0000 7fabc37fe640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fabbc06e2b0 0x7fabbc070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.338+0000 7fabc37fe640 1 -- 172.21.15.7:0/3452326074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fabc80ae720 con 0x7fabd41033e0 2024-09-21T04:31:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.338+0000 7fabd259c640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fabbc06e2b0 0x7fabbc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.339+0000 7fabd259c640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fabbc06e2b0 0x7fabbc070770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabc400bbe0 tx=0x7fabc400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.343+0000 7fabc37fe640 1 -- 172.21.15.7:0/3452326074 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fabc8080070 con 0x7fabd41033e0 2024-09-21T04:31:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.520+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabd4105f60 con 0x7fabd41033e0 2024-09-21T04:31:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.521+0000 7fabc37fe640 1 -- 172.21.15.7:0/3452326074 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+11321 (secure 0 0 0) 0x7fabc80848a0 con 0x7fabd41033e0 2024-09-21T04:31:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:31:44.524 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"fsid":"70154924-77d1-11ef-baf5-efdc52797490","created":"2024-09-21T04:25:32.557057+0000","modified":"2024-09-21T04:31:29.949365+0000","last_up_change":"2024-09-21T04:31:28.943860+0000","last_in_change":"2024-09-21T04:31:05.743550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T04:30:11.508639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7692119a-1db2-417f-a086-0181309481b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6803","nonce":2931271991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6805","nonce":2931271991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6809","nonce":2931271991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2931271991},{"type":"v1","addr":"172.21.15.7:6807","nonce":2931271991}]},"public_addr":"172.21.15.7:6803/2931271991","cluster_addr":"172.21.15.7:6805/2931271991","heartbeat_back_addr":"172.21.15.7:6809/2931271991","heartbeat_front_addr":"172.21.15.7:6807/2931271991","state":["exists","up"]},{"osd":1,"uuid":"09975c57-317c-4d48-9926-8cfa7786b7b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6811","nonce":3959135592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6813","nonce":3959135592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6817","nonce":3959135592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3959135592},{"type":"v1","addr":"172.21.15.7:6815","nonce":3959135592}]},"public_addr":"172.21.15.7:6811/3959135592","cluster_addr":"172.21.15.7:6813/3959135592","heartbeat_back_addr":"172.21.15.7:6817/3959135592","heartbeat_front_addr":"172.21.15.7:6815/3959135592","state":["exists","up"]},{"osd":2,"uuid":"63457a4d-975d-4ea1-b6ac-770bf49aead3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6819","nonce":2953002876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6821","nonce":2953002876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6825","nonce":2953002876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2953002876},{"type":"v1","addr":"172.21.15.7:6823","nonce":2953002876}]},"public_addr":"172.21.15.7:6819/2953002876","cluster_addr":"172.21.15.7:6821/2953002876","heartbeat_back_addr":"172.21.15.7:6825/2953002876","heartbeat_front_addr":"172.21.15.7:6823/2953002876","state":["exists","up"]},{"osd":3,"uuid":"d758b507-1cd4-4c67-b7fb-917f65419e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6801","nonce":3034241373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6803","nonce":3034241373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6807","nonce":3034241373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3034241373},{"type":"v1","addr":"172.21.15.53:6805","nonce":3034241373}]},"public_addr":"172.21.15.53:6801/3034241373","cluster_addr":"172.21.15.53:6803/3034241373","heartbeat_back_addr":"172.21.15.53:6807/3034241373","heartbeat_front_addr":"172.21.15.53:6805/3034241373","state":["exists","up"]},{"osd":4,"uuid":"8bea39b5-2b2e-4ba0-8224-c8642be0d3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6809","nonce":1120118794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6811","nonce":1120118794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6815","nonce":1120118794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1120118794},{"type":"v1","addr":"172.21.15.53:6813","nonce":1120118794}]},"public_addr":"172.21.15.53:6809/1120118794","cluster_addr":"172.21.15.53:6811/1120118794","heartbeat_back_addr":"172.21.15.53:6815/1120118794","heartbeat_front_addr":"172.21.15.53:6813/1120118794","state":["exists","up"]},{"osd":5,"uuid":"af9dd7dd-6063-4e79-9571-ee72f9bea315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6817","nonce":4215104838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6819","nonce":4215104838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6823","nonce":4215104838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":4215104838},{"type":"v1","addr":"172.21.15.53:6821","nonce":4215104838}]},"public_addr":"172.21.15.53:6817/4215104838","cluster_addr":"172.21.15.53:6819/4215104838","heartbeat_back_addr":"172.21.15.53:6823/4215104838","heartbeat_front_addr":"172.21.15.53:6821/4215104838","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:29:07.047481+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:29:38.183480+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:30:07.825963+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:30:33.713760+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:31:00.254040+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:31:27.450032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6800/2645587315":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/2202116322":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/2211484432":"2024-09-22T04:25:58.338381+0000","172.21.15.7:6801/2645587315":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/1594571982":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/3304037039":"2024-09-22T04:25:58.338381+0000","172.21.15.7:0/554975665":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6801/3995670699":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/2851424446":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6800/3808261568":"2024-09-22T04:26:33.198427+0000","172.21.15.7:6801/3808261568":"2024-09-22T04:26:33.198427+0000","172.21.15.7:0/4161500397":"2024-09-22T04:26:33.198427+0000","172.21.15.7:0/3340019345":"2024-09-22T04:27:31.004325+0000","172.21.15.7:6800/3995670699":"2024-09-22T04:27:31.004325+0000","172.21.15.7:0/3901749065":"2024-09-22T04:27:31.004325+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:31:44.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fabbc06e2b0 msgr2=0x7fabbc070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:44.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fabbc06e2b0 0x7fabbc070770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabc400bbe0 tx=0x7fabc400b390 comp rx=0 tx=0).stop 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 msgr2=0x7fabd419dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 0x7fabd419dba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fabc8031460 tx=0x7fabc8004530 comp rx=0 tx=0).stop 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 shutdown_connections 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fabbc06e2b0 0x7fabbc070770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd41033e0 0x7fabd419dba0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 --2- 172.21.15.7:0/3452326074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabd4102ac0 0x7fabd419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.523+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 >> 172.21.15.7:0/3452326074 conn(0x7fabd40fc520 msgr2=0x7fabd4108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.524+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 shutdown_connections 2024-09-21T04:31:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:44.524+0000 7fabd8b50640 1 -- 172.21.15.7:0/3452326074 wait complete. 2024-09-21T04:31:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:44 smithi007 ceph-mon[27954]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:44 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3452326074' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:31:45.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T04:31:45.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T04:31:45.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T04:31:45.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T04:31:45.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T04:31:45.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T04:31:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:44 smithi053 ceph-mon[35554]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:44 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3452326074' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-21T04:31:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.361+0000 7f4305a76640 1 -- 172.21.15.7:0/2754247102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300106800 msgr2=0x7f4300106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.361+0000 7f4305a76640 1 --2- 172.21.15.7:0/2754247102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300106800 0x7f4300106be0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009880 tx=0x7f42f002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.361+0000 7f4305a76640 1 -- 172.21.15.7:0/2754247102 shutdown_connections 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.361+0000 7f4305a76640 1 --2- 172.21.15.7:0/2754247102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4300100800 0x7f4300100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.361+0000 7f4305a76640 1 --2- 172.21.15.7:0/2754247102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300106800 0x7f4300106be0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.361+0000 7f4305a76640 1 -- 172.21.15.7:0/2754247102 >> 172.21.15.7:0/2754247102 conn(0x7f43000fc520 msgr2=0x7f43000fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.362+0000 7f4305a76640 1 -- 172.21.15.7:0/2754247102 shutdown_connections 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.362+0000 7f4305a76640 1 -- 172.21.15.7:0/2754247102 wait complete. 2024-09-21T04:31:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.362+0000 7f4305a76640 1 Processor -- start 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.362+0000 7f4305a76640 1 -- start start 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f4305a76640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300100800 0x7f4300198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f4305a76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4300106800 0x7f4300198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f4305a76640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f430010dd30 con 0x7f4300100800 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f4305a76640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f430010dbb0 con 0x7f4300106800 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300100800 0x7f4300198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300100800 0x7f4300198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51962/0 (socket says 172.21.15.7:51962) 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42ff7fe640 1 -- 172.21.15.7:0/2862535647 learned_addr learned my addr 172.21.15.7:0/2862535647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42feffd640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4300106800 0x7f4300198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42ff7fe640 1 -- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4300106800 msgr2=0x7f4300198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42ff7fe640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4300106800 0x7f4300198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42ff7fe640 1 -- 172.21.15.7:0/2862535647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43001993c0 con 0x7f4300100800 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42ff7fe640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300100800 0x7f4300198830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f42f0031460 tx=0x7f42f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42f003f070 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4300199650 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42f0004cf0 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.363+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42f0035030 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.364+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43001a1c00 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.364+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43000628c0 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.365+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f42f0042070 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.365+0000 7f42fcff9640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f42e806e2b0 0x7f42e8070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.366+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f42f00ae910 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.366+0000 7f42feffd640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f42e806e2b0 0x7f42e8070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.366+0000 7f42fcff9640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] conn(0x7f42e80740a0 0x7f42e80764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.366+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 --> [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42e8076b70 con 0x7f42e80740a0 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.366+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f42f00b32a0 con 0x7f4300100800 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.366+0000 7f42feffd640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f42e806e2b0 0x7f42e8070770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f430006ad40 tx=0x7f42f4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.366+0000 7f42fffff640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] conn(0x7f42e80740a0 0x7f42e80764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.366+0000 7f42fffff640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] conn(0x7f42e80740a0 0x7f42e80764c0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.367+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 <== osd.5 v2:172.21.15.53:6816/4215104838 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42e8076b70 con 0x7f42e80740a0 2024-09-21T04:31:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.385+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 --> [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4300062d40 con 0x7f42e80740a0 2024-09-21T04:31:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f42fcff9640 1 -- 172.21.15.7:0/2862535647 <== osd.5 v2:172.21.15.53:6816/4215104838 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4300062d40 con 0x7f42e80740a0 2024-09-21T04:31:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] conn(0x7f42e80740a0 msgr2=0x7f42e80764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] conn(0x7f42e80740a0 0x7f42e80764c0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f42e806e2b0 msgr2=0x7f42e8070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f42e806e2b0 0x7f42e8070770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f430006ad40 tx=0x7f42f4008040 comp rx=0 tx=0).stop 2024-09-21T04:31:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300100800 msgr2=0x7f4300198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300100800 0x7f4300198830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f42f0031460 tx=0x7f42f00048f0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 shutdown_connections 2024-09-21T04:31:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:6816/4215104838,v1:172.21.15.53:6817/4215104838] conn(0x7f42e80740a0 0x7f42e80764c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f42e806e2b0 0x7f42e8070770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4300106800 0x7f4300198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 --2- 172.21.15.7:0/2862535647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4300100800 0x7f4300198830 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 >> 172.21.15.7:0/2862535647 conn(0x7f43000fc520 msgr2=0x7f43000fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 shutdown_connections 2024-09-21T04:31:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:46.386+0000 7f4305a76640 1 -- 172.21.15.7:0/2862535647 wait complete. 2024-09-21T04:31:47.134 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:46 smithi007 ceph-mon[27954]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:47.135 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:31:47.135 INFO:teuthology.orchestra.run.smithi007.stdout:158913789958 2024-09-21T04:31:47.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T04:31:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.350+0000 7fa6221de640 1 -- 172.21.15.7:0/3498103417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 msgr2=0x7fa61c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:47.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.350+0000 7fa6221de640 1 --2- 172.21.15.7:0/3498103417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 0x7fa61c102ef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0098b0 tx=0x7fa60c02a560 comp rx=0 tx=0).stop 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.350+0000 7fa6221de640 1 -- 172.21.15.7:0/3498103417 shutdown_connections 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.350+0000 7fa6221de640 1 --2- 172.21.15.7:0/3498103417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa61c103430 0x7fa61c10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.350+0000 7fa6221de640 1 --2- 172.21.15.7:0/3498103417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 0x7fa61c102ef0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.350+0000 7fa6221de640 1 -- 172.21.15.7:0/3498103417 >> 172.21.15.7:0/3498103417 conn(0x7fa61c0fc4d0 msgr2=0x7fa61c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.350+0000 7fa6221de640 1 -- 172.21.15.7:0/3498103417 shutdown_connections 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.351+0000 7fa6221de640 1 -- 172.21.15.7:0/3498103417 wait complete. 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.351+0000 7fa6221de640 1 Processor -- start 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.351+0000 7fa6221de640 1 -- start start 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.351+0000 7fa6221de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 0x7fa61c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.351+0000 7fa6221de640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa61c103430 0x7fa61c19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.351+0000 7fa6221de640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10edc0 con 0x7fa61c102b10 2024-09-21T04:31:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa6221de640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10ec40 con 0x7fa61c103430 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa61b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 0x7fa61c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa61b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 0x7fa61c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51984/0 (socket says 172.21.15.7:51984) 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa61b7fe640 1 -- 172.21.15.7:0/3277254272 learned_addr learned my addr 172.21.15.7:0/3277254272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa61affd640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa61c103430 0x7fa61c19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa61b7fe640 1 -- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa61c103430 msgr2=0x7fa61c19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa61b7fe640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa61c103430 0x7fa61c19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa61b7fe640 1 -- 172.21.15.7:0/3277254272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1977f0 con 0x7fa61c102b10 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa61b7fe640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 0x7fa61c19d670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa60c031460 tx=0x7fa60c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa60c03f070 con 0x7fa61c102b10 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c197a80 con 0x7fa61c102b10 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa60c0035f0 con 0x7fa61c102b10 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa60c035030 con 0x7fa61c102b10 2024-09-21T04:31:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.352+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c197fb0 con 0x7fa61c102b10 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.353+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa61c198630 con 0x7fa61c102b10 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.355+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fa60c042070 con 0x7fa61c102b10 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.355+0000 7fa618ff9640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa60406e2b0 0x7fa604070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.355+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fa60c0aed50 con 0x7fa61c102b10 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.355+0000 7fa61affd640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa60406e2b0 0x7fa604070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.355+0000 7fa618ff9640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] conn(0x7fa6040740a0 0x7fa6040764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.355+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 --> [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa60c003940 con 0x7fa6040740a0 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.355+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa60c0af1f0 con 0x7fa61c102b10 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.356+0000 7fa61bfff640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] conn(0x7fa6040740a0 0x7fa6040764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.356+0000 7fa61affd640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa60406e2b0 0x7fa604070770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa61c108970 tx=0x7fa61000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.356+0000 7fa61bfff640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] conn(0x7fa6040740a0 0x7fa6040764c0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.357+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 <== osd.3 v2:172.21.15.53:6800/3034241373 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa60c003940 con 0x7fa6040740a0 2024-09-21T04:31:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.381+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 --> [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa61c062880 con 0x7fa6040740a0 2024-09-21T04:31:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.382+0000 7fa618ff9640 1 -- 172.21.15.7:0/3277254272 <== osd.3 v2:172.21.15.53:6800/3034241373 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa61c062880 con 0x7fa6040740a0 2024-09-21T04:31:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] conn(0x7fa6040740a0 msgr2=0x7fa6040764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] conn(0x7fa6040740a0 0x7fa6040764c0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa60406e2b0 msgr2=0x7fa604070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa60406e2b0 0x7fa604070770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa61c108970 tx=0x7fa61000b390 comp rx=0 tx=0).stop 2024-09-21T04:31:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 msgr2=0x7fa61c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:47.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 0x7fa61c19d670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa60c031460 tx=0x7fa60c0045c0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 shutdown_connections 2024-09-21T04:31:47.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:6800/3034241373,v1:172.21.15.53:6801/3034241373] conn(0x7fa6040740a0 0x7fa6040764c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa60406e2b0 0x7fa604070770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa61c103430 0x7fa61c19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 --2- 172.21.15.7:0/3277254272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa61c102b10 0x7fa61c19d670 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa60c031460 tx=0x7fa60c0045c0 comp rx=0 tx=0).stop 2024-09-21T04:31:47.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.383+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 >> 172.21.15.7:0/3277254272 conn(0x7fa61c0fc4d0 msgr2=0x7fa61c107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:47.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.384+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 shutdown_connections 2024-09-21T04:31:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:47.388+0000 7fa6221de640 1 -- 172.21.15.7:0/3277254272 wait complete. 2024-09-21T04:31:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:46 smithi053 ceph-mon[35554]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:31:48.144 INFO:teuthology.orchestra.run.smithi007.stdout:107374182417 2024-09-21T04:31:48.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T04:31:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:48.160 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:48.163 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:48.168 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:48.217 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:49.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.088+0000 7f002a888640 1 -- 172.21.15.7:0/595513770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0024101050 msgr2=0x7f002410dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.088+0000 7f002a888640 1 --2- 172.21.15.7:0/595513770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0024101050 0x7f002410dd10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0018009880 tx=0x7f001802a490 comp rx=0 tx=0).stop 2024-09-21T04:31:49.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.088+0000 7f002a888640 1 -- 172.21.15.7:0/595513770 shutdown_connections 2024-09-21T04:31:49.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.088+0000 7f002a888640 1 --2- 172.21.15.7:0/595513770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0024101050 0x7f002410dd10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.088+0000 7f002a888640 1 --2- 172.21.15.7:0/595513770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00241006a0 0x7f0024100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.088+0000 7f002a888640 1 -- 172.21.15.7:0/595513770 >> 172.21.15.7:0/595513770 conn(0x7f00240fc550 msgr2=0x7f00240fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:49.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.088+0000 7f002a888640 1 -- 172.21.15.7:0/595513770 shutdown_connections 2024-09-21T04:31:49.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.088+0000 7f002a888640 1 -- 172.21.15.7:0/595513770 wait complete. 2024-09-21T04:31:49.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f002a888640 1 Processor -- start 2024-09-21T04:31:49.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f002a888640 1 -- start start 2024-09-21T04:31:49.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f002a888640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00241006a0 0x7f002419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f002a888640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0024101050 0x7f002419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f002a888640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0024110f30 con 0x7f00241006a0 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f002a888640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0024110db0 con 0x7f0024101050 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f0023fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00241006a0 0x7f002419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f0023fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00241006a0 0x7f002419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51998/0 (socket says 172.21.15.7:51998) 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.089+0000 7f0023fff640 1 -- 172.21.15.7:0/637423807 learned_addr learned my addr 172.21.15.7:0/637423807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f00237fe640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0024101050 0x7f002419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f0023fff640 1 -- 172.21.15.7:0/637423807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0024101050 msgr2=0x7f002419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f0023fff640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0024101050 0x7f002419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f0023fff640 1 -- 172.21.15.7:0/637423807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0024199940 con 0x7f00241006a0 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f0023fff640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00241006a0 0x7f002419f780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f001400d6f0 tx=0x7f001400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0014017070 con 0x7f00241006a0 2024-09-21T04:31:49.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f001400de00 con 0x7f00241006a0 2024-09-21T04:31:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0024199c30 con 0x7f00241006a0 2024-09-21T04:31:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0014004bb0 con 0x7f00241006a0 2024-09-21T04:31:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.090+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002419a190 con 0x7f00241006a0 2024-09-21T04:31:49.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.091+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f002419a750 con 0x7f00241006a0 2024-09-21T04:31:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.092+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f0014004e60 con 0x7f00241006a0 2024-09-21T04:31:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.092+0000 7f00217fa640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f000c06e2b0 0x7f000c070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.092+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f001408d970 con 0x7f00241006a0 2024-09-21T04:31:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.092+0000 7f00237fe640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f000c06e2b0 0x7f000c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.092+0000 7f00217fa640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] conn(0x7f000c0740a0 0x7f000c0764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.092+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 --> [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f000c076b70 con 0x7f000c0740a0 2024-09-21T04:31:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.092+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f00140922a0 con 0x7f00241006a0 2024-09-21T04:31:49.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.092+0000 7f0028e25640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] conn(0x7f000c0740a0 0x7f000c0764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:49.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.093+0000 7f00237fe640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f000c06e2b0 0x7f000c070770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f002419b130 tx=0x7f0018031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:49.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.093+0000 7f0028e25640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] conn(0x7f000c0740a0 0x7f000c0764c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:49.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.093+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 <== osd.2 v2:172.21.15.7:6818/2953002876 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f000c076b70 con 0x7f000c0740a0 2024-09-21T04:31:49.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.107+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 --> [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0024062880 con 0x7f000c0740a0 2024-09-21T04:31:49.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f00217fa640 1 -- 172.21.15.7:0/637423807 <== osd.2 v2:172.21.15.7:6818/2953002876 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0024062880 con 0x7f000c0740a0 2024-09-21T04:31:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] conn(0x7f000c0740a0 msgr2=0x7f000c0764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] conn(0x7f000c0740a0 0x7f000c0764c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f000c06e2b0 msgr2=0x7f000c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f000c06e2b0 0x7f000c070770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f002419b130 tx=0x7f0018031400 comp rx=0 tx=0).stop 2024-09-21T04:31:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00241006a0 msgr2=0x7f002419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00241006a0 0x7f002419f780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f001400d6f0 tx=0x7f001400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 shutdown_connections 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6818/2953002876,v1:172.21.15.7:6819/2953002876] conn(0x7f000c0740a0 0x7f000c0764c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f000c06e2b0 0x7f000c070770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0024101050 0x7f002419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 --2- 172.21.15.7:0/637423807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00241006a0 0x7f002419f780 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 >> 172.21.15.7:0/637423807 conn(0x7f00240fc550 msgr2=0x7f00240fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 shutdown_connections 2024-09-21T04:31:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.108+0000 7f002a888640 1 -- 172.21.15.7:0/637423807 wait complete. 2024-09-21T04:31:49.275 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:48 smithi007 ceph-mon[27954]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:49.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.287+0000 7f7696da4640 1 -- 172.21.15.7:0/3783852083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690106800 msgr2=0x7f7690106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.287+0000 7f7696da4640 1 --2- 172.21.15.7:0/3783852083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690106800 0x7f7690106be0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7684009880 tx=0x7f768402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.287+0000 7f7696da4640 1 -- 172.21.15.7:0/3783852083 shutdown_connections 2024-09-21T04:31:49.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.287+0000 7f7696da4640 1 --2- 172.21.15.7:0/3783852083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690100800 0x7f7690100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.287+0000 7f7696da4640 1 --2- 172.21.15.7:0/3783852083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690106800 0x7f7690106be0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.287+0000 7f7696da4640 1 -- 172.21.15.7:0/3783852083 >> 172.21.15.7:0/3783852083 conn(0x7f76900fc520 msgr2=0x7f76900fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:49.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.287+0000 7f7696da4640 1 -- 172.21.15.7:0/3783852083 shutdown_connections 2024-09-21T04:31:49.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.287+0000 7f7696da4640 1 -- 172.21.15.7:0/3783852083 wait complete. 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7696da4640 1 Processor -- start 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7696da4640 1 -- start start 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7696da4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690100800 0x7f769019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7696da4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690106800 0x7f769019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7696da4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f769010dcb0 con 0x7f7690100800 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7696da4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f769010db30 con 0x7f7690106800 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7694b40640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690100800 0x7f769019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7694b40640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690100800 0x7f769019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52016/0 (socket says 172.21.15.7:52016) 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f7694b40640 1 -- 172.21.15.7:0/52238300 learned_addr learned my addr 172.21.15.7:0/52238300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:49.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.288+0000 7f768ffff640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690106800 0x7f769019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f7694b40640 1 -- 172.21.15.7:0/52238300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690106800 msgr2=0x7f769019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f7694b40640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690106800 0x7f769019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f7694b40640 1 -- 172.21.15.7:0/52238300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7690197820 con 0x7f7690100800 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f7694b40640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690100800 0x7f769019d6a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7684031460 tx=0x7f76840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f768403f070 con 0x7f7690100800 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f76840035f0 con 0x7f7690100800 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7690197ab0 con 0x7f7690100800 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7684035030 con 0x7f7690100800 2024-09-21T04:31:49.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.289+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7690197fe0 con 0x7f7690100800 2024-09-21T04:31:49.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.290+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76901985e0 con 0x7f7690100800 2024-09-21T04:31:49.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.291+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f7684042070 con 0x7f7690100800 2024-09-21T04:31:49.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.291+0000 7f768dffb640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f768006e310 0x7f76800707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:49.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.291+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f76840ae710 con 0x7f7690100800 2024-09-21T04:31:49.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.291+0000 7f768ffff640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f768006e310 0x7f76800707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:49.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.291+0000 7f768dffb640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] conn(0x7f7680074100 0x7f7680076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:49.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.291+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 --> [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76840054e0 con 0x7f7680074100 2024-09-21T04:31:49.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.292+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f76840b5050 con 0x7f7690100800 2024-09-21T04:31:49.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.292+0000 7f7695341640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] conn(0x7f7680074100 0x7f7680076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:49.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.292+0000 7f768ffff640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f768006e310 0x7f76800707d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76900ffb10 tx=0x7f767c00b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:49.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.292+0000 7f7695341640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] conn(0x7f7680074100 0x7f7680076520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:49.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.293+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 <== osd.0 v2:172.21.15.7:6802/2931271991 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76840054e0 con 0x7f7680074100 2024-09-21T04:31:49.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.307+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 --> [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f769006ac30 con 0x7f7680074100 2024-09-21T04:31:49.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.307+0000 7f768dffb640 1 -- 172.21.15.7:0/52238300 <== osd.0 v2:172.21.15.7:6802/2931271991 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f769006ac30 con 0x7f7680074100 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.307+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] conn(0x7f7680074100 msgr2=0x7f7680076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.307+0000 7f7696da4640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] conn(0x7f7680074100 0x7f7680076520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.307+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f768006e310 msgr2=0x7f76800707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f768006e310 0x7f76800707d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76900ffb10 tx=0x7f767c00b390 comp rx=0 tx=0).stop 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690100800 msgr2=0x7f769019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690100800 0x7f769019d6a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7684031460 tx=0x7f76840045e0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 shutdown_connections 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6802/2931271991,v1:172.21.15.7:6803/2931271991] conn(0x7f7680074100 0x7f7680076520 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f768006e310 0x7f76800707d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7690106800 0x7f769019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 --2- 172.21.15.7:0/52238300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7690100800 0x7f769019d6a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 >> 172.21.15.7:0/52238300 conn(0x7f76900fc520 msgr2=0x7f769006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:49.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 shutdown_connections 2024-09-21T04:31:49.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:49.308+0000 7f7696da4640 1 -- 172.21.15.7:0/52238300 wait complete. 2024-09-21T04:31:49.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:48 smithi053 ceph-mon[35554]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:50.453 INFO:teuthology.orchestra.run.smithi007.stdout:38654705699 2024-09-21T04:31:50.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T04:31:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.607+0000 7fc692a10640 1 -- 172.21.15.7:0/3153207781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 msgr2=0x7fc68c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.607+0000 7fc692a10640 1 --2- 172.21.15.7:0/3153207781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 0x7fc68c100c50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc680009880 tx=0x7fc68002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.607+0000 7fc692a10640 1 -- 172.21.15.7:0/3153207781 shutdown_connections 2024-09-21T04:31:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.607+0000 7fc692a10640 1 --2- 172.21.15.7:0/3153207781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 0x7fc68c100c50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.607+0000 7fc692a10640 1 --2- 172.21.15.7:0/3153207781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc68c1067f0 0x7fc68c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.607+0000 7fc692a10640 1 -- 172.21.15.7:0/3153207781 >> 172.21.15.7:0/3153207781 conn(0x7fc68c0fc550 msgr2=0x7fc68c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.607+0000 7fc692a10640 1 -- 172.21.15.7:0/3153207781 shutdown_connections 2024-09-21T04:31:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.607+0000 7fc692a10640 1 -- 172.21.15.7:0/3153207781 wait complete. 2024-09-21T04:31:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.608+0000 7fc692a10640 1 Processor -- start 2024-09-21T04:31:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.608+0000 7fc692a10640 1 -- start start 2024-09-21T04:31:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.608+0000 7fc692a10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 0x7fc68c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.608+0000 7fc692a10640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc68c1067f0 0x7fc68c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.608+0000 7fc692a10640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10dcc0 con 0x7fc68c1007f0 2024-09-21T04:31:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.608+0000 7fc692a10640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c10db40 con 0x7fc68c1067f0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc68bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 0x7fc68c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc68bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 0x7fc68c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52044/0 (socket says 172.21.15.7:52044) 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc68bfff640 1 -- 172.21.15.7:0/2144761129 learned_addr learned my addr 172.21.15.7:0/2144761129 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc68b7fe640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc68c1067f0 0x7fc68c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc68bfff640 1 -- 172.21.15.7:0/2144761129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc68c1067f0 msgr2=0x7fc68c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc68bfff640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc68c1067f0 0x7fc68c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc68bfff640 1 -- 172.21.15.7:0/2144761129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c199330 con 0x7fc68c1007f0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc68bfff640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 0x7fc68c1987f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00d6f0 tx=0x7fc67c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc67c017070 con 0x7fc68c1007f0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc67c00de00 con 0x7fc68c1007f0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c199620 con 0x7fc68c1007f0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.609+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc67c004bb0 con 0x7fc68c1007f0 2024-09-21T04:31:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.610+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1a1c20 con 0x7fc68c1007f0 2024-09-21T04:31:50.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.610+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc68c062880 con 0x7fc68c1007f0 2024-09-21T04:31:50.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.611+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fc67c009a30 con 0x7fc68c1007f0 2024-09-21T04:31:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.612+0000 7fc6897fa640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc67406e310 0x7fc6740707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.612+0000 7fc68b7fe640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc67406e310 0x7fc6740707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.612+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fc67c08d8d0 con 0x7fc68c1007f0 2024-09-21T04:31:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.612+0000 7fc6897fa640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] conn(0x7fc674074100 0x7fc674076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.612+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 --> [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc674076bd0 con 0x7fc674074100 2024-09-21T04:31:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.612+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc67c0922a0 con 0x7fc68c1007f0 2024-09-21T04:31:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.612+0000 7fc690fad640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] conn(0x7fc674074100 0x7fc674076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.612+0000 7fc68b7fe640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc67406e310 0x7fc6740707d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc68c06a7f0 tx=0x7fc680031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:50.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.613+0000 7fc690fad640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] conn(0x7fc674074100 0x7fc674076520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:50.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.613+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 <== osd.1 v2:172.21.15.7:6810/3959135592 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc674076bd0 con 0x7fc674074100 2024-09-21T04:31:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.627+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 --> [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc68c062d00 con 0x7fc674074100 2024-09-21T04:31:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.627+0000 7fc6897fa640 1 -- 172.21.15.7:0/2144761129 <== osd.1 v2:172.21.15.7:6810/3959135592 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc68c062d00 con 0x7fc674074100 2024-09-21T04:31:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.627+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] conn(0x7fc674074100 msgr2=0x7fc674076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.627+0000 7fc692a10640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] conn(0x7fc674074100 0x7fc674076520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc67406e310 msgr2=0x7fc6740707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc67406e310 0x7fc6740707d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc68c06a7f0 tx=0x7fc680031400 comp rx=0 tx=0).stop 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 msgr2=0x7fc68c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 0x7fc68c1987f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00d6f0 tx=0x7fc67c00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 shutdown_connections 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6810/3959135592,v1:172.21.15.7:6811/3959135592] conn(0x7fc674074100 0x7fc674076520 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc67406e310 0x7fc6740707d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc68c1067f0 0x7fc68c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 --2- 172.21.15.7:0/2144761129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c1007f0 0x7fc68c1987f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 >> 172.21.15.7:0/2144761129 conn(0x7fc68c0fc550 msgr2=0x7fc68c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 shutdown_connections 2024-09-21T04:31:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:50.628+0000 7fc692a10640 1 -- 172.21.15.7:0/2144761129 wait complete. 2024-09-21T04:31:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.169+0000 7f772d560640 1 -- 172.21.15.7:0/2062666032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728107900 msgr2=0x7f7728107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.169+0000 7f772d560640 1 --2- 172.21.15.7:0/2062666032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728107900 0x7f7728107ce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7718009880 tx=0x7f771802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.169+0000 7f772d560640 1 -- 172.21.15.7:0/2062666032 shutdown_connections 2024-09-21T04:31:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.169+0000 7f772d560640 1 --2- 172.21.15.7:0/2062666032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7728101900 0x7f7728101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.169+0000 7f772d560640 1 --2- 172.21.15.7:0/2062666032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728107900 0x7f7728107ce0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.169+0000 7f772d560640 1 -- 172.21.15.7:0/2062666032 >> 172.21.15.7:0/2062666032 conn(0x7f77280fd600 msgr2=0x7f77280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.169+0000 7f772d560640 1 -- 172.21.15.7:0/2062666032 shutdown_connections 2024-09-21T04:31:51.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.169+0000 7f772d560640 1 -- 172.21.15.7:0/2062666032 wait complete. 2024-09-21T04:31:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f772d560640 1 Processor -- start 2024-09-21T04:31:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f772d560640 1 -- start start 2024-09-21T04:31:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f772d560640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728101900 0x7f77280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f772d560640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7728107900 0x7f7728074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f772d560640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f772810edb0 con 0x7f7728101900 2024-09-21T04:31:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f772d560640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f772810ec30 con 0x7f7728107900 2024-09-21T04:31:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f7726ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728101900 0x7f77280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f7726ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728101900 0x7f77280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52060/0 (socket says 172.21.15.7:52060) 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.170+0000 7f7726ffd640 1 -- 172.21.15.7:0/2483519290 learned_addr learned my addr 172.21.15.7:0/2483519290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f771ffff640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7728107900 0x7f7728074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f7726ffd640 1 -- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7728107900 msgr2=0x7f7728074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f7726ffd640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7728107900 0x7f7728074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f7726ffd640 1 -- 172.21.15.7:0/2483519290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7728074f70 con 0x7f7728101900 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f7726ffd640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728101900 0x7f77280744f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7718031460 tx=0x7f77180045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f771803f070 con 0x7f7728101900 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7728075200 con 0x7f7728101900 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77180035f0 con 0x7f7728101900 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7718035030 con 0x7f7728101900 2024-09-21T04:31:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.171+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77281a3da0 con 0x7f7728101900 2024-09-21T04:31:51.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.172+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77280628c0 con 0x7f7728101900 2024-09-21T04:31:51.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.173+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f7718042070 con 0x7f7728101900 2024-09-21T04:31:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.174+0000 7f7724ff9640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f771406e310 0x7f77140707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.174+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f77180ae670 con 0x7f7728101900 2024-09-21T04:31:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.174+0000 7f771ffff640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f771406e310 0x7f77140707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.174+0000 7f7724ff9640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] conn(0x7f7714074100 0x7f7714076520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.174+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 --> [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77180054e0 con 0x7f7714074100 2024-09-21T04:31:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.174+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f77180b5050 con 0x7f7728101900 2024-09-21T04:31:51.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.174+0000 7f77277fe640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] conn(0x7f7714074100 0x7f7714076520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:51.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.174+0000 7f771ffff640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f771406e310 0x7f77140707d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7728075f80 tx=0x7f771000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:51.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.175+0000 7f77277fe640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] conn(0x7f7714074100 0x7f7714076520 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:51.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.176+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 <== osd.4 v2:172.21.15.53:6808/1120118794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77180054e0 con 0x7f7714074100 2024-09-21T04:31:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.192+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 --> [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7728062d40 con 0x7f7714074100 2024-09-21T04:31:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f7724ff9640 1 -- 172.21.15.7:0/2483519290 <== osd.4 v2:172.21.15.53:6808/1120118794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7728062d40 con 0x7f7714074100 2024-09-21T04:31:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] conn(0x7f7714074100 msgr2=0x7f7714076520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] conn(0x7f7714074100 0x7f7714076520 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f771406e310 msgr2=0x7f77140707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f771406e310 0x7f77140707d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7728075f80 tx=0x7f771000b410 comp rx=0 tx=0).stop 2024-09-21T04:31:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728101900 msgr2=0x7f77280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728101900 0x7f77280744f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7718031460 tx=0x7f77180045e0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 shutdown_connections 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:6808/1120118794,v1:172.21.15.53:6809/1120118794] conn(0x7f7714074100 0x7f7714076520 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f771406e310 0x7f77140707d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7728107900 0x7f7728074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 --2- 172.21.15.7:0/2483519290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7728101900 0x7f77280744f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 >> 172.21.15.7:0/2483519290 conn(0x7f77280fd600 msgr2=0x7f77280fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 shutdown_connections 2024-09-21T04:31:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:51.193+0000 7f772d560640 1 -- 172.21.15.7:0/2483519290 wait complete. 2024-09-21T04:31:51.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:50 smithi007 ceph-mon[27954]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:50 smithi053 ceph-mon[35554]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:51.829 INFO:teuthology.orchestra.run.smithi007.stdout:60129542173 2024-09-21T04:31:51.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T04:31:52.687 INFO:teuthology.orchestra.run.smithi007.stdout:137438953484 2024-09-21T04:31:52.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T04:31:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.853+0000 7f186bfc0640 1 -- 172.21.15.7:0/798195829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1864102ab0 msgr2=0x7f1864102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.853+0000 7f186bfc0640 1 --2- 172.21.15.7:0/798195829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1864102ab0 0x7f1864102e90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1860009880 tx=0x7f186002a530 comp rx=0 tx=0).stop 2024-09-21T04:31:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.853+0000 7f186bfc0640 1 -- 172.21.15.7:0/798195829 shutdown_connections 2024-09-21T04:31:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.853+0000 7f186bfc0640 1 --2- 172.21.15.7:0/798195829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18641033d0 0x7f186410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.853+0000 7f186bfc0640 1 --2- 172.21.15.7:0/798195829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1864102ab0 0x7f1864102e90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.853+0000 7f186bfc0640 1 -- 172.21.15.7:0/798195829 >> 172.21.15.7:0/798195829 conn(0x7f18640fc550 msgr2=0x7f18640fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.853+0000 7f186bfc0640 1 -- 172.21.15.7:0/798195829 shutdown_connections 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.853+0000 7f186bfc0640 1 -- 172.21.15.7:0/798195829 wait complete. 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186bfc0640 1 Processor -- start 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186bfc0640 1 -- start start 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186bfc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1864102ab0 0x7f186419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186bfc0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18641033d0 0x7f186419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186bfc0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f186410edc0 con 0x7f18641033d0 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186bfc0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f186410ec40 con 0x7f1864102ab0 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186955b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18641033d0 0x7f186419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186955b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18641033d0 0x7f186419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52084/0 (socket says 172.21.15.7:52084) 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f186955b640 1 -- 172.21.15.7:0/27154324 learned_addr learned my addr 172.21.15.7:0/27154324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.854+0000 7f1869d5c640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1864102ab0 0x7f186419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f186955b640 1 -- 172.21.15.7:0/27154324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1864102ab0 msgr2=0x7f186419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f186955b640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1864102ab0 0x7f186419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f186955b640 1 -- 172.21.15.7:0/27154324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18641977b0 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f186955b640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18641033d0 0x7f186419db70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f185400d6c0 tx=0x7f185400db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f185affd640 1 -- 172.21.15.7:0/27154324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1854017070 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1864197aa0 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f185affd640 1 -- 172.21.15.7:0/27154324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1854009ce0 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f185affd640 1 -- 172.21.15.7:0/27154324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f185400f3c0 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.855+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1864198030 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.856+0000 7f185affd640 1 -- 172.21.15.7:0/27154324 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f1854010460 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.857+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186406a580 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.857+0000 7f185affd640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f185006e2b0 0x7f1850070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.857+0000 7f185affd640 1 -- 172.21.15.7:0/27154324 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f185408dbf0 con 0x7f18641033d0 2024-09-21T04:31:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.857+0000 7f1869d5c640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f185006e2b0 0x7f1850070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:52.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.858+0000 7f1869d5c640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f185006e2b0 0x7f1850070770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1860031460 tx=0x7f18600095c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:52.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:52.865+0000 7f185affd640 1 -- 172.21.15.7:0/27154324 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f185405b890 con 0x7f18641033d0 2024-09-21T04:31:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.048+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1864198d10 con 0x7f18641033d0 2024-09-21T04:31:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.048+0000 7f185affd640 1 -- 172.21.15.7:0/27154324 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1864198d10 con 0x7f18641033d0 2024-09-21T04:31:53.051 INFO:teuthology.orchestra.run.smithi007.stdout:158913789959 2024-09-21T04:31:53.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f185006e2b0 msgr2=0x7f1850070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:53.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f185006e2b0 0x7f1850070770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1860031460 tx=0x7f18600095c0 comp rx=0 tx=0).stop 2024-09-21T04:31:53.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18641033d0 msgr2=0x7f186419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18641033d0 0x7f186419db70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f185400d6c0 tx=0x7f185400db80 comp rx=0 tx=0).stop 2024-09-21T04:31:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 shutdown_connections 2024-09-21T04:31:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f185006e2b0 0x7f1850070770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18641033d0 0x7f186419db70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 --2- 172.21.15.7:0/27154324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1864102ab0 0x7f186419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 >> 172.21.15.7:0/27154324 conn(0x7f18640fc550 msgr2=0x7f1864108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 shutdown_connections 2024-09-21T04:31:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:53.050+0000 7f186bfc0640 1 -- 172.21.15.7:0/27154324 wait complete. 2024-09-21T04:31:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:52 smithi007 ceph-mon[27954]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:53.363 INFO:teuthology.orchestra.run.smithi007.stdout:81604378647 2024-09-21T04:31:53.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T04:31:53.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:52 smithi053 ceph-mon[35554]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:54.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789958 got 158913789959 for osd.5 2024-09-21T04:31:54.047 DEBUG:teuthology.parallel:result is None 2024-09-21T04:31:54.085 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:54.085 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:54.315 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:53 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/27154324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-21T04:31:54.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:53 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/27154324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.562+0000 7fc3e984b640 1 -- 172.21.15.7:0/2998409418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 msgr2=0x7fc3e4100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.562+0000 7fc3e984b640 1 --2- 172.21.15.7:0/2998409418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 0x7fc3e4100220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009880 tx=0x7fc3cc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.562+0000 7fc3e984b640 1 -- 172.21.15.7:0/2998409418 shutdown_connections 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.562+0000 7fc3e984b640 1 --2- 172.21.15.7:0/2998409418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 0x7fc3e4100220 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.562+0000 7fc3e984b640 1 --2- 172.21.15.7:0/2998409418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41069a0 0x7fc3e4106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.562+0000 7fc3e984b640 1 -- 172.21.15.7:0/2998409418 >> 172.21.15.7:0/2998409418 conn(0x7fc3e40fbc90 msgr2=0x7fc3e40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.562+0000 7fc3e984b640 1 -- 172.21.15.7:0/2998409418 shutdown_connections 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.562+0000 7fc3e984b640 1 -- 172.21.15.7:0/2998409418 wait complete. 2024-09-21T04:31:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.563+0000 7fc3e984b640 1 Processor -- start 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.563+0000 7fc3e984b640 1 -- start start 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.563+0000 7fc3e984b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 0x7fc3e4100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.563+0000 7fc3e984b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41069a0 0x7fc3e41014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.563+0000 7fc3e984b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e410ef40 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.563+0000 7fc3e984b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e410edc0 con 0x7fc3e41069a0 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.563+0000 7fc3e2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 0x7fc3e4100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 0x7fc3e4100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45668/0 (socket says 172.21.15.7:45668) 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e2ffd640 1 -- 172.21.15.7:0/2348694488 learned_addr learned my addr 172.21.15.7:0/2348694488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3da7fc640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41069a0 0x7fc3e41014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e2ffd640 1 -- 172.21.15.7:0/2348694488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41069a0 msgr2=0x7fc3e41014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e2ffd640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41069a0 0x7fc3e41014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e2ffd640 1 -- 172.21.15.7:0/2348694488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e4101a20 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e2ffd640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 0x7fc3e4100fa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d6f0 tx=0x7fc3d400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e8849640 1 -- 172.21.15.7:0/2348694488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3d4017070 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e4101d10 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e8849640 1 -- 172.21.15.7:0/2348694488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3d400de00 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e8849640 1 -- 172.21.15.7:0/2348694488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3d4004bb0 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.564+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e41a3f70 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.565+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e41030e0 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.566+0000 7fc3e8849640 1 -- 172.21.15.7:0/2348694488 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fc3d4004e60 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.566+0000 7fc3e8849640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc3d006e2b0 0x7fc3d0070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.567+0000 7fc3e8849640 1 -- 172.21.15.7:0/2348694488 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fc3d408d9e0 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.567+0000 7fc3da7fc640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc3d006e2b0 0x7fc3d0070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.567+0000 7fc3da7fc640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc3d006e2b0 0x7fc3d0070770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4102a30 tx=0x7fc3cc031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:54.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.571+0000 7fc3e8849640 1 -- 172.21.15.7:0/2348694488 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc3d405f330 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.603 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.750+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc3e4100220 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.750+0000 7fc3e8849640 1 -- 172.21.15.7:0/2348694488 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc3d4063b60 con 0x7fc3e40ffdc0 2024-09-21T04:31:54.753 INFO:teuthology.orchestra.run.smithi007.stdout:107374182418 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc3d006e2b0 msgr2=0x7fc3d0070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc3d006e2b0 0x7fc3d0070770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4102a30 tx=0x7fc3cc031400 comp rx=0 tx=0).stop 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 msgr2=0x7fc3e4100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 0x7fc3e4100fa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d6f0 tx=0x7fc3d400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 shutdown_connections 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc3d006e2b0 0x7fc3d0070770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41069a0 0x7fc3e41014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 --2- 172.21.15.7:0/2348694488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3e40ffdc0 0x7fc3e4100fa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 >> 172.21.15.7:0/2348694488 conn(0x7fc3e40fbc90 msgr2=0x7fc3e40fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:54.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 shutdown_connections 2024-09-21T04:31:54.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:54.752+0000 7fc3e984b640 1 -- 172.21.15.7:0/2348694488 wait complete. 2024-09-21T04:31:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:55 smithi007 ceph-mon[27954]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:55 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2348694488' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-21T04:31:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:55 smithi053 ceph-mon[35554]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:55 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2348694488' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-21T04:31:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 -- 172.21.15.7:0/3211178670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4101900 msgr2=0x7f13f4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 --2- 172.21.15.7:0/3211178670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4101900 0x7f13f4101d60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009880 tx=0x7f13e802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 -- 172.21.15.7:0/3211178670 shutdown_connections 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 --2- 172.21.15.7:0/3211178670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4101900 0x7f13f4101d60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 --2- 172.21.15.7:0/3211178670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13f4107900 0x7f13f4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 -- 172.21.15.7:0/3211178670 >> 172.21.15.7:0/3211178670 conn(0x7f13f40fd600 msgr2=0x7f13f40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 -- 172.21.15.7:0/3211178670 shutdown_connections 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 -- 172.21.15.7:0/3211178670 wait complete. 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 Processor -- start 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.945+0000 7f13fb779640 1 -- start start 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13fb779640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13f4101900 0x7f13f419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13fb779640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4107900 0x7f13f419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:55.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13fb779640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13f410ee30 con 0x7f13f4107900 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13fb779640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13f410ecb0 con 0x7f13f4101900 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f8d14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4107900 0x7f13f419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f8d14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4107900 0x7f13f419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45700/0 (socket says 172.21.15.7:45700) 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f8d14640 1 -- 172.21.15.7:0/1127668201 learned_addr learned my addr 172.21.15.7:0/1127668201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f9515640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13f4101900 0x7f13f419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f8d14640 1 -- 172.21.15.7:0/1127668201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13f4101900 msgr2=0x7f13f419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f8d14640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13f4101900 0x7f13f419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f8d14640 1 -- 172.21.15.7:0/1127668201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f4199a50 con 0x7f13f4107900 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f8d14640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4107900 0x7f13f419fdd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f13e8031460 tx=0x7f13e80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f27fc640 1 -- 172.21.15.7:0/1127668201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13e803f070 con 0x7f13f4107900 2024-09-21T04:31:55.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f4199ce0 con 0x7f13f4107900 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.946+0000 7f13f27fc640 1 -- 172.21.15.7:0/1127668201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13e8004cf0 con 0x7f13f4107900 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.947+0000 7f13f27fc640 1 -- 172.21.15.7:0/1127668201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13e8035030 con 0x7f13f4107900 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.947+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f419a210 con 0x7f13f4107900 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.948+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f4103040 con 0x7f13f4107900 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.948+0000 7f13f27fc640 1 -- 172.21.15.7:0/1127668201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f13e8042070 con 0x7f13f4107900 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.949+0000 7f13f27fc640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f13dc06e2b0 0x7f13dc070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.949+0000 7f13f27fc640 1 -- 172.21.15.7:0/1127668201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f13e80ae910 con 0x7f13f4107900 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.949+0000 7f13f9515640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f13dc06e2b0 0x7f13dc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.949+0000 7f13f9515640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f13dc06e2b0 0x7f13dc070770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13f419ab90 tx=0x7f13e40094f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:55.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.957+0000 7f13f27fc640 1 -- 172.21.15.7:0/1127668201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f13e807c5b0 con 0x7f13f4107900 2024-09-21T04:31:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 -- 172.21.15.7:0/1583717892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 msgr2=0x7fd664102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 --2- 172.21.15.7:0/1583717892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 0x7fd664102f30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd654009880 tx=0x7fd65402a530 comp rx=0 tx=0).stop 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 -- 172.21.15.7:0/1583717892 shutdown_connections 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 --2- 172.21.15.7:0/1583717892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd664103470 0x7fd66410b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 --2- 172.21.15.7:0/1583717892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 0x7fd664102f30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 -- 172.21.15.7:0/1583717892 >> 172.21.15.7:0/1583717892 conn(0x7fd6640fc550 msgr2=0x7fd6640fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 -- 172.21.15.7:0/1583717892 shutdown_connections 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 -- 172.21.15.7:0/1583717892 wait complete. 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 Processor -- start 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.829+0000 7fd66a8f1640 1 -- start start 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd66a8f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 0x7fd66419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd66a8f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd664103470 0x7fd66419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd66a8f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410ee40 con 0x7fd664102b50 2024-09-21T04:31:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd66a8f1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410ecc0 con 0x7fd664103470 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd663fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 0x7fd66419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd663fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 0x7fd66419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45688/0 (socket says 172.21.15.7:45688) 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd663fff640 1 -- 172.21.15.7:0/3257876376 learned_addr learned my addr 172.21.15.7:0/3257876376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd663fff640 1 -- 172.21.15.7:0/3257876376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd664103470 msgr2=0x7fd66419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd663fff640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd664103470 0x7fd66419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd663fff640 1 -- 172.21.15.7:0/3257876376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd664197830 con 0x7fd664102b50 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.830+0000 7fd663fff640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 0x7fd66419d6b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd654004f90 tx=0x7fd6540048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.831+0000 7fd6617fa640 1 -- 172.21.15.7:0/3257876376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd65403f070 con 0x7fd664102b50 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.831+0000 7fd6617fa640 1 -- 172.21.15.7:0/3257876376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd654004cf0 con 0x7fd664102b50 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.831+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd664197ac0 con 0x7fd664102b50 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.831+0000 7fd6617fa640 1 -- 172.21.15.7:0/3257876376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd654035030 con 0x7fd664102b50 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.831+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd664197ff0 con 0x7fd664102b50 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.832+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66406a4e0 con 0x7fd664102b50 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.832+0000 7fd6617fa640 1 -- 172.21.15.7:0/3257876376 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fd654042070 con 0x7fd664102b50 2024-09-21T04:31:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.833+0000 7fd6617fa640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd64c06e310 0x7fd64c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.833+0000 7fd6617fa640 1 -- 172.21.15.7:0/3257876376 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fd6540ae860 con 0x7fd664102b50 2024-09-21T04:31:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.833+0000 7fd6637fe640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd64c06e310 0x7fd64c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.833+0000 7fd6637fe640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd64c06e310 0x7fd64c0707d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd664108920 tx=0x7fd658006c90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:55.837+0000 7fd6617fa640 1 -- 172.21.15.7:0/3257876376 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd65407c580 con 0x7fd664102b50 2024-09-21T04:31:56.093 INFO:teuthology.orchestra.run.smithi007.stdout:81604378648 2024-09-21T04:31:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.022+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd664105fb0 con 0x7fd664102b50 2024-09-21T04:31:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.022+0000 7fd6617fa640 1 -- 172.21.15.7:0/3257876376 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd654080230 con 0x7fd664102b50 2024-09-21T04:31:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd64c06e310 msgr2=0x7fd64c0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd64c06e310 0x7fd64c0707d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd664108920 tx=0x7fd658006c90 comp rx=0 tx=0).stop 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 msgr2=0x7fd66419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 0x7fd66419d6b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd654004f90 tx=0x7fd6540048f0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 shutdown_connections 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd64c06e310 0x7fd64c0707d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd664103470 0x7fd66419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 --2- 172.21.15.7:0/3257876376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd664102b50 0x7fd66419d6b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 >> 172.21.15.7:0/3257876376 conn(0x7fd6640fc550 msgr2=0x7fd664108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 shutdown_connections 2024-09-21T04:31:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.024+0000 7fd66a8f1640 1 -- 172.21.15.7:0/3257876376 wait complete. 2024-09-21T04:31:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.141+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f13f419afe0 con 0x7f13f4107900 2024-09-21T04:31:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.142+0000 7f13f27fc640 1 -- 172.21.15.7:0/1127668201 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13e8080260 con 0x7f13f4107900 2024-09-21T04:31:56.145 INFO:teuthology.orchestra.run.smithi007.stdout:60129542174 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f13dc06e2b0 msgr2=0x7f13dc070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f13dc06e2b0 0x7f13dc070770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13f419ab90 tx=0x7f13e40094f0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4107900 msgr2=0x7f13f419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4107900 0x7f13f419fdd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f13e8031460 tx=0x7f13e80048f0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 shutdown_connections 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f13dc06e2b0 0x7f13dc070770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13f4107900 0x7f13f419fdd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 --2- 172.21.15.7:0/1127668201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13f4101900 0x7f13f419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 >> 172.21.15.7:0/1127668201 conn(0x7f13f40fd600 msgr2=0x7f13f4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 shutdown_connections 2024-09-21T04:31:56.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:56.144+0000 7f13fb779640 1 -- 172.21.15.7:0/1127668201 wait complete. 2024-09-21T04:31:56.700 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378647 got 81604378648 for osd.2 2024-09-21T04:31:56.701 DEBUG:teuthology.parallel:result is None 2024-09-21T04:31:57.306 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:57 smithi007 ceph-mon[27954]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:57.306 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:57 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3257876376' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-21T04:31:57.306 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:57 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1127668201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-21T04:31:57.340 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542174 for osd.1 2024-09-21T04:31:57.340 DEBUG:teuthology.parallel:result is None 2024-09-21T04:31:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:31:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:57 smithi053 ceph-mon[35554]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:57 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3257876376' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-21T04:31:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:57 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1127668201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-21T04:31:57.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182417 got 107374182418 for osd.3 2024-09-21T04:31:57.463 DEBUG:teuthology.parallel:result is None 2024-09-21T04:31:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.053+0000 7ff60a864640 1 -- 172.21.15.7:0/1520161082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6041033e0 msgr2=0x7ff60410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.053+0000 7ff60a864640 1 --2- 172.21.15.7:0/1520161082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6041033e0 0x7ff60410b8d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009880 tx=0x7ff5f802a530 comp rx=0 tx=0).stop 2024-09-21T04:31:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.054+0000 7ff60a864640 1 -- 172.21.15.7:0/1520161082 shutdown_connections 2024-09-21T04:31:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.054+0000 7ff60a864640 1 --2- 172.21.15.7:0/1520161082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6041033e0 0x7ff60410b8d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.054+0000 7ff60a864640 1 --2- 172.21.15.7:0/1520161082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff604102ac0 0x7ff604102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.054+0000 7ff60a864640 1 -- 172.21.15.7:0/1520161082 >> 172.21.15.7:0/1520161082 conn(0x7ff6040fc520 msgr2=0x7ff6040fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.054+0000 7ff60a864640 1 -- 172.21.15.7:0/1520161082 shutdown_connections 2024-09-21T04:31:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.054+0000 7ff60a864640 1 -- 172.21.15.7:0/1520161082 wait complete. 2024-09-21T04:31:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.054+0000 7ff60a864640 1 Processor -- start 2024-09-21T04:31:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.054+0000 7ff60a864640 1 -- start start 2024-09-21T04:31:58.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff60a864640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff604102ac0 0x7ff604100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff60a864640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6041033e0 0x7ff604100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff60a864640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410ee50 con 0x7ff604102ac0 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff60a864640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410ecd0 con 0x7ff6041033e0 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff603fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff604102ac0 0x7ff604100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff603fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff604102ac0 0x7ff604100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45718/0 (socket says 172.21.15.7:45718) 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff603fff640 1 -- 172.21.15.7:0/2175614756 learned_addr learned my addr 172.21.15.7:0/2175614756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff6037fe640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6041033e0 0x7ff604100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff603fff640 1 -- 172.21.15.7:0/2175614756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6041033e0 msgr2=0x7ff604100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff603fff640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6041033e0 0x7ff604100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff603fff640 1 -- 172.21.15.7:0/2175614756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6041011a0 con 0x7ff604102ac0 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff603fff640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff604102ac0 0x7ff604100720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b420 tx=0x7ff5f400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff6017fa640 1 -- 172.21.15.7:0/2175614756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff5f4011630 con 0x7ff604102ac0 2024-09-21T04:31:58.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff6017fa640 1 -- 172.21.15.7:0/2175614756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5f4016070 con 0x7ff604102ac0 2024-09-21T04:31:58.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.055+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff604101490 con 0x7ff604102ac0 2024-09-21T04:31:58.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.056+0000 7ff6017fa640 1 -- 172.21.15.7:0/2175614756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff5f4010d40 con 0x7ff604102ac0 2024-09-21T04:31:58.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.056+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6041a3ec0 con 0x7ff604102ac0 2024-09-21T04:31:58.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.057+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60406aa90 con 0x7ff604102ac0 2024-09-21T04:31:58.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.057+0000 7ff6017fa640 1 -- 172.21.15.7:0/2175614756 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7ff5f401b460 con 0x7ff604102ac0 2024-09-21T04:31:58.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.058+0000 7ff6017fa640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff5ec06e310 0x7ff5ec0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:58.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.058+0000 7ff6037fe640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff5ec06e310 0x7ff5ec0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:58.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.058+0000 7ff6017fa640 1 -- 172.21.15.7:0/2175614756 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7ff5f408d610 con 0x7ff604102ac0 2024-09-21T04:31:58.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.058+0000 7ff6037fe640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff5ec06e310 0x7ff5ec0707d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6041021b0 tx=0x7ff5f8031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:58.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.062+0000 7ff6017fa640 1 -- 172.21.15.7:0/2175614756 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff5f405b330 con 0x7ff604102ac0 2024-09-21T04:31:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.238+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff604105f60 con 0x7ff604102ac0 2024-09-21T04:31:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.238+0000 7ff6017fa640 1 -- 172.21.15.7:0/2175614756 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff5f408d430 con 0x7ff604102ac0 2024-09-21T04:31:58.241 INFO:teuthology.orchestra.run.smithi007.stdout:38654705701 2024-09-21T04:31:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.240+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff5ec06e310 msgr2=0x7ff5ec0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.240+0000 7ff60a864640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff5ec06e310 0x7ff5ec0707d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6041021b0 tx=0x7ff5f8031400 comp rx=0 tx=0).stop 2024-09-21T04:31:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.240+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff604102ac0 msgr2=0x7ff604100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.240+0000 7ff60a864640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff604102ac0 0x7ff604100720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b420 tx=0x7ff5f400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.241+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 shutdown_connections 2024-09-21T04:31:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.241+0000 7ff60a864640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff5ec06e310 0x7ff5ec0707d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.241+0000 7ff60a864640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6041033e0 0x7ff604100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.241+0000 7ff60a864640 1 --2- 172.21.15.7:0/2175614756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff604102ac0 0x7ff604100720 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.241+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 >> 172.21.15.7:0/2175614756 conn(0x7ff6040fc520 msgr2=0x7ff604108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.241+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 shutdown_connections 2024-09-21T04:31:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.241+0000 7ff60a864640 1 -- 172.21.15.7:0/2175614756 wait complete. 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.286+0000 7f4b6099d640 1 -- 172.21.15.7:0/1501880348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c1033e0 msgr2=0x7f4b5c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.286+0000 7f4b6099d640 1 --2- 172.21.15.7:0/1501880348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c1033e0 0x7f4b5c10b8d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009880 tx=0x7f4b5002a530 comp rx=0 tx=0).stop 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.286+0000 7f4b6099d640 1 -- 172.21.15.7:0/1501880348 shutdown_connections 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.286+0000 7f4b6099d640 1 --2- 172.21.15.7:0/1501880348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c1033e0 0x7f4b5c10b8d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.286+0000 7f4b6099d640 1 --2- 172.21.15.7:0/1501880348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c102ac0 0x7f4b5c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.286+0000 7f4b6099d640 1 -- 172.21.15.7:0/1501880348 >> 172.21.15.7:0/1501880348 conn(0x7f4b5c0fc520 msgr2=0x7f4b5c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.286+0000 7f4b6099d640 1 -- 172.21.15.7:0/1501880348 shutdown_connections 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.286+0000 7f4b6099d640 1 -- 172.21.15.7:0/1501880348 wait complete. 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b6099d640 1 Processor -- start 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b6099d640 1 -- start start 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b6099d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c102ac0 0x7f4b5c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b6099d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c1033e0 0x7f4b5c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b6099d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10edd0 con 0x7f4b5c102ac0 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b6099d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10ec50 con 0x7f4b5c1033e0 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b5a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c102ac0 0x7f4b5c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b5a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c102ac0 0x7f4b5c19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45736/0 (socket says 172.21.15.7:45736) 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b5a59c640 1 -- 172.21.15.7:0/1584237305 learned_addr learned my addr 172.21.15.7:0/1584237305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b59d9b640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c1033e0 0x7f4b5c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b5a59c640 1 -- 172.21.15.7:0/1584237305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c1033e0 msgr2=0x7f4b5c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.287+0000 7f4b5a59c640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c1033e0 0x7f4b5c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.288+0000 7f4b5a59c640 1 -- 172.21.15.7:0/1584237305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c1977e0 con 0x7f4b5c102ac0 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.288+0000 7f4b59d9b640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c1033e0 0x7f4b5c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.288+0000 7f4b5a59c640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c102ac0 0x7f4b5c19d660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c810 tx=0x7f4b4c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.288+0000 7f4b4b7fe640 1 -- 172.21.15.7:0/1584237305 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4b4c004020 con 0x7f4b5c102ac0 2024-09-21T04:31:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.288+0000 7f4b4b7fe640 1 -- 172.21.15.7:0/1584237305 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b4c0041c0 con 0x7f4b5c102ac0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.288+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c197ad0 con 0x7f4b5c102ac0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.288+0000 7f4b4b7fe640 1 -- 172.21.15.7:0/1584237305 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4b4c0047a0 con 0x7f4b5c102ac0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.288+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c198060 con 0x7f4b5c102ac0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.289+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c06aa90 con 0x7f4b5c102ac0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.289+0000 7f4b4b7fe640 1 -- 172.21.15.7:0/1584237305 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f4b4c0079b0 con 0x7f4b5c102ac0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.290+0000 7f4b4b7fe640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4b4406e2b0 0x7f4b44070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.290+0000 7f4b59d9b640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4b4406e2b0 0x7f4b44070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.290+0000 7f4b4b7fe640 1 -- 172.21.15.7:0/1584237305 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f4b4c08dd30 con 0x7f4b5c102ac0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.291+0000 7f4b59d9b640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4b4406e2b0 0x7f4b44070770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b50031460 tx=0x7f4b50004280 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:31:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.295+0000 7f4b4b7fe640 1 -- 172.21.15.7:0/1584237305 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4b4c014030 con 0x7f4b5c102ac0 2024-09-21T04:31:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.483+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4b5c105f60 con 0x7f4b5c102ac0 2024-09-21T04:31:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.484+0000 7f4b4b7fe640 1 -- 172.21.15.7:0/1584237305 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4b4c05b9d0 con 0x7f4b5c102ac0 2024-09-21T04:31:58.487 INFO:teuthology.orchestra.run.smithi007.stdout:137438953486 2024-09-21T04:31:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4b4406e2b0 msgr2=0x7f4b44070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4b4406e2b0 0x7f4b44070770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b50031460 tx=0x7f4b50004280 comp rx=0 tx=0).stop 2024-09-21T04:31:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c102ac0 msgr2=0x7f4b5c19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:31:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c102ac0 0x7f4b5c19d660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c810 tx=0x7f4b4c00ccd0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 shutdown_connections 2024-09-21T04:31:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f4b4406e2b0 0x7f4b44070770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b5c1033e0 0x7f4b5c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 --2- 172.21.15.7:0/1584237305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b5c102ac0 0x7f4b5c19d660 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:31:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 >> 172.21.15.7:0/1584237305 conn(0x7f4b5c0fc520 msgr2=0x7f4b5c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:31:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 shutdown_connections 2024-09-21T04:31:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:31:58.486+0000 7f4b6099d640 1 -- 172.21.15.7:0/1584237305 wait complete. 2024-09-21T04:31:58.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-09-21T04:31:58.920 DEBUG:teuthology.parallel:result is None 2024-09-21T04:31:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:59 smithi007 ceph-mon[27954]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:59 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2175614756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-21T04:31:59.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:31:59 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1584237305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-21T04:31:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:59 smithi053 ceph-mon[35554]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:31:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:59 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2175614756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-21T04:31:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:31:59 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1584237305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-21T04:31:59.607 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953484 got 137438953486 for osd.4 2024-09-21T04:31:59.608 DEBUG:teuthology.parallel:result is None 2024-09-21T04:31:59.608 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T04:31:59.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T04:31:59.877 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.312+0000 7f847bdc2640 1 -- 172.21.15.7:0/1470077266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 msgr2=0x7f8474107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 --2- 172.21.15.7:0/1470077266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 0x7f8474107ce0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f846c009880 tx=0x7f846c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 -- 172.21.15.7:0/1470077266 shutdown_connections 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 --2- 172.21.15.7:0/1470077266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8474101900 0x7f8474101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 --2- 172.21.15.7:0/1470077266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 0x7f8474107ce0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 -- 172.21.15.7:0/1470077266 >> 172.21.15.7:0/1470077266 conn(0x7f84740fd620 msgr2=0x7f84740ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 -- 172.21.15.7:0/1470077266 shutdown_connections 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 -- 172.21.15.7:0/1470077266 wait complete. 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 Processor -- start 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.313+0000 7f847bdc2640 1 -- start start 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847bdc2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8474101900 0x7f8474195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847bdc2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 0x7f8474195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847bdc2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f847410edb0 con 0x7f8474107900 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847bdc2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f847410ec30 con 0x7f8474101900 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847935d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 0x7f8474195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847935d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 0x7f8474195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45742/0 (socket says 172.21.15.7:45742) 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847935d640 1 -- 172.21.15.7:0/128599875 learned_addr learned my addr 172.21.15.7:0/128599875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f8479b5e640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8474101900 0x7f8474195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847935d640 1 -- 172.21.15.7:0/128599875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8474101900 msgr2=0x7f8474195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847935d640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8474101900 0x7f8474195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847935d640 1 -- 172.21.15.7:0/128599875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8474196000 con 0x7f8474107900 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f847935d640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 0x7f8474195ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f846800c810 tx=0x7f846800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f8472ffd640 1 -- 172.21.15.7:0/128599875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8468004020 con 0x7f8474107900 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.314+0000 7f8472ffd640 1 -- 172.21.15.7:0/128599875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f84680041c0 con 0x7f8474107900 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.315+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84741962f0 con 0x7f8474107900 2024-09-21T04:32:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.315+0000 7f8472ffd640 1 -- 172.21.15.7:0/128599875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f84680047a0 con 0x7f8474107900 2024-09-21T04:32:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.315+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84741a81a0 con 0x7f8474107900 2024-09-21T04:32:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.316+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8474103040 con 0x7f8474107900 2024-09-21T04:32:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.316+0000 7f8472ffd640 1 -- 172.21.15.7:0/128599875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f84680079b0 con 0x7f8474107900 2024-09-21T04:32:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.317+0000 7f8472ffd640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f845c06e310 0x7f845c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.317+0000 7f8472ffd640 1 -- 172.21.15.7:0/128599875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f846808db10 con 0x7f8474107900 2024-09-21T04:32:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.317+0000 7f8479b5e640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f845c06e310 0x7f845c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.317+0000 7f8479b5e640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f845c06e310 0x7f845c0707d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f846c000c00 tx=0x7f846c031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:00.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.321+0000 7f8472ffd640 1 -- 172.21.15.7:0/128599875 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8468014030 con 0x7f8474107900 2024-09-21T04:32:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.492+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84740ff420 con 0x7f845c06e310 2024-09-21T04:32:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.493+0000 7f8472ffd640 1 -- 172.21.15.7:0/128599875 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12321 (secure 0 0 0) 0x7f84740ff420 con 0x7f845c06e310 2024-09-21T04:32:00.496 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-21T04:32:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f845c06e310 msgr2=0x7f845c0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f845c06e310 0x7f845c0707d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f846c000c00 tx=0x7f846c031400 comp rx=0 tx=0).stop 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 msgr2=0x7f8474195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 0x7f8474195ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f846800c810 tx=0x7f846800ccd0 comp rx=0 tx=0).stop 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 shutdown_connections 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f845c06e310 0x7f845c0707d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8474107900 0x7f8474195ac0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 --2- 172.21.15.7:0/128599875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8474101900 0x7f8474195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 >> 172.21.15.7:0/128599875 conn(0x7f84740fd620 msgr2=0x7f84740fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 shutdown_connections 2024-09-21T04:32:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:00.495+0000 7f847bdc2640 1 -- 172.21.15.7:0/128599875 wait complete. 2024-09-21T04:32:01.108 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-21T04:31:59.052910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":182600,"kb_used_data":3780,"kb_used_omap":9,"kb_used_meta":178678,"kb_avail":562288312,"statfs":{"total":575970213888,"available":575783231488,"internally_reserved":0,"allocated":3870720,"data_stored":2567496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9566,"internal_metadata":182966946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001934"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":191,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T04:31:29.955611+0000","last_change":"2024-09-21T04:31:01.761818+0000","last_active":"2024-09-21T04:31:29.955611+0000","last_peered":"2024-09-21T04:31:29.955611+0000","last_clean":"2024-09-21T04:31:29.955611+0000","last_became_active":"2024-09-21T04:31:01.761565+0000","last_became_peered":"2024-09-21T04:31:01.761565+0000","last_unstale":"2024-09-21T04:31:29.955611+0000","last_undegraded":"2024-09-21T04:31:29.955611+0000","last_fullsized":"2024-09-21T04:31:29.955611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:30:11.631732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:30:11.631732+0000","last_clean_scrub_stamp":"2024-09-21T04:30:11.631732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:15.459559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":37,"seq":158913789960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717784,"statfs":{"total":95995035648,"available":95967010816,"internally_reserved":0,"allocated":348160,"data_stored":132732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35688,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709464,"statfs":{"total":95995035648,"available":95958491136,"internally_reserved":0,"allocated":348160,"data_stored":132732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182419,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":723100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378649,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35688,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709464,"statfs":{"total":95995035648,"available":95958491136,"internally_reserved":0,"allocated":348160,"data_stored":132732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":942080,"data_stored":723100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705701,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":723100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:32:01.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T04:32:01.365 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:01 smithi053 ceph-mon[35554]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:01 smithi007 ceph-mon[27954]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.821+0000 7f9481ddd640 1 -- 172.21.15.7:0/3192290513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 msgr2=0x7f947c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.821+0000 7f9481ddd640 1 --2- 172.21.15.7:0/3192290513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 0x7f947c102ea0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f946c009880 tx=0x7f946c02a530 comp rx=0 tx=0).stop 2024-09-21T04:32:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.821+0000 7f9481ddd640 1 -- 172.21.15.7:0/3192290513 shutdown_connections 2024-09-21T04:32:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.821+0000 7f9481ddd640 1 --2- 172.21.15.7:0/3192290513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f947c1033e0 0x7f947c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.821+0000 7f9481ddd640 1 --2- 172.21.15.7:0/3192290513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 0x7f947c102ea0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.821+0000 7f9481ddd640 1 -- 172.21.15.7:0/3192290513 >> 172.21.15.7:0/3192290513 conn(0x7f947c0fc520 msgr2=0x7f947c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.821+0000 7f9481ddd640 1 -- 172.21.15.7:0/3192290513 shutdown_connections 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.821+0000 7f9481ddd640 1 -- 172.21.15.7:0/3192290513 wait complete. 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f9481ddd640 1 Processor -- start 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f9481ddd640 1 -- start start 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f9481ddd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 0x7f947c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f9481ddd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f947c1033e0 0x7f947c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f9481ddd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f947c10ee50 con 0x7f947c102ac0 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f9481ddd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f947c10ecd0 con 0x7f947c1033e0 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f947b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 0x7f947c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f947b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 0x7f947c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45758/0 (socket says 172.21.15.7:45758) 2024-09-21T04:32:01.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f947b7fe640 1 -- 172.21.15.7:0/175838066 learned_addr learned my addr 172.21.15.7:0/175838066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f947affd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f947c1033e0 0x7f947c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f947b7fe640 1 -- 172.21.15.7:0/175838066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f947c1033e0 msgr2=0x7f947c100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.822+0000 7f947b7fe640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f947c1033e0 0x7f947c100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.823+0000 7f947b7fe640 1 -- 172.21.15.7:0/175838066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c1011a0 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.823+0000 7f947affd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f947c1033e0 0x7f947c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.823+0000 7f947b7fe640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 0x7f947c100720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f946c031460 tx=0x7f946c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.823+0000 7f9478ff9640 1 -- 172.21.15.7:0/175838066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f946c03f070 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.823+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947c101430 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.823+0000 7f9478ff9640 1 -- 172.21.15.7:0/175838066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f946c004cf0 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.823+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947c1a3e20 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.823+0000 7f9478ff9640 1 -- 172.21.15.7:0/175838066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f946c035030 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.824+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9468005290 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.825+0000 7f9478ff9640 1 -- 172.21.15.7:0/175838066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f946c042070 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.825+0000 7f9478ff9640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f946406e2b0 0x7f9464070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.825+0000 7f9478ff9640 1 -- 172.21.15.7:0/175838066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f946c0ae860 con 0x7f947c102ac0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.825+0000 7f947affd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f946406e2b0 0x7f9464070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.826+0000 7f947affd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f946406e2b0 0x7f9464070770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9470006fd0 tx=0x7f9470008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:01.829+0000 7f9478ff9640 1 -- 172.21.15.7:0/175838066 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f946c07c500 con 0x7f947c102ac0 2024-09-21T04:32:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.001+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9468000bc0 con 0x7f946406e2b0 2024-09-21T04:32:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.001+0000 7f9478ff9640 1 -- 172.21.15.7:0/175838066 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12321 (secure 0 0 0) 0x7f9468000bc0 con 0x7f946406e2b0 2024-09-21T04:32:02.004 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-21T04:32:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.003+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f946406e2b0 msgr2=0x7f9464070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.003+0000 7f9481ddd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f946406e2b0 0x7f9464070770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9470006fd0 tx=0x7f9470008040 comp rx=0 tx=0).stop 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.003+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 msgr2=0x7f947c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.003+0000 7f9481ddd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 0x7f947c100720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f946c031460 tx=0x7f946c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.004+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 shutdown_connections 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.004+0000 7f9481ddd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f946406e2b0 0x7f9464070770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.004+0000 7f9481ddd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f947c1033e0 0x7f947c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.004+0000 7f9481ddd640 1 --2- 172.21.15.7:0/175838066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f947c102ac0 0x7f947c100720 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.004+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 >> 172.21.15.7:0/175838066 conn(0x7f947c0fc520 msgr2=0x7f947c10af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.004+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 shutdown_connections 2024-09-21T04:32:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:02.004+0000 7f9481ddd640 1 -- 172.21.15.7:0/175838066 wait complete. 2024-09-21T04:32:02.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:02 smithi053 ceph-mon[35554]: from='client.14518 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:32:02.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:02 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:32:02.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:02 smithi007 ceph-mon[27954]: from='client.14518 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:32:02.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:02 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:32:02.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-21T04:32:01.053281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":182600,"kb_used_data":3780,"kb_used_omap":9,"kb_used_meta":178678,"kb_avail":562288312,"statfs":{"total":575970213888,"available":575783231488,"internally_reserved":0,"allocated":3870720,"data_stored":2567496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9566,"internal_metadata":182966946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001999"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":191,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-21T04:31:29.955611+0000","last_change":"2024-09-21T04:31:01.761818+0000","last_active":"2024-09-21T04:31:29.955611+0000","last_peered":"2024-09-21T04:31:29.955611+0000","last_clean":"2024-09-21T04:31:29.955611+0000","last_became_active":"2024-09-21T04:31:01.761565+0000","last_became_peered":"2024-09-21T04:31:01.761565+0000","last_unstale":"2024-09-21T04:31:29.955611+0000","last_undegraded":"2024-09-21T04:31:29.955611+0000","last_fullsized":"2024-09-21T04:31:29.955611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:30:11.631732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:30:11.631732+0000","last_clean_scrub_stamp":"2024-09-21T04:30:11.631732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:15.459559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":37,"seq":158913789961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717784,"statfs":{"total":95995035648,"available":95967010816,"internally_reserved":0,"allocated":348160,"data_stored":132732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35688,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709464,"statfs":{"total":95995035648,"available":95958491136,"internally_reserved":0,"allocated":348160,"data_stored":132732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182420,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":942080,"data_stored":723100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378650,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35688,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709464,"statfs":{"total":95995035648,"available":95958491136,"internally_reserved":0,"allocated":348160,"data_stored":132732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":942080,"data_stored":723100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":723100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:32:02.748 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T04:32:02.748 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T04:32:02.748 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T04:32:02.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T04:32:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:03.112 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:03 smithi007 ceph-mon[27954]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:03.113 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:03 smithi007 ceph-mon[27954]: from='client.14522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.426+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/731551498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 msgr2=0x7f6b14107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.426+0000 7f6b1a7c5640 1 --2- 172.21.15.7:0/731551498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 0x7f6b14107ce0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009880 tx=0x7f6b0402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.426+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/731551498 shutdown_connections 2024-09-21T04:32:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.426+0000 7f6b1a7c5640 1 --2- 172.21.15.7:0/731551498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14101900 0x7f6b14101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.426+0000 7f6b1a7c5640 1 --2- 172.21.15.7:0/731551498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 0x7f6b14107ce0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.426+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/731551498 >> 172.21.15.7:0/731551498 conn(0x7f6b140fd5e0 msgr2=0x7f6b140ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.426+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/731551498 shutdown_connections 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.426+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/731551498 wait complete. 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.427+0000 7f6b1a7c5640 1 Processor -- start 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.427+0000 7f6b1a7c5640 1 -- start start 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.427+0000 7f6b1a7c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14101900 0x7f6b1419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.427+0000 7f6b1a7c5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 0x7f6b1419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.427+0000 7f6b1a7c5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410edb0 con 0x7f6b14107900 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.427+0000 7f6b1a7c5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410ec30 con 0x7f6b14101900 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.427+0000 7f6b137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 0x7f6b1419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 0x7f6b1419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45776/0 (socket says 172.21.15.7:45776) 2024-09-21T04:32:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b137fe640 1 -- 172.21.15.7:0/1273337304 learned_addr learned my addr 172.21.15.7:0/1273337304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b13fff640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14101900 0x7f6b1419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b137fe640 1 -- 172.21.15.7:0/1273337304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14101900 msgr2=0x7f6b1419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b137fe640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14101900 0x7f6b1419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b137fe640 1 -- 172.21.15.7:0/1273337304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b141977a0 con 0x7f6b14107900 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b137fe640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 0x7f6b1419db60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c810 tx=0x7f6b0800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b117fa640 1 -- 172.21.15.7:0/1273337304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6b08004020 con 0x7f6b14107900 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b14197a90 con 0x7f6b14107900 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b117fa640 1 -- 172.21.15.7:0/1273337304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b080041c0 con 0x7f6b14107900 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b117fa640 1 -- 172.21.15.7:0/1273337304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6b080047a0 con 0x7f6b14107900 2024-09-21T04:32:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.428+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b14198020 con 0x7f6b14107900 2024-09-21T04:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.430+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14103040 con 0x7f6b14107900 2024-09-21T04:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.431+0000 7f6b117fa640 1 -- 172.21.15.7:0/1273337304 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f6b080079b0 con 0x7f6b14107900 2024-09-21T04:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.431+0000 7f6b117fa640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f6afc06e2b0 0x7f6afc070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.431+0000 7f6b117fa640 1 -- 172.21.15.7:0/1273337304 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f6b0808dcc0 con 0x7f6b14107900 2024-09-21T04:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.431+0000 7f6b13fff640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f6afc06e2b0 0x7f6afc070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.432+0000 7f6b13fff640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f6afc06e2b0 0x7f6afc070770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b04000c00 tx=0x7f6b04031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.436+0000 7f6b117fa640 1 -- 172.21.15.7:0/1273337304 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b08014030 con 0x7f6b14107900 2024-09-21T04:32:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:03 smithi053 ceph-mon[35554]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:03 smithi053 ceph-mon[35554]: from='client.14522 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:32:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.648+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6b14104b10 con 0x7f6b14107900 2024-09-21T04:32:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.648+0000 7f6b117fa640 1 -- 172.21.15.7:0/1273337304 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6b08055e20 con 0x7f6b14107900 2024-09-21T04:32:03.651 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:03.651 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T04:32:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f6afc06e2b0 msgr2=0x7f6afc070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f6afc06e2b0 0x7f6afc070770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b04000c00 tx=0x7f6b04031400 comp rx=0 tx=0).stop 2024-09-21T04:32:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 msgr2=0x7f6b1419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 0x7f6b1419db60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c810 tx=0x7f6b0800ccd0 comp rx=0 tx=0).stop 2024-09-21T04:32:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 shutdown_connections 2024-09-21T04:32:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f6afc06e2b0 0x7f6afc070770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b14107900 0x7f6b1419db60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 --2- 172.21.15.7:0/1273337304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14101900 0x7f6b1419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 >> 172.21.15.7:0/1273337304 conn(0x7f6b140fd5e0 msgr2=0x7f6b140feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 shutdown_connections 2024-09-21T04:32:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:03.650+0000 7f6b1a7c5640 1 -- 172.21.15.7:0/1273337304 wait complete. 2024-09-21T04:32:04.336 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:04 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1273337304' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-21T04:32:04.372 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T04:32:04.372 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T04:32:04.372 INFO:teuthology.run_tasks:Running task print... 2024-09-21T04:32:04.381 INFO:teuthology.task.print:**** done end installing squid cephadm ... 2024-09-21T04:32:04.381 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:32:04.394 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:32:04.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-09-21T04:32:04.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:04 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1273337304' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-21T04:32:04.662 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:05.047 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:05 smithi007 ceph-mon[27954]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 -- 172.21.15.7:0/3614381536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 msgr2=0x7fb514107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 --2- 172.21.15.7:0/3614381536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 0x7fb514107170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb508009880 tx=0x7fb50802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 -- 172.21.15.7:0/3614381536 shutdown_connections 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 --2- 172.21.15.7:0/3614381536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 0x7fb514107170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 --2- 172.21.15.7:0/3614381536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb514102450 0x7fb514104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 -- 172.21.15.7:0/3614381536 >> 172.21.15.7:0/3614381536 conn(0x7fb5140fc010 msgr2=0x7fb5140fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 -- 172.21.15.7:0/3614381536 shutdown_connections 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 -- 172.21.15.7:0/3614381536 wait complete. 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 Processor -- start 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.132+0000 7fb51a1cb640 1 -- start start 2024-09-21T04:32:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb51a1cb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb514102450 0x7fb5141974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb51a1cb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 0x7fb514197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb51a1cb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb51410a740 con 0x7fb514104d80 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb51a1cb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb51410a5c0 con 0x7fb514102450 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb512ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 0x7fb514197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb512ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 0x7fb514197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38738/0 (socket says 172.21.15.7:38738) 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb512ffd640 1 -- 172.21.15.7:0/2099799257 learned_addr learned my addr 172.21.15.7:0/2099799257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb5137fe640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb514102450 0x7fb5141974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb512ffd640 1 -- 172.21.15.7:0/2099799257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb514102450 msgr2=0x7fb5141974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb512ffd640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb514102450 0x7fb5141974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb512ffd640 1 -- 172.21.15.7:0/2099799257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb514198080 con 0x7fb514104d80 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb512ffd640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 0x7fb514197a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb508006010 tx=0x7fb5080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb510ff9640 1 -- 172.21.15.7:0/2099799257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb50803f070 con 0x7fb514104d80 2024-09-21T04:32:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb510ff9640 1 -- 172.21.15.7:0/2099799257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb508004a50 con 0x7fb514104d80 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51419cac0 con 0x7fb514104d80 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.133+0000 7fb510ff9640 1 -- 172.21.15.7:0/2099799257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb508035030 con 0x7fb514104d80 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.134+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51419d050 con 0x7fb514104d80 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.134+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb514102830 con 0x7fb514104d80 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.135+0000 7fb510ff9640 1 -- 172.21.15.7:0/2099799257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fb508042070 con 0x7fb514104d80 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.135+0000 7fb510ff9640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb4fc06e310 0x7fb4fc0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.135+0000 7fb510ff9640 1 -- 172.21.15.7:0/2099799257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fb5080ae850 con 0x7fb514104d80 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.135+0000 7fb5137fe640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb4fc06e310 0x7fb4fc0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.136+0000 7fb5137fe640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb4fc06e310 0x7fb4fc0707d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb504005fd0 tx=0x7fb50400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.140+0000 7fb510ff9640 1 -- 172.21.15.7:0/2099799257 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb508080220 con 0x7fb514104d80 2024-09-21T04:32:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.315+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) -- 0x7fb51419d960 con 0x7fb514104d80 2024-09-21T04:32:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.319+0000 7fb510ff9640 1 -- 172.21.15.7:0/2099799257 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v40)=0 v40) ==== 143+0+0 (secure 0 0 0) 0x7fb508084a50 con 0x7fb514104d80 2024-09-21T04:32:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.321+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb4fc06e310 msgr2=0x7fb4fc0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.321+0000 7fb51a1cb640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb4fc06e310 0x7fb4fc0707d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb504005fd0 tx=0x7fb50400b390 comp rx=0 tx=0).stop 2024-09-21T04:32:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.321+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 msgr2=0x7fb514197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.321+0000 7fb51a1cb640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 0x7fb514197a00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb508006010 tx=0x7fb5080045e0 comp rx=0 tx=0).stop 2024-09-21T04:32:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.322+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 shutdown_connections 2024-09-21T04:32:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.322+0000 7fb51a1cb640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb4fc06e310 0x7fb4fc0707d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.322+0000 7fb51a1cb640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb514104d80 0x7fb514197a00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.322+0000 7fb51a1cb640 1 --2- 172.21.15.7:0/2099799257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb514102450 0x7fb5141974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.322+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 >> 172.21.15.7:0/2099799257 conn(0x7fb5140fc010 msgr2=0x7fb5140fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.322+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 shutdown_connections 2024-09-21T04:32:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:05.322+0000 7fb51a1cb640 1 -- 172.21.15.7:0/2099799257 wait complete. 2024-09-21T04:32:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:05 smithi053 ceph-mon[35554]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:06.006 INFO:teuthology.run_tasks:Running task print... 2024-09-21T04:32:06.018 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-09-21T04:32:06.018 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:32:06.038 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:32:06.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T04:32:06.303 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:06.574 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:06 smithi007 ceph-mon[27954]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:06.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:06 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2099799257' entity='client.admin' 2024-09-21T04:32:06.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:06 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:32:06.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:06 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:06.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:06 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:32:06.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:06 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:06.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:06 smithi053 ceph-mon[35554]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:06.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:06 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2099799257' entity='client.admin' 2024-09-21T04:32:06.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:06 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:32:06.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:06 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:06.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:06 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:32:06.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:06 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.744+0000 7f00d0dd4640 1 -- 172.21.15.7:0/2707978104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc104d80 msgr2=0x7f00cc107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.744+0000 7f00d0dd4640 1 --2- 172.21.15.7:0/2707978104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc104d80 0x7f00cc107170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009880 tx=0x7f00b402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.744+0000 7f00d0dd4640 1 -- 172.21.15.7:0/2707978104 shutdown_connections 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.744+0000 7f00d0dd4640 1 --2- 172.21.15.7:0/2707978104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc104d80 0x7f00cc107170 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.744+0000 7f00d0dd4640 1 --2- 172.21.15.7:0/2707978104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00cc102450 0x7f00cc104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.744+0000 7f00d0dd4640 1 -- 172.21.15.7:0/2707978104 >> 172.21.15.7:0/2707978104 conn(0x7f00cc0fc010 msgr2=0x7f00cc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.744+0000 7f00d0dd4640 1 -- 172.21.15.7:0/2707978104 shutdown_connections 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.744+0000 7f00d0dd4640 1 -- 172.21.15.7:0/2707978104 wait complete. 2024-09-21T04:32:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00d0dd4640 1 Processor -- start 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00d0dd4640 1 -- start start 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00d0dd4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc102450 0x7f00cc101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00d0dd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00cc104d80 0x7f00cc100400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00d0dd4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc10a720 con 0x7f00cc102450 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00d0dd4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc10a5a0 con 0x7f00cc104d80 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc102450 0x7f00cc101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc102450 0x7f00cc101d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38752/0 (socket says 172.21.15.7:38752) 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.745+0000 7f00ca59c640 1 -- 172.21.15.7:0/1968057259 learned_addr learned my addr 172.21.15.7:0/1968057259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00c3fff640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00cc104d80 0x7f00cc100400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00ca59c640 1 -- 172.21.15.7:0/1968057259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00cc104d80 msgr2=0x7f00cc100400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00ca59c640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00cc104d80 0x7f00cc100400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00ca59c640 1 -- 172.21.15.7:0/1968057259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc100940 con 0x7f00cc102450 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00ca59c640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc102450 0x7f00cc101d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c8a0 tx=0x7f00bc00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00c37fe640 1 -- 172.21.15.7:0/1968057259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00bc0079b0 con 0x7f00cc102450 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00c37fe640 1 -- 172.21.15.7:0/1968057259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00bc007b50 con 0x7f00cc102450 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc100c30 con 0x7f00cc102450 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00c37fe640 1 -- 172.21.15.7:0/1968057259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00bc018700 con 0x7f00cc102450 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.746+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc101190 con 0x7f00cc102450 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.747+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00cc102830 con 0x7f00cc102450 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.748+0000 7f00c37fe640 1 -- 172.21.15.7:0/1968057259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f00bc007d00 con 0x7f00cc102450 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.748+0000 7f00c37fe640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f00b806e310 0x7f00b80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.748+0000 7f00c37fe640 1 -- 172.21.15.7:0/1968057259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f00bc08e020 con 0x7f00cc102450 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.748+0000 7f00c3fff640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f00b806e310 0x7f00b80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.749+0000 7f00c3fff640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f00b806e310 0x7f00b80707d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00b4000c00 tx=0x7f00b4005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.752+0000 7f00c37fe640 1 -- 172.21.15.7:0/1968057259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f00bc014030 con 0x7f00cc102450 2024-09-21T04:32:06.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.945+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f00cc19f510 con 0x7f00b806e310 2024-09-21T04:32:06.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.946+0000 7f00c37fe640 1 -- 172.21.15.7:0/1968057259 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f00cc19f510 con 0x7f00b806e310 2024-09-21T04:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-09-21T04:32:06.949 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-09-21T04:32:06.949 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-09-21T04:32:06.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f00b806e310 msgr2=0x7f00b80707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f00b806e310 0x7f00b80707d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00b4000c00 tx=0x7f00b4005a20 comp rx=0 tx=0).stop 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc102450 msgr2=0x7f00cc101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc102450 0x7f00cc101d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00c8a0 tx=0x7f00bc00cd60 comp rx=0 tx=0).stop 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 shutdown_connections 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f00b806e310 0x7f00b80707d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00cc104d80 0x7f00cc100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 --2- 172.21.15.7:0/1968057259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc102450 0x7f00cc101d40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 >> 172.21.15.7:0/1968057259 conn(0x7f00cc0fc010 msgr2=0x7f00cc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 shutdown_connections 2024-09-21T04:32:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:06.948+0000 7f00d0dd4640 1 -- 172.21.15.7:0/1968057259 wait complete. 2024-09-21T04:32:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:07 smithi053 ceph-mon[35554]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:07.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T04:32:07.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:07 smithi007 ceph-mon[27954]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:07.814 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:08.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.229+0000 7f642ba69640 1 -- 172.21.15.7:0/1188060649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 msgr2=0x7f6424100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.229+0000 7f642ba69640 1 --2- 172.21.15.7:0/1188060649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 0x7f6424100f80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6414009880 tx=0x7f641402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 -- 172.21.15.7:0/1188060649 shutdown_connections 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 --2- 172.21.15.7:0/1188060649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 0x7f6424100f80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 --2- 172.21.15.7:0/1188060649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241001e0 0x7f64241005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 -- 172.21.15.7:0/1188060649 >> 172.21.15.7:0/1188060649 conn(0x7f64240fc010 msgr2=0x7f64240fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 -- 172.21.15.7:0/1188060649 shutdown_connections 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 -- 172.21.15.7:0/1188060649 wait complete. 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 Processor -- start 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 -- start start 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241001e0 0x7f64241996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 0x7f6424199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f642410c990 con 0x7f6424100b20 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f642ba69640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f642410c810 con 0x7f64241001e0 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.230+0000 7f6429004640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 0x7f6424199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f6429004640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 0x7f6424199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38770/0 (socket says 172.21.15.7:38770) 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f6429004640 1 -- 172.21.15.7:0/2978769253 learned_addr learned my addr 172.21.15.7:0/2978769253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f6429805640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241001e0 0x7f64241996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f6429004640 1 -- 172.21.15.7:0/2978769253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241001e0 msgr2=0x7f64241996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f6429004640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241001e0 0x7f64241996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f6429004640 1 -- 172.21.15.7:0/2978769253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642419a290 con 0x7f6424100b20 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f6429004640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 0x7f6424199c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6414006010 tx=0x7f64140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f641affd640 1 -- 172.21.15.7:0/2978769253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f641403f070 con 0x7f6424100b20 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642419ec70 con 0x7f6424100b20 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f641affd640 1 -- 172.21.15.7:0/2978769253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6414004ac0 con 0x7f6424100b20 2024-09-21T04:32:08.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f641affd640 1 -- 172.21.15.7:0/2978769253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6414035030 con 0x7f6424100b20 2024-09-21T04:32:08.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.231+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642419f1b0 con 0x7f6424100b20 2024-09-21T04:32:08.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.232+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64241005e0 con 0x7f6424100b20 2024-09-21T04:32:08.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.233+0000 7f641affd640 1 -- 172.21.15.7:0/2978769253 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f6414042070 con 0x7f6424100b20 2024-09-21T04:32:08.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.233+0000 7f641affd640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f641006e2b0 0x7f6410070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:08.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.233+0000 7f641affd640 1 -- 172.21.15.7:0/2978769253 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f64140ae9e0 con 0x7f6424100b20 2024-09-21T04:32:08.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.233+0000 7f6429805640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f641006e2b0 0x7f6410070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:08.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.234+0000 7f6429805640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f641006e2b0 0x7f6410070770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:08.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.238+0000 7f641affd640 1 -- 172.21.15.7:0/2978769253 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6414080330 con 0x7f6424100b20 2024-09-21T04:32:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.421+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6424062630 con 0x7f641006e2b0 2024-09-21T04:32:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.430+0000 7f641affd640 1 -- 172.21.15.7:0/2978769253 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3120 (secure 0 0 0) 0x7f6424062630 con 0x7f641006e2b0 2024-09-21T04:32:08.433 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:32:08.433 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (3m) 112s ago 5m 25.9M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:32:08.433 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (5m) 112s ago 5m 7830k - 19.1.1-373-g4e45e6b4 164465d2334a f80c59780c91 2024-09-21T04:32:08.433 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (4m) 43s ago 4m 5268k - 19.1.1-373-g4e45e6b4 164465d2334a 828ddad2141a 2024-09-21T04:32:08.433 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (5m) 112s ago 5m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (4m) 43s ago 4m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (3m) 112s ago 4m 72.9M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:9283,8765,8443 running (6m) 112s ago 6m 489M - 19.1.1-373-g4e45e6b4 164465d2334a c324ee791e08 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (4m) 43s ago 4m 441M - 19.1.1-373-g4e45e6b4 164465d2334a e9db9486fd2f 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (6m) 112s ago 6m 46.0M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (4m) 43s ago 4m 43.5M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (5m) 112s ago 5m 16.1M - 1.5.0 0da6a335fe13 b4a8255a594a 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (4m) 43s ago 4m 16.6M - 1.5.0 0da6a335fe13 88246b15ca5d 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (3m) 112s ago 3m 40.0M 3948M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (2m) 112s ago 2m 39.9M 3948M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (2m) 112s ago 2m 44.6M 3948M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (102s) 43s ago 102s 39.9M 4972M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (75s) 43s ago 75s 44.5M 4972M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:32:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (48s) 43s ago 48s 11.2M 4972M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:32:08.435 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (3m) 112s ago 4m 42.9M - 2.43.0 a07b618ecd1d 74a720908c7f 2024-09-21T04:32:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f641006e2b0 msgr2=0x7f6410070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f641006e2b0 0x7f6410070770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f642000b390 comp rx=0 tx=0).stop 2024-09-21T04:32:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 msgr2=0x7f6424199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 0x7f6424199c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6414006010 tx=0x7f64140046c0 comp rx=0 tx=0).stop 2024-09-21T04:32:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 shutdown_connections 2024-09-21T04:32:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f641006e2b0 0x7f6410070770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424100b20 0x7f6424199c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 --2- 172.21.15.7:0/2978769253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64241001e0 0x7f64241996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 >> 172.21.15.7:0/2978769253 conn(0x7f64240fc010 msgr2=0x7f6424108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 shutdown_connections 2024-09-21T04:32:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:08.432+0000 7f642ba69640 1 -- 172.21.15.7:0/2978769253 wait complete. 2024-09-21T04:32:08.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:08 smithi053 ceph-mon[35554]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:08.728 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:08 smithi007 ceph-mon[27954]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:09.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T04:32:09.400 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:09 smithi007 ceph-mon[27954]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:09.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:09 smithi053 ceph-mon[35554]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.834+0000 7fb15afe7640 1 -- 172.21.15.7:0/2193235239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb154104d20 msgr2=0x7fb154107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.834+0000 7fb15afe7640 1 --2- 172.21.15.7:0/2193235239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb154104d20 0x7fb154107110 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0098b0 tx=0x7fb13c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:32:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.834+0000 7fb15afe7640 1 -- 172.21.15.7:0/2193235239 shutdown_connections 2024-09-21T04:32:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.834+0000 7fb15afe7640 1 --2- 172.21.15.7:0/2193235239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb154104d20 0x7fb154107110 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.834+0000 7fb15afe7640 1 --2- 172.21.15.7:0/2193235239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1541023f0 0x7fb1541047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.834+0000 7fb15afe7640 1 -- 172.21.15.7:0/2193235239 >> 172.21.15.7:0/2193235239 conn(0x7fb1540fc010 msgr2=0x7fb1540fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.834+0000 7fb15afe7640 1 -- 172.21.15.7:0/2193235239 shutdown_connections 2024-09-21T04:32:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.834+0000 7fb15afe7640 1 -- 172.21.15.7:0/2193235239 wait complete. 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb15afe7640 1 Processor -- start 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb15afe7640 1 -- start start 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb15afe7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1541023f0 0x7fb154197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb15afe7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb154104d20 0x7fb154197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb15afe7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb15410a810 con 0x7fb1541023f0 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb15afe7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb15410a690 con 0x7fb154104d20 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb158d83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1541023f0 0x7fb154197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb158d83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1541023f0 0x7fb154197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38788/0 (socket says 172.21.15.7:38788) 2024-09-21T04:32:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.835+0000 7fb158d83640 1 -- 172.21.15.7:0/171832785 learned_addr learned my addr 172.21.15.7:0/171832785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb153fff640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb154104d20 0x7fb154197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb158d83640 1 -- 172.21.15.7:0/171832785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb154104d20 msgr2=0x7fb154197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb158d83640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb154104d20 0x7fb154197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb158d83640 1 -- 172.21.15.7:0/171832785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb154198150 con 0x7fb1541023f0 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb158d83640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1541023f0 0x7fb154197590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00b420 tx=0x7fb14c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb151ffb640 1 -- 172.21.15.7:0/171832785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb14c011630 con 0x7fb1541023f0 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb151ffb640 1 -- 172.21.15.7:0/171832785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb14c016070 con 0x7fb1541023f0 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15419cb90 con 0x7fb1541023f0 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb151ffb640 1 -- 172.21.15.7:0/171832785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb14c010d40 con 0x7fb1541023f0 2024-09-21T04:32:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.836+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb15419d0f0 con 0x7fb1541023f0 2024-09-21T04:32:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.837+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb15410b020 con 0x7fb1541023f0 2024-09-21T04:32:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.838+0000 7fb151ffb640 1 -- 172.21.15.7:0/171832785 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fb14c01b460 con 0x7fb1541023f0 2024-09-21T04:32:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.838+0000 7fb151ffb640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb14006e2b0 0x7fb140070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.838+0000 7fb151ffb640 1 -- 172.21.15.7:0/171832785 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fb14c08d7d0 con 0x7fb1541023f0 2024-09-21T04:32:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.838+0000 7fb153fff640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb14006e2b0 0x7fb140070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.839+0000 7fb153fff640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb14006e2b0 0x7fb140070770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb13c006010 tx=0x7fb13c009580 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:09.844+0000 7fb151ffb640 1 -- 172.21.15.7:0/171832785 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb15410b020 con 0x7fb1541023f0 2024-09-21T04:32:10.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.027+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb154105780 con 0x7fb14006e2b0 2024-09-21T04:32:10.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.031+0000 7fb151ffb640 1 -- 172.21.15.7:0/171832785 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1280 (secure 0 0 0) 0x7fb154105780 con 0x7fb14006e2b0 2024-09-21T04:32:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T04:32:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager ?:9093,9094 1/1 114s ago 5m count:1 2024-09-21T04:32:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter 2/2 114s ago 5m * 2024-09-21T04:32:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:crash 2/2 114s ago 5m * 2024-09-21T04:32:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:grafana ?:3000 1/1 114s ago 5m count:1 2024-09-21T04:32:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 2/2 114s ago 5m count:2 2024-09-21T04:32:10.035 INFO:teuthology.orchestra.run.smithi007.stdout:mon 2/2 114s ago 5m smithi007:172.21.15.7=smithi007;smithi053:172.21.15.53=smithi053;count:2 2024-09-21T04:32:10.035 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter ?:9100 2/2 114s ago 5m * 2024-09-21T04:32:10.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd 6 114s ago - 2024-09-21T04:32:10.035 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus ?:9095 1/1 114s ago 5m count:1 2024-09-21T04:32:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb14006e2b0 msgr2=0x7fb140070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb14006e2b0 0x7fb140070770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb13c006010 tx=0x7fb13c009580 comp rx=0 tx=0).stop 2024-09-21T04:32:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1541023f0 msgr2=0x7fb154197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1541023f0 0x7fb154197590 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00b420 tx=0x7fb14c00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:32:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 shutdown_connections 2024-09-21T04:32:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb14006e2b0 0x7fb140070770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb154104d20 0x7fb154197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 --2- 172.21.15.7:0/171832785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1541023f0 0x7fb154197590 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.033+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 >> 172.21.15.7:0/171832785 conn(0x7fb1540fc010 msgr2=0x7fb1540fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.034+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 shutdown_connections 2024-09-21T04:32:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:10.034+0000 7fb15afe7640 1 -- 172.21.15.7:0/171832785 wait complete. 2024-09-21T04:32:10.648 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:10 smithi007 ceph-mon[27954]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:10.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T04:32:10.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:10 smithi053 ceph-mon[35554]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.386+0000 7fbae3993640 1 -- 172.21.15.7:0/3171463530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc102b60 msgr2=0x7fbadc102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.386+0000 7fbae3993640 1 --2- 172.21.15.7:0/3171463530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc102b60 0x7fbadc102fe0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009880 tx=0x7fbacc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.386+0000 7fbae3993640 1 -- 172.21.15.7:0/3171463530 shutdown_connections 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.386+0000 7fbae3993640 1 --2- 172.21.15.7:0/3171463530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc102b60 0x7fbadc102fe0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.386+0000 7fbae3993640 1 --2- 172.21.15.7:0/3171463530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbadc101960 0x7fbadc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.386+0000 7fbae3993640 1 -- 172.21.15.7:0/3171463530 >> 172.21.15.7:0/3171463530 conn(0x7fbadc0fd0f0 msgr2=0x7fbadc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.386+0000 7fbae3993640 1 -- 172.21.15.7:0/3171463530 shutdown_connections 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.386+0000 7fbae3993640 1 -- 172.21.15.7:0/3171463530 wait complete. 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae3993640 1 Processor -- start 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae3993640 1 -- start start 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae3993640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc101960 0x7fbadc197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae3993640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbadc102b60 0x7fbadc1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae3993640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc10a780 con 0x7fbadc101960 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae3993640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbadc10a600 con 0x7fbadc102b60 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae172f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc101960 0x7fbadc197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae172f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc101960 0x7fbadc197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38808/0 (socket says 172.21.15.7:38808) 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae172f640 1 -- 172.21.15.7:0/3993018653 learned_addr learned my addr 172.21.15.7:0/3993018653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae172f640 1 -- 172.21.15.7:0/3993018653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbadc102b60 msgr2=0x7fbadc1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae172f640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbadc102b60 0x7fbadc1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.387+0000 7fbae172f640 1 -- 172.21.15.7:0/3993018653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbadc198030 con 0x7fbadc101960 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.388+0000 7fbae172f640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc101960 0x7fbadc197470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c8f0 tx=0x7fbad800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.388+0000 7fbad27fc640 1 -- 172.21.15.7:0/3993018653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbad80079b0 con 0x7fbadc101960 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.388+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbadc19ca70 con 0x7fbadc101960 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.388+0000 7fbad27fc640 1 -- 172.21.15.7:0/3993018653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbad8007b50 con 0x7fbadc101960 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.388+0000 7fbad27fc640 1 -- 172.21.15.7:0/3993018653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbad8018610 con 0x7fbadc101960 2024-09-21T04:32:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.388+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbadc19d000 con 0x7fbadc101960 2024-09-21T04:32:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.389+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbadc101d60 con 0x7fbadc101960 2024-09-21T04:32:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.389+0000 7fbad27fc640 1 -- 172.21.15.7:0/3993018653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fbad8007d00 con 0x7fbadc101960 2024-09-21T04:32:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.390+0000 7fbad27fc640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbac806e2b0 0x7fbac8070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.390+0000 7fbad27fc640 1 -- 172.21.15.7:0/3993018653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fbad808e120 con 0x7fbadc101960 2024-09-21T04:32:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.390+0000 7fbae0f2e640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbac806e2b0 0x7fbac8070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.390+0000 7fbae0f2e640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbac806e2b0 0x7fbac8070770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0096c0 tx=0x7fbacc000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.394+0000 7fbad27fc640 1 -- 172.21.15.7:0/3993018653 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbad8014030 con 0x7fbadc101960 2024-09-21T04:32:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.575+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fbadc0626f0 con 0x7fbac806e2b0 2024-09-21T04:32:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.576+0000 7fbad27fc640 1 -- 172.21.15.7:0/3993018653 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+145 (secure 0 0 0) 0x7fbadc0626f0 con 0x7fbac806e2b0 2024-09-21T04:32:11.579 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-09-21T04:32:11.579 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-09-21T04:32:11.579 INFO:teuthology.orchestra.run.smithi007.stdout:smithi053 172.21.15.53 2024-09-21T04:32:11.579 INFO:teuthology.orchestra.run.smithi007.stdout:2 hosts in cluster 2024-09-21T04:32:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbac806e2b0 msgr2=0x7fbac8070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbac806e2b0 0x7fbac8070770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0096c0 tx=0x7fbacc000bc0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc101960 msgr2=0x7fbadc197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc101960 0x7fbadc197470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c8f0 tx=0x7fbad800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 shutdown_connections 2024-09-21T04:32:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fbac806e2b0 0x7fbac8070770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbadc102b60 0x7fbadc1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 --2- 172.21.15.7:0/3993018653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbadc101960 0x7fbadc197470 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 >> 172.21.15.7:0/3993018653 conn(0x7fbadc0fd0f0 msgr2=0x7fbadc105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 shutdown_connections 2024-09-21T04:32:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:11.578+0000 7fbae3993640 1 -- 172.21.15.7:0/3993018653 wait complete. 2024-09-21T04:32:11.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:11 smithi007 ceph-mon[27954]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:11.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:11 smithi053 ceph-mon[35554]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:12.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T04:32:12.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:12 smithi053 ceph-mon[35554]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:12.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:12 smithi053 ceph-mon[35554]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:12.586 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:12.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:12 smithi007 ceph-mon[27954]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:12.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:12 smithi007 ceph-mon[27954]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.038+0000 7fd106863640 1 -- 172.21.15.7:0/3332871538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 msgr2=0x7fd100103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.038+0000 7fd106863640 1 --2- 172.21.15.7:0/3332871538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 0x7fd100103340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009880 tx=0x7fd0f002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.038+0000 7fd106863640 1 -- 172.21.15.7:0/3332871538 shutdown_connections 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.038+0000 7fd106863640 1 --2- 172.21.15.7:0/3332871538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd100103910 0x7fd1000ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.038+0000 7fd106863640 1 --2- 172.21.15.7:0/3332871538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 0x7fd100103340 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.038+0000 7fd106863640 1 -- 172.21.15.7:0/3332871538 >> 172.21.15.7:0/3332871538 conn(0x7fd1000fb770 msgr2=0x7fd1000fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.038+0000 7fd106863640 1 -- 172.21.15.7:0/3332871538 shutdown_connections 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.038+0000 7fd106863640 1 -- 172.21.15.7:0/3332871538 wait complete. 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd106863640 1 Processor -- start 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd106863640 1 -- start start 2024-09-21T04:32:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd106863640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 0x7fd1001025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd106863640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd100103910 0x7fd100100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd106863640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd10010a830 con 0x7fd100102f40 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd106863640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd10010a6b0 con 0x7fd100103910 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd0fffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 0x7fd1001025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd0fffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 0x7fd1001025d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38824/0 (socket says 172.21.15.7:38824) 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd0fffff640 1 -- 172.21.15.7:0/1617708337 learned_addr learned my addr 172.21.15.7:0/1617708337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd0ff7fe640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd100103910 0x7fd100100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd0fffff640 1 -- 172.21.15.7:0/1617708337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd100103910 msgr2=0x7fd100100c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd0fffff640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd100103910 0x7fd100100c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd0fffff640 1 -- 172.21.15.7:0/1617708337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1001011d0 con 0x7fd100102f40 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.039+0000 7fd0ff7fe640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd100103910 0x7fd100100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.040+0000 7fd0fffff640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 0x7fd1001025d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0006010 tx=0x7fd0f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.040+0000 7fd0fd7fa640 1 -- 172.21.15.7:0/1617708337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd0f003f070 con 0x7fd100102f40 2024-09-21T04:32:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.040+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd100101460 con 0x7fd100102f40 2024-09-21T04:32:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.040+0000 7fd0fd7fa640 1 -- 172.21.15.7:0/1617708337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0f00044d0 con 0x7fd100102f40 2024-09-21T04:32:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.040+0000 7fd0fd7fa640 1 -- 172.21.15.7:0/1617708337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd0f0035030 con 0x7fd100102f40 2024-09-21T04:32:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.040+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd100101a10 con 0x7fd100102f40 2024-09-21T04:32:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.041+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd100103340 con 0x7fd100102f40 2024-09-21T04:32:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.041+0000 7fd0fd7fa640 1 -- 172.21.15.7:0/1617708337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fd0f0042070 con 0x7fd100102f40 2024-09-21T04:32:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.042+0000 7fd0fd7fa640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd0e806e310 0x7fd0e80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.042+0000 7fd0fd7fa640 1 -- 172.21.15.7:0/1617708337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fd0f00aea10 con 0x7fd100102f40 2024-09-21T04:32:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.042+0000 7fd0ff7fe640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd0e806e310 0x7fd0e80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.042+0000 7fd0ff7fe640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd0e806e310 0x7fd0e80707d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006fd0 tx=0x7fd0f4008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.046+0000 7fd0fd7fa640 1 -- 172.21.15.7:0/1617708337 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd0f007c730 con 0x7fd100102f40 2024-09-21T04:32:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.229+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd10002db70 con 0x7fd0e806e310 2024-09-21T04:32:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.231+0000 7fd0fd7fa640 1 -- 172.21.15.7:0/1617708337 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+399 (secure 0 0 0) 0x7fd10002db70 con 0x7fd0e806e310 2024-09-21T04:32:13.233 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T04:32:13.233 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-21T04:32:13.233 INFO:teuthology.orchestra.run.smithi007.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 44s ago Has a FileSystem, LVM detected 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd0e806e310 msgr2=0x7fd0e80707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd0e806e310 0x7fd0e80707d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006fd0 tx=0x7fd0f4008040 comp rx=0 tx=0).stop 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 msgr2=0x7fd1001025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 0x7fd1001025d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0006010 tx=0x7fd0f00048f0 comp rx=0 tx=0).stop 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 shutdown_connections 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd0e806e310 0x7fd0e80707d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd100103910 0x7fd100100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 --2- 172.21.15.7:0/1617708337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd100102f40 0x7fd1001025d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 >> 172.21.15.7:0/1617708337 conn(0x7fd1000fb770 msgr2=0x7fd1001061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 shutdown_connections 2024-09-21T04:32:13.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:13.233+0000 7fd106863640 1 -- 172.21.15.7:0/1617708337 wait complete. 2024-09-21T04:32:13.893 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:32:13.906 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:32:13.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-09-21T04:32:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:14.597 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:14 smithi007 ceph-mon[27954]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:14.597 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:14 smithi007 ceph-mon[27954]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.597+0000 7f8cc3871640 1 -- 172.21.15.7:0/3733914771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 msgr2=0x7f8cbc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.597+0000 7f8cc3871640 1 --2- 172.21.15.7:0/3733914771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 0x7f8cbc100040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009880 tx=0x7f8cb802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.597+0000 7f8cc3871640 1 -- 172.21.15.7:0/3733914771 shutdown_connections 2024-09-21T04:32:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.597+0000 7f8cc3871640 1 --2- 172.21.15.7:0/3733914771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc102cc0 0x7f8cbc103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:14.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.597+0000 7f8cc3871640 1 --2- 172.21.15.7:0/3733914771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 0x7f8cbc100040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.597+0000 7f8cc3871640 1 -- 172.21.15.7:0/3733914771 >> 172.21.15.7:0/3733914771 conn(0x7f8cbc0fb790 msgr2=0x7f8cbc0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.597+0000 7f8cc3871640 1 -- 172.21.15.7:0/3733914771 shutdown_connections 2024-09-21T04:32:14.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.597+0000 7f8cc3871640 1 -- 172.21.15.7:0/3733914771 wait complete. 2024-09-21T04:32:14.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc3871640 1 Processor -- start 2024-09-21T04:32:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc3871640 1 -- start start 2024-09-21T04:32:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc3871640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 0x7f8cbc1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc3871640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc102cc0 0x7f8cbc199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc3871640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc10a820 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc3871640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc10a6a0 con 0x7f8cbc102cc0 2024-09-21T04:32:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc160d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 0x7f8cbc1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc160d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 0x7f8cbc1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54164/0 (socket says 172.21.15.7:54164) 2024-09-21T04:32:14.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc160d640 1 -- 172.21.15.7:0/3366883752 learned_addr learned my addr 172.21.15.7:0/3366883752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:14.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc0e0c640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc102cc0 0x7f8cbc199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:14.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc160d640 1 -- 172.21.15.7:0/3366883752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc102cc0 msgr2=0x7f8cbc199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:14.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc160d640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc102cc0 0x7f8cbc199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:14.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.598+0000 7f8cc160d640 1 -- 172.21.15.7:0/3366883752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc19a3b0 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.599+0000 7f8cc160d640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 0x7f8cbc1997f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006010 tx=0x7f8cb80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:14.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.599+0000 7f8cb27fc640 1 -- 172.21.15.7:0/3366883752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cb803f070 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.599+0000 7f8cb27fc640 1 -- 172.21.15.7:0/3366883752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cb80044d0 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.599+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc19ed90 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.599+0000 7f8cb27fc640 1 -- 172.21.15.7:0/3366883752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cb8035030 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.599+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc19f2d0 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.600+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cbc10b000 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.600+0000 7f8cb27fc640 1 -- 172.21.15.7:0/3366883752 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f8cb8042070 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.601+0000 7f8cb27fc640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e2b0 0x7f8ca8070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.601+0000 7f8cb27fc640 1 -- 172.21.15.7:0/3366883752 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f8cb80aea40 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.601+0000 7f8cc0e0c640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e2b0 0x7f8ca8070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:14.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.601+0000 7f8cc0e0c640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e2b0 0x7f8ca8070770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc19aca0 tx=0x7f8cac008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:14.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.605+0000 7f8cb27fc640 1 -- 172.21.15.7:0/3366883752 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8cb807c6e0 con 0x7f8cbc0ffc40 2024-09-21T04:32:14.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:14 smithi053 ceph-mon[35554]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:14 smithi053 ceph-mon[35554]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:14.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:14.805+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) -- 0x7f8cbc102500 con 0x7f8ca806e2b0 2024-09-21T04:32:15.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:15 smithi053 ceph-mon[35554]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:15.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:15 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-21T04:32:15.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:15 smithi007 ceph-mon[27954]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:15.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:15 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-21T04:32:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.377+0000 7f8cb27fc640 1 -- 172.21.15.7:0/3366883752 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8cbc102500 con 0x7f8ca806e2b0 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e2b0 msgr2=0x7f8ca8070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e2b0 0x7f8ca8070770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc19aca0 tx=0x7f8cac008040 comp rx=0 tx=0).stop 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 msgr2=0x7f8cbc1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 0x7f8cbc1997f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006010 tx=0x7f8cb80048f0 comp rx=0 tx=0).stop 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 shutdown_connections 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8ca806e2b0 0x7f8ca8070770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cbc102cc0 0x7f8cbc199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 --2- 172.21.15.7:0/3366883752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cbc0ffc40 0x7f8cbc1997f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 >> 172.21.15.7:0/3366883752 conn(0x7f8cbc0fb790 msgr2=0x7f8cbc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 shutdown_connections 2024-09-21T04:32:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:16.380+0000 7f8cc3871640 1 -- 172.21.15.7:0/3366883752 wait complete. 2024-09-21T04:32:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:16 smithi053 ceph-mon[35554]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-21T04:32:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:16 smithi053 ceph-mon[35554]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T04:32:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-21T04:32:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:32:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:16 smithi007 ceph-mon[27954]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-21T04:32:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:16 smithi007 ceph-mon[27954]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T04:32:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-21T04:32:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:32:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:16 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[27950]: 2024-09-21T04:32:16.357+0000 7fa0c3435640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:32:17.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: osdmap e40: 6 total, 6 up, 6 in 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: osdmap e41: 6 total, 6 up, 6 in 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: fsmap cephfs:0 2024-09-21T04:32:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: Saving service mds.cephfs spec with placement count:4 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.oxcnqk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.oxcnqk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:17.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:17 smithi053 ceph-mon[35554]: Deploying daemon mds.cephfs.smithi007.oxcnqk on smithi007 2024-09-21T04:32:17.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-21T04:32:17.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: osdmap e40: 6 total, 6 up, 6 in 2024-09-21T04:32:17.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-21T04:32:17.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:32:17.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:32:17.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-21T04:32:17.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: osdmap e41: 6 total, 6 up, 6 in 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: fsmap cephfs:0 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: Saving service mds.cephfs spec with placement count:4 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.oxcnqk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.oxcnqk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:17.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:17 smithi007 ceph-mon[27954]: Deploying daemon mds.cephfs.smithi007.oxcnqk on smithi007 2024-09-21T04:32:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:18.556 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:18 smithi007 ceph-mon[27954]: pgmap v172: 65 pgs: 2 creating+activating, 21 creating+peering, 5 active+clean, 37 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:18.556 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:18 smithi007 ceph-mon[27954]: osdmap e42: 6 total, 6 up, 6 in 2024-09-21T04:32:18.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:18 smithi053 ceph-mon[35554]: pgmap v172: 65 pgs: 2 creating+activating, 21 creating+peering, 5 active+clean, 37 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:18 smithi053 ceph-mon[35554]: osdmap e42: 6 total, 6 up, 6 in 2024-09-21T04:32:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.966+0000 7ff068b34640 1 -- 172.21.15.7:0/2278907272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 msgr2=0x7ff064101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.966+0000 7ff068b34640 1 --2- 172.21.15.7:0/2278907272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 0x7ff064101ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff058009880 tx=0x7ff05802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.966+0000 7ff068b34640 1 -- 172.21.15.7:0/2278907272 shutdown_connections 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.966+0000 7ff068b34640 1 --2- 172.21.15.7:0/2278907272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 0x7ff064101ee0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.966+0000 7ff068b34640 1 --2- 172.21.15.7:0/2278907272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff064100860 0x7ff064100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.966+0000 7ff068b34640 1 -- 172.21.15.7:0/2278907272 >> 172.21.15.7:0/2278907272 conn(0x7ff0640fc010 msgr2=0x7ff0640fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.966+0000 7ff068b34640 1 -- 172.21.15.7:0/2278907272 shutdown_connections 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.966+0000 7ff068b34640 1 -- 172.21.15.7:0/2278907272 wait complete. 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff068b34640 1 Processor -- start 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff068b34640 1 -- start start 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff068b34640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff064100860 0x7ff0641996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff068b34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 0x7ff064199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff068b34640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff064109680 con 0x7ff064101a60 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff068b34640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff064109500 con 0x7ff064100860 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff061d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 0x7ff064199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff061d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 0x7ff064199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54168/0 (socket says 172.21.15.7:54168) 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff061d9b640 1 -- 172.21.15.7:0/3501331091 learned_addr learned my addr 172.21.15.7:0/3501331091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff06259c640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff064100860 0x7ff0641996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff061d9b640 1 -- 172.21.15.7:0/3501331091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff064100860 msgr2=0x7ff0641996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff061d9b640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff064100860 0x7ff0641996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff061d9b640 1 -- 172.21.15.7:0/3501331091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06419a200 con 0x7ff064101a60 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.967+0000 7ff061d9b640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 0x7ff064199c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff058006010 tx=0x7ff0580048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.968+0000 7ff0537fe640 1 -- 172.21.15.7:0/3501331091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff05803f070 con 0x7ff064101a60 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.968+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06419ec10 con 0x7ff064101a60 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.968+0000 7ff0537fe640 1 -- 172.21.15.7:0/3501331091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff0580044d0 con 0x7ff064101a60 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.968+0000 7ff0537fe640 1 -- 172.21.15.7:0/3501331091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff058035030 con 0x7ff064101a60 2024-09-21T04:32:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.968+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06419f110 con 0x7ff064101a60 2024-09-21T04:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.969+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff064100c60 con 0x7ff064101a60 2024-09-21T04:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.969+0000 7ff0537fe640 1 -- 172.21.15.7:0/3501331091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7ff058042070 con 0x7ff064101a60 2024-09-21T04:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.970+0000 7ff0537fe640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff04c06e2b0 0x7ff04c070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.970+0000 7ff0537fe640 1 -- 172.21.15.7:0/3501331091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7ff0580aef10 con 0x7ff064101a60 2024-09-21T04:32:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.970+0000 7ff06259c640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff04c06e2b0 0x7ff04c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.971+0000 7ff06259c640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff04c06e2b0 0x7ff04c070770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0540058c0 tx=0x7ff054005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:18.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:18.975+0000 7ff0537fe640 1 -- 172.21.15.7:0/3501331091 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff05807c870 con 0x7ff064101a60 2024-09-21T04:32:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.213+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff06419fa10 con 0x7ff064101a60 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.214+0000 7ff0537fe640 1 -- 172.21.15.7:0/3501331091 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) ==== 75+0+1272 (secure 0 0 0) 0x7ff058080520 con 0x7ff064101a60 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout:e2 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:16:358692+0000 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:32:19.217 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 2 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:16.358433+0000 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:32:19.218 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 1 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:in 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:19.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:19.220 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-09-21T04:32:19.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff04c06e2b0 msgr2=0x7ff04c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff04c06e2b0 0x7ff04c070770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0540058c0 tx=0x7ff054005850 comp rx=0 tx=0).stop 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 msgr2=0x7ff064199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 0x7ff064199c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff058006010 tx=0x7ff0580048f0 comp rx=0 tx=0).stop 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 shutdown_connections 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff04c06e2b0 0x7ff04c070770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff064101a60 0x7ff064199c10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 --2- 172.21.15.7:0/3501331091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff064100860 0x7ff0641996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 >> 172.21.15.7:0/3501331091 conn(0x7ff0640fc010 msgr2=0x7ff06406a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 shutdown_connections 2024-09-21T04:32:19.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:19.216+0000 7ff068b34640 1 -- 172.21.15.7:0/3501331091 wait complete. 2024-09-21T04:32:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:19 smithi007 ceph-mon[27954]: osdmap e43: 6 total, 6 up, 6 in 2024-09-21T04:32:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:19 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3501331091' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:32:19.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:19 smithi053 ceph-mon[35554]: osdmap e43: 6 total, 6 up, 6 in 2024-09-21T04:32:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:19 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3501331091' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:32:20.137 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:32:20.154 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:32:20.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-09-21T04:32:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:20.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: pgmap v175: 65 pgs: 2 creating+activating, 21 creating+peering, 5 active+clean, 37 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.wdnfat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.wdnfat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: Deploying daemon mds.cephfs.smithi053.wdnfat on smithi053 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6826/2872814664,v1:172.21.15.7:6827/2872814664] up:boot 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: daemon mds.cephfs.smithi007.oxcnqk assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:32:20.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:32:20.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: Cluster is now healthy 2024-09-21T04:32:20.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: fsmap cephfs:0 1 up:standby 2024-09-21T04:32:20.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"}]: dispatch 2024-09-21T04:32:20.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: fsmap cephfs:1 {0=cephfs.smithi007.oxcnqk=up:creating} 2024-09-21T04:32:20.807 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:20 smithi007 ceph-mon[27954]: daemon mds.cephfs.smithi007.oxcnqk is now active in filesystem cephfs as rank 0 2024-09-21T04:32:20.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 -- 172.21.15.7:0/4093906512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8101a60 msgr2=0x7f60a8101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 --2- 172.21.15.7:0/4093906512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8101a60 0x7f60a8101ee0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f609c009880 tx=0x7f609c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 -- 172.21.15.7:0/4093906512 shutdown_connections 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 --2- 172.21.15.7:0/4093906512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8101a60 0x7f60a8101ee0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 --2- 172.21.15.7:0/4093906512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a8100860 0x7f60a8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 -- 172.21.15.7:0/4093906512 >> 172.21.15.7:0/4093906512 conn(0x7f60a80fc010 msgr2=0x7f60a80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 -- 172.21.15.7:0/4093906512 shutdown_connections 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 -- 172.21.15.7:0/4093906512 wait complete. 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.890+0000 7f60ad989640 1 Processor -- start 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60ad989640 1 -- start start 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60ad989640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8100860 0x7f60a81975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60ad989640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a8101a60 0x7f60a8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60ad989640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8109720 con 0x7f60a8100860 2024-09-21T04:32:20.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60ad989640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60a81095a0 con 0x7f60a8101a60 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60a6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8100860 0x7f60a81975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60a6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8100860 0x7f60a81975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54218/0 (socket says 172.21.15.7:54218) 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60a6ffd640 1 -- 172.21.15.7:0/2080569773 learned_addr learned my addr 172.21.15.7:0/2080569773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60a67fc640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a8101a60 0x7f60a8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60a6ffd640 1 -- 172.21.15.7:0/2080569773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a8101a60 msgr2=0x7f60a8197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60a6ffd640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a8101a60 0x7f60a8197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60a6ffd640 1 -- 172.21.15.7:0/2080569773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a81981b0 con 0x7f60a8100860 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.891+0000 7f60a6ffd640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8100860 0x7f60a81975f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f609800b3d0 tx=0x7f609800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.892+0000 7f60ac987640 1 -- 172.21.15.7:0/2080569773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6098015020 con 0x7f60a8100860 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.892+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a819cb80 con 0x7f60a8100860 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.892+0000 7f60ac987640 1 -- 172.21.15.7:0/2080569773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6098011af0 con 0x7f60a8100860 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.892+0000 7f60ac987640 1 -- 172.21.15.7:0/2080569773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6098010da0 con 0x7f60a8100860 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.892+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a819d0e0 con 0x7f60a8100860 2024-09-21T04:32:20.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.893+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a8100c60 con 0x7f60a8100860 2024-09-21T04:32:20.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.893+0000 7f60ac987640 1 -- 172.21.15.7:0/2080569773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f6098018020 con 0x7f60a8100860 2024-09-21T04:32:20.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.894+0000 7f60ac987640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f609006e2b0 0x7f6090070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:20.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.894+0000 7f60a67fc640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f609006e2b0 0x7f6090070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:20.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.894+0000 7f60ac987640 1 -- 172.21.15.7:0/2080569773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f609808dc70 con 0x7f60a8100860 2024-09-21T04:32:20.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.894+0000 7f60a67fc640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f609006e2b0 0x7f6090070770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f609c000c00 tx=0x7f609c0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:20.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:20.899+0000 7f60ac987640 1 -- 172.21.15.7:0/2080569773 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f609801f020 con 0x7f60a8100860 2024-09-21T04:32:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.112+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7f60a819dae0 con 0x7f60a8100860 2024-09-21T04:32:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: pgmap v175: 65 pgs: 2 creating+activating, 21 creating+peering, 5 active+clean, 37 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:32:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.wdnfat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:32:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.wdnfat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:32:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: Deploying daemon mds.cephfs.smithi053.wdnfat on smithi053 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6826/2872814664,v1:172.21.15.7:6827/2872814664] up:boot 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: daemon mds.cephfs.smithi007.oxcnqk assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: Cluster is now healthy 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: fsmap cephfs:0 1 up:standby 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"}]: dispatch 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: fsmap cephfs:1 {0=cephfs.smithi007.oxcnqk=up:creating} 2024-09-21T04:32:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:20 smithi053 ceph-mon[35554]: daemon mds.cephfs.smithi007.oxcnqk is now active in filesystem cephfs as rank 0 2024-09-21T04:32:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.377+0000 7f60ac987640 1 -- 172.21.15.7:0/2080569773 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v5) ==== 105+0+0 (secure 0 0 0) 0x7f609805b5d0 con 0x7f60a8100860 2024-09-21T04:32:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f609006e2b0 msgr2=0x7f6090070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f609006e2b0 0x7f6090070770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f609c000c00 tx=0x7f609c0058d0 comp rx=0 tx=0).stop 2024-09-21T04:32:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8100860 msgr2=0x7f60a81975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8100860 0x7f60a81975f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f609800b3d0 tx=0x7f609800b890 comp rx=0 tx=0).stop 2024-09-21T04:32:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 shutdown_connections 2024-09-21T04:32:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f609006e2b0 0x7f6090070770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60a8101a60 0x7f60a8197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 --2- 172.21.15.7:0/2080569773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a8100860 0x7f60a81975f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 >> 172.21.15.7:0/2080569773 conn(0x7f60a80fc010 msgr2=0x7f60a806a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 shutdown_connections 2024-09-21T04:32:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:21.380+0000 7f60ad989640 1 -- 172.21.15.7:0/2080569773 wait complete. 2024-09-21T04:32:21.929 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:21 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2080569773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-21T04:32:21.929 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:21 smithi053 ceph-mon[35554]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:32:21.929 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:21 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2080569773' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-21T04:32:21.929 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:21 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6826/2872814664,v1:172.21.15.7:6827/2872814664] up:active 2024-09-21T04:32:21.929 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:21 smithi053 ceph-mon[35554]: fsmap cephfs:1 {0=cephfs.smithi007.oxcnqk=up:active} 2024-09-21T04:32:22.143 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:21 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2080569773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-21T04:32:22.143 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:21 smithi007 ceph-mon[27954]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:32:22.143 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:21 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2080569773' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-21T04:32:22.143 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:21 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6826/2872814664,v1:172.21.15.7:6827/2872814664] up:active 2024-09-21T04:32:22.143 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:21 smithi007 ceph-mon[27954]: fsmap cephfs:1 {0=cephfs.smithi007.oxcnqk=up:active} 2024-09-21T04:32:22.179 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:32:22.190 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:32:22.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-09-21T04:32:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: pgmap v176: 65 pgs: 2 creating+activating, 63 active+clean; 578 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 7 op/s 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.eyygpg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.eyygpg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: daemon mds.cephfs.smithi053.wdnfat assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T04:32:22.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-21T04:32:22.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:32:22.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] up:boot 2024-09-21T04:32:22.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:creating} 2024-09-21T04:32:22.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"}]: dispatch 2024-09-21T04:32:22.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:22 smithi007 ceph-mon[27954]: daemon mds.cephfs.smithi053.wdnfat is now active in filesystem cephfs as rank 1 2024-09-21T04:32:23.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.091+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/2258019994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac1023f0 msgr2=0x7f0cac1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:23.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.091+0000 7f0cb1d0e640 1 --2- 172.21.15.7:0/2258019994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac1023f0 0x7f0cac1047e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0098b0 tx=0x7f0c9c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.091+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/2258019994 shutdown_connections 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.091+0000 7f0cb1d0e640 1 --2- 172.21.15.7:0/2258019994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cac104d20 0x7f0cac107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.091+0000 7f0cb1d0e640 1 --2- 172.21.15.7:0/2258019994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac1023f0 0x7f0cac1047e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.091+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/2258019994 >> 172.21.15.7:0/2258019994 conn(0x7f0cac0fc010 msgr2=0x7f0cac0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.091+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/2258019994 shutdown_connections 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.091+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/2258019994 wait complete. 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.092+0000 7f0cb1d0e640 1 Processor -- start 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.092+0000 7f0cb1d0e640 1 -- start start 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.092+0000 7f0cb1d0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cac1023f0 0x7f0cac101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.092+0000 7f0cb1d0e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac104d20 0x7f0cac1003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.092+0000 7f0cb1d0e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac10a7e0 con 0x7f0cac104d20 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.092+0000 7f0cb1d0e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac10a660 con 0x7f0cac1023f0 2024-09-21T04:32:23.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.092+0000 7f0caaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac104d20 0x7f0cac1003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0caaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac104d20 0x7f0cac1003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54246/0 (socket says 172.21.15.7:54246) 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0caaffd640 1 -- 172.21.15.7:0/1241216249 learned_addr learned my addr 172.21.15.7:0/1241216249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0cab7fe640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cac1023f0 0x7f0cac101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0caaffd640 1 -- 172.21.15.7:0/1241216249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cac1023f0 msgr2=0x7f0cac101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0caaffd640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cac1023f0 0x7f0cac101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0caaffd640 1 -- 172.21.15.7:0/1241216249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac1008e0 con 0x7f0cac104d20 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0caaffd640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac104d20 0x7f0cac1003a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000e970 tx=0x7f0ca000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0ca8ff9640 1 -- 172.21.15.7:0/1241216249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ca000cac0 con 0x7f0cac104d20 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0ca8ff9640 1 -- 172.21.15.7:0/1241216249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ca00054e0 con 0x7f0cac104d20 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cac100bd0 con 0x7f0cac104d20 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0ca8ff9640 1 -- 172.21.15.7:0/1241216249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ca0010430 con 0x7f0cac104d20 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.093+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cac101160 con 0x7f0cac104d20 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.094+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cac1027d0 con 0x7f0cac104d20 2024-09-21T04:32:23.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.095+0000 7f0ca8ff9640 1 -- 172.21.15.7:0/1241216249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f0ca0004ec0 con 0x7f0cac104d20 2024-09-21T04:32:23.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.095+0000 7f0ca8ff9640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c9406e2b0 0x7f0c94070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:23.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.095+0000 7f0cab7fe640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c9406e2b0 0x7f0c94070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:23.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.096+0000 7f0ca8ff9640 1 -- 172.21.15.7:0/1241216249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f0ca008e830 con 0x7f0cac104d20 2024-09-21T04:32:23.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.096+0000 7f0cab7fe640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c9406e2b0 0x7f0c94070770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c000c00 tx=0x7f0c9c005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:23.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.100+0000 7f0ca8ff9640 1 -- 172.21.15.7:0/1241216249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ca005c190 con 0x7f0cac104d20 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: pgmap v176: 65 pgs: 2 creating+activating, 63 active+clean; 578 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 7 op/s 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.eyygpg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.eyygpg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: daemon mds.cephfs.smithi053.wdnfat assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:32:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] up:boot 2024-09-21T04:32:23.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:creating} 2024-09-21T04:32:23.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"}]: dispatch 2024-09-21T04:32:23.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:22 smithi053 ceph-mon[35554]: daemon mds.cephfs.smithi053.wdnfat is now active in filesystem cephfs as rank 1 2024-09-21T04:32:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.319+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0cac19f620 con 0x7f0cac104d20 2024-09-21T04:32:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.573+0000 7f0ca8ff9640 1 -- 172.21.15.7:0/1241216249 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v7) ==== 121+0+0 (secure 0 0 0) 0x7f0ca005fe40 con 0x7f0cac104d20 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c9406e2b0 msgr2=0x7f0c94070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c9406e2b0 0x7f0c94070770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c000c00 tx=0x7f0c9c005a20 comp rx=0 tx=0).stop 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac104d20 msgr2=0x7f0cac1003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac104d20 0x7f0cac1003a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000e970 tx=0x7f0ca000ee30 comp rx=0 tx=0).stop 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 shutdown_connections 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f0c9406e2b0 0x7f0c94070770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cac104d20 0x7f0cac1003a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 --2- 172.21.15.7:0/1241216249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cac1023f0 0x7f0cac101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.576+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 >> 172.21.15.7:0/1241216249 conn(0x7f0cac0fc010 msgr2=0x7f0cac0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.577+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 shutdown_connections 2024-09-21T04:32:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:23.577+0000 7f0cb1d0e640 1 -- 172.21.15.7:0/1241216249 wait complete. 2024-09-21T04:32:23.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:23 smithi053 ceph-mon[35554]: Deploying daemon mds.cephfs.smithi007.eyygpg on smithi007 2024-09-21T04:32:23.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:23 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1241216249' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-21T04:32:23.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:23 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1241216249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T04:32:23.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:23 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] up:active 2024-09-21T04:32:23.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:23 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 2024-09-21T04:32:24.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:23 smithi007 ceph-mon[27954]: Deploying daemon mds.cephfs.smithi007.eyygpg on smithi007 2024-09-21T04:32:24.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:23 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1241216249' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-21T04:32:24.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:23 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1241216249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T04:32:24.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:23 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] up:active 2024-09-21T04:32:24.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:23 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 2024-09-21T04:32:24.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:24 smithi053 ceph-mon[35554]: pgmap v177: 65 pgs: 65 active+clean; 578 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 917 B/s wr, 5 op/s 2024-09-21T04:32:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:24 smithi007 ceph-mon[27954]: pgmap v177: 65 pgs: 65 active+clean; 578 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 917 B/s wr, 5 op/s 2024-09-21T04:32:25.232 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:32:25.250 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:32:25.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-09-21T04:32:25.843 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:26.090 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:25 smithi007 ceph-mon[27954]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-21T04:32:26.090 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:25 smithi007 ceph-mon[27954]: Cluster is now healthy 2024-09-21T04:32:26.090 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:25 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:boot 2024-09-21T04:32:26.090 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:25 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby 2024-09-21T04:32:26.090 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:25 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"}]: dispatch 2024-09-21T04:32:26.090 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:25 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 2024-09-21T04:32:26.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:25 smithi053 ceph-mon[35554]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-21T04:32:26.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:25 smithi053 ceph-mon[35554]: Cluster is now healthy 2024-09-21T04:32:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:25 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:boot 2024-09-21T04:32:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:25 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby 2024-09-21T04:32:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:25 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"}]: dispatch 2024-09-21T04:32:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:25 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 2024-09-21T04:32:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.289+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/1744499158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 msgr2=0x7ff4cc074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.289+0000 7ff4d2a6f640 1 --2- 172.21.15.7:0/1744499158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 0x7ff4cc074680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009880 tx=0x7ff4c802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/1744499158 shutdown_connections 2024-09-21T04:32:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 --2- 172.21.15.7:0/1744499158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4cc074c50 0x7ff4cc0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 --2- 172.21.15.7:0/1744499158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 0x7ff4cc074680 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/1744499158 >> 172.21.15.7:0/1744499158 conn(0x7ff4cc0fd0f0 msgr2=0x7ff4cc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/1744499158 shutdown_connections 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/1744499158 wait complete. 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 Processor -- start 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 -- start start 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4cc074c50 0x7ff4cc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 0x7ff4cc199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc10c9a0 con 0x7ff4cc076220 2024-09-21T04:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.290+0000 7ff4d2a6f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc10c820 con 0x7ff4cc074c50 2024-09-21T04:32:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 0x7ff4cc199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4d080b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4cc074c50 0x7ff4cc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 0x7ff4cc199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57764/0 (socket says 172.21.15.7:57764) 2024-09-21T04:32:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c3fff640 1 -- 172.21.15.7:0/2277013256 learned_addr learned my addr 172.21.15.7:0/2277013256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c3fff640 1 -- 172.21.15.7:0/2277013256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4cc074c50 msgr2=0x7ff4cc1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c3fff640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4cc074c50 0x7ff4cc1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:26.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c3fff640 1 -- 172.21.15.7:0/2277013256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc19a290 con 0x7ff4cc076220 2024-09-21T04:32:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c3fff640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 0x7ff4cc199c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00e970 tx=0x7ff4bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c1ffb640 1 -- 172.21.15.7:0/2277013256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc00cac0 con 0x7ff4cc076220 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c1ffb640 1 -- 172.21.15.7:0/2277013256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4bc0054e0 con 0x7ff4cc076220 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc19ecd0 con 0x7ff4cc076220 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4c1ffb640 1 -- 172.21.15.7:0/2277013256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc010430 con 0x7ff4cc076220 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.291+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc19f200 con 0x7ff4cc076220 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.292+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc0750b0 con 0x7ff4cc076220 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.293+0000 7ff4c1ffb640 1 -- 172.21.15.7:0/2277013256 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7ff4bc004e80 con 0x7ff4cc076220 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.293+0000 7ff4c1ffb640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff4b406e2b0 0x7ff4b4070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.293+0000 7ff4c1ffb640 1 -- 172.21.15.7:0/2277013256 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7ff4bc08e7e0 con 0x7ff4cc076220 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.293+0000 7ff4d080b640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff4b406e2b0 0x7ff4b4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.294+0000 7ff4d080b640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff4b406e2b0 0x7ff4b4070770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009230 tx=0x7ff4c80058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:26.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.298+0000 7ff4c1ffb640 1 -- 172.21.15.7:0/2277013256 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff4bc05c140 con 0x7ff4cc076220 2024-09-21T04:32:26.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.509+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff4cc062fb0 con 0x7ff4cc076220 2024-09-21T04:32:26.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.868+0000 7ff4c1ffb640 1 -- 172.21.15.7:0/2277013256 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v10) ==== 168+0+0 (secure 0 0 0) 0x7ff4bc05fdf0 con 0x7ff4cc076220 2024-09-21T04:32:26.871 INFO:teuthology.orchestra.run.smithi007.stderr:inline data enabled 2024-09-21T04:32:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.871+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff4b406e2b0 msgr2=0x7ff4b4070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.871+0000 7ff4d2a6f640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff4b406e2b0 0x7ff4b4070770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009230 tx=0x7ff4c80058d0 comp rx=0 tx=0).stop 2024-09-21T04:32:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.871+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 msgr2=0x7ff4cc199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.871+0000 7ff4d2a6f640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 0x7ff4cc199c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00e970 tx=0x7ff4bc00ed70 comp rx=0 tx=0).stop 2024-09-21T04:32:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.872+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 shutdown_connections 2024-09-21T04:32:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.872+0000 7ff4d2a6f640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff4b406e2b0 0x7ff4b4070770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.872+0000 7ff4d2a6f640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4cc076220 0x7ff4cc199c10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.872+0000 7ff4d2a6f640 1 --2- 172.21.15.7:0/2277013256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4cc074c50 0x7ff4cc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.872+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 >> 172.21.15.7:0/2277013256 conn(0x7ff4cc0fd0f0 msgr2=0x7ff4cc1079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.872+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 shutdown_connections 2024-09-21T04:32:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:26.872+0000 7ff4d2a6f640 1 -- 172.21.15.7:0/2277013256 wait complete. 2024-09-21T04:32:26.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-21T04:32:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.jvuwme", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:32:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.jvuwme", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:32:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: Deploying daemon mds.cephfs.smithi053.jvuwme on smithi053 2024-09-21T04:32:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:26 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2277013256' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-21T04:32:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 7 op/s 2024-09-21T04:32:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.jvuwme", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-21T04:32:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.jvuwme", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:32:27.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:27.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: Deploying daemon mds.cephfs.smithi053.jvuwme on smithi053 2024-09-21T04:32:27.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:26 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2277013256' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-21T04:32:27.509 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:32:27.524 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:32:27.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T04:32:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:27.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:27 smithi053 ceph-mon[35554]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-21T04:32:27.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:27 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2277013256' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T04:32:27.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:27 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 2024-09-21T04:32:28.153 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:27 smithi007 ceph-mon[27954]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-21T04:32:28.153 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:27 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2277013256' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T04:32:28.153 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:27 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 -- 172.21.15.7:0/94789869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 msgr2=0x7f0590103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 --2- 172.21.15.7:0/94789869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 0x7f0590103000 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0584009880 tx=0x7f058402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 -- 172.21.15.7:0/94789869 shutdown_connections 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 --2- 172.21.15.7:0/94789869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 0x7f0590103000 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 --2- 172.21.15.7:0/94789869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0590101980 0x7f0590101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 -- 172.21.15.7:0/94789869 >> 172.21.15.7:0/94789869 conn(0x7f05900fd110 msgr2=0x7f05900ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 -- 172.21.15.7:0/94789869 shutdown_connections 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 -- 172.21.15.7:0/94789869 wait complete. 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 Processor -- start 2024-09-21T04:32:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.234+0000 7f059625c640 1 -- start start 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f059625c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0590101980 0x7f0590197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f059625c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 0x7f05901979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f059625c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f059010a7a0 con 0x7f0590102b80 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f059625c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f059010a620 con 0x7f0590101980 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 0x7f05901979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 0x7f05901979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57788/0 (socket says 172.21.15.7:57788) 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058effd640 1 -- 172.21.15.7:0/4285001576 learned_addr learned my addr 172.21.15.7:0/4285001576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058f7fe640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0590101980 0x7f0590197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058effd640 1 -- 172.21.15.7:0/4285001576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0590101980 msgr2=0x7f0590197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058effd640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0590101980 0x7f0590197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058effd640 1 -- 172.21.15.7:0/4285001576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0590198050 con 0x7f0590102b80 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058effd640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 0x7f05901979d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0584006010 tx=0x7f05840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058cff9640 1 -- 172.21.15.7:0/4285001576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f058403f070 con 0x7f0590102b80 2024-09-21T04:32:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059019ca30 con 0x7f0590102b80 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058cff9640 1 -- 172.21.15.7:0/4285001576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0584004ac0 con 0x7f0590102b80 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f058cff9640 1 -- 172.21.15.7:0/4285001576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0584035030 con 0x7f0590102b80 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.235+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059019cfc0 con 0x7f0590102b80 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.236+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0590101d80 con 0x7f0590102b80 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.237+0000 7f058cff9640 1 -- 172.21.15.7:0/4285001576 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f0584042070 con 0x7f0590102b80 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.237+0000 7f058cff9640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f057806e2b0 0x7f0578070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.237+0000 7f058f7fe640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f057806e2b0 0x7f0578070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.237+0000 7f058cff9640 1 -- 172.21.15.7:0/4285001576 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f05840aecd0 con 0x7f0590102b80 2024-09-21T04:32:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.238+0000 7f058f7fe640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f057806e2b0 0x7f0578070770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f05901029e0 tx=0x7f0580009720 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.242+0000 7f058cff9640 1 -- 172.21.15.7:0/4285001576 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05840802e0 con 0x7f0590102b80 2024-09-21T04:32:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.458+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0590062df0 con 0x7f0590102b80 2024-09-21T04:32:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.459+0000 7f058cff9640 1 -- 172.21.15.7:0/4285001576 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 10 v10) ==== 76+0+1815 (secure 0 0 0) 0x7f0584084b10 con 0x7f0590102b80 2024-09-21T04:32:28.462 INFO:teuthology.orchestra.run.smithi007.stdout:e10 2024-09-21T04:32:28.463 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:26:868156+0000 2024-09-21T04:32:28.463 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:32:28.463 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:32:28.463 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:32:28.463 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:28.463 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:32:28.464 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:32:28.464 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 10 2024-09-21T04:32:28.464 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:32:28.464 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:32:28.464 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:26.511188+0000 2024-09-21T04:32:28.464 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-09-21T04:32:28.465 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:32:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:32:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:32:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14562,1=24327} 2024-09-21T04:32:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:32:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:32:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:32:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:32:28.466 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14562 members: 24327,14562 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14562} state up:active seq 2 addr [v2:172.21.15.7:6826/2872814664,v1:172.21.15.7:6827/2872814664] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:standby-replay seq 1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 2 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:32:28.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:28.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f057806e2b0 msgr2=0x7f0578070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f057806e2b0 0x7f0578070770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f05901029e0 tx=0x7f0580009720 comp rx=0 tx=0).stop 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 msgr2=0x7f05901979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 0x7f05901979d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0584006010 tx=0x7f05840046c0 comp rx=0 tx=0).stop 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 shutdown_connections 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f057806e2b0 0x7f0578070770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0590102b80 0x7f05901979d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 --2- 172.21.15.7:0/4285001576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0590101980 0x7f0590197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 >> 172.21.15.7:0/4285001576 conn(0x7f05900fd110 msgr2=0x7f0590106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.461+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 shutdown_connections 2024-09-21T04:32:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:28.462+0000 7f059625c640 1 -- 172.21.15.7:0/4285001576 wait complete. 2024-09-21T04:32:29.145 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:28 smithi053 ceph-mon[35554]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 211 B/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-21T04:32:29.145 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:28 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/4285001576' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:32:29.145 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.145 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.145 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.145 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.145 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.146 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:28 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:32:29.149 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:28 smithi007 ceph-mon[27954]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 211 B/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-21T04:32:29.149 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:28 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/4285001576' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:32:29.149 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.149 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.149 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.149 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.149 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:29.149 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:28 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:32:29.177 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-09-21T04:32:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:30.154 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:29 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] up:boot 2024-09-21T04:32:30.155 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:29 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:30.155 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:29 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"}]: dispatch 2024-09-21T04:32:30.155 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:29 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:32:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:29 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] up:boot 2024-09-21T04:32:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:29 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:29 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"}]: dispatch 2024-09-21T04:32:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:29 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:32:30.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 -- 172.21.15.7:0/1356594169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8100860 msgr2=0x7f33a8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:30.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 --2- 172.21.15.7:0/1356594169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8100860 0x7f33a8100c60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3398009880 tx=0x7f339802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:30.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 -- 172.21.15.7:0/1356594169 shutdown_connections 2024-09-21T04:32:30.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 --2- 172.21.15.7:0/1356594169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33a8101a60 0x7f33a8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:30.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 --2- 172.21.15.7:0/1356594169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8100860 0x7f33a8100c60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:30.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 -- 172.21.15.7:0/1356594169 >> 172.21.15.7:0/1356594169 conn(0x7f33a80fc010 msgr2=0x7f33a80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:30.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 -- 172.21.15.7:0/1356594169 shutdown_connections 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 -- 172.21.15.7:0/1356594169 wait complete. 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 Processor -- start 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 -- start start 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33a8100860 0x7f33a8197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8101a60 0x7f33a8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.562+0000 7f33ae8b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81096a0 con 0x7f33a8101a60 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f33ae8b3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8109520 con 0x7f33a8100860 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f339ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8101a60 0x7f33a8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f33a7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33a8100860 0x7f33a8197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f339ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8101a60 0x7f33a8197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57822/0 (socket says 172.21.15.7:57822) 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f339ffff640 1 -- 172.21.15.7:0/4248188142 learned_addr learned my addr 172.21.15.7:0/4248188142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f339ffff640 1 -- 172.21.15.7:0/4248188142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33a8100860 msgr2=0x7f33a8197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f339ffff640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33a8100860 0x7f33a8197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f339ffff640 1 -- 172.21.15.7:0/4248188142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a8198100 con 0x7f33a8101a60 2024-09-21T04:32:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f33a7fff640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33a8100860 0x7f33a8197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:32:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f339ffff640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8101a60 0x7f33a8197a80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f339000e970 tx=0x7f339000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f33a5ffb640 1 -- 172.21.15.7:0/4248188142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f339000cac0 con 0x7f33a8101a60 2024-09-21T04:32:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f33a5ffb640 1 -- 172.21.15.7:0/4248188142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33900054e0 con 0x7f33a8101a60 2024-09-21T04:32:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a819cb40 con 0x7f33a8101a60 2024-09-21T04:32:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f33a5ffb640 1 -- 172.21.15.7:0/4248188142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3390010430 con 0x7f33a8101a60 2024-09-21T04:32:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.563+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a819d0d0 con 0x7f33a8101a60 2024-09-21T04:32:30.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.564+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a8100c60 con 0x7f33a8101a60 2024-09-21T04:32:30.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.565+0000 7f33a5ffb640 1 -- 172.21.15.7:0/4248188142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f3390004ee0 con 0x7f33a8101a60 2024-09-21T04:32:30.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.566+0000 7f33a5ffb640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f339406e310 0x7f33940707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:30.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.566+0000 7f33a5ffb640 1 -- 172.21.15.7:0/4248188142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f339008e910 con 0x7f33a8101a60 2024-09-21T04:32:30.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.566+0000 7f33a7fff640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f339406e310 0x7f33940707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:30.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.566+0000 7f33a7fff640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f339406e310 0x7f33940707d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3398009230 tx=0x7f33980058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:30.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.570+0000 7f33a5ffb640 1 -- 172.21.15.7:0/4248188142 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3390054210 con 0x7f33a8101a60 2024-09-21T04:32:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.800+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33a819d530 con 0x7f33a8101a60 2024-09-21T04:32:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.801+0000 7f33a5ffb640 1 -- 172.21.15.7:0/4248188142 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+4489 (secure 0 0 0) 0x7f339005c330 con 0x7f33a8101a60 2024-09-21T04:32:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 14 2024-09-21T04:32:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f339406e310 msgr2=0x7f33940707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f339406e310 0x7f33940707d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3398009230 tx=0x7f33980058d0 comp rx=0 tx=0).stop 2024-09-21T04:32:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8101a60 msgr2=0x7f33a8197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8101a60 0x7f33a8197a80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f339000e970 tx=0x7f339000ed70 comp rx=0 tx=0).stop 2024-09-21T04:32:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 shutdown_connections 2024-09-21T04:32:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f339406e310 0x7f33940707d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a8101a60 0x7f33a8197a80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 --2- 172.21.15.7:0/4248188142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33a8100860 0x7f33a8197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 >> 172.21.15.7:0/4248188142 conn(0x7f33a80fc010 msgr2=0x7f33a806a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 shutdown_connections 2024-09-21T04:32:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:30.803+0000 7f33ae8b3640 1 -- 172.21.15.7:0/4248188142 wait complete. 2024-09-21T04:32:30.817 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 191 B/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:standby-replay 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: Dropping low affinity active daemon mds.cephfs.smithi007.oxcnqk in favor of higher affinity standby. 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: Replacing daemon mds.cephfs.smithi007.oxcnqk as rank 0 with standby daemon mds.cephfs.smithi007.eyygpg 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: osdmap e44: 6 total, 6 up, 6 in 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: fsmap cephfs:2/2 {0=cephfs.smithi007.eyygpg=up:replay,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:30 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/4248188142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 191 B/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:standby-replay 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: Dropping low affinity active daemon mds.cephfs.smithi007.oxcnqk in favor of higher affinity standby. 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: Replacing daemon mds.cephfs.smithi007.oxcnqk as rank 0 with standby daemon mds.cephfs.smithi007.eyygpg 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: osdmap e44: 6 total, 6 up, 6 in 2024-09-21T04:32:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: fsmap cephfs:2/2 {0=cephfs.smithi007.eyygpg=up:replay,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 2024-09-21T04:32:31.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:31.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:31.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:31.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:30 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/4248188142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-21T04:32:31.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:31 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] up:boot 2024-09-21T04:32:31.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:31 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] up:active 2024-09-21T04:32:31.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:31 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:resolve 2024-09-21T04:32:31.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:31 smithi007 ceph-mon[27954]: fsmap cephfs:2/2 {0=cephfs.smithi007.eyygpg=up:resolve,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:31.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"}]: dispatch 2024-09-21T04:32:31.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:31.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:32:32.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-09-21T04:32:32.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:31 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] up:boot 2024-09-21T04:32:32.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:31 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] up:active 2024-09-21T04:32:32.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:31 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:resolve 2024-09-21T04:32:32.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:31 smithi053 ceph-mon[35554]: fsmap cephfs:2/2 {0=cephfs.smithi007.eyygpg=up:resolve,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:32.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:31 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"}]: dispatch 2024-09-21T04:32:32.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:31 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:32.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:31 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:32:32.774 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:33.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:32 smithi053 ceph-mon[35554]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-21T04:32:33.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:32 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:reconnect 2024-09-21T04:32:33.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:32 smithi053 ceph-mon[35554]: fsmap cephfs:2/2 {0=cephfs.smithi007.eyygpg=up:reconnect,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:32 smithi007 ceph-mon[27954]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-21T04:32:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:32 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:reconnect 2024-09-21T04:32:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:32 smithi007 ceph-mon[27954]: fsmap cephfs:2/2 {0=cephfs.smithi007.eyygpg=up:reconnect,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:33.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.778+0000 7ff4209a9640 1 -- 172.21.15.7:0/3668362059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 msgr2=0x7ff41c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:33.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.778+0000 7ff4209a9640 1 --2- 172.21.15.7:0/3668362059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 0x7ff41c100c60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009c20 tx=0x7ff40c02a970 comp rx=0 tx=0).stop 2024-09-21T04:32:33.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.778+0000 7ff4209a9640 1 -- 172.21.15.7:0/3668362059 shutdown_connections 2024-09-21T04:32:33.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.778+0000 7ff4209a9640 1 --2- 172.21.15.7:0/3668362059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c101a60 0x7ff41c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:33.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.778+0000 7ff4209a9640 1 --2- 172.21.15.7:0/3668362059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 0x7ff41c100c60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:33.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.778+0000 7ff4209a9640 1 -- 172.21.15.7:0/3668362059 >> 172.21.15.7:0/3668362059 conn(0x7ff41c0fc010 msgr2=0x7ff41c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:33.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.778+0000 7ff4209a9640 1 -- 172.21.15.7:0/3668362059 shutdown_connections 2024-09-21T04:32:33.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.778+0000 7ff4209a9640 1 -- 172.21.15.7:0/3668362059 wait complete. 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff4209a9640 1 Processor -- start 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff4209a9640 1 -- start start 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff4209a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 0x7ff41c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff4209a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c101a60 0x7ff41c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff4209a9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c109680 con 0x7ff41c100860 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff4209a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c109500 con 0x7ff41c101a60 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff41a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 0x7ff41c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff41a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 0x7ff41c199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47178/0 (socket says 172.21.15.7:47178) 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff41a59c640 1 -- 172.21.15.7:0/2544017921 learned_addr learned my addr 172.21.15.7:0/2544017921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.779+0000 7ff419d9b640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c101a60 0x7ff41c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:33.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff41a59c640 1 -- 172.21.15.7:0/2544017921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c101a60 msgr2=0x7ff41c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff41a59c640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c101a60 0x7ff41c199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff41a59c640 1 -- 172.21.15.7:0/2544017921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c19a0c0 con 0x7ff41c100860 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff41a59c640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 0x7ff41c199500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009bf0 tx=0x7ff40c02f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff40b7fe640 1 -- 172.21.15.7:0/2544017921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff40c047070 con 0x7ff41c100860 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c19eaa0 con 0x7ff41c100860 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff40b7fe640 1 -- 172.21.15.7:0/2544017921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff40c02fea0 con 0x7ff41c100860 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff40b7fe640 1 -- 172.21.15.7:0/2544017921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff40c034070 con 0x7ff41c100860 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.780+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c19f030 con 0x7ff41c100860 2024-09-21T04:32:33.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.781+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c0a1310 con 0x7ff41c100860 2024-09-21T04:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.782+0000 7ff40b7fe640 1 -- 172.21.15.7:0/2544017921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7ff40c04a070 con 0x7ff41c100860 2024-09-21T04:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.782+0000 7ff40b7fe640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff40406e2b0 0x7ff404070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.783+0000 7ff40b7fe640 1 -- 172.21.15.7:0/2544017921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7ff40c0af270 con 0x7ff41c100860 2024-09-21T04:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.783+0000 7ff419d9b640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff40406e2b0 0x7ff404070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.783+0000 7ff419d9b640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff40406e2b0 0x7ff404070770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff41c19a9b0 tx=0x7ff41000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:33.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:33.787+0000 7ff40b7fe640 1 -- 172.21.15.7:0/2544017921 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff40c03f070 con 0x7ff41c100860 2024-09-21T04:32:33.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:33 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:rejoin 2024-09-21T04:32:33.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:33 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] up:standby-replay 2024-09-21T04:32:33.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:33 smithi007 ceph-mon[27954]: fsmap cephfs:2/2 {0=cephfs.smithi007.eyygpg=up:rejoin,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:33.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:33 smithi007 ceph-mon[27954]: daemon mds.cephfs.smithi007.eyygpg is now active in filesystem cephfs as rank 0 2024-09-21T04:32:33.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:33 smithi007 ceph-mon[27954]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:32:34.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.042+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) -- 0x7ff41c19f450 con 0x7ff41c100860 2024-09-21T04:32:34.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.043+0000 7ff40b7fe640 1 -- 172.21.15.7:0/2544017921 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v19) ==== 78+0+98 (secure 0 0 0) 0x7ff40c080780 con 0x7ff41c100860 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff40406e2b0 msgr2=0x7ff404070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff40406e2b0 0x7ff404070770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff41c19a9b0 tx=0x7ff41000b390 comp rx=0 tx=0).stop 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 msgr2=0x7ff41c199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 0x7ff41c199500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009bf0 tx=0x7ff40c02f5f0 comp rx=0 tx=0).stop 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 shutdown_connections 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7ff40406e2b0 0x7ff404070770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c101a60 0x7ff41c199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 --2- 172.21.15.7:0/2544017921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c100860 0x7ff41c199500 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 >> 172.21.15.7:0/2544017921 conn(0x7ff41c0fc010 msgr2=0x7ff41c0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 shutdown_connections 2024-09-21T04:32:34.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:34.045+0000 7ff4209a9640 1 -- 172.21.15.7:0/2544017921 wait complete. 2024-09-21T04:32:34.058 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:32:34.162 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:33 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:rejoin 2024-09-21T04:32:34.162 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:33 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] up:standby-replay 2024-09-21T04:32:34.162 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:33 smithi053 ceph-mon[35554]: fsmap cephfs:2/2 {0=cephfs.smithi007.eyygpg=up:rejoin,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:34.162 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:33 smithi053 ceph-mon[35554]: daemon mds.cephfs.smithi007.eyygpg is now active in filesystem cephfs as rank 0 2024-09-21T04:32:34.162 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:33 smithi053 ceph-mon[35554]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:32:34.761 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-09-21T04:32:34.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:32:35.146 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:34 smithi053 ceph-mon[35554]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-21T04:32:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:34 smithi053 ceph-mon[35554]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:active 2024-09-21T04:32:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:34 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:34 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:32:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:34 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2544017921' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-21T04:32:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:34 smithi007 ceph-mon[27954]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-21T04:32:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:34 smithi007 ceph-mon[27954]: mds.? [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] up:active 2024-09-21T04:32:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:34 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 1 up:standby-replay 1 up:standby 2024-09-21T04:32:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:34 smithi007 ceph-mon[27954]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:32:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:34 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2544017921' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-21T04:32:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.589+0000 7f6984151640 1 -- 172.21.15.7:0/310821793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c100800 msgr2=0x7f697c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.589+0000 7f6984151640 1 --2- 172.21.15.7:0/310821793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c100800 0x7f697c100c60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f696c009880 tx=0x7f696c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.589+0000 7f6984151640 1 -- 172.21.15.7:0/310821793 shutdown_connections 2024-09-21T04:32:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.589+0000 7f6984151640 1 --2- 172.21.15.7:0/310821793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c100800 0x7f697c100c60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.589+0000 7f6984151640 1 --2- 172.21.15.7:0/310821793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c106800 0x7f697c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.589+0000 7f6984151640 1 -- 172.21.15.7:0/310821793 >> 172.21.15.7:0/310821793 conn(0x7f697c0fc520 msgr2=0x7f697c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.589+0000 7f6984151640 1 -- 172.21.15.7:0/310821793 shutdown_connections 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.590+0000 7f6984151640 1 -- 172.21.15.7:0/310821793 wait complete. 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.590+0000 7f6984151640 1 Processor -- start 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.590+0000 7f6984151640 1 -- start start 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.590+0000 7f6984151640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c100800 0x7f697c0fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.590+0000 7f6984151640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c106800 0x7f697c0fd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.590+0000 7f6984151640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f697c10dcd0 con 0x7f697c106800 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.590+0000 7f6984151640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f697c10db50 con 0x7f697c100800 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f69816ec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c106800 0x7f697c0fd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f69816ec640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c106800 0x7f697c0fd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47194/0 (socket says 172.21.15.7:47194) 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f69816ec640 1 -- 172.21.15.7:0/547042386 learned_addr learned my addr 172.21.15.7:0/547042386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f6981eed640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c100800 0x7f697c0fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f69816ec640 1 -- 172.21.15.7:0/547042386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c100800 msgr2=0x7f697c0fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f69816ec640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c100800 0x7f697c0fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f69816ec640 1 -- 172.21.15.7:0/547042386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c0fd710 con 0x7f697c106800 2024-09-21T04:32:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f69816ec640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c106800 0x7f697c0fd1d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f696c031460 tx=0x7f696c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f6972ffd640 1 -- 172.21.15.7:0/547042386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f696c03f070 con 0x7f697c106800 2024-09-21T04:32:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f6972ffd640 1 -- 172.21.15.7:0/547042386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f696c0035f0 con 0x7f697c106800 2024-09-21T04:32:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c0fd9a0 con 0x7f697c106800 2024-09-21T04:32:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.591+0000 7f6972ffd640 1 -- 172.21.15.7:0/547042386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f696c035030 con 0x7f697c106800 2024-09-21T04:32:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.592+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c1a3da0 con 0x7f697c106800 2024-09-21T04:32:35.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.592+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c101f40 con 0x7f697c106800 2024-09-21T04:32:35.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.593+0000 7f6972ffd640 1 -- 172.21.15.7:0/547042386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f696c042070 con 0x7f697c106800 2024-09-21T04:32:35.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.594+0000 7f6972ffd640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f696806e310 0x7f69680707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:35.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.594+0000 7f6981eed640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f696806e310 0x7f69680707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:35.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.594+0000 7f6972ffd640 1 -- 172.21.15.7:0/547042386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f696c0aea10 con 0x7f697c106800 2024-09-21T04:32:35.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.594+0000 7f6981eed640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f696806e310 0x7f69680707d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6978005fd0 tx=0x7f697800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:35.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.598+0000 7f6972ffd640 1 -- 172.21.15.7:0/547042386 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f696c080040 con 0x7f697c106800 2024-09-21T04:32:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.815+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f697c109d80 con 0x7f697c106800 2024-09-21T04:32:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.815+0000 7f6972ffd640 1 -- 172.21.15.7:0/547042386 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5344 (secure 0 0 0) 0x7f696c084870 con 0x7f697c106800 2024-09-21T04:32:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:32:35.819 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"btime":"2024-09-21T04:32:33:892197+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:32:33.892184+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14578,"mds_1":24327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14578":{"gid":14578,"name":"cephfs.smithi007.eyygpg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.7:6829/406067991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":406067991},{"type":"v1","addr":"172.21.15.7:6829","nonce":406067991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14594":{"gid":14594,"name":"cephfs.smithi007.oxcnqk","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.7:6827/3426051215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3426051215},{"type":"v1","addr":"172.21.15.7:6827","nonce":3426051215}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24327":{"gid":24327,"name":"cephfs.smithi053.wdnfat","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6825/2105303008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2105303008},{"type":"v1","addr":"172.21.15.53:6825","nonce":2105303008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24345":{"gid":24345,"name":"cephfs.smithi053.jvuwme","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.53:6827/2642617317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2642617317},{"type":"v1","addr":"172.21.15.53:6827","nonce":2642617317}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14578,"qdb_cluster":[24327,14578]},"id":1}]} 2024-09-21T04:32:35.820 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:32:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.817+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f696806e310 msgr2=0x7f69680707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.817+0000 7f6984151640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f696806e310 0x7f69680707d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6978005fd0 tx=0x7f697800b390 comp rx=0 tx=0).stop 2024-09-21T04:32:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c106800 msgr2=0x7f697c0fd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c106800 0x7f697c0fd1d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f696c031460 tx=0x7f696c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:32:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 shutdown_connections 2024-09-21T04:32:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f696806e310 0x7f69680707d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697c106800 0x7f697c0fd1d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 --2- 172.21.15.7:0/547042386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c100800 0x7f697c0fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 >> 172.21.15.7:0/547042386 conn(0x7f697c0fc520 msgr2=0x7f697c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 shutdown_connections 2024-09-21T04:32:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:35.818+0000 7f6984151640 1 -- 172.21.15.7:0/547042386 wait complete. 2024-09-21T04:32:36.499 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 19, 'max_mds': 2, 'flags': 50} 2024-09-21T04:32:36.499 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-21T04:32:36.558 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-21T04:32:36.558 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-09-21T04:32:36.558 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-21T04:32:36.558 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T04:32:36.559 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:36.559 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-21T04:32:36.559 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T04:32:36.559 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:36.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:36.559 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-09-21T04:32:36.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:36.582 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2024-09-21T04:32:36.657 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2024-09-21T04:32:36.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:32:36.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:36.659 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-21T04:32:36.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:36.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-21T04:32:36.738 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:36 smithi007 ceph-mon[27954]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-21T04:32:36.738 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:36.738 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:36.738 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:36 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/547042386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-21T04:32:36.738 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:36.738 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:32:36.738 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:36 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:36.744 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-09-21T04:32:36.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:32:36.746 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-21T04:32:36.746 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T04:32:36.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T04:32:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:36 smithi053 ceph-mon[35554]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 4.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-21T04:32:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:36 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/547042386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-21T04:32:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:32:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:36 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:37.054 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.524+0000 7fa11969c640 1 -- 172.21.15.7:0/3398662819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 msgr2=0x7fa11410b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.524+0000 7fa11969c640 1 --2- 172.21.15.7:0/3398662819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 0x7fa11410b920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0098b0 tx=0x7fa0fc02a560 comp rx=0 tx=0).stop 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.524+0000 7fa11969c640 1 -- 172.21.15.7:0/3398662819 shutdown_connections 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.524+0000 7fa11969c640 1 --2- 172.21.15.7:0/3398662819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 0x7fa11410b920 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.524+0000 7fa11969c640 1 --2- 172.21.15.7:0/3398662819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa114102b10 0x7fa114102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.524+0000 7fa11969c640 1 -- 172.21.15.7:0/3398662819 >> 172.21.15.7:0/3398662819 conn(0x7fa1140fc550 msgr2=0x7fa1140fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.524+0000 7fa11969c640 1 -- 172.21.15.7:0/3398662819 shutdown_connections 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.524+0000 7fa11969c640 1 -- 172.21.15.7:0/3398662819 wait complete. 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa11969c640 1 Processor -- start 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa11969c640 1 -- start start 2024-09-21T04:32:37.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa11969c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa114102b10 0x7fa114100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa11969c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 0x7fa114100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa11969c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa11410edc0 con 0x7fa114103430 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa11969c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa11410ec40 con 0x7fa114102b10 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa1127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 0x7fa114100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa1127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 0x7fa114100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47210/0 (socket says 172.21.15.7:47210) 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa1127fc640 1 -- 172.21.15.7:0/1823520628 learned_addr learned my addr 172.21.15.7:0/1823520628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.525+0000 7fa112ffd640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa114102b10 0x7fa114100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa1127fc640 1 -- 172.21.15.7:0/1823520628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa114102b10 msgr2=0x7fa114100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa1127fc640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa114102b10 0x7fa114100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa1127fc640 1 -- 172.21.15.7:0/1823520628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1141011f0 con 0x7fa114103430 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa112ffd640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa114102b10 0x7fa114100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa1127fc640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 0x7fa114100cb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc031460 tx=0x7fa0fc0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa10bfff640 1 -- 172.21.15.7:0/1823520628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0fc03f070 con 0x7fa114103430 2024-09-21T04:32:37.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa10bfff640 1 -- 172.21.15.7:0/1823520628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa0fc0035f0 con 0x7fa114103430 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa114101480 con 0x7fa114103430 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa10bfff640 1 -- 172.21.15.7:0/1823520628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0fc035030 con 0x7fa114103430 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.526+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1141a3db0 con 0x7fa114103430 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.528+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11406a520 con 0x7fa114103430 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.528+0000 7fa10bfff640 1 -- 172.21.15.7:0/1823520628 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fa0fc042070 con 0x7fa114103430 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.529+0000 7fa10bfff640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa10006e2b0 0x7fa100070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.529+0000 7fa10bfff640 1 -- 172.21.15.7:0/1823520628 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7fa0fc0aea90 con 0x7fa114103430 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.529+0000 7fa112ffd640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa10006e2b0 0x7fa100070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.529+0000 7fa112ffd640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa10006e2b0 0x7fa100070770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa10400bfd0 tx=0x7fa10400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.534+0000 7fa10bfff640 1 -- 172.21.15.7:0/1823520628 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0fc080040 con 0x7fa114103430 2024-09-21T04:32:37.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.749+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa114105fa0 con 0x7fa114103430 2024-09-21T04:32:37.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.749+0000 7fa10bfff640 1 -- 172.21.15.7:0/1823520628 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+83 (secure 0 0 0) 0x7fa0fc084870 con 0x7fa114103430 2024-09-21T04:32:37.754 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-21T04:32:37.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.751+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa10006e2b0 msgr2=0x7fa100070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa10006e2b0 0x7fa100070770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa10400bfd0 tx=0x7fa10400b390 comp rx=0 tx=0).stop 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 msgr2=0x7fa114100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 0x7fa114100cb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc031460 tx=0x7fa0fc0045c0 comp rx=0 tx=0).stop 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 shutdown_connections 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fa10006e2b0 0x7fa100070770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa114103430 0x7fa114100cb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 --2- 172.21.15.7:0/1823520628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa114102b10 0x7fa114100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 >> 172.21.15.7:0/1823520628 conn(0x7fa1140fc550 msgr2=0x7fa114103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:37.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 shutdown_connections 2024-09-21T04:32:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:37.752+0000 7fa11969c640 1 -- 172.21.15.7:0/1823520628 wait complete. 2024-09-21T04:32:38.095 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:37 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:32:38.113 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:37 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:32:38.520 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T04:32:38.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T04:32:38.520 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-21T04:32:38.520 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T04:32:38.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:38.521 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T04:32:38.521 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T04:32:38.521 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T04:32:38.521 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T04:32:38.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:38.521 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-21T04:32:38.538 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T04:32:38.538 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:32:38.539 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-21T04:32:38.540 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-21T04:32:38.540 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T04:32:38.540 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-21T04:32:38.540 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-21T04:32:38.540 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T04:32:38.540 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42333sec preferred_lft 42333sec 2024-09-21T04:32:38.540 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-21T04:32:38.540 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:32:38.541 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:32:38.541 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-21T04:32:38.541 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-21T04:32:38.541 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T04:32:38.542 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:38.542 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-21T04:32:38.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-09-21T04:32:38.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-09-21T04:32:38.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-09-21T04:32:38.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T04:32:38.542 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-21T04:32:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:38.768 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:38 smithi007 ceph-mon[27954]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T04:32:38.769 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:38 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1823520628' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-21T04:32:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:38.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:38.824 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T04:32:38.850 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-21T04:32:38.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:38.852 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-21T04:32:38.948 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-21T04:32:38.948 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T04:32:38.948 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T04:32:38.948 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T04:32:38.948 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T04:32:38.948 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:38.948 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-21T04:32:38.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T04:32:38.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T04:32:38.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T04:32:38.949 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-21T04:32:39.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:39.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:38 smithi053 ceph-mon[35554]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T04:32:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:38 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1823520628' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-21T04:32:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:39.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:39.203 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-21T04:32:39.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:39.221 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-21T04:32:39.278 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:39.278 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-21T04:32:39.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:32:39.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T04:32:39.278 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-21T04:32:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:39.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:39.397 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T04:32:39.397 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:39.397 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-21T04:32:39.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T04:32:39.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T04:32:39.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T04:32:39.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T04:32:39.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T04:32:39.397 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-21T04:32:39.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:39.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:39.540 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:39.541 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-21T04:32:39.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-21T04:32:39.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T04:32:39.541 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-21T04:32:39.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:39.622 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-21T04:32:39.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:32:39.623 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:39.643 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T04:32:39.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:32:39.643 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:39.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-21T04:32:39.783 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T04:32:39.843 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-09-21T04:32:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-09-21T04:32:39.845 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4d2bb75872f3e453fdd628348f8d28cb4f5da5d0e1b469a778cf9bfc79395c53/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/041bad7c36c3589711a5f181724a733d979044a5a6c54659790eba7ebc67918c/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/40b3652793c6e145455e9955e87ec3ece3ec10653caac58183e5e02700d852bd/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/be6f489203032c61600a1b9d720b8e1589371c99ba5d5dc3db1e7b250c66facf/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/c9e6c852d420c68bbedcab28f38e8d083b5c8ee6425a884289449887195c73d5/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f7f946f3c7f23443474399c928ba53f06fea6c7326d7a84b2d68420fdcebb151/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/555a98c0de1e0576b4a9acf8a7f3dfd1cfe7eafa7b03525722f36528d62df847/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/71bd0791d235c8b4c8d8960c6bd687ec9fbe95c16eae88728745b24ffca06654/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/405f0a96934574f597658f445647b2e6a5fb1884a66b7718d38253d7237d4e9d/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/e58f2c594ebbea326646428e3748bb83dab839c627f8a28ce30ec7708a12f183/merged 2024-09-21T04:32:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/703b8ec9654053c3e70541a349de15f4db984102536fdce604656baff4cf0228/merged 2024-09-21T04:32:39.847 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/9d066c00a732883b0d7880ac4b59aa631a201b3f2f01376934500301a4d46df5/merged 2024-09-21T04:32:39.847 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0e76abce49a46e07b69166cf7de199cf7a6755904500030a1129bcf95e3ab420/merged 2024-09-21T04:32:39.847 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-09-21T04:32:39.847 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:32:39.847 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:32:39.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:39.848 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-21T04:32:39.906 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T04:32:39.907 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-21T04:32:39.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-21T04:32:39.979 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T04:32:40.036 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-09-21T04:32:40.036 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-09-21T04:32:40.036 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-09-21T04:32:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-09-21T04:32:40.038 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T04:32:40.038 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T04:32:40.038 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T04:32:40.038 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T04:32:40.038 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4d2bb75872f3e453fdd628348f8d28cb4f5da5d0e1b469a778cf9bfc79395c53/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/041bad7c36c3589711a5f181724a733d979044a5a6c54659790eba7ebc67918c/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/40b3652793c6e145455e9955e87ec3ece3ec10653caac58183e5e02700d852bd/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/be6f489203032c61600a1b9d720b8e1589371c99ba5d5dc3db1e7b250c66facf/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/c9e6c852d420c68bbedcab28f38e8d083b5c8ee6425a884289449887195c73d5/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f7f946f3c7f23443474399c928ba53f06fea6c7326d7a84b2d68420fdcebb151/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/555a98c0de1e0576b4a9acf8a7f3dfd1cfe7eafa7b03525722f36528d62df847/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/71bd0791d235c8b4c8d8960c6bd687ec9fbe95c16eae88728745b24ffca06654/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/405f0a96934574f597658f445647b2e6a5fb1884a66b7718d38253d7237d4e9d/merged 2024-09-21T04:32:40.039 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/e58f2c594ebbea326646428e3748bb83dab839c627f8a28ce30ec7708a12f183/merged 2024-09-21T04:32:40.040 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/703b8ec9654053c3e70541a349de15f4db984102536fdce604656baff4cf0228/merged 2024-09-21T04:32:40.040 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/9d066c00a732883b0d7880ac4b59aa631a201b3f2f01376934500301a4d46df5/merged 2024-09-21T04:32:40.040 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0e76abce49a46e07b69166cf7de199cf7a6755904500030a1129bcf95e3ab420/merged 2024-09-21T04:32:40.040 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-09-21T04:32:40.040 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:32:40.040 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:32:40.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:40.040 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-21T04:32:40.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2024-09-21T04:32:40.129+0000 7f316362e480 -1 init, newargv = 0x56375bbee050 newargc=15 2024-09-21T04:32:40.133 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[75710]: starting ceph client 2024-09-21T04:32:40.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[75710]: starting fuse 2024-09-21T04:32:40.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:39 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:40.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:39 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:39 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:41.146 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:40 smithi007 ceph-mon[27954]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T04:32:41.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:40 smithi053 ceph-mon[35554]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T04:32:41.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-21T04:32:41.222 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T04:32:41.279 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-09-21T04:32:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/4d2bb75872f3e453fdd628348f8d28cb4f5da5d0e1b469a778cf9bfc79395c53/merged 2024-09-21T04:32:41.281 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/041bad7c36c3589711a5f181724a733d979044a5a6c54659790eba7ebc67918c/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/0 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/40b3652793c6e145455e9955e87ec3ece3ec10653caac58183e5e02700d852bd/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/be6f489203032c61600a1b9d720b8e1589371c99ba5d5dc3db1e7b250c66facf/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/c9e6c852d420c68bbedcab28f38e8d083b5c8ee6425a884289449887195c73d5/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/f7f946f3c7f23443474399c928ba53f06fea6c7326d7a84b2d68420fdcebb151/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/555a98c0de1e0576b4a9acf8a7f3dfd1cfe7eafa7b03525722f36528d62df847/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/71bd0791d235c8b4c8d8960c6bd687ec9fbe95c16eae88728745b24ffca06654/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/405f0a96934574f597658f445647b2e6a5fb1884a66b7718d38253d7237d4e9d/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/e58f2c594ebbea326646428e3748bb83dab839c627f8a28ce30ec7708a12f183/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/703b8ec9654053c3e70541a349de15f4db984102536fdce604656baff4cf0228/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/9d066c00a732883b0d7880ac4b59aa631a201b3f2f01376934500301a4d46df5/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/containers/storage/overlay/0e76abce49a46e07b69166cf7de199cf7a6755904500030a1129bcf95e3ab420/merged 2024-09-21T04:32:41.282 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-09-21T04:32:41.283 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:32:41.283 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:32:41.283 INFO:teuthology.orchestra.run.smithi007.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:41.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:41.283 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-21T04:32:41.336 INFO:teuthology.orchestra.run.smithi007.stdout:165 2024-09-21T04:32:41.336 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [165] 2024-09-21T04:32:41.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo stdin-killer -- python3 -c ' 2024-09-21T04:32:41.336 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-09-21T04:32:41.336 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-09-21T04:32:41.336 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-21T04:32:41.336 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-09-21T04:32:41.336 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-21T04:32:41.336 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T04:32:41.337 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-21T04:32:41.338 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2024-09-21T04:32:41.338 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-21T04:32:41.467 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.75710.asok 2024-09-21T04:32:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:41.479 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75710.asok 2024-09-21T04:32:41.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:41.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75710.asok status 2024-09-21T04:32:41.616 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:32:41.616 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-09-21T04:32:41.616 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "4e45e6b490e6cbe082cdf155fea780438ec53fc7", 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)", 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "75710", 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 14614, 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 14614 2024-09-21T04:32:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1658045521 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1658045521 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.14614 192.168.0.1:0/1658045521", 2024-09-21T04:32:41.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/1658045521", 2024-09-21T04:32:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-09-21T04:32:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 19, 2024-09-21T04:32:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 44, 2024-09-21T04:32:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-09-21T04:32:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-09-21T04:32:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-09-21T04:32:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:32:41.625 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-21T04:32:41.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T04:32:42.248 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.670+0000 7fb7a463e640 1 -- 172.21.15.7:0/417301575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 msgr2=0x7fb79c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.670+0000 7fb7a463e640 1 --2- 172.21.15.7:0/417301575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 0x7fb79c107ce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb798009880 tx=0x7fb79802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.670+0000 7fb7a463e640 1 -- 172.21.15.7:0/417301575 shutdown_connections 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.671+0000 7fb7a463e640 1 --2- 172.21.15.7:0/417301575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb79c101900 0x7fb79c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.671+0000 7fb7a463e640 1 --2- 172.21.15.7:0/417301575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 0x7fb79c107ce0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.671+0000 7fb7a463e640 1 -- 172.21.15.7:0/417301575 >> 172.21.15.7:0/417301575 conn(0x7fb79c0fd600 msgr2=0x7fb79c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.671+0000 7fb7a463e640 1 -- 172.21.15.7:0/417301575 shutdown_connections 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.671+0000 7fb7a463e640 1 -- 172.21.15.7:0/417301575 wait complete. 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.671+0000 7fb7a463e640 1 Processor -- start 2024-09-21T04:32:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.671+0000 7fb7a463e640 1 -- start start 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a463e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb79c101900 0x7fb79c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a463e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 0x7fb79c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a463e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10edb0 con 0x7fb79c107900 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a463e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10ec30 con 0x7fb79c101900 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a1bd9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 0x7fb79c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a23da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb79c101900 0x7fb79c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a1bd9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 0x7fb79c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47226/0 (socket says 172.21.15.7:47226) 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a1bd9640 1 -- 172.21.15.7:0/493753571 learned_addr learned my addr 172.21.15.7:0/493753571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a1bd9640 1 -- 172.21.15.7:0/493753571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb79c101900 msgr2=0x7fb79c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a1bd9640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb79c101900 0x7fb79c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a1bd9640 1 -- 172.21.15.7:0/493753571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c1977c0 con 0x7fb79c107900 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7a1bd9640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 0x7fb79c19db80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00d8d0 tx=0x7fb78c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.672+0000 7fb7937fe640 1 -- 172.21.15.7:0/493753571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb78c017070 con 0x7fb79c107900 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.673+0000 7fb7937fe640 1 -- 172.21.15.7:0/493753571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb78c010460 con 0x7fb79c107900 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.673+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c197ab0 con 0x7fb79c107900 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.673+0000 7fb7937fe640 1 -- 172.21.15.7:0/493753571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb78c00f5f0 con 0x7fb79c107900 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.673+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c198040 con 0x7fb79c107900 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.674+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c103040 con 0x7fb79c107900 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.674+0000 7fb7937fe640 1 -- 172.21.15.7:0/493753571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fb78c009a30 con 0x7fb79c107900 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.675+0000 7fb7937fe640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb78806e2b0 0x7fb788070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.675+0000 7fb7937fe640 1 -- 172.21.15.7:0/493753571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7fb78c08e020 con 0x7fb79c107900 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.675+0000 7fb7a23da640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb78806e2b0 0x7fb788070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.675+0000 7fb7a23da640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb78806e2b0 0x7fb788070770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7980096c0 tx=0x7fb798031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.679+0000 7fb7937fe640 1 -- 172.21.15.7:0/493753571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb78c05f5d0 con 0x7fb79c107900 2024-09-21T04:32:42.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.901+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb79c104b10 con 0x7fb79c107900 2024-09-21T04:32:42.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.901+0000 7fb7937fe640 1 -- 172.21.15.7:0/493753571 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+83 (secure 0 0 0) 0x7fb78c063e00 con 0x7fb79c107900 2024-09-21T04:32:42.905 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-21T04:32:42.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb78806e2b0 msgr2=0x7fb788070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb78806e2b0 0x7fb788070770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7980096c0 tx=0x7fb798031400 comp rx=0 tx=0).stop 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 msgr2=0x7fb79c19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 0x7fb79c19db80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00d8d0 tx=0x7fb78c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 shutdown_connections 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fb78806e2b0 0x7fb788070770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb79c107900 0x7fb79c19db80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 --2- 172.21.15.7:0/493753571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb79c101900 0x7fb79c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 >> 172.21.15.7:0/493753571 conn(0x7fb79c0fd600 msgr2=0x7fb79c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 shutdown_connections 2024-09-21T04:32:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:42.904+0000 7fb7a463e640 1 -- 172.21.15.7:0/493753571 wait complete. 2024-09-21T04:32:42.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:42 smithi007 ceph-mon[27954]: pgmap v187: 65 pgs: 65 active+clean; 581 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 274 B/s wr, 12 op/s 2024-09-21T04:32:43.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:42 smithi053 ceph-mon[35554]: pgmap v187: 65 pgs: 65 active+clean; 581 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 274 B/s wr, 12 op/s 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T04:32:43.532 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T04:32:43.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:43.533 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2024-09-21T04:32:43.549 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T04:32:43.549 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T04:32:43.549 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T04:32:43.549 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:32:43.549 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2024-09-21T04:32:43.549 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:32:43.549 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42309sec preferred_lft 42309sec 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:32:43.550 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:32:43.551 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2024-09-21T04:32:43.551 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T04:32:43.551 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:43.551 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-21T04:32:43.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2024-09-21T04:32:43.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2024-09-21T04:32:43.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2024-09-21T04:32:43.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T04:32:43.551 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-21T04:32:43.639 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:43.773 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:43.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:43.780 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T04:32:43.805 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-09-21T04:32:43.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:43.807 DEBUG:teuthology.orchestra.run.smithi053:> route 2024-09-21T04:32:43.883 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2024-09-21T04:32:43.883 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T04:32:43.883 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T04:32:43.883 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T04:32:43.883 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T04:32:43.883 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:43.883 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-21T04:32:43.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T04:32:43.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T04:32:43.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T04:32:43.884 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-21T04:32:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:43 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:43 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:43 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:43 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:32:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:43 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:43.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:43 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/493753571' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-21T04:32:43.973 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:44.177 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:44.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:44.184 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2024-09-21T04:32:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:32:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:32:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:43 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:43 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/493753571' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-21T04:32:44.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:44.241 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2024-09-21T04:32:44.296 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:44.296 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-21T04:32:44.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:32:44.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T04:32:44.297 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-21T04:32:44.385 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:44.409 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:44.415 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T04:32:44.415 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:44.416 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-21T04:32:44.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T04:32:44.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T04:32:44.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T04:32:44.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T04:32:44.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T04:32:44.416 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-21T04:32:44.465 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:44.549 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:32:44.549 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2024-09-21T04:32:44.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2024-09-21T04:32:44.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T04:32:44.549 DEBUG:teuthology.orchestra.run.smithi053:> ') 2024-09-21T04:32:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:32:44.621 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:44.628 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-21T04:32:44.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:32:44.628 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:44.643 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T04:32:44.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:32:44.643 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:44.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2024-09-21T04:32:44.765 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T04:32:44.822 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2024-09-21T04:32:44.822 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2024-09-21T04:32:44.822 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2024-09-21T04:32:44.822 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2024-09-21T04:32:44.822 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2024-09-21T04:32:44.822 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2024-09-21T04:32:44.822 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/bpf 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T04:32:44.823 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/0 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/b3e2d7dc39a9266bed165ce7023b41a45cea58f36afde1b5c3383cbb7cefd910/merged 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/5beb9889b4798389318f5a97df3f69ba16e061e536f2000d7dec8b4ca126ce52/merged 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/f3633bca930183725d45bfc7d4a464eaad03b3bed9b3d649ee09286c473b48eb/merged 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/7b93bf9685cee36e2b32ab6e33a18ef05f3ccd88539d4a7a6f20a1c51f873b0c/merged 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/559594843ad6499edd278783ac09049f2b16a3b894f336e414b08274af9c7b82/merged 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/5aaac1de1fcc59ce84259391f7f83bb2ab1c602da75a2548b6005157011d190e/merged 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/fb63c8369bb344393c0d6debf1e81bf0f869322027ea9007fe64330e185cced6/merged 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/f68b15612ae9328558f9a5543bbb55d83c7166f31c44523f8b0d458a6baf7c8b/merged 2024-09-21T04:32:44.824 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/21e555d790696e2b8bab8b4754b3aae1ffc64a94f225a03c258db88d31b63bd5/merged 2024-09-21T04:32:44.825 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/297561b4421302ca090e1ad29d6927adaa13f2a64811f27c66e9fb2227585410/merged 2024-09-21T04:32:44.825 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2024-09-21T04:32:44.825 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:32:44.825 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:32:44.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:44.825 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2024-09-21T04:32:44.878 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T04:32:44.878 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-21T04:32:44.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2024-09-21T04:32:44.945 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T04:32:45.002 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/bpf 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2024-09-21T04:32:45.003 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/0 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/b3e2d7dc39a9266bed165ce7023b41a45cea58f36afde1b5c3383cbb7cefd910/merged 2024-09-21T04:32:45.004 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/5beb9889b4798389318f5a97df3f69ba16e061e536f2000d7dec8b4ca126ce52/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/f3633bca930183725d45bfc7d4a464eaad03b3bed9b3d649ee09286c473b48eb/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/7b93bf9685cee36e2b32ab6e33a18ef05f3ccd88539d4a7a6f20a1c51f873b0c/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/559594843ad6499edd278783ac09049f2b16a3b894f336e414b08274af9c7b82/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/5aaac1de1fcc59ce84259391f7f83bb2ab1c602da75a2548b6005157011d190e/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/fb63c8369bb344393c0d6debf1e81bf0f869322027ea9007fe64330e185cced6/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/f68b15612ae9328558f9a5543bbb55d83c7166f31c44523f8b0d458a6baf7c8b/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/21e555d790696e2b8bab8b4754b3aae1ffc64a94f225a03c258db88d31b63bd5/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/297561b4421302ca090e1ad29d6927adaa13f2a64811f27c66e9fb2227585410/merged 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:32:45.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:45.005 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2024-09-21T04:32:45.058 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi053.stderr:2024-09-21T04:32:45.056+0000 7f427381d480 -1 init, newargv = 0x56126a2f4140 newargc=15 2024-09-21T04:32:45.058 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi053.stderr:ceph-fuse[61101]: starting ceph client 2024-09-21T04:32:45.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi053.stderr:ceph-fuse[61101]: starting fuse 2024-09-21T04:32:45.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:44 smithi053 ceph-mon[35554]: pgmap v188: 65 pgs: 65 active+clean; 581 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T04:32:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:44 smithi007 ceph-mon[27954]: pgmap v188: 65 pgs: 65 active+clean; 581 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T04:32:46.161 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2024-09-21T04:32:46.195 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/bpf 2024-09-21T04:32:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2024-09-21T04:32:46.256 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/0 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/b3e2d7dc39a9266bed165ce7023b41a45cea58f36afde1b5c3383cbb7cefd910/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/5beb9889b4798389318f5a97df3f69ba16e061e536f2000d7dec8b4ca126ce52/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/f3633bca930183725d45bfc7d4a464eaad03b3bed9b3d649ee09286c473b48eb/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/7b93bf9685cee36e2b32ab6e33a18ef05f3ccd88539d4a7a6f20a1c51f873b0c/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/559594843ad6499edd278783ac09049f2b16a3b894f336e414b08274af9c7b82/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/5aaac1de1fcc59ce84259391f7f83bb2ab1c602da75a2548b6005157011d190e/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/fb63c8369bb344393c0d6debf1e81bf0f869322027ea9007fe64330e185cced6/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/f68b15612ae9328558f9a5543bbb55d83c7166f31c44523f8b0d458a6baf7c8b/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/21e555d790696e2b8bab8b4754b3aae1ffc64a94f225a03c258db88d31b63bd5/merged 2024-09-21T04:32:46.257 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/containers/storage/overlay/297561b4421302ca090e1ad29d6927adaa13f2a64811f27c66e9fb2227585410/merged 2024-09-21T04:32:46.258 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2024-09-21T04:32:46.258 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:32:46.258 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T04:32:46.258 INFO:teuthology.orchestra.run.smithi053.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:46.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:46.259 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2024-09-21T04:32:46.317 INFO:teuthology.orchestra.run.smithi053.stdout:129 2024-09-21T04:32:46.317 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [129] 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo stdin-killer -- python3 -c ' 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> import glob 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> import re 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> import os 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> import subprocess 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> def _find_admin_socket(client_name): 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> files = glob.glob(asok_path) 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> # Given a non-glob path, it better be there 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> if "*" not in asok_path: 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> assert(len(files) == 1) 2024-09-21T04:32:46.318 DEBUG:teuthology.orchestra.run.smithi053:> return files[0] 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> for f in files: 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> contents = proc_f.read() 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> if mountpoint in contents: 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> return f 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> print(_find_admin_socket("client.1")) 2024-09-21T04:32:46.319 DEBUG:teuthology.orchestra.run.smithi053:> ' 2024-09-21T04:32:46.446 INFO:teuthology.orchestra.run.smithi053.stdout:/var/run/ceph/ceph-client.1.61101.asok 2024-09-21T04:32:46.449 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-21T04:32:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:32:46.457 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61101.asok 2024-09-21T04:32:46.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:46.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61101.asok status 2024-09-21T04:32:46.596 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-21T04:32:46.596 INFO:teuthology.orchestra.run.smithi053.stdout: "metadata": { 2024-09-21T04:32:46.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_sha1": "4e45e6b490e6cbe082cdf155fea780438ec53fc7", 2024-09-21T04:32:46.596 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)", 2024-09-21T04:32:46.596 INFO:teuthology.orchestra.run.smithi053.stdout: "entity_id": "1", 2024-09-21T04:32:46.596 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": "smithi053", 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "pid": "61101", 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "root": "/" 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_count": 0, 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_pinned_count": 0, 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 24375, 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "inst": { 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "name": { 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "client", 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "num": 24375 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "192.168.0.1:0", 2024-09-21T04:32:46.597 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3970860440 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "192.168.0.1:0", 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3970860440 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "inst_str": "client.24375 192.168.0.1:0/3970860440", 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "addr_str": "192.168.0.1:0/3970860440", 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "inode_count": 1, 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_epoch": 19, 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch": 44, 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch_barrier": 0, 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklisted": false, 2024-09-21T04:32:46.598 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs" 2024-09-21T04:32:46.599 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-21T04:32:46.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:46.605 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-09-21T04:32:46.605 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:46.625 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-09-21T04:32:46.625 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:46.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:46.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:46.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:46.700 DEBUG:teuthology.orchestra.run.smithi053:> stat --file-system '--printf=%T 2024-09-21T04:32:46.700 DEBUG:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:46.716 INFO:teuthology.orchestra.run.smithi053.stdout:fuseblk 2024-09-21T04:32:46.716 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:46.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:32:46.717 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:46.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:46 smithi007 ceph-mon[27954]: pgmap v189: 65 pgs: 65 active+clean; 581 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 8.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T04:32:46.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:46.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:32:47.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:46 smithi053 ceph-mon[35554]: pgmap v189: 65 pgs: 65 active+clean; 581 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 8.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T04:32:47.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:32:47.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:32:49.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:48 smithi053 ceph-mon[35554]: pgmap v190: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T04:32:49.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:48 smithi007 ceph-mon[27954]: pgmap v190: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T04:32:50.769 INFO:teuthology.run_tasks:Running task print... 2024-09-21T04:32:50.780 INFO:teuthology.task.print:**** done client 2024-09-21T04:32:50.780 INFO:teuthology.run_tasks:Running task parallel... 2024-09-21T04:32:50.803 INFO:teuthology.task.parallel:starting parallel... 2024-09-21T04:32:50.803 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-21T04:32:50.804 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-21T04:32:50.804 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:32:50.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-21T04:32:50.805 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-21T04:32:50.805 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-09-21T04:32:50.808 INFO:tasks.workunit:Pulling workunits from ref f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:32:50.809 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T04:32:50.809 INFO:tasks.workunit:timeout=3h 2024-09-21T04:32:50.809 INFO:tasks.workunit:cleanup=True 2024-09-21T04:32:50.809 DEBUG:teuthology.orchestra.run.smithi007:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout:Device: a5h/165d Inode: 1 Links: 2 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-21 04:32:20.378661374 +0000 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-21 04:32:46.694582788 +0000 2024-09-21T04:32:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-21T04:32:50.832 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T04:32:50.832 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T04:32:50.907 DEBUG:teuthology.orchestra.run.smithi053:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-21T04:32:50.929 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 81h/129d Inode: 1 Links: 3 2024-09-21T04:32:50.929 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:32:50.929 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-21T04:32:50.929 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-21T04:32:50.929 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-21 04:32:50.897906529 +0000 2024-09-21T04:32:50.929 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-21 04:32:50.897906529 +0000 2024-09-21T04:32:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-09-21T04:32:50.929 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-21T04:32:50.929 DEBUG:teuthology.orchestra.run.smithi053:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-21T04:32:51.004 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:32:51.005 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:32:51.022 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T04:32:51.062 INFO:tasks.workunit.client.1.smithi053.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-21T04:32:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:51.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:50 smithi053 ceph-mon[35554]: pgmap v191: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-21T04:32:51.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:50 smithi007 ceph-mon[27954]: pgmap v191: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-21T04:32:51.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.576+0000 7fab11a9f640 1 -- 172.21.15.7:0/2623335808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c100b20 msgr2=0x7fab0c100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:51.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.576+0000 7fab11a9f640 1 --2- 172.21.15.7:0/2623335808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c100b20 0x7fab0c100f80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fab00009880 tx=0x7fab0002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.577+0000 7fab11a9f640 1 -- 172.21.15.7:0/2623335808 shutdown_connections 2024-09-21T04:32:51.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.577+0000 7fab11a9f640 1 --2- 172.21.15.7:0/2623335808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c100b20 0x7fab0c100f80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.577+0000 7fab11a9f640 1 --2- 172.21.15.7:0/2623335808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c1001e0 0x7fab0c1005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.577+0000 7fab11a9f640 1 -- 172.21.15.7:0/2623335808 >> 172.21.15.7:0/2623335808 conn(0x7fab0c0fc010 msgr2=0x7fab0c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.577+0000 7fab11a9f640 1 -- 172.21.15.7:0/2623335808 shutdown_connections 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.577+0000 7fab11a9f640 1 -- 172.21.15.7:0/2623335808 wait complete. 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.577+0000 7fab11a9f640 1 Processor -- start 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.577+0000 7fab11a9f640 1 -- start start 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab11a9f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c1001e0 0x7fab0c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab11a9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c100b20 0x7fab0c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab11a9f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10c9f0 con 0x7fab0c1001e0 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab11a9f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c10c870 con 0x7fab0c100b20 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c1001e0 0x7fab0c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c1001e0 0x7fab0c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52904/0 (socket says 172.21.15.7:52904) 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab0b7fe640 1 -- 172.21.15.7:0/349592859 learned_addr learned my addr 172.21.15.7:0/349592859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab0affd640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c100b20 0x7fab0c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:51.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab0b7fe640 1 -- 172.21.15.7:0/349592859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c100b20 msgr2=0x7fab0c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab0b7fe640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c100b20 0x7fab0c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab0b7fe640 1 -- 172.21.15.7:0/349592859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c19a2e0 con 0x7fab0c1001e0 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab0b7fe640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c1001e0 0x7fab0c199720 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b420 tx=0x7faafc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab08ff9640 1 -- 172.21.15.7:0/349592859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faafc011630 con 0x7fab0c1001e0 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c19ed20 con 0x7fab0c1001e0 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab08ff9640 1 -- 172.21.15.7:0/349592859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faafc016070 con 0x7fab0c1001e0 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab08ff9640 1 -- 172.21.15.7:0/349592859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faafc010d40 con 0x7fab0c1001e0 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.578+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c19f240 con 0x7fab0c1001e0 2024-09-21T04:32:51.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.580+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c1005e0 con 0x7fab0c1001e0 2024-09-21T04:32:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.580+0000 7fab08ff9640 1 -- 172.21.15.7:0/349592859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7faafc0117d0 con 0x7fab0c1001e0 2024-09-21T04:32:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.580+0000 7fab08ff9640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faaf406e2b0 0x7faaf4070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.581+0000 7fab08ff9640 1 -- 172.21.15.7:0/349592859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7faafc08da90 con 0x7fab0c1001e0 2024-09-21T04:32:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.581+0000 7fab0affd640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faaf406e2b0 0x7faaf4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.581+0000 7fab0affd640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faaf406e2b0 0x7faaf4070770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fab00000c00 tx=0x7fab000058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.585+0000 7fab08ff9640 1 -- 172.21.15.7:0/349592859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faafc05b2f0 con 0x7fab0c1001e0 2024-09-21T04:32:51.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.762+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) -- 0x7fab0c193800 con 0x7fab0c1001e0 2024-09-21T04:32:51.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.762+0000 7fab08ff9640 1 -- 172.21.15.7:0/349592859 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v41)=0 v41) ==== 155+0+0 (secure 0 0 0) 0x7faafc05efa0 con 0x7fab0c1001e0 2024-09-21T04:32:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faaf406e2b0 msgr2=0x7faaf4070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faaf406e2b0 0x7faaf4070770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fab00000c00 tx=0x7fab000058d0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c1001e0 msgr2=0x7fab0c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c1001e0 0x7fab0c199720 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b420 tx=0x7faafc00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 shutdown_connections 2024-09-21T04:32:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faaf406e2b0 0x7faaf4070770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab0c100b20 0x7fab0c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 --2- 172.21.15.7:0/349592859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0c1001e0 0x7fab0c199720 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.764+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 >> 172.21.15.7:0/349592859 conn(0x7fab0c0fc010 msgr2=0x7fab0c108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.765+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 shutdown_connections 2024-09-21T04:32:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:51.765+0000 7fab11a9f640 1 -- 172.21.15.7:0/349592859 wait complete. 2024-09-21T04:32:52.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-21T04:32:52.741 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:53.156 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:52 smithi007 ceph-mon[27954]: pgmap v192: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T04:32:53.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:52 smithi053 ceph-mon[35554]: pgmap v192: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-21T04:32:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.243+0000 7f7f23d31640 1 -- 172.21.15.7:0/1888309845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 msgr2=0x7f7f1c101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.243+0000 7f7f23d31640 1 --2- 172.21.15.7:0/1888309845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 0x7f7f1c101ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009880 tx=0x7f7f0c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.243+0000 7f7f23d31640 1 -- 172.21.15.7:0/1888309845 shutdown_connections 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.243+0000 7f7f23d31640 1 --2- 172.21.15.7:0/1888309845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 0x7f7f1c101ee0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.243+0000 7f7f23d31640 1 --2- 172.21.15.7:0/1888309845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f1c100860 0x7f7f1c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.243+0000 7f7f23d31640 1 -- 172.21.15.7:0/1888309845 >> 172.21.15.7:0/1888309845 conn(0x7f7f1c0fc010 msgr2=0x7f7f1c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f23d31640 1 -- 172.21.15.7:0/1888309845 shutdown_connections 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f23d31640 1 -- 172.21.15.7:0/1888309845 wait complete. 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f23d31640 1 Processor -- start 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f23d31640 1 -- start start 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f23d31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f1c100860 0x7f7f1c197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f23d31640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 0x7f7f1c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f23d31640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c109720 con 0x7f7f1c101a60 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f23d31640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1095a0 con 0x7f7f1c100860 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f212cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 0x7f7f1c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f212cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 0x7f7f1c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52922/0 (socket says 172.21.15.7:52922) 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.244+0000 7f7f212cc640 1 -- 172.21.15.7:0/695929901 learned_addr learned my addr 172.21.15.7:0/695929901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f21acd640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f1c100860 0x7f7f1c197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f212cc640 1 -- 172.21.15.7:0/695929901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f1c100860 msgr2=0x7f7f1c197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f212cc640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f1c100860 0x7f7f1c197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f212cc640 1 -- 172.21.15.7:0/695929901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c198120 con 0x7f7f1c101a60 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f212cc640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 0x7f7f1c197aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c006010 tx=0x7f7f0c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f12ffd640 1 -- 172.21.15.7:0/695929901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7f0c03f070 con 0x7f7f1c101a60 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f12ffd640 1 -- 172.21.15.7:0/695929901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f0c0044d0 con 0x7f7f1c101a60 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c19cb00 con 0x7f7f1c101a60 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f12ffd640 1 -- 172.21.15.7:0/695929901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7f0c035030 con 0x7f7f1c101a60 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.245+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c19d110 con 0x7f7f1c101a60 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.246+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c100c40 con 0x7f7f1c101a60 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.247+0000 7f7f12ffd640 1 -- 172.21.15.7:0/695929901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f7f0c042070 con 0x7f7f1c101a60 2024-09-21T04:32:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.247+0000 7f7f12ffd640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f7f0806e2b0 0x7f7f08070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.248+0000 7f7f21acd640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f7f0806e2b0 0x7f7f08070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.248+0000 7f7f12ffd640 1 -- 172.21.15.7:0/695929901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f7f0c0aee80 con 0x7f7f1c101a60 2024-09-21T04:32:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.248+0000 7f7f21acd640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f7f0806e2b0 0x7f7f08070770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c1018c0 tx=0x7f7f18009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.252+0000 7f7f12ffd640 1 -- 172.21.15.7:0/695929901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7f0c07c780 con 0x7f7f1c101a60 2024-09-21T04:32:53.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.431+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) -- 0x7f7f1c19d8f0 con 0x7f7f1c101a60 2024-09-21T04:32:53.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.431+0000 7f7f12ffd640 1 -- 172.21.15.7:0/695929901 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v41)=0 v41) ==== 163+0+0 (secure 0 0 0) 0x7f7f0c080430 con 0x7f7f1c101a60 2024-09-21T04:32:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f7f0806e2b0 msgr2=0x7f7f08070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f7f0806e2b0 0x7f7f08070770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c1018c0 tx=0x7f7f18009110 comp rx=0 tx=0).stop 2024-09-21T04:32:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 msgr2=0x7f7f1c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 0x7f7f1c197aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c006010 tx=0x7f7f0c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:32:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 shutdown_connections 2024-09-21T04:32:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f7f0806e2b0 0x7f7f08070770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f1c101a60 0x7f7f1c197aa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:53.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 --2- 172.21.15.7:0/695929901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f1c100860 0x7f7f1c197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:53.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 >> 172.21.15.7:0/695929901 conn(0x7f7f1c0fc010 msgr2=0x7f7f1c069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:53.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 shutdown_connections 2024-09-21T04:32:53.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:53.434+0000 7f7f23d31640 1 -- 172.21.15.7:0/695929901 wait complete. 2024-09-21T04:32:54.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-21T04:32:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:54.906 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:54 smithi007 ceph-mon[27954]: pgmap v193: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:32:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/1253437853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 msgr2=0x7fe89c1071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 --2- 172.21.15.7:0/1253437853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 0x7fe89c1071d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe890009880 tx=0x7fe89002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/1253437853 shutdown_connections 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 --2- 172.21.15.7:0/1253437853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 0x7fe89c1071d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 --2- 172.21.15.7:0/1253437853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1024b0 0x7fe89c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/1253437853 >> 172.21.15.7:0/1253437853 conn(0x7fe89c0fc010 msgr2=0x7fe89c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/1253437853 shutdown_connections 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/1253437853 wait complete. 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.992+0000 7fe8a1ca3640 1 Processor -- start 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe8a1ca3640 1 -- start start 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe8a1ca3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1024b0 0x7fe89c197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe8a1ca3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 0x7fe89c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe8a1ca3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c10a780 con 0x7fe89c104de0 2024-09-21T04:32:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe8a1ca3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c10a600 con 0x7fe89c1024b0 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe89affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 0x7fe89c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe89affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 0x7fe89c197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57478/0 (socket says 172.21.15.7:57478) 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe89affd640 1 -- 172.21.15.7:0/3078280085 learned_addr learned my addr 172.21.15.7:0/3078280085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe89b7fe640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1024b0 0x7fe89c197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe89affd640 1 -- 172.21.15.7:0/3078280085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1024b0 msgr2=0x7fe89c197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe89affd640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1024b0 0x7fe89c197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe89affd640 1 -- 172.21.15.7:0/3078280085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c198040 con 0x7fe89c104de0 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.993+0000 7fe89b7fe640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1024b0 0x7fe89c197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.994+0000 7fe89affd640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 0x7fe89c197a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe890006010 tx=0x7fe8900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.994+0000 7fe898ff9640 1 -- 172.21.15.7:0/3078280085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe89003f070 con 0x7fe89c104de0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.994+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c19ca50 con 0x7fe89c104de0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.994+0000 7fe898ff9640 1 -- 172.21.15.7:0/3078280085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe8900044d0 con 0x7fe89c104de0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.994+0000 7fe898ff9640 1 -- 172.21.15.7:0/3078280085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe890035030 con 0x7fe89c104de0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.994+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c19cf50 con 0x7fe89c104de0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.995+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89c102890 con 0x7fe89c104de0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.995+0000 7fe898ff9640 1 -- 172.21.15.7:0/3078280085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fe890042070 con 0x7fe89c104de0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.996+0000 7fe898ff9640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fe88406e2b0 0x7fe884070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.996+0000 7fe898ff9640 1 -- 172.21.15.7:0/3078280085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7fe8900aee50 con 0x7fe89c104de0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.996+0000 7fe89b7fe640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fe88406e2b0 0x7fe884070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:54.996+0000 7fe89b7fe640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fe88406e2b0 0x7fe884070770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00abe0 tx=0x7fe88c00a5c0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.000+0000 7fe898ff9640 1 -- 172.21.15.7:0/3078280085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe89007c750 con 0x7fe89c104de0 2024-09-21T04:32:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.176+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) -- 0x7fe89c19d200 con 0x7fe89c104de0 2024-09-21T04:32:55.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.177+0000 7fe898ff9640 1 -- 172.21.15.7:0/3078280085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v41)=0 v41) ==== 135+0+0 (secure 0 0 0) 0x7fe890080400 con 0x7fe89c104de0 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fe88406e2b0 msgr2=0x7fe884070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fe88406e2b0 0x7fe884070770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00abe0 tx=0x7fe88c00a5c0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 msgr2=0x7fe89c197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 0x7fe89c197a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe890006010 tx=0x7fe8900048f0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 shutdown_connections 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fe88406e2b0 0x7fe884070770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe89c104de0 0x7fe89c197a50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 --2- 172.21.15.7:0/3078280085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1024b0 0x7fe89c197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 >> 172.21.15.7:0/3078280085 conn(0x7fe89c0fc010 msgr2=0x7fe89c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 shutdown_connections 2024-09-21T04:32:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:55.179+0000 7fe8a1ca3640 1 -- 172.21.15.7:0/3078280085 wait complete. 2024-09-21T04:32:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:54 smithi053 ceph-mon[35554]: pgmap v193: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:32:55.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-09-21T04:32:56.052 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:32:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.501+0000 7f714a2d0640 1 -- 172.21.15.7:0/478439070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144100860 msgr2=0x7f7144100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.501+0000 7f714a2d0640 1 --2- 172.21.15.7:0/478439070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144100860 0x7f7144100c60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7134009880 tx=0x7f713402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:32:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.501+0000 7f714a2d0640 1 -- 172.21.15.7:0/478439070 shutdown_connections 2024-09-21T04:32:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.501+0000 7f714a2d0640 1 --2- 172.21.15.7:0/478439070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7144101a60 0x7f7144101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.501+0000 7f714a2d0640 1 --2- 172.21.15.7:0/478439070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144100860 0x7f7144100c60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.501+0000 7f714a2d0640 1 -- 172.21.15.7:0/478439070 >> 172.21.15.7:0/478439070 conn(0x7f71440fc010 msgr2=0x7f71440fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:32:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.501+0000 7f714a2d0640 1 -- 172.21.15.7:0/478439070 shutdown_connections 2024-09-21T04:32:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.501+0000 7f714a2d0640 1 -- 172.21.15.7:0/478439070 wait complete. 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f714a2d0640 1 Processor -- start 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f714a2d0640 1 -- start start 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f714a2d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7144100860 0x7f7144104730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f714a2d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144101a60 0x7f7144102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f714a2d0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71441096a0 con 0x7f7144101a60 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f714a2d0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7144109520 con 0x7f7144100860 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f71437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144101a60 0x7f7144102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f71437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144101a60 0x7f7144102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57504/0 (socket says 172.21.15.7:57504) 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f71437fe640 1 -- 172.21.15.7:0/3282241879 learned_addr learned my addr 172.21.15.7:0/3282241879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f7143fff640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7144100860 0x7f7144104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f71437fe640 1 -- 172.21.15.7:0/3282241879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7144100860 msgr2=0x7f7144104730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:32:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f71437fe640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7144100860 0x7f7144104730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:32:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f71437fe640 1 -- 172.21.15.7:0/3282241879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7144103380 con 0x7f7144101a60 2024-09-21T04:32:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.502+0000 7f7143fff640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7144100860 0x7f7144104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:32:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.503+0000 7f71437fe640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144101a60 0x7f7144102e40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f713800d6f0 tx=0x7f713800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.503+0000 7f71417fa640 1 -- 172.21.15.7:0/3282241879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7138017070 con 0x7f7144101a60 2024-09-21T04:32:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.503+0000 7f71417fa640 1 -- 172.21.15.7:0/3282241879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f713800de00 con 0x7f7144101a60 2024-09-21T04:32:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.503+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7144103670 con 0x7f7144101a60 2024-09-21T04:32:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.503+0000 7f71417fa640 1 -- 172.21.15.7:0/3282241879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7138004bb0 con 0x7f7144101a60 2024-09-21T04:32:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.503+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7144103c00 con 0x7f7144101a60 2024-09-21T04:32:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.504+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7144100c60 con 0x7f7144101a60 2024-09-21T04:32:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.505+0000 7f71417fa640 1 -- 172.21.15.7:0/3282241879 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f7138009a30 con 0x7f7144101a60 2024-09-21T04:32:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.505+0000 7f71417fa640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f712c06e310 0x7f712c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:32:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.505+0000 7f71417fa640 1 -- 172.21.15.7:0/3282241879 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f713808dd20 con 0x7f7144101a60 2024-09-21T04:32:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.505+0000 7f7143fff640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f712c06e310 0x7f712c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:32:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.506+0000 7f7143fff640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f712c06e310 0x7f712c0707d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7134000c00 tx=0x7f7134005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:32:56.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.509+0000 7f71417fa640 1 -- 172.21.15.7:0/3282241879 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f713805b6a0 con 0x7f7144101a60 2024-09-21T04:32:56.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:32:56.699+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) -- 0x7f714406a930 con 0x7f712c06e310 2024-09-21T04:32:57.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:56 smithi053 ceph-mon[35554]: pgmap v194: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:32:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:56 smithi007 ceph-mon[27954]: pgmap v194: 65 pgs: 65 active+clean; 582 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:32:58.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:57 smithi053 ceph-mon[35554]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:58.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:57 smithi007 ceph-mon[27954]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:32:59.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:32:58 smithi053 ceph-mon[35554]: pgmap v195: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-21T04:32:59.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:32:58 smithi007 ceph-mon[27954]: pgmap v195: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-21T04:33:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:00 smithi053 ceph-mon[35554]: pgmap v196: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-21T04:33:01.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:00 smithi007 ceph-mon[27954]: pgmap v196: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-21T04:33:02.119 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:33:02.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:33:03.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:02 smithi053 ceph-mon[35554]: pgmap v197: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T04:33:03.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:02 smithi007 ceph-mon[27954]: pgmap v197: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T04:33:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:04 smithi053 ceph-mon[35554]: pgmap v198: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-21T04:33:05.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:04 smithi007 ceph-mon[27954]: pgmap v198: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-21T04:33:07.120 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:06 smithi053 ceph-mon[35554]: pgmap v199: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-21T04:33:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:06 smithi007 ceph-mon[27954]: pgmap v199: 65 pgs: 65 active+clean; 587 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-21T04:33:09.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:08 smithi053 ceph-mon[35554]: pgmap v200: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-21T04:33:09.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:08 smithi007 ceph-mon[27954]: pgmap v200: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-21T04:33:11.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:10 smithi053 ceph-mon[35554]: pgmap v201: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T04:33:11.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:10 smithi007 ceph-mon[27954]: pgmap v201: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T04:33:13.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:12 smithi053 ceph-mon[35554]: pgmap v202: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-21T04:33:13.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:12 smithi007 ceph-mon[27954]: pgmap v202: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-21T04:33:15.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:14 smithi053 ceph-mon[35554]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T04:33:15.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:14 smithi007 ceph-mon[27954]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T04:33:17.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:16 smithi053 ceph-mon[35554]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T04:33:17.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:33:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:16 smithi007 ceph-mon[27954]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-21T04:33:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:33:19.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:18 smithi053 ceph-mon[35554]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:33:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:18 smithi007 ceph-mon[27954]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:33:21.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:20 smithi053 ceph-mon[35554]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:20 smithi007 ceph-mon[27954]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:22.880 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:22 smithi007 ceph-mon[27954]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:33:23.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:22 smithi053 ceph-mon[35554]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:33:25.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:24 smithi053 ceph-mon[35554]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:24 smithi007 ceph-mon[27954]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:27.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:26 smithi053 ceph-mon[35554]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:33:27.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:26 smithi007 ceph-mon[27954]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:33:29.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:28 smithi053 ceph-mon[35554]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-21T04:33:29.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:28 smithi007 ceph-mon[27954]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-21T04:33:30.684 INFO:teuthology.orchestra.run.smithi007.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:33:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.681+0000 7f71417fa640 1 -- 172.21.15.7:0/3282241879 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+89 (secure 0 0 0) 0x7f714406a930 con 0x7f712c06e310 2024-09-21T04:33:30.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f712c06e310 msgr2=0x7f712c0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:30.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f712c06e310 0x7f712c0707d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7134000c00 tx=0x7f7134005a20 comp rx=0 tx=0).stop 2024-09-21T04:33:30.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144101a60 msgr2=0x7f7144102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144101a60 0x7f7144102e40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f713800d6f0 tx=0x7f713800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:33:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 shutdown_connections 2024-09-21T04:33:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f712c06e310 0x7f712c0707d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7144101a60 0x7f7144102e40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 --2- 172.21.15.7:0/3282241879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7144100860 0x7f7144104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 >> 172.21.15.7:0/3282241879 conn(0x7f71440fc010 msgr2=0x7f714406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 shutdown_connections 2024-09-21T04:33:30.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:30.684+0000 7f714a2d0640 1 -- 172.21.15.7:0/3282241879 wait complete. 2024-09-21T04:33:31.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:30 smithi053 ceph-mon[35554]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-21T04:33:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:33:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:33:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:33:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:33:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:30 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:33:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:30 smithi007 ceph-mon[27954]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-21T04:33:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:33:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:33:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:33:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:33:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:30 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:33:31.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-09-21T04:33:31.769 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:33:32.161 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:31 smithi007 ceph-mon[27954]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:33:32.161 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:31 smithi007 ceph-mon[27954]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:33:32.161 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:31 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:33:32.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:31 smithi053 ceph-mon[35554]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:33:32.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:31 smithi053 ceph-mon[35554]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:33:32.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:31 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.427+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3808607130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 msgr2=0x7fd5e81047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.427+0000 7fd5ef33a640 1 --2- 172.21.15.7:0/3808607130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 0x7fd5e81047e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40098b0 tx=0x7fd5e402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.427+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3808607130 shutdown_connections 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.427+0000 7fd5ef33a640 1 --2- 172.21.15.7:0/3808607130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e8104d20 0x7fd5e8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.427+0000 7fd5ef33a640 1 --2- 172.21.15.7:0/3808607130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 0x7fd5e81047e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.427+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3808607130 >> 172.21.15.7:0/3808607130 conn(0x7fd5e80fc010 msgr2=0x7fd5e80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.427+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3808607130 shutdown_connections 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.427+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3808607130 wait complete. 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.428+0000 7fd5ef33a640 1 Processor -- start 2024-09-21T04:33:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.428+0000 7fd5ef33a640 1 -- start start 2024-09-21T04:33:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.429+0000 7fd5ef33a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 0x7fd5e8073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.429+0000 7fd5ef33a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e8104d20 0x7fd5e8073790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.429+0000 7fd5ef33a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8101de0 con 0x7fd5e81023f0 2024-09-21T04:33:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.429+0000 7fd5ef33a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e8101c60 con 0x7fd5e8104d20 2024-09-21T04:33:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.429+0000 7fd5ec8d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e8104d20 0x7fd5e8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.429+0000 7fd5ec8d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e8104d20 0x7fd5e8073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:41452/0 (socket says 172.21.15.7:41452) 2024-09-21T04:33:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.429+0000 7fd5ec8d5640 1 -- 172.21.15.7:0/3244175493 learned_addr learned my addr 172.21.15.7:0/3244175493 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:33:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.430+0000 7fd5ed0d6640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 0x7fd5e8073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.430+0000 7fd5ec8d5640 1 -- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 msgr2=0x7fd5e8073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.430+0000 7fd5ec8d5640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 0x7fd5e8073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.430+0000 7fd5ec8d5640 1 -- 172.21.15.7:0/3244175493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e8073e30 con 0x7fd5e8104d20 2024-09-21T04:33:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.430+0000 7fd5ed0d6640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 0x7fd5e8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:33:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.430+0000 7fd5ec8d5640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e8104d20 0x7fd5e8073790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800e970 tx=0x7fd5d800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:32.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.433+0000 7fd5de7fc640 1 -- 172.21.15.7:0/3244175493 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd5d800cac0 con 0x7fd5e8104d20 2024-09-21T04:33:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.433+0000 7fd5de7fc640 1 -- 172.21.15.7:0/3244175493 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5d80054e0 con 0x7fd5e8104d20 2024-09-21T04:33:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.433+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e81a75a0 con 0x7fd5e8104d20 2024-09-21T04:33:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.433+0000 7fd5de7fc640 1 -- 172.21.15.7:0/3244175493 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd5d8010430 con 0x7fd5e8104d20 2024-09-21T04:33:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.433+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e81a7ae0 con 0x7fd5e8104d20 2024-09-21T04:33:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.435+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc0052c0 con 0x7fd5e8104d20 2024-09-21T04:33:32.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.436+0000 7fd5de7fc640 1 -- 172.21.15.7:0/3244175493 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fd5d80046c0 con 0x7fd5e8104d20 2024-09-21T04:33:32.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.437+0000 7fd5de7fc640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd5d406e350 0x7fd5d4070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.437+0000 7fd5ed0d6640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd5d406e350 0x7fd5d4070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.437+0000 7fd5de7fc640 1 -- 172.21.15.7:0/3244175493 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7fd5d808e600 con 0x7fd5e8104d20 2024-09-21T04:33:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.438+0000 7fd5ed0d6640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd5d406e350 0x7fd5d4070810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4000c00 tx=0x7fd5e4005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:32.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.446+0000 7fd5de7fc640 1 -- 172.21.15.7:0/3244175493 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd5d805c020 con 0x7fd5e8104d20 2024-09-21T04:33:32.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.660+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd5cc000bf0 con 0x7fd5d406e350 2024-09-21T04:33:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.661+0000 7fd5de7fc640 1 -- 172.21.15.7:0/3244175493 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fd5cc000bf0 con 0x7fd5d406e350 2024-09-21T04:33:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd5d406e350 msgr2=0x7fd5d4070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd5d406e350 0x7fd5d4070810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4000c00 tx=0x7fd5e4005a20 comp rx=0 tx=0).stop 2024-09-21T04:33:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e8104d20 msgr2=0x7fd5e8073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e8104d20 0x7fd5e8073790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800e970 tx=0x7fd5d800ee30 comp rx=0 tx=0).stop 2024-09-21T04:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 shutdown_connections 2024-09-21T04:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fd5d406e350 0x7fd5d4070810 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5e8104d20 0x7fd5e8073790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 --2- 172.21.15.7:0/3244175493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e81023f0 0x7fd5e8073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 >> 172.21.15.7:0/3244175493 conn(0x7fd5e80fc010 msgr2=0x7fd5e80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 shutdown_connections 2024-09-21T04:33:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.664+0000 7fd5ef33a640 1 -- 172.21.15.7:0/3244175493 wait complete. 2024-09-21T04:33:32.676 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:33:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.758+0000 7fafb2507640 1 -- 172.21.15.7:0/481373359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac102b60 msgr2=0x7fafac102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.758+0000 7fafb2507640 1 --2- 172.21.15.7:0/481373359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac102b60 0x7fafac102fe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009880 tx=0x7fafa002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.758+0000 7fafb2507640 1 -- 172.21.15.7:0/481373359 shutdown_connections 2024-09-21T04:33:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.758+0000 7fafb2507640 1 --2- 172.21.15.7:0/481373359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac102b60 0x7fafac102fe0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.758+0000 7fafb2507640 1 --2- 172.21.15.7:0/481373359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac101960 0x7fafac101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.758+0000 7fafb2507640 1 -- 172.21.15.7:0/481373359 >> 172.21.15.7:0/481373359 conn(0x7fafac0fd110 msgr2=0x7fafac0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.758+0000 7fafb2507640 1 -- 172.21.15.7:0/481373359 shutdown_connections 2024-09-21T04:33:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.758+0000 7fafb2507640 1 -- 172.21.15.7:0/481373359 wait complete. 2024-09-21T04:33:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.759+0000 7fafb2507640 1 Processor -- start 2024-09-21T04:33:32.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.759+0000 7fafb2507640 1 -- start start 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafb2507640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac101960 0x7fafac1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafb2507640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac197a20 0x7fafac19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafb2507640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fafac10a7a0 con 0x7fafac101960 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafb2507640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fafac10a620 con 0x7fafac197a20 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac101960 0x7fafac1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac101960 0x7fafac1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33816/0 (socket says 172.21.15.7:33816) 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafabfff640 1 -- 172.21.15.7:0/385484150 learned_addr learned my addr 172.21.15.7:0/385484150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafab7fe640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac197a20 0x7fafac19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafabfff640 1 -- 172.21.15.7:0/385484150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac197a20 msgr2=0x7fafac19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:32.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafabfff640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac197a20 0x7fafac19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.760+0000 7fafabfff640 1 -- 172.21.15.7:0/385484150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac197f50 con 0x7fafac101960 2024-09-21T04:33:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.761+0000 7fafabfff640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac101960 0x7fafac1974e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00e940 tx=0x7faf9c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.761+0000 7fafa97fa640 1 -- 172.21.15.7:0/385484150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faf9c00cac0 con 0x7fafac101960 2024-09-21T04:33:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.761+0000 7fafa97fa640 1 -- 172.21.15.7:0/385484150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faf9c00cc60 con 0x7fafac101960 2024-09-21T04:33:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.761+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac19cf60 con 0x7fafac101960 2024-09-21T04:33:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.761+0000 7fafa97fa640 1 -- 172.21.15.7:0/385484150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faf9c003e30 con 0x7fafac101960 2024-09-21T04:33:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.761+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac19d3c0 con 0x7fafac101960 2024-09-21T04:33:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.762+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafac1915b0 con 0x7fafac101960 2024-09-21T04:33:32.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.763+0000 7fafa97fa640 1 -- 172.21.15.7:0/385484150 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7faf9c004000 con 0x7fafac101960 2024-09-21T04:33:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.764+0000 7fafa97fa640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faf9406e310 0x7faf940707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.764+0000 7fafa97fa640 1 -- 172.21.15.7:0/385484150 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7faf9c08e730 con 0x7fafac101960 2024-09-21T04:33:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.764+0000 7fafab7fe640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faf9406e310 0x7faf940707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.765+0000 7fafab7fe640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faf9406e310 0x7faf940707d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fafac198990 tx=0x7fafa0000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:32.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.773+0000 7fafa97fa640 1 -- 172.21.15.7:0/385484150 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faf9c05bf80 con 0x7fafac101960 2024-09-21T04:33:32.976 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:32 smithi007 ceph-mon[27954]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:33:32.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.976+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fafac062630 con 0x7faf9406e310 2024-09-21T04:33:32.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.978+0000 7fafa97fa640 1 -- 172.21.15.7:0/385484150 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fafac062630 con 0x7faf9406e310 2024-09-21T04:33:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faf9406e310 msgr2=0x7faf940707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faf9406e310 0x7faf940707d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fafac198990 tx=0x7fafa0000bc0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac101960 msgr2=0x7fafac1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac101960 0x7fafac1974e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00e940 tx=0x7faf9c00ee00 comp rx=0 tx=0).stop 2024-09-21T04:33:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 shutdown_connections 2024-09-21T04:33:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7faf9406e310 0x7faf940707d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac197a20 0x7fafac19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 --2- 172.21.15.7:0/385484150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafac101960 0x7fafac1974e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:32.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.981+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 >> 172.21.15.7:0/385484150 conn(0x7fafac0fd110 msgr2=0x7fafac105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.984+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 shutdown_connections 2024-09-21T04:33:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:32.985+0000 7fafb2507640 1 -- 172.21.15.7:0/385484150 wait complete. 2024-09-21T04:33:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.080+0000 7f8492709640 1 -- 172.21.15.7:0/2030888845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 msgr2=0x7f848c101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.080+0000 7f8492709640 1 --2- 172.21.15.7:0/2030888845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 0x7f848c101ee0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8480009880 tx=0x7f848002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.080+0000 7f8492709640 1 -- 172.21.15.7:0/2030888845 shutdown_connections 2024-09-21T04:33:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.080+0000 7f8492709640 1 --2- 172.21.15.7:0/2030888845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 0x7f848c101ee0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.080+0000 7f8492709640 1 --2- 172.21.15.7:0/2030888845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c100860 0x7f848c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.080+0000 7f8492709640 1 -- 172.21.15.7:0/2030888845 >> 172.21.15.7:0/2030888845 conn(0x7f848c0fc010 msgr2=0x7f848c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.080+0000 7f8492709640 1 -- 172.21.15.7:0/2030888845 shutdown_connections 2024-09-21T04:33:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.081+0000 7f8492709640 1 -- 172.21.15.7:0/2030888845 wait complete. 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.081+0000 7f8492709640 1 Processor -- start 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f8492709640 1 -- start start 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f8492709640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c100860 0x7f848c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f8492709640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 0x7f848c196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f8492709640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f848c109680 con 0x7f848c101a60 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f8492709640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f848c109500 con 0x7f848c100860 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f848b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 0x7f848c196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f848b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 0x7f848c196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33830/0 (socket says 172.21.15.7:33830) 2024-09-21T04:33:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f848b7fe640 1 -- 172.21.15.7:0/3296155400 learned_addr learned my addr 172.21.15.7:0/3296155400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:33:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.082+0000 7f848bfff640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c100860 0x7f848c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f848bfff640 1 -- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 msgr2=0x7f848c196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f848bfff640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 0x7f848c196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f848bfff640 1 -- 172.21.15.7:0/3296155400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c196f80 con 0x7f848c100860 2024-09-21T04:33:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f848b7fe640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 0x7f848c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:33:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f848bfff640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c100860 0x7f848c1963c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f847c00e940 tx=0x7f847c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f84897fa640 1 -- 172.21.15.7:0/3296155400 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f847c00cac0 con 0x7f848c100860 2024-09-21T04:33:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848c069d60 con 0x7f848c100860 2024-09-21T04:33:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f84897fa640 1 -- 172.21.15.7:0/3296155400 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f847c00cc60 con 0x7f848c100860 2024-09-21T04:33:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.083+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848c06a2f0 con 0x7f848c100860 2024-09-21T04:33:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.084+0000 7f84897fa640 1 -- 172.21.15.7:0/3296155400 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f847c003e10 con 0x7f848c100860 2024-09-21T04:33:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.085+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848c100c60 con 0x7f848c100860 2024-09-21T04:33:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.086+0000 7f84897fa640 1 -- 172.21.15.7:0/3296155400 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f847c004020 con 0x7f848c100860 2024-09-21T04:33:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.087+0000 7f84897fa640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f847406e310 0x7f84740707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.087+0000 7f848b7fe640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f847406e310 0x7f84740707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.087+0000 7f848b7fe640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f847406e310 0x7f84740707d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f84800096c0 tx=0x7f8480005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.088+0000 7f84897fa640 1 -- 172.21.15.7:0/3296155400 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f847c08e740 con 0x7f848c100860 2024-09-21T04:33:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.095+0000 7f84897fa640 1 -- 172.21.15.7:0/3296155400 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f847c05c020 con 0x7f848c100860 2024-09-21T04:33:33.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:32 smithi053 ceph-mon[35554]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:33:33.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.467+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f848c06a660 con 0x7f847406e310 2024-09-21T04:33:33.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.484+0000 7f84897fa640 1 -- 172.21.15.7:0/3296155400 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3792 (secure 0 0 0) 0x7f848c06a660 con 0x7f847406e310 2024-09-21T04:33:33.487 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (5m) 50s ago 6m 24.6M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (6m) 50s ago 6m 8316k - 19.1.1-373-g4e45e6b4 164465d2334a f80c59780c91 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (5m) 54s ago 5m 5268k - 19.1.1-373-g4e45e6b4 164465d2334a 828ddad2141a 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (6m) 50s ago 6m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (5m) 54s ago 5m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (4m) 50s ago 6m 77.8M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (68s) 50s ago 67s 16.6M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (74s) 50s ago 73s 12.6M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (65s) 54s ago 65s 14.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (71s) 54s ago 71s 20.2M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:9283,8765,8443 running (7m) 50s ago 7m 498M - 19.1.1-373-g4e45e6b4 164465d2334a c324ee791e08 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (5m) 54s ago 5m 441M - 19.1.1-373-g4e45e6b4 164465d2334a e9db9486fd2f 2024-09-21T04:33:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (7m) 50s ago 8m 47.5M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (5m) 54s ago 5m 38.4M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (6m) 50s ago 6m 15.9M - 1.5.0 0da6a335fe13 b4a8255a594a 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (5m) 54s ago 5m 16.5M - 1.5.0 0da6a335fe13 88246b15ca5d 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (4m) 50s ago 4m 48.5M 3948M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (4m) 50s ago 4m 50.4M 3948M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (3m) 50s ago 3m 51.8M 3948M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (3m) 54s ago 3m 45.7M 4972M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (2m) 54s ago 2m 48.5M 4972M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (2m) 54s ago 2m 41.5M 4972M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:33:33.489 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (4m) 50s ago 6m 56.2M - 2.43.0 a07b618ecd1d 74a720908c7f 2024-09-21T04:33:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f847406e310 msgr2=0x7f84740707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f847406e310 0x7f84740707d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f84800096c0 tx=0x7f8480005a20 comp rx=0 tx=0).stop 2024-09-21T04:33:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c100860 msgr2=0x7f848c1963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c100860 0x7f848c1963c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f847c00e940 tx=0x7f847c00ee00 comp rx=0 tx=0).stop 2024-09-21T04:33:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 shutdown_connections 2024-09-21T04:33:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f847406e310 0x7f84740707d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f848c101a60 0x7f848c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 --2- 172.21.15.7:0/3296155400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c100860 0x7f848c1963c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 >> 172.21.15.7:0/3296155400 conn(0x7f848c0fc010 msgr2=0x7f848c104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:33.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.487+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 shutdown_connections 2024-09-21T04:33:33.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.488+0000 7f8492709640 1 -- 172.21.15.7:0/3296155400 wait complete. 2024-09-21T04:33:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.609+0000 7f326ffd8640 1 -- 172.21.15.7:0/1363573212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3268100860 msgr2=0x7f3268100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.609+0000 7f326ffd8640 1 --2- 172.21.15.7:0/1363573212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3268100860 0x7f3268100c60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3264009880 tx=0x7f326402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.610+0000 7f326ffd8640 1 -- 172.21.15.7:0/1363573212 shutdown_connections 2024-09-21T04:33:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.610+0000 7f326ffd8640 1 --2- 172.21.15.7:0/1363573212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268101a60 0x7f3268101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.610+0000 7f326ffd8640 1 --2- 172.21.15.7:0/1363573212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3268100860 0x7f3268100c60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.610+0000 7f326ffd8640 1 -- 172.21.15.7:0/1363573212 >> 172.21.15.7:0/1363573212 conn(0x7f32680fc010 msgr2=0x7f32680fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.610+0000 7f326ffd8640 1 -- 172.21.15.7:0/1363573212 shutdown_connections 2024-09-21T04:33:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.610+0000 7f326ffd8640 1 -- 172.21.15.7:0/1363573212 wait complete. 2024-09-21T04:33:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.611+0000 7f326ffd8640 1 Processor -- start 2024-09-21T04:33:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.611+0000 7f326ffd8640 1 -- start start 2024-09-21T04:33:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.612+0000 7f326ffd8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268101a60 0x7f326806a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.612+0000 7f326ffd8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326819c830 0x7f326819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.612+0000 7f326ffd8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32681096a0 con 0x7f326819c830 2024-09-21T04:33:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.612+0000 7f326ffd8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3268109520 con 0x7f3268101a60 2024-09-21T04:33:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.612+0000 7f326dd74640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268101a60 0x7f326806a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.612+0000 7f326dd74640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268101a60 0x7f326806a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:43678/0 (socket says 172.21.15.7:43678) 2024-09-21T04:33:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.612+0000 7f326dd74640 1 -- 172.21.15.7:0/3556543054 learned_addr learned my addr 172.21.15.7:0/3556543054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:33:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.612+0000 7f326d573640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326819c830 0x7f326819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f326dd74640 1 -- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326819c830 msgr2=0x7f326819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f326dd74640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326819c830 0x7f326819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f326dd74640 1 -- 172.21.15.7:0/3556543054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326819f160 con 0x7f3268101a60 2024-09-21T04:33:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f326dd74640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268101a60 0x7f326806a790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32681018c0 tx=0x7f32640045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f325effd640 1 -- 172.21.15.7:0/3556543054 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f326403f070 con 0x7f3268101a60 2024-09-21T04:33:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326819f330 con 0x7f3268101a60 2024-09-21T04:33:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f325effd640 1 -- 172.21.15.7:0/3556543054 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3264004a50 con 0x7f3268101a60 2024-09-21T04:33:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f325effd640 1 -- 172.21.15.7:0/3556543054 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3264035030 con 0x7f3268101a60 2024-09-21T04:33:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.613+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326819f7e0 con 0x7f3268101a60 2024-09-21T04:33:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.615+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3268109e80 con 0x7f3268101a60 2024-09-21T04:33:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.616+0000 7f325effd640 1 -- 172.21.15.7:0/3556543054 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f3264042070 con 0x7f3268101a60 2024-09-21T04:33:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.616+0000 7f325effd640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f325406e360 0x7f3254070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.617+0000 7f326d573640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f325406e360 0x7f3254070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.617+0000 7f325effd640 1 -- 172.21.15.7:0/3556543054 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f32640aefa0 con 0x7f3268101a60 2024-09-21T04:33:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.617+0000 7f326d573640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f325406e360 0x7f3254070820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f326806a510 tx=0x7f325800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:33.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.625+0000 7f325effd640 1 -- 172.21.15.7:0/3556543054 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f32640804a0 con 0x7f3268101a60 2024-09-21T04:33:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.900+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f32681a0120 con 0x7f3268101a60 2024-09-21T04:33:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.901+0000 7f325effd640 1 -- 172.21.15.7:0/3556543054 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+633 (secure 0 0 0) 0x7f3264084cd0 con 0x7f3268101a60 2024-09-21T04:33:33.903 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:33:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 14 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:33:33.904 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:33:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f325406e360 msgr2=0x7f3254070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f325406e360 0x7f3254070820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f326806a510 tx=0x7f325800b390 comp rx=0 tx=0).stop 2024-09-21T04:33:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268101a60 msgr2=0x7f326806a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268101a60 0x7f326806a790 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32681018c0 tx=0x7f32640045c0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 shutdown_connections 2024-09-21T04:33:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f325406e360 0x7f3254070820 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326819c830 0x7f326819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 --2- 172.21.15.7:0/3556543054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268101a60 0x7f326806a790 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 >> 172.21.15.7:0/3556543054 conn(0x7f32680fc010 msgr2=0x7f32680fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 shutdown_connections 2024-09-21T04:33:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.903+0000 7f326ffd8640 1 -- 172.21.15.7:0/3556543054 wait complete. 2024-09-21T04:33:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.993+0000 7f8c40f80640 1 -- 172.21.15.7:0/2567141432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100240 msgr2=0x7f8c3c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.993+0000 7f8c40f80640 1 --2- 172.21.15.7:0/2567141432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100240 0x7f8c3c100640 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009880 tx=0x7f8c2c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.993+0000 7f8c40f80640 1 -- 172.21.15.7:0/2567141432 shutdown_connections 2024-09-21T04:33:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.993+0000 7f8c40f80640 1 --2- 172.21.15.7:0/2567141432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c3c100b80 0x7f8c3c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.993+0000 7f8c40f80640 1 --2- 172.21.15.7:0/2567141432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100240 0x7f8c3c100640 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.993+0000 7f8c40f80640 1 -- 172.21.15.7:0/2567141432 >> 172.21.15.7:0/2567141432 conn(0x7f8c3c0fc010 msgr2=0x7f8c3c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.993+0000 7f8c40f80640 1 -- 172.21.15.7:0/2567141432 shutdown_connections 2024-09-21T04:33:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.993+0000 7f8c40f80640 1 -- 172.21.15.7:0/2567141432 wait complete. 2024-09-21T04:33:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.994+0000 7f8c40f80640 1 Processor -- start 2024-09-21T04:33:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.994+0000 7f8c40f80640 1 -- start start 2024-09-21T04:33:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.994+0000 7f8c40f80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100b80 0x7f8c3c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.994+0000 7f8c40f80640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c3c199e00 0x7f8c3c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.994+0000 7f8c40f80640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c10c990 con 0x7f8c3c100b80 2024-09-21T04:33:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.994+0000 7f8c40f80640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c10c810 con 0x7f8c3c199e00 2024-09-21T04:33:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c3a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100b80 0x7f8c3c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c3a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100b80 0x7f8c3c1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34924/0 (socket says 172.21.15.7:34924) 2024-09-21T04:33:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c3a59c640 1 -- 172.21.15.7:0/634822977 learned_addr learned my addr 172.21.15.7:0/634822977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:33:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c39d9b640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c3c199e00 0x7f8c3c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c3a59c640 1 -- 172.21.15.7:0/634822977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c3c199e00 msgr2=0x7f8c3c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c3a59c640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c3c199e00 0x7f8c3c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c3a59c640 1 -- 172.21.15.7:0/634822977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3c19a330 con 0x7f8c3c100b80 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c3a59c640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100b80 0x7f8c3c1998c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006010 tx=0x7f8c2c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c2b7fe640 1 -- 172.21.15.7:0/634822977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8c2c03f070 con 0x7f8c3c100b80 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.995+0000 7f8c2b7fe640 1 -- 172.21.15.7:0/634822977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c2c0049d0 con 0x7f8c3c100b80 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.996+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3c19f340 con 0x7f8c3c100b80 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.996+0000 7f8c2b7fe640 1 -- 172.21.15.7:0/634822977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8c2c035030 con 0x7f8c3c100b80 2024-09-21T04:33:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.996+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3c19f790 con 0x7f8c3c100b80 2024-09-21T04:33:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.997+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c3c100640 con 0x7f8c3c100b80 2024-09-21T04:33:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.997+0000 7f8c2b7fe640 1 -- 172.21.15.7:0/634822977 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f8c2c042070 con 0x7f8c3c100b80 2024-09-21T04:33:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.998+0000 7f8c2b7fe640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8c2406e350 0x7f8c24070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:33:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.998+0000 7f8c2b7fe640 1 -- 172.21.15.7:0/634822977 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f8c2c0aea30 con 0x7f8c3c100b80 2024-09-21T04:33:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.999+0000 7f8c39d9b640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8c2406e350 0x7f8c24070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:33:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:33.999+0000 7f8c39d9b640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8c2406e350 0x7f8c24070810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c19ad70 tx=0x7f8c30005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:33:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.006+0000 7f8c2b7fe640 1 -- 172.21.15.7:0/634822977 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8c2c080070 con 0x7f8c3c100b80 2024-09-21T04:33:34.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:33 smithi053 ceph-mon[35554]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:33:34.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:33 smithi053 ceph-mon[35554]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:33:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:33 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:33:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:33 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:33:34.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:33 smithi007 ceph-mon[27954]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:33:34.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:33 smithi007 ceph-mon[27954]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:33:34.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:33 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:33:34.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:33 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:33:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.227+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8c3c109cb0 con 0x7f8c2406e350 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.228+0000 7f8c2b7fe640 1 -- 172.21.15.7:0/634822977 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+439 (secure 0 0 0) 0x7f8c3c109cb0 con 0x7f8c2406e350 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "0/2 daemons upgraded", 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 image on host smithi053", 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:33:34.231 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:33:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8c2406e350 msgr2=0x7f8c24070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8c2406e350 0x7f8c24070810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c19ad70 tx=0x7f8c30005f90 comp rx=0 tx=0).stop 2024-09-21T04:33:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100b80 msgr2=0x7f8c3c1998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:33:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100b80 0x7f8c3c1998c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006010 tx=0x7f8c2c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:33:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 shutdown_connections 2024-09-21T04:33:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f8c2406e350 0x7f8c24070810 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c3c199e00 0x7f8c3c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 --2- 172.21.15.7:0/634822977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c3c100b80 0x7f8c3c1998c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:33:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.230+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 >> 172.21.15.7:0/634822977 conn(0x7f8c3c0fc010 msgr2=0x7f8c3c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:33:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.231+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 shutdown_connections 2024-09-21T04:33:34.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:33:34.231+0000 7f8c40f80640 1 -- 172.21.15.7:0/634822977 wait complete. 2024-09-21T04:33:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:34 smithi053 ceph-mon[35554]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-21T04:33:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:34 smithi053 ceph-mon[35554]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:33:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:34 smithi053 ceph-mon[35554]: Upgrade: Target is version 19.3.0-5103-gf36283e7 (squid) 2024-09-21T04:33:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:34 smithi053 ceph-mon[35554]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00'] 2024-09-21T04:33:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:34 smithi053 ceph-mon[35554]: Upgrade: Need to upgrade myself (mgr.smithi007.huohfl) 2024-09-21T04:33:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:34 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3556543054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:33:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:34 smithi007 ceph-mon[27954]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-21T04:33:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:34 smithi007 ceph-mon[27954]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:33:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:34 smithi007 ceph-mon[27954]: Upgrade: Target is version 19.3.0-5103-gf36283e7 (squid) 2024-09-21T04:33:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:34 smithi007 ceph-mon[27954]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00'] 2024-09-21T04:33:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:34 smithi007 ceph-mon[27954]: Upgrade: Need to upgrade myself (mgr.smithi007.huohfl) 2024-09-21T04:33:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:34 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3556543054' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:33:36.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:35 smithi053 ceph-mon[35554]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 on smithi053 2024-09-21T04:33:36.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:35 smithi053 ceph-mon[35554]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:33:36.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:35 smithi007 ceph-mon[27954]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 on smithi053 2024-09-21T04:33:36.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:35 smithi007 ceph-mon[27954]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:33:37.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:36 smithi053 ceph-mon[35554]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:33:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:36 smithi007 ceph-mon[27954]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:33:39.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:38 smithi053 ceph-mon[35554]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:33:39.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:38 smithi007 ceph-mon[27954]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:33:41.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:40 smithi053 ceph-mon[35554]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:40 smithi007 ceph-mon[27954]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:42.911 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:42 smithi007 ceph-mon[27954]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:33:43.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:42 smithi053 ceph-mon[35554]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:33:45.168 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:44 smithi053 ceph-mon[35554]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:44 smithi007 ceph-mon[27954]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:47.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:46 smithi053 ceph-mon[35554]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-21T04:33:47.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:46 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:33:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:46 smithi007 ceph-mon[27954]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-21T04:33:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:46 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:33:49.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:48 smithi053 ceph-mon[35554]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:33:49.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:48 smithi007 ceph-mon[27954]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:33:51.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:50 smithi053 ceph-mon[35554]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:51.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:50 smithi007 ceph-mon[27954]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:52.923 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:52 smithi007 ceph-mon[27954]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:33:53.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:52 smithi053 ceph-mon[35554]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:33:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:54 smithi053 ceph-mon[35554]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:54 smithi007 ceph-mon[27954]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:33:57.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:56 smithi053 ceph-mon[35554]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-21T04:33:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:56 smithi007 ceph-mon[27954]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-21T04:33:58.344 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 92% (11753/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T04:33:59.045 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'. 2024-09-21T04:33:59.045 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-21T04:33:59.045 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T04:33:59.045 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T04:33:59.046 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-21T04:33:59.047 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at f36283e7c91 ceph-volume: fix OSD lvm/tpm2 activation 2024-09-21T04:33:59.051 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T04:33:59.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:33:58 smithi053 ceph-mon[35554]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:33:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:33:58 smithi007 ceph-mon[27954]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:33:59.242 INFO:tasks.workunit.client.0.smithi007.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T04:33:59.245 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T04:33:59.245 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T04:34:00.465 INFO:tasks.workunit.client.1.smithi053.stderr:Updating files: 79% (10107/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T04:34:01.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:00 smithi053 ceph-mon[35554]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:34:01.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:00 smithi007 ceph-mon[27954]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr:Note: switching to 'f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2'. 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr: 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr: 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr: 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr: git switch -c 2024-09-21T04:34:01.371 INFO:tasks.workunit.client.1.smithi053.stderr: 2024-09-21T04:34:01.372 INFO:tasks.workunit.client.1.smithi053.stderr:Or undo this operation with: 2024-09-21T04:34:01.372 INFO:tasks.workunit.client.1.smithi053.stderr: 2024-09-21T04:34:01.372 INFO:tasks.workunit.client.1.smithi053.stderr: git switch - 2024-09-21T04:34:01.372 INFO:tasks.workunit.client.1.smithi053.stderr: 2024-09-21T04:34:01.372 INFO:tasks.workunit.client.1.smithi053.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T04:34:01.372 INFO:tasks.workunit.client.1.smithi053.stderr: 2024-09-21T04:34:01.372 INFO:tasks.workunit.client.1.smithi053.stderr:HEAD is now at f36283e7c91 ceph-volume: fix OSD lvm/tpm2 activation 2024-09-21T04:34:01.376 DEBUG:teuthology.orchestra.run.smithi053:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-21T04:34:01.447 INFO:tasks.workunit.client.1.smithi053.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T04:34:01.449 INFO:tasks.workunit.client.1.smithi053.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-21T04:34:01.450 INFO:tasks.workunit.client.1.smithi053.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T04:34:02.119 INFO:tasks.workunit.client.1.smithi053.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T04:34:02.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:01 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:02.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:01 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:02.214 INFO:tasks.workunit.client.1.smithi053.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T04:34:02.265 INFO:tasks.workunit.client.1.smithi053.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-21T04:34:02.267 INFO:tasks.workunit.client.1.smithi053.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-21T04:34:02.267 INFO:tasks.workunit.client.1.smithi053.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T04:34:02.321 INFO:tasks.workunit.client.1.smithi053.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-21T04:34:02.325 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-21T04:34:02.325 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-21T04:34:02.384 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-09-21T04:34:02.386 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-21T04:34:02.386 DEBUG:teuthology.orchestra.run.smithi053:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-09-21T04:34:02.459 INFO:tasks.workunit.client.1.smithi053.stderr:+ mkdir -p fsstress 2024-09-21T04:34:02.462 INFO:tasks.workunit.client.1.smithi053.stderr:+ pushd fsstress 2024-09-21T04:34:02.463 INFO:tasks.workunit.client.1.smithi053.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-21T04:34:02.464 INFO:tasks.workunit.client.1.smithi053.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-21T04:34:03.070 INFO:tasks.workunit.client.1.smithi053.stderr:+ tar xzf ltp-full.tgz 2024-09-21T04:34:03.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:02 smithi053 ceph-mon[35554]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:34:03.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:02 smithi007 ceph-mon[27954]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:34:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.339+0000 7f1653f9a640 1 -- 172.21.15.7:0/698545794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 msgr2=0x7f164c103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.339+0000 7f1653f9a640 1 --2- 172.21.15.7:0/698545794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 0x7f164c103f60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1648009c20 tx=0x7f164802a970 comp rx=0 tx=0).stop 2024-09-21T04:34:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.340+0000 7f1653f9a640 1 -- 172.21.15.7:0/698545794 shutdown_connections 2024-09-21T04:34:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.340+0000 7f1653f9a640 1 --2- 172.21.15.7:0/698545794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c104d60 0x7f164c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.340+0000 7f1653f9a640 1 --2- 172.21.15.7:0/698545794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 0x7f164c103f60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.340+0000 7f1653f9a640 1 -- 172.21.15.7:0/698545794 >> 172.21.15.7:0/698545794 conn(0x7f164c0ff2f0 msgr2=0x7f164c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.340+0000 7f1653f9a640 1 -- 172.21.15.7:0/698545794 shutdown_connections 2024-09-21T04:34:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.340+0000 7f1653f9a640 1 -- 172.21.15.7:0/698545794 wait complete. 2024-09-21T04:34:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1653f9a640 1 Processor -- start 2024-09-21T04:34:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1653f9a640 1 -- start start 2024-09-21T04:34:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1653f9a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 0x7f164c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1653f9a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c104d60 0x7f164c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1653f9a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f164c10c9a0 con 0x7f164c103b60 2024-09-21T04:34:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1653f9a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f164c10c820 con 0x7f164c104d60 2024-09-21T04:34:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1651d36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 0x7f164c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1651d36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 0x7f164c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50544/0 (socket says 172.21.15.7:50544) 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1651d36640 1 -- 172.21.15.7:0/2597861168 learned_addr learned my addr 172.21.15.7:0/2597861168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1651535640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c104d60 0x7f164c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1651d36640 1 -- 172.21.15.7:0/2597861168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c104d60 msgr2=0x7f164c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1651d36640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c104d60 0x7f164c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1651d36640 1 -- 172.21.15.7:0/2597861168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c19a2c0 con 0x7f164c103b60 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.341+0000 7f1651535640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c104d60 0x7f164c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.342+0000 7f1651d36640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 0x7f164c199700 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1648006010 tx=0x7f164802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.342+0000 7f1642ffd640 1 -- 172.21.15.7:0/2597861168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1648047070 con 0x7f164c103b60 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.342+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164c19eca0 con 0x7f164c103b60 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.342+0000 7f1642ffd640 1 -- 172.21.15.7:0/2597861168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1648003e30 con 0x7f164c103b60 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.342+0000 7f1642ffd640 1 -- 172.21.15.7:0/2597861168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1648034070 con 0x7f164c103b60 2024-09-21T04:34:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.342+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164c19f200 con 0x7f164c103b60 2024-09-21T04:34:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.343+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164c103f60 con 0x7f164c103b60 2024-09-21T04:34:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.344+0000 7f1642ffd640 1 -- 172.21.15.7:0/2597861168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f164804a070 con 0x7f164c103b60 2024-09-21T04:34:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.345+0000 7f1642ffd640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f163806e310 0x7f16380707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.345+0000 7f1642ffd640 1 -- 172.21.15.7:0/2597861168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f16480aee20 con 0x7f164c103b60 2024-09-21T04:34:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.345+0000 7f1651535640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f163806e310 0x7f16380707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.345+0000 7f1651535640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f163806e310 0x7f16380707d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f164c19abb0 tx=0x7f163c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.350+0000 7f1642ffd640 1 -- 172.21.15.7:0/2597861168 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f164803f070 con 0x7f164c103b60 2024-09-21T04:34:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.566+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f164c0626f0 con 0x7f163806e310 2024-09-21T04:34:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.567+0000 7f1642ffd640 1 -- 172.21.15.7:0/2597861168 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+439 (secure 0 0 0) 0x7f164c0626f0 con 0x7f163806e310 2024-09-21T04:34:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f163806e310 msgr2=0x7f16380707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f163806e310 0x7f16380707d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f164c19abb0 tx=0x7f163c005f90 comp rx=0 tx=0).stop 2024-09-21T04:34:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 msgr2=0x7f164c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 0x7f164c199700 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1648006010 tx=0x7f164802f5d0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 shutdown_connections 2024-09-21T04:34:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f163806e310 0x7f16380707d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164c104d60 0x7f164c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 --2- 172.21.15.7:0/2597861168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c103b60 0x7f164c199700 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 >> 172.21.15.7:0/2597861168 conn(0x7f164c0ff2f0 msgr2=0x7f164c108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 shutdown_connections 2024-09-21T04:34:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.569+0000 7f1653f9a640 1 -- 172.21.15.7:0/2597861168 wait complete. 2024-09-21T04:34:04.581 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:34:04.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.660+0000 7f011ed2c640 1 -- 172.21.15.7:0/1419455458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118101a60 msgr2=0x7f0118101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.660+0000 7f011ed2c640 1 --2- 172.21.15.7:0/1419455458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118101a60 0x7f0118101ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0104009880 tx=0x7f010402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.661+0000 7f011ed2c640 1 -- 172.21.15.7:0/1419455458 shutdown_connections 2024-09-21T04:34:04.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.661+0000 7f011ed2c640 1 --2- 172.21.15.7:0/1419455458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118101a60 0x7f0118101ee0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.661+0000 7f011ed2c640 1 --2- 172.21.15.7:0/1419455458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0118100860 0x7f0118100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.661+0000 7f011ed2c640 1 -- 172.21.15.7:0/1419455458 >> 172.21.15.7:0/1419455458 conn(0x7f01180fc010 msgr2=0x7f01180fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:04.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.661+0000 7f011ed2c640 1 -- 172.21.15.7:0/1419455458 shutdown_connections 2024-09-21T04:34:04.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.661+0000 7f011ed2c640 1 -- 172.21.15.7:0/1419455458 wait complete. 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.661+0000 7f011ed2c640 1 Processor -- start 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.661+0000 7f011ed2c640 1 -- start start 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011ed2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118100860 0x7f011806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011ed2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0118101a60 0x7f011819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011ed2c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0118109680 con 0x7f0118100860 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011ed2c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0118109500 con 0x7f0118101a60 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011cac8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118100860 0x7f011806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011cac8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118100860 0x7f011806a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50562/0 (socket says 172.21.15.7:50562) 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011cac8640 1 -- 172.21.15.7:0/846862759 learned_addr learned my addr 172.21.15.7:0/846862759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f0117fff640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0118101a60 0x7f011819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011cac8640 1 -- 172.21.15.7:0/846862759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0118101a60 msgr2=0x7f011819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011cac8640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0118101a60 0x7f011819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011cac8640 1 -- 172.21.15.7:0/846862759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011819c7d0 con 0x7f0118100860 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.662+0000 7f011cac8640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118100860 0x7f011806a7d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e940 tx=0x7f010c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.663+0000 7f0115ffb640 1 -- 172.21.15.7:0/846862759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f010c00cac0 con 0x7f0118100860 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.663+0000 7f0115ffb640 1 -- 172.21.15.7:0/846862759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f010c00cc60 con 0x7f0118100860 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.663+0000 7f0115ffb640 1 -- 172.21.15.7:0/846862759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f010c003de0 con 0x7f0118100860 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.663+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011819cac0 con 0x7f0118100860 2024-09-21T04:34:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.663+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011819d020 con 0x7f0118100860 2024-09-21T04:34:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.664+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0118100c60 con 0x7f0118100860 2024-09-21T04:34:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.665+0000 7f0115ffb640 1 -- 172.21.15.7:0/846862759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f010c003fe0 con 0x7f0118100860 2024-09-21T04:34:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.665+0000 7f0115ffb640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f010806e2b0 0x7f0108070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.665+0000 7f0115ffb640 1 -- 172.21.15.7:0/846862759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f010c08e960 con 0x7f0118100860 2024-09-21T04:34:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.665+0000 7f0117fff640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f010806e2b0 0x7f0108070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.666+0000 7f0117fff640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f010806e2b0 0x7f0108070770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0104000c00 tx=0x7f01040058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:04.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.670+0000 7f0115ffb640 1 -- 172.21.15.7:0/846862759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f010c05c130 con 0x7f0118100860 2024-09-21T04:34:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.882+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0118062630 con 0x7f010806e2b0 2024-09-21T04:34:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.883+0000 7f0115ffb640 1 -- 172.21.15.7:0/846862759 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+439 (secure 0 0 0) 0x7f0118062630 con 0x7f010806e2b0 2024-09-21T04:34:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f010806e2b0 msgr2=0x7f0108070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f010806e2b0 0x7f0108070770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0104000c00 tx=0x7f01040058d0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118100860 msgr2=0x7f011806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118100860 0x7f011806a7d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f010c00e940 tx=0x7f010c00ee00 comp rx=0 tx=0).stop 2024-09-21T04:34:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 shutdown_connections 2024-09-21T04:34:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f010806e2b0 0x7f0108070770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0118101a60 0x7f011819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 --2- 172.21.15.7:0/846862759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118100860 0x7f011806a7d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.885+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 >> 172.21.15.7:0/846862759 conn(0x7f01180fc010 msgr2=0x7f0118104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.886+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 shutdown_connections 2024-09-21T04:34:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.886+0000 7f011ed2c640 1 -- 172.21.15.7:0/846862759 wait complete. 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.974+0000 7f0775915640 1 -- 172.21.15.7:0/56768416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0770104d20 msgr2=0x7f0770107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.974+0000 7f0775915640 1 --2- 172.21.15.7:0/56768416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0770104d20 0x7f0770107110 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f07640098b0 tx=0x7f076402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.974+0000 7f0775915640 1 -- 172.21.15.7:0/56768416 shutdown_connections 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.974+0000 7f0775915640 1 --2- 172.21.15.7:0/56768416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0770104d20 0x7f0770107110 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.974+0000 7f0775915640 1 --2- 172.21.15.7:0/56768416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07701023f0 0x7f07701047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.975+0000 7f0775915640 1 -- 172.21.15.7:0/56768416 >> 172.21.15.7:0/56768416 conn(0x7f07700fc010 msgr2=0x7f07700fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.975+0000 7f0775915640 1 -- 172.21.15.7:0/56768416 shutdown_connections 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.975+0000 7f0775915640 1 -- 172.21.15.7:0/56768416 wait complete. 2024-09-21T04:34:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.975+0000 7f0775915640 1 Processor -- start 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.975+0000 7f0775915640 1 -- start start 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f0775915640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07701023f0 0x7f0770197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f0775915640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0770104d20 0x7f0770197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f0775915640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f077010a790 con 0x7f07701023f0 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f0775915640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f077010a610 con 0x7f0770104d20 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f076effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07701023f0 0x7f0770197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f076effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07701023f0 0x7f0770197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50588/0 (socket says 172.21.15.7:50588) 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f076effd640 1 -- 172.21.15.7:0/1612698027 learned_addr learned my addr 172.21.15.7:0/1612698027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f076e7fc640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0770104d20 0x7f0770197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f076effd640 1 -- 172.21.15.7:0/1612698027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0770104d20 msgr2=0x7f0770197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f076effd640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0770104d20 0x7f0770197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f076effd640 1 -- 172.21.15.7:0/1612698027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07701980d0 con 0x7f07701023f0 2024-09-21T04:34:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.976+0000 7f076effd640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07701023f0 0x7f0770197510 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f076000d8d0 tx=0x7f076000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.977+0000 7f0774913640 1 -- 172.21.15.7:0/1612698027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0760017030 con 0x7f07701023f0 2024-09-21T04:34:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.977+0000 7f0774913640 1 -- 172.21.15.7:0/1612698027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0760004020 con 0x7f07701023f0 2024-09-21T04:34:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.977+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077019cb10 con 0x7f07701023f0 2024-09-21T04:34:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.977+0000 7f0774913640 1 -- 172.21.15.7:0/1612698027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f07600047a0 con 0x7f07701023f0 2024-09-21T04:34:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.977+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077019d070 con 0x7f07701023f0 2024-09-21T04:34:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.978+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077010af70 con 0x7f07701023f0 2024-09-21T04:34:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.979+0000 7f0774913640 1 -- 172.21.15.7:0/1612698027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7f07600041d0 con 0x7f07701023f0 2024-09-21T04:34:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.979+0000 7f0774913640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f075806e2b0 0x7f0758070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.979+0000 7f0774913640 1 -- 172.21.15.7:0/1612698027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7f0760018070 con 0x7f07701023f0 2024-09-21T04:34:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.979+0000 7f076e7fc640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f075806e2b0 0x7f0758070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.980+0000 7f076e7fc640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f075806e2b0 0x7f0758070770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0764009880 tx=0x7f07640058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:04.984+0000 7f0774913640 1 -- 172.21.15.7:0/1612698027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f076005b900 con 0x7f07701023f0 2024-09-21T04:34:05.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:04 smithi053 ceph-mon[35554]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:34:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.190+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0770105780 con 0x7f075806e2b0 2024-09-21T04:34:05.193 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:04 smithi007 ceph-mon[27954]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:34:05.206 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:34:05.206 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (5m) 82s ago 7m 24.6M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (7m) 82s ago 7m 8316k - 19.1.1-373-g4e45e6b4 164465d2334a f80c59780c91 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (6m) 86s ago 6m 5268k - 19.1.1-373-g4e45e6b4 164465d2334a 828ddad2141a 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (7m) 82s ago 7m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (6m) 86s ago 6m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (5m) 82s ago 6m 77.8M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (99s) 82s ago 99s 16.6M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (106s) 82s ago 105s 12.6M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (97s) 86s ago 96s 14.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (103s) 86s ago 102s 20.2M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:9283,8765,8443 running (8m) 82s ago 8m 498M - 19.1.1-373-g4e45e6b4 164465d2334a c324ee791e08 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (6m) 86s ago 6m 441M - 19.1.1-373-g4e45e6b4 164465d2334a e9db9486fd2f 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (8m) 82s ago 8m 47.5M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:34:05.207 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (6m) 86s ago 6m 38.4M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (7m) 82s ago 7m 15.9M - 1.5.0 0da6a335fe13 b4a8255a594a 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (6m) 86s ago 6m 16.5M - 1.5.0 0da6a335fe13 88246b15ca5d 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (5m) 82s ago 5m 48.5M 3948M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (4m) 82s ago 4m 50.4M 3948M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (4m) 82s ago 4m 51.8M 3948M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (3m) 86s ago 3m 45.7M 4972M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (3m) 86s ago 3m 48.5M 4972M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (2m) 86s ago 2m 41.5M 4972M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:34:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (5m) 82s ago 6m 56.2M - 2.43.0 a07b618ecd1d 74a720908c7f 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.203+0000 7f0774913640 1 -- 172.21.15.7:0/1612698027 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f0770105780 con 0x7f075806e2b0 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f075806e2b0 msgr2=0x7f0758070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f075806e2b0 0x7f0758070770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0764009880 tx=0x7f07640058d0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07701023f0 msgr2=0x7f0770197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07701023f0 0x7f0770197510 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f076000d8d0 tx=0x7f076000dd90 comp rx=0 tx=0).stop 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 shutdown_connections 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7f075806e2b0 0x7f0758070770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0770104d20 0x7f0770197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 --2- 172.21.15.7:0/1612698027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07701023f0 0x7f0770197510 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 >> 172.21.15.7:0/1612698027 conn(0x7f07700fc010 msgr2=0x7f07700fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 shutdown_connections 2024-09-21T04:34:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.206+0000 7f0775915640 1 -- 172.21.15.7:0/1612698027 wait complete. 2024-09-21T04:34:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.312+0000 7fc61a5cc640 1 -- 172.21.15.7:0/2418268221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614101960 msgr2=0x7fc614101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.312+0000 7fc61a5cc640 1 --2- 172.21.15.7:0/2418268221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614101960 0x7fc614101d60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc604009880 tx=0x7fc60402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.312+0000 7fc61a5cc640 1 -- 172.21.15.7:0/2418268221 shutdown_connections 2024-09-21T04:34:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.312+0000 7fc61a5cc640 1 --2- 172.21.15.7:0/2418268221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc614102b60 0x7fc614102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.312+0000 7fc61a5cc640 1 --2- 172.21.15.7:0/2418268221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614101960 0x7fc614101d60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.312+0000 7fc61a5cc640 1 -- 172.21.15.7:0/2418268221 >> 172.21.15.7:0/2418268221 conn(0x7fc6140fd0f0 msgr2=0x7fc6140ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.312+0000 7fc61a5cc640 1 -- 172.21.15.7:0/2418268221 shutdown_connections 2024-09-21T04:34:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.312+0000 7fc61a5cc640 1 -- 172.21.15.7:0/2418268221 wait complete. 2024-09-21T04:34:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.313+0000 7fc61a5cc640 1 Processor -- start 2024-09-21T04:34:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.313+0000 7fc61a5cc640 1 -- start start 2024-09-21T04:34:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.313+0000 7fc61a5cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc614101960 0x7fc6141974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.313+0000 7fc61a5cc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614102b60 0x7fc6141979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.313+0000 7fc61a5cc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc61410a780 con 0x7fc614101960 2024-09-21T04:34:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.313+0000 7fc61a5cc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc61410a600 con 0x7fc614102b60 2024-09-21T04:34:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614102b60 0x7fc6141979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc613fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc614101960 0x7fc6141974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614102b60 0x7fc6141979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:38314/0 (socket says 172.21.15.7:38314) 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc613fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc614101960 0x7fc6141974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50610/0 (socket says 172.21.15.7:50610) 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6137fe640 1 -- 172.21.15.7:0/487924784 learned_addr learned my addr 172.21.15.7:0/487924784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6137fe640 1 -- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc614101960 msgr2=0x7fc6141974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6137fe640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc614101960 0x7fc6141974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6137fe640 1 -- 172.21.15.7:0/487924784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc614198070 con 0x7fc614102b60 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc613fff640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc614101960 0x7fc6141974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6137fe640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614102b60 0x7fc6141979f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e9e0 tx=0x7fc60800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6117fa640 1 -- 172.21.15.7:0/487924784 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc60800cb50 con 0x7fc614102b60 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.314+0000 7fc6117fa640 1 -- 172.21.15.7:0/487924784 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc60800ccf0 con 0x7fc614102b60 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.315+0000 7fc6117fa640 1 -- 172.21.15.7:0/487924784 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc6080049d0 con 0x7fc614102b60 2024-09-21T04:34:05.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.315+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61419cab0 con 0x7fc614102b60 2024-09-21T04:34:05.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.315+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61419d040 con 0x7fc614102b60 2024-09-21T04:34:05.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.316+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc614101d60 con 0x7fc614102b60 2024-09-21T04:34:05.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.316+0000 7fc6117fa640 1 -- 172.21.15.7:0/487924784 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fc608004020 con 0x7fc614102b60 2024-09-21T04:34:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.317+0000 7fc6117fa640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc5fc06e360 0x7fc5fc070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.317+0000 7fc613fff640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc5fc06e360 0x7fc5fc070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.317+0000 7fc6117fa640 1 -- 172.21.15.7:0/487924784 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7fc60808e300 con 0x7fc614102b60 2024-09-21T04:34:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.318+0000 7fc613fff640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc5fc06e360 0x7fc5fc070820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc604009230 tx=0x7fc6040058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.322+0000 7fc6117fa640 1 -- 172.21.15.7:0/487924784 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc60805bbe0 con 0x7fc614102b60 2024-09-21T04:34:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.631+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc614062df0 con 0x7fc614102b60 2024-09-21T04:34:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.631+0000 7fc6117fa640 1 -- 172.21.15.7:0/487924784 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+633 (secure 0 0 0) 0x7fc60808e0c0 con 0x7fc614102b60 2024-09-21T04:34:05.634 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:34:05.635 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:34:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:34:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:34:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:34:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 14 2024-09-21T04:34:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:34:05.636 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc5fc06e360 msgr2=0x7fc5fc070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc5fc06e360 0x7fc5fc070820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc604009230 tx=0x7fc6040058d0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614102b60 msgr2=0x7fc6141979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614102b60 0x7fc6141979f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e9e0 tx=0x7fc60800eea0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 shutdown_connections 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc5fc06e360 0x7fc5fc070820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc614102b60 0x7fc6141979f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 --2- 172.21.15.7:0/487924784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc614101960 0x7fc6141974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 >> 172.21.15.7:0/487924784 conn(0x7fc6140fd0f0 msgr2=0x7fc614106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:05.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 shutdown_connections 2024-09-21T04:34:05.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.634+0000 7fc61a5cc640 1 -- 172.21.15.7:0/487924784 wait complete. 2024-09-21T04:34:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.722+0000 7fc6d6473640 1 -- 172.21.15.7:0/2627086752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 msgr2=0x7fc6d0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.722+0000 7fc6d6473640 1 --2- 172.21.15.7:0/2627086752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 0x7fc6d0100c60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009880 tx=0x7fc6c002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.722+0000 7fc6d6473640 1 -- 172.21.15.7:0/2627086752 shutdown_connections 2024-09-21T04:34:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.722+0000 7fc6d6473640 1 --2- 172.21.15.7:0/2627086752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d0101a60 0x7fc6d0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.722+0000 7fc6d6473640 1 --2- 172.21.15.7:0/2627086752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 0x7fc6d0100c60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.722+0000 7fc6d6473640 1 -- 172.21.15.7:0/2627086752 >> 172.21.15.7:0/2627086752 conn(0x7fc6d00fc010 msgr2=0x7fc6d00fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.722+0000 7fc6d6473640 1 -- 172.21.15.7:0/2627086752 shutdown_connections 2024-09-21T04:34:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.722+0000 7fc6d6473640 1 -- 172.21.15.7:0/2627086752 wait complete. 2024-09-21T04:34:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6d6473640 1 Processor -- start 2024-09-21T04:34:05.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6d6473640 1 -- start start 2024-09-21T04:34:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6d6473640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 0x7fc6d0197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6d6473640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d0101a60 0x7fc6d0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6d6473640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d01096a0 con 0x7fc6d0100860 2024-09-21T04:34:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6d6473640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0109520 con 0x7fc6d0101a60 2024-09-21T04:34:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 0x7fc6d0197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 0x7fc6d0197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50628/0 (socket says 172.21.15.7:50628) 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6cffff640 1 -- 172.21.15.7:0/2460224739 learned_addr learned my addr 172.21.15.7:0/2460224739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.723+0000 7fc6cf7fe640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d0101a60 0x7fc6d0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6cffff640 1 -- 172.21.15.7:0/2460224739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d0101a60 msgr2=0x7fc6d0197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6cffff640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d0101a60 0x7fc6d0197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6cffff640 1 -- 172.21.15.7:0/2460224739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d01980e0 con 0x7fc6d0100860 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6cffff640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 0x7fc6d0197520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006010 tx=0x7fc6c00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6cd7fa640 1 -- 172.21.15.7:0/2460224739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc6c003f070 con 0x7fc6d0100860 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6cd7fa640 1 -- 172.21.15.7:0/2460224739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc6c00049d0 con 0x7fc6d0100860 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d019cac0 con 0x7fc6d0100860 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6cd7fa640 1 -- 172.21.15.7:0/2460224739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc6c0035030 con 0x7fc6d0100860 2024-09-21T04:34:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.724+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d019cf90 con 0x7fc6d0100860 2024-09-21T04:34:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.725+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d0100c60 con 0x7fc6d0100860 2024-09-21T04:34:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.726+0000 7fc6cd7fa640 1 -- 172.21.15.7:0/2460224739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) ==== 92549+0+0 (secure 0 0 0) 0x7fc6c0042070 con 0x7fc6d0100860 2024-09-21T04:34:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.726+0000 7fc6cd7fa640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc6b806e310 0x7fc6b80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.727+0000 7fc6cd7fa640 1 -- 172.21.15.7:0/2460224739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5554+0+0 (secure 0 0 0) 0x7fc6c00aebc0 con 0x7fc6d0100860 2024-09-21T04:34:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.727+0000 7fc6cf7fe640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc6b806e310 0x7fc6b80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.727+0000 7fc6cf7fe640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc6b806e310 0x7fc6b80707d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc6d01989d0 tx=0x7fc6c4005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.731+0000 7fc6cd7fa640 1 -- 172.21.15.7:0/2460224739 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc6c00801f0 con 0x7fc6d0100860 2024-09-21T04:34:05.926 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T04:34:05.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.977+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 --> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc6d006a460 con 0x7fc6b806e310 2024-09-21T04:34:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.979+0000 7fc6cd7fa640 1 -- 172.21.15.7:0/2460224739 <== mgr.14217 v2:172.21.15.7:6800/4030185662 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+439 (secure 0 0 0) 0x7fc6d006a460 con 0x7fc6b806e310 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "0/2 daemons upgraded", 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 image on host smithi053", 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:34:05.982 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:34:05.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc6b806e310 msgr2=0x7fc6b80707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc6b806e310 0x7fc6b80707d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc6d01989d0 tx=0x7fc6c4005f90 comp rx=0 tx=0).stop 2024-09-21T04:34:05.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 msgr2=0x7fc6d0197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:05.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 0x7fc6d0197520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006010 tx=0x7fc6c00045e0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 shutdown_connections 2024-09-21T04:34:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:6800/4030185662,v1:172.21.15.7:6801/4030185662] conn(0x7fc6b806e310 0x7fc6b80707d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6d0101a60 0x7fc6d0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 --2- 172.21.15.7:0/2460224739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d0100860 0x7fc6d0197520 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 >> 172.21.15.7:0/2460224739 conn(0x7fc6d00fc010 msgr2=0x7fc6d0069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 shutdown_connections 2024-09-21T04:34:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:05.982+0000 7fc6d6473640 1 -- 172.21.15.7:0/2460224739 wait complete. 2024-09-21T04:34:06.176 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T04:34:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:05 smithi007 ceph-mon[27954]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:05 smithi007 ceph-mon[27954]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:05 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/487924784' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:34:06.293 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T04:34:06.294 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T04:34:06.294 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T04:34:06.334 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T04:34:06.338 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T04:34:06.338 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T04:34:06.354 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-09-21T04:34:06.355 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-21T04:34:06.355 DEBUG:teuthology.orchestra.run.smithi007:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-09-21T04:34:06.421 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir -p fsstress 2024-09-21T04:34:06.424 INFO:tasks.workunit.client.0.smithi007.stderr:+ pushd fsstress 2024-09-21T04:34:06.426 INFO:tasks.workunit.client.0.smithi007.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-21T04:34:06.426 INFO:tasks.workunit.client.0.smithi007.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-21T04:34:06.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:05 smithi053 ceph-mon[35554]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:06.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:05 smithi053 ceph-mon[35554]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:06.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:05 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/487924784' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:34:07.123 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:06 smithi053 ceph-mon[35554]: pgmap v229: 65 pgs: 65 active+clean; 5.3 MiB data, 196 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 405 KiB/s wr, 14 op/s 2024-09-21T04:34:07.123 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:06 smithi053 ceph-mon[35554]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:07.123 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:06 smithi053 ceph-mon[35554]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:07.173 INFO:tasks.workunit.client.0.smithi007.stderr:+ tar xzf ltp-full.tgz 2024-09-21T04:34:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:06 smithi007 ceph-mon[27954]: pgmap v229: 65 pgs: 65 active+clean; 5.3 MiB data, 196 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 405 KiB/s wr, 14 op/s 2024-09-21T04:34:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:06 smithi007 ceph-mon[27954]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:06 smithi007 ceph-mon[27954]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:08.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:08 smithi053 ceph-mon[35554]: pgmap v230: 65 pgs: 65 active+clean; 14 MiB data, 229 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-09-21T04:34:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:08 smithi053 ceph-mon[35554]: Upgrade: Updating mgr.smithi053.lnadao 2024-09-21T04:34:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:08 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:08 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:34:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:08 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:34:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:08 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:34:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:08 smithi053 ceph-mon[35554]: Deploying daemon mgr.smithi053.lnadao on smithi053 2024-09-21T04:34:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:08 smithi007 ceph-mon[27954]: pgmap v230: 65 pgs: 65 active+clean; 14 MiB data, 229 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-09-21T04:34:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:08 smithi007 ceph-mon[27954]: Upgrade: Updating mgr.smithi053.lnadao 2024-09-21T04:34:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:08 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:08 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:34:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:08 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:34:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:08 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:34:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:08 smithi007 ceph-mon[27954]: Deploying daemon mgr.smithi053.lnadao on smithi053 2024-09-21T04:34:10.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:10 smithi007 ceph-mon[27954]: pgmap v231: 65 pgs: 65 active+clean; 14 MiB data, 229 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 34 op/s 2024-09-21T04:34:10.908 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:10 smithi053 ceph-mon[35554]: pgmap v231: 65 pgs: 65 active+clean; 14 MiB data, 229 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 34 op/s 2024-09-21T04:34:12.755 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:12 smithi053 ceph-mon[35554]: pgmap v232: 65 pgs: 65 active+clean; 35 MiB data, 286 MiB used, 536 GiB / 536 GiB avail; 685 KiB/s rd, 2.9 MiB/s wr, 106 op/s 2024-09-21T04:34:12.755 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:12 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:12.755 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:12 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:12.755 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:12 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:34:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:12 smithi007 ceph-mon[27954]: pgmap v232: 65 pgs: 65 active+clean; 35 MiB data, 286 MiB used, 536 GiB / 536 GiB avail; 685 KiB/s rd, 2.9 MiB/s wr, 106 op/s 2024-09-21T04:34:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:12 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:12 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:12 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:34:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:14 smithi053 ceph-mon[35554]: pgmap v233: 65 pgs: 65 active+clean; 35 MiB data, 292 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 3.0 MiB/s wr, 111 op/s 2024-09-21T04:34:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:14 smithi007 ceph-mon[27954]: pgmap v233: 65 pgs: 65 active+clean; 35 MiB data, 292 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 3.0 MiB/s wr, 111 op/s 2024-09-21T04:34:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:16 smithi053 ceph-mon[35554]: pgmap v234: 65 pgs: 65 active+clean; 41 MiB data, 316 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 127 op/s 2024-09-21T04:34:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:16 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:16 smithi007 ceph-mon[27954]: pgmap v234: 65 pgs: 65 active+clean; 41 MiB data, 316 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 127 op/s 2024-09-21T04:34:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:16 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:18 smithi053 ceph-mon[35554]: pgmap v235: 65 pgs: 65 active+clean; 52 MiB data, 355 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 147 op/s 2024-09-21T04:34:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:18 smithi053 ceph-mon[35554]: Standby manager daemon smithi053.lnadao restarted 2024-09-21T04:34:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:18 smithi053 ceph-mon[35554]: Standby manager daemon smithi053.lnadao started 2024-09-21T04:34:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:18 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/crt"}]: dispatch 2024-09-21T04:34:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:18 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:34:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:18 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/key"}]: dispatch 2024-09-21T04:34:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:18 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:34:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:18 smithi007 ceph-mon[27954]: pgmap v235: 65 pgs: 65 active+clean; 52 MiB data, 355 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 147 op/s 2024-09-21T04:34:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:18 smithi007 ceph-mon[27954]: Standby manager daemon smithi053.lnadao restarted 2024-09-21T04:34:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:18 smithi007 ceph-mon[27954]: Standby manager daemon smithi053.lnadao started 2024-09-21T04:34:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:18 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/crt"}]: dispatch 2024-09-21T04:34:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:18 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:34:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:18 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/key"}]: dispatch 2024-09-21T04:34:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:18 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:34:19.773 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:19 smithi053 ceph-mon[35554]: mgrmap e20: smithi007.huohfl(active, since 6m), standbys: smithi053.lnadao 2024-09-21T04:34:19.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:19 smithi007 ceph-mon[27954]: mgrmap e20: smithi007.huohfl(active, since 6m), standbys: smithi053.lnadao 2024-09-21T04:34:20.937 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:20 smithi053 ceph-mon[35554]: pgmap v236: 65 pgs: 65 active+clean; 52 MiB data, 355 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 124 op/s 2024-09-21T04:34:20.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:20 smithi007 ceph-mon[27954]: pgmap v236: 65 pgs: 65 active+clean; 52 MiB data, 355 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 124 op/s 2024-09-21T04:34:22.745 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:22 smithi007 ceph-mon[27954]: pgmap v237: 65 pgs: 65 active+clean; 75 MiB data, 424 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 208 op/s 2024-09-21T04:34:22.754 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:22 smithi053 ceph-mon[35554]: pgmap v237: 65 pgs: 65 active+clean; 75 MiB data, 424 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 208 op/s 2024-09-21T04:34:24.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:24.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:24.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:34:24.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:34:24.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:24.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:34:24.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr fail", "who": "smithi007.huohfl"}]: dispatch 2024-09-21T04:34:24.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: Activating manager daemon smithi053.lnadao 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "mgr fail", "who": "smithi007.huohfl"}]': finished 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: osdmap e45: 6 total, 6 up, 6 in 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr metadata", "who": "smithi053.lnadao", "id": "smithi053.lnadao"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: mgrmap e21: smithi053.lnadao(active, starting, since 0.0109607s) 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:34:24.416 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:34:24.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:34:24.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:34:24.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:34:24.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:34:24.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:34:24.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:24 smithi053 ceph-mon[35554]: Manager daemon smithi053.lnadao is now available 2024-09-21T04:34:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:34:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:34:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr fail", "who": "smithi007.huohfl"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: Activating manager daemon smithi053.lnadao 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.14217 172.21.15.7:0/3640135938' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "mgr fail", "who": "smithi007.huohfl"}]': finished 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: osdmap e45: 6 total, 6 up, 6 in 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr metadata", "who": "smithi053.lnadao", "id": "smithi053.lnadao"}]: dispatch 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: mgrmap e21: smithi053.lnadao(active, starting, since 0.0109607s) 2024-09-21T04:34:24.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:34:24.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:24 smithi007 ceph-mon[27954]: Manager daemon smithi053.lnadao is now available 2024-09-21T04:34:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.lnadao/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.lnadao/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.lnadao/trash_purge_schedule"}]: dispatch 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.lnadao/trash_purge_schedule"}]: dispatch 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: mgrmap e22: smithi053.lnadao(active, since 1.01157s) 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:34:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:25 smithi053 ceph-mon[35554]: pgmap v3: 65 pgs: 65 active+clean; 93 MiB data, 478 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.lnadao/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.lnadao/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.lnadao/trash_purge_schedule"}]: dispatch 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi053.lnadao/trash_purge_schedule"}]: dispatch 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: mgrmap e22: smithi053.lnadao(active, since 1.01157s) 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:34:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:25 smithi007 ceph-mon[27954]: pgmap v3: 65 pgs: 65 active+clean; 93 MiB data, 478 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:34:26.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:26 smithi053 ceph-mon[35554]: Deploying cephadm binary to smithi053 2024-09-21T04:34:26.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:26 smithi053 ceph-mon[35554]: Deploying cephadm binary to smithi007 2024-09-21T04:34:26.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:26 smithi053 ceph-mon[35554]: pgmap v4: 65 pgs: 65 active+clean; 93 MiB data, 478 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:34:26.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:26 smithi007 ceph-mon[27954]: Deploying cephadm binary to smithi053 2024-09-21T04:34:26.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:26 smithi007 ceph-mon[27954]: Deploying cephadm binary to smithi007 2024-09-21T04:34:26.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:26 smithi007 ceph-mon[27954]: pgmap v4: 65 pgs: 65 active+clean; 93 MiB data, 478 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:34:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:27 smithi053 ceph-mon[35554]: [21/Sep/2024:04:34:25] ENGINE Bus STARTING 2024-09-21T04:34:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:27 smithi053 ceph-mon[35554]: [21/Sep/2024:04:34:25] ENGINE Serving on https://172.21.15.53:7150 2024-09-21T04:34:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:27 smithi053 ceph-mon[35554]: [21/Sep/2024:04:34:25] ENGINE Error in HTTPServer.serve 2024-09-21T04:34:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Traceback (most recent call last): 2024-09-21T04:34:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:34:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:34:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._run(expiration_interval) 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s = self.context.wrap_socket( 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: return self.sslsocket_class._create( 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self.do_handshake() 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._sslobj.do_handshake() 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:27 smithi053 ceph-mon[35554]: [21/Sep/2024:04:34:25] ENGINE Serving on http://172.21.15.53:8765 2024-09-21T04:34:27.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:27 smithi053 ceph-mon[35554]: [21/Sep/2024:04:34:25] ENGINE Bus STARTED 2024-09-21T04:34:27.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:27 smithi053 ceph-mon[35554]: mgrmap e23: smithi053.lnadao(active, since 3s) 2024-09-21T04:34:27.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:27 smithi007 ceph-mon[27954]: [21/Sep/2024:04:34:25] ENGINE Bus STARTING 2024-09-21T04:34:27.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:27 smithi007 ceph-mon[27954]: [21/Sep/2024:04:34:25] ENGINE Serving on https://172.21.15.53:7150 2024-09-21T04:34:27.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:27 smithi007 ceph-mon[27954]: [21/Sep/2024:04:34:25] ENGINE Error in HTTPServer.serve 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Traceback (most recent call last): 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._run(expiration_interval) 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s = self.context.wrap_socket( 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:34:27.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: return self.sslsocket_class._create( 2024-09-21T04:34:27.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:34:27.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self.do_handshake() 2024-09-21T04:34:27.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:34:27.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._sslobj.do_handshake() 2024-09-21T04:34:27.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:34:27.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:27 smithi007 ceph-mon[27954]: [21/Sep/2024:04:34:25] ENGINE Serving on http://172.21.15.53:8765 2024-09-21T04:34:27.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:27 smithi007 ceph-mon[27954]: [21/Sep/2024:04:34:25] ENGINE Bus STARTED 2024-09-21T04:34:27.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:27 smithi007 ceph-mon[27954]: mgrmap e23: smithi053.lnadao(active, since 3s) 2024-09-21T04:34:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:28 smithi007 ceph-mon[27954]: pgmap v5: 65 pgs: 65 active+clean; 93 MiB data, 478 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:34:28.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:28 smithi053 ceph-mon[35554]: pgmap v5: 65 pgs: 65 active+clean; 93 MiB data, 478 MiB used, 536 GiB / 536 GiB avail 2024-09-21T04:34:29.876 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:29 smithi007 ceph-mon[27954]: mgrmap e24: smithi053.lnadao(active, since 5s) 2024-09-21T04:34:29.876 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:29 smithi007 ceph-mon[27954]: pgmap v6: 65 pgs: 65 active+clean; 93 MiB data, 477 MiB used, 536 GiB / 536 GiB avail; 205 B/s rd, 163 KiB/s wr, 34 op/s 2024-09-21T04:34:29.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:29 smithi053 ceph-mon[35554]: mgrmap e24: smithi053.lnadao(active, since 5s) 2024-09-21T04:34:29.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:29 smithi053 ceph-mon[35554]: pgmap v6: 65 pgs: 65 active+clean; 93 MiB data, 477 MiB used, 536 GiB / 536 GiB avail; 205 B/s rd, 163 KiB/s wr, 34 op/s 2024-09-21T04:34:30.915 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:30 smithi007 ceph-mon[27954]: Standby manager daemon smithi007.huohfl started 2024-09-21T04:34:30.915 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:30 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.7:0/496377377' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.huohfl/crt"}]: dispatch 2024-09-21T04:34:30.915 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:30 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.7:0/496377377' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:34:30.915 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:30 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.7:0/496377377' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.huohfl/key"}]: dispatch 2024-09-21T04:34:30.915 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:30 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.7:0/496377377' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:34:30.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:30 smithi053 ceph-mon[35554]: Standby manager daemon smithi007.huohfl started 2024-09-21T04:34:30.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:30 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.7:0/496377377' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.huohfl/crt"}]: dispatch 2024-09-21T04:34:30.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:30 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.7:0/496377377' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:34:30.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:30 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.7:0/496377377' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.huohfl/key"}]: dispatch 2024-09-21T04:34:30.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:30 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.7:0/496377377' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:34:31.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:31 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr metadata", "who": "smithi007.huohfl", "id": "smithi007.huohfl"}]: dispatch 2024-09-21T04:34:31.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:31 smithi007 ceph-mon[27954]: mgrmap e25: smithi053.lnadao(active, since 7s), standbys: smithi007.huohfl 2024-09-21T04:34:31.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:31 smithi007 ceph-mon[27954]: pgmap v7: 65 pgs: 65 active+clean; 103 MiB data, 510 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-21T04:34:31.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:31 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr metadata", "who": "smithi007.huohfl", "id": "smithi007.huohfl"}]: dispatch 2024-09-21T04:34:31.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:31 smithi053 ceph-mon[35554]: mgrmap e25: smithi053.lnadao(active, since 7s), standbys: smithi007.huohfl 2024-09-21T04:34:31.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:31 smithi053 ceph-mon[35554]: pgmap v7: 65 pgs: 65 active+clean; 103 MiB data, 510 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-21T04:34:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:33 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:33.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:33 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:34.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:34 smithi053 ceph-mon[35554]: Adjusting osd_memory_target on smithi007 to 1217M 2024-09-21T04:34:34.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:34 smithi053 ceph-mon[35554]: pgmap v8: 65 pgs: 65 active+clean; 103 MiB data, 510 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-09-21T04:34:34.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:34 smithi007 ceph-mon[27954]: Adjusting osd_memory_target on smithi007 to 1217M 2024-09-21T04:34:34.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:34 smithi007 ceph-mon[27954]: pgmap v8: 65 pgs: 65 active+clean; 103 MiB data, 510 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-09-21T04:34:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.084+0000 7fbc9a63e640 1 -- 172.21.15.7:0/2357314784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941023f0 msgr2=0x7fbc941047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.084+0000 7fbc9a63e640 1 --2- 172.21.15.7:0/2357314784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941023f0 0x7fbc941047e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc840098b0 tx=0x7fbc8402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.084+0000 7fbc9a63e640 1 -- 172.21.15.7:0/2357314784 shutdown_connections 2024-09-21T04:34:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.084+0000 7fbc9a63e640 1 --2- 172.21.15.7:0/2357314784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc94104d20 0x7fbc94107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.084+0000 7fbc9a63e640 1 --2- 172.21.15.7:0/2357314784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941023f0 0x7fbc941047e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.084+0000 7fbc9a63e640 1 -- 172.21.15.7:0/2357314784 >> 172.21.15.7:0/2357314784 conn(0x7fbc940fc010 msgr2=0x7fbc940fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.084+0000 7fbc9a63e640 1 -- 172.21.15.7:0/2357314784 shutdown_connections 2024-09-21T04:34:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.084+0000 7fbc9a63e640 1 -- 172.21.15.7:0/2357314784 wait complete. 2024-09-21T04:34:36.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc9a63e640 1 Processor -- start 2024-09-21T04:34:36.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc9a63e640 1 -- start start 2024-09-21T04:34:36.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc9a63e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc941023f0 0x7fbc94101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc9a63e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94104d20 0x7fbc941003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc9a63e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9410a7c0 con 0x7fbc94104d20 2024-09-21T04:34:36.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc9a63e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9410a640 con 0x7fbc941023f0 2024-09-21T04:34:36.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc8bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94104d20 0x7fbc941003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc8bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94104d20 0x7fbc941003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56970/0 (socket says 172.21.15.7:56970) 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc8bfff640 1 -- 172.21.15.7:0/1219291002 learned_addr learned my addr 172.21.15.7:0/1219291002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.085+0000 7fbc93fff640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc941023f0 0x7fbc94101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.086+0000 7fbc8bfff640 1 -- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc941023f0 msgr2=0x7fbc94101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.086+0000 7fbc8bfff640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc941023f0 0x7fbc94101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.086+0000 7fbc8bfff640 1 -- 172.21.15.7:0/1219291002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9419b130 con 0x7fbc94104d20 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.086+0000 7fbc8bfff640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94104d20 0x7fbc941003a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00e940 tx=0x7fbc7c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.086+0000 7fbc91ffb640 1 -- 172.21.15.7:0/1219291002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbc7c00cac0 con 0x7fbc94104d20 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.086+0000 7fbc91ffb640 1 -- 172.21.15.7:0/1219291002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc7c0054e0 con 0x7fbc94104d20 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.086+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc941008e0 con 0x7fbc94104d20 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.086+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc94100e10 con 0x7fbc94104d20 2024-09-21T04:34:36.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.087+0000 7fbc91ffb640 1 -- 172.21.15.7:0/1219291002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbc7c004e40 con 0x7fbc94104d20 2024-09-21T04:34:36.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.088+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc9410afd0 con 0x7fbc94104d20 2024-09-21T04:34:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.089+0000 7fbc91ffb640 1 -- 172.21.15.7:0/1219291002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7fbc7c004fe0 con 0x7fbc94104d20 2024-09-21T04:34:36.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.090+0000 7fbc91ffb640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fbc80074ad0 0x7fbc80076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.090+0000 7fbc91ffb640 1 -- 172.21.15.7:0/1219291002 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7fbc7c0968a0 con 0x7fbc94104d20 2024-09-21T04:34:36.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.090+0000 7fbc93fff640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fbc80074ad0 0x7fbc80076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.091+0000 7fbc93fff640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fbc80074ad0 0x7fbc80076f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbc84006010 tx=0x7fbc84005a20 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:36.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.096+0000 7fbc91ffb640 1 -- 172.21.15.7:0/1219291002 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fbc9410afd0 con 0x7fbc94104d20 2024-09-21T04:34:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.393+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbc94105680 con 0x7fbc80074ad0 2024-09-21T04:34:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.395+0000 7fbc91ffb640 1 -- 172.21.15.7:0/1219291002 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7fbc94105680 con 0x7fbc80074ad0 2024-09-21T04:34:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.398+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fbc80074ad0 msgr2=0x7fbc80076f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.398+0000 7fbc9a63e640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fbc80074ad0 0x7fbc80076f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbc84006010 tx=0x7fbc84005a20 comp rx=0 tx=0).stop 2024-09-21T04:34:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.398+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94104d20 msgr2=0x7fbc941003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.398+0000 7fbc9a63e640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94104d20 0x7fbc941003a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00e940 tx=0x7fbc7c00ee00 comp rx=0 tx=0).stop 2024-09-21T04:34:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.399+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 shutdown_connections 2024-09-21T04:34:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.399+0000 7fbc9a63e640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fbc80074ad0 0x7fbc80076f90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.399+0000 7fbc9a63e640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94104d20 0x7fbc941003a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.399+0000 7fbc9a63e640 1 --2- 172.21.15.7:0/1219291002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc941023f0 0x7fbc94101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.399+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 >> 172.21.15.7:0/1219291002 conn(0x7fbc940fc010 msgr2=0x7fbc940fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.399+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 shutdown_connections 2024-09-21T04:34:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.399+0000 7fbc9a63e640 1 -- 172.21.15.7:0/1219291002 wait complete. 2024-09-21T04:34:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:34:36.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.505+0000 7fee782b2640 1 -- 172.21.15.7:0/1360156594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 msgr2=0x7fee70101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.505+0000 7fee782b2640 1 --2- 172.21.15.7:0/1360156594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 0x7fee70101f00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fee60009880 tx=0x7fee6002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.505+0000 7fee782b2640 1 -- 172.21.15.7:0/1360156594 shutdown_connections 2024-09-21T04:34:36.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.505+0000 7fee782b2640 1 --2- 172.21.15.7:0/1360156594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 0x7fee70101f00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.505+0000 7fee782b2640 1 --2- 172.21.15.7:0/1360156594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee70100880 0x7fee70100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.505+0000 7fee782b2640 1 -- 172.21.15.7:0/1360156594 >> 172.21.15.7:0/1360156594 conn(0x7fee700fc010 msgr2=0x7fee700fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:36.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.505+0000 7fee782b2640 1 -- 172.21.15.7:0/1360156594 shutdown_connections 2024-09-21T04:34:36.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.505+0000 7fee782b2640 1 -- 172.21.15.7:0/1360156594 wait complete. 2024-09-21T04:34:36.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.506+0000 7fee782b2640 1 Processor -- start 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.506+0000 7fee782b2640 1 -- start start 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.506+0000 7fee782b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee70100880 0x7fee701974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.506+0000 7fee782b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 0x7fee70197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.506+0000 7fee782b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee701096a0 con 0x7fee70101a80 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.506+0000 7fee782b2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee70109520 con 0x7fee70100880 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.507+0000 7fee7584d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 0x7fee70197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.507+0000 7fee7584d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 0x7fee70197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56990/0 (socket says 172.21.15.7:56990) 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.507+0000 7fee7584d640 1 -- 172.21.15.7:0/3944330545 learned_addr learned my addr 172.21.15.7:0/3944330545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.507+0000 7fee7604e640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee70100880 0x7fee701974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.507+0000 7fee7584d640 1 -- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee70100880 msgr2=0x7fee701974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.507+0000 7fee7584d640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee70100880 0x7fee701974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.507+0000 7fee7584d640 1 -- 172.21.15.7:0/3944330545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee701980a0 con 0x7fee70101a80 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.507+0000 7fee7584d640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 0x7fee70197a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fee60009230 tx=0x7fee60004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.508+0000 7fee677fe640 1 -- 172.21.15.7:0/3944330545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee6003f070 con 0x7fee70101a80 2024-09-21T04:34:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.508+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7019ca80 con 0x7fee70101a80 2024-09-21T04:34:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.508+0000 7fee677fe640 1 -- 172.21.15.7:0/3944330545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee60004a90 con 0x7fee70101a80 2024-09-21T04:34:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.508+0000 7fee677fe640 1 -- 172.21.15.7:0/3944330545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee60035030 con 0x7fee70101a80 2024-09-21T04:34:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.508+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee70102ca0 con 0x7fee70101a80 2024-09-21T04:34:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.510+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee70100c80 con 0x7fee70101a80 2024-09-21T04:34:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.510+0000 7fee677fe640 1 -- 172.21.15.7:0/3944330545 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7fee60042070 con 0x7fee70101a80 2024-09-21T04:34:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.511+0000 7fee677fe640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fee5c078e90 0x7fee5c07b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.511+0000 7fee677fe640 1 -- 172.21.15.7:0/3944330545 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7fee600b6ef0 con 0x7fee70101a80 2024-09-21T04:34:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.511+0000 7fee7604e640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fee5c078e90 0x7fee5c07b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.512+0000 7fee7604e640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fee5c078e90 0x7fee5c07b350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee701018e0 tx=0x7fee6c005f90 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:36.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.519+0000 7fee677fe640 1 -- 172.21.15.7:0/3944330545 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fee60082400 con 0x7fee70101a80 2024-09-21T04:34:36.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:36 smithi053 ceph-mon[35554]: pgmap v9: 65 pgs: 65 active+clean; 118 MiB data, 565 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-09-21T04:34:36.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:36 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:36.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:36 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:36 smithi007 ceph-mon[27954]: pgmap v9: 65 pgs: 65 active+clean; 118 MiB data, 565 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-09-21T04:34:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:36 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:36 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:36.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.795+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fee7006ac40 con 0x7fee5c078e90 2024-09-21T04:34:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.797+0000 7fee677fe640 1 -- 172.21.15.7:0/3944330545 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7fee7006ac40 con 0x7fee5c078e90 2024-09-21T04:34:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fee5c078e90 msgr2=0x7fee5c07b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fee5c078e90 0x7fee5c07b350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee701018e0 tx=0x7fee6c005f90 comp rx=0 tx=0).stop 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 msgr2=0x7fee70197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 0x7fee70197a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fee60009230 tx=0x7fee60004570 comp rx=0 tx=0).stop 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 shutdown_connections 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fee5c078e90 0x7fee5c07b350 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee70101a80 0x7fee70197a20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 --2- 172.21.15.7:0/3944330545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee70100880 0x7fee701974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 >> 172.21.15.7:0/3944330545 conn(0x7fee700fc010 msgr2=0x7fee7006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 shutdown_connections 2024-09-21T04:34:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.799+0000 7fee782b2640 1 -- 172.21.15.7:0/3944330545 wait complete. 2024-09-21T04:34:36.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.901+0000 7f7336b97640 1 -- 172.21.15.7:0/1596715151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 msgr2=0x7f7330100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.901+0000 7f7336b97640 1 --2- 172.21.15.7:0/1596715151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 0x7f7330100c60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7324009880 tx=0x7f732402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.901+0000 7f7336b97640 1 -- 172.21.15.7:0/1596715151 shutdown_connections 2024-09-21T04:34:36.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.901+0000 7f7336b97640 1 --2- 172.21.15.7:0/1596715151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330101a60 0x7f7330101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.901+0000 7f7336b97640 1 --2- 172.21.15.7:0/1596715151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 0x7f7330100c60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.901+0000 7f7336b97640 1 -- 172.21.15.7:0/1596715151 >> 172.21.15.7:0/1596715151 conn(0x7f73300fc010 msgr2=0x7f73300fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:36.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.901+0000 7f7336b97640 1 -- 172.21.15.7:0/1596715151 shutdown_connections 2024-09-21T04:34:36.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.901+0000 7f7336b97640 1 -- 172.21.15.7:0/1596715151 wait complete. 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.902+0000 7f7336b97640 1 Processor -- start 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.902+0000 7f7336b97640 1 -- start start 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.902+0000 7f7336b97640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 0x7f7330104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.902+0000 7f7336b97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330101a60 0x7f7330102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.902+0000 7f7336b97640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73301096a0 con 0x7f7330100860 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.902+0000 7f7336b97640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7330109520 con 0x7f7330101a60 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f7334933640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 0x7f7330104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f7334933640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 0x7f7330104780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57008/0 (socket says 172.21.15.7:57008) 2024-09-21T04:34:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f7334933640 1 -- 172.21.15.7:0/1698587200 learned_addr learned my addr 172.21.15.7:0/1698587200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:36.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f732ffff640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330101a60 0x7f7330102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f7334933640 1 -- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330101a60 msgr2=0x7f7330102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:36.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f7334933640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330101a60 0x7f7330102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:36.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f7334933640 1 -- 172.21.15.7:0/1698587200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7330103380 con 0x7f7330100860 2024-09-21T04:34:36.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f732ffff640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330101a60 0x7f7330102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:34:36.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.903+0000 7f7334933640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 0x7f7330104780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7324006010 tx=0x7f73240045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.904+0000 7f732dffb640 1 -- 172.21.15.7:0/1698587200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f732403f070 con 0x7f7330100860 2024-09-21T04:34:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.904+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7330103610 con 0x7f7330100860 2024-09-21T04:34:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.904+0000 7f732dffb640 1 -- 172.21.15.7:0/1698587200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f73240049d0 con 0x7f7330100860 2024-09-21T04:34:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.904+0000 7f732dffb640 1 -- 172.21.15.7:0/1698587200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7324035030 con 0x7f7330100860 2024-09-21T04:34:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.904+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7330103b40 con 0x7f7330100860 2024-09-21T04:34:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.905+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7330100c60 con 0x7f7330100860 2024-09-21T04:34:36.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.907+0000 7f732dffb640 1 -- 172.21.15.7:0/1698587200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f7324042070 con 0x7f7330100860 2024-09-21T04:34:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.908+0000 7f732dffb640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f7320074a80 0x7f7320076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.908+0000 7f732dffb640 1 -- 172.21.15.7:0/1698587200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f73240b70f0 con 0x7f7330100860 2024-09-21T04:34:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.908+0000 7f732ffff640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f7320074a80 0x7f7320076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:36.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.909+0000 7f732ffff640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f7320074a80 0x7f7320076f40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7330104500 tx=0x7f731c005f90 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:36.913+0000 7f732dffb640 1 -- 172.21.15.7:0/1698587200 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f7324082740 con 0x7f7330100860 2024-09-21T04:34:37.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.208+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f733019f550 con 0x7f7320074a80 2024-09-21T04:34:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.219+0000 7f732dffb640 1 -- 172.21.15.7:0/1698587200 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f733019f550 con 0x7f7320074a80 2024-09-21T04:34:37.222 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (6m) 5s ago 7m 24.6M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (7m) 5s ago 7m 8656k - 19.1.1-373-g4e45e6b4 164465d2334a f80c59780c91 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (6m) 1s ago 6m 5268k - 19.1.1-373-g4e45e6b4 164465d2334a 828ddad2141a 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (7m) 5s ago 7m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (6m) 1s ago 6m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (5m) 5s ago 7m 77.5M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (2m) 5s ago 2m 122M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (2m) 5s ago 2m 103M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (2m) 1s ago 2m 19.7M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (2m) 1s ago 2m 25.7M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:9283,8765,8443 running (8m) 5s ago 8m 250M - 19.1.1-373-g4e45e6b4 164465d2334a c324ee791e08 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (26s) 1s ago 6m 499M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7d35aae8ab52 2024-09-21T04:34:37.223 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (9m) 5s ago 9m 52.9M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (6m) 1s ago 6m 43.9M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (7m) 5s ago 7m 15.8M - 1.5.0 0da6a335fe13 b4a8255a594a 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (6m) 1s ago 6m 16.8M - 1.5.0 0da6a335fe13 88246b15ca5d 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (5m) 5s ago 5m 70.3M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (5m) 5s ago 5m 78.0M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (4m) 5s ago 4m 73.5M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (4m) 1s ago 4m 76.8M 4972M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (3m) 1s ago 3m 80.3M 4972M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (3m) 1s ago 3m 64.5M 4972M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:34:37.224 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (5m) 5s ago 7m 56.1M - 2.43.0 a07b618ecd1d 74a720908c7f 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f7320074a80 msgr2=0x7f7320076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f7320074a80 0x7f7320076f40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7330104500 tx=0x7f731c005f90 comp rx=0 tx=0).stop 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 msgr2=0x7f7330104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 0x7f7330104780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7324006010 tx=0x7f73240045e0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 shutdown_connections 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f7320074a80 0x7f7320076f40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330101a60 0x7f7330102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 --2- 172.21.15.7:0/1698587200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7330100860 0x7f7330104780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 >> 172.21.15.7:0/1698587200 conn(0x7f73300fc010 msgr2=0x7f7330069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 shutdown_connections 2024-09-21T04:34:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.223+0000 7f7336b97640 1 -- 172.21.15.7:0/1698587200 wait complete. 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.339+0000 7f1448d4b640 1 -- 172.21.15.7:0/3612469113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 msgr2=0x7f1444100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.339+0000 7f144159a640 1 -- 172.21.15.7:0/3612469113 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1434035020 con 0x7f14440ffc10 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.339+0000 7f1448d4b640 1 --2- 172.21.15.7:0/3612469113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 0x7f1444100010 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1434009880 tx=0x7f143402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.340+0000 7f1448d4b640 1 -- 172.21.15.7:0/3612469113 shutdown_connections 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.340+0000 7f1448d4b640 1 --2- 172.21.15.7:0/3612469113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1444102bf0 0x7f1444103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.340+0000 7f1448d4b640 1 --2- 172.21.15.7:0/3612469113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 0x7f1444100010 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.340+0000 7f1448d4b640 1 -- 172.21.15.7:0/3612469113 >> 172.21.15.7:0/3612469113 conn(0x7f14440fb760 msgr2=0x7f14440fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.340+0000 7f1448d4b640 1 -- 172.21.15.7:0/3612469113 shutdown_connections 2024-09-21T04:34:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.340+0000 7f1448d4b640 1 -- 172.21.15.7:0/3612469113 wait complete. 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.341+0000 7f1448d4b640 1 Processor -- start 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.341+0000 7f1448d4b640 1 -- start start 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.341+0000 7f1448d4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 0x7f144419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.341+0000 7f1448d4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1444102bf0 0x7f144419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.341+0000 7f1448d4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f144410a770 con 0x7f14440ffc10 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.341+0000 7f1448d4b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f144410a5f0 con 0x7f1444102bf0 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.341+0000 7f144259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 0x7f144419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.342+0000 7f144259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 0x7f144419daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57032/0 (socket says 172.21.15.7:57032) 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.342+0000 7f144259c640 1 -- 172.21.15.7:0/3300167514 learned_addr learned my addr 172.21.15.7:0/3300167514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.342+0000 7f144259c640 1 -- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1444102bf0 msgr2=0x7f144419e030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.342+0000 7f144259c640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1444102bf0 0x7f144419e030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.342+0000 7f144259c640 1 -- 172.21.15.7:0/3300167514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144419e6b0 con 0x7f14440ffc10 2024-09-21T04:34:37.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.342+0000 7f144259c640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 0x7f144419daf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1434004f60 tx=0x7f14340045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:37.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.343+0000 7f14337fe640 1 -- 172.21.15.7:0/3300167514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f143403f070 con 0x7f14440ffc10 2024-09-21T04:34:37.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.343+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14441a3090 con 0x7f14440ffc10 2024-09-21T04:34:37.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.343+0000 7f14337fe640 1 -- 172.21.15.7:0/3300167514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14340049d0 con 0x7f14440ffc10 2024-09-21T04:34:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.344+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14441a3620 con 0x7f14440ffc10 2024-09-21T04:34:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.344+0000 7f14337fe640 1 -- 172.21.15.7:0/3300167514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1434035030 con 0x7f14440ffc10 2024-09-21T04:34:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.345+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1424005290 con 0x7f14440ffc10 2024-09-21T04:34:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.346+0000 7f14337fe640 1 -- 172.21.15.7:0/3300167514 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f1434042070 con 0x7f14440ffc10 2024-09-21T04:34:37.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.347+0000 7f14337fe640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f142c074ad0 0x7f142c076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:37.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.347+0000 7f14337fe640 1 -- 172.21.15.7:0/3300167514 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f14340b6d80 con 0x7f14440ffc10 2024-09-21T04:34:37.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.347+0000 7f1441d9b640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f142c074ad0 0x7f142c076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:37.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.348+0000 7f1441d9b640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f142c074ad0 0x7f142c076f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f144419efa0 tx=0x7f1438005f90 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.353+0000 7f14337fe640 1 -- 172.21.15.7:0/3300167514 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f1434082290 con 0x7f14440ffc10 2024-09-21T04:34:37.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:37 smithi053 ceph-mon[35554]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:37.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:37 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:37 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:37 smithi007 ceph-mon[27954]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:37 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:37 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:37.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.732+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f1424005850 con 0x7f14440ffc10 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.733+0000 7f14337fe640 1 -- 172.21.15.7:0/3300167514 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+841 (secure 0 0 0) 0x7f1434087140 con 0x7f14440ffc10 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:34:37.736 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 13, 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:34:37.737 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:34:37.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.735+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f142c074ad0 msgr2=0x7f142c076f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:37.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f142c074ad0 0x7f142c076f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f144419efa0 tx=0x7f1438005f90 comp rx=0 tx=0).stop 2024-09-21T04:34:37.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 msgr2=0x7f144419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:37.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 0x7f144419daf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1434004f60 tx=0x7f14340045e0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 shutdown_connections 2024-09-21T04:34:37.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f142c074ad0 0x7f142c076f90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1444102bf0 0x7f144419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 --2- 172.21.15.7:0/3300167514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14440ffc10 0x7f144419daf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 >> 172.21.15.7:0/3300167514 conn(0x7f14440fb760 msgr2=0x7f1444100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:37.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 shutdown_connections 2024-09-21T04:34:37.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.736+0000 7f1448d4b640 1 -- 172.21.15.7:0/3300167514 wait complete. 2024-09-21T04:34:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.854+0000 7f3fe8b79640 1 -- 172.21.15.7:0/3843863386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 msgr2=0x7f3fe41047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.854+0000 7f3fe8b79640 1 --2- 172.21.15.7:0/3843863386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 0x7f3fe41047e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40098b0 tx=0x7f3fd402a4d0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.854+0000 7f3fe8b79640 1 -- 172.21.15.7:0/3843863386 shutdown_connections 2024-09-21T04:34:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.854+0000 7f3fe8b79640 1 --2- 172.21.15.7:0/3843863386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4104d20 0x7f3fe4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.854+0000 7f3fe8b79640 1 --2- 172.21.15.7:0/3843863386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 0x7f3fe41047e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.854+0000 7f3fe8b79640 1 -- 172.21.15.7:0/3843863386 >> 172.21.15.7:0/3843863386 conn(0x7f3fe40fc010 msgr2=0x7f3fe40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.855+0000 7f3fe8b79640 1 -- 172.21.15.7:0/3843863386 shutdown_connections 2024-09-21T04:34:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.855+0000 7f3fe8b79640 1 -- 172.21.15.7:0/3843863386 wait complete. 2024-09-21T04:34:37.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.856+0000 7f3fe8b79640 1 Processor -- start 2024-09-21T04:34:37.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.856+0000 7f3fe8b79640 1 -- start start 2024-09-21T04:34:37.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.856+0000 7f3fe8b79640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 0x7f3fe41974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:37.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.856+0000 7f3fe8b79640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4104d20 0x7f3fe4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:37.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.856+0000 7f3fe8b79640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe410a7c0 con 0x7f3fe41023f0 2024-09-21T04:34:37.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.856+0000 7f3fe8b79640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe410a640 con 0x7f3fe4104d20 2024-09-21T04:34:37.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fd9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4104d20 0x7f3fe4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:37.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fe259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 0x7f3fe41974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fd9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4104d20 0x7f3fe4197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:35390/0 (socket says 172.21.15.7:35390) 2024-09-21T04:34:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fe259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 0x7f3fe41974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57050/0 (socket says 172.21.15.7:57050) 2024-09-21T04:34:37.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fd9d9b640 1 -- 172.21.15.7:0/1446860375 learned_addr learned my addr 172.21.15.7:0/1446860375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fd9d9b640 1 -- 172.21.15.7:0/1446860375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 msgr2=0x7f3fe41974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fd9d9b640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 0x7f3fe41974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fd9d9b640 1 -- 172.21.15.7:0/1446860375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe41980a0 con 0x7f3fe4104d20 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.857+0000 7f3fd9d9b640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4104d20 0x7f3fe4197a20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00e920 tx=0x7f3fcc00ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.858+0000 7f3fdbfff640 1 -- 172.21.15.7:0/1446860375 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3fcc00cb80 con 0x7f3fe4104d20 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.858+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe419cae0 con 0x7f3fe4104d20 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.858+0000 7f3fdbfff640 1 -- 172.21.15.7:0/1446860375 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3fcc00cd20 con 0x7f3fe4104d20 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.858+0000 7f3fdbfff640 1 -- 172.21.15.7:0/1446860375 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3fcc0049d0 con 0x7f3fe4104d20 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.858+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe419d070 con 0x7f3fe4104d20 2024-09-21T04:34:37.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.859+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe41027d0 con 0x7f3fe4104d20 2024-09-21T04:34:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.860+0000 7f3fdbfff640 1 -- 172.21.15.7:0/1446860375 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f3fcc004c80 con 0x7f3fe4104d20 2024-09-21T04:34:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.861+0000 7f3fdbfff640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f3fd0074a30 0x7f3fd0076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:34:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.861+0000 7f3fdbfff640 1 -- 172.21.15.7:0/1446860375 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f3fcc096550 con 0x7f3fe4104d20 2024-09-21T04:34:37.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.861+0000 7f3fe259c640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f3fd0074a30 0x7f3fd0076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:34:37.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.861+0000 7f3fe259c640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f3fd0074a30 0x7f3fd0076ef0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4006010 tx=0x7f3fd40058d0 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:34:37.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:37.866+0000 7f3fdbfff640 1 -- 172.21.15.7:0/1446860375 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f3fcc061a20 con 0x7f3fe4104d20 2024-09-21T04:34:38.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.153+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3fe4105680 con 0x7f3fd0074a30 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.155+0000 7f3fdbfff640 1 -- 172.21.15.7:0/1446860375 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7f3fe4105680 con 0x7f3fd0074a30 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "1/2 daemons upgraded", 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f3fd0074a30 msgr2=0x7f3fd0076ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f3fd0074a30 0x7f3fd0076ef0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4006010 tx=0x7f3fd40058d0 comp rx=0 tx=0).stop 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4104d20 msgr2=0x7f3fe4197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4104d20 0x7f3fe4197a20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00e920 tx=0x7f3fcc00ede0 comp rx=0 tx=0).stop 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 shutdown_connections 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f3fd0074a30 0x7f3fd0076ef0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fe4104d20 0x7f3fe4197a20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 --2- 172.21.15.7:0/1446860375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fe41023f0 0x7f3fe41974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:34:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 >> 172.21.15.7:0/1446860375 conn(0x7f3fe40fc010 msgr2=0x7f3fe40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:34:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 shutdown_connections 2024-09-21T04:34:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:34:38.158+0000 7f3fe8b79640 1 -- 172.21.15.7:0/1446860375 wait complete. 2024-09-21T04:34:38.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:38 smithi007 ceph-mon[27954]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:38.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:38 smithi007 ceph-mon[27954]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:38.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:38 smithi007 ceph-mon[27954]: pgmap v10: 65 pgs: 65 active+clean; 118 MiB data, 565 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-09-21T04:34:38.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:38 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3300167514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:34:38.591 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:38 smithi053 ceph-mon[35554]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:38.591 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:38 smithi053 ceph-mon[35554]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:38.591 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:38 smithi053 ceph-mon[35554]: pgmap v10: 65 pgs: 65 active+clean; 118 MiB data, 565 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-09-21T04:34:38.591 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:38 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3300167514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:34:39.603 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:39 smithi053 ceph-mon[35554]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:39.603 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:39 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:39 smithi007 ceph-mon[27954]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:34:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:39 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: pgmap v11: 65 pgs: 65 active+clean; 126 MiB data, 609 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:34:40.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:40 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:34:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: pgmap v11: 65 pgs: 65 active+clean; 126 MiB data, 609 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-09-21T04:34:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-21T04:34:40.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:40.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:34:40.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:40 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:34:41.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: Adjusting osd_memory_target on smithi053 to 2241M 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:34:41.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:34:41.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:41 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:34:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: Adjusting osd_memory_target on smithi053 to 2241M 2024-09-21T04:34:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-21T04:34:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-21T04:34:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:34:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:34:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:34:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:41 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:34:42.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:42 smithi053 ceph-mon[35554]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:34:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:42 smithi053 ceph-mon[35554]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:34:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:42 smithi053 ceph-mon[35554]: Reconfiguring ceph-exporter.smithi007 (dependencies changed)... 2024-09-21T04:34:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:42 smithi053 ceph-mon[35554]: Deploying daemon ceph-exporter.smithi007 on smithi007 2024-09-21T04:34:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:42 smithi053 ceph-mon[35554]: pgmap v12: 65 pgs: 65 active+clean; 138 MiB data, 634 MiB used, 536 GiB / 536 GiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-09-21T04:34:42.601 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:42 smithi007 ceph-mon[27954]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:34:42.601 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:42 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:34:42.601 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:42 smithi007 ceph-mon[27954]: Reconfiguring ceph-exporter.smithi007 (dependencies changed)... 2024-09-21T04:34:42.601 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:42 smithi007 ceph-mon[27954]: Deploying daemon ceph-exporter.smithi007 on smithi007 2024-09-21T04:34:42.601 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:42 smithi007 ceph-mon[27954]: pgmap v12: 65 pgs: 65 active+clean; 138 MiB data, 634 MiB used, 536 GiB / 536 GiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-09-21T04:34:44.618 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:44 smithi007 ceph-mon[27954]: pgmap v13: 65 pgs: 65 active+clean; 138 MiB data, 634 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 177 op/s 2024-09-21T04:34:44.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:44 smithi053 ceph-mon[35554]: pgmap v13: 65 pgs: 65 active+clean; 138 MiB data, 634 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 177 op/s 2024-09-21T04:34:46.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:46 smithi053 ceph-mon[35554]: pgmap v14: 65 pgs: 65 active+clean; 152 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-21T04:34:46.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:46 smithi007 ceph-mon[27954]: pgmap v14: 65 pgs: 65 active+clean; 152 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-21T04:34:48.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:48 smithi053 ceph-mon[35554]: pgmap v15: 65 pgs: 65 active+clean; 152 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 185 op/s 2024-09-21T04:34:48.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:48 smithi007 ceph-mon[27954]: pgmap v15: 65 pgs: 65 active+clean; 152 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 185 op/s 2024-09-21T04:34:50.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:50 smithi007 ceph-mon[27954]: pgmap v16: 65 pgs: 65 active+clean; 160 MiB data, 693 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 259 op/s 2024-09-21T04:34:50.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:50 smithi053 ceph-mon[35554]: pgmap v16: 65 pgs: 65 active+clean; 160 MiB data, 693 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 259 op/s 2024-09-21T04:34:52.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:52 smithi053 ceph-mon[35554]: pgmap v17: 65 pgs: 65 active+clean; 169 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-09-21T04:34:52.601 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:52 smithi007 ceph-mon[27954]: pgmap v17: 65 pgs: 65 active+clean; 169 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-09-21T04:34:53.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:53 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:53.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:53 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:53.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:53 smithi053 ceph-mon[35554]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-09-21T04:34:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:53 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:53 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:34:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:53 smithi007 ceph-mon[27954]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-09-21T04:34:54.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:54 smithi053 ceph-mon[35554]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-09-21T04:34:54.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:54 smithi053 ceph-mon[35554]: pgmap v18: 65 pgs: 65 active+clean; 169 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 248 op/s 2024-09-21T04:34:54.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:54 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:54 smithi007 ceph-mon[27954]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-09-21T04:34:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:54 smithi007 ceph-mon[27954]: pgmap v18: 65 pgs: 65 active+clean; 169 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 248 op/s 2024-09-21T04:34:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:54 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:34:56.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:56 smithi007 ceph-mon[27954]: pgmap v19: 65 pgs: 65 active+clean; 191 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 403 op/s 2024-09-21T04:34:56.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:56 smithi053 ceph-mon[35554]: pgmap v19: 65 pgs: 65 active+clean; 191 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 403 op/s 2024-09-21T04:34:56.840 INFO:tasks.workunit.client.1.smithi053.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-21T04:34:56.846 INFO:tasks.workunit.client.1.smithi053.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-21T04:34:56.846 INFO:tasks.workunit.client.1.smithi053.stderr:+ make 2024-09-21T04:34:57.116 INFO:tasks.workunit.client.1.smithi053.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-21T04:34:57.522 INFO:tasks.workunit.client.1.smithi053.stderr:++ readlink -f fsstress 2024-09-21T04:34:57.523 INFO:tasks.workunit.client.1.smithi053.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-21T04:34:57.523 INFO:tasks.workunit.client.1.smithi053.stderr:+ popd 2024-09-21T04:34:57.524 INFO:tasks.workunit.client.1.smithi053.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-21T04:34:57.525 INFO:tasks.workunit.client.1.smithi053.stderr:+ popd 2024-09-21T04:34:57.525 INFO:tasks.workunit.client.1.smithi053.stdout:~/cephtest/mnt.1/client.1/tmp 2024-09-21T04:34:57.525 INFO:tasks.workunit.client.1.smithi053.stderr:++ mktemp -d -p . 2024-09-21T04:34:57.528 INFO:tasks.workunit.client.1.smithi053.stderr:+ T=./tmp.92DWlWLYWz 2024-09-21T04:34:57.528 INFO:tasks.workunit.client.1.smithi053.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.92DWlWLYWz -l 1 -n 1000 -p 10 -v 2024-09-21T04:34:57.532 INFO:tasks.workunit.client.1.smithi053.stdout:seed = 1726371022 2024-09-21T04:34:57.535 INFO:tasks.workunit.client.1.smithi053.stdout:0/0: fdatasync - no filename 2024-09-21T04:34:57.535 INFO:tasks.workunit.client.1.smithi053.stdout:0/1: dwrite - no filename 2024-09-21T04:34:57.536 INFO:tasks.workunit.client.1.smithi053.stdout:0/2: dread - no filename 2024-09-21T04:34:57.536 INFO:tasks.workunit.client.1.smithi053.stdout:0/3: rmdir - no directory 2024-09-21T04:34:57.536 INFO:tasks.workunit.client.1.smithi053.stdout:1/0: fdatasync - no filename 2024-09-21T04:34:57.537 INFO:tasks.workunit.client.1.smithi053.stdout:1/1: dwrite - no filename 2024-09-21T04:34:57.537 INFO:tasks.workunit.client.1.smithi053.stdout:1/2: rename - no filename 2024-09-21T04:34:57.537 INFO:tasks.workunit.client.1.smithi053.stdout:1/3: truncate - no filename 2024-09-21T04:34:57.537 INFO:tasks.workunit.client.1.smithi053.stdout:1/4: stat - no entries 2024-09-21T04:34:57.537 INFO:tasks.workunit.client.1.smithi053.stdout:1/5: readlink - no filename 2024-09-21T04:34:57.537 INFO:tasks.workunit.client.1.smithi053.stdout:0/4: getdents . 0 2024-09-21T04:34:57.539 INFO:tasks.workunit.client.1.smithi053.stdout:1/6: creat f0 x:0 0 0 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/0: link - no file 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/1: read - no filename 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/2: write - no filename 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/3: chown . 8315382 1 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/4: dwrite - no filename 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/5: write - no filename 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/6: dwrite - no filename 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/7: fsync - no filename 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/8: link - no file 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/9: unlink - no file 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/10: fdatasync - no filename 2024-09-21T04:34:57.541 INFO:tasks.workunit.client.1.smithi053.stdout:3/11: dwrite - no filename 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:3/12: chown . 621551 1 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:3/13: unlink - no file 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:3/14: read - no filename 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:3/15: chown . 697513244 1 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:3/16: rmdir - no directory 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:3/17: dread - no filename 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:3/18: stat - no entries 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:2/0: getdents . 0 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:2/1: dread - no filename 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:2/2: chown . 0 1 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:2/3: readlink - no filename 2024-09-21T04:34:57.542 INFO:tasks.workunit.client.1.smithi053.stdout:2/4: rename - no filename 2024-09-21T04:34:57.543 INFO:tasks.workunit.client.1.smithi053.stdout:4/0: rename - no filename 2024-09-21T04:34:57.543 INFO:tasks.workunit.client.1.smithi053.stdout:1/7: link f0 f1 0 2024-09-21T04:34:57.545 INFO:tasks.workunit.client.1.smithi053.stdout:5/0: rename - no filename 2024-09-21T04:34:57.545 INFO:tasks.workunit.client.1.smithi053.stdout:5/1: link - no file 2024-09-21T04:34:57.545 INFO:tasks.workunit.client.1.smithi053.stdout:5/2: read - no filename 2024-09-21T04:34:57.545 INFO:tasks.workunit.client.1.smithi053.stdout:5/3: truncate - no filename 2024-09-21T04:34:57.545 INFO:tasks.workunit.client.1.smithi053.stdout:3/19: mknod c0 0 2024-09-21T04:34:57.545 INFO:tasks.workunit.client.1.smithi053.stdout:3/20: dwrite - no filename 2024-09-21T04:34:57.547 INFO:tasks.workunit.client.1.smithi053.stdout:2/5: creat f0 x:0 0 0 2024-09-21T04:34:57.547 INFO:tasks.workunit.client.1.smithi053.stdout:4/1: mknod c0 0 2024-09-21T04:34:57.547 INFO:tasks.workunit.client.1.smithi053.stdout:2/6: write f0 [928620,105754] 0 2024-09-21T04:34:57.548 INFO:tasks.workunit.client.1.smithi053.stdout:4/2: stat c0 0 2024-09-21T04:34:57.548 INFO:tasks.workunit.client.1.smithi053.stdout:6/0: chown . 5482 1 2024-09-21T04:34:57.550 INFO:tasks.workunit.client.1.smithi053.stdout:3/21: mknod c1 0 2024-09-21T04:34:57.550 INFO:tasks.workunit.client.1.smithi053.stdout:3/22: chown c0 63012 1 2024-09-21T04:34:57.550 INFO:tasks.workunit.client.1.smithi053.stdout:3/23: rmdir - no directory 2024-09-21T04:34:57.551 INFO:tasks.workunit.client.1.smithi053.stdout:3/24: chown c0 5 1 2024-09-21T04:34:57.552 INFO:tasks.workunit.client.1.smithi053.stdout:2/7: mkdir d1 0 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/4: mkdir d0 0 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/5: fsync - no filename 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/6: dread - no filename 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/7: dread - no filename 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/8: dwrite - no filename 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/9: dwrite - no filename 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/10: chown d0 186764 1 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/11: fsync - no filename 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/12: write - no filename 2024-09-21T04:34:57.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/13: chown d0 4965528 1 2024-09-21T04:34:57.554 INFO:tasks.workunit.client.1.smithi053.stdout:7/0: rename - no filename 2024-09-21T04:34:57.554 INFO:tasks.workunit.client.1.smithi053.stdout:7/1: truncate - no filename 2024-09-21T04:34:57.554 INFO:tasks.workunit.client.1.smithi053.stdout:7/2: rename - no filename 2024-09-21T04:34:57.554 INFO:tasks.workunit.client.1.smithi053.stdout:7/3: dwrite - no filename 2024-09-21T04:34:57.555 INFO:tasks.workunit.client.1.smithi053.stdout:7/4: rename - no filename 2024-09-21T04:34:57.555 INFO:tasks.workunit.client.1.smithi053.stdout:4/3: mkdir d1 0 2024-09-21T04:34:57.555 INFO:tasks.workunit.client.1.smithi053.stdout:4/4: readlink - no filename 2024-09-21T04:34:57.555 INFO:tasks.workunit.client.1.smithi053.stdout:4/5: stat c0 0 2024-09-21T04:34:57.557 INFO:tasks.workunit.client.1.smithi053.stdout:3/25: symlink l2 0 2024-09-21T04:34:57.560 INFO:tasks.workunit.client.1.smithi053.stdout:8/0: unlink - no file 2024-09-21T04:34:57.562 INFO:tasks.workunit.client.1.smithi053.stdout:4/6: unlink c0 0 2024-09-21T04:34:57.563 INFO:tasks.workunit.client.1.smithi053.stdout:4/7: dread - no filename 2024-09-21T04:34:57.563 INFO:tasks.workunit.client.1.smithi053.stdout:6/1: symlink l0 0 2024-09-21T04:34:57.564 INFO:tasks.workunit.client.1.smithi053.stdout:2/8: symlink d1/l2 0 2024-09-21T04:34:57.566 INFO:tasks.workunit.client.1.smithi053.stdout:3/26: creat f3 x:0 0 0 2024-09-21T04:34:57.566 INFO:tasks.workunit.client.1.smithi053.stdout:5/14: symlink d0/l1 0 2024-09-21T04:34:57.566 INFO:tasks.workunit.client.1.smithi053.stdout:5/15: fdatasync - no filename 2024-09-21T04:34:57.566 INFO:tasks.workunit.client.1.smithi053.stdout:5/16: dwrite - no filename 2024-09-21T04:34:57.567 INFO:tasks.workunit.client.1.smithi053.stdout:5/17: readlink d0/l1 0 2024-09-21T04:34:57.567 INFO:tasks.workunit.client.1.smithi053.stdout:5/18: write - no filename 2024-09-21T04:34:57.567 INFO:tasks.workunit.client.1.smithi053.stdout:5/19: write - no filename 2024-09-21T04:34:57.567 INFO:tasks.workunit.client.1.smithi053.stdout:5/20: dread - no filename 2024-09-21T04:34:57.567 INFO:tasks.workunit.client.1.smithi053.stdout:5/21: chown d0 26 1 2024-09-21T04:34:57.567 INFO:tasks.workunit.client.1.smithi053.stdout:5/22: dread - no filename 2024-09-21T04:34:57.567 INFO:tasks.workunit.client.1.smithi053.stdout:5/23: dwrite - no filename 2024-09-21T04:34:57.568 INFO:tasks.workunit.client.1.smithi053.stdout:5/24: fdatasync - no filename 2024-09-21T04:34:57.568 INFO:tasks.workunit.client.1.smithi053.stdout:5/25: chown d0/l1 28453109 1 2024-09-21T04:34:57.570 INFO:tasks.workunit.client.1.smithi053.stdout:7/5: mkdir d0 0 2024-09-21T04:34:57.570 INFO:tasks.workunit.client.1.smithi053.stdout:7/6: readlink - no filename 2024-09-21T04:34:57.571 INFO:tasks.workunit.client.1.smithi053.stdout:6/2: creat f1 x:0 0 0 2024-09-21T04:34:57.572 INFO:tasks.workunit.client.1.smithi053.stdout:6/3: truncate f1 307505 0 2024-09-21T04:34:57.572 INFO:tasks.workunit.client.1.smithi053.stdout:2/9: mknod d1/c3 0 2024-09-21T04:34:57.573 INFO:tasks.workunit.client.1.smithi053.stdout:5/26: symlink d0/l2 0 2024-09-21T04:34:57.579 INFO:tasks.workunit.client.1.smithi053.stdout:8/1: symlink l0 0 2024-09-21T04:34:57.579 INFO:tasks.workunit.client.1.smithi053.stdout:8/2: dwrite - no filename 2024-09-21T04:34:57.579 INFO:tasks.workunit.client.1.smithi053.stdout:4/8: symlink d1/l2 0 2024-09-21T04:34:57.580 INFO:tasks.workunit.client.1.smithi053.stdout:6/4: mkdir d2 0 2024-09-21T04:34:57.582 INFO:tasks.workunit.client.1.smithi053.stdout:2/10: creat d1/f4 x:0 0 0 2024-09-21T04:34:57.584 INFO:tasks.workunit.client.1.smithi053.stdout:9/0: mkdir d0 0 2024-09-21T04:34:57.586 INFO:tasks.workunit.client.1.smithi053.stdout:5/27: creat d0/f3 x:0 0 0 2024-09-21T04:34:57.587 INFO:tasks.workunit.client.1.smithi053.stdout:8/3: symlink l1 0 2024-09-21T04:34:57.587 INFO:tasks.workunit.client.1.smithi053.stdout:8/4: truncate - no filename 2024-09-21T04:34:57.588 INFO:tasks.workunit.client.1.smithi053.stdout:5/28: chown d0/f3 57500 1 2024-09-21T04:34:57.589 INFO:tasks.workunit.client.1.smithi053.stdout:4/9: creat d1/f3 x:0 0 0 2024-09-21T04:34:57.589 INFO:tasks.workunit.client.1.smithi053.stdout:6/5: dwrite f1 [0,4194304] 0 2024-09-21T04:34:57.590 INFO:tasks.workunit.client.1.smithi053.stdout:6/6: readlink l0 0 2024-09-21T04:34:57.590 INFO:tasks.workunit.client.1.smithi053.stdout:4/10: chown d1/l2 27603945 1 2024-09-21T04:34:57.591 INFO:tasks.workunit.client.1.smithi053.stdout:2/11: creat d1/f5 x:0 0 0 2024-09-21T04:34:57.594 INFO:tasks.workunit.client.1.smithi053.stdout:8/5: rename l0 to l2 0 2024-09-21T04:34:57.594 INFO:tasks.workunit.client.1.smithi053.stdout:8/6: write - no filename 2024-09-21T04:34:57.594 INFO:tasks.workunit.client.1.smithi053.stdout:8/7: truncate - no filename 2024-09-21T04:34:57.594 INFO:tasks.workunit.client.1.smithi053.stdout:8/8: dwrite - no filename 2024-09-21T04:34:57.594 INFO:tasks.workunit.client.1.smithi053.stdout:8/9: dread - no filename 2024-09-21T04:34:57.596 INFO:tasks.workunit.client.1.smithi053.stdout:5/29: mknod d0/c4 0 2024-09-21T04:34:57.598 INFO:tasks.workunit.client.1.smithi053.stdout:4/11: creat d1/f4 x:0 0 0 2024-09-21T04:34:57.599 INFO:tasks.workunit.client.1.smithi053.stdout:4/12: write d1/f4 [688734,109451] 0 2024-09-21T04:34:57.600 INFO:tasks.workunit.client.1.smithi053.stdout:4/13: dread - d1/f3 zero size 2024-09-21T04:34:57.601 INFO:tasks.workunit.client.1.smithi053.stdout:9/1: creat d0/f1 x:0 0 0 2024-09-21T04:34:57.602 INFO:tasks.workunit.client.1.smithi053.stdout:2/12: creat d1/f6 x:0 0 0 2024-09-21T04:34:57.605 INFO:tasks.workunit.client.1.smithi053.stdout:8/10: mkdir d3 0 2024-09-21T04:34:57.607 INFO:tasks.workunit.client.1.smithi053.stdout:5/30: creat d0/f5 x:0 0 0 2024-09-21T04:34:57.607 INFO:tasks.workunit.client.1.smithi053.stdout:6/7: creat d2/f3 x:0 0 0 2024-09-21T04:34:57.609 INFO:tasks.workunit.client.1.smithi053.stdout:4/14: symlink d1/l5 0 2024-09-21T04:34:57.611 INFO:tasks.workunit.client.1.smithi053.stdout:9/2: mkdir d0/d2 0 2024-09-21T04:34:57.611 INFO:tasks.workunit.client.1.smithi053.stdout:2/13: dwrite d1/f5 [0,4194304] 0 2024-09-21T04:34:57.612 INFO:tasks.workunit.client.1.smithi053.stdout:8/11: unlink l1 0 2024-09-21T04:34:57.613 INFO:tasks.workunit.client.1.smithi053.stdout:8/12: stat d3 0 2024-09-21T04:34:57.613 INFO:tasks.workunit.client.1.smithi053.stdout:8/13: truncate - no filename 2024-09-21T04:34:57.613 INFO:tasks.workunit.client.1.smithi053.stdout:8/14: fsync - no filename 2024-09-21T04:34:57.613 INFO:tasks.workunit.client.1.smithi053.stdout:5/31: rmdir d0 39 2024-09-21T04:34:57.613 INFO:tasks.workunit.client.1.smithi053.stdout:4/15: dread d1/f4 [0,4194304] 0 2024-09-21T04:34:57.617 INFO:tasks.workunit.client.1.smithi053.stdout:9/3: symlink d0/l3 0 2024-09-21T04:34:57.623 INFO:tasks.workunit.client.1.smithi053.stdout:4/16: dwrite d1/f4 [0,4194304] 0 2024-09-21T04:34:57.624 INFO:tasks.workunit.client.1.smithi053.stdout:9/4: creat d0/f4 x:0 0 0 2024-09-21T04:34:57.624 INFO:tasks.workunit.client.1.smithi053.stdout:8/15: mkdir d3/d4 0 2024-09-21T04:34:57.629 INFO:tasks.workunit.client.1.smithi053.stdout:8/16: rename d3/d4 to d3/d5 0 2024-09-21T04:34:57.629 INFO:tasks.workunit.client.1.smithi053.stdout:8/17: fsync - no filename 2024-09-21T04:34:57.629 INFO:tasks.workunit.client.1.smithi053.stdout:8/18: write - no filename 2024-09-21T04:34:57.799 INFO:tasks.workunit.client.1.smithi053.stdout:1/8: fsync f1 0 2024-09-21T04:34:57.808 INFO:tasks.workunit.client.1.smithi053.stdout:5/32: truncate d0/f5 892993 0 2024-09-21T04:34:57.809 INFO:tasks.workunit.client.1.smithi053.stdout:5/33: write d0/f5 [412766,90307] 0 2024-09-21T04:34:57.840 INFO:tasks.workunit.client.1.smithi053.stdout:9/5: fsync d0/f1 0 2024-09-21T04:34:57.841 INFO:tasks.workunit.client.1.smithi053.stdout:2/14: getdents d1 0 2024-09-21T04:34:57.842 INFO:tasks.workunit.client.1.smithi053.stdout:2/15: read - d1/f6 zero size 2024-09-21T04:34:57.842 INFO:tasks.workunit.client.1.smithi053.stdout:2/16: stat d1/f4 0 2024-09-21T04:34:57.845 INFO:tasks.workunit.client.1.smithi053.stdout:6/8: fsync d2/f3 0 2024-09-21T04:34:57.846 INFO:tasks.workunit.client.1.smithi053.stdout:6/9: write f1 [2416306,110936] 0 2024-09-21T04:34:57.846 INFO:tasks.workunit.client.1.smithi053.stdout:2/17: dread f0 [0,4194304] 0 2024-09-21T04:34:57.847 INFO:tasks.workunit.client.1.smithi053.stdout:4/17: getdents d1 0 2024-09-21T04:34:57.848 INFO:tasks.workunit.client.1.smithi053.stdout:2/18: write d1/f5 [3321949,118686] 0 2024-09-21T04:34:57.907 INFO:tasks.workunit.client.1.smithi053.stdout:1/9: mknod c2 0 2024-09-21T04:34:57.908 INFO:tasks.workunit.client.1.smithi053.stdout:5/34: mknod d0/c6 0 2024-09-21T04:34:57.909 INFO:tasks.workunit.client.1.smithi053.stdout:5/35: write d0/f5 [245507,39368] 0 2024-09-21T04:34:57.910 INFO:tasks.workunit.client.1.smithi053.stdout:8/19: unlink l2 0 2024-09-21T04:34:57.910 INFO:tasks.workunit.client.1.smithi053.stdout:8/20: dwrite - no filename 2024-09-21T04:34:57.911 INFO:tasks.workunit.client.1.smithi053.stdout:9/6: creat d0/f5 x:0 0 0 2024-09-21T04:34:57.914 INFO:tasks.workunit.client.1.smithi053.stdout:6/10: mkdir d2/d4 0 2024-09-21T04:34:57.916 INFO:tasks.workunit.client.1.smithi053.stdout:2/19: mknod d1/c7 0 2024-09-21T04:34:57.917 INFO:tasks.workunit.client.1.smithi053.stdout:1/10: dread - f0 zero size 2024-09-21T04:34:57.918 INFO:tasks.workunit.client.1.smithi053.stdout:2/20: chown d1/f5 22748 1 2024-09-21T04:34:57.918 INFO:tasks.workunit.client.1.smithi053.stdout:5/36: rename d0/f5 to d0/f7 0 2024-09-21T04:34:57.919 INFO:tasks.workunit.client.1.smithi053.stdout:8/21: mknod d3/c6 0 2024-09-21T04:34:57.921 INFO:tasks.workunit.client.1.smithi053.stdout:9/7: dwrite d0/f5 [0,4194304] 0 2024-09-21T04:34:57.932 INFO:tasks.workunit.client.1.smithi053.stdout:2/21: mknod d1/c8 0 2024-09-21T04:34:57.934 INFO:tasks.workunit.client.1.smithi053.stdout:8/22: creat d3/f7 x:0 0 0 2024-09-21T04:34:57.935 INFO:tasks.workunit.client.1.smithi053.stdout:9/8: mknod d0/c6 0 2024-09-21T04:34:57.936 INFO:tasks.workunit.client.1.smithi053.stdout:9/9: stat d0/l3 0 2024-09-21T04:34:57.938 INFO:tasks.workunit.client.1.smithi053.stdout:9/10: chown d0/f4 12 1 2024-09-21T04:34:57.939 INFO:tasks.workunit.client.1.smithi053.stdout:1/11: dwrite f0 [0,4194304] 0 2024-09-21T04:34:57.945 INFO:tasks.workunit.client.1.smithi053.stdout:9/11: mknod d0/d2/c7 0 2024-09-21T04:34:57.946 INFO:tasks.workunit.client.1.smithi053.stdout:8/23: dwrite d3/f7 [0,4194304] 0 2024-09-21T04:34:57.946 INFO:tasks.workunit.client.1.smithi053.stdout:9/12: chown d0/f1 775125 1 2024-09-21T04:34:57.949 INFO:tasks.workunit.client.1.smithi053.stdout:8/24: symlink d3/d5/l8 0 2024-09-21T04:34:57.953 INFO:tasks.workunit.client.1.smithi053.stdout:1/12: fsync f1 0 2024-09-21T04:34:57.958 INFO:tasks.workunit.client.1.smithi053.stdout:8/25: link d3/f7 d3/f9 0 2024-09-21T04:34:57.959 INFO:tasks.workunit.client.1.smithi053.stdout:1/13: dwrite f0 [0,4194304] 0 2024-09-21T04:34:57.960 INFO:tasks.workunit.client.1.smithi053.stdout:1/14: chown f1 0 1 2024-09-21T04:34:57.963 INFO:tasks.workunit.client.1.smithi053.stdout:1/15: creat f3 x:0 0 0 2024-09-21T04:34:57.964 INFO:tasks.workunit.client.1.smithi053.stdout:8/26: dread d3/f7 [0,4194304] 0 2024-09-21T04:34:57.965 INFO:tasks.workunit.client.1.smithi053.stdout:1/16: creat f4 x:0 0 0 2024-09-21T04:34:57.972 INFO:tasks.workunit.client.1.smithi053.stdout:8/27: dwrite d3/f7 [0,4194304] 0 2024-09-21T04:34:57.974 INFO:tasks.workunit.client.1.smithi053.stdout:1/17: fdatasync f1 0 2024-09-21T04:34:57.976 INFO:tasks.workunit.client.1.smithi053.stdout:8/28: creat d3/fa x:0 0 0 2024-09-21T04:34:57.980 INFO:tasks.workunit.client.1.smithi053.stdout:8/29: mknod d3/cb 0 2024-09-21T04:34:58.066 INFO:tasks.workunit.client.1.smithi053.stdout:2/22: truncate f0 1550937 0 2024-09-21T04:34:58.068 INFO:tasks.workunit.client.1.smithi053.stdout:4/18: truncate d1/f4 514505 0 2024-09-21T04:34:58.070 INFO:tasks.workunit.client.1.smithi053.stdout:2/23: creat d1/f9 x:0 0 0 2024-09-21T04:34:58.070 INFO:tasks.workunit.client.1.smithi053.stdout:2/24: truncate d1/f9 895404 0 2024-09-21T04:34:58.075 INFO:tasks.workunit.client.1.smithi053.stdout:2/25: rename d1/c7 to d1/ca 0 2024-09-21T04:34:58.079 INFO:tasks.workunit.client.1.smithi053.stdout:2/26: rename d1/f6 to d1/fb 0 2024-09-21T04:34:58.079 INFO:tasks.workunit.client.1.smithi053.stdout:6/11: truncate f1 4361 0 2024-09-21T04:34:58.080 INFO:tasks.workunit.client.1.smithi053.stdout:2/27: read d1/f5 [609361,128547] 0 2024-09-21T04:34:58.086 INFO:tasks.workunit.client.1.smithi053.stdout:9/13: truncate d0/f5 1632467 0 2024-09-21T04:34:58.087 INFO:tasks.workunit.client.1.smithi053.stdout:2/28: symlink d1/lc 0 2024-09-21T04:34:58.092 INFO:tasks.workunit.client.1.smithi053.stdout:1/18: fsync f4 0 2024-09-21T04:34:58.094 INFO:tasks.workunit.client.1.smithi053.stdout:9/14: mknod d0/d2/c8 0 2024-09-21T04:34:58.097 INFO:tasks.workunit.client.1.smithi053.stdout:2/29: mknod d1/cd 0 2024-09-21T04:34:58.097 INFO:tasks.workunit.client.1.smithi053.stdout:2/30: write d1/f4 [210600,100463] 0 2024-09-21T04:34:58.101 INFO:tasks.workunit.client.1.smithi053.stdout:8/30: truncate d3/f9 109200 0 2024-09-21T04:34:58.101 INFO:tasks.workunit.client.1.smithi053.stdout:8/31: stat d3/d5 0 2024-09-21T04:34:58.101 INFO:tasks.workunit.client.1.smithi053.stdout:1/19: creat f5 x:0 0 0 2024-09-21T04:34:58.101 INFO:tasks.workunit.client.1.smithi053.stdout:1/20: chown f0 0 1 2024-09-21T04:34:58.102 INFO:tasks.workunit.client.1.smithi053.stdout:1/21: write f4 [906609,129564] 0 2024-09-21T04:34:58.103 INFO:tasks.workunit.client.1.smithi053.stdout:1/22: write f5 [223461,81516] 0 2024-09-21T04:34:58.108 INFO:tasks.workunit.client.1.smithi053.stdout:1/23: symlink l6 0 2024-09-21T04:34:58.111 INFO:tasks.workunit.client.1.smithi053.stdout:1/24: mkdir d7 0 2024-09-21T04:34:58.111 INFO:tasks.workunit.client.1.smithi053.stdout:1/25: chown f5 0 1 2024-09-21T04:34:58.114 INFO:tasks.workunit.client.1.smithi053.stdout:1/26: unlink l6 0 2024-09-21T04:34:58.119 INFO:tasks.workunit.client.1.smithi053.stdout:1/27: getdents d7 0 2024-09-21T04:34:58.119 INFO:tasks.workunit.client.1.smithi053.stdout:1/28: fdatasync f1 0 2024-09-21T04:34:58.124 INFO:tasks.workunit.client.1.smithi053.stdout:1/29: mknod d7/c8 0 2024-09-21T04:34:58.127 INFO:tasks.workunit.client.1.smithi053.stdout:1/30: mkdir d7/d9 0 2024-09-21T04:34:58.132 INFO:tasks.workunit.client.1.smithi053.stdout:6/12: fsync f1 0 2024-09-21T04:34:58.132 INFO:tasks.workunit.client.1.smithi053.stdout:6/13: dread - d2/f3 zero size 2024-09-21T04:34:58.133 INFO:tasks.workunit.client.1.smithi053.stdout:1/31: mkdir d7/d9/da 0 2024-09-21T04:34:58.134 INFO:tasks.workunit.client.1.smithi053.stdout:1/32: write f1 [966103,121964] 0 2024-09-21T04:34:58.134 INFO:tasks.workunit.client.1.smithi053.stdout:6/14: dread f1 [0,4194304] 0 2024-09-21T04:34:58.134 INFO:tasks.workunit.client.1.smithi053.stdout:1/33: read - f3 zero size 2024-09-21T04:34:58.135 INFO:tasks.workunit.client.1.smithi053.stdout:1/34: fsync f3 0 2024-09-21T04:34:58.139 INFO:tasks.workunit.client.1.smithi053.stdout:9/15: dread d0/f5 [0,4194304] 0 2024-09-21T04:34:58.146 INFO:tasks.workunit.client.1.smithi053.stdout:9/16: mkdir d0/d2/d9 0 2024-09-21T04:34:58.151 INFO:tasks.workunit.client.1.smithi053.stdout:9/17: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:34:58.151 INFO:tasks.workunit.client.1.smithi053.stdout:9/18: chown d0/f1 26700 1 2024-09-21T04:34:58.244 INFO:tasks.workunit.client.1.smithi053.stdout:9/19: creat d0/d2/fa x:0 0 0 2024-09-21T04:34:58.245 INFO:tasks.workunit.client.1.smithi053.stdout:9/20: stat d0/d2/c7 0 2024-09-21T04:34:58.245 INFO:tasks.workunit.client.1.smithi053.stdout:9/21: readlink d0/l3 0 2024-09-21T04:34:58.249 INFO:tasks.workunit.client.1.smithi053.stdout:9/22: link d0/f5 d0/d2/d9/fb 0 2024-09-21T04:34:58.250 INFO:tasks.workunit.client.1.smithi053.stdout:9/23: chown d0/d2 1201802 1 2024-09-21T04:34:58.251 INFO:tasks.workunit.client.1.smithi053.stdout:9/24: write d0/f1 [568463,55073] 0 2024-09-21T04:34:58.261 INFO:tasks.workunit.client.1.smithi053.stdout:4/19: truncate d1/f4 1012332 0 2024-09-21T04:34:58.261 INFO:tasks.workunit.client.1.smithi053.stdout:4/20: write d1/f3 [247090,111545] 0 2024-09-21T04:34:58.265 INFO:tasks.workunit.client.1.smithi053.stdout:4/21: dread d1/f3 [0,4194304] 0 2024-09-21T04:34:58.267 INFO:tasks.workunit.client.1.smithi053.stdout:4/22: rmdir d1 39 2024-09-21T04:34:58.269 INFO:tasks.workunit.client.1.smithi053.stdout:1/35: getdents d7/d9 0 2024-09-21T04:34:58.273 INFO:tasks.workunit.client.1.smithi053.stdout:6/15: dwrite f1 [0,4194304] 0 2024-09-21T04:34:58.275 INFO:tasks.workunit.client.1.smithi053.stdout:4/23: fdatasync d1/f4 0 2024-09-21T04:34:58.277 INFO:tasks.workunit.client.1.smithi053.stdout:1/36: symlink d7/d9/da/lb 0 2024-09-21T04:34:58.281 INFO:tasks.workunit.client.1.smithi053.stdout:1/37: dread f1 [0,4194304] 0 2024-09-21T04:34:58.281 INFO:tasks.workunit.client.1.smithi053.stdout:1/38: chown c2 2952259 1 2024-09-21T04:34:58.283 INFO:tasks.workunit.client.1.smithi053.stdout:4/24: symlink d1/l6 0 2024-09-21T04:34:58.285 INFO:tasks.workunit.client.1.smithi053.stdout:9/25: getdents d0/d2 0 2024-09-21T04:34:58.286 INFO:tasks.workunit.client.1.smithi053.stdout:9/26: write d0/d2/fa [378894,394] 0 2024-09-21T04:34:58.286 INFO:tasks.workunit.client.1.smithi053.stdout:9/27: chown d0 10 1 2024-09-21T04:34:58.287 INFO:tasks.workunit.client.1.smithi053.stdout:1/39: creat d7/fc x:0 0 0 2024-09-21T04:34:58.287 INFO:tasks.workunit.client.1.smithi053.stdout:1/40: stat f1 0 2024-09-21T04:34:58.289 INFO:tasks.workunit.client.1.smithi053.stdout:4/25: symlink d1/l7 0 2024-09-21T04:34:58.291 INFO:tasks.workunit.client.1.smithi053.stdout:9/28: dread d0/d2/fa [0,4194304] 0 2024-09-21T04:34:58.292 INFO:tasks.workunit.client.1.smithi053.stdout:9/29: dread d0/f5 [0,4194304] 0 2024-09-21T04:34:58.293 INFO:tasks.workunit.client.1.smithi053.stdout:1/41: rename d7/fc to d7/fd 0 2024-09-21T04:34:58.302 INFO:tasks.workunit.client.1.smithi053.stdout:1/42: creat d7/fe x:0 0 0 2024-09-21T04:34:58.305 INFO:tasks.workunit.client.1.smithi053.stdout:9/30: creat d0/d2/d9/fc x:0 0 0 2024-09-21T04:34:58.306 INFO:tasks.workunit.client.1.smithi053.stdout:1/43: rmdir d7/d9/da 39 2024-09-21T04:34:58.307 INFO:tasks.workunit.client.1.smithi053.stdout:4/26: link d1/l7 d1/l8 0 2024-09-21T04:34:58.311 INFO:tasks.workunit.client.1.smithi053.stdout:1/44: mknod d7/cf 0 2024-09-21T04:34:58.319 INFO:tasks.workunit.client.1.smithi053.stdout:1/45: symlink d7/d9/da/l10 0 2024-09-21T04:34:58.319 INFO:tasks.workunit.client.1.smithi053.stdout:6/16: dwrite f1 [4194304,4194304] 0 2024-09-21T04:34:58.320 INFO:tasks.workunit.client.1.smithi053.stdout:9/31: dwrite d0/d2/fa [0,4194304] 0 2024-09-21T04:34:58.327 INFO:tasks.workunit.client.1.smithi053.stdout:4/27: dwrite d1/f3 [0,4194304] 0 2024-09-21T04:34:58.327 INFO:tasks.workunit.client.1.smithi053.stdout:9/32: write d0/d2/d9/fc [544519,78483] 0 2024-09-21T04:34:58.330 INFO:tasks.workunit.client.1.smithi053.stdout:4/28: chown d1/l5 412 1 2024-09-21T04:34:58.332 INFO:tasks.workunit.client.1.smithi053.stdout:4/29: write d1/f3 [974023,83323] 0 2024-09-21T04:34:58.334 INFO:tasks.workunit.client.1.smithi053.stdout:1/46: dwrite f4 [0,4194304] 0 2024-09-21T04:34:58.334 INFO:tasks.workunit.client.1.smithi053.stdout:6/17: rmdir d2/d4 0 2024-09-21T04:34:58.343 INFO:tasks.workunit.client.1.smithi053.stdout:4/30: symlink d1/l9 0 2024-09-21T04:34:58.347 INFO:tasks.workunit.client.1.smithi053.stdout:6/18: creat d2/f5 x:0 0 0 2024-09-21T04:34:58.348 INFO:tasks.workunit.client.1.smithi053.stdout:1/47: dread f0 [0,4194304] 0 2024-09-21T04:34:58.349 INFO:tasks.workunit.client.1.smithi053.stdout:1/48: truncate f5 1343484 0 2024-09-21T04:34:58.350 INFO:tasks.workunit.client.1.smithi053.stdout:6/19: symlink d2/l6 0 2024-09-21T04:34:58.355 INFO:tasks.workunit.client.1.smithi053.stdout:6/20: dread f1 [0,4194304] 0 2024-09-21T04:34:58.358 INFO:tasks.workunit.client.1.smithi053.stdout:1/49: dwrite d7/fd [0,4194304] 0 2024-09-21T04:34:58.359 INFO:tasks.workunit.client.1.smithi053.stdout:1/50: chown d7/d9/da 1 1 2024-09-21T04:34:58.364 INFO:tasks.workunit.client.1.smithi053.stdout:9/33: write d0/d2/d9/fb [2289098,126788] 0 2024-09-21T04:34:58.365 INFO:tasks.workunit.client.1.smithi053.stdout:1/51: mkdir d7/d9/da/d11 0 2024-09-21T04:34:58.365 INFO:tasks.workunit.client.1.smithi053.stdout:6/21: dwrite d2/f5 [0,4194304] 0 2024-09-21T04:34:58.367 INFO:tasks.workunit.client.1.smithi053.stdout:9/34: chown d0/f5 1901853 1 2024-09-21T04:34:58.372 INFO:tasks.workunit.client.1.smithi053.stdout:9/35: fsync d0/f5 0 2024-09-21T04:34:58.374 INFO:tasks.workunit.client.1.smithi053.stdout:1/52: dwrite f4 [0,4194304] 0 2024-09-21T04:34:58.378 INFO:tasks.workunit.client.1.smithi053.stdout:9/36: dread d0/f1 [0,4194304] 0 2024-09-21T04:34:58.380 INFO:tasks.workunit.client.1.smithi053.stdout:9/37: chown d0/d2/fa 1511354 1 2024-09-21T04:34:58.390 INFO:tasks.workunit.client.1.smithi053.stdout:6/22: unlink l0 0 2024-09-21T04:34:58.395 INFO:tasks.workunit.client.1.smithi053.stdout:1/53: creat d7/d9/da/f12 x:0 0 0 2024-09-21T04:34:58.397 INFO:tasks.workunit.client.1.smithi053.stdout:9/38: unlink d0/c6 0 2024-09-21T04:34:58.401 INFO:tasks.workunit.client.1.smithi053.stdout:1/54: rmdir d7 39 2024-09-21T04:34:58.401 INFO:tasks.workunit.client.1.smithi053.stdout:1/55: write f3 [1008660,62731] 0 2024-09-21T04:34:58.402 INFO:tasks.workunit.client.1.smithi053.stdout:9/39: creat d0/d2/fd x:0 0 0 2024-09-21T04:34:58.407 INFO:tasks.workunit.client.1.smithi053.stdout:9/40: creat d0/d2/fe x:0 0 0 2024-09-21T04:34:58.408 INFO:tasks.workunit.client.1.smithi053.stdout:1/56: rmdir d7/d9 39 2024-09-21T04:34:58.410 INFO:tasks.workunit.client.1.smithi053.stdout:9/41: unlink d0/f5 0 2024-09-21T04:34:58.414 INFO:tasks.workunit.client.1.smithi053.stdout:9/42: mkdir d0/df 0 2024-09-21T04:34:58.416 INFO:tasks.workunit.client.1.smithi053.stdout:1/57: creat d7/d9/da/f13 x:0 0 0 2024-09-21T04:34:58.416 INFO:tasks.workunit.client.1.smithi053.stdout:1/58: readlink d7/d9/da/l10 0 2024-09-21T04:34:58.417 INFO:tasks.workunit.client.1.smithi053.stdout:9/43: symlink d0/d2/l10 0 2024-09-21T04:34:58.423 INFO:tasks.workunit.client.1.smithi053.stdout:9/44: symlink d0/df/l11 0 2024-09-21T04:34:58.426 INFO:tasks.workunit.client.1.smithi053.stdout:6/23: truncate f1 1102891 0 2024-09-21T04:34:58.428 INFO:tasks.workunit.client.1.smithi053.stdout:6/24: rename d2/f5 to d2/f7 0 2024-09-21T04:34:58.432 INFO:tasks.workunit.client.1.smithi053.stdout:6/25: write d2/f7 [5157021,30621] 0 2024-09-21T04:34:58.440 INFO:tasks.workunit.client.1.smithi053.stdout:1/59: write f0 [2061148,82749] 0 2024-09-21T04:34:58.442 INFO:tasks.workunit.client.1.smithi053.stdout:1/60: truncate d7/d9/da/f13 19507 0 2024-09-21T04:34:58.446 INFO:tasks.workunit.client.1.smithi053.stdout:1/61: dread f5 [0,4194304] 0 2024-09-21T04:34:58.447 INFO:tasks.workunit.client.1.smithi053.stdout:6/26: dread f1 [0,4194304] 0 2024-09-21T04:34:58.452 INFO:tasks.workunit.client.1.smithi053.stdout:6/27: dread d2/f7 [4194304,4194304] 0 2024-09-21T04:34:58.452 INFO:tasks.workunit.client.1.smithi053.stdout:6/28: dread - d2/f3 zero size 2024-09-21T04:34:58.462 INFO:tasks.workunit.client.1.smithi053.stdout:6/29: dread d2/f7 [0,4194304] 0 2024-09-21T04:34:58.462 INFO:tasks.workunit.client.1.smithi053.stdout:6/30: read f1 [644509,28945] 0 2024-09-21T04:34:58.465 INFO:tasks.workunit.client.1.smithi053.stdout:6/31: creat d2/f8 x:0 0 0 2024-09-21T04:34:58.465 INFO:tasks.workunit.client.1.smithi053.stdout:6/32: write d2/f3 [769618,124632] 0 2024-09-21T04:34:58.469 INFO:tasks.workunit.client.1.smithi053.stdout:1/62: truncate d7/fd 2656917 0 2024-09-21T04:34:58.469 INFO:tasks.workunit.client.1.smithi053.stdout:1/63: dread - d7/d9/da/f12 zero size 2024-09-21T04:34:58.506 INFO:tasks.workunit.client.1.smithi053.stdout:1/64: write d7/fd [901137,86128] 0 2024-09-21T04:34:58.507 INFO:tasks.workunit.client.1.smithi053.stdout:1/65: write d7/d9/da/f13 [235168,15694] 0 2024-09-21T04:34:58.508 INFO:tasks.workunit.client.1.smithi053.stdout:1/66: read - d7/d9/da/f12 zero size 2024-09-21T04:34:58.508 INFO:tasks.workunit.client.1.smithi053.stdout:1/67: stat f3 0 2024-09-21T04:34:58.509 INFO:tasks.workunit.client.1.smithi053.stdout:1/68: write f3 [20546,53140] 0 2024-09-21T04:34:58.513 INFO:tasks.workunit.client.1.smithi053.stdout:1/69: rename d7/d9/da/f12 to d7/d9/da/d11/f14 0 2024-09-21T04:34:58.513 INFO:tasks.workunit.client.1.smithi053.stdout:1/70: fsync f4 0 2024-09-21T04:34:58.515 INFO:tasks.workunit.client.1.smithi053.stdout:1/71: mknod d7/d9/da/d11/c15 0 2024-09-21T04:34:58.516 INFO:tasks.workunit.client.1.smithi053.stdout:1/72: symlink d7/d9/da/d11/l16 0 2024-09-21T04:34:58.517 INFO:tasks.workunit.client.1.smithi053.stdout:1/73: write f0 [1032848,36333] 0 2024-09-21T04:34:58.532 INFO:tasks.workunit.client.1.smithi053.stdout:1/74: dwrite f5 [0,4194304] 0 2024-09-21T04:34:58.533 INFO:tasks.workunit.client.1.smithi053.stdout:1/75: write d7/fd [3625639,57403] 0 2024-09-21T04:34:58.536 INFO:tasks.workunit.client.1.smithi053.stdout:1/76: mknod d7/d9/da/c17 0 2024-09-21T04:34:58.540 INFO:tasks.workunit.client.1.smithi053.stdout:1/77: dread d7/fd [0,4194304] 0 2024-09-21T04:34:58.543 INFO:tasks.workunit.client.1.smithi053.stdout:1/78: unlink d7/cf 0 2024-09-21T04:34:58.543 INFO:tasks.workunit.client.1.smithi053.stdout:1/79: chown f1 1 1 2024-09-21T04:34:58.546 INFO:tasks.workunit.client.1.smithi053.stdout:1/80: mknod d7/d9/da/d11/c18 0 2024-09-21T04:34:58.551 INFO:tasks.workunit.client.1.smithi053.stdout:1/81: rename d7/fe to d7/d9/f19 0 2024-09-21T04:34:58.551 INFO:tasks.workunit.client.1.smithi053.stdout:1/82: chown c2 0 1 2024-09-21T04:34:58.558 INFO:tasks.workunit.client.1.smithi053.stdout:1/83: dwrite d7/fd [0,4194304] 0 2024-09-21T04:34:58.563 INFO:tasks.workunit.client.1.smithi053.stdout:1/84: link d7/d9/da/d11/c18 d7/d9/da/c1a 0 2024-09-21T04:34:58.567 INFO:tasks.workunit.client.1.smithi053.stdout:1/85: dread f5 [0,4194304] 0 2024-09-21T04:34:58.571 INFO:tasks.workunit.client.1.smithi053.stdout:1/86: rename d7/fd to d7/d9/f1b 0 2024-09-21T04:34:58.574 INFO:tasks.workunit.client.1.smithi053.stdout:1/87: mknod d7/c1c 0 2024-09-21T04:34:58.575 INFO:tasks.workunit.client.1.smithi053.stdout:1/88: read d7/d9/f1b [3573872,117883] 0 2024-09-21T04:34:58.576 INFO:tasks.workunit.client.1.smithi053.stdout:1/89: write f5 [1795932,52352] 0 2024-09-21T04:34:58.581 INFO:tasks.workunit.client.1.smithi053.stdout:1/90: unlink d7/d9/da/f13 0 2024-09-21T04:34:58.592 INFO:tasks.workunit.client.1.smithi053.stdout:1/91: truncate f3 316656 0 2024-09-21T04:34:58.595 INFO:tasks.workunit.client.1.smithi053.stdout:1/92: mkdir d7/d9/da/d1d 0 2024-09-21T04:34:58.597 INFO:tasks.workunit.client.1.smithi053.stdout:1/93: symlink d7/d9/da/d11/l1e 0 2024-09-21T04:34:58.599 INFO:tasks.workunit.client.1.smithi053.stdout:1/94: write d7/d9/f1b [1457221,9095] 0 2024-09-21T04:34:58.607 INFO:tasks.workunit.client.1.smithi053.stdout:1/95: dread f3 [0,4194304] 0 2024-09-21T04:34:58.608 INFO:tasks.workunit.client.1.smithi053.stdout:1/96: chown d7/d9/da/c17 24697 1 2024-09-21T04:34:58.612 INFO:tasks.workunit.client.1.smithi053.stdout:1/97: dread f0 [0,4194304] 0 2024-09-21T04:34:58.614 INFO:tasks.workunit.client.1.smithi053.stdout:1/98: mknod d7/c1f 0 2024-09-21T04:34:58.614 INFO:tasks.workunit.client.1.smithi053.stdout:1/99: read f3 [160908,112620] 0 2024-09-21T04:34:58.619 INFO:tasks.workunit.client.1.smithi053.stdout:1/100: truncate f3 694794 0 2024-09-21T04:34:58.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:34:58 smithi053 ceph-mon[35554]: pgmap v20: 65 pgs: 65 active+clean; 191 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 317 op/s 2024-09-21T04:34:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:34:58 smithi007 ceph-mon[27954]: pgmap v20: 65 pgs: 65 active+clean; 191 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 317 op/s 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:5/37: sync 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:4/31: sync 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:3/27: sync 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:2/31: sync 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:6/33: sync 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:9/45: sync 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:8/32: sync 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:7/7: sync 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:7/8: dwrite - no filename 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:7/9: dwrite - no filename 2024-09-21T04:34:59.560 INFO:tasks.workunit.client.1.smithi053.stdout:7/10: dwrite - no filename 2024-09-21T04:34:59.561 INFO:tasks.workunit.client.1.smithi053.stdout:0/5: sync 2024-09-21T04:34:59.561 INFO:tasks.workunit.client.1.smithi053.stdout:0/6: fdatasync - no filename 2024-09-21T04:34:59.561 INFO:tasks.workunit.client.1.smithi053.stdout:1/101: sync 2024-09-21T04:34:59.561 INFO:tasks.workunit.client.1.smithi053.stdout:5/38: stat d0/f7 0 2024-09-21T04:34:59.561 INFO:tasks.workunit.client.1.smithi053.stdout:3/28: dread - f3 zero size 2024-09-21T04:34:59.562 INFO:tasks.workunit.client.1.smithi053.stdout:8/33: read d3/f7 [33166,106818] 0 2024-09-21T04:34:59.562 INFO:tasks.workunit.client.1.smithi053.stdout:3/29: chown c0 2087855478 1 2024-09-21T04:34:59.562 INFO:tasks.workunit.client.1.smithi053.stdout:3/30: write f3 [959396,1166] 0 2024-09-21T04:34:59.565 INFO:tasks.workunit.client.1.smithi053.stdout:5/39: mkdir d0/d8 0 2024-09-21T04:34:59.568 INFO:tasks.workunit.client.1.smithi053.stdout:0/7: mknod c0 0 2024-09-21T04:34:59.568 INFO:tasks.workunit.client.1.smithi053.stdout:0/8: truncate - no filename 2024-09-21T04:34:59.568 INFO:tasks.workunit.client.1.smithi053.stdout:0/9: readlink - no filename 2024-09-21T04:34:59.568 INFO:tasks.workunit.client.1.smithi053.stdout:0/10: stat c0 0 2024-09-21T04:34:59.568 INFO:tasks.workunit.client.1.smithi053.stdout:0/11: write - no filename 2024-09-21T04:34:59.569 INFO:tasks.workunit.client.1.smithi053.stdout:2/32: dwrite d1/f9 [0,4194304] 0 2024-09-21T04:34:59.571 INFO:tasks.workunit.client.1.smithi053.stdout:3/31: symlink l4 0 2024-09-21T04:34:59.576 INFO:tasks.workunit.client.1.smithi053.stdout:4/32: creat d1/fa x:0 0 0 2024-09-21T04:34:59.576 INFO:tasks.workunit.client.1.smithi053.stdout:5/40: dwrite d0/f3 [0,4194304] 0 2024-09-21T04:34:59.581 INFO:tasks.workunit.client.1.smithi053.stdout:0/12: creat f1 x:0 0 0 2024-09-21T04:34:59.582 INFO:tasks.workunit.client.1.smithi053.stdout:1/102: creat d7/d9/da/d11/f20 x:0 0 0 2024-09-21T04:34:59.591 INFO:tasks.workunit.client.1.smithi053.stdout:4/33: mknod d1/cb 0 2024-09-21T04:34:59.592 INFO:tasks.workunit.client.1.smithi053.stdout:5/41: fsync d0/f7 0 2024-09-21T04:34:59.597 INFO:tasks.workunit.client.1.smithi053.stdout:2/33: truncate f0 453844 0 2024-09-21T04:34:59.600 INFO:tasks.workunit.client.1.smithi053.stdout:4/34: readlink d1/l8 0 2024-09-21T04:34:59.604 INFO:tasks.workunit.client.1.smithi053.stdout:8/34: getdents d3 0 2024-09-21T04:34:59.604 INFO:tasks.workunit.client.1.smithi053.stdout:1/103: symlink d7/d9/l21 0 2024-09-21T04:34:59.605 INFO:tasks.workunit.client.1.smithi053.stdout:1/104: read f0 [2061444,76112] 0 2024-09-21T04:34:59.606 INFO:tasks.workunit.client.1.smithi053.stdout:2/34: unlink d1/lc 0 2024-09-21T04:34:59.606 INFO:tasks.workunit.client.1.smithi053.stdout:4/35: dwrite d1/f3 [0,4194304] 0 2024-09-21T04:34:59.608 INFO:tasks.workunit.client.1.smithi053.stdout:3/32: link l4 l5 0 2024-09-21T04:34:59.610 INFO:tasks.workunit.client.1.smithi053.stdout:5/42: mknod d0/d8/c9 0 2024-09-21T04:34:59.612 INFO:tasks.workunit.client.1.smithi053.stdout:8/35: symlink d3/lc 0 2024-09-21T04:34:59.613 INFO:tasks.workunit.client.1.smithi053.stdout:4/36: dwrite d1/f3 [0,4194304] 0 2024-09-21T04:34:59.613 INFO:tasks.workunit.client.1.smithi053.stdout:4/37: stat d1/l9 0 2024-09-21T04:34:59.614 INFO:tasks.workunit.client.1.smithi053.stdout:4/38: dread - d1/fa zero size 2024-09-21T04:34:59.614 INFO:tasks.workunit.client.1.smithi053.stdout:4/39: write d1/fa [919088,93650] 0 2024-09-21T04:34:59.615 INFO:tasks.workunit.client.1.smithi053.stdout:4/40: stat d1/cb 0 2024-09-21T04:34:59.623 INFO:tasks.workunit.client.1.smithi053.stdout:5/43: rename d0/d8/c9 to d0/ca 0 2024-09-21T04:34:59.625 INFO:tasks.workunit.client.1.smithi053.stdout:8/36: mkdir d3/dd 0 2024-09-21T04:34:59.628 INFO:tasks.workunit.client.1.smithi053.stdout:1/105: dwrite d7/d9/da/d11/f14 [0,4194304] 0 2024-09-21T04:34:59.628 INFO:tasks.workunit.client.1.smithi053.stdout:4/41: dwrite d1/f3 [0,4194304] 0 2024-09-21T04:34:59.629 INFO:tasks.workunit.client.1.smithi053.stdout:8/37: write d3/fa [30624,59486] 0 2024-09-21T04:34:59.629 INFO:tasks.workunit.client.1.smithi053.stdout:4/42: stat d1/l6 0 2024-09-21T04:34:59.635 INFO:tasks.workunit.client.1.smithi053.stdout:1/106: dread f1 [0,4194304] 0 2024-09-21T04:34:59.635 INFO:tasks.workunit.client.1.smithi053.stdout:8/38: fsync d3/f7 0 2024-09-21T04:34:59.635 INFO:tasks.workunit.client.1.smithi053.stdout:1/107: chown f1 50230 1 2024-09-21T04:34:59.636 INFO:tasks.workunit.client.1.smithi053.stdout:4/43: dread d1/fa [0,4194304] 0 2024-09-21T04:34:59.637 INFO:tasks.workunit.client.1.smithi053.stdout:4/44: write d1/f4 [532476,119560] 0 2024-09-21T04:34:59.637 INFO:tasks.workunit.client.1.smithi053.stdout:4/45: readlink d1/l7 0 2024-09-21T04:34:59.642 INFO:tasks.workunit.client.1.smithi053.stdout:8/39: symlink d3/le 0 2024-09-21T04:34:59.643 INFO:tasks.workunit.client.1.smithi053.stdout:8/40: dread d3/f7 [0,4194304] 0 2024-09-21T04:34:59.644 INFO:tasks.workunit.client.1.smithi053.stdout:8/41: write d3/f7 [903283,124392] 0 2024-09-21T04:34:59.645 INFO:tasks.workunit.client.1.smithi053.stdout:4/46: creat d1/fc x:0 0 0 2024-09-21T04:34:59.647 INFO:tasks.workunit.client.1.smithi053.stdout:4/47: fdatasync d1/f4 0 2024-09-21T04:34:59.647 INFO:tasks.workunit.client.1.smithi053.stdout:8/42: rmdir d3/d5 39 2024-09-21T04:34:59.649 INFO:tasks.workunit.client.1.smithi053.stdout:4/48: write d1/fc [491214,36356] 0 2024-09-21T04:34:59.650 INFO:tasks.workunit.client.1.smithi053.stdout:4/49: chown d1/fa 12877 1 2024-09-21T04:34:59.650 INFO:tasks.workunit.client.1.smithi053.stdout:1/108: dwrite f0 [0,4194304] 0 2024-09-21T04:34:59.652 INFO:tasks.workunit.client.1.smithi053.stdout:8/43: mkdir d3/df 0 2024-09-21T04:34:59.654 INFO:tasks.workunit.client.1.smithi053.stdout:4/50: mkdir d1/dd 0 2024-09-21T04:34:59.655 INFO:tasks.workunit.client.1.smithi053.stdout:8/44: write d3/fa [887657,82549] 0 2024-09-21T04:34:59.661 INFO:tasks.workunit.client.1.smithi053.stdout:1/109: rmdir d7/d9/da/d1d 0 2024-09-21T04:34:59.662 INFO:tasks.workunit.client.1.smithi053.stdout:4/51: mkdir d1/dd/de 0 2024-09-21T04:34:59.667 INFO:tasks.workunit.client.1.smithi053.stdout:1/110: mknod d7/c22 0 2024-09-21T04:34:59.669 INFO:tasks.workunit.client.1.smithi053.stdout:4/52: dwrite d1/fc [0,4194304] 0 2024-09-21T04:34:59.669 INFO:tasks.workunit.client.1.smithi053.stdout:1/111: chown d7/d9/da/l10 9517 1 2024-09-21T04:34:59.676 INFO:tasks.workunit.client.1.smithi053.stdout:1/112: dwrite f0 [0,4194304] 0 2024-09-21T04:34:59.681 INFO:tasks.workunit.client.1.smithi053.stdout:1/113: rename d7/c8 to d7/d9/da/c23 0 2024-09-21T04:34:59.688 INFO:tasks.workunit.client.1.smithi053.stdout:1/114: dwrite d7/d9/da/d11/f20 [0,4194304] 0 2024-09-21T04:34:59.692 INFO:tasks.workunit.client.1.smithi053.stdout:1/115: mkdir d7/d24 0 2024-09-21T04:34:59.695 INFO:tasks.workunit.client.1.smithi053.stdout:1/116: unlink d7/d9/da/d11/c15 0 2024-09-21T04:34:59.695 INFO:tasks.workunit.client.1.smithi053.stdout:1/117: write f1 [973435,16944] 0 2024-09-21T04:34:59.696 INFO:tasks.workunit.client.1.smithi053.stdout:1/118: chown d7/d9 15785 1 2024-09-21T04:34:59.697 INFO:tasks.workunit.client.1.smithi053.stdout:1/119: chown d7/d9/da/d11/l1e 20503189 1 2024-09-21T04:34:59.699 INFO:tasks.workunit.client.1.smithi053.stdout:7/11: sync 2024-09-21T04:34:59.713 INFO:tasks.workunit.client.1.smithi053.stdout:7/12: getdents d0 0 2024-09-21T04:34:59.713 INFO:tasks.workunit.client.1.smithi053.stdout:7/13: fdatasync - no filename 2024-09-21T04:34:59.713 INFO:tasks.workunit.client.1.smithi053.stdout:1/120: dwrite d7/d9/da/d11/f14 [0,4194304] 0 2024-09-21T04:34:59.919 INFO:tasks.workunit.client.1.smithi053.stdout:6/34: dwrite d2/f7 [4194304,4194304] 0 2024-09-21T04:34:59.923 INFO:tasks.workunit.client.1.smithi053.stdout:9/46: write d0/f1 [2304230,52529] 0 2024-09-21T04:34:59.924 INFO:tasks.workunit.client.1.smithi053.stdout:0/13: fsync f1 0 2024-09-21T04:34:59.935 INFO:tasks.workunit.client.1.smithi053.stdout:2/35: unlink f0 0 2024-09-21T04:34:59.935 INFO:tasks.workunit.client.1.smithi053.stdout:2/36: readlink d1/l2 0 2024-09-21T04:34:59.939 INFO:tasks.workunit.client.1.smithi053.stdout:5/44: truncate d0/f7 1216777 0 2024-09-21T04:34:59.939 INFO:tasks.workunit.client.1.smithi053.stdout:5/45: read d0/f3 [1384668,28474] 0 2024-09-21T04:34:59.944 INFO:tasks.workunit.client.1.smithi053.stdout:1/121: rename f4 to d7/d24/f25 0 2024-09-21T04:34:59.946 INFO:tasks.workunit.client.1.smithi053.stdout:6/35: rmdir d2 39 2024-09-21T04:34:59.947 INFO:tasks.workunit.client.1.smithi053.stdout:0/14: symlink l2 0 2024-09-21T04:34:59.947 INFO:tasks.workunit.client.1.smithi053.stdout:2/37: rmdir d1 39 2024-09-21T04:34:59.951 INFO:tasks.workunit.client.1.smithi053.stdout:7/14: getdents d0 0 2024-09-21T04:34:59.951 INFO:tasks.workunit.client.1.smithi053.stdout:7/15: read - no filename 2024-09-21T04:34:59.951 INFO:tasks.workunit.client.1.smithi053.stdout:7/16: fsync - no filename 2024-09-21T04:34:59.952 INFO:tasks.workunit.client.1.smithi053.stdout:3/33: truncate f3 513398 0 2024-09-21T04:34:59.954 INFO:tasks.workunit.client.1.smithi053.stdout:1/122: creat d7/d9/da/d11/f26 x:0 0 0 2024-09-21T04:34:59.955 INFO:tasks.workunit.client.1.smithi053.stdout:0/15: creat f3 x:0 0 0 2024-09-21T04:34:59.956 INFO:tasks.workunit.client.1.smithi053.stdout:0/16: truncate f3 704222 0 2024-09-21T04:34:59.961 INFO:tasks.workunit.client.1.smithi053.stdout:3/34: mknod c6 0 2024-09-21T04:34:59.962 INFO:tasks.workunit.client.1.smithi053.stdout:0/17: dwrite f3 [0,4194304] 0 2024-09-21T04:34:59.964 INFO:tasks.workunit.client.1.smithi053.stdout:2/38: symlink d1/le 0 2024-09-21T04:34:59.966 INFO:tasks.workunit.client.1.smithi053.stdout:8/45: truncate d3/f7 850671 0 2024-09-21T04:34:59.970 INFO:tasks.workunit.client.1.smithi053.stdout:7/17: symlink d0/l1 0 2024-09-21T04:34:59.974 INFO:tasks.workunit.client.1.smithi053.stdout:1/123: mknod d7/d9/c27 0 2024-09-21T04:34:59.975 INFO:tasks.workunit.client.1.smithi053.stdout:0/18: symlink l4 0 2024-09-21T04:34:59.976 INFO:tasks.workunit.client.1.smithi053.stdout:2/39: mknod d1/cf 0 2024-09-21T04:34:59.976 INFO:tasks.workunit.client.1.smithi053.stdout:2/40: stat d1 0 2024-09-21T04:34:59.978 INFO:tasks.workunit.client.1.smithi053.stdout:4/53: truncate d1/fc 1058502 0 2024-09-21T04:34:59.979 INFO:tasks.workunit.client.1.smithi053.stdout:5/46: getdents d0 0 2024-09-21T04:34:59.982 INFO:tasks.workunit.client.1.smithi053.stdout:7/18: creat d0/f2 x:0 0 0 2024-09-21T04:34:59.983 INFO:tasks.workunit.client.1.smithi053.stdout:7/19: chown d0/f2 9 1 2024-09-21T04:34:59.983 INFO:tasks.workunit.client.1.smithi053.stdout:7/20: rename d0 to d0/d3 22 2024-09-21T04:34:59.984 INFO:tasks.workunit.client.1.smithi053.stdout:1/124: mkdir d7/d24/d28 0 2024-09-21T04:34:59.985 INFO:tasks.workunit.client.1.smithi053.stdout:1/125: write f0 [259845,120097] 0 2024-09-21T04:34:59.988 INFO:tasks.workunit.client.1.smithi053.stdout:2/41: read d1/f5 [3068532,10681] 0 2024-09-21T04:34:59.989 INFO:tasks.workunit.client.1.smithi053.stdout:4/54: mknod d1/dd/cf 0 2024-09-21T04:34:59.990 INFO:tasks.workunit.client.1.smithi053.stdout:2/42: dread d1/f4 [0,4194304] 0 2024-09-21T04:34:59.992 INFO:tasks.workunit.client.1.smithi053.stdout:5/47: mkdir d0/d8/db 0 2024-09-21T04:34:59.994 INFO:tasks.workunit.client.1.smithi053.stdout:7/21: creat d0/f4 x:0 0 0 2024-09-21T04:34:59.996 INFO:tasks.workunit.client.1.smithi053.stdout:1/126: chown f3 3995 1 2024-09-21T04:34:59.997 INFO:tasks.workunit.client.1.smithi053.stdout:1/127: read f1 [1697078,78608] 0 2024-09-21T04:34:59.997 INFO:tasks.workunit.client.1.smithi053.stdout:1/128: chown d7/d24/d28 0 1 2024-09-21T04:34:59.998 INFO:tasks.workunit.client.1.smithi053.stdout:1/129: write d7/d9/da/d11/f20 [1089500,10818] 0 2024-09-21T04:35:00.000 INFO:tasks.workunit.client.1.smithi053.stdout:2/43: mknod d1/c10 0 2024-09-21T04:35:00.002 INFO:tasks.workunit.client.1.smithi053.stdout:5/48: rename d0/c6 to d0/d8/db/cc 0 2024-09-21T04:35:00.005 INFO:tasks.workunit.client.1.smithi053.stdout:8/46: rmdir d3/dd 0 2024-09-21T04:35:00.007 INFO:tasks.workunit.client.1.smithi053.stdout:7/22: creat d0/f5 x:0 0 0 2024-09-21T04:35:00.007 INFO:tasks.workunit.client.1.smithi053.stdout:7/23: truncate d0/f5 937465 0 2024-09-21T04:35:00.008 INFO:tasks.workunit.client.1.smithi053.stdout:7/24: read d0/f5 [35187,69896] 0 2024-09-21T04:35:00.008 INFO:tasks.workunit.client.1.smithi053.stdout:7/25: dread - d0/f2 zero size 2024-09-21T04:35:00.011 INFO:tasks.workunit.client.1.smithi053.stdout:1/130: rmdir d7/d9/da/d11 39 2024-09-21T04:35:00.014 INFO:tasks.workunit.client.1.smithi053.stdout:5/49: fsync d0/f3 0 2024-09-21T04:35:00.017 INFO:tasks.workunit.client.1.smithi053.stdout:5/50: dread d0/f3 [0,4194304] 0 2024-09-21T04:35:00.018 INFO:tasks.workunit.client.1.smithi053.stdout:1/131: write d7/d9/da/d11/f14 [1799222,50342] 0 2024-09-21T04:35:00.021 INFO:tasks.workunit.client.1.smithi053.stdout:8/47: chown d3/f9 37615 1 2024-09-21T04:35:00.021 INFO:tasks.workunit.client.1.smithi053.stdout:5/51: dread d0/f3 [0,4194304] 0 2024-09-21T04:35:00.021 INFO:tasks.workunit.client.1.smithi053.stdout:8/48: chown d3/fa 6002588 1 2024-09-21T04:35:00.024 INFO:tasks.workunit.client.1.smithi053.stdout:2/44: link d1/cd d1/c11 0 2024-09-21T04:35:00.026 INFO:tasks.workunit.client.1.smithi053.stdout:2/45: rmdir d1 39 2024-09-21T04:35:00.034 INFO:tasks.workunit.client.1.smithi053.stdout:2/46: dwrite d1/f9 [0,4194304] 0 2024-09-21T04:35:00.036 INFO:tasks.workunit.client.1.smithi053.stdout:2/47: write d1/f9 [5210542,115079] 0 2024-09-21T04:35:00.039 INFO:tasks.workunit.client.1.smithi053.stdout:2/48: creat d1/f12 x:0 0 0 2024-09-21T04:35:00.039 INFO:tasks.workunit.client.1.smithi053.stdout:2/49: dread - d1/fb zero size 2024-09-21T04:35:00.040 INFO:tasks.workunit.client.1.smithi053.stdout:2/50: truncate d1/f12 255733 0 2024-09-21T04:35:00.048 INFO:tasks.workunit.client.1.smithi053.stdout:2/51: dwrite d1/f12 [0,4194304] 0 2024-09-21T04:35:00.070 INFO:tasks.workunit.client.1.smithi053.stdout:9/47: sync 2024-09-21T04:35:00.071 INFO:tasks.workunit.client.1.smithi053.stdout:6/36: sync 2024-09-21T04:35:00.076 INFO:tasks.workunit.client.1.smithi053.stdout:6/37: dread d2/f7 [4194304,4194304] 0 2024-09-21T04:35:00.076 INFO:tasks.workunit.client.1.smithi053.stdout:6/38: chown d2/f3 46246237 1 2024-09-21T04:35:00.084 INFO:tasks.workunit.client.1.smithi053.stdout:9/48: dwrite d0/d2/d9/fc [0,4194304] 0 2024-09-21T04:35:00.085 INFO:tasks.workunit.client.1.smithi053.stdout:6/39: dwrite d2/f3 [0,4194304] 0 2024-09-21T04:35:00.086 INFO:tasks.workunit.client.1.smithi053.stdout:9/49: truncate d0/f4 441655 0 2024-09-21T04:35:00.090 INFO:tasks.workunit.client.1.smithi053.stdout:6/40: mkdir d2/d9 0 2024-09-21T04:35:00.091 INFO:tasks.workunit.client.1.smithi053.stdout:6/41: chown d2/f7 202432 1 2024-09-21T04:35:00.093 INFO:tasks.workunit.client.1.smithi053.stdout:6/42: read d2/f7 [4922647,83870] 0 2024-09-21T04:35:00.094 INFO:tasks.workunit.client.1.smithi053.stdout:9/50: dwrite d0/d2/fa [0,4194304] 0 2024-09-21T04:35:00.098 INFO:tasks.workunit.client.1.smithi053.stdout:6/43: creat d2/fa x:0 0 0 2024-09-21T04:35:00.102 INFO:tasks.workunit.client.1.smithi053.stdout:6/44: creat d2/fb x:0 0 0 2024-09-21T04:35:00.102 INFO:tasks.workunit.client.1.smithi053.stdout:6/45: chown d2/d9 121082 1 2024-09-21T04:35:00.103 INFO:tasks.workunit.client.1.smithi053.stdout:9/51: symlink d0/d2/d9/l12 0 2024-09-21T04:35:00.105 INFO:tasks.workunit.client.1.smithi053.stdout:9/52: read - d0/d2/fd zero size 2024-09-21T04:35:00.111 INFO:tasks.workunit.client.1.smithi053.stdout:6/46: dwrite d2/fa [0,4194304] 0 2024-09-21T04:35:00.114 INFO:tasks.workunit.client.1.smithi053.stdout:6/47: truncate f1 1981593 0 2024-09-21T04:35:00.115 INFO:tasks.workunit.client.1.smithi053.stdout:1/132: sync 2024-09-21T04:35:00.120 INFO:tasks.workunit.client.1.smithi053.stdout:6/48: chown f1 23 1 2024-09-21T04:35:00.120 INFO:tasks.workunit.client.1.smithi053.stdout:6/49: chown d2/fb 108614340 1 2024-09-21T04:35:00.123 INFO:tasks.workunit.client.1.smithi053.stdout:6/50: creat d2/fc x:0 0 0 2024-09-21T04:35:00.126 INFO:tasks.workunit.client.1.smithi053.stdout:6/51: dread d2/f7 [0,4194304] 0 2024-09-21T04:35:00.129 INFO:tasks.workunit.client.1.smithi053.stdout:6/52: creat d2/fd x:0 0 0 2024-09-21T04:35:00.133 INFO:tasks.workunit.client.1.smithi053.stdout:6/53: mkdir d2/d9/de 0 2024-09-21T04:35:00.135 INFO:tasks.workunit.client.1.smithi053.stdout:6/54: unlink d2/fc 0 2024-09-21T04:35:00.136 INFO:tasks.workunit.client.1.smithi053.stdout:6/55: stat d2/f7 0 2024-09-21T04:35:00.136 INFO:tasks.workunit.client.1.smithi053.stdout:6/56: dread - d2/fb zero size 2024-09-21T04:35:00.143 INFO:tasks.workunit.client.1.smithi053.stdout:6/57: dwrite d2/fb [0,4194304] 0 2024-09-21T04:35:00.144 INFO:tasks.workunit.client.1.smithi053.stdout:6/58: write d2/fd [919060,116851] 0 2024-09-21T04:35:00.148 INFO:tasks.workunit.client.1.smithi053.stdout:6/59: symlink d2/lf 0 2024-09-21T04:35:00.148 INFO:tasks.workunit.client.1.smithi053.stdout:6/60: chown d2 81268888 1 2024-09-21T04:35:00.148 INFO:tasks.workunit.client.1.smithi053.stdout:8/49: sync 2024-09-21T04:35:00.149 INFO:tasks.workunit.client.1.smithi053.stdout:8/50: write d3/fa [686187,54661] 0 2024-09-21T04:35:00.151 INFO:tasks.workunit.client.1.smithi053.stdout:6/61: read d2/fd [702004,75755] 0 2024-09-21T04:35:00.157 INFO:tasks.workunit.client.1.smithi053.stdout:6/62: creat d2/f10 x:0 0 0 2024-09-21T04:35:00.157 INFO:tasks.workunit.client.1.smithi053.stdout:6/63: fdatasync d2/f8 0 2024-09-21T04:35:00.161 INFO:tasks.workunit.client.1.smithi053.stdout:8/51: mkdir d3/df/d10 0 2024-09-21T04:35:00.163 INFO:tasks.workunit.client.1.smithi053.stdout:6/64: creat d2/d9/f11 x:0 0 0 2024-09-21T04:35:00.165 INFO:tasks.workunit.client.1.smithi053.stdout:8/52: creat d3/df/f11 x:0 0 0 2024-09-21T04:35:00.173 INFO:tasks.workunit.client.1.smithi053.stdout:6/65: dwrite d2/f7 [4194304,4194304] 0 2024-09-21T04:35:00.177 INFO:tasks.workunit.client.1.smithi053.stdout:6/66: unlink d2/f7 0 2024-09-21T04:35:00.178 INFO:tasks.workunit.client.1.smithi053.stdout:8/53: link d3/lc d3/l12 0 2024-09-21T04:35:00.188 INFO:tasks.workunit.client.1.smithi053.stdout:8/54: creat d3/d5/f13 x:0 0 0 2024-09-21T04:35:00.188 INFO:tasks.workunit.client.1.smithi053.stdout:8/55: chown d3/df/f11 1188 1 2024-09-21T04:35:00.192 INFO:tasks.workunit.client.1.smithi053.stdout:5/52: fsync d0/f7 0 2024-09-21T04:35:00.197 INFO:tasks.workunit.client.1.smithi053.stdout:1/133: rename d7/d24/f25 to d7/d24/f29 0 2024-09-21T04:35:00.199 INFO:tasks.workunit.client.1.smithi053.stdout:5/53: mkdir d0/dd 0 2024-09-21T04:35:00.200 INFO:tasks.workunit.client.1.smithi053.stdout:5/54: chown d0/c4 0 1 2024-09-21T04:35:00.203 INFO:tasks.workunit.client.1.smithi053.stdout:1/134: dwrite d7/d9/da/d11/f20 [0,4194304] 0 2024-09-21T04:35:00.205 INFO:tasks.workunit.client.1.smithi053.stdout:5/55: creat d0/fe x:0 0 0 2024-09-21T04:35:00.207 INFO:tasks.workunit.client.1.smithi053.stdout:1/135: write f1 [4954726,68541] 0 2024-09-21T04:35:00.210 INFO:tasks.workunit.client.1.smithi053.stdout:5/56: mknod d0/d8/db/cf 0 2024-09-21T04:35:00.212 INFO:tasks.workunit.client.1.smithi053.stdout:1/136: mkdir d7/d24/d2a 0 2024-09-21T04:35:00.219 INFO:tasks.workunit.client.1.smithi053.stdout:0/19: dwrite f3 [4194304,4194304] 0 2024-09-21T04:35:00.219 INFO:tasks.workunit.client.1.smithi053.stdout:0/20: fdatasync f1 0 2024-09-21T04:35:00.220 INFO:tasks.workunit.client.1.smithi053.stdout:0/21: chown f1 0 1 2024-09-21T04:35:00.229 INFO:tasks.workunit.client.1.smithi053.stdout:0/22: dread f3 [0,4194304] 0 2024-09-21T04:35:00.229 INFO:tasks.workunit.client.1.smithi053.stdout:4/55: dwrite d1/fc [0,4194304] 0 2024-09-21T04:35:00.229 INFO:tasks.workunit.client.1.smithi053.stdout:0/23: stat f3 0 2024-09-21T04:35:00.238 INFO:tasks.workunit.client.1.smithi053.stdout:4/56: dwrite d1/f4 [0,4194304] 0 2024-09-21T04:35:00.241 INFO:tasks.workunit.client.1.smithi053.stdout:7/26: fsync d0/f5 0 2024-09-21T04:35:00.243 INFO:tasks.workunit.client.1.smithi053.stdout:7/27: dread - d0/f2 zero size 2024-09-21T04:35:00.244 INFO:tasks.workunit.client.1.smithi053.stdout:4/57: symlink d1/dd/l10 0 2024-09-21T04:35:00.244 INFO:tasks.workunit.client.1.smithi053.stdout:4/58: chown d1/l8 1258 1 2024-09-21T04:35:00.244 INFO:tasks.workunit.client.1.smithi053.stdout:4/59: stat d1 0 2024-09-21T04:35:00.245 INFO:tasks.workunit.client.1.smithi053.stdout:7/28: mkdir d0/d6 0 2024-09-21T04:35:00.246 INFO:tasks.workunit.client.1.smithi053.stdout:7/29: write d0/f2 [593557,13042] 0 2024-09-21T04:35:00.246 INFO:tasks.workunit.client.1.smithi053.stdout:5/57: truncate d0/f3 1809802 0 2024-09-21T04:35:00.248 INFO:tasks.workunit.client.1.smithi053.stdout:4/60: creat d1/dd/f11 x:0 0 0 2024-09-21T04:35:00.251 INFO:tasks.workunit.client.1.smithi053.stdout:7/30: unlink d0/f5 0 2024-09-21T04:35:00.253 INFO:tasks.workunit.client.1.smithi053.stdout:5/58: creat d0/d8/f10 x:0 0 0 2024-09-21T04:35:00.254 INFO:tasks.workunit.client.1.smithi053.stdout:5/59: chown d0/fe 77557762 1 2024-09-21T04:35:00.255 INFO:tasks.workunit.client.1.smithi053.stdout:5/60: write d0/d8/f10 [237996,46677] 0 2024-09-21T04:35:00.262 INFO:tasks.workunit.client.1.smithi053.stdout:5/61: symlink d0/l11 0 2024-09-21T04:35:00.264 INFO:tasks.workunit.client.1.smithi053.stdout:2/52: getdents d1 0 2024-09-21T04:35:00.268 INFO:tasks.workunit.client.1.smithi053.stdout:7/31: creat d0/d6/f7 x:0 0 0 2024-09-21T04:35:00.270 INFO:tasks.workunit.client.1.smithi053.stdout:5/62: symlink d0/l12 0 2024-09-21T04:35:00.270 INFO:tasks.workunit.client.1.smithi053.stdout:5/63: chown d0/fe 17420 1 2024-09-21T04:35:00.291 INFO:tasks.workunit.client.1.smithi053.stdout:6/67: fsync d2/fb 0 2024-09-21T04:35:00.293 INFO:tasks.workunit.client.1.smithi053.stdout:6/68: creat d2/d9/f12 x:0 0 0 2024-09-21T04:35:00.293 INFO:tasks.workunit.client.1.smithi053.stdout:6/69: stat d2/d9 0 2024-09-21T04:35:00.299 INFO:tasks.workunit.client.1.smithi053.stdout:6/70: dread d2/fb [0,4194304] 0 2024-09-21T04:35:00.299 INFO:tasks.workunit.client.1.smithi053.stdout:6/71: rename d2/d9 to d2/d9/de/d13 22 2024-09-21T04:35:00.300 INFO:tasks.workunit.client.1.smithi053.stdout:9/53: dwrite d0/d2/d9/fc [4194304,4194304] 0 2024-09-21T04:35:00.301 INFO:tasks.workunit.client.1.smithi053.stdout:9/54: read - d0/d2/fd zero size 2024-09-21T04:35:00.302 INFO:tasks.workunit.client.1.smithi053.stdout:9/55: truncate d0/d2/fd 1017339 0 2024-09-21T04:35:00.303 INFO:tasks.workunit.client.1.smithi053.stdout:9/56: write d0/d2/fe [831542,50244] 0 2024-09-21T04:35:00.306 INFO:tasks.workunit.client.1.smithi053.stdout:9/57: creat d0/d2/f13 x:0 0 0 2024-09-21T04:35:00.309 INFO:tasks.workunit.client.1.smithi053.stdout:9/58: mknod d0/df/c14 0 2024-09-21T04:35:00.311 INFO:tasks.workunit.client.1.smithi053.stdout:9/59: dread d0/d2/fd [0,4194304] 0 2024-09-21T04:35:00.312 INFO:tasks.workunit.client.1.smithi053.stdout:9/60: creat d0/f15 x:0 0 0 2024-09-21T04:35:00.315 INFO:tasks.workunit.client.1.smithi053.stdout:9/61: mknod d0/d2/c16 0 2024-09-21T04:35:00.319 INFO:tasks.workunit.client.1.smithi053.stdout:9/62: dread d0/d2/fe [0,4194304] 0 2024-09-21T04:35:00.322 INFO:tasks.workunit.client.1.smithi053.stdout:9/63: dread d0/d2/fd [0,4194304] 0 2024-09-21T04:35:00.324 INFO:tasks.workunit.client.1.smithi053.stdout:9/64: mknod d0/d2/d9/c17 0 2024-09-21T04:35:00.327 INFO:tasks.workunit.client.1.smithi053.stdout:9/65: dread d0/d2/fa [0,4194304] 0 2024-09-21T04:35:00.350 INFO:tasks.workunit.client.1.smithi053.stdout:3/35: dwrite f3 [0,4194304] 0 2024-09-21T04:35:00.352 INFO:tasks.workunit.client.1.smithi053.stdout:3/36: write f3 [3325220,93053] 0 2024-09-21T04:35:00.355 INFO:tasks.workunit.client.1.smithi053.stdout:0/24: truncate f3 1460879 0 2024-09-21T04:35:00.356 INFO:tasks.workunit.client.1.smithi053.stdout:0/25: readlink l4 0 2024-09-21T04:35:00.357 INFO:tasks.workunit.client.1.smithi053.stdout:8/56: dwrite d3/f7 [0,4194304] 0 2024-09-21T04:35:00.360 INFO:tasks.workunit.client.1.smithi053.stdout:0/26: mkdir d5 0 2024-09-21T04:35:00.360 INFO:tasks.workunit.client.1.smithi053.stdout:3/37: dread f3 [0,4194304] 0 2024-09-21T04:35:00.360 INFO:tasks.workunit.client.1.smithi053.stdout:1/137: sync 2024-09-21T04:35:00.361 INFO:tasks.workunit.client.1.smithi053.stdout:8/57: mkdir d3/df/d14 0 2024-09-21T04:35:00.362 INFO:tasks.workunit.client.1.smithi053.stdout:1/138: chown d7/d9/c27 0 1 2024-09-21T04:35:00.362 INFO:tasks.workunit.client.1.smithi053.stdout:8/58: chown d3/df/f11 8802 1 2024-09-21T04:35:00.366 INFO:tasks.workunit.client.1.smithi053.stdout:3/38: dread f3 [0,4194304] 0 2024-09-21T04:35:00.372 INFO:tasks.workunit.client.1.smithi053.stdout:3/39: write f3 [396764,120512] 0 2024-09-21T04:35:00.373 INFO:tasks.workunit.client.1.smithi053.stdout:1/139: rmdir d7/d24/d28 0 2024-09-21T04:35:00.374 INFO:tasks.workunit.client.1.smithi053.stdout:1/140: readlink d7/d9/l21 0 2024-09-21T04:35:00.376 INFO:tasks.workunit.client.1.smithi053.stdout:1/141: rmdir d7 39 2024-09-21T04:35:00.378 INFO:tasks.workunit.client.1.smithi053.stdout:3/40: dwrite f3 [0,4194304] 0 2024-09-21T04:35:00.380 INFO:tasks.workunit.client.1.smithi053.stdout:3/41: write f3 [1430076,57974] 0 2024-09-21T04:35:00.380 INFO:tasks.workunit.client.1.smithi053.stdout:1/142: chown d7/d9/da/c17 4 1 2024-09-21T04:35:00.380 INFO:tasks.workunit.client.1.smithi053.stdout:1/143: stat f3 0 2024-09-21T04:35:00.381 INFO:tasks.workunit.client.1.smithi053.stdout:1/144: read d7/d9/da/d11/f20 [1225931,119747] 0 2024-09-21T04:35:00.382 INFO:tasks.workunit.client.1.smithi053.stdout:3/42: creat f7 x:0 0 0 2024-09-21T04:35:00.385 INFO:tasks.workunit.client.1.smithi053.stdout:1/145: dread f1 [0,4194304] 0 2024-09-21T04:35:00.388 INFO:tasks.workunit.client.1.smithi053.stdout:3/43: symlink l8 0 2024-09-21T04:35:00.389 INFO:tasks.workunit.client.1.smithi053.stdout:3/44: write f7 [4642,101499] 0 2024-09-21T04:35:00.390 INFO:tasks.workunit.client.1.smithi053.stdout:3/45: stat l2 0 2024-09-21T04:35:00.392 INFO:tasks.workunit.client.1.smithi053.stdout:4/61: truncate d1/f3 1774359 0 2024-09-21T04:35:00.393 INFO:tasks.workunit.client.1.smithi053.stdout:1/146: dwrite d7/d9/da/d11/f26 [0,4194304] 0 2024-09-21T04:35:00.394 INFO:tasks.workunit.client.1.smithi053.stdout:3/46: creat f9 x:0 0 0 2024-09-21T04:35:00.397 INFO:tasks.workunit.client.1.smithi053.stdout:3/47: rename f3 to fa 0 2024-09-21T04:35:00.398 INFO:tasks.workunit.client.1.smithi053.stdout:3/48: write f9 [568220,7450] 0 2024-09-21T04:35:00.400 INFO:tasks.workunit.client.1.smithi053.stdout:1/147: mkdir d7/d9/da/d11/d2b 0 2024-09-21T04:35:00.400 INFO:tasks.workunit.client.1.smithi053.stdout:1/148: chown d7/d9 2495 1 2024-09-21T04:35:00.401 INFO:tasks.workunit.client.1.smithi053.stdout:4/62: getdents d1/dd/de 0 2024-09-21T04:35:00.404 INFO:tasks.workunit.client.1.smithi053.stdout:1/149: fsync d7/d24/f29 0 2024-09-21T04:35:00.409 INFO:tasks.workunit.client.1.smithi053.stdout:3/49: dwrite fa [0,4194304] 0 2024-09-21T04:35:00.410 INFO:tasks.workunit.client.1.smithi053.stdout:3/50: chown fa 349601 1 2024-09-21T04:35:00.412 INFO:tasks.workunit.client.1.smithi053.stdout:3/51: symlink lb 0 2024-09-21T04:35:00.414 INFO:tasks.workunit.client.1.smithi053.stdout:1/150: rename d7/d9/da/c1a to d7/d9/da/c2c 0 2024-09-21T04:35:00.416 INFO:tasks.workunit.client.1.smithi053.stdout:3/52: creat fc x:0 0 0 2024-09-21T04:35:00.418 INFO:tasks.workunit.client.1.smithi053.stdout:6/72: write f1 [1294102,102771] 0 2024-09-21T04:35:00.420 INFO:tasks.workunit.client.1.smithi053.stdout:1/151: symlink d7/d9/l2d 0 2024-09-21T04:35:00.424 INFO:tasks.workunit.client.1.smithi053.stdout:3/53: dwrite f9 [0,4194304] 0 2024-09-21T04:35:00.424 INFO:tasks.workunit.client.1.smithi053.stdout:6/73: creat d2/d9/de/f14 x:0 0 0 2024-09-21T04:35:00.427 INFO:tasks.workunit.client.1.smithi053.stdout:3/54: creat fd x:0 0 0 2024-09-21T04:35:00.429 INFO:tasks.workunit.client.1.smithi053.stdout:3/55: creat fe x:0 0 0 2024-09-21T04:35:00.430 INFO:tasks.workunit.client.1.smithi053.stdout:1/152: dread f0 [0,4194304] 0 2024-09-21T04:35:00.433 INFO:tasks.workunit.client.1.smithi053.stdout:2/53: sync 2024-09-21T04:35:00.436 INFO:tasks.workunit.client.1.smithi053.stdout:1/153: unlink d7/d9/da/d11/f14 0 2024-09-21T04:35:00.436 INFO:tasks.workunit.client.1.smithi053.stdout:6/74: dwrite d2/d9/f11 [0,4194304] 0 2024-09-21T04:35:00.439 INFO:tasks.workunit.client.1.smithi053.stdout:6/75: truncate f1 2568743 0 2024-09-21T04:35:00.439 INFO:tasks.workunit.client.1.smithi053.stdout:2/54: truncate d1/f5 2108377 0 2024-09-21T04:35:00.439 INFO:tasks.workunit.client.1.smithi053.stdout:1/154: rmdir d7/d9 39 2024-09-21T04:35:00.441 INFO:tasks.workunit.client.1.smithi053.stdout:1/155: stat d7/d24 0 2024-09-21T04:35:00.441 INFO:tasks.workunit.client.1.smithi053.stdout:2/55: stat d1/l2 0 2024-09-21T04:35:00.443 INFO:tasks.workunit.client.1.smithi053.stdout:3/56: dwrite f9 [0,4194304] 0 2024-09-21T04:35:00.444 INFO:tasks.workunit.client.1.smithi053.stdout:2/56: symlink d1/l13 0 2024-09-21T04:35:00.446 INFO:tasks.workunit.client.1.smithi053.stdout:3/57: readlink l5 0 2024-09-21T04:35:00.447 INFO:tasks.workunit.client.1.smithi053.stdout:1/156: dread d7/d24/f29 [0,4194304] 0 2024-09-21T04:35:00.447 INFO:tasks.workunit.client.1.smithi053.stdout:2/57: rmdir d1 39 2024-09-21T04:35:00.451 INFO:tasks.workunit.client.1.smithi053.stdout:3/58: mkdir df 0 2024-09-21T04:35:00.455 INFO:tasks.workunit.client.1.smithi053.stdout:0/27: truncate f3 2210857 0 2024-09-21T04:35:00.458 INFO:tasks.workunit.client.1.smithi053.stdout:2/58: rename d1/c10 to d1/c14 0 2024-09-21T04:35:00.461 INFO:tasks.workunit.client.1.smithi053.stdout:1/157: mknod d7/d9/c2e 0 2024-09-21T04:35:00.462 INFO:tasks.workunit.client.1.smithi053.stdout:8/59: dwrite d3/f7 [4194304,4194304] 0 2024-09-21T04:35:00.463 INFO:tasks.workunit.client.1.smithi053.stdout:0/28: mkdir d5/d6 0 2024-09-21T04:35:00.464 INFO:tasks.workunit.client.1.smithi053.stdout:8/60: write d3/df/f11 [774612,16381] 0 2024-09-21T04:35:00.467 INFO:tasks.workunit.client.1.smithi053.stdout:9/66: sync 2024-09-21T04:35:00.467 INFO:tasks.workunit.client.1.smithi053.stdout:7/32: sync 2024-09-21T04:35:00.467 INFO:tasks.workunit.client.1.smithi053.stdout:8/61: chown d3/le 80288 1 2024-09-21T04:35:00.468 INFO:tasks.workunit.client.1.smithi053.stdout:2/59: stat d1/f5 0 2024-09-21T04:35:00.469 INFO:tasks.workunit.client.1.smithi053.stdout:5/64: write d0/f3 [1555297,38761] 0 2024-09-21T04:35:00.473 INFO:tasks.workunit.client.1.smithi053.stdout:2/60: dread d1/f4 [0,4194304] 0 2024-09-21T04:35:00.474 INFO:tasks.workunit.client.1.smithi053.stdout:1/158: dread d7/d9/da/d11/f26 [0,4194304] 0 2024-09-21T04:35:00.475 INFO:tasks.workunit.client.1.smithi053.stdout:0/29: dwrite f1 [0,4194304] 0 2024-09-21T04:35:00.477 INFO:tasks.workunit.client.1.smithi053.stdout:7/33: rename d0/f2 to d0/d6/f8 0 2024-09-21T04:35:00.477 INFO:tasks.workunit.client.1.smithi053.stdout:5/65: creat d0/d8/f13 x:0 0 0 2024-09-21T04:35:00.479 INFO:tasks.workunit.client.1.smithi053.stdout:7/34: dread - d0/d6/f7 zero size 2024-09-21T04:35:00.480 INFO:tasks.workunit.client.1.smithi053.stdout:5/66: read d0/f3 [799895,117533] 0 2024-09-21T04:35:00.482 INFO:tasks.workunit.client.1.smithi053.stdout:0/30: dread f1 [0,4194304] 0 2024-09-21T04:35:00.490 INFO:tasks.workunit.client.1.smithi053.stdout:4/63: truncate d1/f4 1149602 0 2024-09-21T04:35:00.491 INFO:tasks.workunit.client.1.smithi053.stdout:7/35: creat d0/d6/f9 x:0 0 0 2024-09-21T04:35:00.491 INFO:tasks.workunit.client.1.smithi053.stdout:5/67: dwrite d0/d8/f10 [0,4194304] 0 2024-09-21T04:35:00.493 INFO:tasks.workunit.client.1.smithi053.stdout:0/31: dwrite f1 [0,4194304] 0 2024-09-21T04:35:00.499 INFO:tasks.workunit.client.1.smithi053.stdout:9/67: getdents d0/d2 0 2024-09-21T04:35:00.500 INFO:tasks.workunit.client.1.smithi053.stdout:9/68: chown d0/d2/c16 20605387 1 2024-09-21T04:35:00.501 INFO:tasks.workunit.client.1.smithi053.stdout:7/36: creat d0/fa x:0 0 0 2024-09-21T04:35:00.504 INFO:tasks.workunit.client.1.smithi053.stdout:3/59: fsync fe 0 2024-09-21T04:35:00.507 INFO:tasks.workunit.client.1.smithi053.stdout:1/159: creat d7/d9/da/f2f x:0 0 0 2024-09-21T04:35:00.508 INFO:tasks.workunit.client.1.smithi053.stdout:4/64: getdents d1/dd/de 0 2024-09-21T04:35:00.510 INFO:tasks.workunit.client.1.smithi053.stdout:9/69: mkdir d0/df/d18 0 2024-09-21T04:35:00.511 INFO:tasks.workunit.client.1.smithi053.stdout:4/65: chown d1/l9 966277 1 2024-09-21T04:35:00.514 INFO:tasks.workunit.client.1.smithi053.stdout:7/37: dwrite d0/d6/f9 [0,4194304] 0 2024-09-21T04:35:00.514 INFO:tasks.workunit.client.1.smithi053.stdout:6/76: write d2/fd [1366549,13580] 0 2024-09-21T04:35:00.515 INFO:tasks.workunit.client.1.smithi053.stdout:5/68: dwrite d0/f3 [0,4194304] 0 2024-09-21T04:35:00.516 INFO:tasks.workunit.client.1.smithi053.stdout:4/66: write d1/dd/f11 [766472,111275] 0 2024-09-21T04:35:00.519 INFO:tasks.workunit.client.1.smithi053.stdout:7/38: fdatasync d0/d6/f7 0 2024-09-21T04:35:00.522 INFO:tasks.workunit.client.1.smithi053.stdout:0/32: write f3 [1040980,96322] 0 2024-09-21T04:35:00.522 INFO:tasks.workunit.client.1.smithi053.stdout:0/33: chown f1 510 1 2024-09-21T04:35:00.523 INFO:tasks.workunit.client.1.smithi053.stdout:5/69: unlink d0/ca 0 2024-09-21T04:35:00.526 INFO:tasks.workunit.client.1.smithi053.stdout:0/34: dread f1 [0,4194304] 0 2024-09-21T04:35:00.530 INFO:tasks.workunit.client.1.smithi053.stdout:4/67: symlink d1/dd/l12 0 2024-09-21T04:35:00.533 INFO:tasks.workunit.client.1.smithi053.stdout:7/39: creat d0/fb x:0 0 0 2024-09-21T04:35:00.535 INFO:tasks.workunit.client.1.smithi053.stdout:4/68: dread d1/dd/f11 [0,4194304] 0 2024-09-21T04:35:00.535 INFO:tasks.workunit.client.1.smithi053.stdout:4/69: readlink d1/l6 0 2024-09-21T04:35:00.536 INFO:tasks.workunit.client.1.smithi053.stdout:3/60: rmdir df 0 2024-09-21T04:35:00.540 INFO:tasks.workunit.client.1.smithi053.stdout:6/77: link d2/d9/f11 d2/d9/de/f15 0 2024-09-21T04:35:00.541 INFO:tasks.workunit.client.1.smithi053.stdout:6/78: dread - d2/f10 zero size 2024-09-21T04:35:00.542 INFO:tasks.workunit.client.1.smithi053.stdout:0/35: mknod d5/c7 0 2024-09-21T04:35:00.542 INFO:tasks.workunit.client.1.smithi053.stdout:3/61: dwrite fc [0,4194304] 0 2024-09-21T04:35:00.543 INFO:tasks.workunit.client.1.smithi053.stdout:9/70: creat d0/d2/d9/f19 x:0 0 0 2024-09-21T04:35:00.547 INFO:tasks.workunit.client.1.smithi053.stdout:7/40: mknod d0/cc 0 2024-09-21T04:35:00.549 INFO:tasks.workunit.client.1.smithi053.stdout:7/41: read - d0/fa zero size 2024-09-21T04:35:00.551 INFO:tasks.workunit.client.1.smithi053.stdout:7/42: readlink d0/l1 0 2024-09-21T04:35:00.552 INFO:tasks.workunit.client.1.smithi053.stdout:6/79: dwrite d2/f10 [0,4194304] 0 2024-09-21T04:35:00.557 INFO:tasks.workunit.client.1.smithi053.stdout:0/36: mkdir d5/d8 0 2024-09-21T04:35:00.558 INFO:tasks.workunit.client.1.smithi053.stdout:3/62: creat f10 x:0 0 0 2024-09-21T04:35:00.559 INFO:tasks.workunit.client.1.smithi053.stdout:7/43: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:35:00.559 INFO:tasks.workunit.client.1.smithi053.stdout:0/37: write f3 [3210373,128438] 0 2024-09-21T04:35:00.559 INFO:tasks.workunit.client.1.smithi053.stdout:9/71: mkdir d0/d1a 0 2024-09-21T04:35:00.560 INFO:tasks.workunit.client.1.smithi053.stdout:7/44: dread - d0/fa zero size 2024-09-21T04:35:00.567 INFO:tasks.workunit.client.1.smithi053.stdout:2/61: write d1/f5 [415992,40199] 0 2024-09-21T04:35:00.567 INFO:tasks.workunit.client.1.smithi053.stdout:5/70: link d0/f7 d0/f14 0 2024-09-21T04:35:00.569 INFO:tasks.workunit.client.1.smithi053.stdout:6/80: creat d2/f16 x:0 0 0 2024-09-21T04:35:00.570 INFO:tasks.workunit.client.1.smithi053.stdout:5/71: dread - d0/d8/f13 zero size 2024-09-21T04:35:00.571 INFO:tasks.workunit.client.1.smithi053.stdout:9/72: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:35:00.572 INFO:tasks.workunit.client.1.smithi053.stdout:1/160: write f1 [2809863,4594] 0 2024-09-21T04:35:00.575 INFO:tasks.workunit.client.1.smithi053.stdout:3/63: creat f11 x:0 0 0 2024-09-21T04:35:00.575 INFO:tasks.workunit.client.1.smithi053.stdout:8/62: sync 2024-09-21T04:35:00.578 INFO:tasks.workunit.client.1.smithi053.stdout:7/45: creat d0/fd x:0 0 0 2024-09-21T04:35:00.579 INFO:tasks.workunit.client.1.smithi053.stdout:3/64: write fa [352738,46229] 0 2024-09-21T04:35:00.585 INFO:tasks.workunit.client.1.smithi053.stdout:5/72: creat d0/d8/db/f15 x:0 0 0 2024-09-21T04:35:00.587 INFO:tasks.workunit.client.1.smithi053.stdout:6/81: dwrite d2/f16 [0,4194304] 0 2024-09-21T04:35:00.591 INFO:tasks.workunit.client.1.smithi053.stdout:0/38: symlink d5/d8/l9 0 2024-09-21T04:35:00.592 INFO:tasks.workunit.client.1.smithi053.stdout:9/73: dwrite d0/d2/d9/fc [0,4194304] 0 2024-09-21T04:35:00.594 INFO:tasks.workunit.client.1.smithi053.stdout:3/65: unlink l4 0 2024-09-21T04:35:00.600 INFO:tasks.workunit.client.1.smithi053.stdout:7/46: dwrite d0/fb [0,4194304] 0 2024-09-21T04:35:00.602 INFO:tasks.workunit.client.1.smithi053.stdout:8/63: rename d3/l12 to d3/d5/l15 0 2024-09-21T04:35:00.605 INFO:tasks.workunit.client.1.smithi053.stdout:0/39: creat d5/fa x:0 0 0 2024-09-21T04:35:00.606 INFO:tasks.workunit.client.1.smithi053.stdout:7/47: fdatasync d0/f4 0 2024-09-21T04:35:00.606 INFO:tasks.workunit.client.1.smithi053.stdout:8/64: read d3/f9 [5365778,89516] 0 2024-09-21T04:35:00.608 INFO:tasks.workunit.client.1.smithi053.stdout:5/73: dwrite d0/f3 [0,4194304] 0 2024-09-21T04:35:00.609 INFO:tasks.workunit.client.1.smithi053.stdout:2/62: link d1/c3 d1/c15 0 2024-09-21T04:35:00.613 INFO:tasks.workunit.client.1.smithi053.stdout:2/63: write d1/f12 [2628153,18550] 0 2024-09-21T04:35:00.615 INFO:tasks.workunit.client.1.smithi053.stdout:9/74: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:35:00.623 INFO:tasks.workunit.client.1.smithi053.stdout:7/48: dwrite d0/fd [0,4194304] 0 2024-09-21T04:35:00.625 INFO:tasks.workunit.client.1.smithi053.stdout:5/74: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:35:00.626 INFO:tasks.workunit.client.1.smithi053.stdout:0/40: rename d5/d8 to d5/db 0 2024-09-21T04:35:00.628 INFO:tasks.workunit.client.1.smithi053.stdout:1/161: getdents d7/d9/da/d11 0 2024-09-21T04:35:00.628 INFO:tasks.workunit.client.1.smithi053.stdout:9/75: dread d0/f4 [0,4194304] 0 2024-09-21T04:35:00.633 INFO:tasks.workunit.client.1.smithi053.stdout:9/76: dread d0/f4 [0,4194304] 0 2024-09-21T04:35:00.633 INFO:tasks.workunit.client.1.smithi053.stdout:9/77: dread - d0/f15 zero size 2024-09-21T04:35:00.637 INFO:tasks.workunit.client.1.smithi053.stdout:8/65: mkdir d3/df/d14/d16 0 2024-09-21T04:35:00.638 INFO:tasks.workunit.client.1.smithi053.stdout:7/49: symlink d0/le 0 2024-09-21T04:35:00.641 INFO:tasks.workunit.client.1.smithi053.stdout:5/75: creat d0/d8/db/f16 x:0 0 0 2024-09-21T04:35:00.642 INFO:tasks.workunit.client.1.smithi053.stdout:0/41: mknod d5/db/cc 0 2024-09-21T04:35:00.645 INFO:tasks.workunit.client.1.smithi053.stdout:5/76: fsync d0/fe 0 2024-09-21T04:35:00.646 INFO:tasks.workunit.client.1.smithi053.stdout:4/70: dwrite d1/f3 [0,4194304] 0 2024-09-21T04:35:00.648 INFO:tasks.workunit.client.1.smithi053.stdout:5/77: readlink d0/l12 0 2024-09-21T04:35:00.648 INFO:tasks.workunit.client.1.smithi053.stdout:7/50: fdatasync d0/d6/f8 0 2024-09-21T04:35:00.652 INFO:tasks.workunit.client.1.smithi053.stdout:2/64: dwrite d1/fb [0,4194304] 0 2024-09-21T04:35:00.653 INFO:tasks.workunit.client.1.smithi053.stdout:4/71: write d1/f3 [1168711,60066] 0 2024-09-21T04:35:00.655 INFO:tasks.workunit.client.1.smithi053.stdout:7/51: write d0/f4 [362239,83254] 0 2024-09-21T04:35:00.656 INFO:tasks.workunit.client.1.smithi053.stdout:6/82: dwrite d2/d9/f11 [0,4194304] 0 2024-09-21T04:35:00.656 INFO:tasks.workunit.client.1.smithi053.stdout:0/42: mkdir d5/dd 0 2024-09-21T04:35:00.658 INFO:tasks.workunit.client.1.smithi053.stdout:9/78: creat d0/d1a/f1b x:0 0 0 2024-09-21T04:35:00.661 INFO:tasks.workunit.client.1.smithi053.stdout:8/66: mknod d3/df/d14/d16/c17 0 2024-09-21T04:35:00.661 INFO:tasks.workunit.client.1.smithi053.stdout:0/43: write d5/fa [647844,89908] 0 2024-09-21T04:35:00.664 INFO:tasks.workunit.client.1.smithi053.stdout:1/162: dwrite d7/d9/f19 [0,4194304] 0 2024-09-21T04:35:00.664 INFO:tasks.workunit.client.1.smithi053.stdout:5/78: creat d0/d8/f17 x:0 0 0 2024-09-21T04:35:00.667 INFO:tasks.workunit.client.1.smithi053.stdout:3/66: getdents . 0 2024-09-21T04:35:00.669 INFO:tasks.workunit.client.1.smithi053.stdout:3/67: truncate f11 898107 0 2024-09-21T04:35:00.673 INFO:tasks.workunit.client.1.smithi053.stdout:6/83: mknod d2/d9/c17 0 2024-09-21T04:35:00.675 INFO:tasks.workunit.client.1.smithi053.stdout:9/79: mkdir d0/d1c 0 2024-09-21T04:35:00.677 INFO:tasks.workunit.client.1.smithi053.stdout:0/44: mkdir d5/de 0 2024-09-21T04:35:00.680 INFO:tasks.workunit.client.1.smithi053.stdout:5/79: rename d0/d8/f13 to d0/d8/db/f18 0 2024-09-21T04:35:00.682 INFO:tasks.workunit.client.1.smithi053.stdout:6/84: dwrite d2/d9/de/f14 [0,4194304] 0 2024-09-21T04:35:00.683 INFO:tasks.workunit.client.1.smithi053.stdout:3/68: unlink l8 0 2024-09-21T04:35:00.684 INFO:tasks.workunit.client.1.smithi053.stdout:4/72: write d1/dd/f11 [544474,118708] 0 2024-09-21T04:35:00.687 INFO:tasks.workunit.client.1.smithi053.stdout:1/163: mknod d7/d24/d2a/c30 0 2024-09-21T04:35:00.691 INFO:tasks.workunit.client.1.smithi053.stdout:2/65: dwrite d1/f4 [0,4194304] 0 2024-09-21T04:35:00.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:00 smithi053 ceph-mon[35554]: pgmap v21: 65 pgs: 65 active+clean; 230 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 379 op/s 2024-09-21T04:35:00.693 INFO:tasks.workunit.client.1.smithi053.stdout:6/85: creat d2/d9/f18 x:0 0 0 2024-09-21T04:35:00.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:00 smithi007 ceph-mon[27954]: pgmap v21: 65 pgs: 65 active+clean; 230 MiB data, 878 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 379 op/s 2024-09-21T04:35:00.696 INFO:tasks.workunit.client.1.smithi053.stdout:8/67: rename d3/f7 to d3/f18 0 2024-09-21T04:35:00.697 INFO:tasks.workunit.client.1.smithi053.stdout:6/86: write d2/d9/f12 [191059,81359] 0 2024-09-21T04:35:00.700 INFO:tasks.workunit.client.1.smithi053.stdout:4/73: dwrite d1/f3 [4194304,4194304] 0 2024-09-21T04:35:00.701 INFO:tasks.workunit.client.1.smithi053.stdout:4/74: write d1/f3 [2552508,60086] 0 2024-09-21T04:35:00.708 INFO:tasks.workunit.client.1.smithi053.stdout:2/66: stat d1/c15 0 2024-09-21T04:35:00.709 INFO:tasks.workunit.client.1.smithi053.stdout:9/80: link d0/l3 d0/d1c/l1d 0 2024-09-21T04:35:00.710 INFO:tasks.workunit.client.1.smithi053.stdout:9/81: dread - d0/f15 zero size 2024-09-21T04:35:00.710 INFO:tasks.workunit.client.1.smithi053.stdout:9/82: stat d0/d2/fa 0 2024-09-21T04:35:00.711 INFO:tasks.workunit.client.1.smithi053.stdout:8/68: stat d3/f18 0 2024-09-21T04:35:00.722 INFO:tasks.workunit.client.1.smithi053.stdout:0/45: truncate f1 1526218 0 2024-09-21T04:35:00.723 INFO:tasks.workunit.client.1.smithi053.stdout:4/75: symlink d1/l13 0 2024-09-21T04:35:00.723 INFO:tasks.workunit.client.1.smithi053.stdout:1/164: symlink d7/d9/da/d11/d2b/l31 0 2024-09-21T04:35:00.724 INFO:tasks.workunit.client.1.smithi053.stdout:2/67: mknod d1/c16 0 2024-09-21T04:35:00.724 INFO:tasks.workunit.client.1.smithi053.stdout:6/87: link d2/f10 d2/d9/f19 0 2024-09-21T04:35:00.725 INFO:tasks.workunit.client.1.smithi053.stdout:2/68: fsync d1/f9 0 2024-09-21T04:35:00.730 INFO:tasks.workunit.client.1.smithi053.stdout:5/80: dwrite d0/f7 [0,4194304] 0 2024-09-21T04:35:00.732 INFO:tasks.workunit.client.1.smithi053.stdout:8/69: dwrite d3/f9 [8388608,4194304] 0 2024-09-21T04:35:00.733 INFO:tasks.workunit.client.1.smithi053.stdout:8/70: chown d3/d5/l8 236 1 2024-09-21T04:35:00.739 INFO:tasks.workunit.client.1.smithi053.stdout:8/71: dread d3/f9 [8388608,4194304] 0 2024-09-21T04:35:00.739 INFO:tasks.workunit.client.1.smithi053.stdout:8/72: stat d3 0 2024-09-21T04:35:00.740 INFO:tasks.workunit.client.1.smithi053.stdout:8/73: readlink d3/d5/l8 0 2024-09-21T04:35:00.741 INFO:tasks.workunit.client.1.smithi053.stdout:8/74: dread - d3/d5/f13 zero size 2024-09-21T04:35:00.743 INFO:tasks.workunit.client.1.smithi053.stdout:7/52: sync 2024-09-21T04:35:00.842 INFO:tasks.workunit.client.1.smithi053.stdout:4/76: dread d1/f4 [0,4194304] 0 2024-09-21T04:35:00.842 INFO:tasks.workunit.client.1.smithi053.stdout:3/69: rename lb to l12 0 2024-09-21T04:35:00.845 INFO:tasks.workunit.client.1.smithi053.stdout:9/83: creat d0/d1c/f1e x:0 0 0 2024-09-21T04:35:00.845 INFO:tasks.workunit.client.1.smithi053.stdout:9/84: dread - d0/d2/d9/f19 zero size 2024-09-21T04:35:00.846 INFO:tasks.workunit.client.1.smithi053.stdout:1/165: write d7/d9/da/d11/f26 [4158786,11930] 0 2024-09-21T04:35:00.847 INFO:tasks.workunit.client.1.smithi053.stdout:6/88: rmdir d2 39 2024-09-21T04:35:00.851 INFO:tasks.workunit.client.1.smithi053.stdout:9/85: dread d0/f4 [0,4194304] 0 2024-09-21T04:35:00.852 INFO:tasks.workunit.client.1.smithi053.stdout:5/81: creat d0/d8/db/f19 x:0 0 0 2024-09-21T04:35:00.853 INFO:tasks.workunit.client.1.smithi053.stdout:1/166: dwrite d7/d9/f19 [0,4194304] 0 2024-09-21T04:35:00.854 INFO:tasks.workunit.client.1.smithi053.stdout:1/167: chown d7/d9/l2d 541243171 1 2024-09-21T04:35:00.857 INFO:tasks.workunit.client.1.smithi053.stdout:7/53: creat d0/d6/ff x:0 0 0 2024-09-21T04:35:00.858 INFO:tasks.workunit.client.1.smithi053.stdout:7/54: read - d0/d6/f7 zero size 2024-09-21T04:35:00.859 INFO:tasks.workunit.client.1.smithi053.stdout:7/55: dread - d0/fa zero size 2024-09-21T04:35:00.859 INFO:tasks.workunit.client.1.smithi053.stdout:4/77: unlink d1/l2 0 2024-09-21T04:35:00.860 INFO:tasks.workunit.client.1.smithi053.stdout:0/46: link f3 d5/dd/ff 0 2024-09-21T04:35:00.863 INFO:tasks.workunit.client.1.smithi053.stdout:6/89: dread - d2/d9/f18 zero size 2024-09-21T04:35:00.869 INFO:tasks.workunit.client.1.smithi053.stdout:6/90: dwrite d2/d9/f11 [0,4194304] 0 2024-09-21T04:35:00.876 INFO:tasks.workunit.client.1.smithi053.stdout:9/86: dread d0/d2/d9/fb [0,4194304] 0 2024-09-21T04:35:00.877 INFO:tasks.workunit.client.1.smithi053.stdout:1/168: mkdir d7/d24/d2a/d32 0 2024-09-21T04:35:00.879 INFO:tasks.workunit.client.1.smithi053.stdout:8/75: rename d3/f9 to d3/d5/f19 0 2024-09-21T04:35:00.879 INFO:tasks.workunit.client.1.smithi053.stdout:1/169: chown d7/c1f 191 1 2024-09-21T04:35:00.881 INFO:tasks.workunit.client.1.smithi053.stdout:5/82: dwrite d0/d8/f10 [4194304,4194304] 0 2024-09-21T04:35:00.881 INFO:tasks.workunit.client.1.smithi053.stdout:7/56: creat d0/d6/f10 x:0 0 0 2024-09-21T04:35:00.882 INFO:tasks.workunit.client.1.smithi053.stdout:5/83: read - d0/d8/db/f18 zero size 2024-09-21T04:35:00.883 INFO:tasks.workunit.client.1.smithi053.stdout:5/84: read - d0/d8/f17 zero size 2024-09-21T04:35:00.886 INFO:tasks.workunit.client.1.smithi053.stdout:1/170: dread f1 [0,4194304] 0 2024-09-21T04:35:00.887 INFO:tasks.workunit.client.1.smithi053.stdout:1/171: truncate d7/d9/da/f2f 332731 0 2024-09-21T04:35:00.887 INFO:tasks.workunit.client.1.smithi053.stdout:6/91: mkdir d2/d9/d1a 0 2024-09-21T04:35:00.888 INFO:tasks.workunit.client.1.smithi053.stdout:1/172: chown d7/d9 15 1 2024-09-21T04:35:00.894 INFO:tasks.workunit.client.1.smithi053.stdout:8/76: mkdir d3/d5/d1a 0 2024-09-21T04:35:00.895 INFO:tasks.workunit.client.1.smithi053.stdout:1/173: dwrite f1 [0,4194304] 0 2024-09-21T04:35:00.896 INFO:tasks.workunit.client.1.smithi053.stdout:1/174: write d7/d9/f1b [342597,2513] 0 2024-09-21T04:35:00.897 INFO:tasks.workunit.client.1.smithi053.stdout:4/78: mkdir d1/dd/de/d14 0 2024-09-21T04:35:00.897 INFO:tasks.workunit.client.1.smithi053.stdout:7/57: creat d0/f11 x:0 0 0 2024-09-21T04:35:00.899 INFO:tasks.workunit.client.1.smithi053.stdout:4/79: chown d1/f3 41 1 2024-09-21T04:35:00.900 INFO:tasks.workunit.client.1.smithi053.stdout:5/85: mknod d0/c1a 0 2024-09-21T04:35:00.901 INFO:tasks.workunit.client.1.smithi053.stdout:6/92: mkdir d2/d1b 0 2024-09-21T04:35:00.905 INFO:tasks.workunit.client.1.smithi053.stdout:8/77: mknod d3/df/d14/d16/c1b 0 2024-09-21T04:35:00.909 INFO:tasks.workunit.client.1.smithi053.stdout:1/175: creat d7/d9/da/d11/f33 x:0 0 0 2024-09-21T04:35:00.912 INFO:tasks.workunit.client.1.smithi053.stdout:7/58: symlink d0/d6/l12 0 2024-09-21T04:35:00.913 INFO:tasks.workunit.client.1.smithi053.stdout:4/80: rmdir d1/dd/de 39 2024-09-21T04:35:00.915 INFO:tasks.workunit.client.1.smithi053.stdout:5/86: rmdir d0 39 2024-09-21T04:35:00.917 INFO:tasks.workunit.client.1.smithi053.stdout:6/93: rename d2/d9/f12 to d2/d1b/f1c 0 2024-09-21T04:35:00.919 INFO:tasks.workunit.client.1.smithi053.stdout:6/94: write d2/f3 [979150,24244] 0 2024-09-21T04:35:00.920 INFO:tasks.workunit.client.1.smithi053.stdout:0/47: getdents d5/db 0 2024-09-21T04:35:00.922 INFO:tasks.workunit.client.1.smithi053.stdout:7/59: creat d0/f13 x:0 0 0 2024-09-21T04:35:00.929 INFO:tasks.workunit.client.1.smithi053.stdout:6/95: mkdir d2/d1d 0 2024-09-21T04:35:00.934 INFO:tasks.workunit.client.1.smithi053.stdout:6/96: mkdir d2/d1b/d1e 0 2024-09-21T04:35:00.935 INFO:tasks.workunit.client.1.smithi053.stdout:1/176: rmdir d7/d24/d2a/d32 0 2024-09-21T04:35:00.936 INFO:tasks.workunit.client.1.smithi053.stdout:4/81: mknod d1/dd/de/d14/c15 0 2024-09-21T04:35:00.942 INFO:tasks.workunit.client.1.smithi053.stdout:1/177: dwrite d7/d9/da/d11/f26 [0,4194304] 0 2024-09-21T04:35:00.942 INFO:tasks.workunit.client.1.smithi053.stdout:1/178: stat d7/d9/c27 0 2024-09-21T04:35:00.944 INFO:tasks.workunit.client.1.smithi053.stdout:4/82: mknod d1/dd/c16 0 2024-09-21T04:35:00.977 INFO:tasks.workunit.client.1.smithi053.stdout:2/69: truncate d1/f5 1462250 0 2024-09-21T04:35:00.978 INFO:tasks.workunit.client.1.smithi053.stdout:2/70: write d1/f9 [3487018,35643] 0 2024-09-21T04:35:00.980 INFO:tasks.workunit.client.1.smithi053.stdout:2/71: fsync d1/f9 0 2024-09-21T04:35:00.981 INFO:tasks.workunit.client.1.smithi053.stdout:2/72: chown d1/fb 21761 1 2024-09-21T04:35:00.983 INFO:tasks.workunit.client.1.smithi053.stdout:2/73: rename d1/le to d1/l17 0 2024-09-21T04:35:00.985 INFO:tasks.workunit.client.1.smithi053.stdout:3/70: sync 2024-09-21T04:35:00.985 INFO:tasks.workunit.client.1.smithi053.stdout:2/74: mkdir d1/d18 0 2024-09-21T04:35:00.986 INFO:tasks.workunit.client.1.smithi053.stdout:2/75: write d1/fb [3565867,97283] 0 2024-09-21T04:35:00.990 INFO:tasks.workunit.client.1.smithi053.stdout:2/76: dread d1/f4 [0,4194304] 0 2024-09-21T04:35:00.991 INFO:tasks.workunit.client.1.smithi053.stdout:8/78: truncate d3/f18 19691 0 2024-09-21T04:35:00.995 INFO:tasks.workunit.client.1.smithi053.stdout:3/71: rename l12 to l13 0 2024-09-21T04:35:00.996 INFO:tasks.workunit.client.1.smithi053.stdout:8/79: mknod d3/d5/c1c 0 2024-09-21T04:35:00.997 INFO:tasks.workunit.client.1.smithi053.stdout:8/80: write d3/df/f11 [1048884,109498] 0 2024-09-21T04:35:00.998 INFO:tasks.workunit.client.1.smithi053.stdout:8/81: chown d3/d5/d1a 301480478 1 2024-09-21T04:35:00.998 INFO:tasks.workunit.client.1.smithi053.stdout:3/72: write f10 [108362,85317] 0 2024-09-21T04:35:01.001 INFO:tasks.workunit.client.1.smithi053.stdout:9/87: write d0/d2/d9/fb [2622061,126054] 0 2024-09-21T04:35:01.001 INFO:tasks.workunit.client.1.smithi053.stdout:7/60: sync 2024-09-21T04:35:01.003 INFO:tasks.workunit.client.1.smithi053.stdout:7/61: dread - d0/d6/f7 zero size 2024-09-21T04:35:01.003 INFO:tasks.workunit.client.1.smithi053.stdout:9/88: dread - d0/d1a/f1b zero size 2024-09-21T04:35:01.003 INFO:tasks.workunit.client.1.smithi053.stdout:8/82: creat d3/df/d14/d16/f1d x:0 0 0 2024-09-21T04:35:01.006 INFO:tasks.workunit.client.1.smithi053.stdout:7/62: dread d0/d6/f9 [0,4194304] 0 2024-09-21T04:35:01.007 INFO:tasks.workunit.client.1.smithi053.stdout:9/89: unlink d0/d1c/f1e 0 2024-09-21T04:35:01.009 INFO:tasks.workunit.client.1.smithi053.stdout:8/83: unlink d3/c6 0 2024-09-21T04:35:01.013 INFO:tasks.workunit.client.1.smithi053.stdout:7/63: mkdir d0/d6/d14 0 2024-09-21T04:35:01.013 INFO:tasks.workunit.client.1.smithi053.stdout:7/64: fsync d0/f11 0 2024-09-21T04:35:01.016 INFO:tasks.workunit.client.1.smithi053.stdout:5/87: getdents d0 0 2024-09-21T04:35:01.016 INFO:tasks.workunit.client.1.smithi053.stdout:3/73: link f7 f14 0 2024-09-21T04:35:01.018 INFO:tasks.workunit.client.1.smithi053.stdout:7/65: creat d0/f15 x:0 0 0 2024-09-21T04:35:01.019 INFO:tasks.workunit.client.1.smithi053.stdout:7/66: write d0/f13 [47552,19727] 0 2024-09-21T04:35:01.019 INFO:tasks.workunit.client.1.smithi053.stdout:7/67: chown d0 501761 1 2024-09-21T04:35:01.020 INFO:tasks.workunit.client.1.smithi053.stdout:9/90: mkdir d0/d2/d9/d1f 0 2024-09-21T04:35:01.020 INFO:tasks.workunit.client.1.smithi053.stdout:9/91: chown d0 5580101 1 2024-09-21T04:35:01.023 INFO:tasks.workunit.client.1.smithi053.stdout:5/88: symlink d0/l1b 0 2024-09-21T04:35:01.025 INFO:tasks.workunit.client.1.smithi053.stdout:7/68: creat d0/f16 x:0 0 0 2024-09-21T04:35:01.027 INFO:tasks.workunit.client.1.smithi053.stdout:5/89: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:35:01.032 INFO:tasks.workunit.client.1.smithi053.stdout:7/69: dwrite d0/f16 [0,4194304] 0 2024-09-21T04:35:01.032 INFO:tasks.workunit.client.1.smithi053.stdout:5/90: fsync d0/d8/db/f18 0 2024-09-21T04:35:01.036 INFO:tasks.workunit.client.1.smithi053.stdout:3/74: creat f15 x:0 0 0 2024-09-21T04:35:01.040 INFO:tasks.workunit.client.1.smithi053.stdout:7/70: symlink d0/l17 0 2024-09-21T04:35:01.040 INFO:tasks.workunit.client.1.smithi053.stdout:7/71: fdatasync d0/f11 0 2024-09-21T04:35:01.041 INFO:tasks.workunit.client.1.smithi053.stdout:7/72: fdatasync d0/d6/f10 0 2024-09-21T04:35:01.042 INFO:tasks.workunit.client.1.smithi053.stdout:7/73: dread - d0/f15 zero size 2024-09-21T04:35:01.045 INFO:tasks.workunit.client.1.smithi053.stdout:5/91: symlink d0/dd/l1c 0 2024-09-21T04:35:01.045 INFO:tasks.workunit.client.1.smithi053.stdout:6/97: truncate d2/d9/de/f15 1262591 0 2024-09-21T04:35:01.045 INFO:tasks.workunit.client.1.smithi053.stdout:5/92: stat d0/d8/f10 0 2024-09-21T04:35:01.045 INFO:tasks.workunit.client.1.smithi053.stdout:6/98: chown d2/f16 284456424 1 2024-09-21T04:35:01.046 INFO:tasks.workunit.client.1.smithi053.stdout:7/74: creat d0/d6/f18 x:0 0 0 2024-09-21T04:35:01.047 INFO:tasks.workunit.client.1.smithi053.stdout:3/75: mkdir d16 0 2024-09-21T04:35:01.049 INFO:tasks.workunit.client.1.smithi053.stdout:3/76: truncate f14 773155 0 2024-09-21T04:35:01.049 INFO:tasks.workunit.client.1.smithi053.stdout:4/83: write d1/f4 [714164,77755] 0 2024-09-21T04:35:01.051 INFO:tasks.workunit.client.1.smithi053.stdout:4/84: readlink d1/l9 0 2024-09-21T04:35:01.057 INFO:tasks.workunit.client.1.smithi053.stdout:4/85: mknod d1/c17 0 2024-09-21T04:35:01.058 INFO:tasks.workunit.client.1.smithi053.stdout:5/93: dwrite d0/d8/f17 [0,4194304] 0 2024-09-21T04:35:01.058 INFO:tasks.workunit.client.1.smithi053.stdout:1/179: dwrite d7/d9/da/d11/f20 [0,4194304] 0 2024-09-21T04:35:01.059 INFO:tasks.workunit.client.1.smithi053.stdout:7/75: rmdir d0/d6/d14 0 2024-09-21T04:35:01.063 INFO:tasks.workunit.client.1.smithi053.stdout:3/77: mkdir d16/d17 0 2024-09-21T04:35:01.064 INFO:tasks.workunit.client.1.smithi053.stdout:5/94: read d0/d8/f17 [1748845,129141] 0 2024-09-21T04:35:01.064 INFO:tasks.workunit.client.1.smithi053.stdout:6/99: dwrite d2/d1b/f1c [0,4194304] 0 2024-09-21T04:35:01.068 INFO:tasks.workunit.client.1.smithi053.stdout:7/76: creat d0/f19 x:0 0 0 2024-09-21T04:35:01.069 INFO:tasks.workunit.client.1.smithi053.stdout:0/48: dwrite f1 [0,4194304] 0 2024-09-21T04:35:01.071 INFO:tasks.workunit.client.1.smithi053.stdout:4/86: dread d1/fc [0,4194304] 0 2024-09-21T04:35:01.075 INFO:tasks.workunit.client.1.smithi053.stdout:1/180: mknod d7/d9/da/d11/c34 0 2024-09-21T04:35:01.078 INFO:tasks.workunit.client.1.smithi053.stdout:0/49: dwrite f1 [0,4194304] 0 2024-09-21T04:35:01.084 INFO:tasks.workunit.client.1.smithi053.stdout:5/95: creat d0/d8/f1d x:0 0 0 2024-09-21T04:35:01.084 INFO:tasks.workunit.client.1.smithi053.stdout:5/96: chown d0/f14 59 1 2024-09-21T04:35:01.086 INFO:tasks.workunit.client.1.smithi053.stdout:2/77: rename d1/l17 to d1/d18/l19 0 2024-09-21T04:35:01.086 INFO:tasks.workunit.client.1.smithi053.stdout:5/97: write d0/d8/db/f18 [322480,106983] 0 2024-09-21T04:35:01.087 INFO:tasks.workunit.client.1.smithi053.stdout:3/78: rename d16 to d16/d18 22 2024-09-21T04:35:01.088 INFO:tasks.workunit.client.1.smithi053.stdout:0/50: dread f3 [0,4194304] 0 2024-09-21T04:35:01.088 INFO:tasks.workunit.client.1.smithi053.stdout:7/77: mknod d0/c1a 0 2024-09-21T04:35:01.091 INFO:tasks.workunit.client.1.smithi053.stdout:0/51: dread f3 [0,4194304] 0 2024-09-21T04:35:01.095 INFO:tasks.workunit.client.1.smithi053.stdout:1/181: unlink d7/d9/da/d11/c34 0 2024-09-21T04:35:01.098 INFO:tasks.workunit.client.1.smithi053.stdout:1/182: dread f1 [0,4194304] 0 2024-09-21T04:35:01.100 INFO:tasks.workunit.client.1.smithi053.stdout:1/183: write d7/d9/da/d11/f20 [3817088,18594] 0 2024-09-21T04:35:01.102 INFO:tasks.workunit.client.1.smithi053.stdout:3/79: mknod d16/c19 0 2024-09-21T04:35:01.103 INFO:tasks.workunit.client.1.smithi053.stdout:1/184: dread d7/d9/f19 [0,4194304] 0 2024-09-21T04:35:01.105 INFO:tasks.workunit.client.1.smithi053.stdout:6/100: rename d2/f16 to d2/d9/de/f1f 0 2024-09-21T04:35:01.105 INFO:tasks.workunit.client.1.smithi053.stdout:8/84: truncate d3/fa 710959 0 2024-09-21T04:35:01.106 INFO:tasks.workunit.client.1.smithi053.stdout:1/185: dread d7/d9/f1b [0,4194304] 0 2024-09-21T04:35:01.116 INFO:tasks.workunit.client.1.smithi053.stdout:0/52: mkdir d5/d10 0 2024-09-21T04:35:01.118 INFO:tasks.workunit.client.1.smithi053.stdout:9/92: dwrite d0/d2/fd [0,4194304] 0 2024-09-21T04:35:01.127 INFO:tasks.workunit.client.1.smithi053.stdout:3/80: stat fa 0 2024-09-21T04:35:01.131 INFO:tasks.workunit.client.1.smithi053.stdout:6/101: fsync d2/d9/f19 0 2024-09-21T04:35:01.135 INFO:tasks.workunit.client.1.smithi053.stdout:1/186: symlink d7/d9/da/d11/l35 0 2024-09-21T04:35:01.143 INFO:tasks.workunit.client.1.smithi053.stdout:0/53: mkdir d5/d11 0 2024-09-21T04:35:01.144 INFO:tasks.workunit.client.1.smithi053.stdout:4/87: link d1/dd/c16 d1/dd/de/c18 0 2024-09-21T04:35:01.151 INFO:tasks.workunit.client.1.smithi053.stdout:4/88: dwrite d1/f3 [0,4194304] 0 2024-09-21T04:35:01.155 INFO:tasks.workunit.client.1.smithi053.stdout:3/81: symlink d16/l1a 0 2024-09-21T04:35:01.164 INFO:tasks.workunit.client.1.smithi053.stdout:6/102: creat d2/d9/f20 x:0 0 0 2024-09-21T04:35:01.164 INFO:tasks.workunit.client.1.smithi053.stdout:6/103: read d2/fb [2257772,18755] 0 2024-09-21T04:35:01.165 INFO:tasks.workunit.client.1.smithi053.stdout:6/104: read d2/f10 [1665358,92326] 0 2024-09-21T04:35:01.166 INFO:tasks.workunit.client.1.smithi053.stdout:1/187: rmdir d7/d9/da/d11/d2b 39 2024-09-21T04:35:01.171 INFO:tasks.workunit.client.1.smithi053.stdout:0/54: rename d5/db/cc to d5/de/c12 0 2024-09-21T04:35:01.176 INFO:tasks.workunit.client.1.smithi053.stdout:5/98: getdents d0 0 2024-09-21T04:35:01.176 INFO:tasks.workunit.client.1.smithi053.stdout:5/99: chown d0/l1b 3 1 2024-09-21T04:35:01.177 INFO:tasks.workunit.client.1.smithi053.stdout:5/100: chown d0/d8/db/f19 245 1 2024-09-21T04:35:01.179 INFO:tasks.workunit.client.1.smithi053.stdout:3/82: fdatasync f9 0 2024-09-21T04:35:01.185 INFO:tasks.workunit.client.1.smithi053.stdout:2/78: getdents d1/d18 0 2024-09-21T04:35:01.187 INFO:tasks.workunit.client.1.smithi053.stdout:7/78: getdents d0 0 2024-09-21T04:35:01.189 INFO:tasks.workunit.client.1.smithi053.stdout:8/85: chown d3/fa 9336 1 2024-09-21T04:35:01.191 INFO:tasks.workunit.client.1.smithi053.stdout:0/55: rename d5/d11 to d5/dd/d13 0 2024-09-21T04:35:01.192 INFO:tasks.workunit.client.1.smithi053.stdout:9/93: link d0/df/l11 d0/d1c/l20 0 2024-09-21T04:35:01.194 INFO:tasks.workunit.client.1.smithi053.stdout:7/79: dwrite d0/d6/f18 [0,4194304] 0 2024-09-21T04:35:01.194 INFO:tasks.workunit.client.1.smithi053.stdout:9/94: dread - d0/d1a/f1b zero size 2024-09-21T04:35:01.196 INFO:tasks.workunit.client.1.smithi053.stdout:5/101: unlink d0/f3 0 2024-09-21T04:35:01.198 INFO:tasks.workunit.client.1.smithi053.stdout:9/95: truncate d0/d2/d9/f19 258786 0 2024-09-21T04:35:01.200 INFO:tasks.workunit.client.1.smithi053.stdout:0/56: dwrite f1 [0,4194304] 0 2024-09-21T04:35:01.200 INFO:tasks.workunit.client.1.smithi053.stdout:3/83: mkdir d16/d1b 0 2024-09-21T04:35:01.200 INFO:tasks.workunit.client.1.smithi053.stdout:3/84: stat f9 0 2024-09-21T04:35:01.201 INFO:tasks.workunit.client.1.smithi053.stdout:3/85: write fa [4592048,72762] 0 2024-09-21T04:35:01.202 INFO:tasks.workunit.client.1.smithi053.stdout:3/86: write f15 [552360,55264] 0 2024-09-21T04:35:01.207 INFO:tasks.workunit.client.1.smithi053.stdout:2/79: mknod d1/d18/c1a 0 2024-09-21T04:35:01.216 INFO:tasks.workunit.client.1.smithi053.stdout:7/80: rmdir d0/d6 39 2024-09-21T04:35:01.217 INFO:tasks.workunit.client.1.smithi053.stdout:4/89: truncate d1/f4 214909 0 2024-09-21T04:35:01.219 INFO:tasks.workunit.client.1.smithi053.stdout:5/102: symlink d0/dd/l1e 0 2024-09-21T04:35:01.221 INFO:tasks.workunit.client.1.smithi053.stdout:9/96: rmdir d0/df 39 2024-09-21T04:35:01.223 INFO:tasks.workunit.client.1.smithi053.stdout:0/57: truncate d5/dd/ff 3443832 0 2024-09-21T04:35:01.224 INFO:tasks.workunit.client.1.smithi053.stdout:1/188: truncate f5 3928720 0 2024-09-21T04:35:01.225 INFO:tasks.workunit.client.1.smithi053.stdout:5/103: dwrite d0/f7 [0,4194304] 0 2024-09-21T04:35:01.227 INFO:tasks.workunit.client.1.smithi053.stdout:5/104: write d0/d8/db/f16 [192123,77732] 0 2024-09-21T04:35:01.229 INFO:tasks.workunit.client.1.smithi053.stdout:3/87: rename f10 to d16/f1c 0 2024-09-21T04:35:01.231 INFO:tasks.workunit.client.1.smithi053.stdout:8/86: truncate d3/f18 295751 0 2024-09-21T04:35:01.233 INFO:tasks.workunit.client.1.smithi053.stdout:7/81: chown d0/d6/f9 16343 1 2024-09-21T04:35:01.236 INFO:tasks.workunit.client.1.smithi053.stdout:9/97: mkdir d0/d1c/d21 0 2024-09-21T04:35:01.237 INFO:tasks.workunit.client.1.smithi053.stdout:0/58: symlink d5/dd/l14 0 2024-09-21T04:35:01.238 INFO:tasks.workunit.client.1.smithi053.stdout:1/189: mkdir d7/d24/d2a/d36 0 2024-09-21T04:35:01.242 INFO:tasks.workunit.client.1.smithi053.stdout:9/98: write d0/f15 [1047084,79472] 0 2024-09-21T04:35:01.243 INFO:tasks.workunit.client.1.smithi053.stdout:5/105: creat d0/d8/f1f x:0 0 0 2024-09-21T04:35:01.244 INFO:tasks.workunit.client.1.smithi053.stdout:6/105: dwrite d2/d9/f11 [0,4194304] 0 2024-09-21T04:35:01.246 INFO:tasks.workunit.client.1.smithi053.stdout:3/88: dread - fd zero size 2024-09-21T04:35:01.246 INFO:tasks.workunit.client.1.smithi053.stdout:7/82: dwrite d0/f16 [0,4194304] 0 2024-09-21T04:35:01.247 INFO:tasks.workunit.client.1.smithi053.stdout:5/106: chown d0/c4 62 1 2024-09-21T04:35:01.249 INFO:tasks.workunit.client.1.smithi053.stdout:8/87: rmdir d3/df 39 2024-09-21T04:35:01.256 INFO:tasks.workunit.client.1.smithi053.stdout:7/83: dwrite d0/fd [0,4194304] 0 2024-09-21T04:35:01.261 INFO:tasks.workunit.client.1.smithi053.stdout:6/106: unlink d2/f10 0 2024-09-21T04:35:01.261 INFO:tasks.workunit.client.1.smithi053.stdout:7/84: dread d0/d6/f9 [0,4194304] 0 2024-09-21T04:35:01.262 INFO:tasks.workunit.client.1.smithi053.stdout:7/85: truncate d0/f15 164376 0 2024-09-21T04:35:01.263 INFO:tasks.workunit.client.1.smithi053.stdout:7/86: dread - d0/f11 zero size 2024-09-21T04:35:01.265 INFO:tasks.workunit.client.1.smithi053.stdout:3/89: read - fe zero size 2024-09-21T04:35:01.268 INFO:tasks.workunit.client.1.smithi053.stdout:9/99: rename d0/d2/l10 to d0/d1a/l22 0 2024-09-21T04:35:01.271 INFO:tasks.workunit.client.1.smithi053.stdout:5/107: symlink d0/d8/db/l20 0 2024-09-21T04:35:01.275 INFO:tasks.workunit.client.1.smithi053.stdout:0/59: fsync d5/dd/ff 0 2024-09-21T04:35:01.276 INFO:tasks.workunit.client.1.smithi053.stdout:2/80: truncate d1/f4 4090307 0 2024-09-21T04:35:01.277 INFO:tasks.workunit.client.1.smithi053.stdout:2/81: readlink d1/l13 0 2024-09-21T04:35:01.278 INFO:tasks.workunit.client.1.smithi053.stdout:1/190: mknod d7/d24/d2a/d36/c37 0 2024-09-21T04:35:01.279 INFO:tasks.workunit.client.1.smithi053.stdout:5/108: dwrite d0/d8/db/f15 [0,4194304] 0 2024-09-21T04:35:01.280 INFO:tasks.workunit.client.1.smithi053.stdout:5/109: write d0/f7 [1038078,56570] 0 2024-09-21T04:35:01.283 INFO:tasks.workunit.client.1.smithi053.stdout:5/110: chown d0/c1a 889 1 2024-09-21T04:35:01.283 INFO:tasks.workunit.client.1.smithi053.stdout:7/87: creat d0/d6/f1b x:0 0 0 2024-09-21T04:35:01.285 INFO:tasks.workunit.client.1.smithi053.stdout:5/111: dread - d0/fe zero size 2024-09-21T04:35:01.286 INFO:tasks.workunit.client.1.smithi053.stdout:3/90: creat d16/f1d x:0 0 0 2024-09-21T04:35:01.287 INFO:tasks.workunit.client.1.smithi053.stdout:7/88: dread d0/f15 [0,4194304] 0 2024-09-21T04:35:01.287 INFO:tasks.workunit.client.1.smithi053.stdout:5/112: readlink d0/l1b 0 2024-09-21T04:35:01.289 INFO:tasks.workunit.client.1.smithi053.stdout:5/113: chown d0/f14 187928 1 2024-09-21T04:35:01.290 INFO:tasks.workunit.client.1.smithi053.stdout:7/89: write d0/fb [540399,6748] 0 2024-09-21T04:35:01.290 INFO:tasks.workunit.client.1.smithi053.stdout:1/191: dwrite d7/d9/da/d11/f26 [4194304,4194304] 0 2024-09-21T04:35:01.292 INFO:tasks.workunit.client.1.smithi053.stdout:0/60: creat d5/f15 x:0 0 0 2024-09-21T04:35:01.295 INFO:tasks.workunit.client.1.smithi053.stdout:5/114: dread d0/d8/db/f18 [0,4194304] 0 2024-09-21T04:35:01.297 INFO:tasks.workunit.client.1.smithi053.stdout:2/82: mknod d1/c1b 0 2024-09-21T04:35:01.298 INFO:tasks.workunit.client.1.smithi053.stdout:5/115: write d0/fe [755236,17942] 0 2024-09-21T04:35:01.300 INFO:tasks.workunit.client.1.smithi053.stdout:7/90: dread d0/f4 [0,4194304] 0 2024-09-21T04:35:01.301 INFO:tasks.workunit.client.1.smithi053.stdout:5/116: chown d0/d8/db/f15 47397467 1 2024-09-21T04:35:01.303 INFO:tasks.workunit.client.1.smithi053.stdout:0/61: dwrite d5/fa [0,4194304] 0 2024-09-21T04:35:01.306 INFO:tasks.workunit.client.1.smithi053.stdout:7/91: dread d0/fd [0,4194304] 0 2024-09-21T04:35:01.309 INFO:tasks.workunit.client.1.smithi053.stdout:9/100: getdents d0/d2/d9/d1f 0 2024-09-21T04:35:01.313 INFO:tasks.workunit.client.1.smithi053.stdout:8/88: truncate d3/d5/f19 987181 0 2024-09-21T04:35:01.318 INFO:tasks.workunit.client.1.smithi053.stdout:2/83: creat d1/d18/f1c x:0 0 0 2024-09-21T04:35:01.321 INFO:tasks.workunit.client.1.smithi053.stdout:5/117: symlink d0/d8/db/l21 0 2024-09-21T04:35:01.328 INFO:tasks.workunit.client.1.smithi053.stdout:3/91: creat d16/d17/f1e x:0 0 0 2024-09-21T04:35:01.329 INFO:tasks.workunit.client.1.smithi053.stdout:3/92: truncate fa 5266556 0 2024-09-21T04:35:01.329 INFO:tasks.workunit.client.1.smithi053.stdout:3/93: chown f7 8450 1 2024-09-21T04:35:01.329 INFO:tasks.workunit.client.1.smithi053.stdout:3/94: stat d16/f1c 0 2024-09-21T04:35:01.336 INFO:tasks.workunit.client.1.smithi053.stdout:6/107: link d2/d9/f19 d2/d9/d1a/f21 0 2024-09-21T04:35:01.336 INFO:tasks.workunit.client.1.smithi053.stdout:6/108: chown f1 1139 1 2024-09-21T04:35:01.341 INFO:tasks.workunit.client.1.smithi053.stdout:5/118: mknod d0/dd/c22 0 2024-09-21T04:35:01.341 INFO:tasks.workunit.client.1.smithi053.stdout:5/119: truncate d0/d8/f1f 255127 0 2024-09-21T04:35:01.344 INFO:tasks.workunit.client.1.smithi053.stdout:0/62: creat d5/d6/f16 x:0 0 0 2024-09-21T04:35:01.347 INFO:tasks.workunit.client.1.smithi053.stdout:7/92: link d0/d6/f9 d0/d6/f1c 0 2024-09-21T04:35:01.349 INFO:tasks.workunit.client.1.smithi053.stdout:9/101: creat d0/df/f23 x:0 0 0 2024-09-21T04:35:01.351 INFO:tasks.workunit.client.1.smithi053.stdout:8/89: fsync d3/fa 0 2024-09-21T04:35:01.352 INFO:tasks.workunit.client.1.smithi053.stdout:9/102: stat d0/d2/d9/d1f 0 2024-09-21T04:35:01.353 INFO:tasks.workunit.client.1.smithi053.stdout:3/95: rename f7 to d16/d1b/f1f 0 2024-09-21T04:35:01.354 INFO:tasks.workunit.client.1.smithi053.stdout:0/63: dwrite d5/f15 [0,4194304] 0 2024-09-21T04:35:01.354 INFO:tasks.workunit.client.1.smithi053.stdout:5/120: unlink d0/d8/db/f16 0 2024-09-21T04:35:01.357 INFO:tasks.workunit.client.1.smithi053.stdout:2/84: fdatasync d1/f4 0 2024-09-21T04:35:01.358 INFO:tasks.workunit.client.1.smithi053.stdout:7/93: dwrite d0/fa [0,4194304] 0 2024-09-21T04:35:01.359 INFO:tasks.workunit.client.1.smithi053.stdout:7/94: readlink d0/d6/l12 0 2024-09-21T04:35:01.361 INFO:tasks.workunit.client.1.smithi053.stdout:8/90: chown d3/f18 121 1 2024-09-21T04:35:01.370 INFO:tasks.workunit.client.1.smithi053.stdout:3/96: dwrite fc [0,4194304] 0 2024-09-21T04:35:01.370 INFO:tasks.workunit.client.1.smithi053.stdout:0/64: unlink f3 0 2024-09-21T04:35:01.377 INFO:tasks.workunit.client.1.smithi053.stdout:7/95: rename d0/cc to d0/d6/c1d 0 2024-09-21T04:35:01.378 INFO:tasks.workunit.client.1.smithi053.stdout:8/91: readlink d3/d5/l15 0 2024-09-21T04:35:01.379 INFO:tasks.workunit.client.1.smithi053.stdout:7/96: dread d0/f13 [0,4194304] 0 2024-09-21T04:35:01.380 INFO:tasks.workunit.client.1.smithi053.stdout:8/92: write d3/d5/f13 [5371,68242] 0 2024-09-21T04:35:01.380 INFO:tasks.workunit.client.1.smithi053.stdout:9/103: getdents d0/d1c/d21 0 2024-09-21T04:35:01.382 INFO:tasks.workunit.client.1.smithi053.stdout:7/97: dread d0/f13 [0,4194304] 0 2024-09-21T04:35:01.383 INFO:tasks.workunit.client.1.smithi053.stdout:5/121: link d0/f14 d0/dd/f23 0 2024-09-21T04:35:01.384 INFO:tasks.workunit.client.1.smithi053.stdout:5/122: dread d0/d8/f1f [0,4194304] 0 2024-09-21T04:35:01.387 INFO:tasks.workunit.client.1.smithi053.stdout:5/123: dread d0/d8/db/f15 [0,4194304] 0 2024-09-21T04:35:01.390 INFO:tasks.workunit.client.1.smithi053.stdout:7/98: creat d0/d6/f1e x:0 0 0 2024-09-21T04:35:01.390 INFO:tasks.workunit.client.1.smithi053.stdout:7/99: read d0/fa [4003479,70751] 0 2024-09-21T04:35:01.398 INFO:tasks.workunit.client.1.smithi053.stdout:5/124: unlink d0/d8/db/l21 0 2024-09-21T04:35:01.403 INFO:tasks.workunit.client.1.smithi053.stdout:7/100: unlink d0/c1a 0 2024-09-21T04:35:01.406 INFO:tasks.workunit.client.1.smithi053.stdout:2/85: getdents d1 0 2024-09-21T04:35:01.407 INFO:tasks.workunit.client.1.smithi053.stdout:9/104: creat d0/f24 x:0 0 0 2024-09-21T04:35:01.407 INFO:tasks.workunit.client.1.smithi053.stdout:9/105: write d0/d2/fd [2785697,67840] 0 2024-09-21T04:35:01.409 INFO:tasks.workunit.client.1.smithi053.stdout:3/97: truncate f11 384157 0 2024-09-21T04:35:01.409 INFO:tasks.workunit.client.1.smithi053.stdout:3/98: chown d16 292 1 2024-09-21T04:35:01.410 INFO:tasks.workunit.client.1.smithi053.stdout:8/93: link d3/d5/l8 d3/d5/l1e 0 2024-09-21T04:35:01.413 INFO:tasks.workunit.client.1.smithi053.stdout:7/101: symlink d0/d6/l1f 0 2024-09-21T04:35:01.413 INFO:tasks.workunit.client.1.smithi053.stdout:8/94: dread d3/d5/f19 [0,4194304] 0 2024-09-21T04:35:01.415 INFO:tasks.workunit.client.1.smithi053.stdout:9/106: unlink d0/d2/fd 0 2024-09-21T04:35:01.416 INFO:tasks.workunit.client.1.smithi053.stdout:3/99: mknod d16/d17/c20 0 2024-09-21T04:35:01.420 INFO:tasks.workunit.client.1.smithi053.stdout:5/125: dwrite d0/d8/f1f [0,4194304] 0 2024-09-21T04:35:01.420 INFO:tasks.workunit.client.1.smithi053.stdout:4/90: sync 2024-09-21T04:35:01.423 INFO:tasks.workunit.client.1.smithi053.stdout:8/95: truncate d3/fa 1002102 0 2024-09-21T04:35:01.424 INFO:tasks.workunit.client.1.smithi053.stdout:7/102: dwrite d0/d6/f8 [0,4194304] 0 2024-09-21T04:35:01.425 INFO:tasks.workunit.client.1.smithi053.stdout:9/107: mkdir d0/d1c/d25 0 2024-09-21T04:35:01.426 INFO:tasks.workunit.client.1.smithi053.stdout:3/100: mkdir d16/d17/d21 0 2024-09-21T04:35:01.427 INFO:tasks.workunit.client.1.smithi053.stdout:7/103: write d0/d6/f10 [423216,108657] 0 2024-09-21T04:35:01.430 INFO:tasks.workunit.client.1.smithi053.stdout:7/104: write d0/f19 [309574,120974] 0 2024-09-21T04:35:01.433 INFO:tasks.workunit.client.1.smithi053.stdout:5/126: rename d0/d8/db/f18 to d0/dd/f24 0 2024-09-21T04:35:01.434 INFO:tasks.workunit.client.1.smithi053.stdout:9/108: fdatasync d0/d2/fe 0 2024-09-21T04:35:01.435 INFO:tasks.workunit.client.1.smithi053.stdout:3/101: symlink d16/d17/d21/l22 0 2024-09-21T04:35:01.435 INFO:tasks.workunit.client.1.smithi053.stdout:5/127: dread - d0/d8/f1d zero size 2024-09-21T04:35:01.437 INFO:tasks.workunit.client.1.smithi053.stdout:2/86: dwrite d1/f4 [0,4194304] 0 2024-09-21T04:35:01.437 INFO:tasks.workunit.client.1.smithi053.stdout:4/91: dread d1/fa [0,4194304] 0 2024-09-21T04:35:01.437 INFO:tasks.workunit.client.1.smithi053.stdout:5/128: chown d0/d8/f10 261899 1 2024-09-21T04:35:01.437 INFO:tasks.workunit.client.1.smithi053.stdout:8/96: rmdir d3/df/d14/d16 39 2024-09-21T04:35:01.438 INFO:tasks.workunit.client.1.smithi053.stdout:5/129: stat d0/dd 0 2024-09-21T04:35:01.440 INFO:tasks.workunit.client.1.smithi053.stdout:2/87: symlink d1/l1d 0 2024-09-21T04:35:01.442 INFO:tasks.workunit.client.1.smithi053.stdout:5/130: stat d0/d8/db/cc 0 2024-09-21T04:35:01.442 INFO:tasks.workunit.client.1.smithi053.stdout:7/105: creat d0/f20 x:0 0 0 2024-09-21T04:35:01.445 INFO:tasks.workunit.client.1.smithi053.stdout:2/88: rename d1/c1b to d1/c1e 0 2024-09-21T04:35:01.482 INFO:tasks.workunit.client.1.smithi053.stdout:4/92: mknod d1/dd/c19 0 2024-09-21T04:35:01.482 INFO:tasks.workunit.client.1.smithi053.stdout:8/97: mknod d3/c1f 0 2024-09-21T04:35:01.483 INFO:tasks.workunit.client.1.smithi053.stdout:9/109: rmdir d0/df/d18 0 2024-09-21T04:35:01.484 INFO:tasks.workunit.client.1.smithi053.stdout:4/93: rmdir d1/dd 39 2024-09-21T04:35:01.485 INFO:tasks.workunit.client.1.smithi053.stdout:7/106: mknod d0/c21 0 2024-09-21T04:35:01.492 INFO:tasks.workunit.client.1.smithi053.stdout:5/131: dwrite d0/f7 [4194304,4194304] 0 2024-09-21T04:35:01.492 INFO:tasks.workunit.client.1.smithi053.stdout:2/89: dwrite d1/f9 [0,4194304] 0 2024-09-21T04:35:01.494 INFO:tasks.workunit.client.1.smithi053.stdout:9/110: creat d0/d2/d9/d1f/f26 x:0 0 0 2024-09-21T04:35:01.494 INFO:tasks.workunit.client.1.smithi053.stdout:4/94: mknod d1/dd/de/d14/c1a 0 2024-09-21T04:35:01.497 INFO:tasks.workunit.client.1.smithi053.stdout:4/95: write d1/dd/f11 [1618448,45026] 0 2024-09-21T04:35:01.502 INFO:tasks.workunit.client.1.smithi053.stdout:9/111: creat d0/df/f27 x:0 0 0 2024-09-21T04:35:01.504 INFO:tasks.workunit.client.1.smithi053.stdout:5/132: dwrite d0/f14 [0,4194304] 0 2024-09-21T04:35:01.506 INFO:tasks.workunit.client.1.smithi053.stdout:4/96: dread d1/dd/f11 [0,4194304] 0 2024-09-21T04:35:01.507 INFO:tasks.workunit.client.1.smithi053.stdout:8/98: write d3/fa [528536,83897] 0 2024-09-21T04:35:01.509 INFO:tasks.workunit.client.1.smithi053.stdout:5/133: rename d0/c1a to d0/dd/c25 0 2024-09-21T04:35:01.510 INFO:tasks.workunit.client.1.smithi053.stdout:9/112: mkdir d0/d28 0 2024-09-21T04:35:01.511 INFO:tasks.workunit.client.1.smithi053.stdout:9/113: truncate d0/f1 4673003 0 2024-09-21T04:35:01.514 INFO:tasks.workunit.client.1.smithi053.stdout:8/99: fdatasync d3/df/d14/d16/f1d 0 2024-09-21T04:35:01.515 INFO:tasks.workunit.client.1.smithi053.stdout:5/134: write d0/d8/db/f15 [2209193,75798] 0 2024-09-21T04:35:01.517 INFO:tasks.workunit.client.1.smithi053.stdout:9/114: mknod d0/d1c/d25/c29 0 2024-09-21T04:35:01.519 INFO:tasks.workunit.client.1.smithi053.stdout:4/97: rename d1/dd/c19 to d1/dd/c1b 0 2024-09-21T04:35:01.520 INFO:tasks.workunit.client.1.smithi053.stdout:8/100: symlink d3/d5/d1a/l20 0 2024-09-21T04:35:01.522 INFO:tasks.workunit.client.1.smithi053.stdout:2/90: dwrite d1/f5 [0,4194304] 0 2024-09-21T04:35:01.523 INFO:tasks.workunit.client.1.smithi053.stdout:8/101: fsync d3/f18 0 2024-09-21T04:35:01.523 INFO:tasks.workunit.client.1.smithi053.stdout:9/115: truncate d0/d2/d9/d1f/f26 986990 0 2024-09-21T04:35:01.525 INFO:tasks.workunit.client.1.smithi053.stdout:2/91: mkdir d1/d18/d1f 0 2024-09-21T04:35:01.527 INFO:tasks.workunit.client.1.smithi053.stdout:4/98: rename d1/f3 to d1/dd/f1c 0 2024-09-21T04:35:01.528 INFO:tasks.workunit.client.1.smithi053.stdout:8/102: mkdir d3/d5/d21 0 2024-09-21T04:35:01.529 INFO:tasks.workunit.client.1.smithi053.stdout:1/192: sync 2024-09-21T04:35:01.529 INFO:tasks.workunit.client.1.smithi053.stdout:6/109: sync 2024-09-21T04:35:01.530 INFO:tasks.workunit.client.1.smithi053.stdout:9/116: mknod d0/d1a/c2a 0 2024-09-21T04:35:01.531 INFO:tasks.workunit.client.1.smithi053.stdout:9/117: write d0/d2/d9/f19 [1242393,89879] 0 2024-09-21T04:35:01.532 INFO:tasks.workunit.client.1.smithi053.stdout:4/99: dread d1/fc [0,4194304] 0 2024-09-21T04:35:01.534 INFO:tasks.workunit.client.1.smithi053.stdout:8/103: mknod d3/df/d14/d16/c22 0 2024-09-21T04:35:01.538 INFO:tasks.workunit.client.1.smithi053.stdout:2/92: mknod d1/d18/d1f/c20 0 2024-09-21T04:35:01.540 INFO:tasks.workunit.client.1.smithi053.stdout:9/118: mkdir d0/d2/d2b 0 2024-09-21T04:35:01.540 INFO:tasks.workunit.client.1.smithi053.stdout:5/135: write d0/dd/f24 [208709,13049] 0 2024-09-21T04:35:01.542 INFO:tasks.workunit.client.1.smithi053.stdout:1/193: symlink d7/l38 0 2024-09-21T04:35:01.543 INFO:tasks.workunit.client.1.smithi053.stdout:8/104: rename d3/f18 to d3/df/d14/d16/f23 0 2024-09-21T04:35:01.544 INFO:tasks.workunit.client.1.smithi053.stdout:6/110: fsync d2/d9/f18 0 2024-09-21T04:35:01.545 INFO:tasks.workunit.client.1.smithi053.stdout:2/93: mknod d1/d18/c21 0 2024-09-21T04:35:01.545 INFO:tasks.workunit.client.1.smithi053.stdout:8/105: dread d3/d5/f13 [0,4194304] 0 2024-09-21T04:35:01.547 INFO:tasks.workunit.client.1.smithi053.stdout:6/111: symlink d2/l22 0 2024-09-21T04:35:01.549 INFO:tasks.workunit.client.1.smithi053.stdout:4/100: creat d1/dd/f1d x:0 0 0 2024-09-21T04:35:01.549 INFO:tasks.workunit.client.1.smithi053.stdout:9/119: getdents d0/d2/d2b 0 2024-09-21T04:35:01.550 INFO:tasks.workunit.client.1.smithi053.stdout:2/94: mkdir d1/d18/d1f/d22 0 2024-09-21T04:35:01.551 INFO:tasks.workunit.client.1.smithi053.stdout:5/136: dwrite d0/d8/f17 [0,4194304] 0 2024-09-21T04:35:01.585 INFO:tasks.workunit.client.1.smithi053.stdout:2/95: write d1/d18/f1c [947184,13738] 0 2024-09-21T04:35:01.585 INFO:tasks.workunit.client.1.smithi053.stdout:6/112: chown d2/d9/d1a 139980 1 2024-09-21T04:35:01.588 INFO:tasks.workunit.client.1.smithi053.stdout:1/194: rename d7/d9/da/d11/d2b/l31 to d7/d9/l39 0 2024-09-21T04:35:01.589 INFO:tasks.workunit.client.1.smithi053.stdout:2/96: rmdir d1 39 2024-09-21T04:35:01.591 INFO:tasks.workunit.client.1.smithi053.stdout:5/137: dread d0/fe [0,4194304] 0 2024-09-21T04:35:01.593 INFO:tasks.workunit.client.1.smithi053.stdout:1/195: unlink d7/d24/f29 0 2024-09-21T04:35:01.594 INFO:tasks.workunit.client.1.smithi053.stdout:2/97: symlink d1/l23 0 2024-09-21T04:35:01.595 INFO:tasks.workunit.client.1.smithi053.stdout:9/120: dwrite d0/d2/d9/f19 [0,4194304] 0 2024-09-21T04:35:01.596 INFO:tasks.workunit.client.1.smithi053.stdout:4/101: dwrite d1/fa [0,4194304] 0 2024-09-21T04:35:01.597 INFO:tasks.workunit.client.1.smithi053.stdout:2/98: fsync d1/f12 0 2024-09-21T04:35:01.602 INFO:tasks.workunit.client.1.smithi053.stdout:5/138: write d0/fe [1173756,118599] 0 2024-09-21T04:35:01.603 INFO:tasks.workunit.client.1.smithi053.stdout:6/113: dread d2/d9/d1a/f21 [0,4194304] 0 2024-09-21T04:35:01.603 INFO:tasks.workunit.client.1.smithi053.stdout:4/102: write d1/dd/f1d [445173,52508] 0 2024-09-21T04:35:01.605 INFO:tasks.workunit.client.1.smithi053.stdout:2/99: write d1/f9 [3334838,127691] 0 2024-09-21T04:35:01.606 INFO:tasks.workunit.client.1.smithi053.stdout:2/100: readlink d1/l1d 0 2024-09-21T04:35:01.607 INFO:tasks.workunit.client.1.smithi053.stdout:5/139: creat d0/f26 x:0 0 0 2024-09-21T04:35:01.610 INFO:tasks.workunit.client.1.smithi053.stdout:2/101: read d1/d18/f1c [338215,101744] 0 2024-09-21T04:35:01.611 INFO:tasks.workunit.client.1.smithi053.stdout:9/121: dread d0/d2/d9/fb [0,4194304] 0 2024-09-21T04:35:01.614 INFO:tasks.workunit.client.1.smithi053.stdout:9/122: chown d0/d1a/f1b 64190645 1 2024-09-21T04:35:01.615 INFO:tasks.workunit.client.1.smithi053.stdout:1/196: dwrite d7/d9/da/d11/f33 [0,4194304] 0 2024-09-21T04:35:01.616 INFO:tasks.workunit.client.1.smithi053.stdout:6/114: mknod d2/d9/de/c23 0 2024-09-21T04:35:01.618 INFO:tasks.workunit.client.1.smithi053.stdout:2/102: creat d1/d18/d1f/f24 x:0 0 0 2024-09-21T04:35:01.620 INFO:tasks.workunit.client.1.smithi053.stdout:5/140: dread d0/dd/f23 [4194304,4194304] 0 2024-09-21T04:35:01.621 INFO:tasks.workunit.client.1.smithi053.stdout:9/123: creat d0/d1a/f2c x:0 0 0 2024-09-21T04:35:01.622 INFO:tasks.workunit.client.1.smithi053.stdout:4/103: dwrite d1/dd/f11 [0,4194304] 0 2024-09-21T04:35:01.624 INFO:tasks.workunit.client.1.smithi053.stdout:4/104: stat d1/dd/cf 0 2024-09-21T04:35:01.627 INFO:tasks.workunit.client.1.smithi053.stdout:2/103: unlink d1/c8 0 2024-09-21T04:35:01.629 INFO:tasks.workunit.client.1.smithi053.stdout:1/197: dwrite d7/d9/da/d11/f26 [8388608,4194304] 0 2024-09-21T04:35:01.631 INFO:tasks.workunit.client.1.smithi053.stdout:9/124: symlink d0/d1c/l2d 0 2024-09-21T04:35:01.637 INFO:tasks.workunit.client.1.smithi053.stdout:2/104: dwrite d1/f12 [0,4194304] 0 2024-09-21T04:35:01.642 INFO:tasks.workunit.client.1.smithi053.stdout:6/115: getdents d2/d9 0 2024-09-21T04:35:01.643 INFO:tasks.workunit.client.1.smithi053.stdout:0/65: sync 2024-09-21T04:35:01.643 INFO:tasks.workunit.client.1.smithi053.stdout:0/66: stat f1 0 2024-09-21T04:35:01.645 INFO:tasks.workunit.client.1.smithi053.stdout:9/125: symlink d0/d2/d9/l2e 0 2024-09-21T04:35:01.645 INFO:tasks.workunit.client.1.smithi053.stdout:9/126: fsync d0/d2/d9/fc 0 2024-09-21T04:35:01.645 INFO:tasks.workunit.client.1.smithi053.stdout:5/141: link d0/dd/l1c d0/d8/l27 0 2024-09-21T04:35:01.648 INFO:tasks.workunit.client.1.smithi053.stdout:1/198: rename d7/d9/da/l10 to d7/d9/da/l3a 0 2024-09-21T04:35:01.650 INFO:tasks.workunit.client.1.smithi053.stdout:1/199: write d7/d9/da/d11/f33 [2423861,53847] 0 2024-09-21T04:35:01.658 INFO:tasks.workunit.client.1.smithi053.stdout:9/127: dwrite d0/d2/fa [0,4194304] 0 2024-09-21T04:35:01.662 INFO:tasks.workunit.client.1.smithi053.stdout:9/128: creat d0/d2/d9/d1f/f2f x:0 0 0 2024-09-21T04:35:01.665 INFO:tasks.workunit.client.1.smithi053.stdout:1/200: getdents d7/d9/da/d11/d2b 0 2024-09-21T04:35:01.667 INFO:tasks.workunit.client.1.smithi053.stdout:9/129: mknod d0/c30 0 2024-09-21T04:35:01.668 INFO:tasks.workunit.client.1.smithi053.stdout:9/130: read d0/f4 [2414714,108180] 0 2024-09-21T04:35:01.668 INFO:tasks.workunit.client.1.smithi053.stdout:9/131: dread - d0/df/f23 zero size 2024-09-21T04:35:01.669 INFO:tasks.workunit.client.1.smithi053.stdout:9/132: write d0/f15 [1719940,111588] 0 2024-09-21T04:35:01.670 INFO:tasks.workunit.client.1.smithi053.stdout:9/133: write d0/df/f23 [561251,90009] 0 2024-09-21T04:35:01.671 INFO:tasks.workunit.client.1.smithi053.stdout:1/201: mkdir d7/d24/d2a/d3b 0 2024-09-21T04:35:01.672 INFO:tasks.workunit.client.1.smithi053.stdout:9/134: truncate d0/d2/f13 253068 0 2024-09-21T04:35:01.678 INFO:tasks.workunit.client.1.smithi053.stdout:9/135: rename d0/d2/c8 to d0/d2/d9/c31 0 2024-09-21T04:35:01.681 INFO:tasks.workunit.client.1.smithi053.stdout:4/105: write d1/f4 [294587,82423] 0 2024-09-21T04:35:01.684 INFO:tasks.workunit.client.1.smithi053.stdout:4/106: mknod d1/dd/de/c1e 0 2024-09-21T04:35:01.684 INFO:tasks.workunit.client.1.smithi053.stdout:4/107: stat d1/dd/de/d14 0 2024-09-21T04:35:01.685 INFO:tasks.workunit.client.1.smithi053.stdout:4/108: truncate d1/fa 4981459 0 2024-09-21T04:35:01.685 INFO:tasks.workunit.client.1.smithi053.stdout:5/142: truncate d0/dd/f23 4175862 0 2024-09-21T04:35:01.714 INFO:tasks.workunit.client.1.smithi053.stdout:4/109: symlink d1/dd/de/d14/l1f 0 2024-09-21T04:35:01.714 INFO:tasks.workunit.client.1.smithi053.stdout:0/67: write d5/dd/ff [1347790,101515] 0 2024-09-21T04:35:01.715 INFO:tasks.workunit.client.1.smithi053.stdout:5/143: write d0/f26 [1027022,113584] 0 2024-09-21T04:35:01.718 INFO:tasks.workunit.client.1.smithi053.stdout:9/136: rename d0/d2/d9/c31 to d0/d1a/c32 0 2024-09-21T04:35:01.719 INFO:tasks.workunit.client.1.smithi053.stdout:9/137: truncate d0/d2/fe 1574024 0 2024-09-21T04:35:01.719 INFO:tasks.workunit.client.1.smithi053.stdout:4/110: mkdir d1/dd/de/d14/d20 0 2024-09-21T04:35:01.720 INFO:tasks.workunit.client.1.smithi053.stdout:4/111: write d1/dd/f11 [1505582,97158] 0 2024-09-21T04:35:01.733 INFO:tasks.workunit.client.1.smithi053.stdout:0/68: link d5/dd/ff d5/dd/f17 0 2024-09-21T04:35:01.733 INFO:tasks.workunit.client.1.smithi053.stdout:0/69: write d5/fa [687763,59816] 0 2024-09-21T04:35:01.738 INFO:tasks.workunit.client.1.smithi053.stdout:9/138: write d0/d2/d9/f19 [4079774,124510] 0 2024-09-21T04:35:01.739 INFO:tasks.workunit.client.1.smithi053.stdout:9/139: chown d0/d1a/f2c 2074123779 1 2024-09-21T04:35:01.739 INFO:tasks.workunit.client.1.smithi053.stdout:9/140: stat d0/d1c/d25/c29 0 2024-09-21T04:35:01.742 INFO:tasks.workunit.client.1.smithi053.stdout:5/144: write d0/dd/f23 [4789731,78553] 0 2024-09-21T04:35:01.744 INFO:tasks.workunit.client.1.smithi053.stdout:3/102: sync 2024-09-21T04:35:01.744 INFO:tasks.workunit.client.1.smithi053.stdout:7/107: sync 2024-09-21T04:35:01.747 INFO:tasks.workunit.client.1.smithi053.stdout:9/141: dwrite d0/d2/f13 [0,4194304] 0 2024-09-21T04:35:01.751 INFO:tasks.workunit.client.1.smithi053.stdout:3/103: dwrite fe [0,4194304] 0 2024-09-21T04:35:01.752 INFO:tasks.workunit.client.1.smithi053.stdout:3/104: chown fa 2109614034 1 2024-09-21T04:35:01.786 INFO:tasks.workunit.client.1.smithi053.stdout:2/105: sync 2024-09-21T04:35:01.787 INFO:tasks.workunit.client.1.smithi053.stdout:8/106: sync 2024-09-21T04:35:01.869 INFO:tasks.workunit.client.1.smithi053.stdout:6/116: sync 2024-09-21T04:35:01.869 INFO:tasks.workunit.client.1.smithi053.stdout:1/202: sync 2024-09-21T04:35:01.870 INFO:tasks.workunit.client.1.smithi053.stdout:6/117: write d2/fd [1936958,59780] 0 2024-09-21T04:35:01.882 INFO:tasks.workunit.client.1.smithi053.stdout:0/70: rename c0 to d5/de/c18 0 2024-09-21T04:35:01.883 INFO:tasks.workunit.client.1.smithi053.stdout:5/145: fdatasync d0/d8/f10 0 2024-09-21T04:35:01.885 INFO:tasks.workunit.client.1.smithi053.stdout:5/146: write d0/dd/f24 [999044,29573] 0 2024-09-21T04:35:01.888 INFO:tasks.workunit.client.1.smithi053.stdout:7/108: dread d0/fd [0,4194304] 0 2024-09-21T04:35:01.888 INFO:tasks.workunit.client.1.smithi053.stdout:2/106: symlink d1/d18/l25 0 2024-09-21T04:35:01.890 INFO:tasks.workunit.client.1.smithi053.stdout:5/147: dread d0/d8/f17 [0,4194304] 0 2024-09-21T04:35:01.890 INFO:tasks.workunit.client.1.smithi053.stdout:1/203: rmdir d7/d9/da 39 2024-09-21T04:35:01.891 INFO:tasks.workunit.client.1.smithi053.stdout:4/112: link d1/l6 d1/dd/de/d14/d20/l21 0 2024-09-21T04:35:01.892 INFO:tasks.workunit.client.1.smithi053.stdout:2/107: dread d1/f5 [0,4194304] 0 2024-09-21T04:35:01.895 INFO:tasks.workunit.client.1.smithi053.stdout:6/118: rename d2/d9/de/c23 to d2/d1d/c24 0 2024-09-21T04:35:01.896 INFO:tasks.workunit.client.1.smithi053.stdout:6/119: truncate d2/f8 555361 0 2024-09-21T04:35:01.898 INFO:tasks.workunit.client.1.smithi053.stdout:9/142: symlink d0/d1c/d21/l33 0 2024-09-21T04:35:01.903 INFO:tasks.workunit.client.1.smithi053.stdout:8/107: sync 2024-09-21T04:35:01.907 INFO:tasks.workunit.client.1.smithi053.stdout:5/148: mknod d0/d8/c28 0 2024-09-21T04:35:01.909 INFO:tasks.workunit.client.1.smithi053.stdout:1/204: write d7/d9/da/d11/f26 [657207,114901] 0 2024-09-21T04:35:01.911 INFO:tasks.workunit.client.1.smithi053.stdout:0/71: symlink d5/dd/l19 0 2024-09-21T04:35:01.914 INFO:tasks.workunit.client.1.smithi053.stdout:2/108: rename d1/c16 to d1/d18/d1f/c26 0 2024-09-21T04:35:01.915 INFO:tasks.workunit.client.1.smithi053.stdout:6/120: mknod d2/d1d/c25 0 2024-09-21T04:35:01.916 INFO:tasks.workunit.client.1.smithi053.stdout:0/72: dwrite d5/f15 [0,4194304] 0 2024-09-21T04:35:01.916 INFO:tasks.workunit.client.1.smithi053.stdout:9/143: stat d0/l3 0 2024-09-21T04:35:01.920 INFO:tasks.workunit.client.1.smithi053.stdout:2/109: dread d1/f9 [0,4194304] 0 2024-09-21T04:35:01.921 INFO:tasks.workunit.client.1.smithi053.stdout:6/121: dread d2/d9/de/f15 [0,4194304] 0 2024-09-21T04:35:01.921 INFO:tasks.workunit.client.1.smithi053.stdout:2/110: truncate d1/f4 4727335 0 2024-09-21T04:35:01.922 INFO:tasks.workunit.client.1.smithi053.stdout:0/73: dread d5/dd/f17 [0,4194304] 0 2024-09-21T04:35:01.923 INFO:tasks.workunit.client.1.smithi053.stdout:2/111: dread - d1/d18/d1f/f24 zero size 2024-09-21T04:35:01.923 INFO:tasks.workunit.client.1.smithi053.stdout:7/109: symlink d0/l22 0 2024-09-21T04:35:01.926 INFO:tasks.workunit.client.1.smithi053.stdout:6/122: dread d2/d9/de/f15 [0,4194304] 0 2024-09-21T04:35:01.927 INFO:tasks.workunit.client.1.smithi053.stdout:6/123: write d2/d1b/f1c [1833049,1956] 0 2024-09-21T04:35:01.927 INFO:tasks.workunit.client.1.smithi053.stdout:5/149: rename d0/d8/db/f15 to d0/d8/db/f29 0 2024-09-21T04:35:01.930 INFO:tasks.workunit.client.1.smithi053.stdout:4/113: symlink d1/l22 0 2024-09-21T04:35:01.941 INFO:tasks.workunit.client.1.smithi053.stdout:2/112: mkdir d1/d18/d1f/d27 0 2024-09-21T04:35:01.942 INFO:tasks.workunit.client.1.smithi053.stdout:2/113: write d1/d18/d1f/f24 [236220,71639] 0 2024-09-21T04:35:01.944 INFO:tasks.workunit.client.1.smithi053.stdout:5/150: mkdir d0/d8/d2a 0 2024-09-21T04:35:01.947 INFO:tasks.workunit.client.1.smithi053.stdout:1/205: mkdir d7/d9/da/d3c 0 2024-09-21T04:35:01.950 INFO:tasks.workunit.client.1.smithi053.stdout:5/151: dwrite d0/dd/f23 [0,4194304] 0 2024-09-21T04:35:01.951 INFO:tasks.workunit.client.1.smithi053.stdout:9/144: mkdir d0/d2/d9/d34 0 2024-09-21T04:35:01.956 INFO:tasks.workunit.client.1.smithi053.stdout:2/114: unlink d1/d18/d1f/c20 0 2024-09-21T04:35:01.965 INFO:tasks.workunit.client.1.smithi053.stdout:0/74: link d5/d6/f16 d5/dd/f1a 0 2024-09-21T04:35:01.965 INFO:tasks.workunit.client.1.smithi053.stdout:0/75: chown d5/d10 930 1 2024-09-21T04:35:01.966 INFO:tasks.workunit.client.1.smithi053.stdout:0/76: readlink d5/db/l9 0 2024-09-21T04:35:01.967 INFO:tasks.workunit.client.1.smithi053.stdout:5/152: rename d0/fe to d0/d8/f2b 0 2024-09-21T04:35:01.968 INFO:tasks.workunit.client.1.smithi053.stdout:5/153: truncate d0/d8/db/f19 592099 0 2024-09-21T04:35:01.973 INFO:tasks.workunit.client.1.smithi053.stdout:4/114: link d1/fa d1/dd/f23 0 2024-09-21T04:35:01.977 INFO:tasks.workunit.client.1.smithi053.stdout:0/77: creat d5/d6/f1b x:0 0 0 2024-09-21T04:35:01.979 INFO:tasks.workunit.client.1.smithi053.stdout:9/145: mkdir d0/d2/d2b/d35 0 2024-09-21T04:35:01.980 INFO:tasks.workunit.client.1.smithi053.stdout:5/154: unlink d0/d8/db/cc 0 2024-09-21T04:35:01.982 INFO:tasks.workunit.client.1.smithi053.stdout:9/146: dread d0/f4 [0,4194304] 0 2024-09-21T04:35:01.985 INFO:tasks.workunit.client.1.smithi053.stdout:5/155: rename d0/l2 to d0/d8/db/l2c 0 2024-09-21T04:35:01.985 INFO:tasks.workunit.client.1.smithi053.stdout:5/156: readlink d0/l12 0 2024-09-21T04:35:01.987 INFO:tasks.workunit.client.1.smithi053.stdout:4/115: dwrite d1/fc [0,4194304] 0 2024-09-21T04:35:01.989 INFO:tasks.workunit.client.1.smithi053.stdout:5/157: mknod d0/d8/c2d 0 2024-09-21T04:35:01.991 INFO:tasks.workunit.client.1.smithi053.stdout:9/147: link d0/d1c/d25/c29 d0/d1a/c36 0 2024-09-21T04:35:01.997 INFO:tasks.workunit.client.1.smithi053.stdout:5/158: dwrite d0/d8/f1f [0,4194304] 0 2024-09-21T04:35:01.998 INFO:tasks.workunit.client.1.smithi053.stdout:9/148: dread d0/d2/d9/f19 [0,4194304] 0 2024-09-21T04:35:01.999 INFO:tasks.workunit.client.1.smithi053.stdout:4/116: link d1/fc d1/dd/f24 0 2024-09-21T04:35:02.000 INFO:tasks.workunit.client.1.smithi053.stdout:4/117: chown d1/dd/de/d14/c1a 351 1 2024-09-21T04:35:02.005 INFO:tasks.workunit.client.1.smithi053.stdout:5/159: dread d0/dd/f23 [0,4194304] 0 2024-09-21T04:35:02.009 INFO:tasks.workunit.client.1.smithi053.stdout:9/149: dwrite d0/df/f23 [0,4194304] 0 2024-09-21T04:35:02.010 INFO:tasks.workunit.client.1.smithi053.stdout:4/118: dread d1/dd/f24 [0,4194304] 0 2024-09-21T04:35:02.010 INFO:tasks.workunit.client.1.smithi053.stdout:9/150: dread - d0/d1a/f2c zero size 2024-09-21T04:35:02.014 INFO:tasks.workunit.client.1.smithi053.stdout:5/160: getdents d0/dd 0 2024-09-21T04:35:02.018 INFO:tasks.workunit.client.1.smithi053.stdout:5/161: write d0/d8/f17 [318508,4284] 0 2024-09-21T04:35:02.018 INFO:tasks.workunit.client.1.smithi053.stdout:9/151: dread d0/d2/d9/fb [0,4194304] 0 2024-09-21T04:35:02.022 INFO:tasks.workunit.client.1.smithi053.stdout:4/119: dwrite d1/fa [4194304,4194304] 0 2024-09-21T04:35:02.029 INFO:tasks.workunit.client.1.smithi053.stdout:9/152: dwrite d0/d1a/f2c [0,4194304] 0 2024-09-21T04:35:02.032 INFO:tasks.workunit.client.1.smithi053.stdout:9/153: write d0/d2/d9/fb [680647,122540] 0 2024-09-21T04:35:02.033 INFO:tasks.workunit.client.1.smithi053.stdout:9/154: write d0/d1a/f1b [161989,128944] 0 2024-09-21T04:35:02.034 INFO:tasks.workunit.client.1.smithi053.stdout:9/155: write d0/f24 [456337,129067] 0 2024-09-21T04:35:02.035 INFO:tasks.workunit.client.1.smithi053.stdout:9/156: stat d0/d2/d9/fb 0 2024-09-21T04:35:02.036 INFO:tasks.workunit.client.1.smithi053.stdout:9/157: write d0/d2/d9/fb [2135857,48297] 0 2024-09-21T04:35:02.071 INFO:tasks.workunit.client.1.smithi053.stdout:3/105: truncate fa 901069 0 2024-09-21T04:35:02.072 INFO:tasks.workunit.client.1.smithi053.stdout:3/106: write fc [3188032,122765] 0 2024-09-21T04:35:02.074 INFO:tasks.workunit.client.1.smithi053.stdout:3/107: readlink l5 0 2024-09-21T04:35:02.076 INFO:tasks.workunit.client.1.smithi053.stdout:3/108: fdatasync d16/d1b/f1f 0 2024-09-21T04:35:02.081 INFO:tasks.workunit.client.1.smithi053.stdout:3/109: dwrite fe [0,4194304] 0 2024-09-21T04:35:02.090 INFO:tasks.workunit.client.1.smithi053.stdout:8/108: truncate d3/fa 48473 0 2024-09-21T04:35:02.091 INFO:tasks.workunit.client.1.smithi053.stdout:8/109: chown d3/df/d14/d16/c22 128818 1 2024-09-21T04:35:02.091 INFO:tasks.workunit.client.1.smithi053.stdout:8/110: stat d3/cb 0 2024-09-21T04:35:02.093 INFO:tasks.workunit.client.1.smithi053.stdout:8/111: dread d3/df/d14/d16/f23 [0,4194304] 0 2024-09-21T04:35:02.093 INFO:tasks.workunit.client.1.smithi053.stdout:7/110: write d0/f15 [723276,10472] 0 2024-09-21T04:35:02.097 INFO:tasks.workunit.client.1.smithi053.stdout:8/112: chown d3/fa 10329409 1 2024-09-21T04:35:02.097 INFO:tasks.workunit.client.1.smithi053.stdout:8/113: rename d3/d5/d1a to d3/d5/d1a/d24 22 2024-09-21T04:35:02.101 INFO:tasks.workunit.client.1.smithi053.stdout:8/114: unlink d3/lc 0 2024-09-21T04:35:02.107 INFO:tasks.workunit.client.1.smithi053.stdout:6/124: truncate d2/d9/f11 3141938 0 2024-09-21T04:35:02.111 INFO:tasks.workunit.client.1.smithi053.stdout:0/78: fsync d5/dd/f1a 0 2024-09-21T04:35:02.112 INFO:tasks.workunit.client.1.smithi053.stdout:6/125: creat d2/d9/d1a/f26 x:0 0 0 2024-09-21T04:35:02.112 INFO:tasks.workunit.client.1.smithi053.stdout:1/206: dwrite f3 [0,4194304] 0 2024-09-21T04:35:02.112 INFO:tasks.workunit.client.1.smithi053.stdout:6/126: fdatasync d2/d9/f20 0 2024-09-21T04:35:02.113 INFO:tasks.workunit.client.1.smithi053.stdout:1/207: fsync d7/d9/da/d11/f33 0 2024-09-21T04:35:02.115 INFO:tasks.workunit.client.1.smithi053.stdout:2/115: write d1/d18/f1c [1293935,110224] 0 2024-09-21T04:35:02.124 INFO:tasks.workunit.client.1.smithi053.stdout:6/127: dread d2/d9/de/f1f [0,4194304] 0 2024-09-21T04:35:02.125 INFO:tasks.workunit.client.1.smithi053.stdout:2/116: symlink d1/d18/d1f/l28 0 2024-09-21T04:35:02.129 INFO:tasks.workunit.client.1.smithi053.stdout:1/208: symlink d7/d9/l3d 0 2024-09-21T04:35:02.130 INFO:tasks.workunit.client.1.smithi053.stdout:1/209: write d7/d9/da/d11/f26 [1544342,66294] 0 2024-09-21T04:35:02.140 INFO:tasks.workunit.client.1.smithi053.stdout:6/128: chown d2/d9/de/f15 5157 1 2024-09-21T04:35:02.144 INFO:tasks.workunit.client.1.smithi053.stdout:0/79: getdents d5/db 0 2024-09-21T04:35:02.149 INFO:tasks.workunit.client.1.smithi053.stdout:1/210: symlink d7/d9/l3e 0 2024-09-21T04:35:02.152 INFO:tasks.workunit.client.1.smithi053.stdout:1/211: dread d7/d9/da/d11/f33 [0,4194304] 0 2024-09-21T04:35:02.155 INFO:tasks.workunit.client.1.smithi053.stdout:5/162: link d0/d8/db/l2c d0/d8/db/l2e 0 2024-09-21T04:35:02.156 INFO:tasks.workunit.client.1.smithi053.stdout:6/129: mkdir d2/d1b/d1e/d27 0 2024-09-21T04:35:02.166 INFO:tasks.workunit.client.1.smithi053.stdout:1/212: rmdir d7/d24/d2a/d3b 0 2024-09-21T04:35:02.166 INFO:tasks.workunit.client.1.smithi053.stdout:5/163: dwrite d0/d8/db/f29 [0,4194304] 0 2024-09-21T04:35:02.170 INFO:tasks.workunit.client.1.smithi053.stdout:5/164: creat d0/f2f x:0 0 0 2024-09-21T04:35:02.172 INFO:tasks.workunit.client.1.smithi053.stdout:5/165: write d0/dd/f24 [2069396,99868] 0 2024-09-21T04:35:02.176 INFO:tasks.workunit.client.1.smithi053.stdout:5/166: mkdir d0/dd/d30 0 2024-09-21T04:35:02.181 INFO:tasks.workunit.client.1.smithi053.stdout:3/110: dwrite f14 [0,4194304] 0 2024-09-21T04:35:02.182 INFO:tasks.workunit.client.1.smithi053.stdout:3/111: write f15 [1646336,105714] 0 2024-09-21T04:35:02.183 INFO:tasks.workunit.client.1.smithi053.stdout:4/120: sync 2024-09-21T04:35:02.185 INFO:tasks.workunit.client.1.smithi053.stdout:3/112: mknod d16/d1b/c23 0 2024-09-21T04:35:02.189 INFO:tasks.workunit.client.1.smithi053.stdout:4/121: dwrite d1/dd/f23 [0,4194304] 0 2024-09-21T04:35:02.190 INFO:tasks.workunit.client.1.smithi053.stdout:4/122: fdatasync d1/dd/f1d 0 2024-09-21T04:35:02.191 INFO:tasks.workunit.client.1.smithi053.stdout:3/113: creat d16/d1b/f24 x:0 0 0 2024-09-21T04:35:02.194 INFO:tasks.workunit.client.1.smithi053.stdout:8/115: truncate d3/df/f11 448818 0 2024-09-21T04:35:02.195 INFO:tasks.workunit.client.1.smithi053.stdout:8/116: chown d3/d5/l1e 0 1 2024-09-21T04:35:02.197 INFO:tasks.workunit.client.1.smithi053.stdout:3/114: symlink d16/d17/l25 0 2024-09-21T04:35:02.202 INFO:tasks.workunit.client.1.smithi053.stdout:4/123: symlink d1/l25 0 2024-09-21T04:35:02.204 INFO:tasks.workunit.client.1.smithi053.stdout:0/80: truncate d5/dd/f1a 740220 0 2024-09-21T04:35:02.204 INFO:tasks.workunit.client.1.smithi053.stdout:0/81: chown d5/db 365453436 1 2024-09-21T04:35:02.205 INFO:tasks.workunit.client.1.smithi053.stdout:4/124: mknod d1/dd/c26 0 2024-09-21T04:35:02.208 INFO:tasks.workunit.client.1.smithi053.stdout:2/117: getdents d1/d18/d1f 0 2024-09-21T04:35:02.211 INFO:tasks.workunit.client.1.smithi053.stdout:2/118: mknod d1/d18/d1f/c29 0 2024-09-21T04:35:02.214 INFO:tasks.workunit.client.1.smithi053.stdout:2/119: creat d1/d18/f2a x:0 0 0 2024-09-21T04:35:02.216 INFO:tasks.workunit.client.1.smithi053.stdout:4/125: getdents d1/dd/de/d14/d20 0 2024-09-21T04:35:02.219 INFO:tasks.workunit.client.1.smithi053.stdout:4/126: mkdir d1/d27 0 2024-09-21T04:35:02.219 INFO:tasks.workunit.client.1.smithi053.stdout:2/120: dwrite d1/f12 [0,4194304] 0 2024-09-21T04:35:02.220 INFO:tasks.workunit.client.1.smithi053.stdout:2/121: fdatasync d1/d18/f2a 0 2024-09-21T04:35:02.222 INFO:tasks.workunit.client.1.smithi053.stdout:2/122: rmdir d1 39 2024-09-21T04:35:02.226 INFO:tasks.workunit.client.1.smithi053.stdout:4/127: dwrite d1/dd/f1c [0,4194304] 0 2024-09-21T04:35:02.230 INFO:tasks.workunit.client.1.smithi053.stdout:2/123: dread d1/d18/d1f/f24 [0,4194304] 0 2024-09-21T04:35:02.233 INFO:tasks.workunit.client.1.smithi053.stdout:1/213: write f1 [3677308,3037] 0 2024-09-21T04:35:02.234 INFO:tasks.workunit.client.1.smithi053.stdout:2/124: write d1/fb [1221561,127315] 0 2024-09-21T04:35:02.238 INFO:tasks.workunit.client.1.smithi053.stdout:4/128: dwrite d1/dd/f23 [0,4194304] 0 2024-09-21T04:35:02.239 INFO:tasks.workunit.client.1.smithi053.stdout:6/130: dwrite d2/d9/f19 [0,4194304] 0 2024-09-21T04:35:02.241 INFO:tasks.workunit.client.1.smithi053.stdout:2/125: mknod d1/d18/c2b 0 2024-09-21T04:35:02.242 INFO:tasks.workunit.client.1.smithi053.stdout:3/115: fdatasync fa 0 2024-09-21T04:35:02.244 INFO:tasks.workunit.client.1.smithi053.stdout:4/129: chown d1/l9 6511 1 2024-09-21T04:35:02.245 INFO:tasks.workunit.client.1.smithi053.stdout:2/126: fsync d1/f12 0 2024-09-21T04:35:02.246 INFO:tasks.workunit.client.1.smithi053.stdout:6/131: creat d2/f28 x:0 0 0 2024-09-21T04:35:02.247 INFO:tasks.workunit.client.1.smithi053.stdout:3/116: rmdir d16/d17/d21 39 2024-09-21T04:35:02.249 INFO:tasks.workunit.client.1.smithi053.stdout:1/214: dwrite f3 [4194304,4194304] 0 2024-09-21T04:35:02.249 INFO:tasks.workunit.client.1.smithi053.stdout:3/117: write d16/d1b/f24 [215245,12021] 0 2024-09-21T04:35:02.251 INFO:tasks.workunit.client.1.smithi053.stdout:1/215: chown d7/d9/l21 22488 1 2024-09-21T04:35:02.252 INFO:tasks.workunit.client.1.smithi053.stdout:1/216: chown d7/d9/da 10644 1 2024-09-21T04:35:02.255 INFO:tasks.workunit.client.1.smithi053.stdout:6/132: symlink d2/l29 0 2024-09-21T04:35:02.257 INFO:tasks.workunit.client.1.smithi053.stdout:4/130: symlink d1/l28 0 2024-09-21T04:35:02.259 INFO:tasks.workunit.client.1.smithi053.stdout:1/217: mkdir d7/d24/d2a/d36/d3f 0 2024-09-21T04:35:02.259 INFO:tasks.workunit.client.1.smithi053.stdout:3/118: dread fa [0,4194304] 0 2024-09-21T04:35:02.261 INFO:tasks.workunit.client.1.smithi053.stdout:6/133: unlink d2/d1b/f1c 0 2024-09-21T04:35:02.264 INFO:tasks.workunit.client.1.smithi053.stdout:4/131: symlink d1/dd/de/d14/l29 0 2024-09-21T04:35:02.273 INFO:tasks.workunit.client.1.smithi053.stdout:4/132: unlink d1/l28 0 2024-09-21T04:35:02.277 INFO:tasks.workunit.client.1.smithi053.stdout:1/218: mkdir d7/d9/da/d40 0 2024-09-21T04:35:02.280 INFO:tasks.workunit.client.1.smithi053.stdout:4/133: rmdir d1/dd 39 2024-09-21T04:35:02.282 INFO:tasks.workunit.client.1.smithi053.stdout:8/117: creat d3/d5/f25 x:0 0 0 2024-09-21T04:35:02.284 INFO:tasks.workunit.client.1.smithi053.stdout:1/219: creat d7/d24/f41 x:0 0 0 2024-09-21T04:35:02.297 INFO:tasks.workunit.client.1.smithi053.stdout:9/158: sync 2024-09-21T04:35:02.297 INFO:tasks.workunit.client.1.smithi053.stdout:7/111: sync 2024-09-21T04:35:02.298 INFO:tasks.workunit.client.1.smithi053.stdout:1/220: write d7/d9/da/d11/f20 [4989899,119097] 0 2024-09-21T04:35:02.300 INFO:tasks.workunit.client.1.smithi053.stdout:0/82: write d5/dd/f1a [1255223,85398] 0 2024-09-21T04:35:02.300 INFO:tasks.workunit.client.1.smithi053.stdout:7/112: dread - d0/d6/f1b zero size 2024-09-21T04:35:02.301 INFO:tasks.workunit.client.1.smithi053.stdout:9/159: mknod d0/d2/c37 0 2024-09-21T04:35:02.301 INFO:tasks.workunit.client.1.smithi053.stdout:4/134: dread d1/dd/f1d [0,4194304] 0 2024-09-21T04:35:02.304 INFO:tasks.workunit.client.1.smithi053.stdout:7/113: rmdir d0 39 2024-09-21T04:35:02.310 INFO:tasks.workunit.client.1.smithi053.stdout:9/160: dwrite d0/f1 [4194304,4194304] 0 2024-09-21T04:35:02.310 INFO:tasks.workunit.client.1.smithi053.stdout:4/135: dread d1/fa [4194304,4194304] 0 2024-09-21T04:35:02.310 INFO:tasks.workunit.client.1.smithi053.stdout:7/114: read - d0/d6/f1b zero size 2024-09-21T04:35:02.311 INFO:tasks.workunit.client.1.smithi053.stdout:8/118: getdents d3/d5/d1a 0 2024-09-21T04:35:02.314 INFO:tasks.workunit.client.1.smithi053.stdout:8/119: write d3/df/d14/d16/f1d [456157,17] 0 2024-09-21T04:35:02.315 INFO:tasks.workunit.client.1.smithi053.stdout:4/136: rename d1/dd/f1c to d1/dd/de/f2a 0 2024-09-21T04:35:02.319 INFO:tasks.workunit.client.1.smithi053.stdout:6/134: write d2/d9/de/f15 [1761098,5269] 0 2024-09-21T04:35:02.320 INFO:tasks.workunit.client.1.smithi053.stdout:9/161: dwrite d0/f1 [4194304,4194304] 0 2024-09-21T04:35:02.323 INFO:tasks.workunit.client.1.smithi053.stdout:8/120: mkdir d3/d5/d21/d26 0 2024-09-21T04:35:02.324 INFO:tasks.workunit.client.1.smithi053.stdout:6/135: mkdir d2/d1b/d1e/d2a 0 2024-09-21T04:35:02.327 INFO:tasks.workunit.client.1.smithi053.stdout:1/221: dwrite d7/d9/f1b [0,4194304] 0 2024-09-21T04:35:02.328 INFO:tasks.workunit.client.1.smithi053.stdout:9/162: creat d0/d28/f38 x:0 0 0 2024-09-21T04:35:02.332 INFO:tasks.workunit.client.1.smithi053.stdout:9/163: write d0/f1 [2761823,68269] 0 2024-09-21T04:35:02.332 INFO:tasks.workunit.client.1.smithi053.stdout:1/222: dread d7/d9/da/f2f [0,4194304] 0 2024-09-21T04:35:02.334 INFO:tasks.workunit.client.1.smithi053.stdout:8/121: creat d3/d5/d21/d26/f27 x:0 0 0 2024-09-21T04:35:02.335 INFO:tasks.workunit.client.1.smithi053.stdout:6/136: link d2/d9/f18 d2/d1b/d1e/d2a/f2b 0 2024-09-21T04:35:02.337 INFO:tasks.workunit.client.1.smithi053.stdout:4/137: dwrite d1/fa [8388608,4194304] 0 2024-09-21T04:35:02.339 INFO:tasks.workunit.client.1.smithi053.stdout:1/223: symlink d7/d9/da/d11/l42 0 2024-09-21T04:35:02.340 INFO:tasks.workunit.client.1.smithi053.stdout:6/137: dread d2/f8 [0,4194304] 0 2024-09-21T04:35:02.340 INFO:tasks.workunit.client.1.smithi053.stdout:8/122: dread d3/df/d14/d16/f23 [0,4194304] 0 2024-09-21T04:35:02.343 INFO:tasks.workunit.client.1.smithi053.stdout:9/164: dwrite d0/d2/fe [0,4194304] 0 2024-09-21T04:35:02.346 INFO:tasks.workunit.client.1.smithi053.stdout:1/224: rmdir d7/d24/d2a 39 2024-09-21T04:35:02.347 INFO:tasks.workunit.client.1.smithi053.stdout:9/165: write d0/d2/d9/d1f/f26 [1831766,4798] 0 2024-09-21T04:35:02.349 INFO:tasks.workunit.client.1.smithi053.stdout:9/166: read d0/f1 [4196291,114458] 0 2024-09-21T04:35:02.350 INFO:tasks.workunit.client.1.smithi053.stdout:8/123: mkdir d3/d28 0 2024-09-21T04:35:02.350 INFO:tasks.workunit.client.1.smithi053.stdout:4/138: dwrite d1/dd/f23 [0,4194304] 0 2024-09-21T04:35:02.352 INFO:tasks.workunit.client.1.smithi053.stdout:8/124: fsync d3/df/d14/d16/f1d 0 2024-09-21T04:35:02.355 INFO:tasks.workunit.client.1.smithi053.stdout:1/225: dread d7/d9/f1b [0,4194304] 0 2024-09-21T04:35:02.358 INFO:tasks.workunit.client.1.smithi053.stdout:4/139: dread d1/dd/f23 [8388608,4194304] 0 2024-09-21T04:35:02.359 INFO:tasks.workunit.client.1.smithi053.stdout:4/140: write d1/dd/f23 [9401715,8284] 0 2024-09-21T04:35:02.361 INFO:tasks.workunit.client.1.smithi053.stdout:1/226: dread d7/d9/f1b [0,4194304] 0 2024-09-21T04:35:02.362 INFO:tasks.workunit.client.1.smithi053.stdout:8/125: creat d3/df/d14/d16/f29 x:0 0 0 2024-09-21T04:35:02.365 INFO:tasks.workunit.client.1.smithi053.stdout:6/138: dread d2/f3 [0,4194304] 0 2024-09-21T04:35:02.370 INFO:tasks.workunit.client.1.smithi053.stdout:2/127: sync 2024-09-21T04:35:02.370 INFO:tasks.workunit.client.1.smithi053.stdout:5/167: sync 2024-09-21T04:35:02.379 INFO:tasks.workunit.client.1.smithi053.stdout:2/128: symlink d1/l2c 0 2024-09-21T04:35:02.381 INFO:tasks.workunit.client.1.smithi053.stdout:5/168: creat d0/d8/f31 x:0 0 0 2024-09-21T04:35:02.384 INFO:tasks.workunit.client.1.smithi053.stdout:2/129: dread d1/f9 [4194304,4194304] 0 2024-09-21T04:35:02.384 INFO:tasks.workunit.client.1.smithi053.stdout:5/169: dread d0/d8/db/f29 [0,4194304] 0 2024-09-21T04:35:02.385 INFO:tasks.workunit.client.1.smithi053.stdout:2/130: write d1/d18/f2a [799694,78719] 0 2024-09-21T04:35:02.387 INFO:tasks.workunit.client.1.smithi053.stdout:2/131: read d1/f5 [2040082,56996] 0 2024-09-21T04:35:02.388 INFO:tasks.workunit.client.1.smithi053.stdout:5/170: dread d0/d8/f2b [0,4194304] 0 2024-09-21T04:35:02.390 INFO:tasks.workunit.client.1.smithi053.stdout:1/227: write f5 [2282193,60419] 0 2024-09-21T04:35:02.390 INFO:tasks.workunit.client.1.smithi053.stdout:5/171: mknod d0/d8/c32 0 2024-09-21T04:35:02.392 INFO:tasks.workunit.client.1.smithi053.stdout:2/132: mkdir d1/d18/d1f/d22/d2d 0 2024-09-21T04:35:02.395 INFO:tasks.workunit.client.1.smithi053.stdout:6/139: truncate d2/d9/de/f14 2981554 0 2024-09-21T04:35:02.396 INFO:tasks.workunit.client.1.smithi053.stdout:8/126: dwrite d3/fa [0,4194304] 0 2024-09-21T04:35:02.398 INFO:tasks.workunit.client.1.smithi053.stdout:8/127: dread - d3/df/d14/d16/f29 zero size 2024-09-21T04:35:02.398 INFO:tasks.workunit.client.1.smithi053.stdout:5/172: mknod d0/dd/d30/c33 0 2024-09-21T04:35:02.399 INFO:tasks.workunit.client.1.smithi053.stdout:2/133: symlink d1/d18/d1f/d22/d2d/l2e 0 2024-09-21T04:35:02.400 INFO:tasks.workunit.client.1.smithi053.stdout:8/128: read d3/d5/f19 [796209,59706] 0 2024-09-21T04:35:02.402 INFO:tasks.workunit.client.1.smithi053.stdout:8/129: read d3/d5/f19 [98734,40531] 0 2024-09-21T04:35:02.408 INFO:tasks.workunit.client.1.smithi053.stdout:1/228: rename d7/d24/d2a/d36/d3f to d7/d9/d43 0 2024-09-21T04:35:02.409 INFO:tasks.workunit.client.1.smithi053.stdout:8/130: rename d3/df/d14/d16/c17 to d3/df/d14/d16/c2a 0 2024-09-21T04:35:02.410 INFO:tasks.workunit.client.1.smithi053.stdout:5/173: dwrite d0/d8/db/f29 [0,4194304] 0 2024-09-21T04:35:02.413 INFO:tasks.workunit.client.1.smithi053.stdout:6/140: rename d2/lf to d2/d1d/l2c 0 2024-09-21T04:35:02.416 INFO:tasks.workunit.client.1.smithi053.stdout:5/174: write d0/f7 [675486,32851] 0 2024-09-21T04:35:02.417 INFO:tasks.workunit.client.1.smithi053.stdout:8/131: mknod d3/d5/c2b 0 2024-09-21T04:35:02.417 INFO:tasks.workunit.client.1.smithi053.stdout:5/175: write d0/d8/f17 [3850117,5195] 0 2024-09-21T04:35:02.420 INFO:tasks.workunit.client.1.smithi053.stdout:6/141: rmdir d2/d1b/d1e/d2a 39 2024-09-21T04:35:02.422 INFO:tasks.workunit.client.1.smithi053.stdout:6/142: chown d2/fd 1952966 1 2024-09-21T04:35:02.425 INFO:tasks.workunit.client.1.smithi053.stdout:8/132: dwrite d3/fa [0,4194304] 0 2024-09-21T04:35:02.426 INFO:tasks.workunit.client.1.smithi053.stdout:2/134: dwrite d1/f5 [0,4194304] 0 2024-09-21T04:35:02.428 INFO:tasks.workunit.client.1.smithi053.stdout:2/135: write d1/d18/f2a [118744,43923] 0 2024-09-21T04:35:02.434 INFO:tasks.workunit.client.1.smithi053.stdout:1/229: truncate d7/d9/da/d11/f20 2548126 0 2024-09-21T04:35:02.441 INFO:tasks.workunit.client.1.smithi053.stdout:8/133: symlink d3/df/d10/l2c 0 2024-09-21T04:35:02.443 INFO:tasks.workunit.client.1.smithi053.stdout:1/230: rename d7/d9/l2d to d7/d24/d2a/d36/l44 0 2024-09-21T04:35:02.443 INFO:tasks.workunit.client.1.smithi053.stdout:5/176: link d0/l11 d0/d8/d2a/l34 0 2024-09-21T04:35:02.444 INFO:tasks.workunit.client.1.smithi053.stdout:3/119: sync 2024-09-21T04:35:02.444 INFO:tasks.workunit.client.1.smithi053.stdout:5/177: stat d0/d8/f31 0 2024-09-21T04:35:02.447 INFO:tasks.workunit.client.1.smithi053.stdout:8/134: symlink d3/d5/d21/d26/l2d 0 2024-09-21T04:35:02.448 INFO:tasks.workunit.client.1.smithi053.stdout:2/136: fsync d1/f5 0 2024-09-21T04:35:02.452 INFO:tasks.workunit.client.1.smithi053.stdout:6/143: truncate d2/d9/de/f15 1889019 0 2024-09-21T04:35:02.452 INFO:tasks.workunit.client.1.smithi053.stdout:2/137: chown d1/fb 13 1 2024-09-21T04:35:02.453 INFO:tasks.workunit.client.1.smithi053.stdout:1/231: dwrite f3 [0,4194304] 0 2024-09-21T04:35:02.459 INFO:tasks.workunit.client.1.smithi053.stdout:3/120: dwrite d16/d17/f1e [0,4194304] 0 2024-09-21T04:35:02.459 INFO:tasks.workunit.client.1.smithi053.stdout:5/178: dwrite d0/dd/f23 [0,4194304] 0 2024-09-21T04:35:02.459 INFO:tasks.workunit.client.1.smithi053.stdout:6/144: fsync d2/d9/de/f1f 0 2024-09-21T04:35:02.461 INFO:tasks.workunit.client.1.smithi053.stdout:5/179: stat d0/dd/d30 0 2024-09-21T04:35:02.462 INFO:tasks.workunit.client.1.smithi053.stdout:1/232: chown d7/d9/da/d11/c18 69 1 2024-09-21T04:35:02.467 INFO:tasks.workunit.client.1.smithi053.stdout:6/145: mknod d2/d9/de/c2d 0 2024-09-21T04:35:02.467 INFO:tasks.workunit.client.1.smithi053.stdout:5/180: mkdir d0/dd/d30/d35 0 2024-09-21T04:35:02.467 INFO:tasks.workunit.client.1.smithi053.stdout:2/138: dwrite d1/f12 [0,4194304] 0 2024-09-21T04:35:02.470 INFO:tasks.workunit.client.1.smithi053.stdout:3/121: mkdir d16/d17/d21/d26 0 2024-09-21T04:35:02.471 INFO:tasks.workunit.client.1.smithi053.stdout:6/146: mkdir d2/d9/d1a/d2e 0 2024-09-21T04:35:02.472 INFO:tasks.workunit.client.1.smithi053.stdout:5/181: rmdir d0/d8/d2a 39 2024-09-21T04:35:02.473 INFO:tasks.workunit.client.1.smithi053.stdout:3/122: chown d16/d17/l25 264372558 1 2024-09-21T04:35:02.474 INFO:tasks.workunit.client.1.smithi053.stdout:2/139: symlink d1/d18/l2f 0 2024-09-21T04:35:02.475 INFO:tasks.workunit.client.1.smithi053.stdout:1/233: mknod d7/c45 0 2024-09-21T04:35:02.476 INFO:tasks.workunit.client.1.smithi053.stdout:1/234: stat d7/d24/d2a/d36/c37 0 2024-09-21T04:35:02.476 INFO:tasks.workunit.client.1.smithi053.stdout:8/135: dwrite d3/d5/f19 [0,4194304] 0 2024-09-21T04:35:02.479 INFO:tasks.workunit.client.1.smithi053.stdout:6/147: creat d2/d1b/f2f x:0 0 0 2024-09-21T04:35:02.480 INFO:tasks.workunit.client.1.smithi053.stdout:2/140: rmdir d1 39 2024-09-21T04:35:02.482 INFO:tasks.workunit.client.1.smithi053.stdout:3/123: creat d16/d1b/f27 x:0 0 0 2024-09-21T04:35:02.483 INFO:tasks.workunit.client.1.smithi053.stdout:3/124: write f9 [1812954,2513] 0 2024-09-21T04:35:02.494 INFO:tasks.workunit.client.1.smithi053.stdout:8/136: rename d3/df/f11 to d3/d28/f2e 0 2024-09-21T04:35:02.497 INFO:tasks.workunit.client.1.smithi053.stdout:6/148: truncate d2/d9/de/f14 103301 0 2024-09-21T04:35:02.497 INFO:tasks.workunit.client.1.smithi053.stdout:9/167: sync 2024-09-21T04:35:02.497 INFO:tasks.workunit.client.1.smithi053.stdout:0/83: sync 2024-09-21T04:35:02.497 INFO:tasks.workunit.client.1.smithi053.stdout:7/115: sync 2024-09-21T04:35:02.497 INFO:tasks.workunit.client.1.smithi053.stdout:2/141: rmdir d1/d18/d1f/d22 39 2024-09-21T04:35:02.499 INFO:tasks.workunit.client.1.smithi053.stdout:2/142: write d1/f12 [2595845,33287] 0 2024-09-21T04:35:02.500 INFO:tasks.workunit.client.1.smithi053.stdout:0/84: mkdir d5/de/d1c 0 2024-09-21T04:35:02.500 INFO:tasks.workunit.client.1.smithi053.stdout:3/125: link d16/c19 d16/d17/d21/d26/c28 0 2024-09-21T04:35:02.501 INFO:tasks.workunit.client.1.smithi053.stdout:1/235: dread d7/d9/f19 [0,4194304] 0 2024-09-21T04:35:02.503 INFO:tasks.workunit.client.1.smithi053.stdout:6/149: mkdir d2/d1b/d30 0 2024-09-21T04:35:02.506 INFO:tasks.workunit.client.1.smithi053.stdout:5/182: getdents d0/d8/d2a 0 2024-09-21T04:35:02.508 INFO:tasks.workunit.client.1.smithi053.stdout:8/137: truncate d3/df/d14/d16/f1d 427424 0 2024-09-21T04:35:02.509 INFO:tasks.workunit.client.1.smithi053.stdout:2/143: unlink d1/f4 0 2024-09-21T04:35:02.509 INFO:tasks.workunit.client.1.smithi053.stdout:9/168: dwrite d0/d1a/f2c [0,4194304] 0 2024-09-21T04:35:02.513 INFO:tasks.workunit.client.1.smithi053.stdout:0/85: dread d5/dd/ff [0,4194304] 0 2024-09-21T04:35:02.514 INFO:tasks.workunit.client.1.smithi053.stdout:3/126: dwrite d16/d1b/f1f [0,4194304] 0 2024-09-21T04:35:02.516 INFO:tasks.workunit.client.1.smithi053.stdout:5/183: chown d0/d8/db/l2c 432702 1 2024-09-21T04:35:02.517 INFO:tasks.workunit.client.1.smithi053.stdout:8/138: creat d3/d5/d21/d26/f2f x:0 0 0 2024-09-21T04:35:02.518 INFO:tasks.workunit.client.1.smithi053.stdout:1/236: dwrite f0 [0,4194304] 0 2024-09-21T04:35:02.519 INFO:tasks.workunit.client.1.smithi053.stdout:5/184: chown d0/d8/db/f29 4 1 2024-09-21T04:35:02.520 INFO:tasks.workunit.client.1.smithi053.stdout:8/139: dread - d3/d5/d21/d26/f27 zero size 2024-09-21T04:35:02.523 INFO:tasks.workunit.client.1.smithi053.stdout:9/169: rename d0/d2/d9/d1f/f2f to d0/d1c/f39 0 2024-09-21T04:35:02.525 INFO:tasks.workunit.client.1.smithi053.stdout:9/170: readlink d0/l3 0 2024-09-21T04:35:02.527 INFO:tasks.workunit.client.1.smithi053.stdout:3/127: mknod d16/c29 0 2024-09-21T04:35:02.529 INFO:tasks.workunit.client.1.smithi053.stdout:3/128: chown d16/d17/d21 480937975 1 2024-09-21T04:35:02.530 INFO:tasks.workunit.client.1.smithi053.stdout:4/141: sync 2024-09-21T04:35:02.531 INFO:tasks.workunit.client.1.smithi053.stdout:5/185: dwrite d0/f2f [0,4194304] 0 2024-09-21T04:35:02.533 INFO:tasks.workunit.client.1.smithi053.stdout:6/150: truncate d2/d9/de/f1f 977485 0 2024-09-21T04:35:02.534 INFO:tasks.workunit.client.1.smithi053.stdout:5/186: write d0/f2f [1072196,75735] 0 2024-09-21T04:35:02.534 INFO:tasks.workunit.client.1.smithi053.stdout:7/116: dwrite d0/fd [0,4194304] 0 2024-09-21T04:35:02.535 INFO:tasks.workunit.client.1.smithi053.stdout:7/117: stat d0/f20 0 2024-09-21T04:35:02.536 INFO:tasks.workunit.client.1.smithi053.stdout:8/140: rmdir d3/d5/d21/d26 39 2024-09-21T04:35:02.536 INFO:tasks.workunit.client.1.smithi053.stdout:7/118: fsync d0/d6/f8 0 2024-09-21T04:35:02.537 INFO:tasks.workunit.client.1.smithi053.stdout:8/141: chown d3/d5/c2b 97 1 2024-09-21T04:35:02.538 INFO:tasks.workunit.client.1.smithi053.stdout:2/144: mknod d1/d18/d1f/d27/c30 0 2024-09-21T04:35:02.544 INFO:tasks.workunit.client.1.smithi053.stdout:9/171: creat d0/d28/f3a x:0 0 0 2024-09-21T04:35:02.545 INFO:tasks.workunit.client.1.smithi053.stdout:9/172: write d0/f24 [186314,83522] 0 2024-09-21T04:35:02.548 INFO:tasks.workunit.client.1.smithi053.stdout:1/237: write d7/d9/da/f2f [93095,60024] 0 2024-09-21T04:35:02.549 INFO:tasks.workunit.client.1.smithi053.stdout:4/142: fsync d1/fc 0 2024-09-21T04:35:02.550 INFO:tasks.workunit.client.1.smithi053.stdout:4/143: chown d1/dd/l12 2833 1 2024-09-21T04:35:02.550 INFO:tasks.workunit.client.1.smithi053.stdout:4/144: write d1/f4 [812089,18020] 0 2024-09-21T04:35:02.554 INFO:tasks.workunit.client.1.smithi053.stdout:5/187: unlink d0/d8/c2d 0 2024-09-21T04:35:02.556 INFO:tasks.workunit.client.1.smithi053.stdout:8/142: rmdir d3/df 39 2024-09-21T04:35:02.556 INFO:tasks.workunit.client.1.smithi053.stdout:2/145: creat d1/d18/f31 x:0 0 0 2024-09-21T04:35:02.557 INFO:tasks.workunit.client.1.smithi053.stdout:2/146: chown d1/f5 3469311 1 2024-09-21T04:35:02.560 INFO:tasks.workunit.client.1.smithi053.stdout:5/188: dwrite d0/d8/db/f29 [0,4194304] 0 2024-09-21T04:35:02.563 INFO:tasks.workunit.client.1.smithi053.stdout:1/238: rename f5 to d7/d9/da/d3c/f46 0 2024-09-21T04:35:02.564 INFO:tasks.workunit.client.1.smithi053.stdout:1/239: stat d7/l38 0 2024-09-21T04:35:02.565 INFO:tasks.workunit.client.1.smithi053.stdout:4/145: creat d1/dd/de/f2b x:0 0 0 2024-09-21T04:35:02.566 INFO:tasks.workunit.client.1.smithi053.stdout:6/151: symlink d2/d1b/d1e/l31 0 2024-09-21T04:35:02.566 INFO:tasks.workunit.client.1.smithi053.stdout:4/146: write d1/f4 [103624,18895] 0 2024-09-21T04:35:02.567 INFO:tasks.workunit.client.1.smithi053.stdout:4/147: stat d1/dd/f1d 0 2024-09-21T04:35:02.568 INFO:tasks.workunit.client.1.smithi053.stdout:0/86: link d5/de/c18 d5/c1d 0 2024-09-21T04:35:02.571 INFO:tasks.workunit.client.1.smithi053.stdout:5/189: mknod d0/c36 0 2024-09-21T04:35:02.572 INFO:tasks.workunit.client.1.smithi053.stdout:3/129: creat d16/d17/f2a x:0 0 0 2024-09-21T04:35:02.574 INFO:tasks.workunit.client.1.smithi053.stdout:6/152: mkdir d2/d9/de/d32 0 2024-09-21T04:35:02.576 INFO:tasks.workunit.client.1.smithi053.stdout:7/119: creat d0/f23 x:0 0 0 2024-09-21T04:35:02.577 INFO:tasks.workunit.client.1.smithi053.stdout:8/143: mknod d3/df/c30 0 2024-09-21T04:35:02.578 INFO:tasks.workunit.client.1.smithi053.stdout:3/130: dwrite d16/f1d [0,4194304] 0 2024-09-21T04:35:02.581 INFO:tasks.workunit.client.1.smithi053.stdout:2/147: rename d1/f9 to d1/d18/d1f/d22/f32 0 2024-09-21T04:35:02.581 INFO:tasks.workunit.client.1.smithi053.stdout:5/190: creat d0/dd/f37 x:0 0 0 2024-09-21T04:35:02.583 INFO:tasks.workunit.client.1.smithi053.stdout:5/191: chown d0/dd/l1e 52499126 1 2024-09-21T04:35:02.583 INFO:tasks.workunit.client.1.smithi053.stdout:1/240: mkdir d7/d47 0 2024-09-21T04:35:02.585 INFO:tasks.workunit.client.1.smithi053.stdout:3/131: dread fe [0,4194304] 0 2024-09-21T04:35:02.588 INFO:tasks.workunit.client.1.smithi053.stdout:2/148: dread d1/d18/f1c [0,4194304] 0 2024-09-21T04:35:02.589 INFO:tasks.workunit.client.1.smithi053.stdout:9/173: getdents d0/d1c/d21 0 2024-09-21T04:35:02.589 INFO:tasks.workunit.client.1.smithi053.stdout:9/174: stat d0/d2/fa 0 2024-09-21T04:35:02.591 INFO:tasks.workunit.client.1.smithi053.stdout:8/144: stat d3/d5/l8 0 2024-09-21T04:35:02.592 INFO:tasks.workunit.client.1.smithi053.stdout:8/145: write d3/df/d14/d16/f23 [271125,74515] 0 2024-09-21T04:35:02.596 INFO:tasks.workunit.client.1.smithi053.stdout:6/153: symlink d2/d1b/d30/l33 0 2024-09-21T04:35:02.596 INFO:tasks.workunit.client.1.smithi053.stdout:6/154: read - d2/f28 zero size 2024-09-21T04:35:02.598 INFO:tasks.workunit.client.1.smithi053.stdout:5/192: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:35:02.599 INFO:tasks.workunit.client.1.smithi053.stdout:7/120: mkdir d0/d24 0 2024-09-21T04:35:02.602 INFO:tasks.workunit.client.1.smithi053.stdout:6/155: dwrite d2/fb [0,4194304] 0 2024-09-21T04:35:02.604 INFO:tasks.workunit.client.1.smithi053.stdout:2/149: creat d1/d18/d1f/d22/f33 x:0 0 0 2024-09-21T04:35:02.608 INFO:tasks.workunit.client.1.smithi053.stdout:8/146: creat d3/d28/f31 x:0 0 0 2024-09-21T04:35:02.610 INFO:tasks.workunit.client.1.smithi053.stdout:8/147: read d3/df/d14/d16/f23 [4109827,11169] 0 2024-09-21T04:35:02.611 INFO:tasks.workunit.client.1.smithi053.stdout:0/87: dwrite d5/dd/f17 [0,4194304] 0 2024-09-21T04:35:02.611 INFO:tasks.workunit.client.1.smithi053.stdout:3/132: mkdir d16/d17/d21/d26/d2b 0 2024-09-21T04:35:02.612 INFO:tasks.workunit.client.1.smithi053.stdout:3/133: chown d16/f1c 7635 1 2024-09-21T04:35:02.613 INFO:tasks.workunit.client.1.smithi053.stdout:4/148: getdents d1 0 2024-09-21T04:35:02.621 INFO:tasks.workunit.client.1.smithi053.stdout:9/175: rename d0/d1a/c32 to d0/d2/d2b/c3b 0 2024-09-21T04:35:02.621 INFO:tasks.workunit.client.1.smithi053.stdout:9/176: write d0/f15 [2335778,2348] 0 2024-09-21T04:35:02.622 INFO:tasks.workunit.client.1.smithi053.stdout:9/177: dread - d0/df/f27 zero size 2024-09-21T04:35:02.625 INFO:tasks.workunit.client.1.smithi053.stdout:1/241: creat d7/d9/da/d11/f48 x:0 0 0 2024-09-21T04:35:02.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:02 smithi007 ceph-mon[27954]: pgmap v22: 65 pgs: 65 active+clean; 252 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 368 op/s 2024-09-21T04:35:02.627 INFO:tasks.workunit.client.1.smithi053.stdout:8/148: creat d3/d5/d1a/f32 x:0 0 0 2024-09-21T04:35:02.631 INFO:tasks.workunit.client.1.smithi053.stdout:4/149: mkdir d1/d2c 0 2024-09-21T04:35:02.633 INFO:tasks.workunit.client.1.smithi053.stdout:8/149: dwrite d3/d5/d1a/f32 [0,4194304] 0 2024-09-21T04:35:02.636 INFO:tasks.workunit.client.1.smithi053.stdout:2/150: mknod d1/d18/d1f/c34 0 2024-09-21T04:35:02.640 INFO:tasks.workunit.client.1.smithi053.stdout:8/150: dwrite d3/df/d14/d16/f29 [0,4194304] 0 2024-09-21T04:35:02.642 INFO:tasks.workunit.client.1.smithi053.stdout:9/178: readlink d0/d1a/l22 0 2024-09-21T04:35:02.654 INFO:tasks.workunit.client.1.smithi053.stdout:6/156: write d2/d9/de/f1f [1868529,15848] 0 2024-09-21T04:35:02.658 INFO:tasks.workunit.client.1.smithi053.stdout:2/151: dread d1/d18/d1f/d22/f32 [4194304,4194304] 0 2024-09-21T04:35:02.660 INFO:tasks.workunit.client.1.smithi053.stdout:0/88: truncate f1 2787444 0 2024-09-21T04:35:02.661 INFO:tasks.workunit.client.1.smithi053.stdout:0/89: write d5/d6/f16 [595595,111088] 0 2024-09-21T04:35:02.661 INFO:tasks.workunit.client.1.smithi053.stdout:9/179: stat d0/d1c/l20 0 2024-09-21T04:35:02.662 INFO:tasks.workunit.client.1.smithi053.stdout:1/242: mkdir d7/d9/da/d40/d49 0 2024-09-21T04:35:02.663 INFO:tasks.workunit.client.1.smithi053.stdout:1/243: chown d7/d9/f19 7058823 1 2024-09-21T04:35:02.665 INFO:tasks.workunit.client.1.smithi053.stdout:4/150: chown d1/d27 6597183 1 2024-09-21T04:35:02.668 INFO:tasks.workunit.client.1.smithi053.stdout:6/157: creat d2/d1b/d1e/d2a/f34 x:0 0 0 2024-09-21T04:35:02.669 INFO:tasks.workunit.client.1.smithi053.stdout:2/152: creat d1/d18/f35 x:0 0 0 2024-09-21T04:35:02.670 INFO:tasks.workunit.client.1.smithi053.stdout:8/151: mknod d3/d5/c33 0 2024-09-21T04:35:02.671 INFO:tasks.workunit.client.1.smithi053.stdout:2/153: write d1/d18/d1f/d22/f33 [105784,89084] 0 2024-09-21T04:35:02.674 INFO:tasks.workunit.client.1.smithi053.stdout:1/244: mknod d7/d9/da/d3c/c4a 0 2024-09-21T04:35:02.674 INFO:tasks.workunit.client.1.smithi053.stdout:3/134: getdents d16/d17 0 2024-09-21T04:35:02.676 INFO:tasks.workunit.client.1.smithi053.stdout:5/193: rename d0/dd/l1c to d0/d8/l38 0 2024-09-21T04:35:02.677 INFO:tasks.workunit.client.1.smithi053.stdout:4/151: stat d1/l5 0 2024-09-21T04:35:02.678 INFO:tasks.workunit.client.1.smithi053.stdout:5/194: fsync d0/dd/f23 0 2024-09-21T04:35:02.678 INFO:tasks.workunit.client.1.smithi053.stdout:6/158: rmdir d2/d9/d1a 39 2024-09-21T04:35:02.683 INFO:tasks.workunit.client.1.smithi053.stdout:3/135: symlink d16/d1b/l2c 0 2024-09-21T04:35:02.688 INFO:tasks.workunit.client.1.smithi053.stdout:0/90: rename d5/dd/l19 to d5/de/d1c/l1e 0 2024-09-21T04:35:02.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:02 smithi053 ceph-mon[35554]: pgmap v22: 65 pgs: 65 active+clean; 252 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 368 op/s 2024-09-21T04:35:02.694 INFO:tasks.workunit.client.1.smithi053.stdout:5/195: symlink d0/d8/d2a/l39 0 2024-09-21T04:35:02.700 INFO:tasks.workunit.client.1.smithi053.stdout:5/196: dwrite d0/f14 [0,4194304] 0 2024-09-21T04:35:02.700 INFO:tasks.workunit.client.1.smithi053.stdout:9/180: link d0/d1a/c2a d0/d2/c3c 0 2024-09-21T04:35:02.701 INFO:tasks.workunit.client.1.smithi053.stdout:9/181: chown d0/f15 4281 1 2024-09-21T04:35:02.701 INFO:tasks.workunit.client.1.smithi053.stdout:2/154: write d1/d18/f1c [2074016,124034] 0 2024-09-21T04:35:02.704 INFO:tasks.workunit.client.1.smithi053.stdout:3/136: rename d16/l1a to d16/d17/d21/d26/d2b/l2d 0 2024-09-21T04:35:02.707 INFO:tasks.workunit.client.1.smithi053.stdout:0/91: mknod d5/de/c1f 0 2024-09-21T04:35:02.710 INFO:tasks.workunit.client.1.smithi053.stdout:5/197: dwrite d0/f26 [0,4194304] 0 2024-09-21T04:35:02.711 INFO:tasks.workunit.client.1.smithi053.stdout:9/182: dwrite d0/d1a/f2c [0,4194304] 0 2024-09-21T04:35:02.712 INFO:tasks.workunit.client.1.smithi053.stdout:9/183: stat d0/df/f27 0 2024-09-21T04:35:02.714 INFO:tasks.workunit.client.1.smithi053.stdout:9/184: truncate d0/d2/d9/d1f/f26 2502443 0 2024-09-21T04:35:02.715 INFO:tasks.workunit.client.1.smithi053.stdout:3/137: mknod d16/d17/d21/c2e 0 2024-09-21T04:35:02.717 INFO:tasks.workunit.client.1.smithi053.stdout:4/152: creat d1/f2d x:0 0 0 2024-09-21T04:35:02.720 INFO:tasks.workunit.client.1.smithi053.stdout:5/198: creat d0/dd/f3a x:0 0 0 2024-09-21T04:35:02.723 INFO:tasks.workunit.client.1.smithi053.stdout:6/159: creat d2/d9/d1a/d2e/f35 x:0 0 0 2024-09-21T04:35:02.724 INFO:tasks.workunit.client.1.smithi053.stdout:8/152: dwrite d3/d28/f2e [0,4194304] 0 2024-09-21T04:35:02.726 INFO:tasks.workunit.client.1.smithi053.stdout:4/153: write d1/dd/f1d [774670,46804] 0 2024-09-21T04:35:02.727 INFO:tasks.workunit.client.1.smithi053.stdout:5/199: stat d0/d8/l27 0 2024-09-21T04:35:02.731 INFO:tasks.workunit.client.1.smithi053.stdout:9/185: dread d0/d2/d9/f19 [0,4194304] 0 2024-09-21T04:35:02.731 INFO:tasks.workunit.client.1.smithi053.stdout:8/153: rmdir d3/d5/d1a 39 2024-09-21T04:35:02.731 INFO:tasks.workunit.client.1.smithi053.stdout:8/154: stat d3/cb 0 2024-09-21T04:35:02.731 INFO:tasks.workunit.client.1.smithi053.stdout:5/200: chown d0/l1 5986003 1 2024-09-21T04:35:02.731 INFO:tasks.workunit.client.1.smithi053.stdout:5/201: fdatasync d0/dd/f3a 0 2024-09-21T04:35:02.732 INFO:tasks.workunit.client.1.smithi053.stdout:0/92: rmdir d5/dd/d13 0 2024-09-21T04:35:02.738 INFO:tasks.workunit.client.1.smithi053.stdout:0/93: chown d5/d6/f1b 18332 1 2024-09-21T04:35:02.739 INFO:tasks.workunit.client.1.smithi053.stdout:6/160: rename f1 to d2/d1b/d30/f36 0 2024-09-21T04:35:02.739 INFO:tasks.workunit.client.1.smithi053.stdout:5/202: write d0/dd/f37 [821073,67956] 0 2024-09-21T04:35:02.740 INFO:tasks.workunit.client.1.smithi053.stdout:6/161: fdatasync d2/fb 0 2024-09-21T04:35:02.741 INFO:tasks.workunit.client.1.smithi053.stdout:0/94: write d5/f15 [245641,24620] 0 2024-09-21T04:35:02.743 INFO:tasks.workunit.client.1.smithi053.stdout:0/95: fsync d5/dd/ff 0 2024-09-21T04:35:02.744 INFO:tasks.workunit.client.1.smithi053.stdout:4/154: dwrite d1/f2d [0,4194304] 0 2024-09-21T04:35:02.744 INFO:tasks.workunit.client.1.smithi053.stdout:4/155: stat d1/dd/de/c1e 0 2024-09-21T04:35:02.759 INFO:tasks.workunit.client.1.smithi053.stdout:3/138: getdents d16/d17/d21/d26/d2b 0 2024-09-21T04:35:02.764 INFO:tasks.workunit.client.1.smithi053.stdout:5/203: rename d0/dd/f24 to d0/f3b 0 2024-09-21T04:35:02.765 INFO:tasks.workunit.client.1.smithi053.stdout:7/121: sync 2024-09-21T04:35:02.771 INFO:tasks.workunit.client.1.smithi053.stdout:3/139: creat d16/d1b/f2f x:0 0 0 2024-09-21T04:35:02.772 INFO:tasks.workunit.client.1.smithi053.stdout:8/155: creat d3/f34 x:0 0 0 2024-09-21T04:35:02.774 INFO:tasks.workunit.client.1.smithi053.stdout:6/162: mknod d2/d9/c37 0 2024-09-21T04:35:02.774 INFO:tasks.workunit.client.1.smithi053.stdout:8/156: stat d3/d28/f2e 0 2024-09-21T04:35:02.776 INFO:tasks.workunit.client.1.smithi053.stdout:4/156: dwrite d1/fc [0,4194304] 0 2024-09-21T04:35:02.780 INFO:tasks.workunit.client.1.smithi053.stdout:6/163: write d2/d1b/d1e/d2a/f34 [402602,73849] 0 2024-09-21T04:35:02.781 INFO:tasks.workunit.client.1.smithi053.stdout:7/122: dread d0/d6/f9 [0,4194304] 0 2024-09-21T04:35:02.782 INFO:tasks.workunit.client.1.smithi053.stdout:5/204: getdents d0/dd/d30/d35 0 2024-09-21T04:35:02.783 INFO:tasks.workunit.client.1.smithi053.stdout:8/157: creat d3/df/d14/d16/f35 x:0 0 0 2024-09-21T04:35:02.785 INFO:tasks.workunit.client.1.smithi053.stdout:3/140: dwrite fa [0,4194304] 0 2024-09-21T04:35:02.787 INFO:tasks.workunit.client.1.smithi053.stdout:3/141: chown d16/d17/d21/d26 72 1 2024-09-21T04:35:02.790 INFO:tasks.workunit.client.1.smithi053.stdout:5/205: symlink d0/d8/d2a/l3c 0 2024-09-21T04:35:02.790 INFO:tasks.workunit.client.1.smithi053.stdout:4/157: dwrite d1/dd/f23 [0,4194304] 0 2024-09-21T04:35:02.791 INFO:tasks.workunit.client.1.smithi053.stdout:0/96: getdents d5/de 0 2024-09-21T04:35:02.792 INFO:tasks.workunit.client.1.smithi053.stdout:0/97: readlink d5/db/l9 0 2024-09-21T04:35:02.792 INFO:tasks.workunit.client.1.smithi053.stdout:5/206: chown d0/d8/db/cf 4 1 2024-09-21T04:35:02.793 INFO:tasks.workunit.client.1.smithi053.stdout:7/123: symlink d0/d6/l25 0 2024-09-21T04:35:02.793 INFO:tasks.workunit.client.1.smithi053.stdout:0/98: fdatasync d5/f15 0 2024-09-21T04:35:02.794 INFO:tasks.workunit.client.1.smithi053.stdout:3/142: dread d16/f1c [0,4194304] 0 2024-09-21T04:35:02.796 INFO:tasks.workunit.client.1.smithi053.stdout:3/143: fdatasync d16/d1b/f24 0 2024-09-21T04:35:02.797 INFO:tasks.workunit.client.1.smithi053.stdout:8/158: dread d3/df/d14/d16/f23 [0,4194304] 0 2024-09-21T04:35:02.797 INFO:tasks.workunit.client.1.smithi053.stdout:6/164: rename d2/d1b/d30 to d2/d9/d1a/d2e/d38 0 2024-09-21T04:35:02.799 INFO:tasks.workunit.client.1.smithi053.stdout:4/158: chown d1/dd/de/c18 705650 1 2024-09-21T04:35:02.800 INFO:tasks.workunit.client.1.smithi053.stdout:5/207: mknod d0/dd/c3d 0 2024-09-21T04:35:02.802 INFO:tasks.workunit.client.1.smithi053.stdout:3/144: rmdir d16 39 2024-09-21T04:35:02.805 INFO:tasks.workunit.client.1.smithi053.stdout:8/159: write d3/d5/d21/d26/f27 [696787,89449] 0 2024-09-21T04:35:02.806 INFO:tasks.workunit.client.1.smithi053.stdout:6/165: symlink d2/d9/d1a/d2e/d38/l39 0 2024-09-21T04:35:02.807 INFO:tasks.workunit.client.1.smithi053.stdout:7/124: symlink d0/d24/l26 0 2024-09-21T04:35:02.809 INFO:tasks.workunit.client.1.smithi053.stdout:7/125: fdatasync d0/fd 0 2024-09-21T04:35:02.811 INFO:tasks.workunit.client.1.smithi053.stdout:3/145: chown d16/d1b/f1f 177 1 2024-09-21T04:35:02.812 INFO:tasks.workunit.client.1.smithi053.stdout:0/99: truncate d5/dd/f17 3328523 0 2024-09-21T04:35:02.813 INFO:tasks.workunit.client.1.smithi053.stdout:6/166: dwrite d2/d9/d1a/f21 [4194304,4194304] 0 2024-09-21T04:35:02.820 INFO:tasks.workunit.client.1.smithi053.stdout:7/126: symlink d0/d24/l27 0 2024-09-21T04:35:02.822 INFO:tasks.workunit.client.1.smithi053.stdout:3/146: truncate fe 4332281 0 2024-09-21T04:35:02.825 INFO:tasks.workunit.client.1.smithi053.stdout:8/160: rename d3/d5/c1c to d3/df/d14/d16/c36 0 2024-09-21T04:35:02.826 INFO:tasks.workunit.client.1.smithi053.stdout:8/161: fsync d3/d5/f25 0 2024-09-21T04:35:02.827 INFO:tasks.workunit.client.1.smithi053.stdout:6/167: creat d2/d1b/f3a x:0 0 0 2024-09-21T04:35:02.829 INFO:tasks.workunit.client.1.smithi053.stdout:4/159: mkdir d1/d27/d2e 0 2024-09-21T04:35:02.830 INFO:tasks.workunit.client.1.smithi053.stdout:5/208: link d0/d8/l38 d0/dd/d30/l3e 0 2024-09-21T04:35:02.832 INFO:tasks.workunit.client.1.smithi053.stdout:7/127: symlink d0/d6/l28 0 2024-09-21T04:35:02.833 INFO:tasks.workunit.client.1.smithi053.stdout:3/147: rename d16/f1c to d16/f30 0 2024-09-21T04:35:02.834 INFO:tasks.workunit.client.1.smithi053.stdout:0/100: symlink d5/d10/l20 0 2024-09-21T04:35:02.836 INFO:tasks.workunit.client.1.smithi053.stdout:1/245: sync 2024-09-21T04:35:02.836 INFO:tasks.workunit.client.1.smithi053.stdout:8/162: symlink d3/df/d10/l37 0 2024-09-21T04:35:02.838 INFO:tasks.workunit.client.1.smithi053.stdout:6/168: unlink d2/d1d/c25 0 2024-09-21T04:35:02.839 INFO:tasks.workunit.client.1.smithi053.stdout:6/169: truncate d2/d1b/f3a 172690 0 2024-09-21T04:35:02.839 INFO:tasks.workunit.client.1.smithi053.stdout:6/170: stat d2/l22 0 2024-09-21T04:35:02.840 INFO:tasks.workunit.client.1.smithi053.stdout:6/171: dread - d2/d1b/f2f zero size 2024-09-21T04:35:02.840 INFO:tasks.workunit.client.1.smithi053.stdout:6/172: chown d2/d1b/f2f 16 1 2024-09-21T04:35:02.843 INFO:tasks.workunit.client.1.smithi053.stdout:7/128: rename d0/f11 to d0/d6/f29 0 2024-09-21T04:35:02.844 INFO:tasks.workunit.client.1.smithi053.stdout:1/246: fsync d7/d9/da/d3c/f46 0 2024-09-21T04:35:02.850 INFO:tasks.workunit.client.1.smithi053.stdout:3/148: link d16/d1b/f27 d16/d17/f31 0 2024-09-21T04:35:02.854 INFO:tasks.workunit.client.1.smithi053.stdout:8/163: mkdir d3/d5/d38 0 2024-09-21T04:35:02.854 INFO:tasks.workunit.client.1.smithi053.stdout:8/164: chown d3/d5/c2b 261262209 1 2024-09-21T04:35:02.855 INFO:tasks.workunit.client.1.smithi053.stdout:1/247: symlink d7/d24/d2a/l4b 0 2024-09-21T04:35:02.860 INFO:tasks.workunit.client.1.smithi053.stdout:3/149: readlink l2 0 2024-09-21T04:35:02.865 INFO:tasks.workunit.client.1.smithi053.stdout:2/155: sync 2024-09-21T04:35:02.865 INFO:tasks.workunit.client.1.smithi053.stdout:9/186: sync 2024-09-21T04:35:02.866 INFO:tasks.workunit.client.1.smithi053.stdout:9/187: truncate d0/d2/d9/f19 4901814 0 2024-09-21T04:35:02.868 INFO:tasks.workunit.client.1.smithi053.stdout:4/160: truncate d1/f2d 913821 0 2024-09-21T04:35:02.869 INFO:tasks.workunit.client.1.smithi053.stdout:5/209: getdents d0/dd/d30 0 2024-09-21T04:35:02.871 INFO:tasks.workunit.client.1.smithi053.stdout:6/173: mknod d2/d9/de/d32/c3b 0 2024-09-21T04:35:02.873 INFO:tasks.workunit.client.1.smithi053.stdout:0/101: getdents d5 0 2024-09-21T04:35:02.875 INFO:tasks.workunit.client.1.smithi053.stdout:2/156: rmdir d1 39 2024-09-21T04:35:02.877 INFO:tasks.workunit.client.1.smithi053.stdout:9/188: symlink d0/d2/d2b/l3d 0 2024-09-21T04:35:02.877 INFO:tasks.workunit.client.1.smithi053.stdout:7/129: truncate d0/fb 2830437 0 2024-09-21T04:35:02.878 INFO:tasks.workunit.client.1.smithi053.stdout:1/248: mkdir d7/d9/da/d4c 0 2024-09-21T04:35:02.880 INFO:tasks.workunit.client.1.smithi053.stdout:5/210: mknod d0/dd/d30/c3f 0 2024-09-21T04:35:02.884 INFO:tasks.workunit.client.1.smithi053.stdout:2/157: read d1/d18/d1f/d22/f32 [4911919,10941] 0 2024-09-21T04:35:02.884 INFO:tasks.workunit.client.1.smithi053.stdout:8/165: rename d3/d5/d21/d26 to d3/d39 0 2024-09-21T04:35:02.886 INFO:tasks.workunit.client.1.smithi053.stdout:4/161: dwrite d1/f4 [0,4194304] 0 2024-09-21T04:35:02.890 INFO:tasks.workunit.client.1.smithi053.stdout:6/174: link d2/d9/de/f1f d2/d1b/d1e/d2a/f3c 0 2024-09-21T04:35:02.893 INFO:tasks.workunit.client.1.smithi053.stdout:5/211: truncate d0/d8/f10 353027 0 2024-09-21T04:35:02.895 INFO:tasks.workunit.client.1.smithi053.stdout:3/150: write f11 [60344,53878] 0 2024-09-21T04:35:02.895 INFO:tasks.workunit.client.1.smithi053.stdout:2/158: unlink d1/d18/d1f/c34 0 2024-09-21T04:35:02.898 INFO:tasks.workunit.client.1.smithi053.stdout:1/249: creat d7/d47/f4d x:0 0 0 2024-09-21T04:35:02.898 INFO:tasks.workunit.client.1.smithi053.stdout:1/250: chown d7/d9/da/c2c 5074 1 2024-09-21T04:35:02.899 INFO:tasks.workunit.client.1.smithi053.stdout:7/130: symlink d0/l2a 0 2024-09-21T04:35:02.901 INFO:tasks.workunit.client.1.smithi053.stdout:6/175: mkdir d2/d1b/d3d 0 2024-09-21T04:35:02.905 INFO:tasks.workunit.client.1.smithi053.stdout:1/251: creat d7/d24/d2a/d36/f4e x:0 0 0 2024-09-21T04:35:02.910 INFO:tasks.workunit.client.1.smithi053.stdout:5/212: dwrite d0/d8/f2b [0,4194304] 0 2024-09-21T04:35:02.911 INFO:tasks.workunit.client.1.smithi053.stdout:8/166: creat d3/f3a x:0 0 0 2024-09-21T04:35:02.912 INFO:tasks.workunit.client.1.smithi053.stdout:9/189: truncate d0/f4 3366311 0 2024-09-21T04:35:02.914 INFO:tasks.workunit.client.1.smithi053.stdout:5/213: write d0/d8/f1f [2487313,55380] 0 2024-09-21T04:35:02.914 INFO:tasks.workunit.client.1.smithi053.stdout:8/167: dread - d3/d5/f25 zero size 2024-09-21T04:35:02.917 INFO:tasks.workunit.client.1.smithi053.stdout:1/252: creat d7/d24/d2a/f4f x:0 0 0 2024-09-21T04:35:02.919 INFO:tasks.workunit.client.1.smithi053.stdout:0/102: dwrite d5/dd/f17 [0,4194304] 0 2024-09-21T04:35:02.919 INFO:tasks.workunit.client.1.smithi053.stdout:1/253: readlink d7/d9/da/lb 0 2024-09-21T04:35:02.920 INFO:tasks.workunit.client.1.smithi053.stdout:7/131: dwrite d0/fa [0,4194304] 0 2024-09-21T04:35:02.920 INFO:tasks.workunit.client.1.smithi053.stdout:6/176: mkdir d2/d1b/d3d/d3e 0 2024-09-21T04:35:02.924 INFO:tasks.workunit.client.1.smithi053.stdout:8/168: mknod d3/d28/c3b 0 2024-09-21T04:35:02.926 INFO:tasks.workunit.client.1.smithi053.stdout:5/214: creat d0/d8/d2a/f40 x:0 0 0 2024-09-21T04:35:02.928 INFO:tasks.workunit.client.1.smithi053.stdout:7/132: rmdir d0 39 2024-09-21T04:35:02.934 INFO:tasks.workunit.client.1.smithi053.stdout:0/103: rename d5/db to d5/d6/d21 0 2024-09-21T04:35:02.934 INFO:tasks.workunit.client.1.smithi053.stdout:3/151: getdents d16/d1b 0 2024-09-21T04:35:02.936 INFO:tasks.workunit.client.1.smithi053.stdout:9/190: getdents d0/d2/d2b/d35 0 2024-09-21T04:35:02.938 INFO:tasks.workunit.client.1.smithi053.stdout:9/191: read d0/d2/d9/fb [2651418,105824] 0 2024-09-21T04:35:02.941 INFO:tasks.workunit.client.1.smithi053.stdout:0/104: dwrite d5/f15 [0,4194304] 0 2024-09-21T04:35:02.941 INFO:tasks.workunit.client.1.smithi053.stdout:7/133: fdatasync d0/d6/f1e 0 2024-09-21T04:35:02.942 INFO:tasks.workunit.client.1.smithi053.stdout:0/105: chown d5/de/c1f 1864697 1 2024-09-21T04:35:02.943 INFO:tasks.workunit.client.1.smithi053.stdout:3/152: symlink d16/l32 0 2024-09-21T04:35:02.945 INFO:tasks.workunit.client.1.smithi053.stdout:0/106: write d5/dd/f17 [239739,74898] 0 2024-09-21T04:35:02.945 INFO:tasks.workunit.client.1.smithi053.stdout:3/153: write d16/f1d [107731,58745] 0 2024-09-21T04:35:02.947 INFO:tasks.workunit.client.1.smithi053.stdout:9/192: symlink d0/df/l3e 0 2024-09-21T04:35:02.948 INFO:tasks.workunit.client.1.smithi053.stdout:9/193: write d0/d28/f38 [675724,74080] 0 2024-09-21T04:35:02.949 INFO:tasks.workunit.client.1.smithi053.stdout:9/194: write d0/d2/fe [1408057,74629] 0 2024-09-21T04:35:02.950 INFO:tasks.workunit.client.1.smithi053.stdout:5/215: mknod d0/dd/d30/d35/c41 0 2024-09-21T04:35:02.953 INFO:tasks.workunit.client.1.smithi053.stdout:1/254: creat d7/d9/da/f50 x:0 0 0 2024-09-21T04:35:02.956 INFO:tasks.workunit.client.1.smithi053.stdout:9/195: dwrite d0/f24 [0,4194304] 0 2024-09-21T04:35:02.960 INFO:tasks.workunit.client.1.smithi053.stdout:0/107: symlink d5/d6/d21/l22 0 2024-09-21T04:35:02.970 INFO:tasks.workunit.client.1.smithi053.stdout:5/216: symlink d0/dd/d30/d35/l42 0 2024-09-21T04:35:02.975 INFO:tasks.workunit.client.1.smithi053.stdout:1/255: write d7/d9/f1b [4473865,69471] 0 2024-09-21T04:35:02.976 INFO:tasks.workunit.client.1.smithi053.stdout:1/256: stat d7/d9/da/d3c 0 2024-09-21T04:35:02.976 INFO:tasks.workunit.client.1.smithi053.stdout:7/134: mkdir d0/d2b 0 2024-09-21T04:35:02.977 INFO:tasks.workunit.client.1.smithi053.stdout:9/196: mknod d0/d1a/c3f 0 2024-09-21T04:35:02.979 INFO:tasks.workunit.client.1.smithi053.stdout:6/177: getdents d2/d1b 0 2024-09-21T04:35:02.980 INFO:tasks.workunit.client.1.smithi053.stdout:0/108: symlink d5/l23 0 2024-09-21T04:35:02.983 INFO:tasks.workunit.client.1.smithi053.stdout:8/169: link d3/d5/d1a/l20 d3/d5/d1a/l3c 0 2024-09-21T04:35:02.985 INFO:tasks.workunit.client.1.smithi053.stdout:7/135: rename d0/l17 to d0/d24/l2c 0 2024-09-21T04:35:02.986 INFO:tasks.workunit.client.1.smithi053.stdout:7/136: write d0/d6/f7 [88973,106938] 0 2024-09-21T04:35:02.987 INFO:tasks.workunit.client.1.smithi053.stdout:7/137: write d0/fa [3744276,7296] 0 2024-09-21T04:35:02.987 INFO:tasks.workunit.client.1.smithi053.stdout:7/138: chown d0/d6/ff 1031 1 2024-09-21T04:35:02.988 INFO:tasks.workunit.client.1.smithi053.stdout:0/109: unlink d5/dd/f17 0 2024-09-21T04:35:02.992 INFO:tasks.workunit.client.1.smithi053.stdout:7/139: dread d0/d6/f10 [0,4194304] 0 2024-09-21T04:35:02.993 INFO:tasks.workunit.client.1.smithi053.stdout:1/257: rename f3 to d7/d9/da/d11/d2b/f51 0 2024-09-21T04:35:02.997 INFO:tasks.workunit.client.1.smithi053.stdout:0/110: dwrite d5/f15 [4194304,4194304] 0 2024-09-21T04:35:02.998 INFO:tasks.workunit.client.1.smithi053.stdout:0/111: write d5/fa [1459230,24602] 0 2024-09-21T04:35:03.002 INFO:tasks.workunit.client.1.smithi053.stdout:7/140: dwrite d0/d6/f7 [0,4194304] 0 2024-09-21T04:35:03.014 INFO:tasks.workunit.client.1.smithi053.stdout:5/217: truncate d0/f7 160040 0 2024-09-21T04:35:03.015 INFO:tasks.workunit.client.1.smithi053.stdout:4/162: sync 2024-09-21T04:35:03.016 INFO:tasks.workunit.client.1.smithi053.stdout:2/159: sync 2024-09-21T04:35:03.018 INFO:tasks.workunit.client.1.smithi053.stdout:8/170: creat d3/d5/d38/f3d x:0 0 0 2024-09-21T04:35:03.018 INFO:tasks.workunit.client.1.smithi053.stdout:4/163: chown d1/dd/de/f2b 133738616 1 2024-09-21T04:35:03.021 INFO:tasks.workunit.client.1.smithi053.stdout:0/112: symlink d5/d6/l24 0 2024-09-21T04:35:03.022 INFO:tasks.workunit.client.1.smithi053.stdout:0/113: chown d5/d6 15841791 1 2024-09-21T04:35:03.022 INFO:tasks.workunit.client.1.smithi053.stdout:9/197: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:35:03.024 INFO:tasks.workunit.client.1.smithi053.stdout:0/114: write d5/dd/f1a [735047,87933] 0 2024-09-21T04:35:03.025 INFO:tasks.workunit.client.1.smithi053.stdout:6/178: creat d2/d1b/d1e/f3f x:0 0 0 2024-09-21T04:35:03.028 INFO:tasks.workunit.client.1.smithi053.stdout:3/154: link d16/d1b/c23 d16/d17/c33 0 2024-09-21T04:35:03.029 INFO:tasks.workunit.client.1.smithi053.stdout:4/164: dwrite d1/dd/f1d [0,4194304] 0 2024-09-21T04:35:03.029 INFO:tasks.workunit.client.1.smithi053.stdout:2/160: creat d1/d18/d1f/d22/f36 x:0 0 0 2024-09-21T04:35:03.034 INFO:tasks.workunit.client.1.smithi053.stdout:1/258: symlink d7/d9/da/d11/l52 0 2024-09-21T04:35:03.034 INFO:tasks.workunit.client.1.smithi053.stdout:2/161: dread - d1/d18/d1f/d22/f36 zero size 2024-09-21T04:35:03.036 INFO:tasks.workunit.client.1.smithi053.stdout:7/141: mknod d0/c2d 0 2024-09-21T04:35:03.038 INFO:tasks.workunit.client.1.smithi053.stdout:1/259: readlink d7/d9/da/d11/l1e 0 2024-09-21T04:35:03.039 INFO:tasks.workunit.client.1.smithi053.stdout:2/162: write d1/d18/d1f/d22/f33 [31052,89261] 0 2024-09-21T04:35:03.040 INFO:tasks.workunit.client.1.smithi053.stdout:9/198: rename d0/d2/d2b/l3d to d0/d2/d9/d1f/l40 0 2024-09-21T04:35:03.042 INFO:tasks.workunit.client.1.smithi053.stdout:1/260: fsync d7/d9/da/d11/f26 0 2024-09-21T04:35:03.043 INFO:tasks.workunit.client.1.smithi053.stdout:6/179: dwrite d2/d9/d1a/f26 [0,4194304] 0 2024-09-21T04:35:03.043 INFO:tasks.workunit.client.1.smithi053.stdout:2/163: fsync d1/d18/f35 0 2024-09-21T04:35:03.044 INFO:tasks.workunit.client.1.smithi053.stdout:3/155: dwrite fa [4194304,4194304] 0 2024-09-21T04:35:03.045 INFO:tasks.workunit.client.1.smithi053.stdout:2/164: chown d1 4218017 1 2024-09-21T04:35:03.046 INFO:tasks.workunit.client.1.smithi053.stdout:3/156: truncate f11 621692 0 2024-09-21T04:35:03.051 INFO:tasks.workunit.client.1.smithi053.stdout:0/115: mknod d5/dd/c25 0 2024-09-21T04:35:03.054 INFO:tasks.workunit.client.1.smithi053.stdout:1/261: dwrite d7/d9/da/d11/f26 [8388608,4194304] 0 2024-09-21T04:35:03.056 INFO:tasks.workunit.client.1.smithi053.stdout:9/199: rename d0/d2/c37 to d0/d2/c41 0 2024-09-21T04:35:03.056 INFO:tasks.workunit.client.1.smithi053.stdout:3/157: dwrite f9 [0,4194304] 0 2024-09-21T04:35:03.059 INFO:tasks.workunit.client.1.smithi053.stdout:8/171: link d3/d5/c33 d3/df/d10/c3e 0 2024-09-21T04:35:03.061 INFO:tasks.workunit.client.1.smithi053.stdout:1/262: dread - d7/d24/d2a/f4f zero size 2024-09-21T04:35:03.061 INFO:tasks.workunit.client.1.smithi053.stdout:9/200: fsync d0/d28/f38 0 2024-09-21T04:35:03.063 INFO:tasks.workunit.client.1.smithi053.stdout:5/218: getdents d0/d8 0 2024-09-21T04:35:03.065 INFO:tasks.workunit.client.1.smithi053.stdout:1/263: readlink d7/d9/da/d11/l42 0 2024-09-21T04:35:03.065 INFO:tasks.workunit.client.1.smithi053.stdout:4/165: getdents d1/d27/d2e 0 2024-09-21T04:35:03.070 INFO:tasks.workunit.client.1.smithi053.stdout:3/158: dwrite f9 [0,4194304] 0 2024-09-21T04:35:03.075 INFO:tasks.workunit.client.1.smithi053.stdout:3/159: write d16/d1b/f1f [1042958,29702] 0 2024-09-21T04:35:03.078 INFO:tasks.workunit.client.1.smithi053.stdout:8/172: creat d3/d39/f3f x:0 0 0 2024-09-21T04:35:03.079 INFO:tasks.workunit.client.1.smithi053.stdout:9/201: dwrite d0/d1a/f2c [4194304,4194304] 0 2024-09-21T04:35:03.085 INFO:tasks.workunit.client.1.smithi053.stdout:1/264: rmdir d7/d24/d2a 39 2024-09-21T04:35:03.088 INFO:tasks.workunit.client.1.smithi053.stdout:8/173: dwrite d3/d39/f3f [0,4194304] 0 2024-09-21T04:35:03.089 INFO:tasks.workunit.client.1.smithi053.stdout:4/166: mkdir d1/dd/de/d14/d2f 0 2024-09-21T04:35:03.095 INFO:tasks.workunit.client.1.smithi053.stdout:2/165: creat d1/d18/d1f/f37 x:0 0 0 2024-09-21T04:35:03.101 INFO:tasks.workunit.client.1.smithi053.stdout:1/265: symlink d7/d9/da/d40/l53 0 2024-09-21T04:35:03.102 INFO:tasks.workunit.client.1.smithi053.stdout:8/174: creat d3/d39/f40 x:0 0 0 2024-09-21T04:35:03.103 INFO:tasks.workunit.client.1.smithi053.stdout:4/167: unlink d1/dd/cf 0 2024-09-21T04:35:03.103 INFO:tasks.workunit.client.1.smithi053.stdout:1/266: fdatasync d7/d9/da/d11/f26 0 2024-09-21T04:35:03.104 INFO:tasks.workunit.client.1.smithi053.stdout:2/166: dwrite d1/d18/f35 [0,4194304] 0 2024-09-21T04:35:03.105 INFO:tasks.workunit.client.1.smithi053.stdout:2/167: chown d1/d18/f35 255 1 2024-09-21T04:35:03.105 INFO:tasks.workunit.client.1.smithi053.stdout:3/160: chown d16/d17/f1e 414324952 1 2024-09-21T04:35:03.106 INFO:tasks.workunit.client.1.smithi053.stdout:3/161: write d16/d1b/f24 [560015,79091] 0 2024-09-21T04:35:03.110 INFO:tasks.workunit.client.1.smithi053.stdout:5/219: getdents d0/dd 0 2024-09-21T04:35:03.110 INFO:tasks.workunit.client.1.smithi053.stdout:8/175: write d3/d5/f19 [446293,19092] 0 2024-09-21T04:35:03.158 INFO:tasks.workunit.client.1.smithi053.stdout:2/168: symlink d1/d18/d1f/d27/l38 0 2024-09-21T04:35:03.160 INFO:tasks.workunit.client.1.smithi053.stdout:4/168: fsync d1/f2d 0 2024-09-21T04:35:03.160 INFO:tasks.workunit.client.1.smithi053.stdout:8/176: write d3/d5/d38/f3d [378819,127656] 0 2024-09-21T04:35:03.162 INFO:tasks.workunit.client.1.smithi053.stdout:0/116: write d5/dd/ff [3188648,92087] 0 2024-09-21T04:35:03.163 INFO:tasks.workunit.client.1.smithi053.stdout:6/180: dwrite d2/d9/de/f1f [0,4194304] 0 2024-09-21T04:35:03.166 INFO:tasks.workunit.client.1.smithi053.stdout:4/169: readlink d1/dd/de/d14/l29 0 2024-09-21T04:35:03.167 INFO:tasks.workunit.client.1.smithi053.stdout:6/181: readlink d2/d9/d1a/d2e/d38/l33 0 2024-09-21T04:35:03.169 INFO:tasks.workunit.client.1.smithi053.stdout:4/170: write d1/fc [380056,17279] 0 2024-09-21T04:35:03.169 INFO:tasks.workunit.client.1.smithi053.stdout:5/220: rename d0/dd/l1e to d0/d8/d2a/l43 0 2024-09-21T04:35:03.169 INFO:tasks.workunit.client.1.smithi053.stdout:3/162: symlink d16/d17/d21/l34 0 2024-09-21T04:35:03.171 INFO:tasks.workunit.client.1.smithi053.stdout:2/169: mknod d1/d18/d1f/d22/c39 0 2024-09-21T04:35:03.173 INFO:tasks.workunit.client.1.smithi053.stdout:5/221: write d0/dd/f3a [421703,37962] 0 2024-09-21T04:35:03.173 INFO:tasks.workunit.client.1.smithi053.stdout:4/171: chown d1/f4 126647033 1 2024-09-21T04:35:03.175 INFO:tasks.workunit.client.1.smithi053.stdout:4/172: fsync d1/f4 0 2024-09-21T04:35:03.176 INFO:tasks.workunit.client.1.smithi053.stdout:6/182: symlink d2/d1b/d1e/d2a/l40 0 2024-09-21T04:35:03.176 INFO:tasks.workunit.client.1.smithi053.stdout:9/202: dwrite d0/f1 [4194304,4194304] 0 2024-09-21T04:35:03.177 INFO:tasks.workunit.client.1.smithi053.stdout:5/222: stat d0/dd/f3a 0 2024-09-21T04:35:03.178 INFO:tasks.workunit.client.1.smithi053.stdout:8/177: dread d3/df/d14/d16/f23 [0,4194304] 0 2024-09-21T04:35:03.179 INFO:tasks.workunit.client.1.smithi053.stdout:8/178: fsync d3/d39/f27 0 2024-09-21T04:35:03.182 INFO:tasks.workunit.client.1.smithi053.stdout:0/117: rename d5/de/c18 to d5/c26 0 2024-09-21T04:35:03.183 INFO:tasks.workunit.client.1.smithi053.stdout:0/118: read d5/f15 [7573552,32919] 0 2024-09-21T04:35:03.184 INFO:tasks.workunit.client.1.smithi053.stdout:0/119: write d5/dd/ff [2313726,110193] 0 2024-09-21T04:35:03.185 INFO:tasks.workunit.client.1.smithi053.stdout:4/173: creat d1/dd/de/d14/d20/f30 x:0 0 0 2024-09-21T04:35:03.187 INFO:tasks.workunit.client.1.smithi053.stdout:3/163: mknod d16/d17/c35 0 2024-09-21T04:35:03.188 INFO:tasks.workunit.client.1.smithi053.stdout:3/164: dread - d16/d17/f31 zero size 2024-09-21T04:35:03.190 INFO:tasks.workunit.client.1.smithi053.stdout:5/223: creat d0/dd/f44 x:0 0 0 2024-09-21T04:35:03.190 INFO:tasks.workunit.client.1.smithi053.stdout:3/165: dread f11 [0,4194304] 0 2024-09-21T04:35:03.192 INFO:tasks.workunit.client.1.smithi053.stdout:5/224: dread d0/d8/db/f19 [0,4194304] 0 2024-09-21T04:35:03.192 INFO:tasks.workunit.client.1.smithi053.stdout:5/225: chown d0/d8/d2a/l39 214317 1 2024-09-21T04:35:03.195 INFO:tasks.workunit.client.1.smithi053.stdout:5/226: dread d0/dd/f3a [0,4194304] 0 2024-09-21T04:35:03.198 INFO:tasks.workunit.client.1.smithi053.stdout:9/203: creat d0/d2/d9/d34/f42 x:0 0 0 2024-09-21T04:35:03.201 INFO:tasks.workunit.client.1.smithi053.stdout:0/120: symlink d5/de/l27 0 2024-09-21T04:35:03.203 INFO:tasks.workunit.client.1.smithi053.stdout:4/174: rename d1/dd/f24 to d1/d27/f31 0 2024-09-21T04:35:03.205 INFO:tasks.workunit.client.1.smithi053.stdout:0/121: dread d5/d6/f16 [0,4194304] 0 2024-09-21T04:35:03.207 INFO:tasks.workunit.client.1.smithi053.stdout:0/122: dread d5/dd/f1a [0,4194304] 0 2024-09-21T04:35:03.209 INFO:tasks.workunit.client.1.smithi053.stdout:3/166: mknod d16/c36 0 2024-09-21T04:35:03.212 INFO:tasks.workunit.client.1.smithi053.stdout:8/179: symlink d3/df/d14/l41 0 2024-09-21T04:35:03.213 INFO:tasks.workunit.client.1.smithi053.stdout:8/180: chown d3/df/c30 128 1 2024-09-21T04:35:03.218 INFO:tasks.workunit.client.1.smithi053.stdout:5/227: dwrite d0/f3b [0,4194304] 0 2024-09-21T04:35:03.222 INFO:tasks.workunit.client.1.smithi053.stdout:7/142: sync 2024-09-21T04:35:03.222 INFO:tasks.workunit.client.1.smithi053.stdout:7/143: stat d0/l2a 0 2024-09-21T04:35:03.225 INFO:tasks.workunit.client.1.smithi053.stdout:4/175: write d1/dd/de/f2a [400534,68201] 0 2024-09-21T04:35:03.230 INFO:tasks.workunit.client.1.smithi053.stdout:0/123: rename d5/d6/d21/l22 to d5/d6/l28 0 2024-09-21T04:35:03.232 INFO:tasks.workunit.client.1.smithi053.stdout:3/167: rmdir d16/d17/d21/d26/d2b 39 2024-09-21T04:35:03.236 INFO:tasks.workunit.client.1.smithi053.stdout:5/228: unlink d0/d8/d2a/f40 0 2024-09-21T04:35:03.244 INFO:tasks.workunit.client.1.smithi053.stdout:7/144: rename d0/f19 to d0/d2b/f2e 0 2024-09-21T04:35:03.245 INFO:tasks.workunit.client.1.smithi053.stdout:3/168: symlink d16/d1b/l37 0 2024-09-21T04:35:03.246 INFO:tasks.workunit.client.1.smithi053.stdout:4/176: dwrite d1/f2d [0,4194304] 0 2024-09-21T04:35:03.248 INFO:tasks.workunit.client.1.smithi053.stdout:8/181: link d3/d39/f27 d3/df/f42 0 2024-09-21T04:35:03.250 INFO:tasks.workunit.client.1.smithi053.stdout:5/229: symlink d0/d8/l45 0 2024-09-21T04:35:03.252 INFO:tasks.workunit.client.1.smithi053.stdout:9/204: creat d0/d2/d9/f43 x:0 0 0 2024-09-21T04:35:03.253 INFO:tasks.workunit.client.1.smithi053.stdout:9/205: readlink d0/d1c/l20 0 2024-09-21T04:35:03.254 INFO:tasks.workunit.client.1.smithi053.stdout:5/230: truncate d0/dd/f44 213906 0 2024-09-21T04:35:03.256 INFO:tasks.workunit.client.1.smithi053.stdout:3/169: dwrite d16/f1d [0,4194304] 0 2024-09-21T04:35:03.258 INFO:tasks.workunit.client.1.smithi053.stdout:7/145: mknod d0/d6/c2f 0 2024-09-21T04:35:03.261 INFO:tasks.workunit.client.1.smithi053.stdout:4/177: readlink d1/l6 0 2024-09-21T04:35:03.262 INFO:tasks.workunit.client.1.smithi053.stdout:9/206: dread d0/f24 [0,4194304] 0 2024-09-21T04:35:03.264 INFO:tasks.workunit.client.1.smithi053.stdout:4/178: write d1/dd/f1d [750792,49617] 0 2024-09-21T04:35:03.265 INFO:tasks.workunit.client.1.smithi053.stdout:4/179: fsync d1/dd/de/f2b 0 2024-09-21T04:35:03.266 INFO:tasks.workunit.client.1.smithi053.stdout:7/146: dread d0/d6/f7 [0,4194304] 0 2024-09-21T04:35:03.268 INFO:tasks.workunit.client.1.smithi053.stdout:3/170: creat d16/d1b/f38 x:0 0 0 2024-09-21T04:35:03.275 INFO:tasks.workunit.client.1.smithi053.stdout:9/207: creat d0/d1c/f44 x:0 0 0 2024-09-21T04:35:03.279 INFO:tasks.workunit.client.1.smithi053.stdout:3/171: creat d16/d17/d21/f39 x:0 0 0 2024-09-21T04:35:03.281 INFO:tasks.workunit.client.1.smithi053.stdout:6/183: dwrite d2/d9/f18 [0,4194304] 0 2024-09-21T04:35:03.285 INFO:tasks.workunit.client.1.smithi053.stdout:9/208: creat d0/df/f45 x:0 0 0 2024-09-21T04:35:03.286 INFO:tasks.workunit.client.1.smithi053.stdout:9/209: fsync d0/d2/d9/fc 0 2024-09-21T04:35:03.286 INFO:tasks.workunit.client.1.smithi053.stdout:9/210: stat d0/d28/f38 0 2024-09-21T04:35:03.290 INFO:tasks.workunit.client.1.smithi053.stdout:4/180: dread d1/fa [4194304,4194304] 0 2024-09-21T04:35:03.292 INFO:tasks.workunit.client.1.smithi053.stdout:3/172: unlink l2 0 2024-09-21T04:35:03.293 INFO:tasks.workunit.client.1.smithi053.stdout:3/173: write fc [277096,48981] 0 2024-09-21T04:35:03.300 INFO:tasks.workunit.client.1.smithi053.stdout:9/211: dread d0/f24 [0,4194304] 0 2024-09-21T04:35:03.300 INFO:tasks.workunit.client.1.smithi053.stdout:3/174: mknod d16/c3a 0 2024-09-21T04:35:03.301 INFO:tasks.workunit.client.1.smithi053.stdout:6/184: getdents d2/d1b/d3d/d3e 0 2024-09-21T04:35:03.303 INFO:tasks.workunit.client.1.smithi053.stdout:4/181: symlink d1/d2c/l32 0 2024-09-21T04:35:03.305 INFO:tasks.workunit.client.1.smithi053.stdout:3/175: symlink d16/l3b 0 2024-09-21T04:35:03.307 INFO:tasks.workunit.client.1.smithi053.stdout:3/176: write d16/d17/d21/f39 [208177,59356] 0 2024-09-21T04:35:03.308 INFO:tasks.workunit.client.1.smithi053.stdout:6/185: dwrite d2/d1b/f3a [0,4194304] 0 2024-09-21T04:35:03.308 INFO:tasks.workunit.client.1.smithi053.stdout:9/212: rename d0/d2/d2b/d35 to d0/d1c/d21/d46 0 2024-09-21T04:35:03.308 INFO:tasks.workunit.client.1.smithi053.stdout:3/177: fsync f9 0 2024-09-21T04:35:03.310 INFO:tasks.workunit.client.1.smithi053.stdout:3/178: fsync fc 0 2024-09-21T04:35:03.310 INFO:tasks.workunit.client.1.smithi053.stdout:4/182: write d1/dd/f23 [11965134,56103] 0 2024-09-21T04:35:03.311 INFO:tasks.workunit.client.1.smithi053.stdout:6/186: chown d2/d9/d1a/d2e/d38 2526323 1 2024-09-21T04:35:03.315 INFO:tasks.workunit.client.1.smithi053.stdout:1/267: sync 2024-09-21T04:35:03.316 INFO:tasks.workunit.client.1.smithi053.stdout:1/268: readlink d7/d9/da/d11/l42 0 2024-09-21T04:35:03.318 INFO:tasks.workunit.client.1.smithi053.stdout:3/179: dwrite fd [0,4194304] 0 2024-09-21T04:35:03.321 INFO:tasks.workunit.client.1.smithi053.stdout:4/183: mkdir d1/d27/d33 0 2024-09-21T04:35:03.324 INFO:tasks.workunit.client.1.smithi053.stdout:6/187: rename d2/d1b/d3d/d3e to d2/d1b/d1e/d27/d41 0 2024-09-21T04:35:03.327 INFO:tasks.workunit.client.1.smithi053.stdout:1/269: mkdir d7/d9/d43/d54 0 2024-09-21T04:35:03.329 INFO:tasks.workunit.client.1.smithi053.stdout:3/180: fsync d16/f30 0 2024-09-21T04:35:03.330 INFO:tasks.workunit.client.1.smithi053.stdout:0/124: write f1 [2620915,67376] 0 2024-09-21T04:35:03.330 INFO:tasks.workunit.client.1.smithi053.stdout:9/213: dread d0/d2/fa [0,4194304] 0 2024-09-21T04:35:03.331 INFO:tasks.workunit.client.1.smithi053.stdout:3/181: write d16/d1b/f24 [442745,30377] 0 2024-09-21T04:35:03.333 INFO:tasks.workunit.client.1.smithi053.stdout:4/184: creat d1/dd/de/d14/d20/f34 x:0 0 0 2024-09-21T04:35:03.335 INFO:tasks.workunit.client.1.smithi053.stdout:5/231: stat d0/d8/f31 0 2024-09-21T04:35:03.337 INFO:tasks.workunit.client.1.smithi053.stdout:1/270: symlink d7/d9/da/d40/l55 0 2024-09-21T04:35:03.340 INFO:tasks.workunit.client.1.smithi053.stdout:9/214: symlink d0/df/l47 0 2024-09-21T04:35:03.340 INFO:tasks.workunit.client.1.smithi053.stdout:1/271: dread d7/d9/da/f2f [0,4194304] 0 2024-09-21T04:35:03.340 INFO:tasks.workunit.client.1.smithi053.stdout:1/272: fsync f0 0 2024-09-21T04:35:03.341 INFO:tasks.workunit.client.1.smithi053.stdout:0/125: rename f1 to d5/de/d1c/f29 0 2024-09-21T04:35:03.342 INFO:tasks.workunit.client.1.smithi053.stdout:0/126: write d5/dd/ff [2051140,105694] 0 2024-09-21T04:35:03.343 INFO:tasks.workunit.client.1.smithi053.stdout:8/182: write d3/d5/f13 [136670,50431] 0 2024-09-21T04:35:03.344 INFO:tasks.workunit.client.1.smithi053.stdout:8/183: write d3/d39/f40 [103626,68384] 0 2024-09-21T04:35:03.344 INFO:tasks.workunit.client.1.smithi053.stdout:4/185: mknod d1/dd/de/d14/c35 0 2024-09-21T04:35:03.345 INFO:tasks.workunit.client.1.smithi053.stdout:4/186: chown d1/d2c 1674536 1 2024-09-21T04:35:03.352 INFO:tasks.workunit.client.1.smithi053.stdout:9/215: creat d0/df/f48 x:0 0 0 2024-09-21T04:35:03.355 INFO:tasks.workunit.client.1.smithi053.stdout:7/147: dwrite d0/d6/f1c [0,4194304] 0 2024-09-21T04:35:03.356 INFO:tasks.workunit.client.1.smithi053.stdout:7/148: write d0/d6/f9 [3681920,94977] 0 2024-09-21T04:35:03.357 INFO:tasks.workunit.client.1.smithi053.stdout:2/170: sync 2024-09-21T04:35:03.358 INFO:tasks.workunit.client.1.smithi053.stdout:7/149: readlink d0/le 0 2024-09-21T04:35:03.360 INFO:tasks.workunit.client.1.smithi053.stdout:3/182: mkdir d16/d17/d3c 0 2024-09-21T04:35:03.364 INFO:tasks.workunit.client.1.smithi053.stdout:4/187: write d1/fc [4447340,49104] 0 2024-09-21T04:35:03.366 INFO:tasks.workunit.client.1.smithi053.stdout:6/188: creat d2/d9/f42 x:0 0 0 2024-09-21T04:35:03.366 INFO:tasks.workunit.client.1.smithi053.stdout:4/188: chown d1/dd/de/d14/l1f 15 1 2024-09-21T04:35:03.368 INFO:tasks.workunit.client.1.smithi053.stdout:4/189: write d1/dd/de/f2b [614793,29705] 0 2024-09-21T04:35:03.369 INFO:tasks.workunit.client.1.smithi053.stdout:4/190: write d1/dd/f11 [3316396,21211] 0 2024-09-21T04:35:03.371 INFO:tasks.workunit.client.1.smithi053.stdout:9/216: symlink d0/d28/l49 0 2024-09-21T04:35:03.371 INFO:tasks.workunit.client.1.smithi053.stdout:9/217: readlink d0/d2/d9/l12 0 2024-09-21T04:35:03.373 INFO:tasks.workunit.client.1.smithi053.stdout:4/191: dread d1/fc [0,4194304] 0 2024-09-21T04:35:03.374 INFO:tasks.workunit.client.1.smithi053.stdout:7/150: fdatasync d0/d2b/f2e 0 2024-09-21T04:35:03.375 INFO:tasks.workunit.client.1.smithi053.stdout:2/171: truncate d1/d18/d1f/d22/f32 989372 0 2024-09-21T04:35:03.376 INFO:tasks.workunit.client.1.smithi053.stdout:4/192: write d1/fa [10524399,10652] 0 2024-09-21T04:35:03.376 INFO:tasks.workunit.client.1.smithi053.stdout:3/183: rmdir d16/d17/d21 39 2024-09-21T04:35:03.378 INFO:tasks.workunit.client.1.smithi053.stdout:6/189: creat d2/d9/de/f43 x:0 0 0 2024-09-21T04:35:03.379 INFO:tasks.workunit.client.1.smithi053.stdout:5/232: rename d0/d8/c28 to d0/d8/d2a/c46 0 2024-09-21T04:35:03.383 INFO:tasks.workunit.client.1.smithi053.stdout:1/273: creat d7/f56 x:0 0 0 2024-09-21T04:35:03.387 INFO:tasks.workunit.client.1.smithi053.stdout:2/172: unlink d1/f12 0 2024-09-21T04:35:03.389 INFO:tasks.workunit.client.1.smithi053.stdout:3/184: creat d16/f3d x:0 0 0 2024-09-21T04:35:03.390 INFO:tasks.workunit.client.1.smithi053.stdout:3/185: chown d16/f3d 2 1 2024-09-21T04:35:03.390 INFO:tasks.workunit.client.1.smithi053.stdout:4/193: mkdir d1/dd/de/d14/d36 0 2024-09-21T04:35:03.394 INFO:tasks.workunit.client.1.smithi053.stdout:9/218: rename d0/d2/d9/d34/f42 to d0/d28/f4a 0 2024-09-21T04:35:03.396 INFO:tasks.workunit.client.1.smithi053.stdout:6/190: dwrite d2/f3 [0,4194304] 0 2024-09-21T04:35:03.397 INFO:tasks.workunit.client.1.smithi053.stdout:1/274: symlink d7/d9/da/d11/d2b/l57 0 2024-09-21T04:35:03.404 INFO:tasks.workunit.client.1.smithi053.stdout:4/194: creat d1/d27/f37 x:0 0 0 2024-09-21T04:35:03.407 INFO:tasks.workunit.client.1.smithi053.stdout:9/219: fsync d0/f24 0 2024-09-21T04:35:03.410 INFO:tasks.workunit.client.1.smithi053.stdout:2/173: dwrite d1/d18/d1f/f24 [0,4194304] 0 2024-09-21T04:35:03.410 INFO:tasks.workunit.client.1.smithi053.stdout:6/191: creat d2/d9/d1a/f44 x:0 0 0 2024-09-21T04:35:03.411 INFO:tasks.workunit.client.1.smithi053.stdout:1/275: dread - d7/d24/d2a/d36/f4e zero size 2024-09-21T04:35:03.413 INFO:tasks.workunit.client.1.smithi053.stdout:8/184: link d3/d5/d1a/l20 d3/l43 0 2024-09-21T04:35:03.414 INFO:tasks.workunit.client.1.smithi053.stdout:1/276: write d7/d24/f41 [615230,55701] 0 2024-09-21T04:35:03.414 INFO:tasks.workunit.client.1.smithi053.stdout:8/185: fsync d3/df/d14/d16/f35 0 2024-09-21T04:35:03.417 INFO:tasks.workunit.client.1.smithi053.stdout:9/220: fdatasync d0/d2/d9/fb 0 2024-09-21T04:35:03.418 INFO:tasks.workunit.client.1.smithi053.stdout:7/151: link d0/d24/l2c d0/d2b/l30 0 2024-09-21T04:35:03.420 INFO:tasks.workunit.client.1.smithi053.stdout:7/152: write d0/d6/f1e [747037,10410] 0 2024-09-21T04:35:03.420 INFO:tasks.workunit.client.1.smithi053.stdout:2/174: creat d1/d18/d1f/d22/f3a x:0 0 0 2024-09-21T04:35:03.421 INFO:tasks.workunit.client.1.smithi053.stdout:8/186: rmdir d3 39 2024-09-21T04:35:03.422 INFO:tasks.workunit.client.1.smithi053.stdout:3/186: mkdir d16/d17/d21/d26/d3e 0 2024-09-21T04:35:03.423 INFO:tasks.workunit.client.1.smithi053.stdout:2/175: chown d1/d18/d1f/d22/d2d 31853520 1 2024-09-21T04:35:03.423 INFO:tasks.workunit.client.1.smithi053.stdout:7/153: write d0/f23 [503650,107407] 0 2024-09-21T04:35:03.424 INFO:tasks.workunit.client.1.smithi053.stdout:3/187: fdatasync f9 0 2024-09-21T04:35:03.429 INFO:tasks.workunit.client.1.smithi053.stdout:4/195: rmdir d1/dd/de/d14/d36 0 2024-09-21T04:35:03.431 INFO:tasks.workunit.client.1.smithi053.stdout:7/154: dread d0/f15 [0,4194304] 0 2024-09-21T04:35:03.436 INFO:tasks.workunit.client.1.smithi053.stdout:9/221: creat d0/f4b x:0 0 0 2024-09-21T04:35:03.439 INFO:tasks.workunit.client.1.smithi053.stdout:0/127: dwrite d5/f15 [0,4194304] 0 2024-09-21T04:35:03.440 INFO:tasks.workunit.client.1.smithi053.stdout:7/155: fdatasync d0/f4 0 2024-09-21T04:35:03.442 INFO:tasks.workunit.client.1.smithi053.stdout:9/222: chown d0/d2/d9/d1f 388599 1 2024-09-21T04:35:03.443 INFO:tasks.workunit.client.1.smithi053.stdout:4/196: dread d1/f4 [0,4194304] 0 2024-09-21T04:35:03.444 INFO:tasks.workunit.client.1.smithi053.stdout:2/176: link d1/d18/c1a d1/d18/d1f/d22/c3b 0 2024-09-21T04:35:03.444 INFO:tasks.workunit.client.1.smithi053.stdout:1/277: dread d7/d9/da/d11/f33 [0,4194304] 0 2024-09-21T04:35:03.447 INFO:tasks.workunit.client.1.smithi053.stdout:7/156: readlink d0/d2b/l30 0 2024-09-21T04:35:03.447 INFO:tasks.workunit.client.1.smithi053.stdout:1/278: chown d7/l38 113495 1 2024-09-21T04:35:03.448 INFO:tasks.workunit.client.1.smithi053.stdout:3/188: dread fc [0,4194304] 0 2024-09-21T04:35:03.450 INFO:tasks.workunit.client.1.smithi053.stdout:9/223: write d0/d2/fa [3817388,43672] 0 2024-09-21T04:35:03.450 INFO:tasks.workunit.client.1.smithi053.stdout:1/279: stat d7/d9/da/f50 0 2024-09-21T04:35:03.450 INFO:tasks.workunit.client.1.smithi053.stdout:0/128: dread d5/dd/f1a [0,4194304] 0 2024-09-21T04:35:03.450 INFO:tasks.workunit.client.1.smithi053.stdout:7/157: chown d0/d24/l26 1234490 1 2024-09-21T04:35:03.452 INFO:tasks.workunit.client.1.smithi053.stdout:8/187: getdents d3/d5/d38 0 2024-09-21T04:35:03.454 INFO:tasks.workunit.client.1.smithi053.stdout:9/224: write d0/df/f45 [833981,86239] 0 2024-09-21T04:35:03.457 INFO:tasks.workunit.client.1.smithi053.stdout:7/158: chown d0/d6/f29 508 1 2024-09-21T04:35:03.457 INFO:tasks.workunit.client.1.smithi053.stdout:2/177: dread d1/d18/f35 [0,4194304] 0 2024-09-21T04:35:03.459 INFO:tasks.workunit.client.1.smithi053.stdout:1/280: truncate d7/d9/da/d11/f33 4456194 0 2024-09-21T04:35:03.460 INFO:tasks.workunit.client.1.smithi053.stdout:0/129: creat d5/de/f2a x:0 0 0 2024-09-21T04:35:03.461 INFO:tasks.workunit.client.1.smithi053.stdout:7/159: write d0/d6/ff [268724,51730] 0 2024-09-21T04:35:03.461 INFO:tasks.workunit.client.1.smithi053.stdout:8/188: symlink d3/df/d14/d16/l44 0 2024-09-21T04:35:03.461 INFO:tasks.workunit.client.1.smithi053.stdout:2/178: fsync d1/d18/d1f/d22/f36 0 2024-09-21T04:35:03.463 INFO:tasks.workunit.client.1.smithi053.stdout:3/189: symlink d16/d17/d21/d26/d2b/l3f 0 2024-09-21T04:35:03.465 INFO:tasks.workunit.client.1.smithi053.stdout:2/179: dread d1/d18/d1f/d22/f33 [0,4194304] 0 2024-09-21T04:35:03.468 INFO:tasks.workunit.client.1.smithi053.stdout:5/233: rename d0/c4 to d0/d8/c47 0 2024-09-21T04:35:03.470 INFO:tasks.workunit.client.1.smithi053.stdout:7/160: creat d0/d6/f31 x:0 0 0 2024-09-21T04:35:03.471 INFO:tasks.workunit.client.1.smithi053.stdout:7/161: stat d0/d6/f1e 0 2024-09-21T04:35:03.471 INFO:tasks.workunit.client.1.smithi053.stdout:8/189: rmdir d3/d5/d38 39 2024-09-21T04:35:03.473 INFO:tasks.workunit.client.1.smithi053.stdout:4/197: link d1/l25 d1/dd/de/l38 0 2024-09-21T04:35:03.478 INFO:tasks.workunit.client.1.smithi053.stdout:5/234: rmdir d0/dd 39 2024-09-21T04:35:03.479 INFO:tasks.workunit.client.1.smithi053.stdout:9/225: truncate d0/d2/d9/fc 7726470 0 2024-09-21T04:35:03.480 INFO:tasks.workunit.client.1.smithi053.stdout:9/226: chown d0/df/c14 1 1 2024-09-21T04:35:03.485 INFO:tasks.workunit.client.1.smithi053.stdout:0/130: fsync d5/de/f2a 0 2024-09-21T04:35:03.486 INFO:tasks.workunit.client.1.smithi053.stdout:0/131: dread - d5/d6/f1b zero size 2024-09-21T04:35:03.486 INFO:tasks.workunit.client.1.smithi053.stdout:5/235: write d0/d8/f1f [3792161,16320] 0 2024-09-21T04:35:03.488 INFO:tasks.workunit.client.1.smithi053.stdout:7/162: fsync d0/fb 0 2024-09-21T04:35:03.489 INFO:tasks.workunit.client.1.smithi053.stdout:9/227: rmdir d0/d1c 39 2024-09-21T04:35:03.490 INFO:tasks.workunit.client.1.smithi053.stdout:9/228: chown d0/df/f27 1146376 1 2024-09-21T04:35:03.491 INFO:tasks.workunit.client.1.smithi053.stdout:9/229: fsync d0/d28/f38 0 2024-09-21T04:35:03.492 INFO:tasks.workunit.client.1.smithi053.stdout:0/132: dread d5/dd/ff [0,4194304] 0 2024-09-21T04:35:03.492 INFO:tasks.workunit.client.1.smithi053.stdout:0/133: readlink d5/d6/l24 0 2024-09-21T04:35:03.495 INFO:tasks.workunit.client.1.smithi053.stdout:4/198: symlink d1/d27/d2e/l39 0 2024-09-21T04:35:03.499 INFO:tasks.workunit.client.1.smithi053.stdout:5/236: truncate d0/d8/f1d 818756 0 2024-09-21T04:35:03.503 INFO:tasks.workunit.client.1.smithi053.stdout:3/190: dwrite d16/d1b/f27 [0,4194304] 0 2024-09-21T04:35:03.503 INFO:tasks.workunit.client.1.smithi053.stdout:7/163: unlink d0/d6/f7 0 2024-09-21T04:35:03.505 INFO:tasks.workunit.client.1.smithi053.stdout:7/164: truncate d0/d6/f1b 68074 0 2024-09-21T04:35:03.505 INFO:tasks.workunit.client.1.smithi053.stdout:9/230: mkdir d0/d1a/d4c 0 2024-09-21T04:35:03.508 INFO:tasks.workunit.client.1.smithi053.stdout:9/231: dread - d0/d28/f4a zero size 2024-09-21T04:35:03.509 INFO:tasks.workunit.client.1.smithi053.stdout:1/281: dwrite d7/d9/da/d11/f20 [0,4194304] 0 2024-09-21T04:35:03.509 INFO:tasks.workunit.client.1.smithi053.stdout:7/165: dread d0/f15 [0,4194304] 0 2024-09-21T04:35:03.511 INFO:tasks.workunit.client.1.smithi053.stdout:1/282: dread - d7/d9/da/d11/f48 zero size 2024-09-21T04:35:03.512 INFO:tasks.workunit.client.1.smithi053.stdout:9/232: chown d0/d2/d9/d1f/f26 51365138 1 2024-09-21T04:35:03.513 INFO:tasks.workunit.client.1.smithi053.stdout:1/283: write d7/d9/f1b [380010,4222] 0 2024-09-21T04:35:03.516 INFO:tasks.workunit.client.1.smithi053.stdout:8/190: truncate d3/fa 1751452 0 2024-09-21T04:35:03.516 INFO:tasks.workunit.client.1.smithi053.stdout:3/191: readlink l13 0 2024-09-21T04:35:03.516 INFO:tasks.workunit.client.1.smithi053.stdout:5/237: mkdir d0/dd/d48 0 2024-09-21T04:35:03.517 INFO:tasks.workunit.client.1.smithi053.stdout:9/233: unlink d0/f1 0 2024-09-21T04:35:03.517 INFO:tasks.workunit.client.1.smithi053.stdout:8/191: chown d3/df/d14/d16 138845551 1 2024-09-21T04:35:03.517 INFO:tasks.workunit.client.1.smithi053.stdout:3/192: stat d16/c3a 0 2024-09-21T04:35:03.519 INFO:tasks.workunit.client.1.smithi053.stdout:1/284: mkdir d7/d9/da/d40/d58 0 2024-09-21T04:35:03.526 INFO:tasks.workunit.client.1.smithi053.stdout:5/238: mknod d0/dd/d30/d35/c49 0 2024-09-21T04:35:03.527 INFO:tasks.workunit.client.1.smithi053.stdout:3/193: creat d16/d1b/f40 x:0 0 0 2024-09-21T04:35:03.527 INFO:tasks.workunit.client.1.smithi053.stdout:3/194: fdatasync f15 0 2024-09-21T04:35:03.530 INFO:tasks.workunit.client.1.smithi053.stdout:8/192: rename d3/df/c30 to d3/df/d10/c45 0 2024-09-21T04:35:03.532 INFO:tasks.workunit.client.1.smithi053.stdout:1/285: mkdir d7/d24/d2a/d59 0 2024-09-21T04:35:03.533 INFO:tasks.workunit.client.1.smithi053.stdout:0/134: getdents d5/d6/d21 0 2024-09-21T04:35:03.533 INFO:tasks.workunit.client.1.smithi053.stdout:1/286: write d7/d9/f1b [1367859,44200] 0 2024-09-21T04:35:03.535 INFO:tasks.workunit.client.1.smithi053.stdout:4/199: link d1/dd/de/d14/c1a d1/c3a 0 2024-09-21T04:35:03.540 INFO:tasks.workunit.client.1.smithi053.stdout:3/195: symlink d16/d17/d21/d26/d2b/l41 0 2024-09-21T04:35:03.543 INFO:tasks.workunit.client.1.smithi053.stdout:0/135: creat d5/d10/f2b x:0 0 0 2024-09-21T04:35:03.546 INFO:tasks.workunit.client.1.smithi053.stdout:9/234: link d0/l3 d0/d1a/l4d 0 2024-09-21T04:35:03.548 INFO:tasks.workunit.client.1.smithi053.stdout:3/196: dwrite d16/d1b/f38 [0,4194304] 0 2024-09-21T04:35:03.548 INFO:tasks.workunit.client.1.smithi053.stdout:4/200: mknod d1/dd/c3b 0 2024-09-21T04:35:03.549 INFO:tasks.workunit.client.1.smithi053.stdout:3/197: chown d16/c36 25758 1 2024-09-21T04:35:03.550 INFO:tasks.workunit.client.1.smithi053.stdout:9/235: truncate d0/d2/fe 4514394 0 2024-09-21T04:35:03.551 INFO:tasks.workunit.client.1.smithi053.stdout:5/239: link d0/f2f d0/d8/d2a/f4a 0 2024-09-21T04:35:03.552 INFO:tasks.workunit.client.1.smithi053.stdout:3/198: chown d16/d1b/l2c 0 1 2024-09-21T04:35:03.554 INFO:tasks.workunit.client.1.smithi053.stdout:8/193: write d3/df/f42 [705283,24496] 0 2024-09-21T04:35:03.555 INFO:tasks.workunit.client.1.smithi053.stdout:8/194: chown d3/df/d14/d16/f29 1060106652 1 2024-09-21T04:35:03.555 INFO:tasks.workunit.client.1.smithi053.stdout:8/195: fsync d3/d39/f2f 0 2024-09-21T04:35:03.557 INFO:tasks.workunit.client.1.smithi053.stdout:1/287: rename d7/d24/d2a to d7/d5a 0 2024-09-21T04:35:03.561 INFO:tasks.workunit.client.1.smithi053.stdout:0/136: write d5/dd/ff [952263,115132] 0 2024-09-21T04:35:03.562 INFO:tasks.workunit.client.1.smithi053.stdout:5/240: stat d0/l11 0 2024-09-21T04:35:03.564 INFO:tasks.workunit.client.1.smithi053.stdout:3/199: mkdir d16/d17/d21/d42 0 2024-09-21T04:35:03.568 INFO:tasks.workunit.client.1.smithi053.stdout:1/288: rmdir d7/d5a 39 2024-09-21T04:35:03.569 INFO:tasks.workunit.client.1.smithi053.stdout:4/201: creat d1/d27/d33/f3c x:0 0 0 2024-09-21T04:35:03.570 INFO:tasks.workunit.client.1.smithi053.stdout:4/202: chown d1/d2c/l32 11942104 1 2024-09-21T04:35:03.570 INFO:tasks.workunit.client.1.smithi053.stdout:9/236: write d0/d2/d9/fb [2065849,87488] 0 2024-09-21T04:35:03.572 INFO:tasks.workunit.client.1.smithi053.stdout:0/137: rename d5/de/c12 to d5/d6/d21/c2c 0 2024-09-21T04:35:03.572 INFO:tasks.workunit.client.1.smithi053.stdout:5/241: symlink d0/dd/l4b 0 2024-09-21T04:35:03.573 INFO:tasks.workunit.client.1.smithi053.stdout:8/196: symlink d3/l46 0 2024-09-21T04:35:03.575 INFO:tasks.workunit.client.1.smithi053.stdout:9/237: mkdir d0/d2/d4e 0 2024-09-21T04:35:03.581 INFO:tasks.workunit.client.1.smithi053.stdout:0/138: dwrite d5/d10/f2b [0,4194304] 0 2024-09-21T04:35:03.582 INFO:tasks.workunit.client.1.smithi053.stdout:1/289: creat d7/d9/da/f5b x:0 0 0 2024-09-21T04:35:03.583 INFO:tasks.workunit.client.1.smithi053.stdout:8/197: creat d3/d5/d38/f47 x:0 0 0 2024-09-21T04:35:03.587 INFO:tasks.workunit.client.1.smithi053.stdout:6/192: sync 2024-09-21T04:35:03.587 INFO:tasks.workunit.client.1.smithi053.stdout:8/198: rmdir d3/d5 39 2024-09-21T04:35:03.590 INFO:tasks.workunit.client.1.smithi053.stdout:3/200: truncate d16/d17/f1e 3056582 0 2024-09-21T04:35:03.591 INFO:tasks.workunit.client.1.smithi053.stdout:9/238: getdents d0/d2/d9/d1f 0 2024-09-21T04:35:03.591 INFO:tasks.workunit.client.1.smithi053.stdout:3/201: readlink d16/d17/d21/d26/d2b/l41 0 2024-09-21T04:35:03.592 INFO:tasks.workunit.client.1.smithi053.stdout:5/242: dread d0/d8/d2a/f4a [0,4194304] 0 2024-09-21T04:35:03.592 INFO:tasks.workunit.client.1.smithi053.stdout:4/203: truncate d1/f2d 679751 0 2024-09-21T04:35:03.592 INFO:tasks.workunit.client.1.smithi053.stdout:9/239: chown d0/d2/d9/f19 13193983 1 2024-09-21T04:35:03.595 INFO:tasks.workunit.client.1.smithi053.stdout:6/193: rename d2/l6 to d2/d9/de/l45 0 2024-09-21T04:35:03.595 INFO:tasks.workunit.client.1.smithi053.stdout:9/240: write d0/d2/d9/f43 [753257,61510] 0 2024-09-21T04:35:03.597 INFO:tasks.workunit.client.1.smithi053.stdout:1/290: creat d7/d9/da/d11/f5c x:0 0 0 2024-09-21T04:35:03.598 INFO:tasks.workunit.client.1.smithi053.stdout:3/202: dread d16/f1d [0,4194304] 0 2024-09-21T04:35:03.598 INFO:tasks.workunit.client.1.smithi053.stdout:5/243: write d0/d8/f31 [549880,44440] 0 2024-09-21T04:35:03.599 INFO:tasks.workunit.client.1.smithi053.stdout:3/203: write f14 [2323311,20752] 0 2024-09-21T04:35:03.601 INFO:tasks.workunit.client.1.smithi053.stdout:9/241: chown d0/d1c/d21/l33 5463 1 2024-09-21T04:35:03.601 INFO:tasks.workunit.client.1.smithi053.stdout:8/199: rename d3/d5/f25 to d3/df/d10/f48 0 2024-09-21T04:35:03.602 INFO:tasks.workunit.client.1.smithi053.stdout:0/139: getdents d5/d6/d21 0 2024-09-21T04:35:03.604 INFO:tasks.workunit.client.1.smithi053.stdout:4/204: mknod d1/c3d 0 2024-09-21T04:35:03.604 INFO:tasks.workunit.client.1.smithi053.stdout:6/194: dread d2/d9/de/f14 [0,4194304] 0 2024-09-21T04:35:03.605 INFO:tasks.workunit.client.1.smithi053.stdout:4/205: truncate d1/dd/f1d 4351405 0 2024-09-21T04:35:03.606 INFO:tasks.workunit.client.1.smithi053.stdout:4/206: write d1/d27/f37 [651278,84337] 0 2024-09-21T04:35:03.607 INFO:tasks.workunit.client.1.smithi053.stdout:8/200: mkdir d3/df/d49 0 2024-09-21T04:35:03.607 INFO:tasks.workunit.client.1.smithi053.stdout:4/207: chown d1/dd/f1d 3066 1 2024-09-21T04:35:03.608 INFO:tasks.workunit.client.1.smithi053.stdout:8/201: chown d3/df/d14/d16/f29 5 1 2024-09-21T04:35:03.609 INFO:tasks.workunit.client.1.smithi053.stdout:8/202: dread - d3/f3a zero size 2024-09-21T04:35:03.609 INFO:tasks.workunit.client.1.smithi053.stdout:0/140: mknod d5/d6/d21/c2d 0 2024-09-21T04:35:03.611 INFO:tasks.workunit.client.1.smithi053.stdout:3/204: mkdir d16/d17/d21/d42/d43 0 2024-09-21T04:35:03.613 INFO:tasks.workunit.client.1.smithi053.stdout:6/195: read d2/f8 [125376,118589] 0 2024-09-21T04:35:03.614 INFO:tasks.workunit.client.1.smithi053.stdout:6/196: truncate d2/fa 4632575 0 2024-09-21T04:35:03.617 INFO:tasks.workunit.client.1.smithi053.stdout:5/244: rename d0/dd/l4b to d0/l4c 0 2024-09-21T04:35:03.618 INFO:tasks.workunit.client.1.smithi053.stdout:1/291: getdents d7/d24 0 2024-09-21T04:35:03.618 INFO:tasks.workunit.client.1.smithi053.stdout:9/242: write d0/df/f23 [5166183,108432] 0 2024-09-21T04:35:03.619 INFO:tasks.workunit.client.1.smithi053.stdout:1/292: write d7/f56 [119276,73178] 0 2024-09-21T04:35:03.662 INFO:tasks.workunit.client.1.smithi053.stdout:5/245: mknod d0/dd/d30/c4d 0 2024-09-21T04:35:03.663 INFO:tasks.workunit.client.1.smithi053.stdout:9/243: unlink d0/d2/d9/l2e 0 2024-09-21T04:35:03.664 INFO:tasks.workunit.client.1.smithi053.stdout:8/203: creat d3/f4a x:0 0 0 2024-09-21T04:35:03.664 INFO:tasks.workunit.client.1.smithi053.stdout:4/208: getdents d1/d27/d33 0 2024-09-21T04:35:03.666 INFO:tasks.workunit.client.1.smithi053.stdout:3/205: rename c6 to d16/c44 0 2024-09-21T04:35:03.668 INFO:tasks.workunit.client.1.smithi053.stdout:5/246: symlink d0/d8/d2a/l4e 0 2024-09-21T04:35:03.671 INFO:tasks.workunit.client.1.smithi053.stdout:1/293: creat d7/d9/da/d40/d58/f5d x:0 0 0 2024-09-21T04:35:03.671 INFO:tasks.workunit.client.1.smithi053.stdout:6/197: dread d2/fb [0,4194304] 0 2024-09-21T04:35:03.672 INFO:tasks.workunit.client.1.smithi053.stdout:8/204: mkdir d3/d5/d4b 0 2024-09-21T04:35:03.673 INFO:tasks.workunit.client.1.smithi053.stdout:9/244: dread d0/f4 [0,4194304] 0 2024-09-21T04:35:03.675 INFO:tasks.workunit.client.1.smithi053.stdout:4/209: dwrite d1/dd/f23 [4194304,4194304] 0 2024-09-21T04:35:03.677 INFO:tasks.workunit.client.1.smithi053.stdout:3/206: rename d16/d17/d21/l22 to d16/d17/d21/d26/l45 0 2024-09-21T04:35:03.678 INFO:tasks.workunit.client.1.smithi053.stdout:6/198: creat d2/d1b/d1e/d27/d41/f46 x:0 0 0 2024-09-21T04:35:03.680 INFO:tasks.workunit.client.1.smithi053.stdout:3/207: write d16/d1b/f2f [225321,48001] 0 2024-09-21T04:35:03.683 INFO:tasks.workunit.client.1.smithi053.stdout:8/205: mkdir d3/d5/d4c 0 2024-09-21T04:35:03.684 INFO:tasks.workunit.client.1.smithi053.stdout:5/247: rename d0/dd/c3d to d0/dd/d30/c4f 0 2024-09-21T04:35:03.687 INFO:tasks.workunit.client.1.smithi053.stdout:3/208: mkdir d16/d46 0 2024-09-21T04:35:03.688 INFO:tasks.workunit.client.1.smithi053.stdout:6/199: mknod d2/d1b/d1e/d27/d41/c47 0 2024-09-21T04:35:03.689 INFO:tasks.workunit.client.1.smithi053.stdout:4/210: dwrite d1/dd/de/d14/d20/f34 [0,4194304] 0 2024-09-21T04:35:03.693 INFO:tasks.workunit.client.1.smithi053.stdout:4/211: stat d1/dd/de/d14/d20/l21 0 2024-09-21T04:35:03.695 INFO:tasks.workunit.client.1.smithi053.stdout:9/245: dwrite d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:03.696 INFO:tasks.workunit.client.1.smithi053.stdout:1/294: dread f0 [0,4194304] 0 2024-09-21T04:35:03.696 INFO:tasks.workunit.client.1.smithi053.stdout:9/246: stat d0/d28 0 2024-09-21T04:35:03.698 INFO:tasks.workunit.client.1.smithi053.stdout:6/200: write d2/d9/f19 [6040633,22436] 0 2024-09-21T04:35:03.699 INFO:tasks.workunit.client.1.smithi053.stdout:3/209: rename d16/c3a to d16/d17/d21/d42/d43/c47 0 2024-09-21T04:35:03.700 INFO:tasks.workunit.client.1.smithi053.stdout:5/248: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:03.703 INFO:tasks.workunit.client.1.smithi053.stdout:8/206: symlink d3/d5/d21/l4d 0 2024-09-21T04:35:03.705 INFO:tasks.workunit.client.1.smithi053.stdout:9/247: unlink d0/df/f45 0 2024-09-21T04:35:03.706 INFO:tasks.workunit.client.1.smithi053.stdout:3/210: mknod d16/d17/d21/d26/d2b/c48 0 2024-09-21T04:35:03.707 INFO:tasks.workunit.client.1.smithi053.stdout:3/211: chown f9 161941 1 2024-09-21T04:35:03.708 INFO:tasks.workunit.client.1.smithi053.stdout:9/248: fdatasync d0/d2/d9/d1f/f26 0 2024-09-21T04:35:03.712 INFO:tasks.workunit.client.1.smithi053.stdout:9/249: chown d0/c30 46375 1 2024-09-21T04:35:03.713 INFO:tasks.workunit.client.1.smithi053.stdout:1/295: creat d7/d9/da/d40/d49/f5e x:0 0 0 2024-09-21T04:35:03.714 INFO:tasks.workunit.client.1.smithi053.stdout:4/212: dread d1/dd/de/d14/d20/f34 [0,4194304] 0 2024-09-21T04:35:03.718 INFO:tasks.workunit.client.1.smithi053.stdout:1/296: readlink d7/d9/da/d11/l1e 0 2024-09-21T04:35:03.718 INFO:tasks.workunit.client.1.smithi053.stdout:3/212: mkdir d16/d17/d21/d26/d49 0 2024-09-21T04:35:03.719 INFO:tasks.workunit.client.1.smithi053.stdout:6/201: dwrite d2/d9/f42 [0,4194304] 0 2024-09-21T04:35:03.720 INFO:tasks.workunit.client.1.smithi053.stdout:8/207: dwrite d3/df/d14/d16/f29 [0,4194304] 0 2024-09-21T04:35:03.724 INFO:tasks.workunit.client.1.smithi053.stdout:4/213: symlink d1/d27/d2e/l3e 0 2024-09-21T04:35:03.725 INFO:tasks.workunit.client.1.smithi053.stdout:4/214: fsync d1/f4 0 2024-09-21T04:35:03.727 INFO:tasks.workunit.client.1.smithi053.stdout:6/202: rmdir d2/d9/d1a 39 2024-09-21T04:35:03.727 INFO:tasks.workunit.client.1.smithi053.stdout:5/249: dwrite d0/d8/f2b [4194304,4194304] 0 2024-09-21T04:35:03.733 INFO:tasks.workunit.client.1.smithi053.stdout:8/208: rename d3/f34 to d3/d5/d38/f4e 0 2024-09-21T04:35:03.734 INFO:tasks.workunit.client.1.smithi053.stdout:1/297: mknod d7/d5a/c5f 0 2024-09-21T04:35:03.737 INFO:tasks.workunit.client.1.smithi053.stdout:3/213: creat d16/d46/f4a x:0 0 0 2024-09-21T04:35:03.743 INFO:tasks.workunit.client.1.smithi053.stdout:9/250: getdents d0 0 2024-09-21T04:35:03.743 INFO:tasks.workunit.client.1.smithi053.stdout:1/298: mknod d7/d9/da/d40/d58/c60 0 2024-09-21T04:35:03.744 INFO:tasks.workunit.client.1.smithi053.stdout:6/203: creat d2/d9/d1a/f48 x:0 0 0 2024-09-21T04:35:03.745 INFO:tasks.workunit.client.1.smithi053.stdout:3/214: symlink d16/d1b/l4b 0 2024-09-21T04:35:03.746 INFO:tasks.workunit.client.1.smithi053.stdout:9/251: creat d0/d2/d9/d1f/f4f x:0 0 0 2024-09-21T04:35:03.751 INFO:tasks.workunit.client.1.smithi053.stdout:9/252: symlink d0/d1a/d4c/l50 0 2024-09-21T04:35:03.753 INFO:tasks.workunit.client.1.smithi053.stdout:9/253: mknod d0/d1c/d21/c51 0 2024-09-21T04:35:03.754 INFO:tasks.workunit.client.1.smithi053.stdout:9/254: chown d0/d1a/c3f 3 1 2024-09-21T04:35:03.756 INFO:tasks.workunit.client.1.smithi053.stdout:9/255: mkdir d0/d1c/d52 0 2024-09-21T04:35:03.759 INFO:tasks.workunit.client.1.smithi053.stdout:9/256: mkdir d0/d1c/d52/d53 0 2024-09-21T04:35:03.761 INFO:tasks.workunit.client.1.smithi053.stdout:3/215: dread d16/d1b/f1f [0,4194304] 0 2024-09-21T04:35:03.762 INFO:tasks.workunit.client.1.smithi053.stdout:9/257: creat d0/d1c/d25/f54 x:0 0 0 2024-09-21T04:35:03.764 INFO:tasks.workunit.client.1.smithi053.stdout:6/204: rename d2/d9 to d2/d1b/d3d/d49 0 2024-09-21T04:35:03.764 INFO:tasks.workunit.client.1.smithi053.stdout:3/216: dread d16/d17/d21/f39 [0,4194304] 0 2024-09-21T04:35:03.764 INFO:tasks.workunit.client.1.smithi053.stdout:5/250: dwrite d0/dd/f3a [0,4194304] 0 2024-09-21T04:35:03.765 INFO:tasks.workunit.client.1.smithi053.stdout:8/209: write d3/df/d14/d16/f1d [1009768,3089] 0 2024-09-21T04:35:03.765 INFO:tasks.workunit.client.1.smithi053.stdout:6/205: chown d2/f3 6778042 1 2024-09-21T04:35:03.769 INFO:tasks.workunit.client.1.smithi053.stdout:6/206: write d2/d1b/d3d/d49/de/f1f [1758950,30082] 0 2024-09-21T04:35:03.859 INFO:tasks.workunit.client.1.smithi053.stdout:2/180: sync 2024-09-21T04:35:03.863 INFO:tasks.workunit.client.1.smithi053.stdout:1/299: truncate d7/d9/da/d11/f20 1559186 0 2024-09-21T04:35:03.865 INFO:tasks.workunit.client.1.smithi053.stdout:8/210: write d3/df/d14/d16/f23 [3909777,37208] 0 2024-09-21T04:35:03.868 INFO:tasks.workunit.client.1.smithi053.stdout:3/217: symlink d16/d17/d21/d26/l4c 0 2024-09-21T04:35:03.869 INFO:tasks.workunit.client.1.smithi053.stdout:2/181: rename d1/d18/d1f/d27/c30 to d1/c3c 0 2024-09-21T04:35:03.869 INFO:tasks.workunit.client.1.smithi053.stdout:5/251: dread d0/d8/f2b [0,4194304] 0 2024-09-21T04:35:03.872 INFO:tasks.workunit.client.1.smithi053.stdout:8/211: dread d3/df/f42 [0,4194304] 0 2024-09-21T04:35:03.872 INFO:tasks.workunit.client.1.smithi053.stdout:5/252: chown d0/dd/d30/c4d 130 1 2024-09-21T04:35:03.872 INFO:tasks.workunit.client.1.smithi053.stdout:2/182: write d1/d18/f31 [670447,15280] 0 2024-09-21T04:35:03.877 INFO:tasks.workunit.client.1.smithi053.stdout:1/300: dwrite f1 [0,4194304] 0 2024-09-21T04:35:03.877 INFO:tasks.workunit.client.1.smithi053.stdout:5/253: symlink d0/d8/d2a/l50 0 2024-09-21T04:35:03.878 INFO:tasks.workunit.client.1.smithi053.stdout:8/212: dread d3/d39/f27 [0,4194304] 0 2024-09-21T04:35:03.881 INFO:tasks.workunit.client.1.smithi053.stdout:3/218: dwrite d16/d17/f31 [0,4194304] 0 2024-09-21T04:35:03.885 INFO:tasks.workunit.client.1.smithi053.stdout:8/213: unlink d3/d5/d1a/f32 0 2024-09-21T04:35:03.889 INFO:tasks.workunit.client.1.smithi053.stdout:2/183: dread d1/d18/d1f/d22/f32 [0,4194304] 0 2024-09-21T04:35:03.895 INFO:tasks.workunit.client.1.smithi053.stdout:8/214: dwrite d3/df/d10/f48 [0,4194304] 0 2024-09-21T04:35:03.899 INFO:tasks.workunit.client.1.smithi053.stdout:8/215: mkdir d3/d4f 0 2024-09-21T04:35:03.907 INFO:tasks.workunit.client.1.smithi053.stdout:8/216: rename d3/df/d14/d16/c2a to d3/df/d14/c50 0 2024-09-21T04:35:03.908 INFO:tasks.workunit.client.1.smithi053.stdout:8/217: read d3/d39/f3f [1375403,105232] 0 2024-09-21T04:35:03.910 INFO:tasks.workunit.client.1.smithi053.stdout:8/218: rename d3/d5/l8 to d3/d5/d4b/l51 0 2024-09-21T04:35:03.912 INFO:tasks.workunit.client.1.smithi053.stdout:6/207: write d2/d1b/d3d/d49/de/f15 [2354620,121118] 0 2024-09-21T04:35:03.912 INFO:tasks.workunit.client.1.smithi053.stdout:6/208: dread - d2/f28 zero size 2024-09-21T04:35:03.913 INFO:tasks.workunit.client.1.smithi053.stdout:1/301: dread d7/d9/da/d11/f20 [0,4194304] 0 2024-09-21T04:35:03.913 INFO:tasks.workunit.client.1.smithi053.stdout:6/209: stat d2/d1b/d3d/d49/f19 0 2024-09-21T04:35:03.915 INFO:tasks.workunit.client.1.smithi053.stdout:6/210: fsync d2/d1b/d3d/d49/de/f15 0 2024-09-21T04:35:03.916 INFO:tasks.workunit.client.1.smithi053.stdout:5/254: truncate d0/f3b 415907 0 2024-09-21T04:35:03.918 INFO:tasks.workunit.client.1.smithi053.stdout:6/211: dread d2/d1b/d1e/d2a/f34 [0,4194304] 0 2024-09-21T04:35:03.921 INFO:tasks.workunit.client.1.smithi053.stdout:5/255: unlink d0/dd/d30/d35/l42 0 2024-09-21T04:35:03.924 INFO:tasks.workunit.client.1.smithi053.stdout:8/219: link d3/d5/c33 d3/d5/d4c/c52 0 2024-09-21T04:35:03.924 INFO:tasks.workunit.client.1.smithi053.stdout:2/184: dwrite d1/d18/d1f/d22/f33 [0,4194304] 0 2024-09-21T04:35:03.925 INFO:tasks.workunit.client.1.smithi053.stdout:7/166: sync 2024-09-21T04:35:03.925 INFO:tasks.workunit.client.1.smithi053.stdout:5/256: write d0/f2f [1449117,113750] 0 2024-09-21T04:35:03.926 INFO:tasks.workunit.client.1.smithi053.stdout:2/185: dread - d1/d18/d1f/f37 zero size 2024-09-21T04:35:03.927 INFO:tasks.workunit.client.1.smithi053.stdout:2/186: stat d1/d18/d1f/l28 0 2024-09-21T04:35:03.929 INFO:tasks.workunit.client.1.smithi053.stdout:2/187: write d1/d18/f31 [1270390,118009] 0 2024-09-21T04:35:03.929 INFO:tasks.workunit.client.1.smithi053.stdout:2/188: readlink d1/l1d 0 2024-09-21T04:35:03.933 INFO:tasks.workunit.client.1.smithi053.stdout:7/167: chown d0/d2b/l30 19519 1 2024-09-21T04:35:03.937 INFO:tasks.workunit.client.1.smithi053.stdout:8/220: mkdir d3/d5/d4c/d53 0 2024-09-21T04:35:03.941 INFO:tasks.workunit.client.1.smithi053.stdout:7/168: creat d0/d2b/f32 x:0 0 0 2024-09-21T04:35:03.941 INFO:tasks.workunit.client.1.smithi053.stdout:7/169: dread - d0/d2b/f32 zero size 2024-09-21T04:35:03.942 INFO:tasks.workunit.client.1.smithi053.stdout:6/212: getdents d2 0 2024-09-21T04:35:03.945 INFO:tasks.workunit.client.1.smithi053.stdout:5/257: truncate d0/f26 1213124 0 2024-09-21T04:35:03.947 INFO:tasks.workunit.client.1.smithi053.stdout:6/213: truncate d2/f8 170738 0 2024-09-21T04:35:03.948 INFO:tasks.workunit.client.1.smithi053.stdout:8/221: rename d3/d5/l1e to d3/d4f/l54 0 2024-09-21T04:35:03.949 INFO:tasks.workunit.client.1.smithi053.stdout:5/258: mkdir d0/dd/d30/d51 0 2024-09-21T04:35:03.952 INFO:tasks.workunit.client.1.smithi053.stdout:2/189: getdents d1/d18 0 2024-09-21T04:35:03.953 INFO:tasks.workunit.client.1.smithi053.stdout:7/170: write d0/fb [3311963,23098] 0 2024-09-21T04:35:03.954 INFO:tasks.workunit.client.1.smithi053.stdout:6/214: unlink d2/f3 0 2024-09-21T04:35:03.954 INFO:tasks.workunit.client.1.smithi053.stdout:2/190: rename d1/d18/d1f/d22/d2d to d1/d18/d1f/d22/d2d/d3d 22 2024-09-21T04:35:03.959 INFO:tasks.workunit.client.1.smithi053.stdout:5/259: mkdir d0/dd/d30/d51/d52 0 2024-09-21T04:35:03.960 INFO:tasks.workunit.client.1.smithi053.stdout:7/171: dwrite d0/f23 [0,4194304] 0 2024-09-21T04:35:03.961 INFO:tasks.workunit.client.1.smithi053.stdout:6/215: creat d2/d1b/d3d/f4a x:0 0 0 2024-09-21T04:35:03.963 INFO:tasks.workunit.client.1.smithi053.stdout:2/191: rename d1/c3 to d1/d18/d1f/d22/d2d/c3e 0 2024-09-21T04:35:03.964 INFO:tasks.workunit.client.1.smithi053.stdout:2/192: read d1/d18/d1f/d22/f33 [89071,30601] 0 2024-09-21T04:35:03.966 INFO:tasks.workunit.client.1.smithi053.stdout:5/260: dwrite d0/d8/d2a/f4a [0,4194304] 0 2024-09-21T04:35:03.966 INFO:tasks.workunit.client.1.smithi053.stdout:7/172: symlink d0/d6/l33 0 2024-09-21T04:35:03.968 INFO:tasks.workunit.client.1.smithi053.stdout:5/261: write d0/d8/f1f [1151055,11578] 0 2024-09-21T04:35:03.973 INFO:tasks.workunit.client.1.smithi053.stdout:5/262: truncate d0/d8/f1d 1257989 0 2024-09-21T04:35:03.973 INFO:tasks.workunit.client.1.smithi053.stdout:6/216: fsync d2/f8 0 2024-09-21T04:35:03.973 INFO:tasks.workunit.client.1.smithi053.stdout:2/193: dwrite d1/d18/d1f/d22/f3a [0,4194304] 0 2024-09-21T04:35:03.975 INFO:tasks.workunit.client.1.smithi053.stdout:2/194: chown d1/d18/d1f/d22/f36 476 1 2024-09-21T04:35:03.976 INFO:tasks.workunit.client.1.smithi053.stdout:2/195: write d1/d18/d1f/f37 [843531,16578] 0 2024-09-21T04:35:03.977 INFO:tasks.workunit.client.1.smithi053.stdout:7/173: mknod d0/c34 0 2024-09-21T04:35:03.978 INFO:tasks.workunit.client.1.smithi053.stdout:7/174: chown d0/l22 25196323 1 2024-09-21T04:35:03.982 INFO:tasks.workunit.client.1.smithi053.stdout:2/196: creat d1/f3f x:0 0 0 2024-09-21T04:35:03.984 INFO:tasks.workunit.client.1.smithi053.stdout:2/197: write d1/d18/f1c [1529787,124245] 0 2024-09-21T04:35:03.985 INFO:tasks.workunit.client.1.smithi053.stdout:7/175: rename d0/l2a to d0/d24/l35 0 2024-09-21T04:35:03.989 INFO:tasks.workunit.client.1.smithi053.stdout:2/198: rename d1/l1d to d1/d18/d1f/d22/l40 0 2024-09-21T04:35:03.990 INFO:tasks.workunit.client.1.smithi053.stdout:7/176: mkdir d0/d2b/d36 0 2024-09-21T04:35:03.993 INFO:tasks.workunit.client.1.smithi053.stdout:2/199: mkdir d1/d18/d1f/d27/d41 0 2024-09-21T04:35:03.994 INFO:tasks.workunit.client.1.smithi053.stdout:7/177: creat d0/d2b/f37 x:0 0 0 2024-09-21T04:35:03.997 INFO:tasks.workunit.client.1.smithi053.stdout:2/200: creat d1/d18/d1f/d27/d41/f42 x:0 0 0 2024-09-21T04:35:04.000 INFO:tasks.workunit.client.1.smithi053.stdout:7/178: symlink d0/l38 0 2024-09-21T04:35:04.000 INFO:tasks.workunit.client.1.smithi053.stdout:7/179: chown d0/l22 59 1 2024-09-21T04:35:04.006 INFO:tasks.workunit.client.1.smithi053.stdout:7/180: rename d0/d24/l35 to d0/d6/l39 0 2024-09-21T04:35:04.008 INFO:tasks.workunit.client.1.smithi053.stdout:6/217: fdatasync d2/f8 0 2024-09-21T04:35:04.009 INFO:tasks.workunit.client.1.smithi053.stdout:7/181: write d0/d6/f1c [5063762,65522] 0 2024-09-21T04:35:04.011 INFO:tasks.workunit.client.1.smithi053.stdout:2/201: write d1/d18/d1f/d22/f32 [321873,10413] 0 2024-09-21T04:35:04.012 INFO:tasks.workunit.client.1.smithi053.stdout:6/218: mkdir d2/d1b/d1e/d2a/d4b 0 2024-09-21T04:35:04.012 INFO:tasks.workunit.client.1.smithi053.stdout:7/182: write d0/d6/f9 [896312,43924] 0 2024-09-21T04:35:04.014 INFO:tasks.workunit.client.1.smithi053.stdout:5/263: dwrite d0/f26 [0,4194304] 0 2024-09-21T04:35:04.015 INFO:tasks.workunit.client.1.smithi053.stdout:2/202: unlink d1/l13 0 2024-09-21T04:35:04.016 INFO:tasks.workunit.client.1.smithi053.stdout:6/219: write d2/d1b/d3d/d49/d1a/d2e/f35 [58998,93558] 0 2024-09-21T04:35:04.017 INFO:tasks.workunit.client.1.smithi053.stdout:6/220: write d2/d1b/d3d/d49/de/f15 [880168,58717] 0 2024-09-21T04:35:04.018 INFO:tasks.workunit.client.1.smithi053.stdout:6/221: chown d2/f28 64372695 1 2024-09-21T04:35:04.019 INFO:tasks.workunit.client.1.smithi053.stdout:5/264: rmdir d0/dd/d30/d51 39 2024-09-21T04:35:04.020 INFO:tasks.workunit.client.1.smithi053.stdout:6/222: dread d2/d1b/d1e/d2a/f34 [0,4194304] 0 2024-09-21T04:35:04.021 INFO:tasks.workunit.client.1.smithi053.stdout:2/203: mkdir d1/d18/d1f/d22/d43 0 2024-09-21T04:35:04.022 INFO:tasks.workunit.client.1.smithi053.stdout:2/204: truncate d1/d18/d1f/d22/f36 755652 0 2024-09-21T04:35:04.023 INFO:tasks.workunit.client.1.smithi053.stdout:2/205: fsync d1/fb 0 2024-09-21T04:35:04.024 INFO:tasks.workunit.client.1.smithi053.stdout:2/206: chown d1/d18/d1f/f37 548667 1 2024-09-21T04:35:04.026 INFO:tasks.workunit.client.1.smithi053.stdout:2/207: stat d1/d18/d1f/d27/d41 0 2024-09-21T04:35:04.026 INFO:tasks.workunit.client.1.smithi053.stdout:5/265: fsync d0/f7 0 2024-09-21T04:35:04.028 INFO:tasks.workunit.client.1.smithi053.stdout:2/208: write d1/fb [542876,97026] 0 2024-09-21T04:35:04.030 INFO:tasks.workunit.client.1.smithi053.stdout:5/266: read d0/d8/f1f [3637684,34576] 0 2024-09-21T04:35:04.031 INFO:tasks.workunit.client.1.smithi053.stdout:2/209: write d1/d18/d1f/f37 [1163565,8054] 0 2024-09-21T04:35:04.032 INFO:tasks.workunit.client.1.smithi053.stdout:5/267: symlink d0/d8/d2a/l53 0 2024-09-21T04:35:04.034 INFO:tasks.workunit.client.1.smithi053.stdout:7/183: truncate d0/f23 4109006 0 2024-09-21T04:35:04.037 INFO:tasks.workunit.client.1.smithi053.stdout:5/268: creat d0/dd/d48/f54 x:0 0 0 2024-09-21T04:35:04.038 INFO:tasks.workunit.client.1.smithi053.stdout:0/141: sync 2024-09-21T04:35:04.038 INFO:tasks.workunit.client.1.smithi053.stdout:4/215: sync 2024-09-21T04:35:04.038 INFO:tasks.workunit.client.1.smithi053.stdout:6/223: dread d2/d1b/d3d/d49/d1a/f21 [4194304,4194304] 0 2024-09-21T04:35:04.040 INFO:tasks.workunit.client.1.smithi053.stdout:2/210: dwrite d1/d18/d1f/f37 [0,4194304] 0 2024-09-21T04:35:04.040 INFO:tasks.workunit.client.1.smithi053.stdout:5/269: chown d0/dd/d30/d35/c49 2 1 2024-09-21T04:35:04.042 INFO:tasks.workunit.client.1.smithi053.stdout:7/184: creat d0/f3a x:0 0 0 2024-09-21T04:35:04.043 INFO:tasks.workunit.client.1.smithi053.stdout:4/216: mkdir d1/dd/de/d14/d3f 0 2024-09-21T04:35:04.043 INFO:tasks.workunit.client.1.smithi053.stdout:4/217: stat d1/fa 0 2024-09-21T04:35:04.045 INFO:tasks.workunit.client.1.smithi053.stdout:4/218: dread - d1/d27/d33/f3c zero size 2024-09-21T04:35:04.045 INFO:tasks.workunit.client.1.smithi053.stdout:2/211: mknod d1/d18/c44 0 2024-09-21T04:35:04.048 INFO:tasks.workunit.client.1.smithi053.stdout:7/185: write d0/d6/f29 [1036702,81128] 0 2024-09-21T04:35:04.049 INFO:tasks.workunit.client.1.smithi053.stdout:5/270: fsync d0/f3b 0 2024-09-21T04:35:04.050 INFO:tasks.workunit.client.1.smithi053.stdout:7/186: write d0/d6/f8 [919276,20901] 0 2024-09-21T04:35:04.051 INFO:tasks.workunit.client.1.smithi053.stdout:5/271: write d0/dd/f44 [1208937,95088] 0 2024-09-21T04:35:04.053 INFO:tasks.workunit.client.1.smithi053.stdout:0/142: rename l4 to d5/dd/l2e 0 2024-09-21T04:35:04.054 INFO:tasks.workunit.client.1.smithi053.stdout:2/212: creat d1/d18/d1f/d22/f45 x:0 0 0 2024-09-21T04:35:04.057 INFO:tasks.workunit.client.1.smithi053.stdout:5/272: rmdir d0/dd/d48 39 2024-09-21T04:35:04.060 INFO:tasks.workunit.client.1.smithi053.stdout:7/187: dwrite d0/d6/f1b [0,4194304] 0 2024-09-21T04:35:04.063 INFO:tasks.workunit.client.1.smithi053.stdout:0/143: dwrite d5/fa [0,4194304] 0 2024-09-21T04:35:04.067 INFO:tasks.workunit.client.1.smithi053.stdout:7/188: dread d0/d6/f1c [0,4194304] 0 2024-09-21T04:35:04.071 INFO:tasks.workunit.client.1.smithi053.stdout:6/224: rename d2/d1b/d1e/d2a/f2b to d2/d1b/d3d/d49/f4c 0 2024-09-21T04:35:04.073 INFO:tasks.workunit.client.1.smithi053.stdout:0/144: creat d5/de/f2f x:0 0 0 2024-09-21T04:35:04.075 INFO:tasks.workunit.client.1.smithi053.stdout:5/273: symlink d0/dd/d48/l55 0 2024-09-21T04:35:04.076 INFO:tasks.workunit.client.1.smithi053.stdout:7/189: rmdir d0/d6 39 2024-09-21T04:35:04.077 INFO:tasks.workunit.client.1.smithi053.stdout:7/190: chown d0/d2b/l30 139 1 2024-09-21T04:35:04.081 INFO:tasks.workunit.client.1.smithi053.stdout:6/225: creat d2/d1b/d1e/d2a/f4d x:0 0 0 2024-09-21T04:35:04.082 INFO:tasks.workunit.client.1.smithi053.stdout:0/145: creat d5/f30 x:0 0 0 2024-09-21T04:35:04.083 INFO:tasks.workunit.client.1.smithi053.stdout:5/274: readlink d0/dd/d30/l3e 0 2024-09-21T04:35:04.083 INFO:tasks.workunit.client.1.smithi053.stdout:6/226: chown d2/d1b/d1e/f3f 5 1 2024-09-21T04:35:04.083 INFO:tasks.workunit.client.1.smithi053.stdout:0/146: fdatasync d5/f15 0 2024-09-21T04:35:04.085 INFO:tasks.workunit.client.1.smithi053.stdout:2/213: link d1/d18/d1f/d22/d2d/c3e d1/d18/d1f/d22/d43/c46 0 2024-09-21T04:35:04.085 INFO:tasks.workunit.client.1.smithi053.stdout:7/191: fsync d0/d6/ff 0 2024-09-21T04:35:04.085 INFO:tasks.workunit.client.1.smithi053.stdout:9/258: sync 2024-09-21T04:35:04.087 INFO:tasks.workunit.client.1.smithi053.stdout:9/259: fsync d0/d2/fe 0 2024-09-21T04:35:04.087 INFO:tasks.workunit.client.1.smithi053.stdout:4/219: dwrite d1/fc [0,4194304] 0 2024-09-21T04:35:04.088 INFO:tasks.workunit.client.1.smithi053.stdout:4/220: chown d1/d2c 1995 1 2024-09-21T04:35:04.089 INFO:tasks.workunit.client.1.smithi053.stdout:6/227: creat d2/d1b/d3d/d49/d1a/f4e x:0 0 0 2024-09-21T04:35:04.091 INFO:tasks.workunit.client.1.smithi053.stdout:0/147: dread d5/f15 [0,4194304] 0 2024-09-21T04:35:04.092 INFO:tasks.workunit.client.1.smithi053.stdout:6/228: stat d2/d1b/d3d/d49/d1a/d2e/d38/f36 0 2024-09-21T04:35:04.093 INFO:tasks.workunit.client.1.smithi053.stdout:2/214: stat d1/c3c 0 2024-09-21T04:35:04.093 INFO:tasks.workunit.client.1.smithi053.stdout:7/192: symlink d0/d2b/l3b 0 2024-09-21T04:35:04.096 INFO:tasks.workunit.client.1.smithi053.stdout:9/260: dwrite d0/d28/f3a [0,4194304] 0 2024-09-21T04:35:04.098 INFO:tasks.workunit.client.1.smithi053.stdout:4/221: stat d1/l7 0 2024-09-21T04:35:04.102 INFO:tasks.workunit.client.1.smithi053.stdout:9/261: dwrite d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:04.104 INFO:tasks.workunit.client.1.smithi053.stdout:0/148: mknod d5/d10/c31 0 2024-09-21T04:35:04.108 INFO:tasks.workunit.client.1.smithi053.stdout:7/193: rename d0/d6/f10 to d0/d2b/d36/f3c 0 2024-09-21T04:35:04.109 INFO:tasks.workunit.client.1.smithi053.stdout:9/262: dread d0/d2/fa [0,4194304] 0 2024-09-21T04:35:04.109 INFO:tasks.workunit.client.1.smithi053.stdout:7/194: fsync d0/f3a 0 2024-09-21T04:35:04.110 INFO:tasks.workunit.client.1.smithi053.stdout:9/263: read - d0/df/f48 zero size 2024-09-21T04:35:04.110 INFO:tasks.workunit.client.1.smithi053.stdout:7/195: chown d0/d6/l33 105 1 2024-09-21T04:35:04.112 INFO:tasks.workunit.client.1.smithi053.stdout:5/275: creat d0/dd/d30/d51/d52/f56 x:0 0 0 2024-09-21T04:35:04.113 INFO:tasks.workunit.client.1.smithi053.stdout:0/149: truncate d5/d6/f16 1799205 0 2024-09-21T04:35:04.113 INFO:tasks.workunit.client.1.smithi053.stdout:7/196: chown d0/d2b/f2e 212249628 1 2024-09-21T04:35:04.113 INFO:tasks.workunit.client.1.smithi053.stdout:4/222: write d1/dd/de/d14/d20/f34 [3043179,86324] 0 2024-09-21T04:35:04.114 INFO:tasks.workunit.client.1.smithi053.stdout:0/150: stat d5/de/d1c/f29 0 2024-09-21T04:35:04.115 INFO:tasks.workunit.client.1.smithi053.stdout:7/197: stat d0/d6/f18 0 2024-09-21T04:35:04.116 INFO:tasks.workunit.client.1.smithi053.stdout:9/264: dread d0/d1a/f1b [0,4194304] 0 2024-09-21T04:35:04.116 INFO:tasks.workunit.client.1.smithi053.stdout:5/276: read d0/dd/f3a [1617081,71216] 0 2024-09-21T04:35:04.117 INFO:tasks.workunit.client.1.smithi053.stdout:0/151: chown d5/d10/c31 1048703 1 2024-09-21T04:35:04.117 INFO:tasks.workunit.client.1.smithi053.stdout:6/229: creat d2/d1b/d1e/f4f x:0 0 0 2024-09-21T04:35:04.119 INFO:tasks.workunit.client.1.smithi053.stdout:9/265: dread - d0/d1c/d25/f54 zero size 2024-09-21T04:35:04.120 INFO:tasks.workunit.client.1.smithi053.stdout:0/152: write d5/f30 [52867,8921] 0 2024-09-21T04:35:04.120 INFO:tasks.workunit.client.1.smithi053.stdout:5/277: dread - d0/dd/d30/d51/d52/f56 zero size 2024-09-21T04:35:04.121 INFO:tasks.workunit.client.1.smithi053.stdout:0/153: fsync d5/d10/f2b 0 2024-09-21T04:35:04.121 INFO:tasks.workunit.client.1.smithi053.stdout:9/266: dread - d0/d1c/f44 zero size 2024-09-21T04:35:04.121 INFO:tasks.workunit.client.1.smithi053.stdout:2/215: getdents d1/d18/d1f 0 2024-09-21T04:35:04.123 INFO:tasks.workunit.client.1.smithi053.stdout:4/223: rename d1/fa to d1/d2c/f40 0 2024-09-21T04:35:04.127 INFO:tasks.workunit.client.1.smithi053.stdout:6/230: creat d2/d1d/f50 x:0 0 0 2024-09-21T04:35:04.132 INFO:tasks.workunit.client.1.smithi053.stdout:5/278: creat d0/d8/d2a/f57 x:0 0 0 2024-09-21T04:35:04.133 INFO:tasks.workunit.client.1.smithi053.stdout:6/231: dwrite d2/fd [0,4194304] 0 2024-09-21T04:35:04.138 INFO:tasks.workunit.client.1.smithi053.stdout:2/216: mknod d1/d18/c47 0 2024-09-21T04:35:04.139 INFO:tasks.workunit.client.1.smithi053.stdout:2/217: truncate d1/d18/d1f/f24 4213403 0 2024-09-21T04:35:04.139 INFO:tasks.workunit.client.1.smithi053.stdout:2/218: chown d1/d18/d1f/d22/f45 131633975 1 2024-09-21T04:35:04.142 INFO:tasks.workunit.client.1.smithi053.stdout:8/222: sync 2024-09-21T04:35:04.142 INFO:tasks.workunit.client.1.smithi053.stdout:1/302: sync 2024-09-21T04:35:04.142 INFO:tasks.workunit.client.1.smithi053.stdout:3/219: sync 2024-09-21T04:35:04.145 INFO:tasks.workunit.client.1.smithi053.stdout:9/267: rename d0/df/f27 to d0/d2/f55 0 2024-09-21T04:35:04.146 INFO:tasks.workunit.client.1.smithi053.stdout:1/303: dread d7/d24/f41 [0,4194304] 0 2024-09-21T04:35:04.146 INFO:tasks.workunit.client.1.smithi053.stdout:4/224: mkdir d1/d27/d33/d41 0 2024-09-21T04:35:04.148 INFO:tasks.workunit.client.1.smithi053.stdout:7/198: link d0/d6/f1e d0/d2b/d36/f3d 0 2024-09-21T04:35:04.155 INFO:tasks.workunit.client.1.smithi053.stdout:6/232: unlink d2/f28 0 2024-09-21T04:35:04.155 INFO:tasks.workunit.client.1.smithi053.stdout:6/233: read - d2/d1b/d3d/d49/d1a/f48 zero size 2024-09-21T04:35:04.161 INFO:tasks.workunit.client.1.smithi053.stdout:1/304: rmdir d7/d9/d43 39 2024-09-21T04:35:04.191 INFO:tasks.workunit.client.1.smithi053.stdout:8/223: creat d3/df/d49/f55 x:0 0 0 2024-09-21T04:35:04.192 INFO:tasks.workunit.client.1.smithi053.stdout:3/220: mkdir d16/d17/d21/d26/d49/d4d 0 2024-09-21T04:35:04.194 INFO:tasks.workunit.client.1.smithi053.stdout:1/305: creat d7/d9/da/d11/d2b/f61 x:0 0 0 2024-09-21T04:35:04.195 INFO:tasks.workunit.client.1.smithi053.stdout:3/221: fsync f11 0 2024-09-21T04:35:04.195 INFO:tasks.workunit.client.1.smithi053.stdout:0/154: dread d5/d6/f16 [0,4194304] 0 2024-09-21T04:35:04.196 INFO:tasks.workunit.client.1.smithi053.stdout:4/225: creat d1/f42 x:0 0 0 2024-09-21T04:35:04.199 INFO:tasks.workunit.client.1.smithi053.stdout:6/234: dwrite d2/d1b/d3d/d49/f4c [4194304,4194304] 0 2024-09-21T04:35:04.200 INFO:tasks.workunit.client.1.smithi053.stdout:1/306: rmdir d7/d9/da/d11 39 2024-09-21T04:35:04.201 INFO:tasks.workunit.client.1.smithi053.stdout:6/235: readlink d2/d1b/d1e/l31 0 2024-09-21T04:35:04.201 INFO:tasks.workunit.client.1.smithi053.stdout:1/307: dread - d7/d47/f4d zero size 2024-09-21T04:35:04.202 INFO:tasks.workunit.client.1.smithi053.stdout:3/222: mkdir d16/d17/d21/d4e 0 2024-09-21T04:35:04.202 INFO:tasks.workunit.client.1.smithi053.stdout:8/224: dwrite d3/d5/f19 [4194304,4194304] 0 2024-09-21T04:35:04.203 INFO:tasks.workunit.client.1.smithi053.stdout:6/236: chown d2/d1b/d1e/d27 0 1 2024-09-21T04:35:04.205 INFO:tasks.workunit.client.1.smithi053.stdout:6/237: dread d2/d1b/d3d/d49/de/f14 [0,4194304] 0 2024-09-21T04:35:04.205 INFO:tasks.workunit.client.1.smithi053.stdout:6/238: fsync d2/fa 0 2024-09-21T04:35:04.206 INFO:tasks.workunit.client.1.smithi053.stdout:0/155: mknod d5/d10/c32 0 2024-09-21T04:35:04.207 INFO:tasks.workunit.client.1.smithi053.stdout:6/239: write d2/f8 [517931,123280] 0 2024-09-21T04:35:04.213 INFO:tasks.workunit.client.1.smithi053.stdout:1/308: fdatasync d7/d9/da/d11/f33 0 2024-09-21T04:35:04.213 INFO:tasks.workunit.client.1.smithi053.stdout:5/279: dwrite d0/dd/f3a [0,4194304] 0 2024-09-21T04:35:04.215 INFO:tasks.workunit.client.1.smithi053.stdout:4/226: symlink d1/dd/l43 0 2024-09-21T04:35:04.217 INFO:tasks.workunit.client.1.smithi053.stdout:0/156: write d5/de/d1c/f29 [3655521,3812] 0 2024-09-21T04:35:04.217 INFO:tasks.workunit.client.1.smithi053.stdout:4/227: chown d1/dd/de/d14/d20/f30 865 1 2024-09-21T04:35:04.218 INFO:tasks.workunit.client.1.smithi053.stdout:3/223: unlink fe 0 2024-09-21T04:35:04.219 INFO:tasks.workunit.client.1.smithi053.stdout:2/219: dwrite d1/d18/d1f/d22/f33 [0,4194304] 0 2024-09-21T04:35:04.221 INFO:tasks.workunit.client.1.smithi053.stdout:1/309: symlink d7/d5a/l62 0 2024-09-21T04:35:04.221 INFO:tasks.workunit.client.1.smithi053.stdout:1/310: chown d7/d9/da/d11/f26 29960 1 2024-09-21T04:35:04.222 INFO:tasks.workunit.client.1.smithi053.stdout:1/311: stat d7 0 2024-09-21T04:35:04.224 INFO:tasks.workunit.client.1.smithi053.stdout:6/240: link d2/d1b/d3d/d49/f42 d2/d1b/d1e/d27/f51 0 2024-09-21T04:35:04.226 INFO:tasks.workunit.client.1.smithi053.stdout:0/157: creat d5/d6/d21/f33 x:0 0 0 2024-09-21T04:35:04.226 INFO:tasks.workunit.client.1.smithi053.stdout:0/158: fsync d5/d10/f2b 0 2024-09-21T04:35:04.226 INFO:tasks.workunit.client.1.smithi053.stdout:0/159: stat d5/f15 0 2024-09-21T04:35:04.230 INFO:tasks.workunit.client.1.smithi053.stdout:3/224: mknod d16/d17/d21/d26/d49/c4f 0 2024-09-21T04:35:04.230 INFO:tasks.workunit.client.1.smithi053.stdout:3/225: stat d16/d17/d21/c2e 0 2024-09-21T04:35:04.230 INFO:tasks.workunit.client.1.smithi053.stdout:3/226: chown d16/d17/d21/d26 18132742 1 2024-09-21T04:35:04.231 INFO:tasks.workunit.client.1.smithi053.stdout:2/220: creat d1/d18/d1f/d22/d2d/f48 x:0 0 0 2024-09-21T04:35:04.232 INFO:tasks.workunit.client.1.smithi053.stdout:5/280: read d0/d8/db/f19 [499181,2011] 0 2024-09-21T04:35:04.232 INFO:tasks.workunit.client.1.smithi053.stdout:1/312: rmdir d7/d9/da/d40 39 2024-09-21T04:35:04.233 INFO:tasks.workunit.client.1.smithi053.stdout:1/313: dread - d7/d5a/f4f zero size 2024-09-21T04:35:04.234 INFO:tasks.workunit.client.1.smithi053.stdout:6/241: dread - d2/d1b/d3d/d49/f20 zero size 2024-09-21T04:35:04.234 INFO:tasks.workunit.client.1.smithi053.stdout:4/228: symlink d1/dd/de/d14/d3f/l44 0 2024-09-21T04:35:04.236 INFO:tasks.workunit.client.1.smithi053.stdout:4/229: readlink d1/dd/l12 0 2024-09-21T04:35:04.236 INFO:tasks.workunit.client.1.smithi053.stdout:8/225: getdents d3/d5/d21 0 2024-09-21T04:35:04.237 INFO:tasks.workunit.client.1.smithi053.stdout:3/227: mknod d16/d46/c50 0 2024-09-21T04:35:04.238 INFO:tasks.workunit.client.1.smithi053.stdout:2/221: creat d1/d18/f49 x:0 0 0 2024-09-21T04:35:04.239 INFO:tasks.workunit.client.1.smithi053.stdout:4/230: write d1/dd/de/d14/d20/f34 [5138197,71217] 0 2024-09-21T04:35:04.240 INFO:tasks.workunit.client.1.smithi053.stdout:5/281: dwrite d0/d8/db/f29 [0,4194304] 0 2024-09-21T04:35:04.243 INFO:tasks.workunit.client.1.smithi053.stdout:1/314: mkdir d7/d63 0 2024-09-21T04:35:04.245 INFO:tasks.workunit.client.1.smithi053.stdout:0/160: creat d5/dd/f34 x:0 0 0 2024-09-21T04:35:04.247 INFO:tasks.workunit.client.1.smithi053.stdout:2/222: rename d1/d18/f1c to d1/d18/d1f/f4a 0 2024-09-21T04:35:04.248 INFO:tasks.workunit.client.1.smithi053.stdout:8/226: mknod d3/d5/c56 0 2024-09-21T04:35:04.249 INFO:tasks.workunit.client.1.smithi053.stdout:0/161: write d5/dd/ff [1575203,109753] 0 2024-09-21T04:35:04.251 INFO:tasks.workunit.client.1.smithi053.stdout:8/227: rename d3 to d3/df/d49/d57 22 2024-09-21T04:35:04.251 INFO:tasks.workunit.client.1.smithi053.stdout:0/162: dread - d5/dd/f34 zero size 2024-09-21T04:35:04.252 INFO:tasks.workunit.client.1.smithi053.stdout:3/228: dwrite d16/d1b/f27 [0,4194304] 0 2024-09-21T04:35:04.253 INFO:tasks.workunit.client.1.smithi053.stdout:6/242: dread d2/d1b/d3d/d49/de/f1f [0,4194304] 0 2024-09-21T04:35:04.254 INFO:tasks.workunit.client.1.smithi053.stdout:0/163: fsync d5/d6/f1b 0 2024-09-21T04:35:04.257 INFO:tasks.workunit.client.1.smithi053.stdout:1/315: mkdir d7/d9/da/d40/d64 0 2024-09-21T04:35:04.258 INFO:tasks.workunit.client.1.smithi053.stdout:1/316: write d7/d9/da/d11/f26 [11745793,50860] 0 2024-09-21T04:35:04.258 INFO:tasks.workunit.client.1.smithi053.stdout:8/228: mknod d3/d4f/c58 0 2024-09-21T04:35:04.258 INFO:tasks.workunit.client.1.smithi053.stdout:1/317: readlink d7/l38 0 2024-09-21T04:35:04.259 INFO:tasks.workunit.client.1.smithi053.stdout:8/229: readlink d3/l46 0 2024-09-21T04:35:04.259 INFO:tasks.workunit.client.1.smithi053.stdout:8/230: readlink d3/df/d14/l41 0 2024-09-21T04:35:04.259 INFO:tasks.workunit.client.1.smithi053.stdout:0/164: creat d5/de/f35 x:0 0 0 2024-09-21T04:35:04.260 INFO:tasks.workunit.client.1.smithi053.stdout:1/318: dread - d7/d5a/d36/f4e zero size 2024-09-21T04:35:04.260 INFO:tasks.workunit.client.1.smithi053.stdout:5/282: link d0/l1 d0/dd/d30/d51/d52/l58 0 2024-09-21T04:35:04.261 INFO:tasks.workunit.client.1.smithi053.stdout:0/165: chown d5/dd/c25 2 1 2024-09-21T04:35:04.263 INFO:tasks.workunit.client.1.smithi053.stdout:3/229: mkdir d16/d17/d21/d4e/d51 0 2024-09-21T04:35:04.265 INFO:tasks.workunit.client.1.smithi053.stdout:0/166: creat d5/d6/f36 x:0 0 0 2024-09-21T04:35:04.268 INFO:tasks.workunit.client.1.smithi053.stdout:1/319: rename d7/d9/da/d40/d58/f5d to d7/d9/da/d3c/f65 0 2024-09-21T04:35:04.269 INFO:tasks.workunit.client.1.smithi053.stdout:6/243: getdents d2/d1b/d1e/d2a/d4b 0 2024-09-21T04:35:04.272 INFO:tasks.workunit.client.1.smithi053.stdout:1/320: symlink d7/d9/da/d40/d49/l66 0 2024-09-21T04:35:04.273 INFO:tasks.workunit.client.1.smithi053.stdout:1/321: write d7/f56 [1094678,85635] 0 2024-09-21T04:35:04.273 INFO:tasks.workunit.client.1.smithi053.stdout:8/231: creat d3/f59 x:0 0 0 2024-09-21T04:35:04.274 INFO:tasks.workunit.client.1.smithi053.stdout:6/244: symlink d2/d1b/d3d/d49/d1a/d2e/d38/l52 0 2024-09-21T04:35:04.275 INFO:tasks.workunit.client.1.smithi053.stdout:3/230: creat d16/d17/d3c/f52 x:0 0 0 2024-09-21T04:35:04.277 INFO:tasks.workunit.client.1.smithi053.stdout:0/167: rename d5/c26 to d5/d6/c37 0 2024-09-21T04:35:04.279 INFO:tasks.workunit.client.1.smithi053.stdout:3/231: chown d16/d17/d21/d26/c28 82 1 2024-09-21T04:35:04.280 INFO:tasks.workunit.client.1.smithi053.stdout:1/322: dwrite d7/d9/da/d11/d2b/f61 [0,4194304] 0 2024-09-21T04:35:04.281 INFO:tasks.workunit.client.1.smithi053.stdout:0/168: creat d5/d6/f38 x:0 0 0 2024-09-21T04:35:04.283 INFO:tasks.workunit.client.1.smithi053.stdout:0/169: chown d5/d6/f16 213156940 1 2024-09-21T04:35:04.283 INFO:tasks.workunit.client.1.smithi053.stdout:1/323: chown d7/d9/da/c23 119162 1 2024-09-21T04:35:04.290 INFO:tasks.workunit.client.1.smithi053.stdout:8/232: rename d3/d4f/l54 to d3/d5/l5a 0 2024-09-21T04:35:04.293 INFO:tasks.workunit.client.1.smithi053.stdout:1/324: write d7/d9/da/d3c/f65 [119431,72474] 0 2024-09-21T04:35:04.299 INFO:tasks.workunit.client.1.smithi053.stdout:0/170: dwrite d5/d6/f1b [0,4194304] 0 2024-09-21T04:35:04.301 INFO:tasks.workunit.client.1.smithi053.stdout:1/325: creat d7/d9/f67 x:0 0 0 2024-09-21T04:35:04.302 INFO:tasks.workunit.client.1.smithi053.stdout:0/171: chown d5/d6/d21/c2d 252625865 1 2024-09-21T04:35:04.303 INFO:tasks.workunit.client.1.smithi053.stdout:0/172: readlink d5/l23 0 2024-09-21T04:35:04.303 INFO:tasks.workunit.client.1.smithi053.stdout:0/173: write d5/d10/f2b [2771315,12720] 0 2024-09-21T04:35:04.307 INFO:tasks.workunit.client.1.smithi053.stdout:0/174: creat d5/f39 x:0 0 0 2024-09-21T04:35:04.307 INFO:tasks.workunit.client.1.smithi053.stdout:1/326: creat d7/f68 x:0 0 0 2024-09-21T04:35:04.309 INFO:tasks.workunit.client.1.smithi053.stdout:0/175: write d5/de/d1c/f29 [126993,109484] 0 2024-09-21T04:35:04.311 INFO:tasks.workunit.client.1.smithi053.stdout:8/233: dread d3/d28/f2e [0,4194304] 0 2024-09-21T04:35:04.311 INFO:tasks.workunit.client.1.smithi053.stdout:0/176: write d5/de/f2f [304802,10300] 0 2024-09-21T04:35:04.311 INFO:tasks.workunit.client.1.smithi053.stdout:1/327: mkdir d7/d9/da/d11/d2b/d69 0 2024-09-21T04:35:04.312 INFO:tasks.workunit.client.1.smithi053.stdout:8/234: chown d3/d28/f31 15454 1 2024-09-21T04:35:04.313 INFO:tasks.workunit.client.1.smithi053.stdout:1/328: readlink d7/d9/da/d11/l42 0 2024-09-21T04:35:04.314 INFO:tasks.workunit.client.1.smithi053.stdout:0/177: unlink d5/f30 0 2024-09-21T04:35:04.317 INFO:tasks.workunit.client.1.smithi053.stdout:8/235: creat d3/d28/f5b x:0 0 0 2024-09-21T04:35:04.318 INFO:tasks.workunit.client.1.smithi053.stdout:1/329: dread f0 [0,4194304] 0 2024-09-21T04:35:04.319 INFO:tasks.workunit.client.1.smithi053.stdout:8/236: mkdir d3/d5/d4b/d5c 0 2024-09-21T04:35:04.321 INFO:tasks.workunit.client.1.smithi053.stdout:0/178: dwrite d5/dd/ff [0,4194304] 0 2024-09-21T04:35:04.324 INFO:tasks.workunit.client.1.smithi053.stdout:8/237: symlink d3/df/d14/l5d 0 2024-09-21T04:35:04.325 INFO:tasks.workunit.client.1.smithi053.stdout:1/330: dwrite d7/d9/da/d11/f33 [0,4194304] 0 2024-09-21T04:35:04.333 INFO:tasks.workunit.client.1.smithi053.stdout:1/331: dread d7/f56 [0,4194304] 0 2024-09-21T04:35:04.334 INFO:tasks.workunit.client.1.smithi053.stdout:8/238: mknod d3/d5/d4c/d53/c5e 0 2024-09-21T04:35:04.334 INFO:tasks.workunit.client.1.smithi053.stdout:0/179: link d5/c7 d5/c3a 0 2024-09-21T04:35:04.335 INFO:tasks.workunit.client.1.smithi053.stdout:8/239: write d3/f3a [41206,22271] 0 2024-09-21T04:35:04.337 INFO:tasks.workunit.client.1.smithi053.stdout:1/332: rename d7/f68 to d7/d63/f6a 0 2024-09-21T04:35:04.338 INFO:tasks.workunit.client.1.smithi053.stdout:0/180: creat d5/f3b x:0 0 0 2024-09-21T04:35:04.345 INFO:tasks.workunit.client.1.smithi053.stdout:0/181: mknod d5/dd/c3c 0 2024-09-21T04:35:04.346 INFO:tasks.workunit.client.1.smithi053.stdout:0/182: fdatasync d5/d6/f36 0 2024-09-21T04:35:04.346 INFO:tasks.workunit.client.1.smithi053.stdout:0/183: chown d5/d6/f38 161941939 1 2024-09-21T04:35:04.348 INFO:tasks.workunit.client.1.smithi053.stdout:0/184: creat d5/de/d1c/f3d x:0 0 0 2024-09-21T04:35:04.349 INFO:tasks.workunit.client.1.smithi053.stdout:0/185: rmdir d5/de 39 2024-09-21T04:35:04.351 INFO:tasks.workunit.client.1.smithi053.stdout:0/186: write d5/de/d1c/f3d [359934,119492] 0 2024-09-21T04:35:04.394 INFO:tasks.workunit.client.1.smithi053.stdout:2/223: mkdir d1/d18/d1f/d27/d41/d4b 0 2024-09-21T04:35:04.398 INFO:tasks.workunit.client.1.smithi053.stdout:2/224: rename d1/d18/d1f/f37 to d1/d18/d1f/d27/d41/f4c 0 2024-09-21T04:35:04.401 INFO:tasks.workunit.client.1.smithi053.stdout:4/231: truncate d1/dd/f11 1795831 0 2024-09-21T04:35:04.405 INFO:tasks.workunit.client.1.smithi053.stdout:4/232: dread d1/d27/f31 [0,4194304] 0 2024-09-21T04:35:04.407 INFO:tasks.workunit.client.1.smithi053.stdout:4/233: creat d1/dd/de/d14/d20/f45 x:0 0 0 2024-09-21T04:35:04.408 INFO:tasks.workunit.client.1.smithi053.stdout:4/234: chown d1/l5 11746548 1 2024-09-21T04:35:04.409 INFO:tasks.workunit.client.1.smithi053.stdout:4/235: truncate d1/f42 366819 0 2024-09-21T04:35:04.410 INFO:tasks.workunit.client.1.smithi053.stdout:4/236: write d1/dd/de/f2b [768370,32443] 0 2024-09-21T04:35:04.412 INFO:tasks.workunit.client.1.smithi053.stdout:4/237: mkdir d1/d27/d2e/d46 0 2024-09-21T04:35:04.413 INFO:tasks.workunit.client.1.smithi053.stdout:4/238: mknod d1/d27/d33/c47 0 2024-09-21T04:35:04.415 INFO:tasks.workunit.client.1.smithi053.stdout:4/239: symlink d1/dd/de/d14/d3f/l48 0 2024-09-21T04:35:04.417 INFO:tasks.workunit.client.1.smithi053.stdout:4/240: mknod d1/dd/de/d14/d3f/c49 0 2024-09-21T04:35:04.437 INFO:tasks.workunit.client.1.smithi053.stdout:8/240: truncate d3/df/f42 282125 0 2024-09-21T04:35:04.438 INFO:tasks.workunit.client.1.smithi053.stdout:8/241: chown d3/d5/d38/f4e 248 1 2024-09-21T04:35:04.440 INFO:tasks.workunit.client.1.smithi053.stdout:1/333: dwrite d7/d24/f41 [0,4194304] 0 2024-09-21T04:35:04.440 INFO:tasks.workunit.client.1.smithi053.stdout:0/187: truncate d5/de/f2f 99272 0 2024-09-21T04:35:04.442 INFO:tasks.workunit.client.1.smithi053.stdout:1/334: write d7/d9/da/d3c/f46 [3116612,20133] 0 2024-09-21T04:35:04.442 INFO:tasks.workunit.client.1.smithi053.stdout:8/242: mknod d3/d5/d21/c5f 0 2024-09-21T04:35:04.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:04 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:04.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:04 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:04.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:04 smithi053 ceph-mon[35554]: Reconfiguring ceph-exporter.smithi053 (dependencies changed)... 2024-09-21T04:35:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:04 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:35:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:04 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:35:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:04 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:35:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:04 smithi053 ceph-mon[35554]: Deploying daemon ceph-exporter.smithi053 on smithi053 2024-09-21T04:35:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:04 smithi053 ceph-mon[35554]: pgmap v23: 65 pgs: 65 active+clean; 252 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 279 op/s 2024-09-21T04:35:04.443 INFO:tasks.workunit.client.1.smithi053.stdout:1/335: dread - d7/d9/da/f5b zero size 2024-09-21T04:35:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:04 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:04 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:04 smithi007 ceph-mon[27954]: Reconfiguring ceph-exporter.smithi053 (dependencies changed)... 2024-09-21T04:35:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:04 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:35:04.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:04 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-21T04:35:04.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:04 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:35:04.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:04 smithi007 ceph-mon[27954]: Deploying daemon ceph-exporter.smithi053 on smithi053 2024-09-21T04:35:04.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:04 smithi007 ceph-mon[27954]: pgmap v23: 65 pgs: 65 active+clean; 252 MiB data, 972 MiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 279 op/s 2024-09-21T04:35:04.451 INFO:tasks.workunit.client.1.smithi053.stdout:8/243: creat d3/df/d14/f60 x:0 0 0 2024-09-21T04:35:04.455 INFO:tasks.workunit.client.1.smithi053.stdout:1/336: dwrite d7/d63/f6a [0,4194304] 0 2024-09-21T04:35:04.457 INFO:tasks.workunit.client.1.smithi053.stdout:2/225: fdatasync d1/d18/d1f/d22/f36 0 2024-09-21T04:35:04.460 INFO:tasks.workunit.client.1.smithi053.stdout:1/337: mkdir d7/d63/d6b 0 2024-09-21T04:35:04.464 INFO:tasks.workunit.client.1.smithi053.stdout:1/338: fdatasync f0 0 2024-09-21T04:35:04.465 INFO:tasks.workunit.client.1.smithi053.stdout:1/339: truncate d7/d9/f67 409220 0 2024-09-21T04:35:04.467 INFO:tasks.workunit.client.1.smithi053.stdout:2/226: mkdir d1/d18/d1f/d27/d41/d4b/d4d 0 2024-09-21T04:35:04.468 INFO:tasks.workunit.client.1.smithi053.stdout:1/340: symlink d7/l6c 0 2024-09-21T04:35:04.469 INFO:tasks.workunit.client.1.smithi053.stdout:2/227: dread d1/d18/d1f/d22/f36 [0,4194304] 0 2024-09-21T04:35:04.471 INFO:tasks.workunit.client.1.smithi053.stdout:2/228: truncate d1/d18/d1f/d27/d41/f42 680012 0 2024-09-21T04:35:04.474 INFO:tasks.workunit.client.1.smithi053.stdout:8/244: chown d3/d5/d21/c5f 130124348 1 2024-09-21T04:35:04.474 INFO:tasks.workunit.client.1.smithi053.stdout:8/245: dread - d3/df/d14/d16/f35 zero size 2024-09-21T04:35:04.475 INFO:tasks.workunit.client.1.smithi053.stdout:1/341: creat d7/d9/d43/f6d x:0 0 0 2024-09-21T04:35:04.476 INFO:tasks.workunit.client.1.smithi053.stdout:0/188: truncate d5/d10/f2b 3841813 0 2024-09-21T04:35:04.476 INFO:tasks.workunit.client.1.smithi053.stdout:8/246: chown d3/df/d14/d16/f29 14201 1 2024-09-21T04:35:04.476 INFO:tasks.workunit.client.1.smithi053.stdout:2/229: rmdir d1/d18/d1f/d27/d41/d4b 39 2024-09-21T04:35:04.476 INFO:tasks.workunit.client.1.smithi053.stdout:0/189: fdatasync d5/de/d1c/f29 0 2024-09-21T04:35:04.477 INFO:tasks.workunit.client.1.smithi053.stdout:2/230: read d1/d18/d1f/d22/f33 [1920200,40866] 0 2024-09-21T04:35:04.479 INFO:tasks.workunit.client.1.smithi053.stdout:2/231: chown d1/d18/d1f/d22/f45 18 1 2024-09-21T04:35:04.481 INFO:tasks.workunit.client.1.smithi053.stdout:8/247: creat d3/df/d49/f61 x:0 0 0 2024-09-21T04:35:04.483 INFO:tasks.workunit.client.1.smithi053.stdout:0/190: creat d5/de/f3e x:0 0 0 2024-09-21T04:35:04.485 INFO:tasks.workunit.client.1.smithi053.stdout:0/191: write d5/dd/f34 [816650,29775] 0 2024-09-21T04:35:04.486 INFO:tasks.workunit.client.1.smithi053.stdout:2/232: dwrite d1/d18/d1f/d22/f45 [0,4194304] 0 2024-09-21T04:35:04.487 INFO:tasks.workunit.client.1.smithi053.stdout:8/248: creat d3/df/f62 x:0 0 0 2024-09-21T04:35:04.492 INFO:tasks.workunit.client.1.smithi053.stdout:4/241: dwrite d1/dd/f11 [0,4194304] 0 2024-09-21T04:35:04.493 INFO:tasks.workunit.client.1.smithi053.stdout:8/249: readlink d3/d5/d4b/l51 0 2024-09-21T04:35:04.494 INFO:tasks.workunit.client.1.smithi053.stdout:8/250: write d3/d5/d38/f47 [729261,2105] 0 2024-09-21T04:35:04.499 INFO:tasks.workunit.client.1.smithi053.stdout:0/192: rename d5/d6/f38 to d5/d6/d21/f3f 0 2024-09-21T04:35:04.499 INFO:tasks.workunit.client.1.smithi053.stdout:1/342: write d7/d9/f19 [2348245,28794] 0 2024-09-21T04:35:04.500 INFO:tasks.workunit.client.1.smithi053.stdout:2/233: symlink d1/d18/d1f/d27/l4e 0 2024-09-21T04:35:04.500 INFO:tasks.workunit.client.1.smithi053.stdout:0/193: chown d5/d6/f36 13963 1 2024-09-21T04:35:04.500 INFO:tasks.workunit.client.1.smithi053.stdout:2/234: fsync d1/fb 0 2024-09-21T04:35:04.501 INFO:tasks.workunit.client.1.smithi053.stdout:0/194: write d5/dd/f34 [40951,103392] 0 2024-09-21T04:35:04.502 INFO:tasks.workunit.client.1.smithi053.stdout:0/195: read d5/d6/f16 [1386495,51543] 0 2024-09-21T04:35:04.503 INFO:tasks.workunit.client.1.smithi053.stdout:6/245: sync 2024-09-21T04:35:04.503 INFO:tasks.workunit.client.1.smithi053.stdout:5/283: sync 2024-09-21T04:35:04.503 INFO:tasks.workunit.client.1.smithi053.stdout:7/199: sync 2024-09-21T04:35:04.503 INFO:tasks.workunit.client.1.smithi053.stdout:9/268: sync 2024-09-21T04:35:04.503 INFO:tasks.workunit.client.1.smithi053.stdout:3/232: sync 2024-09-21T04:35:04.505 INFO:tasks.workunit.client.1.smithi053.stdout:5/284: readlink d0/d8/db/l2e 0 2024-09-21T04:35:04.506 INFO:tasks.workunit.client.1.smithi053.stdout:0/196: truncate d5/d6/f36 386738 0 2024-09-21T04:35:04.506 INFO:tasks.workunit.client.1.smithi053.stdout:6/246: write d2/d1b/d1e/d2a/f4d [729694,41671] 0 2024-09-21T04:35:04.507 INFO:tasks.workunit.client.1.smithi053.stdout:4/242: chown d1/f2d 4847 1 2024-09-21T04:35:04.508 INFO:tasks.workunit.client.1.smithi053.stdout:8/251: chown d3/d5/d4c/c52 9211394 1 2024-09-21T04:35:04.511 INFO:tasks.workunit.client.1.smithi053.stdout:1/343: creat d7/d9/d43/f6e x:0 0 0 2024-09-21T04:35:04.516 INFO:tasks.workunit.client.1.smithi053.stdout:9/269: chown d0/d2/c41 0 1 2024-09-21T04:35:04.516 INFO:tasks.workunit.client.1.smithi053.stdout:7/200: symlink d0/d2b/l3e 0 2024-09-21T04:35:04.517 INFO:tasks.workunit.client.1.smithi053.stdout:3/233: rmdir d16/d1b 39 2024-09-21T04:35:04.517 INFO:tasks.workunit.client.1.smithi053.stdout:1/344: dwrite d7/d9/da/d11/d2b/f61 [0,4194304] 0 2024-09-21T04:35:04.518 INFO:tasks.workunit.client.1.smithi053.stdout:7/201: fdatasync d0/d6/f29 0 2024-09-21T04:35:04.521 INFO:tasks.workunit.client.1.smithi053.stdout:5/285: rename d0/f26 to d0/dd/d30/d51/d52/f59 0 2024-09-21T04:35:04.522 INFO:tasks.workunit.client.1.smithi053.stdout:5/286: fsync d0/d8/f31 0 2024-09-21T04:35:04.526 INFO:tasks.workunit.client.1.smithi053.stdout:6/247: unlink d2/d1b/d3d/d49/de/f1f 0 2024-09-21T04:35:04.529 INFO:tasks.workunit.client.1.smithi053.stdout:2/235: creat d1/d18/d1f/d22/d43/f4f x:0 0 0 2024-09-21T04:35:04.529 INFO:tasks.workunit.client.1.smithi053.stdout:8/252: dread d3/d39/f3f [0,4194304] 0 2024-09-21T04:35:04.530 INFO:tasks.workunit.client.1.smithi053.stdout:2/236: fdatasync d1/d18/f49 0 2024-09-21T04:35:04.531 INFO:tasks.workunit.client.1.smithi053.stdout:9/270: creat d0/d2/d9/d34/f56 x:0 0 0 2024-09-21T04:35:04.532 INFO:tasks.workunit.client.1.smithi053.stdout:9/271: write d0/d1c/f39 [2277607,80737] 0 2024-09-21T04:35:04.534 INFO:tasks.workunit.client.1.smithi053.stdout:8/253: dread d3/d5/d38/f3d [0,4194304] 0 2024-09-21T04:35:04.537 INFO:tasks.workunit.client.1.smithi053.stdout:0/197: mknod d5/c40 0 2024-09-21T04:35:04.539 INFO:tasks.workunit.client.1.smithi053.stdout:7/202: rename d0/d2b/l3e to d0/d2b/l3f 0 2024-09-21T04:35:04.540 INFO:tasks.workunit.client.1.smithi053.stdout:7/203: fdatasync d0/d6/f1b 0 2024-09-21T04:35:04.543 INFO:tasks.workunit.client.1.smithi053.stdout:5/287: symlink d0/dd/d30/d51/l5a 0 2024-09-21T04:35:04.546 INFO:tasks.workunit.client.1.smithi053.stdout:6/248: creat d2/d1b/f53 x:0 0 0 2024-09-21T04:35:04.547 INFO:tasks.workunit.client.1.smithi053.stdout:9/272: dread d0/d1a/f2c [0,4194304] 0 2024-09-21T04:35:04.556 INFO:tasks.workunit.client.1.smithi053.stdout:8/254: unlink d3/df/d14/l5d 0 2024-09-21T04:35:04.556 INFO:tasks.workunit.client.1.smithi053.stdout:1/345: getdents d7/d24 0 2024-09-21T04:35:04.557 INFO:tasks.workunit.client.1.smithi053.stdout:8/255: truncate d3/f59 598773 0 2024-09-21T04:35:04.557 INFO:tasks.workunit.client.1.smithi053.stdout:1/346: dread d7/d9/f67 [0,4194304] 0 2024-09-21T04:35:04.558 INFO:tasks.workunit.client.1.smithi053.stdout:1/347: fdatasync d7/d9/da/d11/d2b/f61 0 2024-09-21T04:35:04.559 INFO:tasks.workunit.client.1.smithi053.stdout:3/234: mkdir d16/d17/d21/d26/d49/d4d/d53 0 2024-09-21T04:35:04.561 INFO:tasks.workunit.client.1.smithi053.stdout:7/204: rename d0/d6/l33 to d0/d2b/d36/l40 0 2024-09-21T04:35:04.563 INFO:tasks.workunit.client.1.smithi053.stdout:6/249: dread d2/d1b/f3a [0,4194304] 0 2024-09-21T04:35:04.563 INFO:tasks.workunit.client.1.smithi053.stdout:7/205: write d0/d6/f1b [1720514,61175] 0 2024-09-21T04:35:04.564 INFO:tasks.workunit.client.1.smithi053.stdout:4/243: creat d1/f4a x:0 0 0 2024-09-21T04:35:04.566 INFO:tasks.workunit.client.1.smithi053.stdout:8/256: dwrite d3/df/d10/f48 [0,4194304] 0 2024-09-21T04:35:04.566 INFO:tasks.workunit.client.1.smithi053.stdout:5/288: write d0/d8/f2b [5253291,74820] 0 2024-09-21T04:35:04.569 INFO:tasks.workunit.client.1.smithi053.stdout:8/257: write d3/d39/f2f [92554,68497] 0 2024-09-21T04:35:04.570 INFO:tasks.workunit.client.1.smithi053.stdout:8/258: fdatasync d3/df/d49/f55 0 2024-09-21T04:35:04.571 INFO:tasks.workunit.client.1.smithi053.stdout:9/273: mknod d0/d2/c57 0 2024-09-21T04:35:04.571 INFO:tasks.workunit.client.1.smithi053.stdout:8/259: write d3/df/d14/d16/f23 [676015,21846] 0 2024-09-21T04:35:04.571 INFO:tasks.workunit.client.1.smithi053.stdout:9/274: read d0/f4 [349991,65622] 0 2024-09-21T04:35:04.582 INFO:tasks.workunit.client.1.smithi053.stdout:1/348: mknod d7/d9/da/d40/d49/c6f 0 2024-09-21T04:35:04.586 INFO:tasks.workunit.client.1.smithi053.stdout:7/206: creat d0/d2b/f41 x:0 0 0 2024-09-21T04:35:04.587 INFO:tasks.workunit.client.1.smithi053.stdout:4/244: symlink d1/d27/d33/l4b 0 2024-09-21T04:35:04.588 INFO:tasks.workunit.client.1.smithi053.stdout:4/245: read d1/dd/de/f2b [441308,40257] 0 2024-09-21T04:35:04.590 INFO:tasks.workunit.client.1.smithi053.stdout:8/260: symlink d3/df/d10/l63 0 2024-09-21T04:35:04.590 INFO:tasks.workunit.client.1.smithi053.stdout:8/261: chown d3 2898 1 2024-09-21T04:35:04.590 INFO:tasks.workunit.client.1.smithi053.stdout:9/275: mkdir d0/d2/d9/d34/d58 0 2024-09-21T04:35:04.592 INFO:tasks.workunit.client.1.smithi053.stdout:2/237: symlink d1/d18/d1f/d27/d41/d4b/d4d/l50 0 2024-09-21T04:35:04.593 INFO:tasks.workunit.client.1.smithi053.stdout:2/238: fdatasync d1/d18/d1f/d22/d2d/f48 0 2024-09-21T04:35:04.594 INFO:tasks.workunit.client.1.smithi053.stdout:3/235: creat d16/d17/d21/d4e/d51/f54 x:0 0 0 2024-09-21T04:35:04.594 INFO:tasks.workunit.client.1.smithi053.stdout:3/236: stat d16/d46/c50 0 2024-09-21T04:35:04.595 INFO:tasks.workunit.client.1.smithi053.stdout:9/276: dread d0/d28/f38 [0,4194304] 0 2024-09-21T04:35:04.597 INFO:tasks.workunit.client.1.smithi053.stdout:1/349: fsync d7/d9/da/f2f 0 2024-09-21T04:35:04.601 INFO:tasks.workunit.client.1.smithi053.stdout:4/246: unlink d1/dd/f11 0 2024-09-21T04:35:04.602 INFO:tasks.workunit.client.1.smithi053.stdout:8/262: mkdir d3/d5/d38/d64 0 2024-09-21T04:35:04.605 INFO:tasks.workunit.client.1.smithi053.stdout:2/239: rename d1/l2c to d1/d18/d1f/d22/d43/l51 0 2024-09-21T04:35:04.605 INFO:tasks.workunit.client.1.smithi053.stdout:8/263: chown d3/d5/c33 0 1 2024-09-21T04:35:04.607 INFO:tasks.workunit.client.1.smithi053.stdout:9/277: dwrite d0/d1c/d25/f54 [0,4194304] 0 2024-09-21T04:35:04.609 INFO:tasks.workunit.client.1.smithi053.stdout:1/350: dwrite d7/d9/f1b [0,4194304] 0 2024-09-21T04:35:04.610 INFO:tasks.workunit.client.1.smithi053.stdout:5/289: write d0/d8/f10 [1047442,130581] 0 2024-09-21T04:35:04.611 INFO:tasks.workunit.client.1.smithi053.stdout:9/278: write d0/d1c/f39 [273754,87589] 0 2024-09-21T04:35:04.612 INFO:tasks.workunit.client.1.smithi053.stdout:3/237: dread d16/f1d [0,4194304] 0 2024-09-21T04:35:04.616 INFO:tasks.workunit.client.1.smithi053.stdout:4/247: mknod d1/dd/de/c4c 0 2024-09-21T04:35:04.619 INFO:tasks.workunit.client.1.smithi053.stdout:5/290: dread d0/f3b [0,4194304] 0 2024-09-21T04:35:04.619 INFO:tasks.workunit.client.1.smithi053.stdout:6/250: getdents d2/d1b 0 2024-09-21T04:35:04.620 INFO:tasks.workunit.client.1.smithi053.stdout:1/351: mkdir d7/d24/d70 0 2024-09-21T04:35:04.622 INFO:tasks.workunit.client.1.smithi053.stdout:9/279: creat d0/d2/d9/d34/f59 x:0 0 0 2024-09-21T04:35:04.623 INFO:tasks.workunit.client.1.smithi053.stdout:4/248: mknod d1/d2c/c4d 0 2024-09-21T04:35:04.623 INFO:tasks.workunit.client.1.smithi053.stdout:9/280: dread d0/d1a/f1b [0,4194304] 0 2024-09-21T04:35:04.625 INFO:tasks.workunit.client.1.smithi053.stdout:9/281: write d0/d1c/f39 [2969139,45078] 0 2024-09-21T04:35:04.631 INFO:tasks.workunit.client.1.smithi053.stdout:1/352: mknod d7/d9/da/d40/d58/c71 0 2024-09-21T04:35:04.633 INFO:tasks.workunit.client.1.smithi053.stdout:4/249: chown d1/cb 56622870 1 2024-09-21T04:35:04.633 INFO:tasks.workunit.client.1.smithi053.stdout:4/250: readlink d1/d2c/l32 0 2024-09-21T04:35:04.637 INFO:tasks.workunit.client.1.smithi053.stdout:1/353: dwrite f1 [0,4194304] 0 2024-09-21T04:35:04.637 INFO:tasks.workunit.client.1.smithi053.stdout:9/282: symlink d0/l5a 0 2024-09-21T04:35:04.639 INFO:tasks.workunit.client.1.smithi053.stdout:2/240: write d1/d18/d1f/d22/f33 [4696478,51878] 0 2024-09-21T04:35:04.641 INFO:tasks.workunit.client.1.smithi053.stdout:6/251: chown d2/d1b/d1e/d2a/f3c 65596891 1 2024-09-21T04:35:04.642 INFO:tasks.workunit.client.1.smithi053.stdout:8/264: getdents d3/df/d14/d16 0 2024-09-21T04:35:04.645 INFO:tasks.workunit.client.1.smithi053.stdout:1/354: dread f0 [0,4194304] 0 2024-09-21T04:35:04.645 INFO:tasks.workunit.client.1.smithi053.stdout:5/291: creat d0/f5b x:0 0 0 2024-09-21T04:35:04.645 INFO:tasks.workunit.client.1.smithi053.stdout:8/265: fdatasync d3/df/d14/f60 0 2024-09-21T04:35:04.646 INFO:tasks.workunit.client.1.smithi053.stdout:6/252: write d2/d1b/d3d/d49/de/f43 [1014108,51094] 0 2024-09-21T04:35:04.648 INFO:tasks.workunit.client.1.smithi053.stdout:2/241: dread d1/d18/d1f/d22/f45 [0,4194304] 0 2024-09-21T04:35:04.653 INFO:tasks.workunit.client.1.smithi053.stdout:8/266: dread d3/d28/f2e [0,4194304] 0 2024-09-21T04:35:04.656 INFO:tasks.workunit.client.1.smithi053.stdout:3/238: dwrite d16/d17/f1e [0,4194304] 0 2024-09-21T04:35:04.657 INFO:tasks.workunit.client.1.smithi053.stdout:4/251: write d1/dd/de/f2b [995109,84644] 0 2024-09-21T04:35:04.659 INFO:tasks.workunit.client.1.smithi053.stdout:4/252: write d1/f42 [781454,111232] 0 2024-09-21T04:35:04.663 INFO:tasks.workunit.client.1.smithi053.stdout:4/253: write d1/dd/de/d14/d20/f30 [893386,1031] 0 2024-09-21T04:35:04.664 INFO:tasks.workunit.client.1.smithi053.stdout:8/267: creat d3/d28/f65 x:0 0 0 2024-09-21T04:35:04.666 INFO:tasks.workunit.client.1.smithi053.stdout:8/268: write d3/df/f62 [83503,69299] 0 2024-09-21T04:35:04.667 INFO:tasks.workunit.client.1.smithi053.stdout:1/355: rename d7/d5a/c30 to d7/d9/c72 0 2024-09-21T04:35:04.668 INFO:tasks.workunit.client.1.smithi053.stdout:9/283: symlink d0/d1c/d21/l5b 0 2024-09-21T04:35:04.670 INFO:tasks.workunit.client.1.smithi053.stdout:4/254: mkdir d1/dd/de/d14/d20/d4e 0 2024-09-21T04:35:04.670 INFO:tasks.workunit.client.1.smithi053.stdout:3/239: unlink d16/d17/c33 0 2024-09-21T04:35:04.672 INFO:tasks.workunit.client.1.smithi053.stdout:2/242: rename d1/d18/d1f/d22/d2d/f48 to d1/d18/d1f/d27/d41/d4b/d4d/f52 0 2024-09-21T04:35:04.672 INFO:tasks.workunit.client.1.smithi053.stdout:1/356: truncate d7/d5a/d36/f4e 478991 0 2024-09-21T04:35:04.673 INFO:tasks.workunit.client.1.smithi053.stdout:5/292: dwrite d0/f3b [0,4194304] 0 2024-09-21T04:35:04.673 INFO:tasks.workunit.client.1.smithi053.stdout:6/253: getdents d2/d1b/d3d/d49/de/d32 0 2024-09-21T04:35:04.675 INFO:tasks.workunit.client.1.smithi053.stdout:2/243: readlink d1/d18/d1f/d27/l38 0 2024-09-21T04:35:04.675 INFO:tasks.workunit.client.1.smithi053.stdout:1/357: truncate d7/f56 1434282 0 2024-09-21T04:35:04.676 INFO:tasks.workunit.client.1.smithi053.stdout:8/269: mkdir d3/d5/d4c/d53/d66 0 2024-09-21T04:35:04.676 INFO:tasks.workunit.client.1.smithi053.stdout:4/255: truncate d1/dd/de/d14/d20/f45 648255 0 2024-09-21T04:35:04.677 INFO:tasks.workunit.client.1.smithi053.stdout:1/358: chown d7/f56 0 1 2024-09-21T04:35:04.677 INFO:tasks.workunit.client.1.smithi053.stdout:6/254: stat d2/d1b/d3d/d49/d1a/d2e/d38/l33 0 2024-09-21T04:35:04.679 INFO:tasks.workunit.client.1.smithi053.stdout:4/256: write d1/f4a [1031639,11479] 0 2024-09-21T04:35:04.682 INFO:tasks.workunit.client.1.smithi053.stdout:5/293: mkdir d0/dd/d5c 0 2024-09-21T04:35:04.683 INFO:tasks.workunit.client.1.smithi053.stdout:9/284: mkdir d0/d2/d4e/d5c 0 2024-09-21T04:35:04.685 INFO:tasks.workunit.client.1.smithi053.stdout:6/255: creat d2/d1d/f54 x:0 0 0 2024-09-21T04:35:04.687 INFO:tasks.workunit.client.1.smithi053.stdout:8/270: dwrite d3/df/d14/d16/f35 [0,4194304] 0 2024-09-21T04:35:04.689 INFO:tasks.workunit.client.1.smithi053.stdout:9/285: truncate d0/d2/d9/d1f/f4f 926211 0 2024-09-21T04:35:04.689 INFO:tasks.workunit.client.1.smithi053.stdout:8/271: truncate d3/f3a 547371 0 2024-09-21T04:35:04.691 INFO:tasks.workunit.client.1.smithi053.stdout:3/240: rename d16/d17/d21/d26/d2b/c48 to d16/d17/d21/d26/c55 0 2024-09-21T04:35:04.692 INFO:tasks.workunit.client.1.smithi053.stdout:8/272: dread - d3/df/d14/f60 zero size 2024-09-21T04:35:04.692 INFO:tasks.workunit.client.1.smithi053.stdout:1/359: mknod d7/d9/da/d4c/c73 0 2024-09-21T04:35:04.692 INFO:tasks.workunit.client.1.smithi053.stdout:9/286: chown d0/d1c/l2d 2527777 1 2024-09-21T04:35:04.696 INFO:tasks.workunit.client.1.smithi053.stdout:5/294: getdents d0/dd/d5c 0 2024-09-21T04:35:04.698 INFO:tasks.workunit.client.1.smithi053.stdout:2/244: rename d1/c11 to d1/d18/d1f/d22/c53 0 2024-09-21T04:35:04.699 INFO:tasks.workunit.client.1.smithi053.stdout:2/245: readlink d1/l23 0 2024-09-21T04:35:04.701 INFO:tasks.workunit.client.1.smithi053.stdout:6/256: dwrite d2/d1b/d1e/f4f [0,4194304] 0 2024-09-21T04:35:04.704 INFO:tasks.workunit.client.1.smithi053.stdout:8/273: creat d3/d5/d38/d64/f67 x:0 0 0 2024-09-21T04:35:04.705 INFO:tasks.workunit.client.1.smithi053.stdout:9/287: mknod d0/d2/d9/d34/d58/c5d 0 2024-09-21T04:35:04.706 INFO:tasks.workunit.client.1.smithi053.stdout:1/360: mkdir d7/d9/da/d40/d64/d74 0 2024-09-21T04:35:04.706 INFO:tasks.workunit.client.1.smithi053.stdout:4/257: dwrite d1/d2c/f40 [0,4194304] 0 2024-09-21T04:35:04.708 INFO:tasks.workunit.client.1.smithi053.stdout:4/258: chown d1/l7 25298 1 2024-09-21T04:35:04.708 INFO:tasks.workunit.client.1.smithi053.stdout:5/295: mknod d0/dd/d5c/c5d 0 2024-09-21T04:35:04.709 INFO:tasks.workunit.client.1.smithi053.stdout:2/246: mkdir d1/d18/d1f/d22/d54 0 2024-09-21T04:35:04.710 INFO:tasks.workunit.client.1.smithi053.stdout:8/274: write d3/df/d10/f48 [4204404,19640] 0 2024-09-21T04:35:04.712 INFO:tasks.workunit.client.1.smithi053.stdout:3/241: getdents d16 0 2024-09-21T04:35:04.714 INFO:tasks.workunit.client.1.smithi053.stdout:2/247: readlink d1/d18/d1f/d22/d2d/l2e 0 2024-09-21T04:35:04.714 INFO:tasks.workunit.client.1.smithi053.stdout:5/296: symlink d0/dd/d5c/l5e 0 2024-09-21T04:35:04.715 INFO:tasks.workunit.client.1.smithi053.stdout:4/259: write d1/fc [151162,127044] 0 2024-09-21T04:35:04.715 INFO:tasks.workunit.client.1.smithi053.stdout:5/297: stat d0/f3b 0 2024-09-21T04:35:04.716 INFO:tasks.workunit.client.1.smithi053.stdout:2/248: fsync d1/d18/d1f/d22/f32 0 2024-09-21T04:35:04.717 INFO:tasks.workunit.client.1.smithi053.stdout:8/275: creat d3/d28/f68 x:0 0 0 2024-09-21T04:35:04.717 INFO:tasks.workunit.client.1.smithi053.stdout:3/242: creat d16/f56 x:0 0 0 2024-09-21T04:35:04.717 INFO:tasks.workunit.client.1.smithi053.stdout:4/260: fdatasync d1/f4a 0 2024-09-21T04:35:04.718 INFO:tasks.workunit.client.1.smithi053.stdout:9/288: creat d0/d1c/d52/d53/f5e x:0 0 0 2024-09-21T04:35:04.720 INFO:tasks.workunit.client.1.smithi053.stdout:3/243: read fd [1498964,70275] 0 2024-09-21T04:35:04.720 INFO:tasks.workunit.client.1.smithi053.stdout:1/361: mkdir d7/d9/da/d40/d64/d74/d75 0 2024-09-21T04:35:04.720 INFO:tasks.workunit.client.1.smithi053.stdout:5/298: write d0/dd/f44 [1498739,2898] 0 2024-09-21T04:35:04.726 INFO:tasks.workunit.client.1.smithi053.stdout:4/261: readlink d1/dd/de/l38 0 2024-09-21T04:35:04.728 INFO:tasks.workunit.client.1.smithi053.stdout:9/289: creat d0/d2/d4e/f5f x:0 0 0 2024-09-21T04:35:04.729 INFO:tasks.workunit.client.1.smithi053.stdout:8/276: dwrite d3/df/d49/f55 [0,4194304] 0 2024-09-21T04:35:04.729 INFO:tasks.workunit.client.1.smithi053.stdout:5/299: creat d0/d8/db/f5f x:0 0 0 2024-09-21T04:35:04.731 INFO:tasks.workunit.client.1.smithi053.stdout:1/362: mkdir d7/d63/d76 0 2024-09-21T04:35:04.733 INFO:tasks.workunit.client.1.smithi053.stdout:4/262: write d1/d2c/f40 [5686311,88389] 0 2024-09-21T04:35:04.736 INFO:tasks.workunit.client.1.smithi053.stdout:6/257: dwrite d2/d1b/d3d/d49/de/f14 [0,4194304] 0 2024-09-21T04:35:04.736 INFO:tasks.workunit.client.1.smithi053.stdout:2/249: fdatasync d1/d18/d1f/f4a 0 2024-09-21T04:35:04.741 INFO:tasks.workunit.client.1.smithi053.stdout:3/244: chown d16/c19 13 1 2024-09-21T04:35:04.747 INFO:tasks.workunit.client.1.smithi053.stdout:5/300: creat d0/dd/d48/f60 x:0 0 0 2024-09-21T04:35:04.747 INFO:tasks.workunit.client.1.smithi053.stdout:4/263: symlink d1/dd/de/l4f 0 2024-09-21T04:35:04.749 INFO:tasks.workunit.client.1.smithi053.stdout:1/363: mkdir d7/d9/da/d40/d49/d77 0 2024-09-21T04:35:04.751 INFO:tasks.workunit.client.1.smithi053.stdout:4/264: fsync d1/dd/de/f2a 0 2024-09-21T04:35:04.753 INFO:tasks.workunit.client.1.smithi053.stdout:8/277: dwrite d3/d39/f40 [0,4194304] 0 2024-09-21T04:35:04.753 INFO:tasks.workunit.client.1.smithi053.stdout:6/258: creat d2/d1b/d3d/d49/d1a/d2e/f55 x:0 0 0 2024-09-21T04:35:04.753 INFO:tasks.workunit.client.1.smithi053.stdout:5/301: mkdir d0/dd/d30/d35/d61 0 2024-09-21T04:35:04.754 INFO:tasks.workunit.client.1.smithi053.stdout:1/364: write d7/d9/f19 [15171,3724] 0 2024-09-21T04:35:04.755 INFO:tasks.workunit.client.1.smithi053.stdout:3/245: mkdir d16/d46/d57 0 2024-09-21T04:35:04.758 INFO:tasks.workunit.client.1.smithi053.stdout:1/365: dread - d7/d9/da/d11/f5c zero size 2024-09-21T04:35:04.758 INFO:tasks.workunit.client.1.smithi053.stdout:8/278: dread d3/df/f62 [0,4194304] 0 2024-09-21T04:35:04.758 INFO:tasks.workunit.client.1.smithi053.stdout:4/265: creat d1/dd/de/d14/d3f/f50 x:0 0 0 2024-09-21T04:35:04.760 INFO:tasks.workunit.client.1.smithi053.stdout:2/250: dwrite d1/d18/d1f/d22/f36 [0,4194304] 0 2024-09-21T04:35:04.761 INFO:tasks.workunit.client.1.smithi053.stdout:4/266: fsync d1/d27/d33/f3c 0 2024-09-21T04:35:04.761 INFO:tasks.workunit.client.1.smithi053.stdout:9/290: dread d0/d2/fe [0,4194304] 0 2024-09-21T04:35:04.761 INFO:tasks.workunit.client.1.smithi053.stdout:6/259: mknod d2/d1b/d1e/d27/c56 0 2024-09-21T04:35:04.767 INFO:tasks.workunit.client.1.smithi053.stdout:5/302: creat d0/dd/f62 x:0 0 0 2024-09-21T04:35:04.769 INFO:tasks.workunit.client.1.smithi053.stdout:5/303: truncate d0/dd/f44 2466200 0 2024-09-21T04:35:04.770 INFO:tasks.workunit.client.1.smithi053.stdout:3/246: dwrite d16/d1b/f24 [0,4194304] 0 2024-09-21T04:35:04.771 INFO:tasks.workunit.client.1.smithi053.stdout:8/279: fdatasync d3/d39/f27 0 2024-09-21T04:35:04.773 INFO:tasks.workunit.client.1.smithi053.stdout:6/260: creat d2/d1b/d3d/d49/d1a/f57 x:0 0 0 2024-09-21T04:35:04.775 INFO:tasks.workunit.client.1.smithi053.stdout:1/366: symlink d7/d9/da/d11/l78 0 2024-09-21T04:35:04.776 INFO:tasks.workunit.client.1.smithi053.stdout:3/247: dread f11 [0,4194304] 0 2024-09-21T04:35:04.776 INFO:tasks.workunit.client.1.smithi053.stdout:8/280: write d3/df/d49/f55 [3273367,26457] 0 2024-09-21T04:35:04.777 INFO:tasks.workunit.client.1.smithi053.stdout:9/291: mknod d0/d2/d9/d34/d58/c60 0 2024-09-21T04:35:04.779 INFO:tasks.workunit.client.1.smithi053.stdout:4/267: mkdir d1/dd/de/d14/d20/d4e/d51 0 2024-09-21T04:35:04.779 INFO:tasks.workunit.client.1.smithi053.stdout:5/304: dread d0/dd/f3a [0,4194304] 0 2024-09-21T04:35:04.780 INFO:tasks.workunit.client.1.smithi053.stdout:4/268: dread - d1/dd/de/d14/d3f/f50 zero size 2024-09-21T04:35:04.781 INFO:tasks.workunit.client.1.smithi053.stdout:4/269: dread - d1/dd/de/d14/d3f/f50 zero size 2024-09-21T04:35:04.784 INFO:tasks.workunit.client.1.smithi053.stdout:1/367: mkdir d7/d47/d79 0 2024-09-21T04:35:04.788 INFO:tasks.workunit.client.1.smithi053.stdout:9/292: rmdir d0/d2/d9/d34 39 2024-09-21T04:35:04.791 INFO:tasks.workunit.client.1.smithi053.stdout:4/270: truncate d1/f2d 1161928 0 2024-09-21T04:35:04.793 INFO:tasks.workunit.client.1.smithi053.stdout:9/293: dwrite d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:04.795 INFO:tasks.workunit.client.1.smithi053.stdout:9/294: fdatasync d0/d2/d9/fb 0 2024-09-21T04:35:04.795 INFO:tasks.workunit.client.1.smithi053.stdout:1/368: rename d7/d5a to d7/d9/da/d11/d2b/d7a 0 2024-09-21T04:35:04.797 INFO:tasks.workunit.client.1.smithi053.stdout:2/251: link d1/d18/d1f/d22/f45 d1/d18/d1f/d27/f55 0 2024-09-21T04:35:04.799 INFO:tasks.workunit.client.1.smithi053.stdout:8/281: symlink d3/d5/d4c/d53/d66/l69 0 2024-09-21T04:35:04.801 INFO:tasks.workunit.client.1.smithi053.stdout:9/295: truncate d0/d2/d9/f43 1185589 0 2024-09-21T04:35:04.801 INFO:tasks.workunit.client.1.smithi053.stdout:3/248: dwrite d16/d17/f2a [0,4194304] 0 2024-09-21T04:35:04.803 INFO:tasks.workunit.client.1.smithi053.stdout:1/369: unlink d7/c22 0 2024-09-21T04:35:04.805 INFO:tasks.workunit.client.1.smithi053.stdout:6/261: rename d2/d1b/d1e/d2a/f3c to d2/d1b/d1e/d2a/d4b/f58 0 2024-09-21T04:35:04.807 INFO:tasks.workunit.client.1.smithi053.stdout:4/271: mknod d1/c52 0 2024-09-21T04:35:04.809 INFO:tasks.workunit.client.1.smithi053.stdout:2/252: rename d1/d18/d1f/d22/d2d/l2e to d1/d18/d1f/d22/d2d/l56 0 2024-09-21T04:35:04.809 INFO:tasks.workunit.client.1.smithi053.stdout:8/282: write d3/d5/d38/f4e [764055,27913] 0 2024-09-21T04:35:04.812 INFO:tasks.workunit.client.1.smithi053.stdout:8/283: dread - d3/d28/f31 zero size 2024-09-21T04:35:04.813 INFO:tasks.workunit.client.1.smithi053.stdout:4/272: mkdir d1/d27/d53 0 2024-09-21T04:35:04.813 INFO:tasks.workunit.client.1.smithi053.stdout:9/296: dwrite d0/d2/d9/f43 [0,4194304] 0 2024-09-21T04:35:04.815 INFO:tasks.workunit.client.1.smithi053.stdout:3/249: dwrite d16/d1b/f2f [0,4194304] 0 2024-09-21T04:35:04.815 INFO:tasks.workunit.client.1.smithi053.stdout:2/253: write d1/d18/d1f/d27/d41/f42 [1516274,49433] 0 2024-09-21T04:35:04.819 INFO:tasks.workunit.client.1.smithi053.stdout:8/284: chown d3/d5/d1a/l20 14085 1 2024-09-21T04:35:04.821 INFO:tasks.workunit.client.1.smithi053.stdout:9/297: mknod d0/d28/c61 0 2024-09-21T04:35:04.822 INFO:tasks.workunit.client.0.smithi007.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-21T04:35:04.824 INFO:tasks.workunit.client.1.smithi053.stdout:9/298: readlink d0/d1c/l20 0 2024-09-21T04:35:04.824 INFO:tasks.workunit.client.1.smithi053.stdout:3/250: getdents d16/d17/d21/d26/d2b 0 2024-09-21T04:35:04.826 INFO:tasks.workunit.client.1.smithi053.stdout:6/262: dwrite d2/d1b/d3d/d49/f19 [4194304,4194304] 0 2024-09-21T04:35:04.827 INFO:tasks.workunit.client.1.smithi053.stdout:8/285: truncate d3/d5/d38/d64/f67 854489 0 2024-09-21T04:35:04.827 INFO:tasks.workunit.client.1.smithi053.stdout:3/251: rename d16 to d16/d17/d21/d4e/d58 22 2024-09-21T04:35:04.828 INFO:tasks.workunit.client.0.smithi007.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-21T04:35:04.828 INFO:tasks.workunit.client.0.smithi007.stderr:+ make 2024-09-21T04:35:04.828 INFO:tasks.workunit.client.1.smithi053.stdout:4/273: dwrite d1/f4a [0,4194304] 0 2024-09-21T04:35:04.829 INFO:tasks.workunit.client.1.smithi053.stdout:3/252: stat fd 0 2024-09-21T04:35:04.831 INFO:tasks.workunit.client.1.smithi053.stdout:8/286: mknod d3/d5/d38/c6a 0 2024-09-21T04:35:04.831 INFO:tasks.workunit.client.1.smithi053.stdout:2/254: dread d1/d18/d1f/d27/f55 [0,4194304] 0 2024-09-21T04:35:04.833 INFO:tasks.workunit.client.1.smithi053.stdout:2/255: readlink d1/d18/l19 0 2024-09-21T04:35:04.834 INFO:tasks.workunit.client.1.smithi053.stdout:2/256: write d1/d18/d1f/d22/f36 [2134006,60277] 0 2024-09-21T04:35:04.835 INFO:tasks.workunit.client.1.smithi053.stdout:8/287: creat d3/d39/f6b x:0 0 0 2024-09-21T04:35:04.835 INFO:tasks.workunit.client.1.smithi053.stdout:4/274: creat d1/d27/d2e/d46/f54 x:0 0 0 2024-09-21T04:35:04.836 INFO:tasks.workunit.client.1.smithi053.stdout:8/288: write d3/df/d14/d16/f29 [1416458,89342] 0 2024-09-21T04:35:04.839 INFO:tasks.workunit.client.1.smithi053.stdout:8/289: write d3/d28/f2e [4241323,72870] 0 2024-09-21T04:35:04.841 INFO:tasks.workunit.client.1.smithi053.stdout:8/290: write d3/df/d49/f55 [429206,44147] 0 2024-09-21T04:35:04.842 INFO:tasks.workunit.client.1.smithi053.stdout:2/257: mknod d1/d18/d1f/d27/d41/c57 0 2024-09-21T04:35:04.844 INFO:tasks.workunit.client.1.smithi053.stdout:8/291: unlink d3/d5/c2b 0 2024-09-21T04:35:04.846 INFO:tasks.workunit.client.1.smithi053.stdout:0/198: sync 2024-09-21T04:35:04.846 INFO:tasks.workunit.client.1.smithi053.stdout:7/207: sync 2024-09-21T04:35:04.849 INFO:tasks.workunit.client.1.smithi053.stdout:2/258: getdents d1/d18 0 2024-09-21T04:35:04.850 INFO:tasks.workunit.client.1.smithi053.stdout:7/208: creat d0/d6/f42 x:0 0 0 2024-09-21T04:35:04.850 INFO:tasks.workunit.client.1.smithi053.stdout:0/199: stat d5/d6/d21/l9 0 2024-09-21T04:35:04.856 INFO:tasks.workunit.client.1.smithi053.stdout:8/292: dwrite d3/f4a [0,4194304] 0 2024-09-21T04:35:04.856 INFO:tasks.workunit.client.1.smithi053.stdout:8/293: read d3/d39/f3f [2320376,115175] 0 2024-09-21T04:35:04.862 INFO:tasks.workunit.client.1.smithi053.stdout:8/294: symlink d3/d5/d38/l6c 0 2024-09-21T04:35:04.864 INFO:tasks.workunit.client.1.smithi053.stdout:0/200: rename d5/de/f2f to d5/de/d1c/f41 0 2024-09-21T04:35:04.865 INFO:tasks.workunit.client.1.smithi053.stdout:0/201: chown d5/dd/c3c 0 1 2024-09-21T04:35:04.866 INFO:tasks.workunit.client.1.smithi053.stdout:2/259: dread d1/d18/f2a [0,4194304] 0 2024-09-21T04:35:04.867 INFO:tasks.workunit.client.1.smithi053.stdout:0/202: mkdir d5/de/d42 0 2024-09-21T04:35:04.869 INFO:tasks.workunit.client.1.smithi053.stdout:2/260: chown d1/d18/d1f/f24 60562 1 2024-09-21T04:35:04.871 INFO:tasks.workunit.client.1.smithi053.stdout:2/261: mkdir d1/d18/d1f/d27/d58 0 2024-09-21T04:35:04.872 INFO:tasks.workunit.client.1.smithi053.stdout:2/262: stat d1/d18/d1f/d27/l38 0 2024-09-21T04:35:04.872 INFO:tasks.workunit.client.1.smithi053.stdout:2/263: chown d1/d18/d1f/d27/d58 1297 1 2024-09-21T04:35:04.873 INFO:tasks.workunit.client.1.smithi053.stdout:7/209: read d0/d6/f8 [1442818,17402] 0 2024-09-21T04:35:04.874 INFO:tasks.workunit.client.1.smithi053.stdout:0/203: link d5/d6/l24 d5/de/d1c/l43 0 2024-09-21T04:35:04.874 INFO:tasks.workunit.client.1.smithi053.stdout:2/264: fdatasync d1/d18/d1f/d22/f36 0 2024-09-21T04:35:04.874 INFO:tasks.workunit.client.1.smithi053.stdout:7/210: chown d0/d2b/f32 128 1 2024-09-21T04:35:04.875 INFO:tasks.workunit.client.1.smithi053.stdout:0/204: chown d5/d6/f16 0 1 2024-09-21T04:35:04.876 INFO:tasks.workunit.client.1.smithi053.stdout:0/205: fsync d5/d6/f36 0 2024-09-21T04:35:04.876 INFO:tasks.workunit.client.1.smithi053.stdout:2/265: write d1/d18/d1f/d22/f32 [1020919,65388] 0 2024-09-21T04:35:04.878 INFO:tasks.workunit.client.1.smithi053.stdout:7/211: rename d0/d6/f1e to d0/d2b/f43 0 2024-09-21T04:35:04.879 INFO:tasks.workunit.client.1.smithi053.stdout:0/206: mkdir d5/d10/d44 0 2024-09-21T04:35:04.879 INFO:tasks.workunit.client.1.smithi053.stdout:0/207: chown d5/de/f2a 5120889 1 2024-09-21T04:35:04.884 INFO:tasks.workunit.client.1.smithi053.stdout:1/370: truncate d7/d9/da/d11/f26 6852235 0 2024-09-21T04:35:04.887 INFO:tasks.workunit.client.1.smithi053.stdout:6/263: write d2/d1b/d3d/d49/f42 [3637678,26954] 0 2024-09-21T04:35:04.887 INFO:tasks.workunit.client.1.smithi053.stdout:3/253: truncate f14 3038351 0 2024-09-21T04:35:04.888 INFO:tasks.workunit.client.1.smithi053.stdout:7/212: dwrite d0/d6/f1c [0,4194304] 0 2024-09-21T04:35:04.890 INFO:tasks.workunit.client.1.smithi053.stdout:0/208: dwrite d5/de/d1c/f29 [0,4194304] 0 2024-09-21T04:35:04.893 INFO:tasks.workunit.client.1.smithi053.stdout:0/209: fdatasync d5/f15 0 2024-09-21T04:35:04.893 INFO:tasks.workunit.client.1.smithi053.stdout:3/254: write d16/d17/d3c/f52 [41676,122003] 0 2024-09-21T04:35:04.895 INFO:tasks.workunit.client.1.smithi053.stdout:1/371: creat d7/d9/da/d11/d2b/d7a/d59/f7b x:0 0 0 2024-09-21T04:35:04.898 INFO:tasks.workunit.client.1.smithi053.stdout:6/264: rename d2/d1b/d3d/d49/f20 to d2/f59 0 2024-09-21T04:35:04.900 INFO:tasks.workunit.client.1.smithi053.stdout:1/372: chown d7/d9/l21 2399789 1 2024-09-21T04:35:04.901 INFO:tasks.workunit.client.1.smithi053.stdout:9/299: dwrite d0/f24 [0,4194304] 0 2024-09-21T04:35:04.901 INFO:tasks.workunit.client.1.smithi053.stdout:0/210: mkdir d5/dd/d45 0 2024-09-21T04:35:04.904 INFO:tasks.workunit.client.0.smithi007.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-21T04:35:04.906 INFO:tasks.workunit.client.1.smithi053.stdout:2/266: truncate d1/d18/d1f/d27/d41/f4c 2310266 0 2024-09-21T04:35:04.909 INFO:tasks.workunit.client.1.smithi053.stdout:0/211: mkdir d5/d10/d46 0 2024-09-21T04:35:04.909 INFO:tasks.workunit.client.1.smithi053.stdout:6/265: creat d2/d1b/f5a x:0 0 0 2024-09-21T04:35:04.910 INFO:tasks.workunit.client.1.smithi053.stdout:7/213: dwrite d0/f20 [0,4194304] 0 2024-09-21T04:35:04.916 INFO:tasks.workunit.client.1.smithi053.stdout:7/214: stat d0/d2b/l30 0 2024-09-21T04:35:04.916 INFO:tasks.workunit.client.1.smithi053.stdout:1/373: symlink d7/d9/da/d40/d49/d77/l7c 0 2024-09-21T04:35:04.916 INFO:tasks.workunit.client.1.smithi053.stdout:0/212: write d5/fa [946106,63097] 0 2024-09-21T04:35:04.918 INFO:tasks.workunit.client.1.smithi053.stdout:4/275: dwrite d1/f2d [0,4194304] 0 2024-09-21T04:35:04.919 INFO:tasks.workunit.client.1.smithi053.stdout:2/267: creat d1/d18/d1f/d22/d43/f59 x:0 0 0 2024-09-21T04:35:04.920 INFO:tasks.workunit.client.1.smithi053.stdout:0/213: truncate d5/d6/f36 1037461 0 2024-09-21T04:35:04.924 INFO:tasks.workunit.client.1.smithi053.stdout:6/266: fdatasync d2/d1b/f3a 0 2024-09-21T04:35:04.926 INFO:tasks.workunit.client.1.smithi053.stdout:1/374: truncate d7/d9/da/d11/d2b/d7a/d59/f7b 528804 0 2024-09-21T04:35:04.930 INFO:tasks.workunit.client.1.smithi053.stdout:0/214: stat d5/de/d1c/l43 0 2024-09-21T04:35:04.933 INFO:tasks.workunit.client.1.smithi053.stdout:9/300: dwrite d0/d1a/f1b [0,4194304] 0 2024-09-21T04:35:04.933 INFO:tasks.workunit.client.1.smithi053.stdout:7/215: creat d0/f44 x:0 0 0 2024-09-21T04:35:04.936 INFO:tasks.workunit.client.1.smithi053.stdout:0/215: unlink d5/d6/d21/l9 0 2024-09-21T04:35:04.939 INFO:tasks.workunit.client.1.smithi053.stdout:4/276: getdents d1/dd/de/d14/d20 0 2024-09-21T04:35:04.940 INFO:tasks.workunit.client.1.smithi053.stdout:6/267: dwrite d2/d1b/f5a [0,4194304] 0 2024-09-21T04:35:04.941 INFO:tasks.workunit.client.1.smithi053.stdout:0/216: write d5/f15 [4730520,38912] 0 2024-09-21T04:35:04.941 INFO:tasks.workunit.client.1.smithi053.stdout:2/268: getdents d1/d18/d1f/d22 0 2024-09-21T04:35:04.942 INFO:tasks.workunit.client.1.smithi053.stdout:7/216: readlink d0/d2b/d36/l40 0 2024-09-21T04:35:04.942 INFO:tasks.workunit.client.1.smithi053.stdout:3/255: dwrite d16/d1b/f1f [0,4194304] 0 2024-09-21T04:35:04.946 INFO:tasks.workunit.client.1.smithi053.stdout:7/217: fsync d0/f20 0 2024-09-21T04:35:04.947 INFO:tasks.workunit.client.1.smithi053.stdout:4/277: write d1/d27/f31 [957147,32692] 0 2024-09-21T04:35:04.949 INFO:tasks.workunit.client.1.smithi053.stdout:6/268: write d2/d1b/d3d/d49/d1a/f57 [169322,48824] 0 2024-09-21T04:35:04.949 INFO:tasks.workunit.client.1.smithi053.stdout:0/217: write d5/fa [1149245,23673] 0 2024-09-21T04:35:04.950 INFO:tasks.workunit.client.1.smithi053.stdout:7/218: chown d0/d24 48446 1 2024-09-21T04:35:04.950 INFO:tasks.workunit.client.1.smithi053.stdout:2/269: dread - d1/d18/d1f/d22/d43/f59 zero size 2024-09-21T04:35:04.951 INFO:tasks.workunit.client.1.smithi053.stdout:3/256: chown d16/d17/d21/d26/d2b/l3f 796941086 1 2024-09-21T04:35:04.952 INFO:tasks.workunit.client.1.smithi053.stdout:0/218: stat d5/f39 0 2024-09-21T04:35:04.953 INFO:tasks.workunit.client.1.smithi053.stdout:7/219: stat d0/c21 0 2024-09-21T04:35:04.955 INFO:tasks.workunit.client.1.smithi053.stdout:4/278: creat d1/d27/f55 x:0 0 0 2024-09-21T04:35:04.956 INFO:tasks.workunit.client.1.smithi053.stdout:3/257: creat d16/d1b/f59 x:0 0 0 2024-09-21T04:35:04.957 INFO:tasks.workunit.client.1.smithi053.stdout:7/220: chown d0/d6/c2f 55 1 2024-09-21T04:35:04.958 INFO:tasks.workunit.client.1.smithi053.stdout:9/301: dwrite d0/d2/d9/d34/f56 [0,4194304] 0 2024-09-21T04:35:04.958 INFO:tasks.workunit.client.1.smithi053.stdout:2/270: creat d1/d18/d1f/d22/f5a x:0 0 0 2024-09-21T04:35:04.959 INFO:tasks.workunit.client.1.smithi053.stdout:6/269: mkdir d2/d1b/d3d/d49/d1a/d2e/d5b 0 2024-09-21T04:35:04.960 INFO:tasks.workunit.client.1.smithi053.stdout:0/219: mkdir d5/d47 0 2024-09-21T04:35:04.963 INFO:tasks.workunit.client.1.smithi053.stdout:9/302: creat d0/d28/f62 x:0 0 0 2024-09-21T04:35:04.963 INFO:tasks.workunit.client.1.smithi053.stdout:9/303: chown d0/d1c/d21 1132260700 1 2024-09-21T04:35:04.964 INFO:tasks.workunit.client.1.smithi053.stdout:9/304: write d0/d2/fe [3582851,108186] 0 2024-09-21T04:35:04.965 INFO:tasks.workunit.client.1.smithi053.stdout:3/258: creat d16/d17/d21/d26/d3e/f5a x:0 0 0 2024-09-21T04:35:04.966 INFO:tasks.workunit.client.1.smithi053.stdout:3/259: write d16/d1b/f59 [349452,128779] 0 2024-09-21T04:35:04.966 INFO:tasks.workunit.client.1.smithi053.stdout:6/270: mkdir d2/d1b/d3d/d49/d5c 0 2024-09-21T04:35:04.967 INFO:tasks.workunit.client.1.smithi053.stdout:0/220: creat d5/de/d42/f48 x:0 0 0 2024-09-21T04:35:04.968 INFO:tasks.workunit.client.1.smithi053.stdout:0/221: readlink d5/l23 0 2024-09-21T04:35:04.970 INFO:tasks.workunit.client.1.smithi053.stdout:0/222: read - d5/d6/d21/f33 zero size 2024-09-21T04:35:04.970 INFO:tasks.workunit.client.1.smithi053.stdout:4/279: rename d1/l8 to d1/dd/l56 0 2024-09-21T04:35:04.971 INFO:tasks.workunit.client.1.smithi053.stdout:7/221: getdents d0/d24 0 2024-09-21T04:35:04.975 INFO:tasks.workunit.client.1.smithi053.stdout:4/280: mknod d1/d27/d33/d41/c57 0 2024-09-21T04:35:04.976 INFO:tasks.workunit.client.1.smithi053.stdout:0/223: symlink d5/d10/d44/l49 0 2024-09-21T04:35:04.976 INFO:tasks.workunit.client.1.smithi053.stdout:9/305: dread d0/f4 [0,4194304] 0 2024-09-21T04:35:04.976 INFO:tasks.workunit.client.1.smithi053.stdout:6/271: dwrite d2/d1b/f3a [0,4194304] 0 2024-09-21T04:35:04.978 INFO:tasks.workunit.client.1.smithi053.stdout:6/272: chown d2/d1b/d1e 41744767 1 2024-09-21T04:35:04.984 INFO:tasks.workunit.client.1.smithi053.stdout:7/222: write d0/f23 [2061210,85048] 0 2024-09-21T04:35:04.986 INFO:tasks.workunit.client.1.smithi053.stdout:6/273: creat d2/d1b/d3d/d49/d5c/f5d x:0 0 0 2024-09-21T04:35:04.990 INFO:tasks.workunit.client.1.smithi053.stdout:7/223: rmdir d0 39 2024-09-21T04:35:04.992 INFO:tasks.workunit.client.1.smithi053.stdout:4/281: dwrite d1/d27/f55 [0,4194304] 0 2024-09-21T04:35:04.994 INFO:tasks.workunit.client.1.smithi053.stdout:0/224: dwrite d5/de/d1c/f3d [0,4194304] 0 2024-09-21T04:35:04.994 INFO:tasks.workunit.client.1.smithi053.stdout:9/306: dwrite d0/d2/fe [0,4194304] 0 2024-09-21T04:35:04.996 INFO:tasks.workunit.client.1.smithi053.stdout:9/307: fsync d0/d1c/f39 0 2024-09-21T04:35:04.999 INFO:tasks.workunit.client.1.smithi053.stdout:4/282: unlink d1/l22 0 2024-09-21T04:35:05.001 INFO:tasks.workunit.client.1.smithi053.stdout:9/308: mkdir d0/d28/d63 0 2024-09-21T04:35:05.002 INFO:tasks.workunit.client.1.smithi053.stdout:0/225: creat d5/f4a x:0 0 0 2024-09-21T04:35:05.003 INFO:tasks.workunit.client.1.smithi053.stdout:9/309: dread - d0/d1c/f44 zero size 2024-09-21T04:35:05.003 INFO:tasks.workunit.client.1.smithi053.stdout:6/274: dwrite d2/fb [0,4194304] 0 2024-09-21T04:35:05.005 INFO:tasks.workunit.client.1.smithi053.stdout:4/283: mknod d1/dd/de/d14/c58 0 2024-09-21T04:35:05.005 INFO:tasks.workunit.client.1.smithi053.stdout:9/310: chown d0/d28/d63 41664 1 2024-09-21T04:35:05.006 INFO:tasks.workunit.client.1.smithi053.stdout:0/226: write d5/d6/f1b [642654,102867] 0 2024-09-21T04:35:05.008 INFO:tasks.workunit.client.1.smithi053.stdout:7/224: dwrite d0/d2b/f2e [0,4194304] 0 2024-09-21T04:35:05.011 INFO:tasks.workunit.client.1.smithi053.stdout:7/225: dread - d0/d6/f31 zero size 2024-09-21T04:35:05.011 INFO:tasks.workunit.client.1.smithi053.stdout:0/227: rmdir d5/de/d1c 39 2024-09-21T04:35:05.015 INFO:tasks.workunit.client.1.smithi053.stdout:6/275: rename d2/d1b/d1e/d27/f51 to d2/d1b/d3d/f5e 0 2024-09-21T04:35:05.021 INFO:tasks.workunit.client.1.smithi053.stdout:7/226: dwrite d0/d6/f9 [4194304,4194304] 0 2024-09-21T04:35:05.021 INFO:tasks.workunit.client.1.smithi053.stdout:4/284: rename d1/d27/d2e/l3e to d1/d27/d53/l59 0 2024-09-21T04:35:05.022 INFO:tasks.workunit.client.1.smithi053.stdout:9/311: getdents d0/d1a 0 2024-09-21T04:35:05.023 INFO:tasks.workunit.client.1.smithi053.stdout:4/285: chown d1/dd/de/c1e 5442 1 2024-09-21T04:35:05.023 INFO:tasks.workunit.client.1.smithi053.stdout:6/276: mkdir d2/d1b/d3d/d49/d1a/d2e/d5f 0 2024-09-21T04:35:05.025 INFO:tasks.workunit.client.1.smithi053.stdout:0/228: getdents d5/de/d42 0 2024-09-21T04:35:05.026 INFO:tasks.workunit.client.1.smithi053.stdout:9/312: symlink d0/d2/d9/d34/d58/l64 0 2024-09-21T04:35:05.030 INFO:tasks.workunit.client.1.smithi053.stdout:6/277: mknod d2/d1b/d3d/d49/d1a/d2e/d5b/c60 0 2024-09-21T04:35:05.031 INFO:tasks.workunit.client.1.smithi053.stdout:0/229: rename d5/c3a to d5/d10/d46/c4b 0 2024-09-21T04:35:05.032 INFO:tasks.workunit.client.1.smithi053.stdout:4/286: getdents d1/d27/d33/d41 0 2024-09-21T04:35:05.033 INFO:tasks.workunit.client.1.smithi053.stdout:7/227: write d0/f4 [3847257,108535] 0 2024-09-21T04:35:05.035 INFO:tasks.workunit.client.1.smithi053.stdout:9/313: rename d0/df/f23 to d0/d2/d9/d34/f65 0 2024-09-21T04:35:05.035 INFO:tasks.workunit.client.1.smithi053.stdout:0/230: mkdir d5/d10/d4c 0 2024-09-21T04:35:05.036 INFO:tasks.workunit.client.1.smithi053.stdout:4/287: unlink d1/f4a 0 2024-09-21T04:35:05.037 INFO:tasks.workunit.client.1.smithi053.stdout:4/288: chown d1/dd/de/d14/c58 3 1 2024-09-21T04:35:05.038 INFO:tasks.workunit.client.1.smithi053.stdout:6/278: write d2/d1b/d1e/d2a/f34 [1509476,97581] 0 2024-09-21T04:35:05.038 INFO:tasks.workunit.client.1.smithi053.stdout:4/289: write d1/d27/f37 [480180,21050] 0 2024-09-21T04:35:05.040 INFO:tasks.workunit.client.1.smithi053.stdout:7/228: unlink d0/f3a 0 2024-09-21T04:35:05.041 INFO:tasks.workunit.client.1.smithi053.stdout:7/229: write d0/d6/f1b [3635354,15516] 0 2024-09-21T04:35:05.041 INFO:tasks.workunit.client.1.smithi053.stdout:9/314: dread - d0/f4b zero size 2024-09-21T04:35:05.044 INFO:tasks.workunit.client.1.smithi053.stdout:4/290: rmdir d1/d27/d2e 39 2024-09-21T04:35:05.047 INFO:tasks.workunit.client.1.smithi053.stdout:9/315: creat d0/d1c/d21/d46/f66 x:0 0 0 2024-09-21T04:35:05.048 INFO:tasks.workunit.client.1.smithi053.stdout:4/291: mknod d1/dd/de/c5a 0 2024-09-21T04:35:05.056 INFO:tasks.workunit.client.1.smithi053.stdout:9/316: symlink d0/d2/d4e/d5c/l67 0 2024-09-21T04:35:05.057 INFO:tasks.workunit.client.1.smithi053.stdout:4/292: getdents d1/dd/de/d14/d20/d4e/d51 0 2024-09-21T04:35:05.060 INFO:tasks.workunit.client.1.smithi053.stdout:7/230: getdents d0/d6 0 2024-09-21T04:35:05.064 INFO:tasks.workunit.client.1.smithi053.stdout:7/231: creat d0/d2b/f45 x:0 0 0 2024-09-21T04:35:05.064 INFO:tasks.workunit.client.1.smithi053.stdout:4/293: dwrite d1/f2d [0,4194304] 0 2024-09-21T04:35:05.067 INFO:tasks.workunit.client.1.smithi053.stdout:6/279: truncate d2/d1b/f5a 3601784 0 2024-09-21T04:35:05.068 INFO:tasks.workunit.client.1.smithi053.stdout:9/317: write d0/d2/d9/fc [1194839,73671] 0 2024-09-21T04:35:05.070 INFO:tasks.workunit.client.1.smithi053.stdout:6/280: chown d2/d1b/f2f 214109914 1 2024-09-21T04:35:05.071 INFO:tasks.workunit.client.1.smithi053.stdout:7/232: getdents d0/d2b/d36 0 2024-09-21T04:35:05.073 INFO:tasks.workunit.client.1.smithi053.stdout:7/233: mkdir d0/d24/d46 0 2024-09-21T04:35:05.074 INFO:tasks.workunit.client.1.smithi053.stdout:4/294: dread d1/dd/de/d14/d20/f34 [4194304,4194304] 0 2024-09-21T04:35:05.075 INFO:tasks.workunit.client.1.smithi053.stdout:7/234: truncate d0/d6/f18 4850206 0 2024-09-21T04:35:05.075 INFO:tasks.workunit.client.1.smithi053.stdout:4/295: write d1/f42 [1452423,35788] 0 2024-09-21T04:35:05.076 INFO:tasks.workunit.client.1.smithi053.stdout:4/296: write d1/f4 [462436,36116] 0 2024-09-21T04:35:05.077 INFO:tasks.workunit.client.1.smithi053.stdout:6/281: link d2/d1b/d3d/d49/d1a/d2e/f35 d2/d1b/d3d/d49/d1a/f61 0 2024-09-21T04:35:05.079 INFO:tasks.workunit.client.1.smithi053.stdout:4/297: write d1/d27/f55 [1107957,45878] 0 2024-09-21T04:35:05.080 INFO:tasks.workunit.client.1.smithi053.stdout:6/282: write d2/fa [4004224,83458] 0 2024-09-21T04:35:05.081 INFO:tasks.workunit.client.1.smithi053.stdout:9/318: truncate d0/f4 638381 0 2024-09-21T04:35:05.083 INFO:tasks.workunit.client.1.smithi053.stdout:7/235: dread d0/f20 [0,4194304] 0 2024-09-21T04:35:05.083 INFO:tasks.workunit.client.1.smithi053.stdout:4/298: write d1/d27/d33/f3c [538570,13513] 0 2024-09-21T04:35:05.084 INFO:tasks.workunit.client.1.smithi053.stdout:9/319: write d0/d2/d9/d34/f59 [887233,92781] 0 2024-09-21T04:35:05.085 INFO:tasks.workunit.client.1.smithi053.stdout:7/236: readlink d0/le 0 2024-09-21T04:35:05.085 INFO:tasks.workunit.client.1.smithi053.stdout:6/283: chown d2/d1b/d3d/d49/de/c2d 380 1 2024-09-21T04:35:05.086 INFO:tasks.workunit.client.1.smithi053.stdout:7/237: chown d0/d6/f8 387863 1 2024-09-21T04:35:05.087 INFO:tasks.workunit.client.1.smithi053.stdout:4/299: mkdir d1/dd/de/d14/d20/d5b 0 2024-09-21T04:35:05.088 INFO:tasks.workunit.client.1.smithi053.stdout:6/284: write d2/d1b/d3d/d49/de/f15 [2865268,47201] 0 2024-09-21T04:35:05.088 INFO:tasks.workunit.client.1.smithi053.stdout:7/238: write d0/fb [2183001,31884] 0 2024-09-21T04:35:05.090 INFO:tasks.workunit.client.1.smithi053.stdout:6/285: fdatasync d2/fd 0 2024-09-21T04:35:05.093 INFO:tasks.workunit.client.1.smithi053.stdout:7/239: chown d0/d2b/f37 462600486 1 2024-09-21T04:35:05.093 INFO:tasks.workunit.client.1.smithi053.stdout:6/286: dread - d2/d1b/d3d/f4a zero size 2024-09-21T04:35:05.095 INFO:tasks.workunit.client.1.smithi053.stdout:9/320: dwrite d0/d1a/f1b [0,4194304] 0 2024-09-21T04:35:05.096 INFO:tasks.workunit.client.1.smithi053.stdout:6/287: dread - d2/d1b/d1e/d27/d41/f46 zero size 2024-09-21T04:35:05.096 INFO:tasks.workunit.client.1.smithi053.stdout:7/240: unlink d0/d6/l28 0 2024-09-21T04:35:05.097 INFO:tasks.workunit.client.1.smithi053.stdout:7/241: chown d0/d2b/l3b 1573563 1 2024-09-21T04:35:05.098 INFO:tasks.workunit.client.1.smithi053.stdout:6/288: symlink d2/l62 0 2024-09-21T04:35:05.099 INFO:tasks.workunit.client.1.smithi053.stdout:6/289: fdatasync d2/d1b/d3d/d49/d1a/d2e/f55 0 2024-09-21T04:35:05.100 INFO:tasks.workunit.client.1.smithi053.stdout:6/290: fsync d2/d1b/d3d/f4a 0 2024-09-21T04:35:05.100 INFO:tasks.workunit.client.1.smithi053.stdout:7/242: mknod d0/c47 0 2024-09-21T04:35:05.103 INFO:tasks.workunit.client.1.smithi053.stdout:4/300: dread d1/f4 [0,4194304] 0 2024-09-21T04:35:05.104 INFO:tasks.workunit.client.1.smithi053.stdout:7/243: mkdir d0/d2b/d36/d48 0 2024-09-21T04:35:05.104 INFO:tasks.workunit.client.1.smithi053.stdout:4/301: fsync d1/dd/de/f2a 0 2024-09-21T04:35:05.105 INFO:tasks.workunit.client.1.smithi053.stdout:7/244: fsync d0/f23 0 2024-09-21T04:35:05.106 INFO:tasks.workunit.client.1.smithi053.stdout:4/302: mknod d1/d2c/c5c 0 2024-09-21T04:35:05.108 INFO:tasks.workunit.client.1.smithi053.stdout:7/245: creat d0/f49 x:0 0 0 2024-09-21T04:35:05.111 INFO:tasks.workunit.client.1.smithi053.stdout:4/303: link d1/d27/f31 d1/dd/de/d14/f5d 0 2024-09-21T04:35:05.112 INFO:tasks.workunit.client.1.smithi053.stdout:6/291: dread d2/d1b/d3d/d49/d1a/d2e/d38/f36 [0,4194304] 0 2024-09-21T04:35:05.112 INFO:tasks.workunit.client.1.smithi053.stdout:4/304: readlink d1/dd/de/l4f 0 2024-09-21T04:35:05.112 INFO:tasks.workunit.client.1.smithi053.stdout:7/246: creat d0/d24/d46/f4a x:0 0 0 2024-09-21T04:35:05.113 INFO:tasks.workunit.client.1.smithi053.stdout:6/292: write d2/d1b/d3d/d49/d5c/f5d [684600,110192] 0 2024-09-21T04:35:05.116 INFO:tasks.workunit.client.1.smithi053.stdout:7/247: rename d0/d6/l1f to d0/d2b/d36/d48/l4b 0 2024-09-21T04:35:05.118 INFO:tasks.workunit.client.1.smithi053.stdout:6/293: rename d2/d1b/d3d/d49/de/d32/c3b to d2/d1b/d3d/d49/d5c/c63 0 2024-09-21T04:35:05.121 INFO:tasks.workunit.client.1.smithi053.stdout:6/294: dread d2/d1b/d3d/d49/d1a/f61 [0,4194304] 0 2024-09-21T04:35:05.121 INFO:tasks.workunit.client.1.smithi053.stdout:6/295: chown d2/d1d 26129060 1 2024-09-21T04:35:05.124 INFO:tasks.workunit.client.1.smithi053.stdout:4/305: truncate d1/d27/f31 4069156 0 2024-09-21T04:35:05.126 INFO:tasks.workunit.client.1.smithi053.stdout:4/306: read - d1/dd/de/d14/d3f/f50 zero size 2024-09-21T04:35:05.126 INFO:tasks.workunit.client.1.smithi053.stdout:7/248: getdents d0/d6 0 2024-09-21T04:35:05.128 INFO:tasks.workunit.client.1.smithi053.stdout:6/296: dwrite d2/d1b/f3a [0,4194304] 0 2024-09-21T04:35:05.129 INFO:tasks.workunit.client.1.smithi053.stdout:6/297: read d2/d1b/d3d/d49/de/f14 [98081,45630] 0 2024-09-21T04:35:05.132 INFO:tasks.workunit.client.1.smithi053.stdout:7/249: rename d0/d2b/d36/l40 to d0/d2b/l4c 0 2024-09-21T04:35:05.132 INFO:tasks.workunit.client.1.smithi053.stdout:7/250: dread - d0/d2b/f32 zero size 2024-09-21T04:35:05.134 INFO:tasks.workunit.client.1.smithi053.stdout:4/307: rename d1/dd/c16 to d1/d27/d53/c5e 0 2024-09-21T04:35:05.135 INFO:tasks.workunit.client.1.smithi053.stdout:7/251: dread d0/f15 [0,4194304] 0 2024-09-21T04:35:05.139 INFO:tasks.workunit.client.1.smithi053.stdout:4/308: symlink d1/dd/de/d14/d3f/l5f 0 2024-09-21T04:35:05.142 INFO:tasks.workunit.client.1.smithi053.stdout:6/298: truncate d2/d1b/d1e/d2a/d4b/f58 978443 0 2024-09-21T04:35:05.147 INFO:tasks.workunit.client.1.smithi053.stdout:4/309: creat d1/dd/de/d14/d20/d5b/f60 x:0 0 0 2024-09-21T04:35:05.148 INFO:tasks.workunit.client.1.smithi053.stdout:4/310: write d1/dd/de/f2b [296798,9470] 0 2024-09-21T04:35:05.149 INFO:tasks.workunit.client.1.smithi053.stdout:4/311: readlink d1/d27/d33/l4b 0 2024-09-21T04:35:05.153 INFO:tasks.workunit.client.1.smithi053.stdout:7/252: truncate d0/d6/f1b 2519876 0 2024-09-21T04:35:05.155 INFO:tasks.workunit.client.1.smithi053.stdout:4/312: dwrite d1/f2d [0,4194304] 0 2024-09-21T04:35:05.160 INFO:tasks.workunit.client.1.smithi053.stdout:6/299: dread d2/d1b/d3d/d49/d1a/f21 [0,4194304] 0 2024-09-21T04:35:05.164 INFO:tasks.workunit.client.1.smithi053.stdout:7/253: getdents d0/d2b/d36/d48 0 2024-09-21T04:35:05.166 INFO:tasks.workunit.client.1.smithi053.stdout:6/300: rename d2/d1b/d1e/d2a/f34 to d2/d1b/d1e/d27/d41/f64 0 2024-09-21T04:35:05.167 INFO:tasks.workunit.client.1.smithi053.stdout:4/313: dwrite d1/dd/de/d14/d20/f45 [0,4194304] 0 2024-09-21T04:35:05.168 INFO:tasks.workunit.client.1.smithi053.stdout:6/301: write d2/d1b/d3d/d49/f4c [9145172,84622] 0 2024-09-21T04:35:05.169 INFO:tasks.workunit.client.1.smithi053.stdout:7/254: rename d0/d2b/f37 to d0/d24/f4d 0 2024-09-21T04:35:05.174 INFO:tasks.workunit.client.1.smithi053.stdout:4/314: creat d1/d27/d2e/f61 x:0 0 0 2024-09-21T04:35:05.177 INFO:tasks.workunit.client.1.smithi053.stdout:6/302: mkdir d2/d1b/d3d/d49/d1a/d2e/d5f/d65 0 2024-09-21T04:35:05.177 INFO:tasks.workunit.client.1.smithi053.stdout:6/303: write d2/d1b/d1e/f4f [2624197,8876] 0 2024-09-21T04:35:05.179 INFO:tasks.workunit.client.1.smithi053.stdout:6/304: chown d2/d1b/d1e/d27/d41/f64 343248 1 2024-09-21T04:35:05.184 INFO:tasks.workunit.client.1.smithi053.stdout:7/255: read d0/d6/f1b [2056328,84927] 0 2024-09-21T04:35:05.185 INFO:tasks.workunit.client.1.smithi053.stdout:7/256: read d0/d6/f9 [2878805,123704] 0 2024-09-21T04:35:05.189 INFO:tasks.workunit.client.1.smithi053.stdout:4/315: getdents d1/dd/de/d14/d20/d5b 0 2024-09-21T04:35:05.193 INFO:tasks.workunit.client.1.smithi053.stdout:7/257: mknod d0/d2b/d36/d48/c4e 0 2024-09-21T04:35:05.200 INFO:tasks.workunit.client.1.smithi053.stdout:7/258: symlink d0/d2b/l4f 0 2024-09-21T04:35:05.201 INFO:tasks.workunit.client.1.smithi053.stdout:7/259: chown d0/le 109 1 2024-09-21T04:35:05.203 INFO:tasks.workunit.client.1.smithi053.stdout:4/316: mkdir d1/dd/d62 0 2024-09-21T04:35:05.205 INFO:tasks.workunit.client.1.smithi053.stdout:7/260: write d0/d24/f4d [898759,88027] 0 2024-09-21T04:35:05.210 INFO:tasks.workunit.client.1.smithi053.stdout:6/305: getdents d2/d1b/d3d/d49/d1a/d2e/d5f 0 2024-09-21T04:35:05.211 INFO:tasks.workunit.client.1.smithi053.stdout:7/261: mkdir d0/d2b/d36/d48/d50 0 2024-09-21T04:35:05.211 INFO:tasks.workunit.client.1.smithi053.stdout:6/306: read - d2/d1b/d1e/d27/d41/f46 zero size 2024-09-21T04:35:05.211 INFO:tasks.workunit.client.1.smithi053.stdout:7/262: chown d0/d2b/d36/d48/d50 134 1 2024-09-21T04:35:05.212 INFO:tasks.workunit.client.1.smithi053.stdout:6/307: chown d2/d1b/d3d/d49/d1a/d2e/f55 1009939711 1 2024-09-21T04:35:05.215 INFO:tasks.workunit.client.1.smithi053.stdout:6/308: symlink d2/d1b/d3d/l66 0 2024-09-21T04:35:05.216 INFO:tasks.workunit.client.1.smithi053.stdout:7/263: symlink d0/d2b/d36/d48/d50/l51 0 2024-09-21T04:35:05.219 INFO:tasks.workunit.client.1.smithi053.stdout:7/264: mkdir d0/d2b/d52 0 2024-09-21T04:35:05.225 INFO:tasks.workunit.client.1.smithi053.stdout:7/265: rmdir d0/d2b/d52 0 2024-09-21T04:35:05.227 INFO:tasks.workunit.client.1.smithi053.stdout:4/317: truncate d1/d2c/f40 10890711 0 2024-09-21T04:35:05.228 INFO:tasks.workunit.client.1.smithi053.stdout:7/266: creat d0/d2b/d36/d48/f53 x:0 0 0 2024-09-21T04:35:05.228 INFO:tasks.workunit.client.1.smithi053.stdout:4/318: write d1/dd/de/d14/d3f/f50 [982016,69428] 0 2024-09-21T04:35:05.231 INFO:tasks.workunit.client.1.smithi053.stdout:7/267: rmdir d0/d2b/d36 39 2024-09-21T04:35:05.232 INFO:tasks.workunit.client.1.smithi053.stdout:4/319: chown d1/d27/d53/l59 4392 1 2024-09-21T04:35:05.234 INFO:tasks.workunit.client.1.smithi053.stdout:4/320: mkdir d1/d27/d33/d41/d63 0 2024-09-21T04:35:05.235 INFO:tasks.workunit.client.1.smithi053.stdout:7/268: creat d0/d2b/d36/d48/d50/f54 x:0 0 0 2024-09-21T04:35:05.240 INFO:tasks.workunit.client.1.smithi053.stdout:7/269: dwrite d0/d2b/f41 [0,4194304] 0 2024-09-21T04:35:05.242 INFO:tasks.workunit.client.1.smithi053.stdout:4/321: link d1/dd/c1b d1/d27/c64 0 2024-09-21T04:35:05.246 INFO:tasks.workunit.client.1.smithi053.stdout:4/322: creat d1/d27/d33/f65 x:0 0 0 2024-09-21T04:35:05.252 INFO:tasks.workunit.client.1.smithi053.stdout:4/323: dwrite d1/d27/d33/f65 [0,4194304] 0 2024-09-21T04:35:05.255 INFO:tasks.workunit.client.1.smithi053.stdout:4/324: readlink d1/l13 0 2024-09-21T04:35:05.260 INFO:tasks.workunit.client.1.smithi053.stdout:4/325: mknod d1/dd/de/d14/d20/d4e/c66 0 2024-09-21T04:35:05.261 INFO:tasks.workunit.client.1.smithi053.stdout:4/326: write d1/d27/d33/f3c [853915,14678] 0 2024-09-21T04:35:05.261 INFO:tasks.workunit.client.1.smithi053.stdout:8/295: sync 2024-09-21T04:35:05.261 INFO:tasks.workunit.client.1.smithi053.stdout:5/305: sync 2024-09-21T04:35:05.265 INFO:tasks.workunit.client.1.smithi053.stdout:4/327: readlink d1/l7 0 2024-09-21T04:35:05.267 INFO:tasks.workunit.client.1.smithi053.stdout:4/328: write d1/d27/f37 [977858,67692] 0 2024-09-21T04:35:05.270 INFO:tasks.workunit.client.1.smithi053.stdout:5/306: unlink d0/dd/d30/l3e 0 2024-09-21T04:35:05.272 INFO:tasks.workunit.client.1.smithi053.stdout:4/329: dread d1/f4 [0,4194304] 0 2024-09-21T04:35:05.275 INFO:tasks.workunit.client.1.smithi053.stdout:5/307: mkdir d0/dd/d5c/d63 0 2024-09-21T04:35:05.278 INFO:tasks.workunit.client.1.smithi053.stdout:4/330: creat d1/d27/d2e/f67 x:0 0 0 2024-09-21T04:35:05.278 INFO:tasks.workunit.client.1.smithi053.stdout:5/308: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:35:05.281 INFO:tasks.workunit.client.1.smithi053.stdout:4/331: chown d1/l9 0 1 2024-09-21T04:35:05.284 INFO:tasks.workunit.client.1.smithi053.stdout:5/309: creat d0/dd/d30/d35/f64 x:0 0 0 2024-09-21T04:35:05.290 INFO:tasks.workunit.client.1.smithi053.stdout:4/332: link d1/dd/de/d14/d20/d5b/f60 d1/dd/de/d14/d20/d4e/f68 0 2024-09-21T04:35:05.297 INFO:tasks.workunit.client.1.smithi053.stdout:5/310: link d0/l1 d0/dd/d30/d51/l65 0 2024-09-21T04:35:05.298 INFO:tasks.workunit.client.1.smithi053.stdout:4/333: dwrite d1/dd/de/d14/d20/d5b/f60 [0,4194304] 0 2024-09-21T04:35:05.304 INFO:tasks.workunit.client.1.smithi053.stdout:5/311: dwrite d0/dd/d48/f54 [0,4194304] 0 2024-09-21T04:35:05.312 INFO:tasks.workunit.client.1.smithi053.stdout:5/312: mkdir d0/d66 0 2024-09-21T04:35:05.317 INFO:tasks.workunit.client.1.smithi053.stdout:5/313: mkdir d0/dd/d67 0 2024-09-21T04:35:05.319 INFO:tasks.workunit.client.1.smithi053.stdout:5/314: read d0/dd/f44 [360888,20772] 0 2024-09-21T04:35:05.322 INFO:tasks.workunit.client.1.smithi053.stdout:5/315: mkdir d0/d8/db/d68 0 2024-09-21T04:35:05.323 INFO:tasks.workunit.client.1.smithi053.stdout:8/296: dwrite d3/d39/f27 [0,4194304] 0 2024-09-21T04:35:05.323 INFO:tasks.workunit.client.1.smithi053.stdout:8/297: read d3/f4a [2019564,102956] 0 2024-09-21T04:35:05.324 INFO:tasks.workunit.client.1.smithi053.stdout:8/298: stat d3/df/d10/c3e 0 2024-09-21T04:35:05.332 INFO:tasks.workunit.client.1.smithi053.stdout:8/299: dwrite d3/d5/d38/f4e [0,4194304] 0 2024-09-21T04:35:05.333 INFO:tasks.workunit.client.1.smithi053.stdout:5/316: unlink d0/l1 0 2024-09-21T04:35:05.337 INFO:tasks.workunit.client.1.smithi053.stdout:8/300: dread d3/df/d14/d16/f35 [0,4194304] 0 2024-09-21T04:35:05.346 INFO:tasks.workunit.client.1.smithi053.stdout:8/301: dwrite d3/df/f62 [0,4194304] 0 2024-09-21T04:35:05.408 INFO:tasks.workunit.client.1.smithi053.stdout:9/321: sync 2024-09-21T04:35:05.408 INFO:tasks.workunit.client.1.smithi053.stdout:2/271: sync 2024-09-21T04:35:05.409 INFO:tasks.workunit.client.1.smithi053.stdout:1/375: sync 2024-09-21T04:35:05.409 INFO:tasks.workunit.client.1.smithi053.stdout:0/231: sync 2024-09-21T04:35:05.409 INFO:tasks.workunit.client.1.smithi053.stdout:3/260: sync 2024-09-21T04:35:05.410 INFO:tasks.workunit.client.1.smithi053.stdout:0/232: chown d5/de/d1c 62128957 1 2024-09-21T04:35:05.411 INFO:tasks.workunit.client.1.smithi053.stdout:1/376: rename f1 to d7/d63/f7d 0 2024-09-21T04:35:05.411 INFO:tasks.workunit.client.1.smithi053.stdout:0/233: read - d5/f39 zero size 2024-09-21T04:35:05.412 INFO:tasks.workunit.client.1.smithi053.stdout:1/377: fdatasync d7/d9/d43/f6e 0 2024-09-21T04:35:05.416 INFO:tasks.workunit.client.1.smithi053.stdout:3/261: dwrite f9 [0,4194304] 0 2024-09-21T04:35:05.418 INFO:tasks.workunit.client.1.smithi053.stdout:1/378: chown d7/d9/da/d40/d49/l66 227362779 1 2024-09-21T04:35:05.421 INFO:tasks.workunit.client.1.smithi053.stdout:3/262: truncate d16/d17/d21/d4e/d51/f54 803369 0 2024-09-21T04:35:05.422 INFO:tasks.workunit.client.1.smithi053.stdout:0/234: dwrite d5/f3b [0,4194304] 0 2024-09-21T04:35:05.424 INFO:tasks.workunit.client.1.smithi053.stdout:1/379: link d7/f56 d7/d9/da/d4c/f7e 0 2024-09-21T04:35:05.427 INFO:tasks.workunit.client.1.smithi053.stdout:0/235: mkdir d5/d10/d4c/d4d 0 2024-09-21T04:35:05.429 INFO:tasks.workunit.client.1.smithi053.stdout:0/236: stat d5/de/f35 0 2024-09-21T04:35:05.431 INFO:tasks.workunit.client.1.smithi053.stdout:3/263: getdents d16/d17/d21/d26/d49 0 2024-09-21T04:35:05.433 INFO:tasks.workunit.client.1.smithi053.stdout:3/264: fdatasync d16/d1b/f40 0 2024-09-21T04:35:05.434 INFO:tasks.workunit.client.1.smithi053.stdout:0/237: unlink d5/d10/c31 0 2024-09-21T04:35:05.434 INFO:tasks.workunit.client.1.smithi053.stdout:1/380: dwrite d7/d9/d43/f6e [0,4194304] 0 2024-09-21T04:35:05.436 INFO:tasks.workunit.client.1.smithi053.stdout:3/265: write d16/d1b/f1f [3859647,2807] 0 2024-09-21T04:35:05.437 INFO:tasks.workunit.client.1.smithi053.stdout:9/322: truncate d0/d2/d9/f19 3428447 0 2024-09-21T04:35:05.439 INFO:tasks.workunit.client.1.smithi053.stdout:1/381: write d7/d9/da/d11/d2b/d7a/d36/f4e [802353,36152] 0 2024-09-21T04:35:05.441 INFO:tasks.workunit.client.1.smithi053.stdout:3/266: mknod d16/d17/d21/d4e/c5b 0 2024-09-21T04:35:05.443 INFO:tasks.workunit.client.1.smithi053.stdout:3/267: write d16/d46/f4a [864207,130409] 0 2024-09-21T04:35:05.444 INFO:tasks.workunit.client.1.smithi053.stdout:1/382: fdatasync d7/d9/da/d4c/f7e 0 2024-09-21T04:35:05.444 INFO:tasks.workunit.client.1.smithi053.stdout:9/323: symlink d0/d1c/d21/d46/l68 0 2024-09-21T04:35:05.444 INFO:tasks.workunit.client.1.smithi053.stdout:0/238: creat d5/d10/d4c/d4d/f4e x:0 0 0 2024-09-21T04:35:05.446 INFO:tasks.workunit.client.1.smithi053.stdout:9/324: dread - d0/d1c/f44 zero size 2024-09-21T04:35:05.451 INFO:tasks.workunit.client.1.smithi053.stdout:9/325: write d0/d2/d9/d1f/f26 [1062817,79447] 0 2024-09-21T04:35:05.455 INFO:tasks.workunit.client.1.smithi053.stdout:1/383: getdents d7/d9/da/d40/d49 0 2024-09-21T04:35:05.459 INFO:tasks.workunit.client.1.smithi053.stdout:9/326: dread d0/d1a/f1b [0,4194304] 0 2024-09-21T04:35:05.463 INFO:tasks.workunit.client.1.smithi053.stdout:3/268: dread d16/d1b/f27 [0,4194304] 0 2024-09-21T04:35:05.464 INFO:tasks.workunit.client.1.smithi053.stdout:9/327: creat d0/d28/d63/f69 x:0 0 0 2024-09-21T04:35:05.464 INFO:tasks.workunit.client.1.smithi053.stdout:1/384: dwrite d7/d9/da/d11/f5c [0,4194304] 0 2024-09-21T04:35:05.465 INFO:tasks.workunit.client.1.smithi053.stdout:0/239: dread d5/fa [0,4194304] 0 2024-09-21T04:35:05.468 INFO:tasks.workunit.client.1.smithi053.stdout:9/328: rename d0/d1a/c3f to d0/df/c6a 0 2024-09-21T04:35:05.470 INFO:tasks.workunit.client.1.smithi053.stdout:3/269: mknod d16/d17/d21/d42/c5c 0 2024-09-21T04:35:05.471 INFO:tasks.workunit.client.1.smithi053.stdout:9/329: write d0/f4b [594892,45842] 0 2024-09-21T04:35:05.472 INFO:tasks.workunit.client.1.smithi053.stdout:0/240: symlink d5/d10/d4c/l4f 0 2024-09-21T04:35:05.474 INFO:tasks.workunit.client.1.smithi053.stdout:3/270: dread d16/d17/d3c/f52 [0,4194304] 0 2024-09-21T04:35:05.474 INFO:tasks.workunit.client.1.smithi053.stdout:0/241: chown d5/d10/d44/l49 5160 1 2024-09-21T04:35:05.475 INFO:tasks.workunit.client.1.smithi053.stdout:1/385: dread d7/d9/d43/f6e [0,4194304] 0 2024-09-21T04:35:05.477 INFO:tasks.workunit.client.1.smithi053.stdout:0/242: chown d5/d10/d4c/l4f 234973034 1 2024-09-21T04:35:05.480 INFO:tasks.workunit.client.1.smithi053.stdout:0/243: mkdir d5/dd/d50 0 2024-09-21T04:35:05.481 INFO:tasks.workunit.client.1.smithi053.stdout:9/330: dwrite d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:05.481 INFO:tasks.workunit.client.1.smithi053.stdout:0/244: fdatasync d5/de/d1c/f3d 0 2024-09-21T04:35:05.483 INFO:tasks.workunit.client.1.smithi053.stdout:1/386: rename d7/d9/da/d11/d2b/d7a to d7/d9/d43/d54/d7f 0 2024-09-21T04:35:05.483 INFO:tasks.workunit.client.1.smithi053.stdout:0/245: write d5/d10/d4c/d4d/f4e [359655,99147] 0 2024-09-21T04:35:05.484 INFO:tasks.workunit.client.1.smithi053.stdout:3/271: link d16/d17/d21/d26/l4c d16/d17/d21/d4e/l5d 0 2024-09-21T04:35:05.485 INFO:tasks.workunit.client.1.smithi053.stdout:9/331: creat d0/d28/f6b x:0 0 0 2024-09-21T04:35:05.489 INFO:tasks.workunit.client.1.smithi053.stdout:3/272: write d16/f1d [4224692,67017] 0 2024-09-21T04:35:05.492 INFO:tasks.workunit.client.1.smithi053.stdout:3/273: unlink d16/d17/d3c/f52 0 2024-09-21T04:35:05.497 INFO:tasks.workunit.client.1.smithi053.stdout:1/387: dread d7/d9/da/d3c/f65 [0,4194304] 0 2024-09-21T04:35:05.497 INFO:tasks.workunit.client.1.smithi053.stdout:7/270: sync 2024-09-21T04:35:05.497 INFO:tasks.workunit.client.1.smithi053.stdout:6/309: sync 2024-09-21T04:35:05.499 INFO:tasks.workunit.client.1.smithi053.stdout:9/332: dwrite d0/f15 [0,4194304] 0 2024-09-21T04:35:05.500 INFO:tasks.workunit.client.1.smithi053.stdout:1/388: truncate f0 5400721 0 2024-09-21T04:35:05.502 INFO:tasks.workunit.client.1.smithi053.stdout:7/271: symlink d0/d2b/d36/l55 0 2024-09-21T04:35:05.503 INFO:tasks.workunit.client.1.smithi053.stdout:9/333: mkdir d0/d6c 0 2024-09-21T04:35:05.508 INFO:tasks.workunit.client.1.smithi053.stdout:3/274: dwrite d16/d1b/f59 [0,4194304] 0 2024-09-21T04:35:05.508 INFO:tasks.workunit.client.1.smithi053.stdout:1/389: rmdir d7/d9/da/d40/d64/d74 39 2024-09-21T04:35:05.509 INFO:tasks.workunit.client.1.smithi053.stdout:7/272: mknod d0/d2b/d36/d48/d50/c56 0 2024-09-21T04:35:05.509 INFO:tasks.workunit.client.1.smithi053.stdout:0/246: dread d5/d10/f2b [0,4194304] 0 2024-09-21T04:35:05.510 INFO:tasks.workunit.client.0.smithi007.stderr:++ readlink -f fsstress 2024-09-21T04:35:05.510 INFO:tasks.workunit.client.1.smithi053.stdout:3/275: write d16/d1b/f2f [2845456,95695] 0 2024-09-21T04:35:05.512 INFO:tasks.workunit.client.0.smithi007.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-21T04:35:05.512 INFO:tasks.workunit.client.0.smithi007.stderr:+ popd 2024-09-21T04:35:05.514 INFO:tasks.workunit.client.0.smithi007.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-21T04:35:05.514 INFO:tasks.workunit.client.0.smithi007.stderr:+ popd 2024-09-21T04:35:05.514 INFO:tasks.workunit.client.0.smithi007.stdout:~/cephtest/mnt.0/client.0/tmp 2024-09-21T04:35:05.515 INFO:tasks.workunit.client.1.smithi053.stdout:0/247: fsync d5/d6/f16 0 2024-09-21T04:35:05.515 INFO:tasks.workunit.client.0.smithi007.stderr:++ mktemp -d -p . 2024-09-21T04:35:05.517 INFO:tasks.workunit.client.0.smithi007.stderr:+ T=./tmp.FBBCV7FZnT 2024-09-21T04:35:05.517 INFO:tasks.workunit.client.0.smithi007.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.FBBCV7FZnT -l 1 -n 1000 -p 10 -v 2024-09-21T04:35:05.518 INFO:tasks.workunit.client.1.smithi053.stdout:7/273: mknod d0/d2b/d36/d48/d50/c57 0 2024-09-21T04:35:05.519 INFO:tasks.workunit.client.1.smithi053.stdout:1/390: creat d7/d9/d43/d54/f80 x:0 0 0 2024-09-21T04:35:05.521 INFO:tasks.workunit.client.1.smithi053.stdout:7/274: chown d0/d6/f1b 53 1 2024-09-21T04:35:05.521 INFO:tasks.workunit.client.1.smithi053.stdout:3/276: dread d16/d1b/f27 [0,4194304] 0 2024-09-21T04:35:05.521 INFO:tasks.workunit.client.0.smithi007.stdout:seed = 1726594266 2024-09-21T04:35:05.523 INFO:tasks.workunit.client.1.smithi053.stdout:6/310: dwrite d2/d1b/d1e/d2a/d4b/f58 [0,4194304] 0 2024-09-21T04:35:05.525 INFO:tasks.workunit.client.1.smithi053.stdout:6/311: write d2/d1b/d3d/d49/d1a/f44 [493161,46675] 0 2024-09-21T04:35:05.526 INFO:tasks.workunit.client.1.smithi053.stdout:3/277: mknod d16/d17/d3c/c5e 0 2024-09-21T04:35:05.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/0: dread - no filename 2024-09-21T04:35:05.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/1: truncate - no filename 2024-09-21T04:35:05.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/2: dread - no filename 2024-09-21T04:35:05.527 INFO:tasks.workunit.client.0.smithi007.stdout:0/3: rename - no filename 2024-09-21T04:35:05.527 INFO:tasks.workunit.client.0.smithi007.stdout:0/4: dwrite - no filename 2024-09-21T04:35:05.528 INFO:tasks.workunit.client.1.smithi053.stdout:3/278: write d16/d1b/f40 [97117,37402] 0 2024-09-21T04:35:05.528 INFO:tasks.workunit.client.1.smithi053.stdout:7/275: rename d0/d2b/d36/d48/l4b to d0/d6/l58 0 2024-09-21T04:35:05.531 INFO:tasks.workunit.client.1.smithi053.stdout:3/279: stat c1 0 2024-09-21T04:35:05.533 INFO:tasks.workunit.client.1.smithi053.stdout:1/391: truncate f0 1680942 0 2024-09-21T04:35:05.535 INFO:tasks.workunit.client.0.smithi007.stdout:2/0: read - no filename 2024-09-21T04:35:05.535 INFO:tasks.workunit.client.0.smithi007.stdout:2/1: dread - no filename 2024-09-21T04:35:05.535 INFO:tasks.workunit.client.0.smithi007.stdout:2/2: write - no filename 2024-09-21T04:35:05.535 INFO:tasks.workunit.client.0.smithi007.stdout:2/3: write - no filename 2024-09-21T04:35:05.535 INFO:tasks.workunit.client.0.smithi007.stdout:2/4: link - no file 2024-09-21T04:35:05.537 INFO:tasks.workunit.client.0.smithi007.stdout:3/0: rename - no filename 2024-09-21T04:35:05.537 INFO:tasks.workunit.client.0.smithi007.stdout:3/1: fsync - no filename 2024-09-21T04:35:05.537 INFO:tasks.workunit.client.0.smithi007.stdout:3/2: stat - no entries 2024-09-21T04:35:05.537 INFO:tasks.workunit.client.0.smithi007.stdout:3/3: fsync - no filename 2024-09-21T04:35:05.538 INFO:tasks.workunit.client.1.smithi053.stdout:6/312: mknod d2/d1b/d3d/d49/d1a/d2e/d5f/d65/c67 0 2024-09-21T04:35:05.538 INFO:tasks.workunit.client.0.smithi007.stdout:1/0: creat f0 x:0 0 0 2024-09-21T04:35:05.538 INFO:tasks.workunit.client.1.smithi053.stdout:1/392: dread d7/d9/d43/d54/d7f/d36/f4e [0,4194304] 0 2024-09-21T04:35:05.539 INFO:tasks.workunit.client.0.smithi007.stdout:1/1: write f0 [160834,75070] 0 2024-09-21T04:35:05.539 INFO:tasks.workunit.client.1.smithi053.stdout:3/280: mkdir d16/d17/d21/d26/d5f 0 2024-09-21T04:35:05.540 INFO:tasks.workunit.client.0.smithi007.stdout:1/2: chown f0 889579 1 2024-09-21T04:35:05.540 INFO:tasks.workunit.client.0.smithi007.stdout:1/3: stat f0 0 2024-09-21T04:35:05.541 INFO:tasks.workunit.client.1.smithi053.stdout:7/276: creat d0/f59 x:0 0 0 2024-09-21T04:35:05.542 INFO:tasks.workunit.client.1.smithi053.stdout:6/313: creat d2/d1b/d1e/d2a/d4b/f68 x:0 0 0 2024-09-21T04:35:05.543 INFO:tasks.workunit.client.0.smithi007.stdout:2/5: getdents . 0 2024-09-21T04:35:05.543 INFO:tasks.workunit.client.0.smithi007.stdout:2/6: dread - no filename 2024-09-21T04:35:05.543 INFO:tasks.workunit.client.0.smithi007.stdout:2/7: dwrite - no filename 2024-09-21T04:35:05.543 INFO:tasks.workunit.client.1.smithi053.stdout:6/314: read - d2/d1d/f50 zero size 2024-09-21T04:35:05.543 INFO:tasks.workunit.client.1.smithi053.stdout:3/281: mkdir d16/d17/d21/d60 0 2024-09-21T04:35:05.544 INFO:tasks.workunit.client.1.smithi053.stdout:7/277: creat d0/d24/f5a x:0 0 0 2024-09-21T04:35:05.545 INFO:tasks.workunit.client.0.smithi007.stdout:5/0: chown . 1022414 1 2024-09-21T04:35:05.546 INFO:tasks.workunit.client.1.smithi053.stdout:6/315: symlink d2/d1b/d3d/d49/de/l69 0 2024-09-21T04:35:05.546 INFO:tasks.workunit.client.1.smithi053.stdout:7/278: truncate d0/d6/f42 587320 0 2024-09-21T04:35:05.546 INFO:tasks.workunit.client.0.smithi007.stdout:3/4: mknod c0 0 2024-09-21T04:35:05.550 INFO:tasks.workunit.client.1.smithi053.stdout:7/279: symlink d0/d2b/d36/d48/l5b 0 2024-09-21T04:35:05.554 INFO:tasks.workunit.client.0.smithi007.stdout:1/4: fsync f0 0 2024-09-21T04:35:05.556 INFO:tasks.workunit.client.0.smithi007.stdout:3/5: creat f1 x:0 0 0 2024-09-21T04:35:05.557 INFO:tasks.workunit.client.0.smithi007.stdout:4/0: creat f0 x:0 0 0 2024-09-21T04:35:05.558 INFO:tasks.workunit.client.0.smithi007.stdout:4/1: write f0 [470307,55804] 0 2024-09-21T04:35:05.558 INFO:tasks.workunit.client.0.smithi007.stdout:2/8: mkdir d0 0 2024-09-21T04:35:05.559 INFO:tasks.workunit.client.0.smithi007.stdout:4/2: truncate f0 1328895 0 2024-09-21T04:35:05.560 INFO:tasks.workunit.client.0.smithi007.stdout:7/0: rename - no filename 2024-09-21T04:35:05.560 INFO:tasks.workunit.client.0.smithi007.stdout:7/1: chown . 1063245 1 2024-09-21T04:35:05.561 INFO:tasks.workunit.client.0.smithi007.stdout:7/2: chown . 16885345 1 2024-09-21T04:35:05.561 INFO:tasks.workunit.client.0.smithi007.stdout:7/3: dread - no filename 2024-09-21T04:35:05.562 INFO:tasks.workunit.client.0.smithi007.stdout:5/1: symlink l0 0 2024-09-21T04:35:05.562 INFO:tasks.workunit.client.0.smithi007.stdout:5/2: truncate - no filename 2024-09-21T04:35:05.562 INFO:tasks.workunit.client.0.smithi007.stdout:5/3: dread - no filename 2024-09-21T04:35:05.563 INFO:tasks.workunit.client.0.smithi007.stdout:1/5: rename f0 to f1 0 2024-09-21T04:35:05.564 INFO:tasks.workunit.client.0.smithi007.stdout:1/6: chown f1 203229820 1 2024-09-21T04:35:05.565 INFO:tasks.workunit.client.0.smithi007.stdout:3/6: rename c0 to c2 0 2024-09-21T04:35:05.566 INFO:tasks.workunit.client.0.smithi007.stdout:1/7: chown f1 8141928 1 2024-09-21T04:35:05.567 INFO:tasks.workunit.client.0.smithi007.stdout:1/8: chown f1 17 1 2024-09-21T04:35:05.568 INFO:tasks.workunit.client.0.smithi007.stdout:4/3: dwrite f0 [0,4194304] 0 2024-09-21T04:35:05.568 INFO:tasks.workunit.client.0.smithi007.stdout:6/0: creat f0 x:0 0 0 2024-09-21T04:35:05.569 INFO:tasks.workunit.client.0.smithi007.stdout:8/0: fdatasync - no filename 2024-09-21T04:35:05.569 INFO:tasks.workunit.client.0.smithi007.stdout:8/1: dwrite - no filename 2024-09-21T04:35:05.571 INFO:tasks.workunit.client.0.smithi007.stdout:5/4: rename l0 to l1 0 2024-09-21T04:35:05.571 INFO:tasks.workunit.client.0.smithi007.stdout:5/5: dread - no filename 2024-09-21T04:35:05.572 INFO:tasks.workunit.client.0.smithi007.stdout:2/9: mkdir d0/d1 0 2024-09-21T04:35:05.572 INFO:tasks.workunit.client.0.smithi007.stdout:2/10: dwrite - no filename 2024-09-21T04:35:05.572 INFO:tasks.workunit.client.0.smithi007.stdout:2/11: unlink - no file 2024-09-21T04:35:05.572 INFO:tasks.workunit.client.0.smithi007.stdout:2/12: read - no filename 2024-09-21T04:35:05.572 INFO:tasks.workunit.client.0.smithi007.stdout:2/13: dwrite - no filename 2024-09-21T04:35:05.574 INFO:tasks.workunit.client.0.smithi007.stdout:7/4: creat f0 x:0 0 0 2024-09-21T04:35:05.576 INFO:tasks.workunit.client.0.smithi007.stdout:7/5: chown f0 1 1 2024-09-21T04:35:05.577 INFO:tasks.workunit.client.0.smithi007.stdout:4/4: mknod c1 0 2024-09-21T04:35:05.577 INFO:tasks.workunit.client.0.smithi007.stdout:9/0: write - no filename 2024-09-21T04:35:05.580 INFO:tasks.workunit.client.0.smithi007.stdout:1/9: link f1 f2 0 2024-09-21T04:35:05.580 INFO:tasks.workunit.client.0.smithi007.stdout:6/1: dwrite f0 [0,4194304] 0 2024-09-21T04:35:05.581 INFO:tasks.workunit.client.0.smithi007.stdout:8/2: mkdir d0 0 2024-09-21T04:35:05.582 INFO:tasks.workunit.client.0.smithi007.stdout:8/3: unlink - no file 2024-09-21T04:35:05.583 INFO:tasks.workunit.client.0.smithi007.stdout:7/6: rename f0 to f1 0 2024-09-21T04:35:05.584 INFO:tasks.workunit.client.0.smithi007.stdout:7/7: truncate f1 772934 0 2024-09-21T04:35:05.585 INFO:tasks.workunit.client.0.smithi007.stdout:4/5: mkdir d2 0 2024-09-21T04:35:05.586 INFO:tasks.workunit.client.0.smithi007.stdout:4/6: chown c1 82613 1 2024-09-21T04:35:05.591 INFO:tasks.workunit.client.0.smithi007.stdout:7/8: mknod c2 0 2024-09-21T04:35:05.594 INFO:tasks.workunit.client.0.smithi007.stdout:9/1: creat f0 x:0 0 0 2024-09-21T04:35:05.595 INFO:tasks.workunit.client.1.smithi053.stdout:4/334: sync 2024-09-21T04:35:05.595 INFO:tasks.workunit.client.1.smithi053.stdout:2/272: sync 2024-09-21T04:35:05.595 INFO:tasks.workunit.client.1.smithi053.stdout:5/317: sync 2024-09-21T04:35:05.595 INFO:tasks.workunit.client.1.smithi053.stdout:8/302: sync 2024-09-21T04:35:05.595 INFO:tasks.workunit.client.1.smithi053.stdout:6/316: creat d2/d1b/d3d/d49/d1a/d2e/d5b/f6a x:0 0 0 2024-09-21T04:35:05.596 INFO:tasks.workunit.client.0.smithi007.stdout:9/2: chown f0 188445 1 2024-09-21T04:35:05.596 INFO:tasks.workunit.client.0.smithi007.stdout:3/7: chown c2 178689 1 2024-09-21T04:35:05.597 INFO:tasks.workunit.client.0.smithi007.stdout:9/3: read - f0 zero size 2024-09-21T04:35:05.598 INFO:tasks.workunit.client.0.smithi007.stdout:8/4: creat d0/f1 x:0 0 0 2024-09-21T04:35:05.598 INFO:tasks.workunit.client.0.smithi007.stdout:1/10: dwrite f2 [0,4194304] 0 2024-09-21T04:35:05.599 INFO:tasks.workunit.client.0.smithi007.stdout:6/2: fdatasync f0 0 2024-09-21T04:35:05.599 INFO:tasks.workunit.client.1.smithi053.stdout:6/317: fsync d2/d1b/d3d/d49/d1a/f57 0 2024-09-21T04:35:05.599 INFO:tasks.workunit.client.1.smithi053.stdout:2/273: write d1/d18/d1f/d22/f32 [835066,73099] 0 2024-09-21T04:35:05.599 INFO:tasks.workunit.client.1.smithi053.stdout:8/303: write d3/d5/f19 [735323,12134] 0 2024-09-21T04:35:05.600 INFO:tasks.workunit.client.0.smithi007.stdout:5/6: unlink l1 0 2024-09-21T04:35:05.601 INFO:tasks.workunit.client.1.smithi053.stdout:8/304: chown d3/d5/d4c 9823148 1 2024-09-21T04:35:05.602 INFO:tasks.workunit.client.1.smithi053.stdout:7/280: unlink d0/d2b/l3f 0 2024-09-21T04:35:05.603 INFO:tasks.workunit.client.1.smithi053.stdout:4/335: truncate d1/d27/d2e/d46/f54 508635 0 2024-09-21T04:35:05.604 INFO:tasks.workunit.client.1.smithi053.stdout:3/282: dread d16/d1b/f24 [0,4194304] 0 2024-09-21T04:35:05.606 INFO:tasks.workunit.client.1.smithi053.stdout:3/283: fdatasync fc 0 2024-09-21T04:35:05.606 INFO:tasks.workunit.client.0.smithi007.stdout:4/7: creat d2/f3 x:0 0 0 2024-09-21T04:35:05.608 INFO:tasks.workunit.client.0.smithi007.stdout:9/4: dwrite f0 [0,4194304] 0 2024-09-21T04:35:05.609 INFO:tasks.workunit.client.1.smithi053.stdout:4/336: write d1/dd/f1d [3889868,62568] 0 2024-09-21T04:35:05.609 INFO:tasks.workunit.client.0.smithi007.stdout:3/8: unlink f1 0 2024-09-21T04:35:05.610 INFO:tasks.workunit.client.0.smithi007.stdout:3/9: dwrite - no filename 2024-09-21T04:35:05.610 INFO:tasks.workunit.client.0.smithi007.stdout:3/10: dwrite - no filename 2024-09-21T04:35:05.611 INFO:tasks.workunit.client.0.smithi007.stdout:9/5: write f0 [3642859,11873] 0 2024-09-21T04:35:05.616 INFO:tasks.workunit.client.0.smithi007.stdout:1/11: dwrite f2 [0,4194304] 0 2024-09-21T04:35:05.617 INFO:tasks.workunit.client.1.smithi053.stdout:2/274: dwrite d1/d18/d1f/d27/d41/f42 [0,4194304] 0 2024-09-21T04:35:05.619 INFO:tasks.workunit.client.0.smithi007.stdout:8/5: symlink d0/l2 0 2024-09-21T04:35:05.619 INFO:tasks.workunit.client.1.smithi053.stdout:2/275: dread - d1/d18/d1f/d22/f5a zero size 2024-09-21T04:35:05.619 INFO:tasks.workunit.client.1.smithi053.stdout:2/276: stat d1/d18/d1f/f4a 0 2024-09-21T04:35:05.620 INFO:tasks.workunit.client.1.smithi053.stdout:5/318: write d0/dd/f44 [1299111,93437] 0 2024-09-21T04:35:05.621 INFO:tasks.workunit.client.1.smithi053.stdout:6/318: creat d2/d1b/f6b x:0 0 0 2024-09-21T04:35:05.621 INFO:tasks.workunit.client.0.smithi007.stdout:6/3: symlink l1 0 2024-09-21T04:35:05.623 INFO:tasks.workunit.client.1.smithi053.stdout:8/305: creat d3/d5/d4c/f6d x:0 0 0 2024-09-21T04:35:05.623 INFO:tasks.workunit.client.0.smithi007.stdout:5/7: mkdir d2 0 2024-09-21T04:35:05.624 INFO:tasks.workunit.client.0.smithi007.stdout:5/8: dread - no filename 2024-09-21T04:35:05.624 INFO:tasks.workunit.client.0.smithi007.stdout:5/9: dread - no filename 2024-09-21T04:35:05.624 INFO:tasks.workunit.client.0.smithi007.stdout:7/9: fsync f1 0 2024-09-21T04:35:05.625 INFO:tasks.workunit.client.1.smithi053.stdout:2/277: write d1/f3f [279925,96208] 0 2024-09-21T04:35:05.626 INFO:tasks.workunit.client.0.smithi007.stdout:4/8: mknod d2/c4 0 2024-09-21T04:35:05.627 INFO:tasks.workunit.client.0.smithi007.stdout:4/9: dread - d2/f3 zero size 2024-09-21T04:35:05.627 INFO:tasks.workunit.client.0.smithi007.stdout:4/10: read - d2/f3 zero size 2024-09-21T04:35:05.628 INFO:tasks.workunit.client.0.smithi007.stdout:4/11: fsync d2/f3 0 2024-09-21T04:35:05.629 INFO:tasks.workunit.client.1.smithi053.stdout:7/281: dread d0/f23 [0,4194304] 0 2024-09-21T04:35:05.630 INFO:tasks.workunit.client.1.smithi053.stdout:7/282: stat d0/d6/l12 0 2024-09-21T04:35:05.630 INFO:tasks.workunit.client.1.smithi053.stdout:7/283: chown d0/d2b/d36/d48 5 1 2024-09-21T04:35:05.631 INFO:tasks.workunit.client.0.smithi007.stdout:3/11: creat f3 x:0 0 0 2024-09-21T04:35:05.631 INFO:tasks.workunit.client.0.smithi007.stdout:3/12: chown c2 302 1 2024-09-21T04:35:05.632 INFO:tasks.workunit.client.1.smithi053.stdout:8/306: mknod d3/d5/d4c/d53/c6e 0 2024-09-21T04:35:05.632 INFO:tasks.workunit.client.1.smithi053.stdout:8/307: readlink d3/d5/l5a 0 2024-09-21T04:35:05.632 INFO:tasks.workunit.client.0.smithi007.stdout:9/6: symlink l1 0 2024-09-21T04:35:05.633 INFO:tasks.workunit.client.1.smithi053.stdout:8/308: stat d3/d5/d4b/d5c 0 2024-09-21T04:35:05.634 INFO:tasks.workunit.client.0.smithi007.stdout:1/12: creat f3 x:0 0 0 2024-09-21T04:35:05.635 INFO:tasks.workunit.client.0.smithi007.stdout:1/13: write f3 [365416,16841] 0 2024-09-21T04:35:05.635 INFO:tasks.workunit.client.1.smithi053.stdout:8/309: dread - d3/df/d49/f61 zero size 2024-09-21T04:35:05.635 INFO:tasks.workunit.client.0.smithi007.stdout:1/14: chown f2 16692101 1 2024-09-21T04:35:05.636 INFO:tasks.workunit.client.0.smithi007.stdout:8/6: mkdir d0/d3 0 2024-09-21T04:35:05.637 INFO:tasks.workunit.client.0.smithi007.stdout:8/7: write d0/f1 [591009,22018] 0 2024-09-21T04:35:05.638 INFO:tasks.workunit.client.0.smithi007.stdout:6/4: unlink l1 0 2024-09-21T04:35:05.642 INFO:tasks.workunit.client.0.smithi007.stdout:7/10: unlink c2 0 2024-09-21T04:35:05.646 INFO:tasks.workunit.client.0.smithi007.stdout:6/5: dwrite f0 [0,4194304] 0 2024-09-21T04:35:05.646 INFO:tasks.workunit.client.0.smithi007.stdout:4/12: unlink c1 0 2024-09-21T04:35:05.648 INFO:tasks.workunit.client.0.smithi007.stdout:3/13: mknod c4 0 2024-09-21T04:35:05.650 INFO:tasks.workunit.client.0.smithi007.stdout:3/14: dread - f3 zero size 2024-09-21T04:35:05.651 INFO:tasks.workunit.client.0.smithi007.stdout:1/15: creat f4 x:0 0 0 2024-09-21T04:35:05.651 INFO:tasks.workunit.client.0.smithi007.stdout:3/15: dread - f3 zero size 2024-09-21T04:35:05.652 INFO:tasks.workunit.client.0.smithi007.stdout:8/8: mknod d0/c4 0 2024-09-21T04:35:05.653 INFO:tasks.workunit.client.0.smithi007.stdout:3/16: stat f3 0 2024-09-21T04:35:05.653 INFO:tasks.workunit.client.0.smithi007.stdout:1/16: chown f3 5 1 2024-09-21T04:35:05.654 INFO:tasks.workunit.client.0.smithi007.stdout:8/9: stat d0/d3 0 2024-09-21T04:35:05.655 INFO:tasks.workunit.client.0.smithi007.stdout:1/17: chown f3 1040 1 2024-09-21T04:35:05.655 INFO:tasks.workunit.client.0.smithi007.stdout:6/6: dread f0 [0,4194304] 0 2024-09-21T04:35:05.655 INFO:tasks.workunit.client.0.smithi007.stdout:7/11: mkdir d3 0 2024-09-21T04:35:05.656 INFO:tasks.workunit.client.0.smithi007.stdout:4/13: dwrite f0 [0,4194304] 0 2024-09-21T04:35:05.656 INFO:tasks.workunit.client.0.smithi007.stdout:8/10: readlink d0/l2 0 2024-09-21T04:35:05.657 INFO:tasks.workunit.client.1.smithi053.stdout:4/337: rmdir d1/dd/de/d14/d2f 0 2024-09-21T04:35:05.659 INFO:tasks.workunit.client.0.smithi007.stdout:5/10: symlink d2/l3 0 2024-09-21T04:35:05.659 INFO:tasks.workunit.client.1.smithi053.stdout:4/338: dread d1/d27/d2e/d46/f54 [0,4194304] 0 2024-09-21T04:35:05.660 INFO:tasks.workunit.client.1.smithi053.stdout:3/284: link d16/d46/c50 d16/d17/d21/d26/d49/d4d/d53/c61 0 2024-09-21T04:35:05.661 INFO:tasks.workunit.client.0.smithi007.stdout:5/11: stat d2 0 2024-09-21T04:35:05.662 INFO:tasks.workunit.client.0.smithi007.stdout:5/12: rename d2 to d2/d4 22 2024-09-21T04:35:05.665 INFO:tasks.workunit.client.0.smithi007.stdout:5/13: rename d2 to d2/d5 22 2024-09-21T04:35:05.665 INFO:tasks.workunit.client.1.smithi053.stdout:5/319: rmdir d0/dd/d5c/d63 0 2024-09-21T04:35:05.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/11: dread d0/f1 [0,4194304] 0 2024-09-21T04:35:05.667 INFO:tasks.workunit.client.1.smithi053.stdout:5/320: truncate d0/d8/db/f5f 6988 0 2024-09-21T04:35:05.668 INFO:tasks.workunit.client.1.smithi053.stdout:6/319: truncate d2/d1b/f3a 3112188 0 2024-09-21T04:35:05.669 INFO:tasks.workunit.client.0.smithi007.stdout:8/12: truncate d0/f1 1638265 0 2024-09-21T04:35:05.669 INFO:tasks.workunit.client.0.smithi007.stdout:6/7: dwrite f0 [0,4194304] 0 2024-09-21T04:35:05.669 INFO:tasks.workunit.client.1.smithi053.stdout:6/320: fdatasync d2/d1b/d3d/d49/de/f43 0 2024-09-21T04:35:05.671 INFO:tasks.workunit.client.0.smithi007.stdout:3/17: unlink f3 0 2024-09-21T04:35:05.671 INFO:tasks.workunit.client.0.smithi007.stdout:3/18: dwrite - no filename 2024-09-21T04:35:05.671 INFO:tasks.workunit.client.0.smithi007.stdout:3/19: truncate - no filename 2024-09-21T04:35:05.671 INFO:tasks.workunit.client.0.smithi007.stdout:4/14: dwrite f0 [0,4194304] 0 2024-09-21T04:35:05.672 INFO:tasks.workunit.client.1.smithi053.stdout:5/321: dread d0/d8/f31 [0,4194304] 0 2024-09-21T04:35:05.673 INFO:tasks.workunit.client.0.smithi007.stdout:4/15: rename d2 to d2/d5 22 2024-09-21T04:35:05.673 INFO:tasks.workunit.client.0.smithi007.stdout:1/18: mkdir d5 0 2024-09-21T04:35:05.673 INFO:tasks.workunit.client.1.smithi053.stdout:2/278: write d1/d18/d1f/d27/d41/f4c [765680,24990] 0 2024-09-21T04:35:05.674 INFO:tasks.workunit.client.0.smithi007.stdout:1/19: dread - f4 zero size 2024-09-21T04:35:05.680 INFO:tasks.workunit.client.1.smithi053.stdout:3/285: creat d16/d17/d21/d26/d3e/f62 x:0 0 0 2024-09-21T04:35:05.681 INFO:tasks.workunit.client.0.smithi007.stdout:8/13: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:35:05.685 INFO:tasks.workunit.client.0.smithi007.stdout:8/14: dread d0/f1 [0,4194304] 0 2024-09-21T04:35:05.685 INFO:tasks.workunit.client.0.smithi007.stdout:5/14: creat d2/f6 x:0 0 0 2024-09-21T04:35:05.690 INFO:tasks.workunit.client.1.smithi053.stdout:4/339: truncate d1/fc 1600822 0 2024-09-21T04:35:05.690 INFO:tasks.workunit.client.0.smithi007.stdout:4/16: creat d2/f6 x:0 0 0 2024-09-21T04:35:05.690 INFO:tasks.workunit.client.1.smithi053.stdout:2/279: creat d1/d18/f5b x:0 0 0 2024-09-21T04:35:05.690 INFO:tasks.workunit.client.1.smithi053.stdout:3/286: dwrite fc [4194304,4194304] 0 2024-09-21T04:35:05.692 INFO:tasks.workunit.client.0.smithi007.stdout:7/12: mknod d3/c4 0 2024-09-21T04:35:05.695 INFO:tasks.workunit.client.0.smithi007.stdout:4/17: dread f0 [0,4194304] 0 2024-09-21T04:35:05.695 INFO:tasks.workunit.client.1.smithi053.stdout:8/310: rename d3/df/d10/c45 to d3/c6f 0 2024-09-21T04:35:05.695 INFO:tasks.workunit.client.1.smithi053.stdout:6/321: dwrite d2/d1b/d3d/d49/d1a/f21 [0,4194304] 0 2024-09-21T04:35:05.696 INFO:tasks.workunit.client.1.smithi053.stdout:2/280: fdatasync d1/d18/f49 0 2024-09-21T04:35:05.696 INFO:tasks.workunit.client.0.smithi007.stdout:5/15: unlink d2/f6 0 2024-09-21T04:35:05.697 INFO:tasks.workunit.client.1.smithi053.stdout:4/340: chown d1/d27/d53/l59 1 1 2024-09-21T04:35:05.698 INFO:tasks.workunit.client.0.smithi007.stdout:6/8: rename f0 to f2 0 2024-09-21T04:35:05.700 INFO:tasks.workunit.client.0.smithi007.stdout:6/9: chown f2 2021882 1 2024-09-21T04:35:05.701 INFO:tasks.workunit.client.0.smithi007.stdout:1/20: mkdir d5/d6 0 2024-09-21T04:35:05.701 INFO:tasks.workunit.client.1.smithi053.stdout:8/311: creat d3/d5/d38/f70 x:0 0 0 2024-09-21T04:35:05.702 INFO:tasks.workunit.client.0.smithi007.stdout:7/13: dwrite f1 [0,4194304] 0 2024-09-21T04:35:05.702 INFO:tasks.workunit.client.0.smithi007.stdout:8/15: creat d0/d3/f5 x:0 0 0 2024-09-21T04:35:05.706 INFO:tasks.workunit.client.0.smithi007.stdout:7/14: write f1 [450330,49310] 0 2024-09-21T04:35:05.707 INFO:tasks.workunit.client.0.smithi007.stdout:7/15: stat d3/c4 0 2024-09-21T04:35:05.710 INFO:tasks.workunit.client.0.smithi007.stdout:4/18: mkdir d2/d7 0 2024-09-21T04:35:05.712 INFO:tasks.workunit.client.0.smithi007.stdout:5/16: rmdir d2 39 2024-09-21T04:35:05.712 INFO:tasks.workunit.client.1.smithi053.stdout:4/341: dread d1/dd/de/d14/d3f/f50 [0,4194304] 0 2024-09-21T04:35:05.714 INFO:tasks.workunit.client.1.smithi053.stdout:4/342: write d1/dd/de/d14/d20/f45 [3804828,75518] 0 2024-09-21T04:35:05.718 INFO:tasks.workunit.client.1.smithi053.stdout:3/287: link d16/d17/d3c/c5e d16/d17/d21/d26/d49/d4d/d53/c63 0 2024-09-21T04:35:05.719 INFO:tasks.workunit.client.0.smithi007.stdout:8/16: write d0/f1 [624866,127114] 0 2024-09-21T04:35:05.719 INFO:tasks.workunit.client.0.smithi007.stdout:8/17: chown d0 60 1 2024-09-21T04:35:05.724 INFO:tasks.workunit.client.0.smithi007.stdout:7/16: creat d3/f5 x:0 0 0 2024-09-21T04:35:05.724 INFO:tasks.workunit.client.0.smithi007.stdout:7/17: chown d3/c4 7603453 1 2024-09-21T04:35:05.725 INFO:tasks.workunit.client.0.smithi007.stdout:7/18: read - d3/f5 zero size 2024-09-21T04:35:05.725 INFO:tasks.workunit.client.1.smithi053.stdout:2/281: mkdir d1/d18/d1f/d27/d58/d5c 0 2024-09-21T04:35:05.727 INFO:tasks.workunit.client.0.smithi007.stdout:4/19: write f0 [1619468,49087] 0 2024-09-21T04:35:05.727 INFO:tasks.workunit.client.1.smithi053.stdout:4/343: symlink d1/d27/d2e/l69 0 2024-09-21T04:35:05.728 INFO:tasks.workunit.client.1.smithi053.stdout:4/344: fdatasync d1/d27/d33/f3c 0 2024-09-21T04:35:05.729 INFO:tasks.workunit.client.0.smithi007.stdout:5/17: chown d2/l3 111443931 1 2024-09-21T04:35:05.730 INFO:tasks.workunit.client.1.smithi053.stdout:8/312: mknod d3/d5/d4b/d5c/c71 0 2024-09-21T04:35:05.731 INFO:tasks.workunit.client.1.smithi053.stdout:8/313: read - d3/d28/f31 zero size 2024-09-21T04:35:05.731 INFO:tasks.workunit.client.0.smithi007.stdout:1/21: getdents d5/d6 0 2024-09-21T04:35:05.731 INFO:tasks.workunit.client.0.smithi007.stdout:5/18: readlink d2/l3 0 2024-09-21T04:35:05.731 INFO:tasks.workunit.client.0.smithi007.stdout:5/19: dread - no filename 2024-09-21T04:35:05.731 INFO:tasks.workunit.client.0.smithi007.stdout:5/20: write - no filename 2024-09-21T04:35:05.732 INFO:tasks.workunit.client.1.smithi053.stdout:2/282: dwrite d1/d18/d1f/d22/f36 [0,4194304] 0 2024-09-21T04:35:05.732 INFO:tasks.workunit.client.0.smithi007.stdout:6/10: unlink f2 0 2024-09-21T04:35:05.733 INFO:tasks.workunit.client.1.smithi053.stdout:2/283: truncate d1/d18/d1f/d22/d43/f59 298236 0 2024-09-21T04:35:05.733 INFO:tasks.workunit.client.1.smithi053.stdout:3/288: creat d16/d17/d21/f64 x:0 0 0 2024-09-21T04:35:05.734 INFO:tasks.workunit.client.1.smithi053.stdout:2/284: write d1/d18/f5b [359507,77956] 0 2024-09-21T04:35:05.735 INFO:tasks.workunit.client.0.smithi007.stdout:7/19: dwrite d3/f5 [0,4194304] 0 2024-09-21T04:35:05.735 INFO:tasks.workunit.client.1.smithi053.stdout:3/289: readlink d16/d17/d21/d26/d2b/l2d 0 2024-09-21T04:35:05.736 INFO:tasks.workunit.client.0.smithi007.stdout:4/20: rmdir d2 39 2024-09-21T04:35:05.736 INFO:tasks.workunit.client.0.smithi007.stdout:7/20: fsync f1 0 2024-09-21T04:35:05.743 INFO:tasks.workunit.client.0.smithi007.stdout:5/21: rmdir d2 39 2024-09-21T04:35:05.743 INFO:tasks.workunit.client.0.smithi007.stdout:7/21: dread d3/f5 [0,4194304] 0 2024-09-21T04:35:05.745 INFO:tasks.workunit.client.1.smithi053.stdout:8/314: unlink d3/cb 0 2024-09-21T04:35:05.749 INFO:tasks.workunit.client.0.smithi007.stdout:8/18: link d0/d3/f5 d0/f6 0 2024-09-21T04:35:05.750 INFO:tasks.workunit.client.1.smithi053.stdout:8/315: write d3/d5/d4c/f6d [597351,73114] 0 2024-09-21T04:35:05.751 INFO:tasks.workunit.client.0.smithi007.stdout:8/19: write d0/f1 [1290844,90325] 0 2024-09-21T04:35:05.751 INFO:tasks.workunit.client.1.smithi053.stdout:6/322: getdents d2/d1b/d3d/d49/d1a 0 2024-09-21T04:35:05.751 INFO:tasks.workunit.client.1.smithi053.stdout:2/285: creat d1/d18/f5d x:0 0 0 2024-09-21T04:35:05.751 INFO:tasks.workunit.client.1.smithi053.stdout:3/290: mkdir d16/d17/d21/d65 0 2024-09-21T04:35:05.756 INFO:tasks.workunit.client.0.smithi007.stdout:5/22: chown d2/l3 66 1 2024-09-21T04:35:05.756 INFO:tasks.workunit.client.0.smithi007.stdout:8/20: fdatasync d0/f1 0 2024-09-21T04:35:05.760 INFO:tasks.workunit.client.1.smithi053.stdout:6/323: unlink d2/d1b/d3d/d49/de/f14 0 2024-09-21T04:35:05.763 INFO:tasks.workunit.client.1.smithi053.stdout:8/316: symlink d3/d4f/l72 0 2024-09-21T04:35:05.765 INFO:tasks.workunit.client.1.smithi053.stdout:3/291: truncate d16/d17/d21/f39 38786 0 2024-09-21T04:35:05.766 INFO:tasks.workunit.client.1.smithi053.stdout:3/292: stat d16/d1b/f38 0 2024-09-21T04:35:05.766 INFO:tasks.workunit.client.1.smithi053.stdout:4/345: getdents d1/d27/d2e 0 2024-09-21T04:35:05.767 INFO:tasks.workunit.client.0.smithi007.stdout:1/22: truncate f2 2572543 0 2024-09-21T04:35:05.767 INFO:tasks.workunit.client.0.smithi007.stdout:1/23: write f3 [1333164,101503] 0 2024-09-21T04:35:05.768 INFO:tasks.workunit.client.1.smithi053.stdout:2/286: dwrite d1/d18/f2a [0,4194304] 0 2024-09-21T04:35:05.768 INFO:tasks.workunit.client.1.smithi053.stdout:6/324: creat d2/d1b/d3d/f6c x:0 0 0 2024-09-21T04:35:05.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/23: mknod d2/c7 0 2024-09-21T04:35:05.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/24: dwrite - no filename 2024-09-21T04:35:05.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/25: chown d2 26 1 2024-09-21T04:35:05.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/26: dread - no filename 2024-09-21T04:35:05.770 INFO:tasks.workunit.client.0.smithi007.stdout:8/21: mknod d0/c7 0 2024-09-21T04:35:05.773 INFO:tasks.workunit.client.0.smithi007.stdout:4/21: creat d2/d7/f8 x:0 0 0 2024-09-21T04:35:05.775 INFO:tasks.workunit.client.1.smithi053.stdout:8/317: dwrite d3/d28/f31 [0,4194304] 0 2024-09-21T04:35:05.783 INFO:tasks.workunit.client.1.smithi053.stdout:4/346: dwrite d1/dd/de/d14/d20/f30 [0,4194304] 0 2024-09-21T04:35:05.796 INFO:tasks.workunit.client.1.smithi053.stdout:3/293: creat d16/d17/d21/d65/f66 x:0 0 0 2024-09-21T04:35:05.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/11: getdents . 0 2024-09-21T04:35:05.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/12: dwrite - no filename 2024-09-21T04:35:05.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/13: link - no file 2024-09-21T04:35:05.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/14: fdatasync - no filename 2024-09-21T04:35:05.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/15: link - no file 2024-09-21T04:35:05.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/16: write - no filename 2024-09-21T04:35:05.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/17: chown . 1755112 1 2024-09-21T04:35:05.804 INFO:tasks.workunit.client.0.smithi007.stdout:4/22: symlink d2/l9 0 2024-09-21T04:35:05.804 INFO:tasks.workunit.client.0.smithi007.stdout:4/23: chown d2/d7/f8 19559 1 2024-09-21T04:35:05.807 INFO:tasks.workunit.client.0.smithi007.stdout:7/22: truncate f1 1324109 0 2024-09-21T04:35:05.810 INFO:tasks.workunit.client.1.smithi053.stdout:3/294: symlink d16/d17/l67 0 2024-09-21T04:35:05.811 INFO:tasks.workunit.client.0.smithi007.stdout:4/24: dwrite d2/f3 [0,4194304] 0 2024-09-21T04:35:05.811 INFO:tasks.workunit.client.0.smithi007.stdout:6/18: mkdir d3 0 2024-09-21T04:35:05.812 INFO:tasks.workunit.client.0.smithi007.stdout:6/19: chown d3 1723 1 2024-09-21T04:35:05.812 INFO:tasks.workunit.client.0.smithi007.stdout:6/20: dwrite - no filename 2024-09-21T04:35:05.812 INFO:tasks.workunit.client.0.smithi007.stdout:4/25: write d2/f3 [56636,36578] 0 2024-09-21T04:35:05.813 INFO:tasks.workunit.client.1.smithi053.stdout:3/295: stat d16/c44 0 2024-09-21T04:35:05.816 INFO:tasks.workunit.client.0.smithi007.stdout:7/23: mkdir d3/d6 0 2024-09-21T04:35:05.821 INFO:tasks.workunit.client.0.smithi007.stdout:4/26: dwrite d2/f3 [0,4194304] 0 2024-09-21T04:35:05.823 INFO:tasks.workunit.client.0.smithi007.stdout:5/27: getdents d2 0 2024-09-21T04:35:05.828 INFO:tasks.workunit.client.0.smithi007.stdout:4/27: creat d2/d7/fa x:0 0 0 2024-09-21T04:35:05.831 INFO:tasks.workunit.client.0.smithi007.stdout:5/28: creat d2/f8 x:0 0 0 2024-09-21T04:35:05.833 INFO:tasks.workunit.client.1.smithi053.stdout:3/296: link d16/d46/c50 d16/d17/d21/d26/d2b/c68 0 2024-09-21T04:35:05.833 INFO:tasks.workunit.client.1.smithi053.stdout:3/297: chown d16/c44 45 1 2024-09-21T04:35:05.834 INFO:tasks.workunit.client.0.smithi007.stdout:4/28: unlink d2/l9 0 2024-09-21T04:35:05.835 INFO:tasks.workunit.client.0.smithi007.stdout:5/29: rmdir d2 39 2024-09-21T04:35:05.836 INFO:tasks.workunit.client.0.smithi007.stdout:4/29: mknod d2/cb 0 2024-09-21T04:35:05.836 INFO:tasks.workunit.client.1.smithi053.stdout:3/298: dread f11 [0,4194304] 0 2024-09-21T04:35:05.837 INFO:tasks.workunit.client.0.smithi007.stdout:5/30: dread - d2/f8 zero size 2024-09-21T04:35:05.838 INFO:tasks.workunit.client.0.smithi007.stdout:4/30: mknod d2/d7/cc 0 2024-09-21T04:35:05.840 INFO:tasks.workunit.client.0.smithi007.stdout:5/31: mknod d2/c9 0 2024-09-21T04:35:05.840 INFO:tasks.workunit.client.0.smithi007.stdout:5/32: dread - d2/f8 zero size 2024-09-21T04:35:05.841 INFO:tasks.workunit.client.0.smithi007.stdout:5/33: write d2/f8 [1007873,95549] 0 2024-09-21T04:35:05.845 INFO:tasks.workunit.client.0.smithi007.stdout:5/34: creat d2/fa x:0 0 0 2024-09-21T04:35:05.846 INFO:tasks.workunit.client.1.smithi053.stdout:3/299: getdents d16/d17/d21 0 2024-09-21T04:35:05.846 INFO:tasks.workunit.client.0.smithi007.stdout:5/35: mknod d2/cb 0 2024-09-21T04:35:05.849 INFO:tasks.workunit.client.0.smithi007.stdout:5/36: creat d2/fc x:0 0 0 2024-09-21T04:35:05.849 INFO:tasks.workunit.client.1.smithi053.stdout:3/300: mknod d16/d17/d21/d26/c69 0 2024-09-21T04:35:05.850 INFO:tasks.workunit.client.0.smithi007.stdout:5/37: mkdir d2/dd 0 2024-09-21T04:35:05.851 INFO:tasks.workunit.client.1.smithi053.stdout:4/347: rmdir d1/dd 39 2024-09-21T04:35:05.851 INFO:tasks.workunit.client.0.smithi007.stdout:5/38: chown d2/fa 7 1 2024-09-21T04:35:05.852 INFO:tasks.workunit.client.0.smithi007.stdout:5/39: truncate d2/fa 992433 0 2024-09-21T04:35:05.852 INFO:tasks.workunit.client.1.smithi053.stdout:3/301: mknod d16/d17/d3c/c6a 0 2024-09-21T04:35:05.853 INFO:tasks.workunit.client.1.smithi053.stdout:4/348: stat d1/dd/de/d14/d20/l21 0 2024-09-21T04:35:05.856 INFO:tasks.workunit.client.1.smithi053.stdout:4/349: write d1/f4 [703448,97795] 0 2024-09-21T04:35:05.857 INFO:tasks.workunit.client.1.smithi053.stdout:4/350: chown d1/l5 4 1 2024-09-21T04:35:05.858 INFO:tasks.workunit.client.1.smithi053.stdout:2/287: write d1/d18/d1f/d22/f45 [2199127,162] 0 2024-09-21T04:35:05.860 INFO:tasks.workunit.client.0.smithi007.stdout:1/24: write f1 [2098614,86311] 0 2024-09-21T04:35:05.860 INFO:tasks.workunit.client.1.smithi053.stdout:2/288: chown d1/d18/d1f/d22/f32 90151658 1 2024-09-21T04:35:05.861 INFO:tasks.workunit.client.0.smithi007.stdout:8/22: write d0/d3/f5 [338156,67920] 0 2024-09-21T04:35:05.861 INFO:tasks.workunit.client.0.smithi007.stdout:1/25: write f4 [670789,122915] 0 2024-09-21T04:35:05.861 INFO:tasks.workunit.client.0.smithi007.stdout:1/26: readlink - no filename 2024-09-21T04:35:05.864 INFO:tasks.workunit.client.1.smithi053.stdout:4/351: dread d1/dd/de/d14/d20/f45 [0,4194304] 0 2024-09-21T04:35:05.867 INFO:tasks.workunit.client.1.smithi053.stdout:9/334: sync 2024-09-21T04:35:05.868 INFO:tasks.workunit.client.1.smithi053.stdout:0/248: sync 2024-09-21T04:35:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:7/24: dread f1 [0,4194304] 0 2024-09-21T04:35:05.868 INFO:tasks.workunit.client.1.smithi053.stdout:6/325: truncate d2/d1b/d3d/d49/f42 3705217 0 2024-09-21T04:35:05.869 INFO:tasks.workunit.client.1.smithi053.stdout:3/302: dread d16/d17/f2a [0,4194304] 0 2024-09-21T04:35:05.870 INFO:tasks.workunit.client.1.smithi053.stdout:4/352: dread d1/dd/de/d14/d20/f30 [0,4194304] 0 2024-09-21T04:35:05.871 INFO:tasks.workunit.client.1.smithi053.stdout:6/326: dread - d2/d1b/d1e/d2a/d4b/f68 zero size 2024-09-21T04:35:05.872 INFO:tasks.workunit.client.1.smithi053.stdout:6/327: readlink d2/d1b/d3d/d49/d1a/d2e/d38/l33 0 2024-09-21T04:35:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:1/27: dwrite f3 [0,4194304] 0 2024-09-21T04:35:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:8/23: dwrite d0/f1 [0,4194304] 0 2024-09-21T04:35:05.875 INFO:tasks.workunit.client.1.smithi053.stdout:3/303: write d16/d17/f2a [177341,39545] 0 2024-09-21T04:35:05.876 INFO:tasks.workunit.client.0.smithi007.stdout:1/28: dread f2 [0,4194304] 0 2024-09-21T04:35:05.878 INFO:tasks.workunit.client.1.smithi053.stdout:9/335: mkdir d0/d1c/d52/d6d 0 2024-09-21T04:35:05.879 INFO:tasks.workunit.client.0.smithi007.stdout:7/25: creat d3/d6/f7 x:0 0 0 2024-09-21T04:35:05.880 INFO:tasks.workunit.client.1.smithi053.stdout:6/328: symlink d2/d1d/l6d 0 2024-09-21T04:35:05.881 INFO:tasks.workunit.client.0.smithi007.stdout:8/24: mkdir d0/d3/d8 0 2024-09-21T04:35:05.887 INFO:tasks.workunit.client.1.smithi053.stdout:6/329: dwrite d2/fd [0,4194304] 0 2024-09-21T04:35:05.891 INFO:tasks.workunit.client.0.smithi007.stdout:8/25: rename d0/f1 to d0/d3/d8/f9 0 2024-09-21T04:35:05.893 INFO:tasks.workunit.client.0.smithi007.stdout:1/29: dwrite f1 [0,4194304] 0 2024-09-21T04:35:05.894 INFO:tasks.workunit.client.0.smithi007.stdout:7/26: link d3/f5 d3/d6/f8 0 2024-09-21T04:35:05.895 INFO:tasks.workunit.client.1.smithi053.stdout:9/336: dwrite d0/d2/f13 [4194304,4194304] 0 2024-09-21T04:35:05.899 INFO:tasks.workunit.client.1.smithi053.stdout:3/304: creat d16/d46/d57/f6b x:0 0 0 2024-09-21T04:35:05.902 INFO:tasks.workunit.client.1.smithi053.stdout:6/330: creat d2/d1b/d1e/d27/f6e x:0 0 0 2024-09-21T04:35:05.903 INFO:tasks.workunit.client.0.smithi007.stdout:8/26: rmdir d0 39 2024-09-21T04:35:05.903 INFO:tasks.workunit.client.0.smithi007.stdout:1/30: dwrite f1 [0,4194304] 0 2024-09-21T04:35:05.905 INFO:tasks.workunit.client.0.smithi007.stdout:4/31: truncate f0 2516313 0 2024-09-21T04:35:05.907 INFO:tasks.workunit.client.1.smithi053.stdout:0/249: getdents d5/de/d1c 0 2024-09-21T04:35:05.908 INFO:tasks.workunit.client.1.smithi053.stdout:3/305: dwrite d16/d1b/f38 [0,4194304] 0 2024-09-21T04:35:05.908 INFO:tasks.workunit.client.1.smithi053.stdout:1/393: sync 2024-09-21T04:35:05.908 INFO:tasks.workunit.client.1.smithi053.stdout:7/284: sync 2024-09-21T04:35:05.908 INFO:tasks.workunit.client.1.smithi053.stdout:5/322: sync 2024-09-21T04:35:05.910 INFO:tasks.workunit.client.1.smithi053.stdout:0/250: chown l2 2 1 2024-09-21T04:35:05.911 INFO:tasks.workunit.client.1.smithi053.stdout:3/306: stat f9 0 2024-09-21T04:35:05.911 INFO:tasks.workunit.client.0.smithi007.stdout:8/27: write d0/f6 [1028512,30496] 0 2024-09-21T04:35:05.912 INFO:tasks.workunit.client.1.smithi053.stdout:3/307: dread - d16/f56 zero size 2024-09-21T04:35:05.916 INFO:tasks.workunit.client.0.smithi007.stdout:4/32: creat d2/fd x:0 0 0 2024-09-21T04:35:05.916 INFO:tasks.workunit.client.0.smithi007.stdout:7/27: dwrite d3/f5 [0,4194304] 0 2024-09-21T04:35:05.917 INFO:tasks.workunit.client.0.smithi007.stdout:4/33: dread - d2/d7/f8 zero size 2024-09-21T04:35:05.918 INFO:tasks.workunit.client.0.smithi007.stdout:5/40: getdents d2 0 2024-09-21T04:35:05.919 INFO:tasks.workunit.client.1.smithi053.stdout:9/337: mknod d0/df/c6e 0 2024-09-21T04:35:05.920 INFO:tasks.workunit.client.0.smithi007.stdout:5/41: fdatasync d2/fc 0 2024-09-21T04:35:05.922 INFO:tasks.workunit.client.0.smithi007.stdout:5/42: write d2/f8 [40144,77693] 0 2024-09-21T04:35:05.922 INFO:tasks.workunit.client.0.smithi007.stdout:5/43: dread - d2/fc zero size 2024-09-21T04:35:05.923 INFO:tasks.workunit.client.1.smithi053.stdout:1/394: dwrite d7/d9/d43/f6d [0,4194304] 0 2024-09-21T04:35:05.924 INFO:tasks.workunit.client.0.smithi007.stdout:1/31: creat d5/d6/f7 x:0 0 0 2024-09-21T04:35:05.929 INFO:tasks.workunit.client.0.smithi007.stdout:7/28: unlink f1 0 2024-09-21T04:35:05.931 INFO:tasks.workunit.client.1.smithi053.stdout:3/308: readlink d16/l3b 0 2024-09-21T04:35:05.933 INFO:tasks.workunit.client.0.smithi007.stdout:4/34: rename d2/cb to d2/d7/ce 0 2024-09-21T04:35:05.935 INFO:tasks.workunit.client.0.smithi007.stdout:8/28: dwrite d0/d3/d8/f9 [4194304,4194304] 0 2024-09-21T04:35:05.935 INFO:tasks.workunit.client.1.smithi053.stdout:8/318: rmdir d3/d5 39 2024-09-21T04:35:05.939 INFO:tasks.workunit.client.1.smithi053.stdout:6/331: dread d2/d1b/d3d/d49/d1a/f26 [0,4194304] 0 2024-09-21T04:35:05.940 INFO:tasks.workunit.client.1.smithi053.stdout:7/285: dread d0/d6/f18 [0,4194304] 0 2024-09-21T04:35:05.941 INFO:tasks.workunit.client.1.smithi053.stdout:2/289: truncate d1/d18/d1f/d27/d41/f42 2992667 0 2024-09-21T04:35:05.943 INFO:tasks.workunit.client.1.smithi053.stdout:7/286: write d0/d6/f29 [206295,93363] 0 2024-09-21T04:35:05.944 INFO:tasks.workunit.client.0.smithi007.stdout:4/35: dwrite d2/fd [0,4194304] 0 2024-09-21T04:35:05.944 INFO:tasks.workunit.client.0.smithi007.stdout:7/29: mknod d3/d6/c9 0 2024-09-21T04:35:05.945 INFO:tasks.workunit.client.0.smithi007.stdout:7/30: stat d3/d6/f8 0 2024-09-21T04:35:05.946 INFO:tasks.workunit.client.1.smithi053.stdout:0/251: getdents d5/dd/d45 0 2024-09-21T04:35:05.947 INFO:tasks.workunit.client.1.smithi053.stdout:7/287: truncate d0/d6/ff 653556 0 2024-09-21T04:35:05.947 INFO:tasks.workunit.client.0.smithi007.stdout:5/44: symlink d2/dd/le 0 2024-09-21T04:35:05.950 INFO:tasks.workunit.client.1.smithi053.stdout:0/252: write d5/de/d1c/f29 [729114,34465] 0 2024-09-21T04:35:05.952 INFO:tasks.workunit.client.0.smithi007.stdout:8/29: rename d0/d3/f5 to d0/d3/d8/fa 0 2024-09-21T04:35:05.953 INFO:tasks.workunit.client.1.smithi053.stdout:9/338: dwrite d0/d28/f38 [0,4194304] 0 2024-09-21T04:35:05.954 INFO:tasks.workunit.client.1.smithi053.stdout:0/253: chown d5/de/f3e 1795467 1 2024-09-21T04:35:05.956 INFO:tasks.workunit.client.1.smithi053.stdout:2/290: dwrite d1/d18/f49 [0,4194304] 0 2024-09-21T04:35:05.956 INFO:tasks.workunit.client.1.smithi053.stdout:3/309: fdatasync f11 0 2024-09-21T04:35:05.958 INFO:tasks.workunit.client.1.smithi053.stdout:0/254: write d5/f15 [562835,77133] 0 2024-09-21T04:35:05.958 INFO:tasks.workunit.client.0.smithi007.stdout:1/32: link f1 d5/d6/f8 0 2024-09-21T04:35:05.959 INFO:tasks.workunit.client.1.smithi053.stdout:7/288: dread d0/d6/ff [0,4194304] 0 2024-09-21T04:35:05.962 INFO:tasks.workunit.client.0.smithi007.stdout:8/30: dwrite d0/d3/d8/f9 [4194304,4194304] 0 2024-09-21T04:35:05.962 INFO:tasks.workunit.client.0.smithi007.stdout:4/36: creat d2/d7/ff x:0 0 0 2024-09-21T04:35:05.963 INFO:tasks.workunit.client.1.smithi053.stdout:7/289: dread - d0/d2b/d36/d48/f53 zero size 2024-09-21T04:35:05.965 INFO:tasks.workunit.client.0.smithi007.stdout:7/31: creat d3/d6/fa x:0 0 0 2024-09-21T04:35:05.965 INFO:tasks.workunit.client.1.smithi053.stdout:4/353: dwrite d1/fc [0,4194304] 0 2024-09-21T04:35:05.966 INFO:tasks.workunit.client.0.smithi007.stdout:7/32: truncate d3/f5 4833581 0 2024-09-21T04:35:05.967 INFO:tasks.workunit.client.1.smithi053.stdout:7/290: dread d0/d6/f42 [0,4194304] 0 2024-09-21T04:35:05.968 INFO:tasks.workunit.client.1.smithi053.stdout:5/323: creat d0/d8/f69 x:0 0 0 2024-09-21T04:35:05.977 INFO:tasks.workunit.client.0.smithi007.stdout:7/33: dwrite d3/d6/f8 [0,4194304] 0 2024-09-21T04:35:05.978 INFO:tasks.workunit.client.0.smithi007.stdout:1/33: write f1 [610715,6628] 0 2024-09-21T04:35:05.982 INFO:tasks.workunit.client.0.smithi007.stdout:1/34: fdatasync d5/d6/f8 0 2024-09-21T04:35:05.987 INFO:tasks.workunit.client.1.smithi053.stdout:9/339: mkdir d0/d1c/d52/d6f 0 2024-09-21T04:35:05.990 INFO:tasks.workunit.client.0.smithi007.stdout:8/31: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:35:05.990 INFO:tasks.workunit.client.1.smithi053.stdout:8/319: mkdir d3/d5/d38/d64/d73 0 2024-09-21T04:35:05.991 INFO:tasks.workunit.client.1.smithi053.stdout:8/320: dread - d3/d28/f68 zero size 2024-09-21T04:35:05.992 INFO:tasks.workunit.client.1.smithi053.stdout:2/291: mknod d1/d18/d1f/d27/d41/d4b/c5e 0 2024-09-21T04:35:05.994 INFO:tasks.workunit.client.0.smithi007.stdout:7/34: rename d3/c4 to d3/d6/cb 0 2024-09-21T04:35:05.998 INFO:tasks.workunit.client.0.smithi007.stdout:7/35: fdatasync d3/d6/f8 0 2024-09-21T04:35:06.000 INFO:tasks.workunit.client.1.smithi053.stdout:2/292: dwrite d1/d18/f5b [0,4194304] 0 2024-09-21T04:35:06.000 INFO:tasks.workunit.client.0.smithi007.stdout:1/35: mknod d5/d6/c9 0 2024-09-21T04:35:06.003 INFO:tasks.workunit.client.0.smithi007.stdout:8/32: rmdir d0/d3 39 2024-09-21T04:35:06.005 INFO:tasks.workunit.client.1.smithi053.stdout:7/291: mkdir d0/d2b/d36/d48/d5c 0 2024-09-21T04:35:06.005 INFO:tasks.workunit.client.1.smithi053.stdout:7/292: chown d0/f59 232853 1 2024-09-21T04:35:06.006 INFO:tasks.workunit.client.1.smithi053.stdout:7/293: read - d0/d6/f31 zero size 2024-09-21T04:35:06.009 INFO:tasks.workunit.client.0.smithi007.stdout:7/36: mknod d3/d6/cc 0 2024-09-21T04:35:06.009 INFO:tasks.workunit.client.0.smithi007.stdout:7/37: stat d3/d6/cc 0 2024-09-21T04:35:06.009 INFO:tasks.workunit.client.0.smithi007.stdout:7/38: write d3/d6/fa [591085,26937] 0 2024-09-21T04:35:06.010 INFO:tasks.workunit.client.0.smithi007.stdout:1/36: creat d5/d6/fa x:0 0 0 2024-09-21T04:35:06.011 INFO:tasks.workunit.client.0.smithi007.stdout:7/39: write d3/f5 [3208746,83202] 0 2024-09-21T04:35:06.012 INFO:tasks.workunit.client.0.smithi007.stdout:7/40: fdatasync d3/d6/fa 0 2024-09-21T04:35:06.013 INFO:tasks.workunit.client.0.smithi007.stdout:7/41: dread - d3/d6/f7 zero size 2024-09-21T04:35:06.013 INFO:tasks.workunit.client.0.smithi007.stdout:4/37: fsync f0 0 2024-09-21T04:35:06.021 INFO:tasks.workunit.client.1.smithi053.stdout:9/340: rename d0/d2/d9/l12 to d0/d1c/d52/d6f/l70 0 2024-09-21T04:35:06.021 INFO:tasks.workunit.client.1.smithi053.stdout:9/341: dread - d0/d2/f55 zero size 2024-09-21T04:35:06.022 INFO:tasks.workunit.client.1.smithi053.stdout:9/342: readlink d0/d2/d9/d1f/l40 0 2024-09-21T04:35:06.022 INFO:tasks.workunit.client.1.smithi053.stdout:8/321: rmdir d3/d5/d4b/d5c 39 2024-09-21T04:35:06.023 INFO:tasks.workunit.client.1.smithi053.stdout:6/332: write d2/d1b/f3a [3333151,64111] 0 2024-09-21T04:35:06.025 INFO:tasks.workunit.client.1.smithi053.stdout:6/333: write d2/d1b/d3d/d49/d1a/f48 [301432,117168] 0 2024-09-21T04:35:06.026 INFO:tasks.workunit.client.1.smithi053.stdout:2/293: symlink d1/d18/d1f/d22/l5f 0 2024-09-21T04:35:06.026 INFO:tasks.workunit.client.1.smithi053.stdout:2/294: stat d1/d18/d1f/d27/d41/f4c 0 2024-09-21T04:35:06.028 INFO:tasks.workunit.client.1.smithi053.stdout:7/294: mkdir d0/d2b/d36/d48/d5d 0 2024-09-21T04:35:06.028 INFO:tasks.workunit.client.1.smithi053.stdout:4/354: getdents d1/d27/d33/d41/d63 0 2024-09-21T04:35:06.029 INFO:tasks.workunit.client.1.smithi053.stdout:4/355: chown d1/l6 782065 1 2024-09-21T04:35:06.030 INFO:tasks.workunit.client.1.smithi053.stdout:1/395: getdents d7/d9/da/d40 0 2024-09-21T04:35:06.031 INFO:tasks.workunit.client.1.smithi053.stdout:3/310: rmdir d16/d17/d21/d26/d5f 0 2024-09-21T04:35:06.034 INFO:tasks.workunit.client.1.smithi053.stdout:8/322: creat d3/d5/d4c/d53/d66/f74 x:0 0 0 2024-09-21T04:35:06.036 INFO:tasks.workunit.client.1.smithi053.stdout:6/334: creat d2/d1b/f6f x:0 0 0 2024-09-21T04:35:06.037 INFO:tasks.workunit.client.1.smithi053.stdout:5/324: creat d0/f6a x:0 0 0 2024-09-21T04:35:06.037 INFO:tasks.workunit.client.1.smithi053.stdout:7/295: dwrite d0/f44 [0,4194304] 0 2024-09-21T04:35:06.038 INFO:tasks.workunit.client.1.smithi053.stdout:2/295: mkdir d1/d60 0 2024-09-21T04:35:06.039 INFO:tasks.workunit.client.1.smithi053.stdout:7/296: readlink d0/le 0 2024-09-21T04:35:06.042 INFO:tasks.workunit.client.1.smithi053.stdout:9/343: rename d0/df/c6a to d0/d2/c71 0 2024-09-21T04:35:06.043 INFO:tasks.workunit.client.1.smithi053.stdout:4/356: dwrite d1/d27/f37 [0,4194304] 0 2024-09-21T04:35:06.043 INFO:tasks.workunit.client.1.smithi053.stdout:2/296: read d1/d18/d1f/d22/f32 [151162,33843] 0 2024-09-21T04:35:06.044 INFO:tasks.workunit.client.1.smithi053.stdout:3/311: unlink d16/d17/d21/d26/c28 0 2024-09-21T04:35:06.045 INFO:tasks.workunit.client.1.smithi053.stdout:4/357: fdatasync d1/dd/de/d14/f5d 0 2024-09-21T04:35:06.045 INFO:tasks.workunit.client.1.smithi053.stdout:0/255: link d5/d6/d21/c2c d5/dd/c51 0 2024-09-21T04:35:06.048 INFO:tasks.workunit.client.1.smithi053.stdout:0/256: write d5/dd/f34 [1221531,103382] 0 2024-09-21T04:35:06.049 INFO:tasks.workunit.client.1.smithi053.stdout:8/323: symlink d3/d5/d38/d64/l75 0 2024-09-21T04:35:06.050 INFO:tasks.workunit.client.1.smithi053.stdout:5/325: rmdir d0/dd/d48 39 2024-09-21T04:35:06.053 INFO:tasks.workunit.client.1.smithi053.stdout:5/326: chown d0/dd/d5c 2217273 1 2024-09-21T04:35:06.055 INFO:tasks.workunit.client.1.smithi053.stdout:3/312: dwrite fa [0,4194304] 0 2024-09-21T04:35:06.057 INFO:tasks.workunit.client.1.smithi053.stdout:9/344: rename d0/d1c/f44 to d0/df/f72 0 2024-09-21T04:35:06.063 INFO:tasks.workunit.client.1.smithi053.stdout:2/297: mkdir d1/d18/d1f/d22/d2d/d61 0 2024-09-21T04:35:06.063 INFO:tasks.workunit.client.1.smithi053.stdout:1/396: write d7/d9/da/f2f [1220449,19306] 0 2024-09-21T04:35:06.065 INFO:tasks.workunit.client.1.smithi053.stdout:4/358: symlink d1/d2c/l6a 0 2024-09-21T04:35:06.067 INFO:tasks.workunit.client.1.smithi053.stdout:0/257: rmdir d5/d10/d4c 39 2024-09-21T04:35:06.068 INFO:tasks.workunit.client.1.smithi053.stdout:0/258: write d5/f15 [30991,96666] 0 2024-09-21T04:35:06.069 INFO:tasks.workunit.client.1.smithi053.stdout:0/259: chown d5/f15 85013066 1 2024-09-21T04:35:06.071 INFO:tasks.workunit.client.1.smithi053.stdout:7/297: getdents d0/d2b/d36/d48/d5d 0 2024-09-21T04:35:06.072 INFO:tasks.workunit.client.1.smithi053.stdout:8/324: dread d3/df/d14/d16/f23 [4194304,4194304] 0 2024-09-21T04:35:06.073 INFO:tasks.workunit.client.1.smithi053.stdout:3/313: creat d16/d17/d21/d4e/f6c x:0 0 0 2024-09-21T04:35:06.078 INFO:tasks.workunit.client.1.smithi053.stdout:6/335: dwrite d2/d1b/d3d/d49/d1a/d2e/f35 [0,4194304] 0 2024-09-21T04:35:06.079 INFO:tasks.workunit.client.1.smithi053.stdout:2/298: creat d1/d18/d1f/d27/d58/d5c/f62 x:0 0 0 2024-09-21T04:35:06.080 INFO:tasks.workunit.client.1.smithi053.stdout:2/299: chown d1/d18/d1f/d22/f33 627872 1 2024-09-21T04:35:06.081 INFO:tasks.workunit.client.1.smithi053.stdout:8/325: mknod d3/d5/d4c/d53/c76 0 2024-09-21T04:35:06.082 INFO:tasks.workunit.client.1.smithi053.stdout:3/314: creat d16/d17/d21/f6d x:0 0 0 2024-09-21T04:35:06.083 INFO:tasks.workunit.client.1.smithi053.stdout:0/260: creat d5/d10/d4c/f52 x:0 0 0 2024-09-21T04:35:06.084 INFO:tasks.workunit.client.1.smithi053.stdout:5/327: creat d0/dd/d30/d51/d52/f6b x:0 0 0 2024-09-21T04:35:06.084 INFO:tasks.workunit.client.1.smithi053.stdout:3/315: chown d16/d17/d3c 42951085 1 2024-09-21T04:35:06.086 INFO:tasks.workunit.client.1.smithi053.stdout:6/336: dwrite d2/d1b/d1e/f4f [4194304,4194304] 0 2024-09-21T04:35:06.087 INFO:tasks.workunit.client.1.smithi053.stdout:4/359: rename d1/dd/de/d14/d20/d5b/f60 to d1/dd/f6b 0 2024-09-21T04:35:06.090 INFO:tasks.workunit.client.1.smithi053.stdout:2/300: mknod d1/d18/d1f/d27/d58/c63 0 2024-09-21T04:35:06.091 INFO:tasks.workunit.client.1.smithi053.stdout:2/301: dread - d1/d18/f5d zero size 2024-09-21T04:35:06.091 INFO:tasks.workunit.client.1.smithi053.stdout:8/326: fsync d3/d5/d38/f3d 0 2024-09-21T04:35:06.092 INFO:tasks.workunit.client.1.smithi053.stdout:5/328: rmdir d0/d8 39 2024-09-21T04:35:06.093 INFO:tasks.workunit.client.1.smithi053.stdout:0/261: mkdir d5/d6/d53 0 2024-09-21T04:35:06.095 INFO:tasks.workunit.client.1.smithi053.stdout:3/316: creat d16/d17/d21/d65/f6e x:0 0 0 2024-09-21T04:35:06.096 INFO:tasks.workunit.client.1.smithi053.stdout:6/337: rmdir d2/d1b/d3d/d49/d1a/d2e/d38 39 2024-09-21T04:35:06.097 INFO:tasks.workunit.client.1.smithi053.stdout:9/345: link d0/l3 d0/d1c/d25/l73 0 2024-09-21T04:35:06.099 INFO:tasks.workunit.client.1.smithi053.stdout:5/329: fdatasync d0/d8/f1d 0 2024-09-21T04:35:06.100 INFO:tasks.workunit.client.1.smithi053.stdout:0/262: symlink d5/de/d42/l54 0 2024-09-21T04:35:06.102 INFO:tasks.workunit.client.1.smithi053.stdout:2/302: dwrite d1/fb [0,4194304] 0 2024-09-21T04:35:06.102 INFO:tasks.workunit.client.1.smithi053.stdout:6/338: rename d2/l22 to d2/d1b/d3d/d49/de/l70 0 2024-09-21T04:35:06.103 INFO:tasks.workunit.client.1.smithi053.stdout:8/327: creat d3/d5/f77 x:0 0 0 2024-09-21T04:35:06.105 INFO:tasks.workunit.client.1.smithi053.stdout:3/317: dread d16/d1b/f27 [0,4194304] 0 2024-09-21T04:35:06.107 INFO:tasks.workunit.client.1.smithi053.stdout:0/263: symlink d5/dd/d45/l55 0 2024-09-21T04:35:06.107 INFO:tasks.workunit.client.1.smithi053.stdout:6/339: rmdir d2/d1b/d1e/d27 39 2024-09-21T04:35:06.109 INFO:tasks.workunit.client.1.smithi053.stdout:9/346: rename d0/d1c/d25/c29 to d0/d28/d63/c74 0 2024-09-21T04:35:06.110 INFO:tasks.workunit.client.1.smithi053.stdout:4/360: dread d1/dd/de/f2a [4194304,4194304] 0 2024-09-21T04:35:06.110 INFO:tasks.workunit.client.1.smithi053.stdout:3/318: symlink d16/d17/d21/d42/l6f 0 2024-09-21T04:35:06.113 INFO:tasks.workunit.client.1.smithi053.stdout:6/340: rmdir d2/d1b/d3d/d49/d1a/d2e/d5b 39 2024-09-21T04:35:06.113 INFO:tasks.workunit.client.1.smithi053.stdout:2/303: dwrite d1/d18/f5d [0,4194304] 0 2024-09-21T04:35:06.114 INFO:tasks.workunit.client.1.smithi053.stdout:8/328: creat d3/d5/d21/f78 x:0 0 0 2024-09-21T04:35:06.115 INFO:tasks.workunit.client.1.smithi053.stdout:6/341: chown d2/f59 672997231 1 2024-09-21T04:35:06.115 INFO:tasks.workunit.client.1.smithi053.stdout:4/361: write d1/dd/de/f2b [1485568,35254] 0 2024-09-21T04:35:06.115 INFO:tasks.workunit.client.1.smithi053.stdout:0/264: truncate d5/de/d1c/f41 210884 0 2024-09-21T04:35:06.117 INFO:tasks.workunit.client.1.smithi053.stdout:8/329: write d3/d5/d38/f47 [745059,125017] 0 2024-09-21T04:35:06.118 INFO:tasks.workunit.client.1.smithi053.stdout:3/319: creat d16/d17/d21/d26/d49/d4d/f70 x:0 0 0 2024-09-21T04:35:06.118 INFO:tasks.workunit.client.1.smithi053.stdout:2/304: chown d1/d18/c44 346998 1 2024-09-21T04:35:06.120 INFO:tasks.workunit.client.1.smithi053.stdout:9/347: mknod d0/d6c/c75 0 2024-09-21T04:35:06.120 INFO:tasks.workunit.client.1.smithi053.stdout:0/265: stat d5/d6/f1b 0 2024-09-21T04:35:06.122 INFO:tasks.workunit.client.1.smithi053.stdout:8/330: write d3/d5/f77 [383914,37676] 0 2024-09-21T04:35:06.124 INFO:tasks.workunit.client.1.smithi053.stdout:4/362: rename d1 to d1/dd/d6c 22 2024-09-21T04:35:06.126 INFO:tasks.workunit.client.0.smithi007.stdout:5/45: truncate d2/fa 37534 0 2024-09-21T04:35:06.127 INFO:tasks.workunit.client.0.smithi007.stdout:5/46: readlink d2/l3 0 2024-09-21T04:35:06.127 INFO:tasks.workunit.client.1.smithi053.stdout:2/305: rmdir d1/d60 0 2024-09-21T04:35:06.128 INFO:tasks.workunit.client.0.smithi007.stdout:7/42: chown d3/d6/cb 57337891 1 2024-09-21T04:35:06.128 INFO:tasks.workunit.client.1.smithi053.stdout:8/331: read d3/df/d14/d16/f1d [898843,48707] 0 2024-09-21T04:35:06.129 INFO:tasks.workunit.client.0.smithi007.stdout:7/43: fdatasync d3/d6/fa 0 2024-09-21T04:35:06.129 INFO:tasks.workunit.client.0.smithi007.stdout:4/38: creat d2/d7/f10 x:0 0 0 2024-09-21T04:35:06.129 INFO:tasks.workunit.client.1.smithi053.stdout:4/363: dread - d1/d27/d2e/f67 zero size 2024-09-21T04:35:06.131 INFO:tasks.workunit.client.0.smithi007.stdout:1/37: truncate d5/d6/f8 2550593 0 2024-09-21T04:35:06.131 INFO:tasks.workunit.client.0.smithi007.stdout:4/39: dread - d2/d7/f8 zero size 2024-09-21T04:35:06.133 INFO:tasks.workunit.client.1.smithi053.stdout:6/342: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:35:06.133 INFO:tasks.workunit.client.1.smithi053.stdout:0/266: creat d5/d10/f56 x:0 0 0 2024-09-21T04:35:06.133 INFO:tasks.workunit.client.0.smithi007.stdout:8/33: creat d0/d3/d8/fb x:0 0 0 2024-09-21T04:35:06.134 INFO:tasks.workunit.client.0.smithi007.stdout:4/40: chown d2/f6 918588 1 2024-09-21T04:35:06.134 INFO:tasks.workunit.client.0.smithi007.stdout:8/34: stat d0/c7 0 2024-09-21T04:35:06.135 INFO:tasks.workunit.client.0.smithi007.stdout:4/41: fsync d2/f6 0 2024-09-21T04:35:06.136 INFO:tasks.workunit.client.0.smithi007.stdout:5/47: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:35:06.137 INFO:tasks.workunit.client.1.smithi053.stdout:8/332: dread d3/d39/f2f [0,4194304] 0 2024-09-21T04:35:06.138 INFO:tasks.workunit.client.1.smithi053.stdout:3/320: dwrite d16/d17/d21/d65/f66 [0,4194304] 0 2024-09-21T04:35:06.139 INFO:tasks.workunit.client.1.smithi053.stdout:0/267: read - d5/d6/d21/f3f zero size 2024-09-21T04:35:06.140 INFO:tasks.workunit.client.1.smithi053.stdout:9/348: dwrite d0/d1c/d52/d53/f5e [0,4194304] 0 2024-09-21T04:35:06.141 INFO:tasks.workunit.client.0.smithi007.stdout:7/44: mknod d3/d6/cd 0 2024-09-21T04:35:06.142 INFO:tasks.workunit.client.0.smithi007.stdout:7/45: chown d3/d6/cd 535893482 1 2024-09-21T04:35:06.144 INFO:tasks.workunit.client.1.smithi053.stdout:3/321: chown d16/d17/d21/d26/d2b/l2d 15825531 1 2024-09-21T04:35:06.147 INFO:tasks.workunit.client.0.smithi007.stdout:4/42: symlink d2/d7/l11 0 2024-09-21T04:35:06.150 INFO:tasks.workunit.client.1.smithi053.stdout:8/333: dwrite d3/df/d49/f55 [0,4194304] 0 2024-09-21T04:35:06.151 INFO:tasks.workunit.client.0.smithi007.stdout:1/38: fsync d5/d6/f8 0 2024-09-21T04:35:06.151 INFO:tasks.workunit.client.1.smithi053.stdout:4/364: dread d1/d27/d2e/d46/f54 [0,4194304] 0 2024-09-21T04:35:06.154 INFO:tasks.workunit.client.0.smithi007.stdout:7/46: symlink d3/le 0 2024-09-21T04:35:06.161 INFO:tasks.workunit.client.1.smithi053.stdout:4/365: dread d1/d27/f37 [0,4194304] 0 2024-09-21T04:35:06.161 INFO:tasks.workunit.client.1.smithi053.stdout:8/334: dwrite d3/df/d49/f55 [0,4194304] 0 2024-09-21T04:35:06.167 INFO:tasks.workunit.client.0.smithi007.stdout:7/47: dread - d3/d6/f7 zero size 2024-09-21T04:35:06.167 INFO:tasks.workunit.client.0.smithi007.stdout:1/39: mknod d5/cb 0 2024-09-21T04:35:06.168 INFO:tasks.workunit.client.1.smithi053.stdout:6/343: symlink d2/d1b/d3d/d49/d1a/d2e/d38/l71 0 2024-09-21T04:35:06.170 INFO:tasks.workunit.client.1.smithi053.stdout:6/344: chown d2/d1b/d1e/f4f 724912215 1 2024-09-21T04:35:06.170 INFO:tasks.workunit.client.1.smithi053.stdout:0/268: mknod d5/dd/d50/c57 0 2024-09-21T04:35:06.171 INFO:tasks.workunit.client.0.smithi007.stdout:1/40: dread f3 [0,4194304] 0 2024-09-21T04:35:06.171 INFO:tasks.workunit.client.1.smithi053.stdout:9/349: mkdir d0/d76 0 2024-09-21T04:35:06.172 INFO:tasks.workunit.client.0.smithi007.stdout:1/41: chown d5/cb 339 1 2024-09-21T04:35:06.173 INFO:tasks.workunit.client.0.smithi007.stdout:1/42: write f4 [62982,34824] 0 2024-09-21T04:35:06.174 INFO:tasks.workunit.client.1.smithi053.stdout:4/366: creat d1/dd/de/f6d x:0 0 0 2024-09-21T04:35:06.175 INFO:tasks.workunit.client.1.smithi053.stdout:8/335: write d3/df/d14/d16/f23 [3878778,8056] 0 2024-09-21T04:35:06.176 INFO:tasks.workunit.client.1.smithi053.stdout:3/322: mkdir d16/d17/d21/d26/d71 0 2024-09-21T04:35:06.179 INFO:tasks.workunit.client.1.smithi053.stdout:2/306: rename d1/d18/f5d to d1/d18/d1f/d27/d41/f64 0 2024-09-21T04:35:06.180 INFO:tasks.workunit.client.1.smithi053.stdout:8/336: dread d3/df/d14/d16/f1d [0,4194304] 0 2024-09-21T04:35:06.180 INFO:tasks.workunit.client.1.smithi053.stdout:2/307: write d1/d18/d1f/f24 [786669,34527] 0 2024-09-21T04:35:06.181 INFO:tasks.workunit.client.1.smithi053.stdout:2/308: fsync d1/f3f 0 2024-09-21T04:35:06.188 INFO:tasks.workunit.client.1.smithi053.stdout:8/337: dwrite d3/d5/f19 [8388608,4194304] 0 2024-09-21T04:35:06.189 INFO:tasks.workunit.client.1.smithi053.stdout:9/350: truncate d0/d2/d9/f19 3269359 0 2024-09-21T04:35:06.193 INFO:tasks.workunit.client.1.smithi053.stdout:0/269: rename d5/d6/f16 to d5/dd/f58 0 2024-09-21T04:35:06.194 INFO:tasks.workunit.client.1.smithi053.stdout:0/270: fdatasync d5/d10/d4c/f52 0 2024-09-21T04:35:06.195 INFO:tasks.workunit.client.1.smithi053.stdout:0/271: chown d5/d6/d21/c2d 7365120 1 2024-09-21T04:35:06.196 INFO:tasks.workunit.client.1.smithi053.stdout:9/351: chown d0/f4 7599332 1 2024-09-21T04:35:06.197 INFO:tasks.workunit.client.1.smithi053.stdout:9/352: stat d0/d2/c57 0 2024-09-21T04:35:06.199 INFO:tasks.workunit.client.1.smithi053.stdout:6/345: rename d2/d1b/d3d/d49/d1a/f48 to d2/d1b/d3d/f72 0 2024-09-21T04:35:06.201 INFO:tasks.workunit.client.1.smithi053.stdout:0/272: chown d5/d10/d46/c4b 2733523 1 2024-09-21T04:35:06.206 INFO:tasks.workunit.client.1.smithi053.stdout:6/346: creat d2/d1b/d3d/d49/de/f73 x:0 0 0 2024-09-21T04:35:06.207 INFO:tasks.workunit.client.1.smithi053.stdout:4/367: write d1/d2c/f40 [7702500,128990] 0 2024-09-21T04:35:06.208 INFO:tasks.workunit.client.1.smithi053.stdout:0/273: symlink d5/d10/d46/l59 0 2024-09-21T04:35:06.209 INFO:tasks.workunit.client.1.smithi053.stdout:0/274: dread - d5/d6/d21/f33 zero size 2024-09-21T04:35:06.209 INFO:tasks.workunit.client.1.smithi053.stdout:3/323: rename d16/d17/c20 to d16/d17/d21/d4e/c72 0 2024-09-21T04:35:06.210 INFO:tasks.workunit.client.1.smithi053.stdout:6/347: creat d2/d1b/d3d/d49/d5c/f74 x:0 0 0 2024-09-21T04:35:06.211 INFO:tasks.workunit.client.1.smithi053.stdout:8/338: getdents d3/d5/d4b 0 2024-09-21T04:35:06.211 INFO:tasks.workunit.client.1.smithi053.stdout:8/339: write d3/f59 [943657,20291] 0 2024-09-21T04:35:06.212 INFO:tasks.workunit.client.1.smithi053.stdout:4/368: creat d1/dd/de/d14/d20/f6e x:0 0 0 2024-09-21T04:35:06.213 INFO:tasks.workunit.client.1.smithi053.stdout:8/340: write d3/d28/f31 [248666,119733] 0 2024-09-21T04:35:06.213 INFO:tasks.workunit.client.1.smithi053.stdout:4/369: write d1/f42 [255511,14760] 0 2024-09-21T04:35:06.214 INFO:tasks.workunit.client.1.smithi053.stdout:2/309: rename d1/d18/d1f/f24 to d1/d18/d1f/d22/f65 0 2024-09-21T04:35:06.220 INFO:tasks.workunit.client.1.smithi053.stdout:8/341: creat d3/d28/f79 x:0 0 0 2024-09-21T04:35:06.221 INFO:tasks.workunit.client.1.smithi053.stdout:8/342: read d3/d5/d38/f4e [3331719,76400] 0 2024-09-21T04:35:06.223 INFO:tasks.workunit.client.1.smithi053.stdout:0/275: creat d5/d47/f5a x:0 0 0 2024-09-21T04:35:06.224 INFO:tasks.workunit.client.1.smithi053.stdout:4/370: creat d1/dd/de/d14/d20/f6f x:0 0 0 2024-09-21T04:35:06.224 INFO:tasks.workunit.client.1.smithi053.stdout:4/371: chown d1/dd/de/d14 53828 1 2024-09-21T04:35:06.228 INFO:tasks.workunit.client.1.smithi053.stdout:3/324: rename d16/d46 to d16/d17/d21/d26/d71/d73 0 2024-09-21T04:35:06.231 INFO:tasks.workunit.client.1.smithi053.stdout:8/343: unlink d3/d28/c3b 0 2024-09-21T04:35:06.240 INFO:tasks.workunit.client.1.smithi053.stdout:4/372: rename d1/d27/d53/c5e to d1/d2c/c70 0 2024-09-21T04:35:06.241 INFO:tasks.workunit.client.1.smithi053.stdout:4/373: write d1/d27/d33/f65 [1933861,107878] 0 2024-09-21T04:35:06.242 INFO:tasks.workunit.client.1.smithi053.stdout:0/276: creat d5/de/f5b x:0 0 0 2024-09-21T04:35:06.245 INFO:tasks.workunit.client.1.smithi053.stdout:2/310: write d1/d18/d1f/d27/d41/f4c [398301,15387] 0 2024-09-21T04:35:06.248 INFO:tasks.workunit.client.1.smithi053.stdout:0/277: symlink d5/d10/d46/l5c 0 2024-09-21T04:35:06.257 INFO:tasks.workunit.client.1.smithi053.stdout:2/311: truncate d1/d18/d1f/d27/d41/f42 1358987 0 2024-09-21T04:35:06.261 INFO:tasks.workunit.client.1.smithi053.stdout:4/374: link d1/dd/de/c4c d1/d2c/c71 0 2024-09-21T04:35:06.263 INFO:tasks.workunit.client.1.smithi053.stdout:0/278: mknod d5/c5d 0 2024-09-21T04:35:06.264 INFO:tasks.workunit.client.1.smithi053.stdout:0/279: write d5/d47/f5a [89604,86038] 0 2024-09-21T04:35:06.265 INFO:tasks.workunit.client.1.smithi053.stdout:2/312: chown d1/d18/c1a 120 1 2024-09-21T04:35:06.269 INFO:tasks.workunit.client.1.smithi053.stdout:0/280: mkdir d5/de/d42/d5e 0 2024-09-21T04:35:06.271 INFO:tasks.workunit.client.1.smithi053.stdout:2/313: rename d1/d18/d1f/d27/d41/f4c to d1/f66 0 2024-09-21T04:35:06.271 INFO:tasks.workunit.client.1.smithi053.stdout:4/375: dwrite d1/dd/de/d14/d20/f45 [0,4194304] 0 2024-09-21T04:35:06.273 INFO:tasks.workunit.client.1.smithi053.stdout:3/325: symlink d16/d17/d21/d26/d2b/l74 0 2024-09-21T04:35:06.274 INFO:tasks.workunit.client.1.smithi053.stdout:0/281: symlink d5/d6/l5f 0 2024-09-21T04:35:06.276 INFO:tasks.workunit.client.1.smithi053.stdout:2/314: chown d1/d18/d1f/d22/f36 47501 1 2024-09-21T04:35:06.276 INFO:tasks.workunit.client.1.smithi053.stdout:3/326: readlink d16/d1b/l37 0 2024-09-21T04:35:06.277 INFO:tasks.workunit.client.1.smithi053.stdout:2/315: chown d1/d18/d1f/d22/c39 126351294 1 2024-09-21T04:35:06.278 INFO:tasks.workunit.client.1.smithi053.stdout:0/282: link d5/dd/f34 d5/d47/f60 0 2024-09-21T04:35:06.280 INFO:tasks.workunit.client.1.smithi053.stdout:3/327: rename f15 to d16/d1b/f75 0 2024-09-21T04:35:06.283 INFO:tasks.workunit.client.1.smithi053.stdout:2/316: link d1/d18/f35 d1/d18/f67 0 2024-09-21T04:35:06.285 INFO:tasks.workunit.client.1.smithi053.stdout:3/328: rename d16/d1b/f38 to d16/d17/d21/d60/f76 0 2024-09-21T04:35:06.286 INFO:tasks.workunit.client.1.smithi053.stdout:3/329: read - d16/d17/d21/d4e/f6c zero size 2024-09-21T04:35:06.289 INFO:tasks.workunit.client.1.smithi053.stdout:3/330: rename d16/f56 to d16/d17/d21/d26/d49/f77 0 2024-09-21T04:35:06.290 INFO:tasks.workunit.client.1.smithi053.stdout:2/317: creat d1/d18/d1f/d22/d54/f68 x:0 0 0 2024-09-21T04:35:06.291 INFO:tasks.workunit.client.1.smithi053.stdout:0/283: dread d5/de/d1c/f29 [0,4194304] 0 2024-09-21T04:35:06.292 INFO:tasks.workunit.client.1.smithi053.stdout:2/318: symlink d1/l69 0 2024-09-21T04:35:06.297 INFO:tasks.workunit.client.1.smithi053.stdout:2/319: creat d1/d18/d1f/d27/f6a x:0 0 0 2024-09-21T04:35:06.301 INFO:tasks.workunit.client.1.smithi053.stdout:2/320: unlink d1/l69 0 2024-09-21T04:35:06.339 INFO:tasks.workunit.client.1.smithi053.stdout:3/331: truncate d16/d17/d21/f6d 88653 0 2024-09-21T04:35:06.344 INFO:tasks.workunit.client.1.smithi053.stdout:0/284: write d5/dd/f58 [959009,82852] 0 2024-09-21T04:35:06.345 INFO:tasks.workunit.client.1.smithi053.stdout:3/332: dwrite d16/d17/d21/d65/f66 [0,4194304] 0 2024-09-21T04:35:06.348 INFO:tasks.workunit.client.0.smithi007.stdout:4/43: link d2/d7/fa d2/f12 0 2024-09-21T04:35:06.348 INFO:tasks.workunit.client.0.smithi007.stdout:4/44: chown d2/f3 7023768 1 2024-09-21T04:35:06.353 INFO:tasks.workunit.client.0.smithi007.stdout:5/48: rename d2/fa to d2/dd/ff 0 2024-09-21T04:35:06.354 INFO:tasks.workunit.client.0.smithi007.stdout:7/48: rename d3 to d3/df 22 2024-09-21T04:35:06.356 INFO:tasks.workunit.client.0.smithi007.stdout:7/49: stat d3/d6/fa 0 2024-09-21T04:35:06.357 INFO:tasks.workunit.client.0.smithi007.stdout:7/50: dread - d3/d6/f7 zero size 2024-09-21T04:35:06.358 INFO:tasks.workunit.client.0.smithi007.stdout:4/45: dwrite d2/d7/ff [0,4194304] 0 2024-09-21T04:35:06.358 INFO:tasks.workunit.client.1.smithi053.stdout:2/321: truncate d1/d18/d1f/d27/f55 2023152 0 2024-09-21T04:35:06.358 INFO:tasks.workunit.client.0.smithi007.stdout:4/46: fdatasync f0 0 2024-09-21T04:35:06.359 INFO:tasks.workunit.client.0.smithi007.stdout:7/51: write d3/d6/f8 [5471754,3829] 0 2024-09-21T04:35:06.362 INFO:tasks.workunit.client.0.smithi007.stdout:7/52: fdatasync d3/f5 0 2024-09-21T04:35:06.363 INFO:tasks.workunit.client.1.smithi053.stdout:2/322: rename d1/ca to d1/d18/d1f/d27/d41/d4b/d4d/c6b 0 2024-09-21T04:35:06.364 INFO:tasks.workunit.client.1.smithi053.stdout:3/333: dwrite d16/d1b/f27 [0,4194304] 0 2024-09-21T04:35:06.365 INFO:tasks.workunit.client.0.smithi007.stdout:1/43: rename d5/d6/f7 to d5/fc 0 2024-09-21T04:35:06.366 INFO:tasks.workunit.client.0.smithi007.stdout:4/47: fdatasync d2/d7/fa 0 2024-09-21T04:35:06.367 INFO:tasks.workunit.client.1.smithi053.stdout:3/334: truncate d16/d17/d21/d26/d3e/f62 228761 0 2024-09-21T04:35:06.368 INFO:tasks.workunit.client.0.smithi007.stdout:7/53: mkdir d3/d6/d10 0 2024-09-21T04:35:06.370 INFO:tasks.workunit.client.0.smithi007.stdout:7/54: write d3/d6/fa [877470,54734] 0 2024-09-21T04:35:06.370 INFO:tasks.workunit.client.1.smithi053.stdout:2/323: creat d1/d18/d1f/d27/d41/d4b/d4d/f6c x:0 0 0 2024-09-21T04:35:06.371 INFO:tasks.workunit.client.1.smithi053.stdout:2/324: chown d1/d18/d1f/d27/d41/d4b 7675708 1 2024-09-21T04:35:06.371 INFO:tasks.workunit.client.0.smithi007.stdout:1/44: symlink d5/ld 0 2024-09-21T04:35:06.374 INFO:tasks.workunit.client.0.smithi007.stdout:1/45: rmdir d5 39 2024-09-21T04:35:06.378 INFO:tasks.workunit.client.0.smithi007.stdout:7/55: mknod d3/d6/d10/c11 0 2024-09-21T04:35:06.380 INFO:tasks.workunit.client.0.smithi007.stdout:1/46: dread - d5/d6/fa zero size 2024-09-21T04:35:06.380 INFO:tasks.workunit.client.0.smithi007.stdout:7/56: mknod d3/d6/c12 0 2024-09-21T04:35:06.381 INFO:tasks.workunit.client.0.smithi007.stdout:7/57: fdatasync d3/d6/f8 0 2024-09-21T04:35:06.381 INFO:tasks.workunit.client.0.smithi007.stdout:1/47: fsync f3 0 2024-09-21T04:35:06.383 INFO:tasks.workunit.client.0.smithi007.stdout:7/58: mkdir d3/d6/d13 0 2024-09-21T04:35:06.386 INFO:tasks.workunit.client.0.smithi007.stdout:1/48: link d5/fc d5/fe 0 2024-09-21T04:35:06.386 INFO:tasks.workunit.client.0.smithi007.stdout:1/49: stat f4 0 2024-09-21T04:35:06.388 INFO:tasks.workunit.client.0.smithi007.stdout:7/59: dread d3/d6/f8 [4194304,4194304] 0 2024-09-21T04:35:06.388 INFO:tasks.workunit.client.0.smithi007.stdout:1/50: creat d5/ff x:0 0 0 2024-09-21T04:35:06.389 INFO:tasks.workunit.client.0.smithi007.stdout:1/51: chown d5/d6 1979322 1 2024-09-21T04:35:06.390 INFO:tasks.workunit.client.0.smithi007.stdout:7/60: mkdir d3/d6/d14 0 2024-09-21T04:35:06.393 INFO:tasks.workunit.client.0.smithi007.stdout:1/52: dread f4 [0,4194304] 0 2024-09-21T04:35:06.397 INFO:tasks.workunit.client.0.smithi007.stdout:7/61: dwrite d3/d6/f7 [0,4194304] 0 2024-09-21T04:35:06.401 INFO:tasks.workunit.client.0.smithi007.stdout:7/62: creat d3/f15 x:0 0 0 2024-09-21T04:35:06.404 INFO:tasks.workunit.client.0.smithi007.stdout:7/63: rename d3/d6/d10/c11 to d3/d6/d13/c16 0 2024-09-21T04:35:06.405 INFO:tasks.workunit.client.1.smithi053.stdout:0/285: dwrite d5/de/d1c/f41 [0,4194304] 0 2024-09-21T04:35:06.408 INFO:tasks.workunit.client.1.smithi053.stdout:0/286: creat d5/d6/f61 x:0 0 0 2024-09-21T04:35:06.409 INFO:tasks.workunit.client.1.smithi053.stdout:0/287: chown d5/dd/f34 63 1 2024-09-21T04:35:06.410 INFO:tasks.workunit.client.1.smithi053.stdout:2/325: write d1/d18/f67 [275328,76427] 0 2024-09-21T04:35:06.411 INFO:tasks.workunit.client.0.smithi007.stdout:4/48: truncate d2/fd 709306 0 2024-09-21T04:35:06.413 INFO:tasks.workunit.client.1.smithi053.stdout:2/326: creat d1/f6d x:0 0 0 2024-09-21T04:35:06.414 INFO:tasks.workunit.client.0.smithi007.stdout:4/49: mkdir d2/d13 0 2024-09-21T04:35:06.416 INFO:tasks.workunit.client.0.smithi007.stdout:4/50: mknod d2/d7/c14 0 2024-09-21T04:35:06.419 INFO:tasks.workunit.client.0.smithi007.stdout:4/51: getdents d2/d13 0 2024-09-21T04:35:06.419 INFO:tasks.workunit.client.0.smithi007.stdout:1/53: write f2 [2092673,17517] 0 2024-09-21T04:35:06.420 INFO:tasks.workunit.client.1.smithi053.stdout:2/327: link d1/d18/d1f/c29 d1/d18/d1f/d27/d58/c6e 0 2024-09-21T04:35:06.421 INFO:tasks.workunit.client.1.smithi053.stdout:2/328: dread - d1/d18/d1f/d22/d43/f4f zero size 2024-09-21T04:35:06.421 INFO:tasks.workunit.client.0.smithi007.stdout:1/54: mknod d5/c10 0 2024-09-21T04:35:06.422 INFO:tasks.workunit.client.0.smithi007.stdout:5/49: write d2/dd/ff [221537,78566] 0 2024-09-21T04:35:06.422 INFO:tasks.workunit.client.1.smithi053.stdout:2/329: write d1/d18/f49 [45749,64720] 0 2024-09-21T04:35:06.422 INFO:tasks.workunit.client.0.smithi007.stdout:1/55: fdatasync d5/d6/fa 0 2024-09-21T04:35:06.423 INFO:tasks.workunit.client.1.smithi053.stdout:2/330: chown d1/d18/d1f/d27/d41/d4b 1252161 1 2024-09-21T04:35:06.423 INFO:tasks.workunit.client.0.smithi007.stdout:7/64: getdents d3 0 2024-09-21T04:35:06.426 INFO:tasks.workunit.client.0.smithi007.stdout:5/50: symlink d2/dd/l10 0 2024-09-21T04:35:06.427 INFO:tasks.workunit.client.0.smithi007.stdout:5/51: fdatasync d2/f8 0 2024-09-21T04:35:06.428 INFO:tasks.workunit.client.0.smithi007.stdout:5/52: chown d2/dd/l10 11495 1 2024-09-21T04:35:06.429 INFO:tasks.workunit.client.0.smithi007.stdout:1/56: symlink d5/d6/l11 0 2024-09-21T04:35:06.429 INFO:tasks.workunit.client.0.smithi007.stdout:1/57: chown d5/ld 0 1 2024-09-21T04:35:06.429 INFO:tasks.workunit.client.0.smithi007.stdout:4/52: link d2/d7/cc d2/d7/c15 0 2024-09-21T04:35:06.430 INFO:tasks.workunit.client.0.smithi007.stdout:1/58: read f4 [754797,117670] 0 2024-09-21T04:35:06.433 INFO:tasks.workunit.client.1.smithi053.stdout:2/331: dread d1/d18/d1f/f4a [0,4194304] 0 2024-09-21T04:35:06.434 INFO:tasks.workunit.client.0.smithi007.stdout:1/59: chown d5/d6/c9 41699018 1 2024-09-21T04:35:06.434 INFO:tasks.workunit.client.0.smithi007.stdout:1/60: dread - d5/fe zero size 2024-09-21T04:35:06.434 INFO:tasks.workunit.client.0.smithi007.stdout:5/53: symlink d2/l11 0 2024-09-21T04:35:06.436 INFO:tasks.workunit.client.0.smithi007.stdout:1/61: creat d5/f12 x:0 0 0 2024-09-21T04:35:06.437 INFO:tasks.workunit.client.1.smithi053.stdout:2/332: mknod d1/d18/d1f/d22/d2d/d61/c6f 0 2024-09-21T04:35:06.439 INFO:tasks.workunit.client.0.smithi007.stdout:5/54: mknod d2/dd/c12 0 2024-09-21T04:35:06.441 INFO:tasks.workunit.client.0.smithi007.stdout:1/62: mknod d5/c13 0 2024-09-21T04:35:06.441 INFO:tasks.workunit.client.0.smithi007.stdout:5/55: rmdir d2/dd 39 2024-09-21T04:35:06.441 INFO:tasks.workunit.client.0.smithi007.stdout:5/56: fsync d2/fc 0 2024-09-21T04:35:06.443 INFO:tasks.workunit.client.0.smithi007.stdout:1/63: mknod d5/c14 0 2024-09-21T04:35:06.444 INFO:tasks.workunit.client.0.smithi007.stdout:5/57: stat d2/dd/c12 0 2024-09-21T04:35:06.445 INFO:tasks.workunit.client.0.smithi007.stdout:5/58: write d2/dd/ff [329787,104306] 0 2024-09-21T04:35:06.446 INFO:tasks.workunit.client.1.smithi053.stdout:2/333: getdents d1/d18 0 2024-09-21T04:35:06.446 INFO:tasks.workunit.client.0.smithi007.stdout:4/53: getdents d2 0 2024-09-21T04:35:06.447 INFO:tasks.workunit.client.1.smithi053.stdout:2/334: write d1/d18/d1f/d22/f5a [680602,1672] 0 2024-09-21T04:35:06.448 INFO:tasks.workunit.client.0.smithi007.stdout:1/64: creat d5/f15 x:0 0 0 2024-09-21T04:35:06.449 INFO:tasks.workunit.client.0.smithi007.stdout:1/65: write f2 [2094901,128663] 0 2024-09-21T04:35:06.450 INFO:tasks.workunit.client.1.smithi053.stdout:2/335: getdents d1/d18/d1f/d22/d54 0 2024-09-21T04:35:06.453 INFO:tasks.workunit.client.0.smithi007.stdout:5/59: dwrite d2/f8 [4194304,4194304] 0 2024-09-21T04:35:06.453 INFO:tasks.workunit.client.0.smithi007.stdout:5/60: fsync d2/fc 0 2024-09-21T04:35:06.454 INFO:tasks.workunit.client.0.smithi007.stdout:1/66: getdents d5/d6 0 2024-09-21T04:35:06.455 INFO:tasks.workunit.client.0.smithi007.stdout:1/67: chown d5/f15 10335682 1 2024-09-21T04:35:06.456 INFO:tasks.workunit.client.0.smithi007.stdout:5/61: mknod d2/dd/c13 0 2024-09-21T04:35:06.457 INFO:tasks.workunit.client.0.smithi007.stdout:5/62: read d2/f8 [5008767,92911] 0 2024-09-21T04:35:06.457 INFO:tasks.workunit.client.0.smithi007.stdout:1/68: creat d5/d6/f16 x:0 0 0 2024-09-21T04:35:06.458 INFO:tasks.workunit.client.0.smithi007.stdout:5/63: chown d2/dd 43072 1 2024-09-21T04:35:06.462 INFO:tasks.workunit.client.1.smithi053.stdout:2/336: dread d1/d18/f35 [0,4194304] 0 2024-09-21T04:35:06.463 INFO:tasks.workunit.client.0.smithi007.stdout:1/69: dread d5/d6/f8 [0,4194304] 0 2024-09-21T04:35:06.464 INFO:tasks.workunit.client.1.smithi053.stdout:2/337: read d1/d18/d1f/d22/f5a [668038,114197] 0 2024-09-21T04:35:06.467 INFO:tasks.workunit.client.0.smithi007.stdout:5/64: dwrite d2/dd/ff [0,4194304] 0 2024-09-21T04:35:06.469 INFO:tasks.workunit.client.0.smithi007.stdout:5/65: fsync d2/f8 0 2024-09-21T04:35:06.471 INFO:tasks.workunit.client.0.smithi007.stdout:5/66: read d2/f8 [7543234,108126] 0 2024-09-21T04:35:06.472 INFO:tasks.workunit.client.0.smithi007.stdout:5/67: stat d2/dd/ff 0 2024-09-21T04:35:06.473 INFO:tasks.workunit.client.0.smithi007.stdout:1/70: dwrite f3 [0,4194304] 0 2024-09-21T04:35:06.475 INFO:tasks.workunit.client.0.smithi007.stdout:5/68: creat d2/dd/f14 x:0 0 0 2024-09-21T04:35:06.478 INFO:tasks.workunit.client.1.smithi053.stdout:2/338: dread d1/d18/d1f/d22/f45 [0,4194304] 0 2024-09-21T04:35:06.479 INFO:tasks.workunit.client.0.smithi007.stdout:5/69: dread d2/dd/ff [0,4194304] 0 2024-09-21T04:35:06.480 INFO:tasks.workunit.client.0.smithi007.stdout:5/70: chown d2/dd/l10 63 1 2024-09-21T04:35:06.481 INFO:tasks.workunit.client.1.smithi053.stdout:2/339: mknod d1/d18/d1f/c70 0 2024-09-21T04:35:06.490 INFO:tasks.workunit.client.0.smithi007.stdout:5/71: dwrite d2/fc [0,4194304] 0 2024-09-21T04:35:06.493 INFO:tasks.workunit.client.0.smithi007.stdout:1/71: dwrite d5/fc [0,4194304] 0 2024-09-21T04:35:06.497 INFO:tasks.workunit.client.0.smithi007.stdout:1/72: rename d5/ff to d5/f17 0 2024-09-21T04:35:06.498 INFO:tasks.workunit.client.0.smithi007.stdout:5/72: getdents d2 0 2024-09-21T04:35:06.499 INFO:tasks.workunit.client.1.smithi053.stdout:2/340: dwrite d1/d18/d1f/d22/f32 [0,4194304] 0 2024-09-21T04:35:06.500 INFO:tasks.workunit.client.0.smithi007.stdout:5/73: rmdir d2 39 2024-09-21T04:35:06.504 INFO:tasks.workunit.client.1.smithi053.stdout:2/341: mkdir d1/d18/d1f/d27/d71 0 2024-09-21T04:35:06.505 INFO:tasks.workunit.client.1.smithi053.stdout:2/342: chown d1/d18/d1f/d27/d41/d4b 9728 1 2024-09-21T04:35:06.507 INFO:tasks.workunit.client.0.smithi007.stdout:1/73: dwrite d5/fe [4194304,4194304] 0 2024-09-21T04:35:06.508 INFO:tasks.workunit.client.1.smithi053.stdout:2/343: write d1/d18/d1f/d27/d41/d4b/d4d/f6c [146659,28511] 0 2024-09-21T04:35:06.511 INFO:tasks.workunit.client.0.smithi007.stdout:5/74: dread d2/f8 [4194304,4194304] 0 2024-09-21T04:35:06.511 INFO:tasks.workunit.client.0.smithi007.stdout:5/75: write d2/fc [1421009,40584] 0 2024-09-21T04:35:06.512 INFO:tasks.workunit.client.0.smithi007.stdout:5/76: write d2/fc [1244225,23172] 0 2024-09-21T04:35:06.514 INFO:tasks.workunit.client.0.smithi007.stdout:1/74: symlink d5/d6/l18 0 2024-09-21T04:35:06.516 INFO:tasks.workunit.client.0.smithi007.stdout:1/75: rmdir d5 39 2024-09-21T04:35:06.517 INFO:tasks.workunit.client.1.smithi053.stdout:2/344: symlink d1/d18/d1f/d27/d71/l72 0 2024-09-21T04:35:06.520 INFO:tasks.workunit.client.0.smithi007.stdout:1/76: unlink d5/d6/l18 0 2024-09-21T04:35:06.522 INFO:tasks.workunit.client.0.smithi007.stdout:5/77: dwrite d2/dd/f14 [0,4194304] 0 2024-09-21T04:35:06.524 INFO:tasks.workunit.client.1.smithi053.stdout:2/345: dwrite d1/d18/d1f/f4a [0,4194304] 0 2024-09-21T04:35:06.525 INFO:tasks.workunit.client.0.smithi007.stdout:1/77: dread f1 [0,4194304] 0 2024-09-21T04:35:06.526 INFO:tasks.workunit.client.1.smithi053.stdout:2/346: fdatasync d1/d18/f35 0 2024-09-21T04:35:06.528 INFO:tasks.workunit.client.0.smithi007.stdout:1/78: write f2 [793078,12295] 0 2024-09-21T04:35:06.530 INFO:tasks.workunit.client.0.smithi007.stdout:5/78: dwrite d2/fc [0,4194304] 0 2024-09-21T04:35:06.530 INFO:tasks.workunit.client.0.smithi007.stdout:1/79: mkdir d5/d6/d19 0 2024-09-21T04:35:06.538 INFO:tasks.workunit.client.0.smithi007.stdout:1/80: dread d5/fe [4194304,4194304] 0 2024-09-21T04:35:06.539 INFO:tasks.workunit.client.1.smithi053.stdout:2/347: dread d1/f66 [0,4194304] 0 2024-09-21T04:35:06.540 INFO:tasks.workunit.client.0.smithi007.stdout:5/79: dwrite d2/fc [0,4194304] 0 2024-09-21T04:35:06.542 INFO:tasks.workunit.client.1.smithi053.stdout:2/348: creat d1/d18/d1f/d22/d43/f73 x:0 0 0 2024-09-21T04:35:06.545 INFO:tasks.workunit.client.0.smithi007.stdout:1/81: link d5/d6/fa d5/f1a 0 2024-09-21T04:35:06.547 INFO:tasks.workunit.client.0.smithi007.stdout:1/82: chown d5/f1a 45156 1 2024-09-21T04:35:06.547 INFO:tasks.workunit.client.1.smithi053.stdout:2/349: unlink d1/d18/d1f/d27/d41/d4b/d4d/l50 0 2024-09-21T04:35:06.549 INFO:tasks.workunit.client.1.smithi053.stdout:2/350: stat d1/d18/d1f/c29 0 2024-09-21T04:35:06.550 INFO:tasks.workunit.client.1.smithi053.stdout:2/351: read d1/d18/f31 [1277771,43689] 0 2024-09-21T04:35:06.550 INFO:tasks.workunit.client.0.smithi007.stdout:5/80: dwrite d2/fc [0,4194304] 0 2024-09-21T04:35:06.550 INFO:tasks.workunit.client.0.smithi007.stdout:1/83: dread f4 [0,4194304] 0 2024-09-21T04:35:06.552 INFO:tasks.workunit.client.1.smithi053.stdout:5/330: sync 2024-09-21T04:35:06.552 INFO:tasks.workunit.client.1.smithi053.stdout:7/298: sync 2024-09-21T04:35:06.552 INFO:tasks.workunit.client.1.smithi053.stdout:1/397: sync 2024-09-21T04:35:06.553 INFO:tasks.workunit.client.0.smithi007.stdout:5/81: mknod d2/c15 0 2024-09-21T04:35:06.555 INFO:tasks.workunit.client.1.smithi053.stdout:2/352: rename d1/d18/f35 to d1/d18/d1f/d27/d58/f74 0 2024-09-21T04:35:06.556 INFO:tasks.workunit.client.1.smithi053.stdout:7/299: creat d0/d2b/d36/f5e x:0 0 0 2024-09-21T04:35:06.558 INFO:tasks.workunit.client.1.smithi053.stdout:5/331: creat d0/dd/d30/d51/d52/f6c x:0 0 0 2024-09-21T04:35:06.562 INFO:tasks.workunit.client.1.smithi053.stdout:1/398: rename d7/d24/d70 to d7/d9/d43/d54/d7f/d36/d81 0 2024-09-21T04:35:06.563 INFO:tasks.workunit.client.0.smithi007.stdout:1/84: link d5/c13 d5/d6/c1b 0 2024-09-21T04:35:06.564 INFO:tasks.workunit.client.1.smithi053.stdout:2/353: mknod d1/d18/d1f/d22/d43/c75 0 2024-09-21T04:35:06.565 INFO:tasks.workunit.client.1.smithi053.stdout:7/300: chown d0/d6/l58 640522956 1 2024-09-21T04:35:06.566 INFO:tasks.workunit.client.1.smithi053.stdout:7/301: fdatasync d0/d2b/f45 0 2024-09-21T04:35:06.568 INFO:tasks.workunit.client.1.smithi053.stdout:1/399: rename d7/d9/da/d11/l1e to d7/d9/d43/d54/d7f/l82 0 2024-09-21T04:35:06.571 INFO:tasks.workunit.client.0.smithi007.stdout:1/85: dwrite d5/fe [0,4194304] 0 2024-09-21T04:35:06.573 INFO:tasks.workunit.client.1.smithi053.stdout:5/332: dread d0/dd/d30/d51/d52/f59 [0,4194304] 0 2024-09-21T04:35:06.574 INFO:tasks.workunit.client.0.smithi007.stdout:5/82: rename d2/dd/ff to d2/dd/f16 0 2024-09-21T04:35:06.575 INFO:tasks.workunit.client.0.smithi007.stdout:1/86: rename d5/d6/f16 to d5/d6/d19/f1c 0 2024-09-21T04:35:06.578 INFO:tasks.workunit.client.0.smithi007.stdout:5/83: symlink d2/dd/l17 0 2024-09-21T04:35:06.580 INFO:tasks.workunit.client.1.smithi053.stdout:1/400: dread d7/d24/f41 [0,4194304] 0 2024-09-21T04:35:06.580 INFO:tasks.workunit.client.1.smithi053.stdout:5/333: mkdir d0/dd/d6d 0 2024-09-21T04:35:06.582 INFO:tasks.workunit.client.0.smithi007.stdout:1/87: dwrite d5/fe [4194304,4194304] 0 2024-09-21T04:35:06.583 INFO:tasks.workunit.client.1.smithi053.stdout:5/334: creat d0/dd/d6d/f6e x:0 0 0 2024-09-21T04:35:06.584 INFO:tasks.workunit.client.0.smithi007.stdout:5/84: dread d2/dd/f16 [0,4194304] 0 2024-09-21T04:35:06.585 INFO:tasks.workunit.client.1.smithi053.stdout:1/401: creat d7/d9/da/d40/d64/f83 x:0 0 0 2024-09-21T04:35:06.587 INFO:tasks.workunit.client.0.smithi007.stdout:5/85: fsync d2/fc 0 2024-09-21T04:35:06.588 INFO:tasks.workunit.client.1.smithi053.stdout:1/402: symlink d7/d9/da/d40/d49/d77/l84 0 2024-09-21T04:35:06.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/88: unlink d5/d6/d19/f1c 0 2024-09-21T04:35:06.595 INFO:tasks.workunit.client.1.smithi053.stdout:5/335: creat d0/dd/d30/f6f x:0 0 0 2024-09-21T04:35:06.597 INFO:tasks.workunit.client.0.smithi007.stdout:5/86: dwrite d2/dd/f16 [0,4194304] 0 2024-09-21T04:35:06.601 INFO:tasks.workunit.client.1.smithi053.stdout:6/348: sync 2024-09-21T04:35:06.601 INFO:tasks.workunit.client.1.smithi053.stdout:9/353: sync 2024-09-21T04:35:06.601 INFO:tasks.workunit.client.1.smithi053.stdout:3/335: sync 2024-09-21T04:35:06.602 INFO:tasks.workunit.client.1.smithi053.stdout:4/376: sync 2024-09-21T04:35:06.602 INFO:tasks.workunit.client.1.smithi053.stdout:8/344: sync 2024-09-21T04:35:06.602 INFO:tasks.workunit.client.1.smithi053.stdout:0/288: sync 2024-09-21T04:35:06.603 INFO:tasks.workunit.client.1.smithi053.stdout:0/289: stat d5/d10/d46/c4b 0 2024-09-21T04:35:06.603 INFO:tasks.workunit.client.1.smithi053.stdout:5/336: rename d0/dd/d30/d51/l65 to d0/d8/db/d68/l70 0 2024-09-21T04:35:06.604 INFO:tasks.workunit.client.1.smithi053.stdout:1/403: truncate d7/d9/f1b 3196339 0 2024-09-21T04:35:06.605 INFO:tasks.workunit.client.0.smithi007.stdout:1/89: rmdir d5/d6/d19 0 2024-09-21T04:35:06.606 INFO:tasks.workunit.client.1.smithi053.stdout:5/337: dread - d0/dd/f62 zero size 2024-09-21T04:35:06.607 INFO:tasks.workunit.client.1.smithi053.stdout:8/345: mknod d3/d28/c7a 0 2024-09-21T04:35:06.608 INFO:tasks.workunit.client.0.smithi007.stdout:1/90: symlink d5/l1d 0 2024-09-21T04:35:06.609 INFO:tasks.workunit.client.0.smithi007.stdout:1/91: dread - d5/f12 zero size 2024-09-21T04:35:06.614 INFO:tasks.workunit.client.1.smithi053.stdout:1/404: creat d7/d9/d43/d54/d7f/d59/f85 x:0 0 0 2024-09-21T04:35:06.615 INFO:tasks.workunit.client.1.smithi053.stdout:5/338: rmdir d0/dd/d30 39 2024-09-21T04:35:06.617 INFO:tasks.workunit.client.1.smithi053.stdout:4/377: dwrite d1/d27/f55 [4194304,4194304] 0 2024-09-21T04:35:06.622 INFO:tasks.workunit.client.1.smithi053.stdout:3/336: dwrite d16/d17/d21/d65/f66 [4194304,4194304] 0 2024-09-21T04:35:06.622 INFO:tasks.workunit.client.1.smithi053.stdout:8/346: symlink d3/d5/d21/l7b 0 2024-09-21T04:35:06.623 INFO:tasks.workunit.client.1.smithi053.stdout:6/349: getdents d2/d1b/d3d 0 2024-09-21T04:35:06.624 INFO:tasks.workunit.client.0.smithi007.stdout:1/92: truncate d5/fe 960521 0 2024-09-21T04:35:06.624 INFO:tasks.workunit.client.1.smithi053.stdout:1/405: stat d7/d9/d43/d54/d7f/d59/f7b 0 2024-09-21T04:35:06.625 INFO:tasks.workunit.client.1.smithi053.stdout:9/354: dwrite d0/d2/d9/d34/f65 [0,4194304] 0 2024-09-21T04:35:06.627 INFO:tasks.workunit.client.1.smithi053.stdout:4/378: unlink d1/dd/de/c1e 0 2024-09-21T04:35:06.630 INFO:tasks.workunit.client.1.smithi053.stdout:8/347: write d3/d5/d21/f78 [731115,122140] 0 2024-09-21T04:35:06.630 INFO:tasks.workunit.client.1.smithi053.stdout:3/337: write d16/d17/f2a [4449583,97710] 0 2024-09-21T04:35:06.631 INFO:tasks.workunit.client.1.smithi053.stdout:0/290: dread d5/f15 [0,4194304] 0 2024-09-21T04:35:06.632 INFO:tasks.workunit.client.1.smithi053.stdout:9/355: mknod d0/d28/c77 0 2024-09-21T04:35:06.634 INFO:tasks.workunit.client.0.smithi007.stdout:1/93: dwrite d5/f17 [0,4194304] 0 2024-09-21T04:35:06.635 INFO:tasks.workunit.client.1.smithi053.stdout:4/379: creat d1/dd/de/d14/d3f/f72 x:0 0 0 2024-09-21T04:35:06.638 INFO:tasks.workunit.client.1.smithi053.stdout:5/339: dwrite d0/f2f [0,4194304] 0 2024-09-21T04:35:06.639 INFO:tasks.workunit.client.1.smithi053.stdout:8/348: write d3/df/d49/f55 [3813829,70047] 0 2024-09-21T04:35:06.639 INFO:tasks.workunit.client.1.smithi053.stdout:8/349: chown d3/d5/d38/c6a 1 1 2024-09-21T04:35:06.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/94: rename d5/ld to d5/l1e 0 2024-09-21T04:35:06.642 INFO:tasks.workunit.client.1.smithi053.stdout:9/356: dread d0/d1c/d52/d53/f5e [0,4194304] 0 2024-09-21T04:35:06.642 INFO:tasks.workunit.client.0.smithi007.stdout:1/95: write d5/f15 [107650,108368] 0 2024-09-21T04:35:06.642 INFO:tasks.workunit.client.1.smithi053.stdout:0/291: mknod d5/de/d42/c62 0 2024-09-21T04:35:06.643 INFO:tasks.workunit.client.1.smithi053.stdout:9/357: read - d0/df/f48 zero size 2024-09-21T04:35:06.646 INFO:tasks.workunit.client.1.smithi053.stdout:3/338: symlink d16/l78 0 2024-09-21T04:35:06.649 INFO:tasks.workunit.client.0.smithi007.stdout:1/96: creat d5/f1f x:0 0 0 2024-09-21T04:35:06.652 INFO:tasks.workunit.client.1.smithi053.stdout:4/380: mkdir d1/dd/de/d14/d20/d73 0 2024-09-21T04:35:06.654 INFO:tasks.workunit.client.1.smithi053.stdout:5/340: mknod d0/d8/db/c71 0 2024-09-21T04:35:06.654 INFO:tasks.workunit.client.1.smithi053.stdout:4/381: write d1/dd/de/f6d [594645,119557] 0 2024-09-21T04:35:06.656 INFO:tasks.workunit.client.0.smithi007.stdout:1/97: rename f2 to d5/d6/f20 0 2024-09-21T04:35:06.657 INFO:tasks.workunit.client.1.smithi053.stdout:8/350: creat d3/d28/f7c x:0 0 0 2024-09-21T04:35:06.658 INFO:tasks.workunit.client.1.smithi053.stdout:8/351: write d3/d28/f68 [883951,70805] 0 2024-09-21T04:35:06.659 INFO:tasks.workunit.client.1.smithi053.stdout:8/352: stat d3/d5/d38/c6a 0 2024-09-21T04:35:06.660 INFO:tasks.workunit.client.1.smithi053.stdout:6/350: link d2/fd d2/d1b/d3d/d49/f75 0 2024-09-21T04:35:06.661 INFO:tasks.workunit.client.0.smithi007.stdout:1/98: mkdir d5/d6/d21 0 2024-09-21T04:35:06.662 INFO:tasks.workunit.client.1.smithi053.stdout:0/292: creat d5/dd/d50/f63 x:0 0 0 2024-09-21T04:35:06.663 INFO:tasks.workunit.client.1.smithi053.stdout:9/358: creat d0/d2/d4e/d5c/f78 x:0 0 0 2024-09-21T04:35:06.664 INFO:tasks.workunit.client.0.smithi007.stdout:1/99: chown d5/c13 9015122 1 2024-09-21T04:35:06.664 INFO:tasks.workunit.client.1.smithi053.stdout:9/359: write d0/d2/d9/d34/f59 [460316,98329] 0 2024-09-21T04:35:06.667 INFO:tasks.workunit.client.1.smithi053.stdout:1/406: rename d7/d9/f19 to d7/d9/da/f86 0 2024-09-21T04:35:06.668 INFO:tasks.workunit.client.1.smithi053.stdout:5/341: write d0/dd/d48/f54 [3751937,65509] 0 2024-09-21T04:35:06.668 INFO:tasks.workunit.client.0.smithi007.stdout:1/100: rename d5/c14 to d5/c22 0 2024-09-21T04:35:06.670 INFO:tasks.workunit.client.0.smithi007.stdout:1/101: dread d5/f15 [0,4194304] 0 2024-09-21T04:35:06.672 INFO:tasks.workunit.client.1.smithi053.stdout:4/382: rename d1/d27/d33/d41/d63 to d1/dd/de/d14/d3f/d74 0 2024-09-21T04:35:06.673 INFO:tasks.workunit.client.0.smithi007.stdout:1/102: write f4 [560719,48310] 0 2024-09-21T04:35:06.674 INFO:tasks.workunit.client.1.smithi053.stdout:1/407: dwrite d7/d9/da/d40/d49/f5e [0,4194304] 0 2024-09-21T04:35:06.675 INFO:tasks.workunit.client.1.smithi053.stdout:9/360: mkdir d0/d76/d79 0 2024-09-21T04:35:06.675 INFO:tasks.workunit.client.1.smithi053.stdout:5/342: chown d0/d8/d2a/c46 124771588 1 2024-09-21T04:35:06.679 INFO:tasks.workunit.client.1.smithi053.stdout:6/351: creat d2/d1b/d1e/d2a/f76 x:0 0 0 2024-09-21T04:35:06.681 INFO:tasks.workunit.client.0.smithi007.stdout:1/103: mkdir d5/d6/d21/d23 0 2024-09-21T04:35:06.682 INFO:tasks.workunit.client.0.smithi007.stdout:1/104: chown d5/f17 13160 1 2024-09-21T04:35:06.683 INFO:tasks.workunit.client.1.smithi053.stdout:8/353: rename d3/d5/d38/c6a to d3/d39/c7d 0 2024-09-21T04:35:06.688 INFO:tasks.workunit.client.1.smithi053.stdout:5/343: symlink d0/dd/d48/l72 0 2024-09-21T04:35:06.689 INFO:tasks.workunit.client.1.smithi053.stdout:5/344: chown d0/f6a 0 1 2024-09-21T04:35:06.690 INFO:tasks.workunit.client.1.smithi053.stdout:5/345: write d0/f2f [632135,102422] 0 2024-09-21T04:35:06.690 INFO:tasks.workunit.client.1.smithi053.stdout:6/352: fsync d2/d1b/d3d/d49/d1a/f26 0 2024-09-21T04:35:06.691 INFO:tasks.workunit.client.1.smithi053.stdout:3/339: getdents d16/d17/d21/d65 0 2024-09-21T04:35:06.692 INFO:tasks.workunit.client.1.smithi053.stdout:6/353: chown d2/d1b/d3d/d49/d1a/f44 3610 1 2024-09-21T04:35:06.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:06 smithi053 ceph-mon[35554]: pgmap v24: 65 pgs: 65 active+clean; 661 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 10 MiB/s rd, 84 MiB/s wr, 428 op/s 2024-09-21T04:35:06.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:06 smithi007 ceph-mon[27954]: pgmap v24: 65 pgs: 65 active+clean; 661 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 10 MiB/s rd, 84 MiB/s wr, 428 op/s 2024-09-21T04:35:06.696 INFO:tasks.workunit.client.1.smithi053.stdout:4/383: link d1/fc d1/dd/de/d14/d20/f75 0 2024-09-21T04:35:06.699 INFO:tasks.workunit.client.1.smithi053.stdout:8/354: dread d3/df/f62 [0,4194304] 0 2024-09-21T04:35:06.700 INFO:tasks.workunit.client.1.smithi053.stdout:6/354: dwrite d2/d1b/d3d/d49/f19 [0,4194304] 0 2024-09-21T04:35:06.700 INFO:tasks.workunit.client.1.smithi053.stdout:3/340: creat d16/d17/d3c/f79 x:0 0 0 2024-09-21T04:35:06.702 INFO:tasks.workunit.client.1.smithi053.stdout:6/355: write d2/d1b/d3d/d49/d1a/f44 [787416,92209] 0 2024-09-21T04:35:06.704 INFO:tasks.workunit.client.1.smithi053.stdout:9/361: creat d0/d2/d9/f7a x:0 0 0 2024-09-21T04:35:06.705 INFO:tasks.workunit.client.1.smithi053.stdout:4/384: mknod d1/d2c/c76 0 2024-09-21T04:35:06.706 INFO:tasks.workunit.client.1.smithi053.stdout:4/385: chown d1/dd/de/d14/d20/f75 16944 1 2024-09-21T04:35:06.708 INFO:tasks.workunit.client.1.smithi053.stdout:5/346: mknod d0/dd/d30/d51/d52/c73 0 2024-09-21T04:35:06.709 INFO:tasks.workunit.client.1.smithi053.stdout:5/347: write d0/f5b [510070,5830] 0 2024-09-21T04:35:06.710 INFO:tasks.workunit.client.1.smithi053.stdout:8/355: rmdir d3/d5/d4c/d53 39 2024-09-21T04:35:06.710 INFO:tasks.workunit.client.1.smithi053.stdout:5/348: chown d0/l11 3847 1 2024-09-21T04:35:06.712 INFO:tasks.workunit.client.0.smithi007.stdout:1/105: dwrite d5/fe [0,4194304] 0 2024-09-21T04:35:06.713 INFO:tasks.workunit.client.0.smithi007.stdout:1/106: dread - d5/f1a zero size 2024-09-21T04:35:06.713 INFO:tasks.workunit.client.1.smithi053.stdout:6/356: unlink d2/d1b/d1e/d2a/f76 0 2024-09-21T04:35:06.714 INFO:tasks.workunit.client.1.smithi053.stdout:1/408: getdents d7/d9/d43/d54/d7f/d59 0 2024-09-21T04:35:06.717 INFO:tasks.workunit.client.1.smithi053.stdout:4/386: creat d1/dd/de/d14/d3f/f77 x:0 0 0 2024-09-21T04:35:06.718 INFO:tasks.workunit.client.1.smithi053.stdout:3/341: dwrite d16/d1b/f40 [0,4194304] 0 2024-09-21T04:35:06.720 INFO:tasks.workunit.client.1.smithi053.stdout:1/409: write d7/d9/da/d11/f33 [2942079,6633] 0 2024-09-21T04:35:06.724 INFO:tasks.workunit.client.0.smithi007.stdout:1/107: getdents d5/d6/d21 0 2024-09-21T04:35:06.726 INFO:tasks.workunit.client.1.smithi053.stdout:4/387: creat d1/d27/d53/f78 x:0 0 0 2024-09-21T04:35:06.727 INFO:tasks.workunit.client.1.smithi053.stdout:5/349: mknod d0/dd/d30/d35/d61/c74 0 2024-09-21T04:35:06.727 INFO:tasks.workunit.client.1.smithi053.stdout:1/410: rmdir d7/d9/da/d3c 39 2024-09-21T04:35:06.727 INFO:tasks.workunit.client.1.smithi053.stdout:1/411: chown d7/d47 288 1 2024-09-21T04:35:06.728 INFO:tasks.workunit.client.1.smithi053.stdout:9/362: rename d0/d28/f62 to d0/d2/d9/f7b 0 2024-09-21T04:35:06.729 INFO:tasks.workunit.client.1.smithi053.stdout:3/342: mknod d16/d17/d21/d42/c7a 0 2024-09-21T04:35:06.729 INFO:tasks.workunit.client.1.smithi053.stdout:4/388: rmdir d1/dd 39 2024-09-21T04:35:06.731 INFO:tasks.workunit.client.1.smithi053.stdout:4/389: read - d1/d27/d2e/f61 zero size 2024-09-21T04:35:06.732 INFO:tasks.workunit.client.1.smithi053.stdout:6/357: rename d2/d1b/d3d/d49/d1a/d2e/d5f to d2/d1b/d1e/d2a/d4b/d77 0 2024-09-21T04:35:06.733 INFO:tasks.workunit.client.1.smithi053.stdout:8/356: getdents d3/d5/d21 0 2024-09-21T04:35:06.735 INFO:tasks.workunit.client.1.smithi053.stdout:8/357: dread - d3/d5/d38/f70 zero size 2024-09-21T04:35:06.736 INFO:tasks.workunit.client.1.smithi053.stdout:3/343: rename d16/d17/d21/l34 to d16/d17/d21/d42/l7b 0 2024-09-21T04:35:06.736 INFO:tasks.workunit.client.1.smithi053.stdout:1/412: link d7/d9/da/d11/l52 d7/d9/d43/d54/d7f/d59/l87 0 2024-09-21T04:35:06.737 INFO:tasks.workunit.client.1.smithi053.stdout:9/363: dwrite d0/d28/f3a [0,4194304] 0 2024-09-21T04:35:06.740 INFO:tasks.workunit.client.1.smithi053.stdout:6/358: mkdir d2/d1b/d1e/d2a/d78 0 2024-09-21T04:35:06.740 INFO:tasks.workunit.client.1.smithi053.stdout:4/390: write d1/dd/de/d14/d20/f34 [969238,3591] 0 2024-09-21T04:35:06.742 INFO:tasks.workunit.client.1.smithi053.stdout:5/350: write d0/dd/f3a [563966,42351] 0 2024-09-21T04:35:06.743 INFO:tasks.workunit.client.1.smithi053.stdout:3/344: write d16/d17/d21/d60/f76 [186690,34872] 0 2024-09-21T04:35:06.745 INFO:tasks.workunit.client.1.smithi053.stdout:8/358: dread d3/d5/f19 [8388608,4194304] 0 2024-09-21T04:35:06.745 INFO:tasks.workunit.client.1.smithi053.stdout:9/364: symlink d0/d1c/d52/l7c 0 2024-09-21T04:35:06.747 INFO:tasks.workunit.client.1.smithi053.stdout:3/345: chown d16/d17/d21/d42/l6f 920783064 1 2024-09-21T04:35:06.748 INFO:tasks.workunit.client.1.smithi053.stdout:9/365: chown d0/d2/fe 2068 1 2024-09-21T04:35:06.749 INFO:tasks.workunit.client.1.smithi053.stdout:1/413: truncate d7/d63/f7d 2234382 0 2024-09-21T04:35:06.749 INFO:tasks.workunit.client.1.smithi053.stdout:9/366: write d0/d28/d63/f69 [51670,106624] 0 2024-09-21T04:35:06.755 INFO:tasks.workunit.client.1.smithi053.stdout:6/359: dread d2/d1b/d1e/d2a/f4d [0,4194304] 0 2024-09-21T04:35:06.760 INFO:tasks.workunit.client.1.smithi053.stdout:7/302: sync 2024-09-21T04:35:06.760 INFO:tasks.workunit.client.1.smithi053.stdout:2/354: sync 2024-09-21T04:35:06.760 INFO:tasks.workunit.client.1.smithi053.stdout:6/360: write d2/f8 [3066668,129648] 0 2024-09-21T04:35:06.762 INFO:tasks.workunit.client.1.smithi053.stdout:5/351: rename d0/d8/l38 to d0/dd/d30/l75 0 2024-09-21T04:35:06.765 INFO:tasks.workunit.client.1.smithi053.stdout:1/414: rename d7/d9/f67 to d7/d9/da/d40/d64/d74/d75/f88 0 2024-09-21T04:35:06.766 INFO:tasks.workunit.client.1.smithi053.stdout:4/391: dread d1/dd/f23 [0,4194304] 0 2024-09-21T04:35:06.769 INFO:tasks.workunit.client.1.smithi053.stdout:8/359: dwrite d3/d39/f2f [0,4194304] 0 2024-09-21T04:35:06.769 INFO:tasks.workunit.client.1.smithi053.stdout:9/367: dread d0/d28/f38 [0,4194304] 0 2024-09-21T04:35:06.772 INFO:tasks.workunit.client.1.smithi053.stdout:5/352: creat d0/d8/db/f76 x:0 0 0 2024-09-21T04:35:06.774 INFO:tasks.workunit.client.1.smithi053.stdout:3/346: dwrite fd [4194304,4194304] 0 2024-09-21T04:35:06.776 INFO:tasks.workunit.client.1.smithi053.stdout:8/360: creat d3/d5/d4b/f7e x:0 0 0 2024-09-21T04:35:06.776 INFO:tasks.workunit.client.1.smithi053.stdout:1/415: chown d7/d9/d43/d54/d7f/d36/l44 5028678 1 2024-09-21T04:35:06.777 INFO:tasks.workunit.client.1.smithi053.stdout:4/392: truncate d1/dd/de/d14/d3f/f72 669064 0 2024-09-21T04:35:06.779 INFO:tasks.workunit.client.1.smithi053.stdout:6/361: rename d2/d1b/d3d/f72 to d2/d1b/d3d/d49/d1a/d2e/f79 0 2024-09-21T04:35:06.779 INFO:tasks.workunit.client.1.smithi053.stdout:1/416: readlink d7/d9/l3d 0 2024-09-21T04:35:06.781 INFO:tasks.workunit.client.1.smithi053.stdout:3/347: chown d16/d17/d21/d42/l7b 0 1 2024-09-21T04:35:06.782 INFO:tasks.workunit.client.1.smithi053.stdout:2/355: dwrite d1/d18/f67 [0,4194304] 0 2024-09-21T04:35:06.783 INFO:tasks.workunit.client.1.smithi053.stdout:6/362: chown d2/d1b/d3d/d49/c37 11656217 1 2024-09-21T04:35:06.785 INFO:tasks.workunit.client.1.smithi053.stdout:4/393: unlink d1/d2c/l32 0 2024-09-21T04:35:06.790 INFO:tasks.workunit.client.1.smithi053.stdout:6/363: dread - d2/d1b/d3d/d49/d1a/d2e/f55 zero size 2024-09-21T04:35:06.791 INFO:tasks.workunit.client.1.smithi053.stdout:5/353: dwrite d0/d8/db/f29 [4194304,4194304] 0 2024-09-21T04:35:06.791 INFO:tasks.workunit.client.1.smithi053.stdout:3/348: symlink d16/d17/d3c/l7c 0 2024-09-21T04:35:06.793 INFO:tasks.workunit.client.1.smithi053.stdout:8/361: creat d3/df/d14/f7f x:0 0 0 2024-09-21T04:35:06.797 INFO:tasks.workunit.client.1.smithi053.stdout:1/417: write f0 [2347139,18394] 0 2024-09-21T04:35:06.799 INFO:tasks.workunit.client.1.smithi053.stdout:4/394: creat d1/dd/de/d14/d20/d73/f79 x:0 0 0 2024-09-21T04:35:06.800 INFO:tasks.workunit.client.1.smithi053.stdout:3/349: truncate d16/d1b/f59 4967124 0 2024-09-21T04:35:06.802 INFO:tasks.workunit.client.1.smithi053.stdout:6/364: mknod d2/d1b/d1e/d2a/d4b/d77/d65/c7a 0 2024-09-21T04:35:06.805 INFO:tasks.workunit.client.1.smithi053.stdout:1/418: write d7/d9/da/d11/f5c [809652,83928] 0 2024-09-21T04:35:06.805 INFO:tasks.workunit.client.1.smithi053.stdout:8/362: mknod d3/df/d14/c80 0 2024-09-21T04:35:06.808 INFO:tasks.workunit.client.1.smithi053.stdout:2/356: dwrite d1/d18/d1f/d27/d58/d5c/f62 [0,4194304] 0 2024-09-21T04:35:06.808 INFO:tasks.workunit.client.1.smithi053.stdout:1/419: stat d7/d9/da/d40/d64 0 2024-09-21T04:35:06.808 INFO:tasks.workunit.client.1.smithi053.stdout:8/363: write d3/d39/f2f [3057314,103549] 0 2024-09-21T04:35:06.809 INFO:tasks.workunit.client.1.smithi053.stdout:4/395: rename d1/dd/de/d14/f5d to d1/dd/de/f7a 0 2024-09-21T04:35:06.811 INFO:tasks.workunit.client.1.smithi053.stdout:1/420: chown d7/f56 20024769 1 2024-09-21T04:35:06.811 INFO:tasks.workunit.client.1.smithi053.stdout:2/357: chown d1/d18/d1f/d22/f65 3807006 1 2024-09-21T04:35:06.811 INFO:tasks.workunit.client.1.smithi053.stdout:3/350: unlink d16/d17/d21/d26/d2b/c68 0 2024-09-21T04:35:06.813 INFO:tasks.workunit.client.1.smithi053.stdout:4/396: write d1/d27/d33/f3c [1551724,36267] 0 2024-09-21T04:35:06.813 INFO:tasks.workunit.client.1.smithi053.stdout:8/364: chown d3/df/d14/f7f 1221 1 2024-09-21T04:35:06.814 INFO:tasks.workunit.client.1.smithi053.stdout:6/365: creat d2/d1b/d1e/d2a/d4b/d77/d65/f7b x:0 0 0 2024-09-21T04:35:06.817 INFO:tasks.workunit.client.1.smithi053.stdout:1/421: mknod d7/d9/da/d40/d49/c89 0 2024-09-21T04:35:06.817 INFO:tasks.workunit.client.1.smithi053.stdout:2/358: chown d1/d18/d1f/d27/d58/f74 669 1 2024-09-21T04:35:06.822 INFO:tasks.workunit.client.1.smithi053.stdout:5/354: dwrite d0/d8/db/f19 [0,4194304] 0 2024-09-21T04:35:06.823 INFO:tasks.workunit.client.1.smithi053.stdout:1/422: creat d7/d63/d6b/f8a x:0 0 0 2024-09-21T04:35:06.824 INFO:tasks.workunit.client.1.smithi053.stdout:3/351: rename d16/d17/d21/d26/d71/d73 to d16/d17/d21/d65/d7d 0 2024-09-21T04:35:06.825 INFO:tasks.workunit.client.1.smithi053.stdout:2/359: unlink d1/d18/d1f/d22/l5f 0 2024-09-21T04:35:06.825 INFO:tasks.workunit.client.1.smithi053.stdout:6/366: truncate d2/fa 1022467 0 2024-09-21T04:35:06.827 INFO:tasks.workunit.client.1.smithi053.stdout:4/397: creat d1/dd/de/d14/d20/d4e/d51/f7b x:0 0 0 2024-09-21T04:35:06.830 INFO:tasks.workunit.client.1.smithi053.stdout:2/360: write d1/d18/f5b [1307433,15332] 0 2024-09-21T04:35:06.830 INFO:tasks.workunit.client.1.smithi053.stdout:1/423: mknod d7/d9/da/d40/d64/d74/c8b 0 2024-09-21T04:35:06.831 INFO:tasks.workunit.client.1.smithi053.stdout:4/398: fdatasync d1/d27/f55 0 2024-09-21T04:35:06.832 INFO:tasks.workunit.client.1.smithi053.stdout:8/365: dwrite d3/d5/d4c/f6d [0,4194304] 0 2024-09-21T04:35:06.834 INFO:tasks.workunit.client.1.smithi053.stdout:3/352: read d16/f1d [2157157,69821] 0 2024-09-21T04:35:06.834 INFO:tasks.workunit.client.1.smithi053.stdout:6/367: read d2/d1b/d1e/d2a/d4b/f58 [543193,118956] 0 2024-09-21T04:35:06.837 INFO:tasks.workunit.client.1.smithi053.stdout:1/424: write d7/d9/da/d11/f5c [3175196,101660] 0 2024-09-21T04:35:06.837 INFO:tasks.workunit.client.1.smithi053.stdout:5/355: write d0/f14 [593775,35191] 0 2024-09-21T04:35:06.839 INFO:tasks.workunit.client.1.smithi053.stdout:3/353: creat d16/d17/d3c/f7e x:0 0 0 2024-09-21T04:35:06.841 INFO:tasks.workunit.client.1.smithi053.stdout:2/361: mknod d1/d18/d1f/d27/d41/c76 0 2024-09-21T04:35:06.844 INFO:tasks.workunit.client.1.smithi053.stdout:3/354: stat d16/d17/d21/d65/d7d/d57/f6b 0 2024-09-21T04:35:06.845 INFO:tasks.workunit.client.1.smithi053.stdout:2/362: mkdir d1/d18/d77 0 2024-09-21T04:35:06.846 INFO:tasks.workunit.client.1.smithi053.stdout:1/425: getdents d7 0 2024-09-21T04:35:06.848 INFO:tasks.workunit.client.1.smithi053.stdout:3/355: mknod d16/d17/d21/d42/c7f 0 2024-09-21T04:35:06.848 INFO:tasks.workunit.client.1.smithi053.stdout:6/368: link d2/fd d2/d1b/d1e/d2a/f7c 0 2024-09-21T04:35:06.848 INFO:tasks.workunit.client.1.smithi053.stdout:8/366: getdents d3/d5/d4b/d5c 0 2024-09-21T04:35:06.850 INFO:tasks.workunit.client.1.smithi053.stdout:2/363: mkdir d1/d18/d1f/d27/d41/d4b/d78 0 2024-09-21T04:35:06.851 INFO:tasks.workunit.client.1.smithi053.stdout:1/426: symlink d7/d9/d43/d54/d7f/l8c 0 2024-09-21T04:35:06.851 INFO:tasks.workunit.client.1.smithi053.stdout:5/356: dwrite d0/d8/f31 [0,4194304] 0 2024-09-21T04:35:06.852 INFO:tasks.workunit.client.1.smithi053.stdout:6/369: stat d2/d1b/d3d/d49/de/c2d 0 2024-09-21T04:35:06.856 INFO:tasks.workunit.client.1.smithi053.stdout:2/364: fsync d1/d18/d1f/d22/f5a 0 2024-09-21T04:35:06.857 INFO:tasks.workunit.client.1.smithi053.stdout:3/356: getdents d16 0 2024-09-21T04:35:06.857 INFO:tasks.workunit.client.1.smithi053.stdout:5/357: dread d0/f14 [0,4194304] 0 2024-09-21T04:35:06.860 INFO:tasks.workunit.client.1.smithi053.stdout:8/367: write d3/d5/d38/f4e [4838699,32750] 0 2024-09-21T04:35:06.861 INFO:tasks.workunit.client.1.smithi053.stdout:6/370: truncate d2/d1b/d3d/d49/d1a/d2e/f55 141587 0 2024-09-21T04:35:06.861 INFO:tasks.workunit.client.1.smithi053.stdout:8/368: stat d3/d5 0 2024-09-21T04:35:06.862 INFO:tasks.workunit.client.1.smithi053.stdout:0/293: sync 2024-09-21T04:35:06.863 INFO:tasks.workunit.client.1.smithi053.stdout:3/357: symlink d16/d17/d3c/l80 0 2024-09-21T04:35:06.866 INFO:tasks.workunit.client.1.smithi053.stdout:1/427: dwrite d7/d9/da/d11/d2b/f61 [0,4194304] 0 2024-09-21T04:35:06.867 INFO:tasks.workunit.client.1.smithi053.stdout:2/365: write d1/d18/d1f/d27/d41/d4b/d4d/f52 [643804,4910] 0 2024-09-21T04:35:06.868 INFO:tasks.workunit.client.1.smithi053.stdout:0/294: dread - d5/de/f5b zero size 2024-09-21T04:35:06.869 INFO:tasks.workunit.client.1.smithi053.stdout:3/358: mknod d16/d17/d21/d4e/c81 0 2024-09-21T04:35:06.870 INFO:tasks.workunit.client.1.smithi053.stdout:6/371: write d2/d1b/d3d/d49/d1a/d2e/f35 [2430465,77056] 0 2024-09-21T04:35:06.872 INFO:tasks.workunit.client.1.smithi053.stdout:8/369: link d3/d5/d38/f70 d3/df/f81 0 2024-09-21T04:35:06.875 INFO:tasks.workunit.client.1.smithi053.stdout:1/428: truncate d7/d9/f1b 2133219 0 2024-09-21T04:35:06.876 INFO:tasks.workunit.client.1.smithi053.stdout:6/372: truncate d2/d1d/f50 886221 0 2024-09-21T04:35:06.877 INFO:tasks.workunit.client.1.smithi053.stdout:5/358: dread d0/dd/f44 [0,4194304] 0 2024-09-21T04:35:06.877 INFO:tasks.workunit.client.1.smithi053.stdout:6/373: write d2/f8 [3223356,99505] 0 2024-09-21T04:35:06.878 INFO:tasks.workunit.client.1.smithi053.stdout:8/370: creat d3/d5/d38/d64/f82 x:0 0 0 2024-09-21T04:35:06.881 INFO:tasks.workunit.client.1.smithi053.stdout:2/366: link d1/d18/f2a d1/d18/d1f/d27/d41/d4b/f79 0 2024-09-21T04:35:06.882 INFO:tasks.workunit.client.1.smithi053.stdout:6/374: dread d2/d1b/d3d/d49/d1a/f21 [0,4194304] 0 2024-09-21T04:35:06.882 INFO:tasks.workunit.client.1.smithi053.stdout:3/359: mkdir d16/d17/d21/d26/d49/d4d/d53/d82 0 2024-09-21T04:35:06.884 INFO:tasks.workunit.client.1.smithi053.stdout:8/371: dread d3/d5/d4c/f6d [0,4194304] 0 2024-09-21T04:35:06.884 INFO:tasks.workunit.client.1.smithi053.stdout:0/295: rename d5/d10/c32 to d5/d10/c64 0 2024-09-21T04:35:06.887 INFO:tasks.workunit.client.1.smithi053.stdout:6/375: chown d2/d1b/d3d/d49/f11 104284 1 2024-09-21T04:35:06.888 INFO:tasks.workunit.client.1.smithi053.stdout:8/372: fsync d3/df/f62 0 2024-09-21T04:35:06.889 INFO:tasks.workunit.client.1.smithi053.stdout:2/367: link d1/f3f d1/d18/d1f/d27/f7a 0 2024-09-21T04:35:06.892 INFO:tasks.workunit.client.1.smithi053.stdout:0/296: rename d5/f3b to d5/d6/f65 0 2024-09-21T04:35:06.895 INFO:tasks.workunit.client.1.smithi053.stdout:3/360: link d16/d17/d21/f6d d16/d17/d21/d4e/f83 0 2024-09-21T04:35:06.897 INFO:tasks.workunit.client.1.smithi053.stdout:5/359: dwrite d0/d8/f17 [0,4194304] 0 2024-09-21T04:35:06.898 INFO:tasks.workunit.client.1.smithi053.stdout:0/297: dread - d5/de/f35 zero size 2024-09-21T04:35:06.902 INFO:tasks.workunit.client.1.smithi053.stdout:8/373: dread d3/d5/f19 [8388608,4194304] 0 2024-09-21T04:35:06.903 INFO:tasks.workunit.client.1.smithi053.stdout:0/298: symlink d5/d47/l66 0 2024-09-21T04:35:06.903 INFO:tasks.workunit.client.1.smithi053.stdout:3/361: symlink d16/d17/d21/d42/l84 0 2024-09-21T04:35:06.904 INFO:tasks.workunit.client.1.smithi053.stdout:1/429: dread d7/d9/da/d3c/f46 [0,4194304] 0 2024-09-21T04:35:06.904 INFO:tasks.workunit.client.1.smithi053.stdout:6/376: dwrite d2/d1b/d3d/d49/f18 [4194304,4194304] 0 2024-09-21T04:35:06.908 INFO:tasks.workunit.client.1.smithi053.stdout:2/368: dwrite d1/d18/d1f/d22/d54/f68 [0,4194304] 0 2024-09-21T04:35:06.909 INFO:tasks.workunit.client.1.smithi053.stdout:6/377: write d2/f59 [984919,122995] 0 2024-09-21T04:35:06.911 INFO:tasks.workunit.client.1.smithi053.stdout:0/299: read d5/d6/f1b [1290678,19109] 0 2024-09-21T04:35:06.915 INFO:tasks.workunit.client.1.smithi053.stdout:6/378: creat d2/d1b/f7d x:0 0 0 2024-09-21T04:35:06.917 INFO:tasks.workunit.client.1.smithi053.stdout:5/360: getdents d0/dd/d30/d35 0 2024-09-21T04:35:06.920 INFO:tasks.workunit.client.1.smithi053.stdout:3/362: dwrite d16/d17/d21/d65/d7d/f4a [0,4194304] 0 2024-09-21T04:35:06.922 INFO:tasks.workunit.client.1.smithi053.stdout:5/361: chown d0/d8/d2a/l39 451619 1 2024-09-21T04:35:06.922 INFO:tasks.workunit.client.1.smithi053.stdout:8/374: getdents d3/d5/d4c/d53 0 2024-09-21T04:35:06.923 INFO:tasks.workunit.client.1.smithi053.stdout:1/430: truncate d7/d9/da/f86 590540 0 2024-09-21T04:35:06.923 INFO:tasks.workunit.client.1.smithi053.stdout:6/379: chown d2/d1b/d1e/d2a/d4b/d77/d65/c67 10886675 1 2024-09-21T04:35:06.927 INFO:tasks.workunit.client.1.smithi053.stdout:1/431: dread - d7/d9/d43/d54/d7f/f4f zero size 2024-09-21T04:35:06.927 INFO:tasks.workunit.client.1.smithi053.stdout:3/363: link d16/f3d d16/d17/d21/d65/d7d/f85 0 2024-09-21T04:35:06.928 INFO:tasks.workunit.client.1.smithi053.stdout:8/375: creat d3/d5/d4c/d53/f83 x:0 0 0 2024-09-21T04:35:06.928 INFO:tasks.workunit.client.1.smithi053.stdout:2/369: dread d1/f5 [0,4194304] 0 2024-09-21T04:35:06.930 INFO:tasks.workunit.client.1.smithi053.stdout:0/300: dwrite d5/de/d1c/f29 [0,4194304] 0 2024-09-21T04:35:06.933 INFO:tasks.workunit.client.1.smithi053.stdout:0/301: truncate d5/d10/f56 709401 0 2024-09-21T04:35:06.934 INFO:tasks.workunit.client.1.smithi053.stdout:2/370: creat d1/f7b x:0 0 0 2024-09-21T04:35:06.934 INFO:tasks.workunit.client.1.smithi053.stdout:5/362: rename d0/d8/c32 to d0/d8/c77 0 2024-09-21T04:35:06.937 INFO:tasks.workunit.client.1.smithi053.stdout:3/364: mkdir d16/d17/d21/d42/d43/d86 0 2024-09-21T04:35:06.939 INFO:tasks.workunit.client.1.smithi053.stdout:5/363: truncate d0/f6a 940764 0 2024-09-21T04:35:06.940 INFO:tasks.workunit.client.1.smithi053.stdout:8/376: dread d3/d5/d38/f47 [0,4194304] 0 2024-09-21T04:35:06.940 INFO:tasks.workunit.client.1.smithi053.stdout:6/380: dread d2/d1b/d1e/d27/d41/f64 [0,4194304] 0 2024-09-21T04:35:06.941 INFO:tasks.workunit.client.1.smithi053.stdout:0/302: creat d5/de/f67 x:0 0 0 2024-09-21T04:35:06.944 INFO:tasks.workunit.client.1.smithi053.stdout:1/432: dwrite d7/d9/da/d11/f5c [0,4194304] 0 2024-09-21T04:35:06.944 INFO:tasks.workunit.client.1.smithi053.stdout:8/377: symlink d3/d5/l84 0 2024-09-21T04:35:06.948 INFO:tasks.workunit.client.1.smithi053.stdout:8/378: dread - d3/d28/f65 zero size 2024-09-21T04:35:06.948 INFO:tasks.workunit.client.1.smithi053.stdout:6/381: rename d2/d1b/d1e/d27/f6e to d2/d1d/f7e 0 2024-09-21T04:35:06.949 INFO:tasks.workunit.client.1.smithi053.stdout:1/433: creat d7/d63/d6b/f8d x:0 0 0 2024-09-21T04:35:06.951 INFO:tasks.workunit.client.1.smithi053.stdout:3/365: getdents d16/d17/d21/d4e 0 2024-09-21T04:35:06.954 INFO:tasks.workunit.client.1.smithi053.stdout:0/303: dwrite d5/d6/f36 [0,4194304] 0 2024-09-21T04:35:06.955 INFO:tasks.workunit.client.1.smithi053.stdout:0/304: dread - d5/de/f35 zero size 2024-09-21T04:35:06.956 INFO:tasks.workunit.client.1.smithi053.stdout:6/382: rename d2/f59 to d2/d1b/d3d/d49/de/f7f 0 2024-09-21T04:35:06.959 INFO:tasks.workunit.client.1.smithi053.stdout:3/366: mknod d16/d17/d21/d26/d49/d4d/c87 0 2024-09-21T04:35:06.959 INFO:tasks.workunit.client.1.smithi053.stdout:0/305: dread d5/d10/f56 [0,4194304] 0 2024-09-21T04:35:06.961 INFO:tasks.workunit.client.1.smithi053.stdout:1/434: rename d7/d9/l21 to d7/d9/d43/d54/l8e 0 2024-09-21T04:35:06.961 INFO:tasks.workunit.client.1.smithi053.stdout:5/364: truncate d0/d8/f17 3584418 0 2024-09-21T04:35:06.962 INFO:tasks.workunit.client.1.smithi053.stdout:5/365: chown d0/f6a 1 1 2024-09-21T04:35:06.963 INFO:tasks.workunit.client.1.smithi053.stdout:0/306: write d5/d6/f65 [618400,122051] 0 2024-09-21T04:35:06.965 INFO:tasks.workunit.client.1.smithi053.stdout:6/383: rename d2/d1b/d3d/d49/d1a to d2/d1b/d3d/d49/de/d32/d80 0 2024-09-21T04:35:06.966 INFO:tasks.workunit.client.1.smithi053.stdout:2/371: truncate d1/d18/d1f/d22/f32 3334829 0 2024-09-21T04:35:06.969 INFO:tasks.workunit.client.1.smithi053.stdout:5/366: truncate d0/dd/d30/d51/d52/f56 207736 0 2024-09-21T04:35:06.972 INFO:tasks.workunit.client.1.smithi053.stdout:6/384: unlink d2/d1b/d3d/d49/de/d32/d80/d2e/f35 0 2024-09-21T04:35:06.973 INFO:tasks.workunit.client.1.smithi053.stdout:1/435: dwrite d7/d9/da/f5b [0,4194304] 0 2024-09-21T04:35:06.973 INFO:tasks.workunit.client.1.smithi053.stdout:5/367: rmdir d0/d8 39 2024-09-21T04:35:06.976 INFO:tasks.workunit.client.1.smithi053.stdout:0/307: dwrite d5/de/d1c/f41 [0,4194304] 0 2024-09-21T04:35:06.976 INFO:tasks.workunit.client.1.smithi053.stdout:2/372: symlink d1/d18/d1f/d27/d41/l7c 0 2024-09-21T04:35:06.976 INFO:tasks.workunit.client.1.smithi053.stdout:1/436: dread - d7/d9/da/d11/f48 zero size 2024-09-21T04:35:06.978 INFO:tasks.workunit.client.1.smithi053.stdout:5/368: stat d0/d8/c47 0 2024-09-21T04:35:06.983 INFO:tasks.workunit.client.1.smithi053.stdout:0/308: symlink d5/d10/d46/l68 0 2024-09-21T04:35:06.985 INFO:tasks.workunit.client.1.smithi053.stdout:1/437: mknod d7/d9/c8f 0 2024-09-21T04:35:06.988 INFO:tasks.workunit.client.1.smithi053.stdout:3/367: truncate d16/d17/f1e 2149568 0 2024-09-21T04:35:06.988 INFO:tasks.workunit.client.1.smithi053.stdout:6/385: dwrite d2/d1b/d3d/d49/de/d32/d80/f4e [0,4194304] 0 2024-09-21T04:35:06.993 INFO:tasks.workunit.client.1.smithi053.stdout:1/438: chown d7/d9/d43/d54/d7f/l62 598628 1 2024-09-21T04:35:06.994 INFO:tasks.workunit.client.1.smithi053.stdout:6/386: read - d2/d1b/f6b zero size 2024-09-21T04:35:06.995 INFO:tasks.workunit.client.1.smithi053.stdout:5/369: dwrite d0/d8/f31 [4194304,4194304] 0 2024-09-21T04:35:07.000 INFO:tasks.workunit.client.1.smithi053.stdout:3/368: creat d16/d17/d21/d65/d7d/f88 x:0 0 0 2024-09-21T04:35:07.000 INFO:tasks.workunit.client.1.smithi053.stdout:5/370: stat d0/l4c 0 2024-09-21T04:35:07.001 INFO:tasks.workunit.client.1.smithi053.stdout:1/439: creat d7/d9/da/d40/d49/d77/f90 x:0 0 0 2024-09-21T04:35:07.005 INFO:tasks.workunit.client.1.smithi053.stdout:6/387: creat d2/d1b/d1e/d2a/d4b/f81 x:0 0 0 2024-09-21T04:35:07.006 INFO:tasks.workunit.client.1.smithi053.stdout:5/371: unlink d0/dd/d30/c33 0 2024-09-21T04:35:07.006 INFO:tasks.workunit.client.1.smithi053.stdout:0/309: dwrite d5/d6/f65 [0,4194304] 0 2024-09-21T04:35:07.009 INFO:tasks.workunit.client.1.smithi053.stdout:3/369: symlink d16/d17/d21/d65/d7d/l89 0 2024-09-21T04:35:07.011 INFO:tasks.workunit.client.1.smithi053.stdout:0/310: read d5/d6/f1b [4081806,115813] 0 2024-09-21T04:35:07.011 INFO:tasks.workunit.client.1.smithi053.stdout:2/373: truncate d1/d18/d1f/d22/f32 1987433 0 2024-09-21T04:35:07.011 INFO:tasks.workunit.client.1.smithi053.stdout:3/370: write d16/d17/d21/d65/f6e [504794,83365] 0 2024-09-21T04:35:07.012 INFO:tasks.workunit.client.1.smithi053.stdout:1/440: getdents d7/d9/da/d40/d64/d74 0 2024-09-21T04:35:07.013 INFO:tasks.workunit.client.1.smithi053.stdout:6/388: fsync d2/d1b/d3d/d49/de/d32/d80/d2e/f55 0 2024-09-21T04:35:07.015 INFO:tasks.workunit.client.1.smithi053.stdout:1/441: readlink d7/d9/d43/d54/d7f/l62 0 2024-09-21T04:35:07.015 INFO:tasks.workunit.client.1.smithi053.stdout:2/374: symlink d1/d18/l7d 0 2024-09-21T04:35:07.016 INFO:tasks.workunit.client.1.smithi053.stdout:5/372: getdents d0/d8/db/d68 0 2024-09-21T04:35:07.018 INFO:tasks.workunit.client.1.smithi053.stdout:3/371: mknod d16/d17/d21/d26/d71/c8a 0 2024-09-21T04:35:07.019 INFO:tasks.workunit.client.1.smithi053.stdout:3/372: write d16/d17/d21/f64 [226592,19250] 0 2024-09-21T04:35:07.019 INFO:tasks.workunit.client.1.smithi053.stdout:0/311: symlink d5/de/d42/d5e/l69 0 2024-09-21T04:35:07.022 INFO:tasks.workunit.client.1.smithi053.stdout:5/373: rename d0/f5b to d0/d8/db/f78 0 2024-09-21T04:35:07.024 INFO:tasks.workunit.client.1.smithi053.stdout:3/373: rmdir d16/d17 39 2024-09-21T04:35:07.027 INFO:tasks.workunit.client.1.smithi053.stdout:0/312: symlink d5/de/d42/l6a 0 2024-09-21T04:35:07.028 INFO:tasks.workunit.client.1.smithi053.stdout:0/313: write d5/d47/f5a [1129686,91764] 0 2024-09-21T04:35:07.033 INFO:tasks.workunit.client.1.smithi053.stdout:6/389: link d2/d1b/d3d/d49/de/f43 d2/d1b/d3d/d49/de/d32/d80/f82 0 2024-09-21T04:35:07.034 INFO:tasks.workunit.client.1.smithi053.stdout:5/374: mknod d0/dd/d67/c79 0 2024-09-21T04:35:07.035 INFO:tasks.workunit.client.1.smithi053.stdout:0/314: mknod d5/dd/d45/c6b 0 2024-09-21T04:35:07.036 INFO:tasks.workunit.client.1.smithi053.stdout:1/442: write d7/d9/f1b [2615550,31073] 0 2024-09-21T04:35:07.039 INFO:tasks.workunit.client.1.smithi053.stdout:2/375: truncate d1/d18/d1f/d27/d41/f64 325375 0 2024-09-21T04:35:07.040 INFO:tasks.workunit.client.1.smithi053.stdout:3/374: truncate d16/d17/d21/d65/d7d/f85 812567 0 2024-09-21T04:35:07.041 INFO:tasks.workunit.client.1.smithi053.stdout:1/443: mkdir d7/d9/da/d40/d64/d74/d91 0 2024-09-21T04:35:07.043 INFO:tasks.workunit.client.1.smithi053.stdout:3/375: write d16/d17/d3c/f79 [504168,35432] 0 2024-09-21T04:35:07.044 INFO:tasks.workunit.client.1.smithi053.stdout:0/315: dwrite d5/dd/f58 [0,4194304] 0 2024-09-21T04:35:07.049 INFO:tasks.workunit.client.1.smithi053.stdout:6/390: write d2/d1b/d3d/d49/de/d32/d80/f61 [1423943,78141] 0 2024-09-21T04:35:07.051 INFO:tasks.workunit.client.1.smithi053.stdout:5/375: truncate d0/d8/d2a/f4a 1777711 0 2024-09-21T04:35:07.054 INFO:tasks.workunit.client.1.smithi053.stdout:5/376: chown d0/dd/c25 113233 1 2024-09-21T04:35:07.055 INFO:tasks.workunit.client.1.smithi053.stdout:6/391: rmdir d2/d1b/d3d/d49/de 39 2024-09-21T04:35:07.056 INFO:tasks.workunit.client.1.smithi053.stdout:0/316: dwrite d5/d10/d4c/d4d/f4e [0,4194304] 0 2024-09-21T04:35:07.058 INFO:tasks.workunit.client.1.smithi053.stdout:6/392: dread - d2/d1b/d1e/d2a/d4b/f68 zero size 2024-09-21T04:35:07.060 INFO:tasks.workunit.client.1.smithi053.stdout:2/376: dread d1/d18/f5b [0,4194304] 0 2024-09-21T04:35:07.060 INFO:tasks.workunit.client.1.smithi053.stdout:0/317: write d5/d6/d21/f3f [269925,33945] 0 2024-09-21T04:35:07.063 INFO:tasks.workunit.client.1.smithi053.stdout:0/318: stat d5/de/d1c/f41 0 2024-09-21T04:35:07.063 INFO:tasks.workunit.client.1.smithi053.stdout:2/377: creat d1/d18/d1f/d22/d2d/f7e x:0 0 0 2024-09-21T04:35:07.069 INFO:tasks.workunit.client.1.smithi053.stdout:6/393: rmdir d2/d1b/d3d/d49/de/d32/d80/d2e/d5b 39 2024-09-21T04:35:07.070 INFO:tasks.workunit.client.1.smithi053.stdout:2/378: mknod d1/d18/d1f/d27/d41/d4b/d4d/c7f 0 2024-09-21T04:35:07.072 INFO:tasks.workunit.client.1.smithi053.stdout:5/377: dwrite d0/d8/db/f78 [0,4194304] 0 2024-09-21T04:35:07.077 INFO:tasks.workunit.client.1.smithi053.stdout:5/378: creat d0/dd/d5c/f7a x:0 0 0 2024-09-21T04:35:07.078 INFO:tasks.workunit.client.1.smithi053.stdout:2/379: creat d1/d18/d1f/d27/d41/d4b/d78/f80 x:0 0 0 2024-09-21T04:35:07.079 INFO:tasks.workunit.client.1.smithi053.stdout:3/376: dwrite d16/d17/d21/d4e/f83 [0,4194304] 0 2024-09-21T04:35:07.080 INFO:tasks.workunit.client.1.smithi053.stdout:0/319: truncate d5/d10/d4c/d4d/f4e 2290925 0 2024-09-21T04:35:07.081 INFO:tasks.workunit.client.1.smithi053.stdout:3/377: chown d16/d17/d21/d26/d49/d4d 51152439 1 2024-09-21T04:35:07.081 INFO:tasks.workunit.client.1.smithi053.stdout:0/320: write d5/d6/f36 [4113724,127293] 0 2024-09-21T04:35:07.082 INFO:tasks.workunit.client.1.smithi053.stdout:5/379: creat d0/dd/d30/d35/f7b x:0 0 0 2024-09-21T04:35:07.083 INFO:tasks.workunit.client.1.smithi053.stdout:0/321: fsync d5/dd/f58 0 2024-09-21T04:35:07.086 INFO:tasks.workunit.client.1.smithi053.stdout:5/380: read d0/d8/f1f [3472550,124143] 0 2024-09-21T04:35:07.088 INFO:tasks.workunit.client.1.smithi053.stdout:2/380: link d1/c1e d1/d18/c81 0 2024-09-21T04:35:07.089 INFO:tasks.workunit.client.1.smithi053.stdout:6/394: dread d2/d1b/d3d/d49/f11 [0,4194304] 0 2024-09-21T04:35:07.090 INFO:tasks.workunit.client.1.smithi053.stdout:5/381: dread d0/d8/db/f5f [0,4194304] 0 2024-09-21T04:35:07.091 INFO:tasks.workunit.client.1.smithi053.stdout:0/322: dread d5/d47/f5a [0,4194304] 0 2024-09-21T04:35:07.092 INFO:tasks.workunit.client.1.smithi053.stdout:2/381: symlink d1/d18/l82 0 2024-09-21T04:35:07.093 INFO:tasks.workunit.client.1.smithi053.stdout:0/323: chown d5/d10/d4c/f52 3 1 2024-09-21T04:35:07.097 INFO:tasks.workunit.client.1.smithi053.stdout:2/382: mkdir d1/d18/d77/d83 0 2024-09-21T04:35:07.098 INFO:tasks.workunit.client.1.smithi053.stdout:2/383: fsync d1/f5 0 2024-09-21T04:35:07.098 INFO:tasks.workunit.client.1.smithi053.stdout:0/324: creat d5/dd/d45/f6c x:0 0 0 2024-09-21T04:35:07.103 INFO:tasks.workunit.client.1.smithi053.stdout:6/395: dwrite d2/d1b/d1e/f3f [0,4194304] 0 2024-09-21T04:35:07.103 INFO:tasks.workunit.client.1.smithi053.stdout:3/378: dread fa [4194304,4194304] 0 2024-09-21T04:35:07.104 INFO:tasks.workunit.client.1.smithi053.stdout:0/325: symlink d5/d10/d4c/d4d/l6d 0 2024-09-21T04:35:07.104 INFO:tasks.workunit.client.1.smithi053.stdout:2/384: mknod d1/d18/d1f/d27/d41/d4b/c84 0 2024-09-21T04:35:07.105 INFO:tasks.workunit.client.1.smithi053.stdout:6/396: rmdir d2/d1b/d1e/d27/d41 39 2024-09-21T04:35:07.106 INFO:tasks.workunit.client.1.smithi053.stdout:0/326: chown d5/d10/d46/l68 87294 1 2024-09-21T04:35:07.106 INFO:tasks.workunit.client.1.smithi053.stdout:3/379: unlink d16/d17/d21/d42/l6f 0 2024-09-21T04:35:07.108 INFO:tasks.workunit.client.1.smithi053.stdout:2/385: creat d1/d18/d1f/d22/d54/f85 x:0 0 0 2024-09-21T04:35:07.108 INFO:tasks.workunit.client.1.smithi053.stdout:0/327: readlink d5/dd/l14 0 2024-09-21T04:35:07.109 INFO:tasks.workunit.client.1.smithi053.stdout:0/328: fdatasync d5/de/f3e 0 2024-09-21T04:35:07.109 INFO:tasks.workunit.client.1.smithi053.stdout:6/397: unlink d2/d1b/d1e/d27/c56 0 2024-09-21T04:35:07.112 INFO:tasks.workunit.client.1.smithi053.stdout:2/386: creat d1/d18/d1f/d22/d2d/f86 x:0 0 0 2024-09-21T04:35:07.112 INFO:tasks.workunit.client.1.smithi053.stdout:5/382: dread d0/d8/db/f29 [0,4194304] 0 2024-09-21T04:35:07.112 INFO:tasks.workunit.client.1.smithi053.stdout:0/329: unlink d5/dd/c3c 0 2024-09-21T04:35:07.114 INFO:tasks.workunit.client.1.smithi053.stdout:6/398: write d2/d1b/d3d/d49/de/d32/d80/f4e [3259535,99488] 0 2024-09-21T04:35:07.115 INFO:tasks.workunit.client.1.smithi053.stdout:2/387: unlink d1/d18/l19 0 2024-09-21T04:35:07.116 INFO:tasks.workunit.client.1.smithi053.stdout:5/383: mknod d0/d8/db/d68/c7c 0 2024-09-21T04:35:07.116 INFO:tasks.workunit.client.1.smithi053.stdout:2/388: fdatasync d1/d18/d1f/d22/d43/f59 0 2024-09-21T04:35:07.118 INFO:tasks.workunit.client.1.smithi053.stdout:5/384: truncate d0/d8/db/f19 4495631 0 2024-09-21T04:35:07.120 INFO:tasks.workunit.client.1.smithi053.stdout:5/385: write d0/dd/d6d/f6e [609364,48867] 0 2024-09-21T04:35:07.123 INFO:tasks.workunit.client.1.smithi053.stdout:5/386: dread - d0/dd/d30/d51/d52/f6c zero size 2024-09-21T04:35:07.123 INFO:tasks.workunit.client.1.smithi053.stdout:6/399: link d2/d1b/d1e/d2a/d4b/f81 d2/d1b/d3d/d49/de/d32/d80/f83 0 2024-09-21T04:35:07.126 INFO:tasks.workunit.client.1.smithi053.stdout:0/330: truncate d5/f15 5598764 0 2024-09-21T04:35:07.127 INFO:tasks.workunit.client.1.smithi053.stdout:3/380: dwrite f11 [0,4194304] 0 2024-09-21T04:35:07.128 INFO:tasks.workunit.client.1.smithi053.stdout:2/389: dread d1/f66 [0,4194304] 0 2024-09-21T04:35:07.133 INFO:tasks.workunit.client.1.smithi053.stdout:3/381: rename d16/d17/d21/d65 to d16/d17/d21/d8b 0 2024-09-21T04:35:07.134 INFO:tasks.workunit.client.1.smithi053.stdout:3/382: fdatasync d16/d17/d21/f64 0 2024-09-21T04:35:07.135 INFO:tasks.workunit.client.1.smithi053.stdout:2/390: creat d1/d18/d1f/d22/d2d/f87 x:0 0 0 2024-09-21T04:35:07.137 INFO:tasks.workunit.client.1.smithi053.stdout:2/391: write d1/d18/d1f/d27/d41/d4b/d4d/f6c [1221287,126958] 0 2024-09-21T04:35:07.138 INFO:tasks.workunit.client.1.smithi053.stdout:3/383: read d16/d17/f31 [4071709,122251] 0 2024-09-21T04:35:07.140 INFO:tasks.workunit.client.1.smithi053.stdout:5/387: link d0/dd/d48/l72 d0/dd/d5c/l7d 0 2024-09-21T04:35:07.142 INFO:tasks.workunit.client.1.smithi053.stdout:6/400: dwrite d2/d1b/d3d/d49/de/d32/d80/f21 [0,4194304] 0 2024-09-21T04:35:07.144 INFO:tasks.workunit.client.1.smithi053.stdout:2/392: rename d1/d18/d1f/d22/f3a to d1/d18/d1f/d27/d41/d4b/d4d/f88 0 2024-09-21T04:35:07.144 INFO:tasks.workunit.client.1.smithi053.stdout:2/393: dread - d1/d18/d1f/d22/d2d/f87 zero size 2024-09-21T04:35:07.145 INFO:tasks.workunit.client.1.smithi053.stdout:6/401: rmdir d2/d1b/d3d/d49/de/d32/d80 39 2024-09-21T04:35:07.147 INFO:tasks.workunit.client.1.smithi053.stdout:2/394: truncate d1/d18/f31 1392466 0 2024-09-21T04:35:07.150 INFO:tasks.workunit.client.1.smithi053.stdout:6/402: unlink d2/d1b/d1e/l31 0 2024-09-21T04:35:07.152 INFO:tasks.workunit.client.1.smithi053.stdout:2/395: mkdir d1/d18/d1f/d27/d41/d4b/d4d/d89 0 2024-09-21T04:35:07.155 INFO:tasks.workunit.client.1.smithi053.stdout:2/396: truncate d1/d18/d1f/d22/d2d/f7e 162310 0 2024-09-21T04:35:07.159 INFO:tasks.workunit.client.1.smithi053.stdout:0/331: dread d5/f15 [0,4194304] 0 2024-09-21T04:35:07.159 INFO:tasks.workunit.client.1.smithi053.stdout:2/397: truncate d1/d18/d1f/d22/d2d/f7e 1083861 0 2024-09-21T04:35:07.159 INFO:tasks.workunit.client.1.smithi053.stdout:6/403: dwrite d2/d1b/d3d/d49/f11 [0,4194304] 0 2024-09-21T04:35:07.160 INFO:tasks.workunit.client.1.smithi053.stdout:0/332: write d5/dd/d50/f63 [591236,106311] 0 2024-09-21T04:35:07.165 INFO:tasks.workunit.client.1.smithi053.stdout:0/333: chown d5/dd/c51 61043732 1 2024-09-21T04:35:07.165 INFO:tasks.workunit.client.1.smithi053.stdout:2/398: creat d1/d18/d1f/f8a x:0 0 0 2024-09-21T04:35:07.166 INFO:tasks.workunit.client.1.smithi053.stdout:0/334: read d5/f15 [2887605,117860] 0 2024-09-21T04:35:07.170 INFO:tasks.workunit.client.1.smithi053.stdout:2/399: symlink d1/d18/d1f/d27/d41/d4b/d78/l8b 0 2024-09-21T04:35:07.174 INFO:tasks.workunit.client.1.smithi053.stdout:0/335: dwrite d5/de/f2a [0,4194304] 0 2024-09-21T04:35:07.174 INFO:tasks.workunit.client.1.smithi053.stdout:2/400: unlink d1/f3f 0 2024-09-21T04:35:07.175 INFO:tasks.workunit.client.1.smithi053.stdout:6/404: dread d2/d1b/d1e/f4f [0,4194304] 0 2024-09-21T04:35:07.176 INFO:tasks.workunit.client.1.smithi053.stdout:6/405: chown d2/fb 9 1 2024-09-21T04:35:07.177 INFO:tasks.workunit.client.1.smithi053.stdout:2/401: creat d1/f8c x:0 0 0 2024-09-21T04:35:07.178 INFO:tasks.workunit.client.1.smithi053.stdout:7/303: sync 2024-09-21T04:35:07.178 INFO:tasks.workunit.client.1.smithi053.stdout:9/368: sync 2024-09-21T04:35:07.178 INFO:tasks.workunit.client.1.smithi053.stdout:4/399: sync 2024-09-21T04:35:07.179 INFO:tasks.workunit.client.1.smithi053.stdout:9/369: chown d0/d76 83767395 1 2024-09-21T04:35:07.180 INFO:tasks.workunit.client.1.smithi053.stdout:4/400: dread - d1/dd/de/d14/d3f/f77 zero size 2024-09-21T04:35:07.180 INFO:tasks.workunit.client.1.smithi053.stdout:7/304: truncate d0/d2b/f32 448458 0 2024-09-21T04:35:07.181 INFO:tasks.workunit.client.1.smithi053.stdout:6/406: fsync d2/d1b/d3d/d49/de/f7f 0 2024-09-21T04:35:07.185 INFO:tasks.workunit.client.1.smithi053.stdout:7/305: creat d0/d24/d46/f5f x:0 0 0 2024-09-21T04:35:07.187 INFO:tasks.workunit.client.1.smithi053.stdout:4/401: dwrite d1/dd/de/d14/d20/f6f [0,4194304] 0 2024-09-21T04:35:07.189 INFO:tasks.workunit.client.1.smithi053.stdout:2/402: creat d1/d18/d77/d83/f8d x:0 0 0 2024-09-21T04:35:07.190 INFO:tasks.workunit.client.1.smithi053.stdout:4/402: fdatasync d1/dd/de/d14/d3f/f72 0 2024-09-21T04:35:07.191 INFO:tasks.workunit.client.1.smithi053.stdout:6/407: creat d2/d1b/d1e/d2a/d78/f84 x:0 0 0 2024-09-21T04:35:07.193 INFO:tasks.workunit.client.1.smithi053.stdout:2/403: write d1/f7b [567785,90126] 0 2024-09-21T04:35:07.194 INFO:tasks.workunit.client.1.smithi053.stdout:7/306: rename d0/d2b/d36/d48/c4e to d0/d2b/d36/c60 0 2024-09-21T04:35:07.199 INFO:tasks.workunit.client.1.smithi053.stdout:9/370: dread d0/d1a/f2c [0,4194304] 0 2024-09-21T04:35:07.204 INFO:tasks.workunit.client.1.smithi053.stdout:6/408: link d2/d1b/f2f d2/d1b/d3d/d49/de/f85 0 2024-09-21T04:35:07.206 INFO:tasks.workunit.client.1.smithi053.stdout:9/371: dwrite d0/d28/d63/f69 [0,4194304] 0 2024-09-21T04:35:07.207 INFO:tasks.workunit.client.1.smithi053.stdout:4/403: creat d1/d27/f7c x:0 0 0 2024-09-21T04:35:07.209 INFO:tasks.workunit.client.1.smithi053.stdout:7/307: creat d0/f61 x:0 0 0 2024-09-21T04:35:07.276 INFO:tasks.workunit.client.1.smithi053.stdout:4/404: mknod d1/dd/de/d14/d20/d73/c7d 0 2024-09-21T04:35:07.279 INFO:tasks.workunit.client.1.smithi053.stdout:4/405: fsync d1/d27/d33/f65 0 2024-09-21T04:35:07.281 INFO:tasks.workunit.client.1.smithi053.stdout:6/409: rename d2/d1d/l2c to d2/d1b/d1e/d2a/l86 0 2024-09-21T04:35:07.284 INFO:tasks.workunit.client.1.smithi053.stdout:7/308: dwrite d0/d2b/f2e [0,4194304] 0 2024-09-21T04:35:07.286 INFO:tasks.workunit.client.1.smithi053.stdout:4/406: creat d1/dd/de/d14/d3f/d74/f7e x:0 0 0 2024-09-21T04:35:07.286 INFO:tasks.workunit.client.1.smithi053.stdout:9/372: dread d0/d2/d9/d34/f65 [0,4194304] 0 2024-09-21T04:35:07.287 INFO:tasks.workunit.client.1.smithi053.stdout:7/309: mknod d0/d24/d46/c62 0 2024-09-21T04:35:07.288 INFO:tasks.workunit.client.1.smithi053.stdout:4/407: write d1/dd/de/f6d [1457779,127431] 0 2024-09-21T04:35:07.289 INFO:tasks.workunit.client.1.smithi053.stdout:6/410: creat d2/d1b/d1e/d27/f87 x:0 0 0 2024-09-21T04:35:07.290 INFO:tasks.workunit.client.1.smithi053.stdout:9/373: write d0/d2/d9/d34/f65 [5463090,30168] 0 2024-09-21T04:35:07.294 INFO:tasks.workunit.client.1.smithi053.stdout:4/408: creat d1/d2c/f7f x:0 0 0 2024-09-21T04:35:07.296 INFO:tasks.workunit.client.1.smithi053.stdout:9/374: symlink d0/d2/d9/d34/l7d 0 2024-09-21T04:35:07.297 INFO:tasks.workunit.client.1.smithi053.stdout:4/409: creat d1/dd/de/d14/d20/d5b/f80 x:0 0 0 2024-09-21T04:35:07.298 INFO:tasks.workunit.client.1.smithi053.stdout:2/404: dread d1/d18/d1f/d27/d41/d4b/f79 [0,4194304] 0 2024-09-21T04:35:07.302 INFO:tasks.workunit.client.1.smithi053.stdout:2/405: truncate d1/d18/f31 2355605 0 2024-09-21T04:35:07.303 INFO:tasks.workunit.client.1.smithi053.stdout:4/410: dread d1/f42 [0,4194304] 0 2024-09-21T04:35:07.304 INFO:tasks.workunit.client.1.smithi053.stdout:2/406: symlink d1/d18/d1f/d22/l8e 0 2024-09-21T04:35:07.305 INFO:tasks.workunit.client.1.smithi053.stdout:4/411: rmdir d1/dd/de/d14/d3f 39 2024-09-21T04:35:07.306 INFO:tasks.workunit.client.1.smithi053.stdout:2/407: mknod d1/d18/d1f/d27/d41/d4b/d4d/d89/c8f 0 2024-09-21T04:35:07.309 INFO:tasks.workunit.client.1.smithi053.stdout:2/408: creat d1/d18/d1f/d22/d2d/d61/f90 x:0 0 0 2024-09-21T04:35:07.311 INFO:tasks.workunit.client.1.smithi053.stdout:2/409: mkdir d1/d18/d1f/d27/d58/d5c/d91 0 2024-09-21T04:35:07.313 INFO:tasks.workunit.client.1.smithi053.stdout:2/410: symlink d1/d18/d1f/d27/d41/d4b/l92 0 2024-09-21T04:35:07.319 INFO:tasks.workunit.client.1.smithi053.stdout:4/412: dread d1/dd/de/f2a [4194304,4194304] 0 2024-09-21T04:35:07.320 INFO:tasks.workunit.client.1.smithi053.stdout:2/411: dwrite d1/d18/d1f/d27/d41/d4b/d4d/f52 [0,4194304] 0 2024-09-21T04:35:07.323 INFO:tasks.workunit.client.1.smithi053.stdout:8/379: sync 2024-09-21T04:35:07.324 INFO:tasks.workunit.client.1.smithi053.stdout:8/380: chown d3/df/d10/f48 2009044 1 2024-09-21T04:35:07.325 INFO:tasks.workunit.client.1.smithi053.stdout:8/381: chown d3/d5/d4b 25564 1 2024-09-21T04:35:07.326 INFO:tasks.workunit.client.1.smithi053.stdout:8/382: chown d3/d5/l15 4310763 1 2024-09-21T04:35:07.328 INFO:tasks.workunit.client.1.smithi053.stdout:8/383: stat d3/d5/d4b/l51 0 2024-09-21T04:35:07.330 INFO:tasks.workunit.client.1.smithi053.stdout:8/384: write d3/df/d14/d16/f1d [983008,32137] 0 2024-09-21T04:35:07.333 INFO:tasks.workunit.client.1.smithi053.stdout:8/385: creat d3/f85 x:0 0 0 2024-09-21T04:35:07.334 INFO:tasks.workunit.client.1.smithi053.stdout:8/386: write d3/d28/f31 [4296028,112669] 0 2024-09-21T04:35:07.334 INFO:tasks.workunit.client.1.smithi053.stdout:8/387: chown d3/d5/d1a 0 1 2024-09-21T04:35:07.340 INFO:tasks.workunit.client.1.smithi053.stdout:8/388: dwrite d3/f85 [0,4194304] 0 2024-09-21T04:35:07.343 INFO:tasks.workunit.client.1.smithi053.stdout:8/389: mkdir d3/d5/d4b/d5c/d86 0 2024-09-21T04:35:07.345 INFO:tasks.workunit.client.1.smithi053.stdout:8/390: fsync d3/d5/d38/f70 0 2024-09-21T04:35:07.348 INFO:tasks.workunit.client.1.smithi053.stdout:8/391: creat d3/d39/f87 x:0 0 0 2024-09-21T04:35:07.351 INFO:tasks.workunit.client.1.smithi053.stdout:8/392: mknod d3/d5/d4b/d5c/c88 0 2024-09-21T04:35:07.352 INFO:tasks.workunit.client.1.smithi053.stdout:8/393: creat d3/df/d10/f89 x:0 0 0 2024-09-21T04:35:07.353 INFO:tasks.workunit.client.1.smithi053.stdout:7/310: write d0/d6/f18 [4497441,48276] 0 2024-09-21T04:35:07.353 INFO:tasks.workunit.client.1.smithi053.stdout:8/394: stat d3/d5/d1a/l20 0 2024-09-21T04:35:07.354 INFO:tasks.workunit.client.1.smithi053.stdout:6/411: write d2/d1b/d3d/d49/de/d32/d80/f83 [375044,62926] 0 2024-09-21T04:35:07.357 INFO:tasks.workunit.client.1.smithi053.stdout:8/395: mknod d3/d5/d4b/d5c/d86/c8a 0 2024-09-21T04:35:07.358 INFO:tasks.workunit.client.1.smithi053.stdout:7/311: dread d0/d6/ff [0,4194304] 0 2024-09-21T04:35:07.359 INFO:tasks.workunit.client.1.smithi053.stdout:2/412: getdents d1/d18/d1f/d27/d41/d4b/d4d/d89 0 2024-09-21T04:35:07.361 INFO:tasks.workunit.client.1.smithi053.stdout:7/312: dread - d0/d2b/d36/d48/f53 zero size 2024-09-21T04:35:07.362 INFO:tasks.workunit.client.1.smithi053.stdout:7/313: chown d0/f23 699 1 2024-09-21T04:35:07.363 INFO:tasks.workunit.client.1.smithi053.stdout:2/413: rmdir d1/d18/d1f/d22/d2d 39 2024-09-21T04:35:07.363 INFO:tasks.workunit.client.1.smithi053.stdout:6/412: dwrite d2/d1b/d1e/d2a/d4b/f68 [0,4194304] 0 2024-09-21T04:35:07.364 INFO:tasks.workunit.client.1.smithi053.stdout:4/413: write d1/d27/d2e/d46/f54 [1555556,79955] 0 2024-09-21T04:35:07.366 INFO:tasks.workunit.client.1.smithi053.stdout:7/314: write d0/d24/d46/f5f [682901,23956] 0 2024-09-21T04:35:07.370 INFO:tasks.workunit.client.1.smithi053.stdout:8/396: dwrite d3/d5/f19 [0,4194304] 0 2024-09-21T04:35:07.370 INFO:tasks.workunit.client.1.smithi053.stdout:2/414: creat d1/d18/d1f/d27/d41/d4b/d4d/f93 x:0 0 0 2024-09-21T04:35:07.371 INFO:tasks.workunit.client.1.smithi053.stdout:8/397: fdatasync d3/df/d14/f60 0 2024-09-21T04:35:07.371 INFO:tasks.workunit.client.1.smithi053.stdout:6/413: mkdir d2/d1b/d1e/d2a/d4b/d77/d88 0 2024-09-21T04:35:07.373 INFO:tasks.workunit.client.1.smithi053.stdout:7/315: creat d0/d2b/d36/d48/f63 x:0 0 0 2024-09-21T04:35:07.374 INFO:tasks.workunit.client.1.smithi053.stdout:4/414: mknod d1/dd/de/d14/d20/d4e/d51/c81 0 2024-09-21T04:35:07.376 INFO:tasks.workunit.client.1.smithi053.stdout:7/316: mkdir d0/d2b/d36/d64 0 2024-09-21T04:35:07.379 INFO:tasks.workunit.client.1.smithi053.stdout:2/415: mknod d1/d18/d1f/d27/c94 0 2024-09-21T04:35:07.382 INFO:tasks.workunit.client.1.smithi053.stdout:4/415: mknod d1/dd/de/d14/d20/d5b/c82 0 2024-09-21T04:35:07.384 INFO:tasks.workunit.client.1.smithi053.stdout:6/414: mkdir d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89 0 2024-09-21T04:35:07.387 INFO:tasks.workunit.client.1.smithi053.stdout:4/416: creat d1/dd/de/f83 x:0 0 0 2024-09-21T04:35:07.387 INFO:tasks.workunit.client.1.smithi053.stdout:1/444: sync 2024-09-21T04:35:07.387 INFO:tasks.workunit.client.1.smithi053.stdout:3/384: sync 2024-09-21T04:35:07.389 INFO:tasks.workunit.client.1.smithi053.stdout:6/415: rename d2/d1b/d3d/f4a to d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f8a 0 2024-09-21T04:35:07.390 INFO:tasks.workunit.client.1.smithi053.stdout:7/317: getdents d0/d2b 0 2024-09-21T04:35:07.390 INFO:tasks.workunit.client.1.smithi053.stdout:3/385: chown d16/d17/d21/d26 0 1 2024-09-21T04:35:07.393 INFO:tasks.workunit.client.1.smithi053.stdout:8/398: dread d3/d39/f40 [0,4194304] 0 2024-09-21T04:35:07.393 INFO:tasks.workunit.client.1.smithi053.stdout:1/445: symlink d7/d47/l92 0 2024-09-21T04:35:07.395 INFO:tasks.workunit.client.1.smithi053.stdout:2/416: write d1/d18/d1f/d27/f55 [2264296,101477] 0 2024-09-21T04:35:07.396 INFO:tasks.workunit.client.1.smithi053.stdout:6/416: dread d2/d1b/d3d/d49/de/d32/d80/f21 [0,4194304] 0 2024-09-21T04:35:07.396 INFO:tasks.workunit.client.1.smithi053.stdout:7/318: creat d0/d24/f65 x:0 0 0 2024-09-21T04:35:07.397 INFO:tasks.workunit.client.1.smithi053.stdout:8/399: truncate d3/d5/f13 562555 0 2024-09-21T04:35:07.399 INFO:tasks.workunit.client.1.smithi053.stdout:6/417: fsync d2/d1b/d1e/f4f 0 2024-09-21T04:35:07.400 INFO:tasks.workunit.client.1.smithi053.stdout:2/417: write d1/f66 [2921929,109196] 0 2024-09-21T04:35:07.400 INFO:tasks.workunit.client.1.smithi053.stdout:7/319: rename d0/l1 to d0/d2b/d36/d48/d50/l66 0 2024-09-21T04:35:07.402 INFO:tasks.workunit.client.1.smithi053.stdout:8/400: fsync d3/df/d14/d16/f35 0 2024-09-21T04:35:07.403 INFO:tasks.workunit.client.1.smithi053.stdout:3/386: getdents d16/d17/d21/d26/d2b 0 2024-09-21T04:35:07.403 INFO:tasks.workunit.client.1.smithi053.stdout:6/418: rmdir d2/d1b/d3d/d49/d5c 39 2024-09-21T04:35:07.403 INFO:tasks.workunit.client.1.smithi053.stdout:7/320: chown d0/d6/f1c 393 1 2024-09-21T04:35:07.405 INFO:tasks.workunit.client.1.smithi053.stdout:3/387: rmdir d16 39 2024-09-21T04:35:07.406 INFO:tasks.workunit.client.1.smithi053.stdout:8/401: read d3/f59 [261931,123516] 0 2024-09-21T04:35:07.406 INFO:tasks.workunit.client.1.smithi053.stdout:4/417: dwrite d1/dd/de/f7a [0,4194304] 0 2024-09-21T04:35:07.408 INFO:tasks.workunit.client.1.smithi053.stdout:1/446: truncate d7/d9/d43/f6d 2776223 0 2024-09-21T04:35:07.410 INFO:tasks.workunit.client.1.smithi053.stdout:8/402: creat d3/d28/f8b x:0 0 0 2024-09-21T04:35:07.411 INFO:tasks.workunit.client.1.smithi053.stdout:7/321: creat d0/d2b/d36/d48/d5c/f67 x:0 0 0 2024-09-21T04:35:07.412 INFO:tasks.workunit.client.1.smithi053.stdout:3/388: mknod d16/d17/d21/c8c 0 2024-09-21T04:35:07.412 INFO:tasks.workunit.client.1.smithi053.stdout:1/447: symlink d7/d9/da/d40/d58/l93 0 2024-09-21T04:35:07.413 INFO:tasks.workunit.client.1.smithi053.stdout:7/322: readlink d0/d24/l2c 0 2024-09-21T04:35:07.414 INFO:tasks.workunit.client.1.smithi053.stdout:1/448: chown d7/d9/c8f 0 1 2024-09-21T04:35:07.417 INFO:tasks.workunit.client.1.smithi053.stdout:6/419: link d2/d1b/d3d/d49/f42 d2/d1b/d3d/d49/de/f8b 0 2024-09-21T04:35:07.418 INFO:tasks.workunit.client.1.smithi053.stdout:7/323: mknod d0/d2b/d36/d48/d5d/c68 0 2024-09-21T04:35:07.422 INFO:tasks.workunit.client.1.smithi053.stdout:2/418: dread d1/d18/d1f/d22/f36 [0,4194304] 0 2024-09-21T04:35:07.424 INFO:tasks.workunit.client.1.smithi053.stdout:8/403: getdents d3/df 0 2024-09-21T04:35:07.425 INFO:tasks.workunit.client.1.smithi053.stdout:3/389: dwrite d16/f30 [0,4194304] 0 2024-09-21T04:35:07.425 INFO:tasks.workunit.client.1.smithi053.stdout:1/449: rmdir d7/d63/d76 0 2024-09-21T04:35:07.426 INFO:tasks.workunit.client.1.smithi053.stdout:4/418: truncate d1/dd/de/d14/d20/d4e/f68 115880 0 2024-09-21T04:35:07.429 INFO:tasks.workunit.client.1.smithi053.stdout:4/419: chown d1/cb 48 1 2024-09-21T04:35:07.431 INFO:tasks.workunit.client.1.smithi053.stdout:1/450: read d7/d9/f1b [691785,87191] 0 2024-09-21T04:35:07.435 INFO:tasks.workunit.client.1.smithi053.stdout:6/420: dwrite d2/d1b/d1e/d27/f87 [0,4194304] 0 2024-09-21T04:35:07.435 INFO:tasks.workunit.client.1.smithi053.stdout:2/419: symlink d1/d18/d1f/d22/d2d/d61/l95 0 2024-09-21T04:35:07.436 INFO:tasks.workunit.client.1.smithi053.stdout:7/324: link d0/c21 d0/d2b/d36/d48/d5d/c69 0 2024-09-21T04:35:07.438 INFO:tasks.workunit.client.1.smithi053.stdout:0/336: sync 2024-09-21T04:35:07.438 INFO:tasks.workunit.client.1.smithi053.stdout:9/375: sync 2024-09-21T04:35:07.438 INFO:tasks.workunit.client.1.smithi053.stdout:5/388: sync 2024-09-21T04:35:07.439 INFO:tasks.workunit.client.1.smithi053.stdout:3/390: truncate d16/d17/d21/f39 594107 0 2024-09-21T04:35:07.440 INFO:tasks.workunit.client.1.smithi053.stdout:7/325: chown d0/d6/l25 447684237 1 2024-09-21T04:35:07.440 INFO:tasks.workunit.client.1.smithi053.stdout:6/421: stat d2/d1b/d3d/d49/de/d32/d80/d2e/d38/f36 0 2024-09-21T04:35:07.441 INFO:tasks.workunit.client.1.smithi053.stdout:5/389: chown d0/dd 3505 1 2024-09-21T04:35:07.441 INFO:tasks.workunit.client.1.smithi053.stdout:1/451: rename d7/d9/da/d11/d2b to d7/d9/d43/d94 0 2024-09-21T04:35:07.443 INFO:tasks.workunit.client.1.smithi053.stdout:3/391: chown d16/d17/d21/d26/d2b 265868958 1 2024-09-21T04:35:07.443 INFO:tasks.workunit.client.1.smithi053.stdout:5/390: fsync d0/dd/f37 0 2024-09-21T04:35:07.443 INFO:tasks.workunit.client.1.smithi053.stdout:0/337: creat d5/dd/d50/f6e x:0 0 0 2024-09-21T04:35:07.444 INFO:tasks.workunit.client.1.smithi053.stdout:2/420: truncate d1/d18/d1f/d22/d43/f59 352897 0 2024-09-21T04:35:07.447 INFO:tasks.workunit.client.1.smithi053.stdout:6/422: chown d2/fd 0 1 2024-09-21T04:35:07.447 INFO:tasks.workunit.client.1.smithi053.stdout:7/326: creat d0/d6/f6a x:0 0 0 2024-09-21T04:35:07.449 INFO:tasks.workunit.client.1.smithi053.stdout:2/421: creat d1/d18/f96 x:0 0 0 2024-09-21T04:35:07.450 INFO:tasks.workunit.client.1.smithi053.stdout:8/404: dread d3/d39/f27 [0,4194304] 0 2024-09-21T04:35:07.454 INFO:tasks.workunit.client.1.smithi053.stdout:0/338: creat d5/d10/d46/f6f x:0 0 0 2024-09-21T04:35:07.454 INFO:tasks.workunit.client.1.smithi053.stdout:1/452: mkdir d7/d9/d43/d54/d7f/d36/d81/d95 0 2024-09-21T04:35:07.455 INFO:tasks.workunit.client.1.smithi053.stdout:6/423: read d2/d1b/d1e/d27/f87 [485545,118566] 0 2024-09-21T04:35:07.457 INFO:tasks.workunit.client.1.smithi053.stdout:7/327: stat d0/d2b/d36/d48/d50/c56 0 2024-09-21T04:35:07.458 INFO:tasks.workunit.client.1.smithi053.stdout:1/453: mkdir d7/d63/d96 0 2024-09-21T04:35:07.459 INFO:tasks.workunit.client.1.smithi053.stdout:9/376: dwrite d0/d2/fe [4194304,4194304] 0 2024-09-21T04:35:07.461 INFO:tasks.workunit.client.1.smithi053.stdout:9/377: stat d0/df/f48 0 2024-09-21T04:35:07.462 INFO:tasks.workunit.client.1.smithi053.stdout:2/422: link d1/fb d1/d18/d1f/d22/d43/f97 0 2024-09-21T04:35:07.465 INFO:tasks.workunit.client.1.smithi053.stdout:5/391: truncate d0/d8/db/f29 7910644 0 2024-09-21T04:35:07.466 INFO:tasks.workunit.client.1.smithi053.stdout:7/328: creat d0/d2b/d36/d48/f6b x:0 0 0 2024-09-21T04:35:07.467 INFO:tasks.workunit.client.1.smithi053.stdout:9/378: chown d0/l5a 1144 1 2024-09-21T04:35:07.467 INFO:tasks.workunit.client.1.smithi053.stdout:6/424: mkdir d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/d8c 0 2024-09-21T04:35:07.469 INFO:tasks.workunit.client.1.smithi053.stdout:4/420: dread d1/d27/f55 [0,4194304] 0 2024-09-21T04:35:07.470 INFO:tasks.workunit.client.1.smithi053.stdout:7/329: fdatasync d0/d24/d46/f5f 0 2024-09-21T04:35:07.472 INFO:tasks.workunit.client.1.smithi053.stdout:5/392: symlink d0/d8/db/l7e 0 2024-09-21T04:35:07.473 INFO:tasks.workunit.client.1.smithi053.stdout:8/405: dwrite d3/d5/f13 [0,4194304] 0 2024-09-21T04:35:07.474 INFO:tasks.workunit.client.1.smithi053.stdout:2/423: readlink d1/d18/d1f/d27/l4e 0 2024-09-21T04:35:07.478 INFO:tasks.workunit.client.1.smithi053.stdout:6/425: rmdir d2/d1b/d3d/d49/de/d32/d80/d2e 39 2024-09-21T04:35:07.484 INFO:tasks.workunit.client.1.smithi053.stdout:3/392: dwrite d16/d1b/f27 [0,4194304] 0 2024-09-21T04:35:07.489 INFO:tasks.workunit.client.1.smithi053.stdout:1/454: dwrite d7/d9/d43/d94/f61 [0,4194304] 0 2024-09-21T04:35:07.491 INFO:tasks.workunit.client.1.smithi053.stdout:8/406: rename d3/d5/f77 to d3/d5/d4c/f8c 0 2024-09-21T04:35:07.495 INFO:tasks.workunit.client.1.smithi053.stdout:9/379: dwrite d0/df/f48 [0,4194304] 0 2024-09-21T04:35:07.497 INFO:tasks.workunit.client.1.smithi053.stdout:8/407: dread - d3/df/f81 zero size 2024-09-21T04:35:07.499 INFO:tasks.workunit.client.1.smithi053.stdout:3/393: symlink d16/d17/d21/d8b/d7d/d57/l8d 0 2024-09-21T04:35:07.499 INFO:tasks.workunit.client.1.smithi053.stdout:2/424: dread d1/f66 [0,4194304] 0 2024-09-21T04:35:07.501 INFO:tasks.workunit.client.1.smithi053.stdout:7/330: dwrite d0/d6/f31 [0,4194304] 0 2024-09-21T04:35:07.501 INFO:tasks.workunit.client.1.smithi053.stdout:1/455: creat d7/d9/d43/d54/d7f/d36/f97 x:0 0 0 2024-09-21T04:35:07.503 INFO:tasks.workunit.client.1.smithi053.stdout:8/408: readlink d3/d5/d38/l6c 0 2024-09-21T04:35:07.504 INFO:tasks.workunit.client.1.smithi053.stdout:6/426: truncate d2/d1b/d1e/f4f 7445013 0 2024-09-21T04:35:07.505 INFO:tasks.workunit.client.1.smithi053.stdout:2/425: stat d1/d18/d1f/d22/d2d/d61 0 2024-09-21T04:35:07.507 INFO:tasks.workunit.client.1.smithi053.stdout:3/394: symlink d16/d17/d21/d42/l8e 0 2024-09-21T04:35:07.509 INFO:tasks.workunit.client.1.smithi053.stdout:5/393: dwrite d0/d8/f17 [0,4194304] 0 2024-09-21T04:35:07.509 INFO:tasks.workunit.client.1.smithi053.stdout:6/427: chown d2/d1b/d3d/d49/de/f85 109991290 1 2024-09-21T04:35:07.510 INFO:tasks.workunit.client.1.smithi053.stdout:7/331: mknod d0/d24/d46/c6c 0 2024-09-21T04:35:07.512 INFO:tasks.workunit.client.1.smithi053.stdout:1/456: fsync d7/d9/d43/f6d 0 2024-09-21T04:35:07.513 INFO:tasks.workunit.client.1.smithi053.stdout:6/428: chown d2/d1b/d3d/d49/de/d32/d80/f83 215428014 1 2024-09-21T04:35:07.513 INFO:tasks.workunit.client.1.smithi053.stdout:3/395: symlink d16/d17/d21/d26/d49/d4d/d53/l8f 0 2024-09-21T04:35:07.515 INFO:tasks.workunit.client.1.smithi053.stdout:6/429: write d2/d1d/f54 [525757,84479] 0 2024-09-21T04:35:07.516 INFO:tasks.workunit.client.1.smithi053.stdout:1/457: truncate d7/d9/d43/d54/d7f/f4f 337904 0 2024-09-21T04:35:07.516 INFO:tasks.workunit.client.1.smithi053.stdout:5/394: dread d0/d8/db/f5f [0,4194304] 0 2024-09-21T04:35:07.517 INFO:tasks.workunit.client.1.smithi053.stdout:2/426: mknod d1/d18/d1f/c98 0 2024-09-21T04:35:07.519 INFO:tasks.workunit.client.1.smithi053.stdout:9/380: dread d0/d2/d9/fc [0,4194304] 0 2024-09-21T04:35:07.526 INFO:tasks.workunit.client.1.smithi053.stdout:9/381: creat d0/d76/f7e x:0 0 0 2024-09-21T04:35:07.526 INFO:tasks.workunit.client.1.smithi053.stdout:3/396: dread d16/d17/d21/f39 [0,4194304] 0 2024-09-21T04:35:07.526 INFO:tasks.workunit.client.1.smithi053.stdout:1/458: dread f0 [0,4194304] 0 2024-09-21T04:35:07.527 INFO:tasks.workunit.client.1.smithi053.stdout:8/409: rename d3/d5/d4b/d5c/d86/c8a to d3/d5/c8d 0 2024-09-21T04:35:07.528 INFO:tasks.workunit.client.1.smithi053.stdout:6/430: mkdir d2/d1b/d3d/d49/d5c/d8d 0 2024-09-21T04:35:07.532 INFO:tasks.workunit.client.1.smithi053.stdout:3/397: mkdir d16/d17/d21/d8b/d7d/d57/d90 0 2024-09-21T04:35:07.532 INFO:tasks.workunit.client.1.smithi053.stdout:7/332: dwrite d0/d6/f9 [0,4194304] 0 2024-09-21T04:35:07.534 INFO:tasks.workunit.client.1.smithi053.stdout:1/459: creat d7/d9/d43/d54/d7f/d36/f98 x:0 0 0 2024-09-21T04:35:07.538 INFO:tasks.workunit.client.1.smithi053.stdout:5/395: write d0/d8/d2a/f4a [597500,115462] 0 2024-09-21T04:35:07.539 INFO:tasks.workunit.client.1.smithi053.stdout:9/382: fsync d0/d2/d9/f19 0 2024-09-21T04:35:07.539 INFO:tasks.workunit.client.1.smithi053.stdout:1/460: write d7/d9/d43/d54/d7f/d36/f97 [196689,53105] 0 2024-09-21T04:35:07.540 INFO:tasks.workunit.client.1.smithi053.stdout:2/427: write d1/d18/d1f/d27/d41/d4b/f79 [5178060,76837] 0 2024-09-21T04:35:07.543 INFO:tasks.workunit.client.1.smithi053.stdout:6/431: write d2/d1b/d1e/d2a/f7c [2378238,60772] 0 2024-09-21T04:35:07.544 INFO:tasks.workunit.client.1.smithi053.stdout:1/461: write d7/d9/d43/d94/f61 [1313294,5342] 0 2024-09-21T04:35:07.545 INFO:tasks.workunit.client.1.smithi053.stdout:7/333: dwrite d0/f61 [0,4194304] 0 2024-09-21T04:35:07.546 INFO:tasks.workunit.client.1.smithi053.stdout:5/396: unlink d0/dd/c22 0 2024-09-21T04:35:07.546 INFO:tasks.workunit.client.1.smithi053.stdout:2/428: read d1/d18/d1f/d22/d2d/f7e [929261,110199] 0 2024-09-21T04:35:07.547 INFO:tasks.workunit.client.1.smithi053.stdout:1/462: stat d7/d47/f4d 0 2024-09-21T04:35:07.548 INFO:tasks.workunit.client.1.smithi053.stdout:9/383: mknod d0/d2/d9/d1f/c7f 0 2024-09-21T04:35:07.549 INFO:tasks.workunit.client.1.smithi053.stdout:8/410: truncate d3/df/d14/d16/f29 388776 0 2024-09-21T04:35:07.549 INFO:tasks.workunit.client.1.smithi053.stdout:2/429: read - d1/d18/d1f/d22/d43/f4f zero size 2024-09-21T04:35:07.553 INFO:tasks.workunit.client.1.smithi053.stdout:3/398: write d16/f3d [412070,83962] 0 2024-09-21T04:35:07.557 INFO:tasks.workunit.client.1.smithi053.stdout:3/399: truncate d16/d17/d3c/f79 1228209 0 2024-09-21T04:35:07.557 INFO:tasks.workunit.client.1.smithi053.stdout:9/384: link d0/d2/d9/fb d0/d1c/d52/d53/f80 0 2024-09-21T04:35:07.558 INFO:tasks.workunit.client.1.smithi053.stdout:7/334: dwrite d0/d24/d46/f5f [0,4194304] 0 2024-09-21T04:35:07.561 INFO:tasks.workunit.client.1.smithi053.stdout:7/335: write d0/d2b/d36/d48/f6b [1023557,128714] 0 2024-09-21T04:35:07.561 INFO:tasks.workunit.client.1.smithi053.stdout:1/463: getdents d7/d63/d6b 0 2024-09-21T04:35:07.562 INFO:tasks.workunit.client.1.smithi053.stdout:7/336: stat d0/d24/d46/f4a 0 2024-09-21T04:35:07.567 INFO:tasks.workunit.client.1.smithi053.stdout:1/464: unlink d7/d9/da/lb 0 2024-09-21T04:35:07.568 INFO:tasks.workunit.client.1.smithi053.stdout:1/465: stat d7/d9/da/c17 0 2024-09-21T04:35:07.568 INFO:tasks.workunit.client.1.smithi053.stdout:3/400: mknod d16/c91 0 2024-09-21T04:35:07.570 INFO:tasks.workunit.client.1.smithi053.stdout:1/466: dread - d7/d63/d6b/f8a zero size 2024-09-21T04:35:07.571 INFO:tasks.workunit.client.1.smithi053.stdout:3/401: write d16/d17/d3c/f7e [861076,75618] 0 2024-09-21T04:35:07.571 INFO:tasks.workunit.client.1.smithi053.stdout:5/397: truncate d0/dd/d30/d51/d52/f59 1890681 0 2024-09-21T04:35:07.572 INFO:tasks.workunit.client.1.smithi053.stdout:8/411: dwrite d3/d5/d38/f3d [0,4194304] 0 2024-09-21T04:35:07.573 INFO:tasks.workunit.client.1.smithi053.stdout:9/385: getdents d0/d1a 0 2024-09-21T04:35:07.574 INFO:tasks.workunit.client.1.smithi053.stdout:2/430: truncate d1/d18/d1f/d27/f55 2319378 0 2024-09-21T04:35:07.577 INFO:tasks.workunit.client.1.smithi053.stdout:5/398: rename d0/d8/f17 to d0/d8/db/f7f 0 2024-09-21T04:35:07.577 INFO:tasks.workunit.client.1.smithi053.stdout:9/386: read d0/f4b [6346,12564] 0 2024-09-21T04:35:07.578 INFO:tasks.workunit.client.1.smithi053.stdout:2/431: fsync d1/d18/d1f/d22/d2d/f86 0 2024-09-21T04:35:07.582 INFO:tasks.workunit.client.1.smithi053.stdout:2/432: write d1/d18/d1f/d27/d41/d4b/d4d/f6c [2040128,58195] 0 2024-09-21T04:35:07.582 INFO:tasks.workunit.client.1.smithi053.stdout:9/387: creat d0/d1a/f81 x:0 0 0 2024-09-21T04:35:07.583 INFO:tasks.workunit.client.1.smithi053.stdout:1/467: dwrite d7/d9/d43/d54/d7f/d59/f7b [0,4194304] 0 2024-09-21T04:35:07.584 INFO:tasks.workunit.client.1.smithi053.stdout:2/433: chown d1/d18/d1f/d27/d41/d4b/d4d/d89 11 1 2024-09-21T04:35:07.585 INFO:tasks.workunit.client.1.smithi053.stdout:1/468: write d7/d9/da/f2f [1673945,88093] 0 2024-09-21T04:35:07.586 INFO:tasks.workunit.client.1.smithi053.stdout:5/399: dread d0/f14 [0,4194304] 0 2024-09-21T04:35:07.586 INFO:tasks.workunit.client.1.smithi053.stdout:1/469: dread - d7/d47/f4d zero size 2024-09-21T04:35:07.588 INFO:tasks.workunit.client.1.smithi053.stdout:9/388: unlink d0/df/l3e 0 2024-09-21T04:35:07.591 INFO:tasks.workunit.client.1.smithi053.stdout:5/400: mkdir d0/dd/d5c/d80 0 2024-09-21T04:35:07.594 INFO:tasks.workunit.client.1.smithi053.stdout:9/389: creat d0/d2/d9/d34/f82 x:0 0 0 2024-09-21T04:35:07.594 INFO:tasks.workunit.client.1.smithi053.stdout:1/470: dwrite d7/d9/d43/d54/d7f/d59/f85 [0,4194304] 0 2024-09-21T04:35:07.594 INFO:tasks.workunit.client.1.smithi053.stdout:8/412: truncate d3/d28/f31 576085 0 2024-09-21T04:35:07.596 INFO:tasks.workunit.client.1.smithi053.stdout:8/413: write d3/d5/d38/f4e [1657455,116076] 0 2024-09-21T04:35:07.597 INFO:tasks.workunit.client.1.smithi053.stdout:5/401: symlink d0/dd/d30/d35/d61/l81 0 2024-09-21T04:35:07.599 INFO:tasks.workunit.client.1.smithi053.stdout:9/390: creat d0/d1c/f83 x:0 0 0 2024-09-21T04:35:07.604 INFO:tasks.workunit.client.1.smithi053.stdout:2/434: dread d1/d18/d1f/d27/d58/f74 [0,4194304] 0 2024-09-21T04:35:07.606 INFO:tasks.workunit.client.1.smithi053.stdout:2/435: chown d1/d18/d1f/d27/d41/c76 51312771 1 2024-09-21T04:35:07.607 INFO:tasks.workunit.client.1.smithi053.stdout:2/436: write d1/d18/d1f/d22/d2d/f86 [42281,25392] 0 2024-09-21T04:35:07.607 INFO:tasks.workunit.client.1.smithi053.stdout:1/471: mkdir d7/d47/d79/d99 0 2024-09-21T04:35:07.608 INFO:tasks.workunit.client.1.smithi053.stdout:8/414: mkdir d3/d5/d4c/d8e 0 2024-09-21T04:35:07.609 INFO:tasks.workunit.client.1.smithi053.stdout:5/402: creat d0/d66/f82 x:0 0 0 2024-09-21T04:35:07.609 INFO:tasks.workunit.client.1.smithi053.stdout:8/415: chown d3/df/d10/l63 913181 1 2024-09-21T04:35:07.610 INFO:tasks.workunit.client.1.smithi053.stdout:2/437: rmdir d1/d18/d1f/d27/d58/d5c 39 2024-09-21T04:35:07.613 INFO:tasks.workunit.client.1.smithi053.stdout:5/403: mkdir d0/dd/d30/d83 0 2024-09-21T04:35:07.615 INFO:tasks.workunit.client.1.smithi053.stdout:2/438: dread d1/d18/f2a [0,4194304] 0 2024-09-21T04:35:07.615 INFO:tasks.workunit.client.1.smithi053.stdout:8/416: creat d3/d5/f8f x:0 0 0 2024-09-21T04:35:07.616 INFO:tasks.workunit.client.1.smithi053.stdout:5/404: mknod d0/dd/c84 0 2024-09-21T04:35:07.616 INFO:tasks.workunit.client.1.smithi053.stdout:8/417: chown d3/d5/d38/f3d 4502119 1 2024-09-21T04:35:07.617 INFO:tasks.workunit.client.1.smithi053.stdout:2/439: symlink d1/d18/d77/l99 0 2024-09-21T04:35:07.617 INFO:tasks.workunit.client.1.smithi053.stdout:5/405: readlink d0/dd/d48/l55 0 2024-09-21T04:35:07.617 INFO:tasks.workunit.client.1.smithi053.stdout:9/391: dread d0/d2/f13 [0,4194304] 0 2024-09-21T04:35:07.618 INFO:tasks.workunit.client.1.smithi053.stdout:8/418: creat d3/df/d10/f90 x:0 0 0 2024-09-21T04:35:07.619 INFO:tasks.workunit.client.1.smithi053.stdout:5/406: write d0/dd/d5c/f7a [748695,42156] 0 2024-09-21T04:35:07.622 INFO:tasks.workunit.client.1.smithi053.stdout:8/419: mkdir d3/d5/d38/d64/d73/d91 0 2024-09-21T04:35:07.624 INFO:tasks.workunit.client.1.smithi053.stdout:8/420: dread - d3/d28/f8b zero size 2024-09-21T04:35:07.626 INFO:tasks.workunit.client.1.smithi053.stdout:9/392: dwrite d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:07.628 INFO:tasks.workunit.client.1.smithi053.stdout:5/407: dwrite d0/d66/f82 [0,4194304] 0 2024-09-21T04:35:07.628 INFO:tasks.workunit.client.1.smithi053.stdout:8/421: fsync d3/d5/d38/f70 0 2024-09-21T04:35:07.629 INFO:tasks.workunit.client.1.smithi053.stdout:8/422: chown d3/d5 47 1 2024-09-21T04:35:07.629 INFO:tasks.workunit.client.1.smithi053.stdout:5/408: stat d0/dd/f62 0 2024-09-21T04:35:07.629 INFO:tasks.workunit.client.1.smithi053.stdout:9/393: mknod d0/d1c/d21/c84 0 2024-09-21T04:35:07.629 INFO:tasks.workunit.client.1.smithi053.stdout:8/423: stat d3/d5/d4c/f8c 0 2024-09-21T04:35:07.630 INFO:tasks.workunit.client.1.smithi053.stdout:5/409: write d0/d8/f1d [2169631,22304] 0 2024-09-21T04:35:07.631 INFO:tasks.workunit.client.1.smithi053.stdout:9/394: symlink d0/d2/d9/l85 0 2024-09-21T04:35:07.635 INFO:tasks.workunit.client.1.smithi053.stdout:8/424: mknod d3/d5/c92 0 2024-09-21T04:35:07.636 INFO:tasks.workunit.client.1.smithi053.stdout:9/395: creat d0/d1a/d4c/f86 x:0 0 0 2024-09-21T04:35:07.644 INFO:tasks.workunit.client.1.smithi053.stdout:9/396: dread d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:07.645 INFO:tasks.workunit.client.1.smithi053.stdout:8/425: creat d3/f93 x:0 0 0 2024-09-21T04:35:07.645 INFO:tasks.workunit.client.1.smithi053.stdout:2/440: dread d1/d18/d1f/d27/d41/f64 [0,4194304] 0 2024-09-21T04:35:07.648 INFO:tasks.workunit.client.1.smithi053.stdout:8/426: creat d3/df/d10/f94 x:0 0 0 2024-09-21T04:35:07.654 INFO:tasks.workunit.client.1.smithi053.stdout:9/397: dread d0/d2/d9/d34/f65 [0,4194304] 0 2024-09-21T04:35:07.659 INFO:tasks.workunit.client.1.smithi053.stdout:9/398: write d0/d1c/f83 [420461,91096] 0 2024-09-21T04:35:07.659 INFO:tasks.workunit.client.1.smithi053.stdout:5/410: dread d0/d8/f1d [0,4194304] 0 2024-09-21T04:35:07.660 INFO:tasks.workunit.client.1.smithi053.stdout:8/427: dwrite d3/d5/d38/f4e [0,4194304] 0 2024-09-21T04:35:07.662 INFO:tasks.workunit.client.1.smithi053.stdout:8/428: readlink d3/d5/d38/l6c 0 2024-09-21T04:35:07.663 INFO:tasks.workunit.client.1.smithi053.stdout:2/441: dwrite d1/d18/d1f/d27/d58/d5c/f62 [0,4194304] 0 2024-09-21T04:35:07.665 INFO:tasks.workunit.client.1.smithi053.stdout:2/442: fdatasync d1/d18/d1f/d27/d41/d4b/d4d/f93 0 2024-09-21T04:35:07.666 INFO:tasks.workunit.client.1.smithi053.stdout:5/411: mknod d0/dd/d30/d51/d52/c85 0 2024-09-21T04:35:07.669 INFO:tasks.workunit.client.1.smithi053.stdout:5/412: write d0/dd/f37 [995144,17353] 0 2024-09-21T04:35:07.671 INFO:tasks.workunit.client.1.smithi053.stdout:8/429: dread d3/d5/f13 [0,4194304] 0 2024-09-21T04:35:07.672 INFO:tasks.workunit.client.1.smithi053.stdout:8/430: fdatasync d3/f93 0 2024-09-21T04:35:07.673 INFO:tasks.workunit.client.1.smithi053.stdout:5/413: rmdir d0/dd/d30 39 2024-09-21T04:35:07.674 INFO:tasks.workunit.client.1.smithi053.stdout:9/399: dwrite d0/d2/d9/f7b [0,4194304] 0 2024-09-21T04:35:07.679 INFO:tasks.workunit.client.1.smithi053.stdout:8/431: rename d3/df/d14/c80 to d3/d5/d38/d64/d73/d91/c95 0 2024-09-21T04:35:07.682 INFO:tasks.workunit.client.1.smithi053.stdout:5/414: mknod d0/dd/d30/d35/c86 0 2024-09-21T04:35:07.684 INFO:tasks.workunit.client.1.smithi053.stdout:5/415: symlink d0/dd/d67/l87 0 2024-09-21T04:35:07.684 INFO:tasks.workunit.client.1.smithi053.stdout:0/339: sync 2024-09-21T04:35:07.686 INFO:tasks.workunit.client.1.smithi053.stdout:5/416: creat d0/dd/d30/d35/d61/f88 x:0 0 0 2024-09-21T04:35:07.686 INFO:tasks.workunit.client.1.smithi053.stdout:2/443: truncate d1/d18/d1f/d27/d41/d4b/d4d/f52 2521197 0 2024-09-21T04:35:07.687 INFO:tasks.workunit.client.1.smithi053.stdout:9/400: truncate d0/d1c/f39 1644392 0 2024-09-21T04:35:07.687 INFO:tasks.workunit.client.1.smithi053.stdout:5/417: chown d0/d8/db 39 1 2024-09-21T04:35:07.690 INFO:tasks.workunit.client.1.smithi053.stdout:5/418: mkdir d0/dd/d5c/d89 0 2024-09-21T04:35:07.691 INFO:tasks.workunit.client.1.smithi053.stdout:0/340: dwrite d5/de/d42/f48 [0,4194304] 0 2024-09-21T04:35:07.693 INFO:tasks.workunit.client.1.smithi053.stdout:9/401: link d0/d2/d9/d34/f59 d0/d1c/d52/d6d/f87 0 2024-09-21T04:35:07.698 INFO:tasks.workunit.client.1.smithi053.stdout:9/402: dread d0/df/f48 [0,4194304] 0 2024-09-21T04:35:07.701 INFO:tasks.workunit.client.1.smithi053.stdout:2/444: getdents d1/d18/d1f/d27/d41/d4b/d4d 0 2024-09-21T04:35:07.701 INFO:tasks.workunit.client.1.smithi053.stdout:2/445: chown d1/d18/l25 65881228 1 2024-09-21T04:35:07.702 INFO:tasks.workunit.client.1.smithi053.stdout:5/419: creat d0/dd/d5c/d80/f8a x:0 0 0 2024-09-21T04:35:07.706 INFO:tasks.workunit.client.1.smithi053.stdout:2/446: mkdir d1/d18/d1f/d27/d58/d5c/d9a 0 2024-09-21T04:35:07.707 INFO:tasks.workunit.client.1.smithi053.stdout:5/420: creat d0/d8/db/d68/f8b x:0 0 0 2024-09-21T04:35:07.708 INFO:tasks.workunit.client.1.smithi053.stdout:0/341: link d5/d6/l5f d5/d6/l70 0 2024-09-21T04:35:07.708 INFO:tasks.workunit.client.1.smithi053.stdout:0/342: stat d5/dd/f1a 0 2024-09-21T04:35:07.711 INFO:tasks.workunit.client.1.smithi053.stdout:2/447: mkdir d1/d18/d77/d9b 0 2024-09-21T04:35:07.711 INFO:tasks.workunit.client.1.smithi053.stdout:2/448: stat d1/d18/d1f/d22/d54/f85 0 2024-09-21T04:35:07.712 INFO:tasks.workunit.client.1.smithi053.stdout:2/449: write d1/d18/d1f/d22/d2d/f87 [674084,90843] 0 2024-09-21T04:35:07.713 INFO:tasks.workunit.client.1.smithi053.stdout:5/421: unlink d0/dd/f44 0 2024-09-21T04:35:07.714 INFO:tasks.workunit.client.1.smithi053.stdout:5/422: dread - d0/dd/d48/f60 zero size 2024-09-21T04:35:07.715 INFO:tasks.workunit.client.1.smithi053.stdout:0/343: getdents d5/d47 0 2024-09-21T04:35:07.716 INFO:tasks.workunit.client.1.smithi053.stdout:5/423: dread d0/d8/db/f5f [0,4194304] 0 2024-09-21T04:35:07.716 INFO:tasks.workunit.client.1.smithi053.stdout:2/450: mknod d1/d18/d1f/d27/d41/d4b/d78/c9c 0 2024-09-21T04:35:07.717 INFO:tasks.workunit.client.1.smithi053.stdout:5/424: mkdir d0/dd/d48/d8c 0 2024-09-21T04:35:07.718 INFO:tasks.workunit.client.1.smithi053.stdout:2/451: write d1/d18/d1f/d27/d41/d4b/f79 [4581337,70086] 0 2024-09-21T04:35:07.718 INFO:tasks.workunit.client.1.smithi053.stdout:0/344: getdents d5/d10/d4c/d4d 0 2024-09-21T04:35:07.720 INFO:tasks.workunit.client.1.smithi053.stdout:0/345: mknod d5/d10/c71 0 2024-09-21T04:35:07.721 INFO:tasks.workunit.client.1.smithi053.stdout:2/452: fsync d1/d18/d1f/d27/f55 0 2024-09-21T04:35:07.723 INFO:tasks.workunit.client.1.smithi053.stdout:0/346: creat d5/d10/f72 x:0 0 0 2024-09-21T04:35:07.724 INFO:tasks.workunit.client.1.smithi053.stdout:0/347: chown d5/dd/d45 22503 1 2024-09-21T04:35:07.729 INFO:tasks.workunit.client.1.smithi053.stdout:0/348: dwrite d5/de/f2a [0,4194304] 0 2024-09-21T04:35:07.730 INFO:tasks.workunit.client.1.smithi053.stdout:0/349: rmdir d5/d6 39 2024-09-21T04:35:07.732 INFO:tasks.workunit.client.1.smithi053.stdout:2/453: dread d1/d18/d1f/d27/f55 [0,4194304] 0 2024-09-21T04:35:07.733 INFO:tasks.workunit.client.1.smithi053.stdout:0/350: symlink d5/l73 0 2024-09-21T04:35:07.734 INFO:tasks.workunit.client.1.smithi053.stdout:0/351: chown d5/d6/f65 112412884 1 2024-09-21T04:35:07.736 INFO:tasks.workunit.client.1.smithi053.stdout:2/454: rename d1/d18/d1f/d22/d2d/d61 to d1/d18/d1f/d9d 0 2024-09-21T04:35:07.741 INFO:tasks.workunit.client.1.smithi053.stdout:0/352: dwrite d5/d10/d46/f6f [0,4194304] 0 2024-09-21T04:35:07.743 INFO:tasks.workunit.client.1.smithi053.stdout:0/353: symlink d5/de/d1c/l74 0 2024-09-21T04:35:07.745 INFO:tasks.workunit.client.1.smithi053.stdout:0/354: mknod d5/de/d42/c75 0 2024-09-21T04:35:07.745 INFO:tasks.workunit.client.1.smithi053.stdout:0/355: write d5/de/f35 [706618,43261] 0 2024-09-21T04:35:07.748 INFO:tasks.workunit.client.1.smithi053.stdout:0/356: mknod d5/dd/d45/c76 0 2024-09-21T04:35:07.750 INFO:tasks.workunit.client.1.smithi053.stdout:0/357: creat d5/d6/d53/f77 x:0 0 0 2024-09-21T04:35:07.759 INFO:tasks.workunit.client.1.smithi053.stdout:0/358: dwrite d5/de/f2a [0,4194304] 0 2024-09-21T04:35:07.764 INFO:tasks.workunit.client.1.smithi053.stdout:0/359: rename d5/d10/d46 to d5/de/d42/d78 0 2024-09-21T04:35:07.769 INFO:tasks.workunit.client.1.smithi053.stdout:0/360: getdents d5/de/d42/d5e 0 2024-09-21T04:35:07.769 INFO:tasks.workunit.client.1.smithi053.stdout:0/361: fdatasync d5/de/d1c/f3d 0 2024-09-21T04:35:07.773 INFO:tasks.workunit.client.1.smithi053.stdout:0/362: dread d5/de/d42/d78/f6f [0,4194304] 0 2024-09-21T04:35:07.776 INFO:tasks.workunit.client.1.smithi053.stdout:0/363: rename d5/dd/l14 to d5/de/d42/d78/l79 0 2024-09-21T04:35:07.777 INFO:tasks.workunit.client.1.smithi053.stdout:0/364: dread - d5/d10/d4c/f52 zero size 2024-09-21T04:35:07.785 INFO:tasks.workunit.client.1.smithi053.stdout:0/365: dwrite d5/d47/f60 [0,4194304] 0 2024-09-21T04:35:07.786 INFO:tasks.workunit.client.1.smithi053.stdout:0/366: write d5/dd/d45/f6c [840918,127724] 0 2024-09-21T04:35:07.789 INFO:tasks.workunit.client.1.smithi053.stdout:0/367: link d5/de/f67 d5/de/d42/d5e/f7a 0 2024-09-21T04:35:07.791 INFO:tasks.workunit.client.1.smithi053.stdout:0/368: mknod d5/de/d42/c7b 0 2024-09-21T04:35:07.793 INFO:tasks.workunit.client.1.smithi053.stdout:0/369: rmdir d5/d10/d44 39 2024-09-21T04:35:07.798 INFO:tasks.workunit.client.1.smithi053.stdout:0/370: dwrite d5/de/f5b [0,4194304] 0 2024-09-21T04:35:07.801 INFO:tasks.workunit.client.1.smithi053.stdout:0/371: write d5/d6/f1b [907773,84710] 0 2024-09-21T04:35:07.805 INFO:tasks.workunit.client.1.smithi053.stdout:0/372: rename d5/d6/l24 to d5/dd/d50/l7c 0 2024-09-21T04:35:07.806 INFO:tasks.workunit.client.1.smithi053.stdout:0/373: chown d5/d10/d4c/d4d/l6d 3 1 2024-09-21T04:35:07.806 INFO:tasks.workunit.client.1.smithi053.stdout:0/374: chown d5/d10/d4c/d4d 41807 1 2024-09-21T04:35:07.808 INFO:tasks.workunit.client.1.smithi053.stdout:0/375: fsync d5/de/f67 0 2024-09-21T04:35:07.810 INFO:tasks.workunit.client.1.smithi053.stdout:0/376: symlink d5/d10/d4c/l7d 0 2024-09-21T04:35:07.814 INFO:tasks.workunit.client.1.smithi053.stdout:0/377: truncate d5/d6/f36 1365003 0 2024-09-21T04:35:07.815 INFO:tasks.workunit.client.1.smithi053.stdout:0/378: creat d5/de/d1c/f7e x:0 0 0 2024-09-21T04:35:07.817 INFO:tasks.workunit.client.1.smithi053.stdout:0/379: mknod d5/d10/d4c/d4d/c7f 0 2024-09-21T04:35:07.819 INFO:tasks.workunit.client.1.smithi053.stdout:0/380: rename d5/de/d42/d5e/l69 to d5/d10/d44/l80 0 2024-09-21T04:35:07.820 INFO:tasks.workunit.client.1.smithi053.stdout:0/381: write d5/de/f2a [4933429,50134] 0 2024-09-21T04:35:07.822 INFO:tasks.workunit.client.1.smithi053.stdout:0/382: mkdir d5/de/d81 0 2024-09-21T04:35:07.824 INFO:tasks.workunit.client.1.smithi053.stdout:0/383: rename d5/d10/f72 to d5/de/f82 0 2024-09-21T04:35:07.856 INFO:tasks.workunit.client.1.smithi053.stdout:4/421: sync 2024-09-21T04:35:07.859 INFO:tasks.workunit.client.1.smithi053.stdout:4/422: rename d1/dd/de/f2a to d1/dd/de/d14/f84 0 2024-09-21T04:35:07.860 INFO:tasks.workunit.client.1.smithi053.stdout:4/423: creat d1/d2c/f85 x:0 0 0 2024-09-21T04:35:07.862 INFO:tasks.workunit.client.1.smithi053.stdout:4/424: chown d1/dd/de/d14/c1a 3068 1 2024-09-21T04:35:07.865 INFO:tasks.workunit.client.1.smithi053.stdout:4/425: truncate d1/d27/d33/f65 1034462 0 2024-09-21T04:35:07.866 INFO:tasks.workunit.client.1.smithi053.stdout:4/426: dread - d1/dd/de/d14/d3f/d74/f7e zero size 2024-09-21T04:35:07.870 INFO:tasks.workunit.client.1.smithi053.stdout:4/427: dread d1/fc [0,4194304] 0 2024-09-21T04:35:07.871 INFO:tasks.workunit.client.1.smithi053.stdout:4/428: stat d1/d27/d33/d41 0 2024-09-21T04:35:07.872 INFO:tasks.workunit.client.1.smithi053.stdout:4/429: dread - d1/dd/de/d14/d3f/d74/f7e zero size 2024-09-21T04:35:07.876 INFO:tasks.workunit.client.1.smithi053.stdout:4/430: creat d1/d27/f86 x:0 0 0 2024-09-21T04:35:07.877 INFO:tasks.workunit.client.1.smithi053.stdout:4/431: write d1/d27/d2e/f67 [913030,129446] 0 2024-09-21T04:35:07.878 INFO:tasks.workunit.client.1.smithi053.stdout:4/432: truncate d1/d27/f7c 318312 0 2024-09-21T04:35:07.880 INFO:tasks.workunit.client.1.smithi053.stdout:4/433: mkdir d1/dd/de/d14/d20/d5b/d87 0 2024-09-21T04:35:07.881 INFO:tasks.workunit.client.1.smithi053.stdout:4/434: readlink d1/dd/de/d14/l29 0 2024-09-21T04:35:07.882 INFO:tasks.workunit.client.1.smithi053.stdout:4/435: mkdir d1/d27/d53/d88 0 2024-09-21T04:35:07.885 INFO:tasks.workunit.client.1.smithi053.stdout:4/436: creat d1/d27/f89 x:0 0 0 2024-09-21T04:35:07.891 INFO:tasks.workunit.client.1.smithi053.stdout:4/437: dwrite d1/d27/f89 [0,4194304] 0 2024-09-21T04:35:07.895 INFO:tasks.workunit.client.1.smithi053.stdout:4/438: creat d1/d27/d53/d88/f8a x:0 0 0 2024-09-21T04:35:07.997 INFO:tasks.workunit.client.1.smithi053.stdout:6/432: sync 2024-09-21T04:35:07.997 INFO:tasks.workunit.client.1.smithi053.stdout:6/433: link d2/d1b/d3d/d49/de/c2d d2/d1b/d3d/d49/de/d32/d80/d2e/c8e 0 2024-09-21T04:35:08.001 INFO:tasks.workunit.client.1.smithi053.stdout:6/434: dread d2/d1b/d3d/d49/de/d32/d80/f82 [0,4194304] 0 2024-09-21T04:35:08.002 INFO:tasks.workunit.client.1.smithi053.stdout:6/435: stat d2/d1b/f6b 0 2024-09-21T04:35:08.003 INFO:tasks.workunit.client.1.smithi053.stdout:6/436: read d2/d1b/d3d/d49/f18 [9138141,63173] 0 2024-09-21T04:35:08.006 INFO:tasks.workunit.client.1.smithi053.stdout:6/437: creat d2/d1b/d3d/d49/f8f x:0 0 0 2024-09-21T04:35:08.018 INFO:tasks.workunit.client.1.smithi053.stdout:6/438: dread d2/d1b/d3d/d49/de/d32/d80/f61 [0,4194304] 0 2024-09-21T04:35:08.020 INFO:tasks.workunit.client.1.smithi053.stdout:6/439: mkdir d2/d1b/d1e/d27/d90 0 2024-09-21T04:35:08.021 INFO:tasks.workunit.client.1.smithi053.stdout:6/440: dread - d2/d1b/f7d zero size 2024-09-21T04:35:08.021 INFO:tasks.workunit.client.1.smithi053.stdout:6/441: dread - d2/d1b/d3d/d49/f8f zero size 2024-09-21T04:35:08.024 INFO:tasks.workunit.client.1.smithi053.stdout:6/442: getdents d2/d1b 0 2024-09-21T04:35:08.026 INFO:tasks.workunit.client.1.smithi053.stdout:6/443: unlink d2/d1b/d1e/d27/f87 0 2024-09-21T04:35:08.028 INFO:tasks.workunit.client.1.smithi053.stdout:6/444: creat d2/d1b/d1e/d27/d41/f91 x:0 0 0 2024-09-21T04:35:08.032 INFO:tasks.workunit.client.1.smithi053.stdout:5/425: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:7/337: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:2/455: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:1/472: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:8/432: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:4/439: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:0/384: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:9/403: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:3/402: sync 2024-09-21T04:35:08.034 INFO:tasks.workunit.client.1.smithi053.stdout:5/426: chown d0/dd/c84 287933615 1 2024-09-21T04:35:08.038 INFO:tasks.workunit.client.1.smithi053.stdout:1/473: dwrite d7/d9/da/d40/d64/f83 [0,4194304] 0 2024-09-21T04:35:08.126 INFO:tasks.workunit.client.1.smithi053.stdout:7/338: symlink d0/d24/l6d 0 2024-09-21T04:35:08.127 INFO:tasks.workunit.client.1.smithi053.stdout:8/433: mknod d3/df/d49/c96 0 2024-09-21T04:35:08.128 INFO:tasks.workunit.client.1.smithi053.stdout:8/434: chown d3/d5/d1a 58316027 1 2024-09-21T04:35:08.169 INFO:tasks.workunit.client.1.smithi053.stdout:0/385: mknod d5/d6/d53/c83 0 2024-09-21T04:35:08.170 INFO:tasks.workunit.client.1.smithi053.stdout:6/445: truncate d2/d1b/d1e/d2a/d4b/f58 356375 0 2024-09-21T04:35:08.171 INFO:tasks.workunit.client.1.smithi053.stdout:9/404: unlink d0/d2/d9/d1f/l40 0 2024-09-21T04:35:08.172 INFO:tasks.workunit.client.1.smithi053.stdout:4/440: creat d1/dd/de/d14/d3f/d74/f8b x:0 0 0 2024-09-21T04:35:08.179 INFO:tasks.workunit.client.1.smithi053.stdout:0/386: mknod d5/de/d42/c84 0 2024-09-21T04:35:08.184 INFO:tasks.workunit.client.1.smithi053.stdout:6/446: mknod d2/d1b/d3d/d49/de/d32/d80/d2e/d38/c92 0 2024-09-21T04:35:08.184 INFO:tasks.workunit.client.1.smithi053.stdout:0/387: dwrite d5/dd/f34 [0,4194304] 0 2024-09-21T04:35:08.185 INFO:tasks.workunit.client.1.smithi053.stdout:0/388: chown d5/dd/ff 55 1 2024-09-21T04:35:08.186 INFO:tasks.workunit.client.1.smithi053.stdout:0/389: dread - d5/d6/f61 zero size 2024-09-21T04:35:08.188 INFO:tasks.workunit.client.1.smithi053.stdout:9/405: rename d0/d2/c16 to d0/d28/d63/c88 0 2024-09-21T04:35:08.196 INFO:tasks.workunit.client.1.smithi053.stdout:0/390: dwrite d5/f4a [0,4194304] 0 2024-09-21T04:35:08.200 INFO:tasks.workunit.client.1.smithi053.stdout:7/339: mkdir d0/d6e 0 2024-09-21T04:35:08.200 INFO:tasks.workunit.client.1.smithi053.stdout:1/474: write d7/d9/d43/f6d [1408873,56096] 0 2024-09-21T04:35:08.201 INFO:tasks.workunit.client.1.smithi053.stdout:7/340: fdatasync d0/f20 0 2024-09-21T04:35:08.207 INFO:tasks.workunit.client.1.smithi053.stdout:3/403: creat d16/d17/d21/d26/f92 x:0 0 0 2024-09-21T04:35:08.209 INFO:tasks.workunit.client.1.smithi053.stdout:9/406: mkdir d0/d1a/d89 0 2024-09-21T04:35:08.210 INFO:tasks.workunit.client.1.smithi053.stdout:1/475: dwrite d7/d63/f6a [4194304,4194304] 0 2024-09-21T04:35:08.210 INFO:tasks.workunit.client.1.smithi053.stdout:2/456: dwrite d1/d18/f31 [0,4194304] 0 2024-09-21T04:35:08.212 INFO:tasks.workunit.client.1.smithi053.stdout:3/404: chown d16/d17/d21/d42/c5c 2779 1 2024-09-21T04:35:08.216 INFO:tasks.workunit.client.1.smithi053.stdout:5/427: write d0/dd/d30/d51/d52/f59 [1192992,29809] 0 2024-09-21T04:35:08.217 INFO:tasks.workunit.client.1.smithi053.stdout:5/428: readlink d0/d8/d2a/l53 0 2024-09-21T04:35:08.217 INFO:tasks.workunit.client.1.smithi053.stdout:5/429: stat d0/dd/d5c/l5e 0 2024-09-21T04:35:08.218 INFO:tasks.workunit.client.1.smithi053.stdout:5/430: chown d0/dd/d30/d35/c41 213335 1 2024-09-21T04:35:08.237 INFO:tasks.workunit.client.1.smithi053.stdout:0/391: truncate d5/de/d1c/f3d 2563762 0 2024-09-21T04:35:08.239 INFO:tasks.workunit.client.1.smithi053.stdout:5/431: symlink d0/dd/d67/l8d 0 2024-09-21T04:35:08.240 INFO:tasks.workunit.client.1.smithi053.stdout:5/432: write d0/dd/d30/d35/f64 [807171,68714] 0 2024-09-21T04:35:08.241 INFO:tasks.workunit.client.1.smithi053.stdout:0/392: dread d5/dd/d50/f63 [0,4194304] 0 2024-09-21T04:35:08.241 INFO:tasks.workunit.client.1.smithi053.stdout:5/433: chown d0/d8/f31 24 1 2024-09-21T04:35:08.241 INFO:tasks.workunit.client.1.smithi053.stdout:5/434: read d0/f7 [491008,15171] 0 2024-09-21T04:35:08.242 INFO:tasks.workunit.client.1.smithi053.stdout:7/341: mkdir d0/d6f 0 2024-09-21T04:35:08.244 INFO:tasks.workunit.client.1.smithi053.stdout:8/435: getdents d3/d5/d4b/d5c/d86 0 2024-09-21T04:35:08.245 INFO:tasks.workunit.client.1.smithi053.stdout:8/436: chown d3/d5/d4c/c52 319 1 2024-09-21T04:35:08.247 INFO:tasks.workunit.client.1.smithi053.stdout:6/447: dwrite d2/d1b/d3d/f5e [0,4194304] 0 2024-09-21T04:35:08.249 INFO:tasks.workunit.client.1.smithi053.stdout:6/448: write d2/d1b/d1e/d2a/d4b/d77/d65/f7b [426391,113129] 0 2024-09-21T04:35:08.253 INFO:tasks.workunit.client.1.smithi053.stdout:4/441: link d1/dd/de/d14/d20/d5b/c82 d1/dd/c8c 0 2024-09-21T04:35:08.258 INFO:tasks.workunit.client.1.smithi053.stdout:7/342: rmdir d0/d24 39 2024-09-21T04:35:08.258 INFO:tasks.workunit.client.1.smithi053.stdout:7/343: chown d0/d2b/d36/d48/d50/f54 5 1 2024-09-21T04:35:08.259 INFO:tasks.workunit.client.1.smithi053.stdout:5/435: creat d0/dd/d5c/d80/f8e x:0 0 0 2024-09-21T04:35:08.260 INFO:tasks.workunit.client.1.smithi053.stdout:8/437: write d3/d5/f13 [1299776,54863] 0 2024-09-21T04:35:08.261 INFO:tasks.workunit.client.1.smithi053.stdout:8/438: truncate d3/f59 1382285 0 2024-09-21T04:35:08.262 INFO:tasks.workunit.client.1.smithi053.stdout:8/439: read - d3/d28/f7c zero size 2024-09-21T04:35:08.263 INFO:tasks.workunit.client.1.smithi053.stdout:6/449: write d2/d1b/d1e/d27/d41/f64 [2052697,15461] 0 2024-09-21T04:35:08.264 INFO:tasks.workunit.client.1.smithi053.stdout:2/457: link d1/d18/d1f/c29 d1/d18/d1f/d27/d41/d4b/d4d/d89/c9e 0 2024-09-21T04:35:08.266 INFO:tasks.workunit.client.1.smithi053.stdout:4/442: creat d1/d27/d2e/d46/f8d x:0 0 0 2024-09-21T04:35:08.268 INFO:tasks.workunit.client.1.smithi053.stdout:8/440: dwrite d3/d5/d21/f78 [0,4194304] 0 2024-09-21T04:35:08.268 INFO:tasks.workunit.client.1.smithi053.stdout:7/344: truncate d0/d6/f1b 2984628 0 2024-09-21T04:35:08.269 INFO:tasks.workunit.client.1.smithi053.stdout:9/407: link d0/df/c6e d0/d2/d9/c8a 0 2024-09-21T04:35:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.282+0000 7f5ee2632640 1 -- 172.21.15.7:0/2981508637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 msgr2=0x7f5edc1005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.282+0000 7f5ee2632640 1 --2- 172.21.15.7:0/2981508637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 0x7f5edc1005e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009880 tx=0x7f5ecc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.282+0000 7f5ee2632640 1 -- 172.21.15.7:0/2981508637 shutdown_connections 2024-09-21T04:35:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.282+0000 7f5ee2632640 1 --2- 172.21.15.7:0/2981508637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc100b20 0x7f5edc100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.282+0000 7f5ee2632640 1 --2- 172.21.15.7:0/2981508637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 0x7f5edc1005e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.282+0000 7f5ee2632640 1 -- 172.21.15.7:0/2981508637 >> 172.21.15.7:0/2981508637 conn(0x7f5edc0fc010 msgr2=0x7f5edc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:08.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.282+0000 7f5ee2632640 1 -- 172.21.15.7:0/2981508637 shutdown_connections 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.283+0000 7f5ee2632640 1 -- 172.21.15.7:0/2981508637 wait complete. 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.283+0000 7f5ee2632640 1 Processor -- start 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.283+0000 7f5ee2632640 1 -- start start 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5ee2632640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 0x7f5edc199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5ee2632640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc100b20 0x7f5edc199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5ee2632640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc10c970 con 0x7f5edc1001e0 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5ee2632640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc10c7f0 con 0x7f5edc100b20 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5edbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 0x7f5edc199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5edbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 0x7f5edc199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35786/0 (socket says 172.21.15.7:35786) 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5edb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc100b20 0x7f5edc199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5edbfff640 1 -- 172.21.15.7:0/4291578812 learned_addr learned my addr 172.21.15.7:0/4291578812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5edbfff640 1 -- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc100b20 msgr2=0x7f5edc199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5edbfff640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc100b20 0x7f5edc199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5edbfff640 1 -- 172.21.15.7:0/4291578812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc19a220 con 0x7f5edc1001e0 2024-09-21T04:35:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.284+0000 7f5edbfff640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 0x7f5edc199660 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc02aa90 tx=0x7f5ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.285+0000 7f5ed97fa640 1 -- 172.21.15.7:0/4291578812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5ecc03f070 con 0x7f5edc1001e0 2024-09-21T04:35:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.285+0000 7f5ed97fa640 1 -- 172.21.15.7:0/4291578812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ecc004a40 con 0x7f5edc1001e0 2024-09-21T04:35:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.285+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc19ec00 con 0x7f5edc1001e0 2024-09-21T04:35:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.285+0000 7f5ed97fa640 1 -- 172.21.15.7:0/4291578812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5ecc035030 con 0x7f5edc1001e0 2024-09-21T04:35:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.285+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc19f190 con 0x7f5edc1001e0 2024-09-21T04:35:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.286+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc1005e0 con 0x7f5edc1001e0 2024-09-21T04:35:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.287+0000 7f5ed97fa640 1 -- 172.21.15.7:0/4291578812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f5ecc042070 con 0x7f5edc1001e0 2024-09-21T04:35:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.287+0000 7f5ed97fa640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f5ec4074ad0 0x7f5ec4076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:08.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.287+0000 7f5ed97fa640 1 -- 172.21.15.7:0/4291578812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f5ecc0b6e20 con 0x7f5edc1001e0 2024-09-21T04:35:08.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.287+0000 7f5edb7fe640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f5ec4074ad0 0x7f5ec4076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:08.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.288+0000 7f5edb7fe640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f5ec4074ad0 0x7f5ec4076f90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5edc19ab10 tx=0x7f5ed000b390 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:08.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.292+0000 7f5ed97fa640 1 -- 172.21.15.7:0/4291578812 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f5ecc082330 con 0x7f5edc1001e0 2024-09-21T04:35:08.298 INFO:tasks.workunit.client.1.smithi053.stdout:6/450: symlink d2/d1b/d1e/d2a/d4b/d77/l93 0 2024-09-21T04:35:08.298 INFO:tasks.workunit.client.1.smithi053.stdout:1/476: getdents d7/d9/da/d3c 0 2024-09-21T04:35:08.298 INFO:tasks.workunit.client.1.smithi053.stdout:4/443: mkdir d1/d2c/d8e 0 2024-09-21T04:35:08.299 INFO:tasks.workunit.client.1.smithi053.stdout:0/393: rename d5/d6/d21/c2c to d5/c85 0 2024-09-21T04:35:08.299 INFO:tasks.workunit.client.1.smithi053.stdout:5/436: creat d0/dd/d30/d83/f8f x:0 0 0 2024-09-21T04:35:08.299 INFO:tasks.workunit.client.1.smithi053.stdout:8/441: creat d3/d39/f97 x:0 0 0 2024-09-21T04:35:08.300 INFO:tasks.workunit.client.1.smithi053.stdout:7/345: rmdir d0/d2b/d36/d48/d5c 39 2024-09-21T04:35:08.301 INFO:tasks.workunit.client.1.smithi053.stdout:0/394: dread - d5/d6/d53/f77 zero size 2024-09-21T04:35:08.302 INFO:tasks.workunit.client.1.smithi053.stdout:1/477: read d7/d9/da/d4c/f7e [355719,100153] 0 2024-09-21T04:35:08.305 INFO:tasks.workunit.client.1.smithi053.stdout:1/478: dread - d7/d9/d43/d54/d7f/d36/f98 zero size 2024-09-21T04:35:08.305 INFO:tasks.workunit.client.1.smithi053.stdout:1/479: stat d7/l6c 0 2024-09-21T04:35:08.306 INFO:tasks.workunit.client.1.smithi053.stdout:6/451: dread d2/d1b/d3d/f5e [0,4194304] 0 2024-09-21T04:35:08.306 INFO:tasks.workunit.client.1.smithi053.stdout:9/408: dread d0/d1c/d52/d53/f80 [0,4194304] 0 2024-09-21T04:35:08.309 INFO:tasks.workunit.client.1.smithi053.stdout:5/437: unlink d0/d8/db/l2e 0 2024-09-21T04:35:08.311 INFO:tasks.workunit.client.1.smithi053.stdout:8/442: creat d3/df/d14/d16/f98 x:0 0 0 2024-09-21T04:35:08.311 INFO:tasks.workunit.client.1.smithi053.stdout:8/443: readlink d3/d5/d21/l7b 0 2024-09-21T04:35:08.312 INFO:tasks.workunit.client.1.smithi053.stdout:7/346: fsync d0/d2b/d36/d48/d5c/f67 0 2024-09-21T04:35:08.312 INFO:tasks.workunit.client.1.smithi053.stdout:7/347: readlink d0/d2b/d36/d48/l5b 0 2024-09-21T04:35:08.313 INFO:tasks.workunit.client.1.smithi053.stdout:7/348: truncate d0/d2b/f2e 4920026 0 2024-09-21T04:35:08.315 INFO:tasks.workunit.client.1.smithi053.stdout:4/444: mknod d1/dd/d62/c8f 0 2024-09-21T04:35:08.322 INFO:tasks.workunit.client.1.smithi053.stdout:5/438: symlink d0/d8/d2a/l90 0 2024-09-21T04:35:08.323 INFO:tasks.workunit.client.1.smithi053.stdout:5/439: read - d0/dd/d5c/d80/f8a zero size 2024-09-21T04:35:08.324 INFO:tasks.workunit.client.1.smithi053.stdout:0/395: creat d5/de/d81/f86 x:0 0 0 2024-09-21T04:35:08.325 INFO:tasks.workunit.client.1.smithi053.stdout:7/349: truncate d0/d6/ff 265553 0 2024-09-21T04:35:08.327 INFO:tasks.workunit.client.1.smithi053.stdout:1/480: creat d7/d9/da/d40/d64/d74/d91/f9a x:0 0 0 2024-09-21T04:35:08.330 INFO:tasks.workunit.client.1.smithi053.stdout:6/452: creat d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/d8c/f94 x:0 0 0 2024-09-21T04:35:08.331 INFO:tasks.workunit.client.1.smithi053.stdout:5/440: readlink d0/d8/d2a/l43 0 2024-09-21T04:35:08.334 INFO:tasks.workunit.client.1.smithi053.stdout:7/350: read d0/f16 [3081256,62464] 0 2024-09-21T04:35:08.334 INFO:tasks.workunit.client.1.smithi053.stdout:3/405: sync 2024-09-21T04:35:08.335 INFO:tasks.workunit.client.1.smithi053.stdout:4/445: mkdir d1/d27/d90 0 2024-09-21T04:35:08.335 INFO:tasks.workunit.client.1.smithi053.stdout:7/351: write d0/f61 [2255528,118547] 0 2024-09-21T04:35:08.336 INFO:tasks.workunit.client.1.smithi053.stdout:4/446: chown d1/dd/de/f2b 3857342 1 2024-09-21T04:35:08.337 INFO:tasks.workunit.client.1.smithi053.stdout:3/406: write d16/d17/d21/d8b/d7d/d57/f6b [58372,12609] 0 2024-09-21T04:35:08.337 INFO:tasks.workunit.client.1.smithi053.stdout:9/409: link d0/d2/d9/d34/d58/l64 d0/d1a/l8b 0 2024-09-21T04:35:08.338 INFO:tasks.workunit.client.1.smithi053.stdout:1/481: dread d7/d9/da/d11/f5c [0,4194304] 0 2024-09-21T04:35:08.344 INFO:tasks.workunit.client.1.smithi053.stdout:3/407: dwrite d16/d17/d21/d60/f76 [0,4194304] 0 2024-09-21T04:35:08.344 INFO:tasks.workunit.client.1.smithi053.stdout:6/453: dread d2/d1b/d3d/d49/de/f8b [0,4194304] 0 2024-09-21T04:35:08.346 INFO:tasks.workunit.client.1.smithi053.stdout:3/408: write d16/d17/d21/d8b/f66 [741247,104314] 0 2024-09-21T04:35:08.346 INFO:tasks.workunit.client.1.smithi053.stdout:5/441: creat d0/dd/d30/d35/d61/f91 x:0 0 0 2024-09-21T04:35:08.347 INFO:tasks.workunit.client.1.smithi053.stdout:5/442: write d0/dd/f3a [4423341,124381] 0 2024-09-21T04:35:08.351 INFO:tasks.workunit.client.1.smithi053.stdout:7/352: chown d0/d24/d46/f4a 118 1 2024-09-21T04:35:08.355 INFO:tasks.workunit.client.1.smithi053.stdout:9/410: fdatasync d0/f15 0 2024-09-21T04:35:08.356 INFO:tasks.workunit.client.1.smithi053.stdout:9/411: write d0/d76/f7e [163952,87488] 0 2024-09-21T04:35:08.358 INFO:tasks.workunit.client.1.smithi053.stdout:9/412: write d0/d2/d4e/d5c/f78 [728354,8263] 0 2024-09-21T04:35:08.361 INFO:tasks.workunit.client.1.smithi053.stdout:4/447: rename d1/dd/de/d14 to d1/d27/d53/d88/d91 0 2024-09-21T04:35:08.373 INFO:tasks.workunit.client.1.smithi053.stdout:3/409: mknod d16/d17/d21/d42/d43/c93 0 2024-09-21T04:35:08.375 INFO:tasks.workunit.client.1.smithi053.stdout:5/443: truncate d0/d8/f10 362778 0 2024-09-21T04:35:08.375 INFO:tasks.workunit.client.1.smithi053.stdout:6/454: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f8a [0,4194304] 0 2024-09-21T04:35:08.377 INFO:tasks.workunit.client.1.smithi053.stdout:7/353: mkdir d0/d6/d70 0 2024-09-21T04:35:08.381 INFO:tasks.workunit.client.1.smithi053.stdout:9/413: mknod d0/d1a/c8c 0 2024-09-21T04:35:08.383 INFO:tasks.workunit.client.1.smithi053.stdout:4/448: mkdir d1/d27/d53/d88/d91/d20/d73/d92 0 2024-09-21T04:35:08.389 INFO:tasks.workunit.client.1.smithi053.stdout:3/410: mknod d16/d17/d21/d8b/d7d/c94 0 2024-09-21T04:35:08.390 INFO:tasks.workunit.client.1.smithi053.stdout:3/411: write d16/d17/d21/d4e/f83 [2326589,16976] 0 2024-09-21T04:35:08.392 INFO:tasks.workunit.client.1.smithi053.stdout:5/444: unlink d0/d8/l45 0 2024-09-21T04:35:08.393 INFO:tasks.workunit.client.1.smithi053.stdout:0/396: truncate d5/de/d42/f48 1227697 0 2024-09-21T04:35:08.396 INFO:tasks.workunit.client.1.smithi053.stdout:7/354: symlink d0/d2b/d36/d48/d5c/l71 0 2024-09-21T04:35:08.400 INFO:tasks.workunit.client.1.smithi053.stdout:0/397: dwrite d5/de/d1c/f7e [0,4194304] 0 2024-09-21T04:35:08.401 INFO:tasks.workunit.client.1.smithi053.stdout:7/355: dread d0/d2b/d36/f3c [0,4194304] 0 2024-09-21T04:35:08.401 INFO:tasks.workunit.client.1.smithi053.stdout:0/398: write d5/dd/f1a [1053234,79362] 0 2024-09-21T04:35:08.404 INFO:tasks.workunit.client.1.smithi053.stdout:0/399: dread - d5/d6/d21/f33 zero size 2024-09-21T04:35:08.405 INFO:tasks.workunit.client.1.smithi053.stdout:5/445: mkdir d0/dd/d5c/d92 0 2024-09-21T04:35:08.407 INFO:tasks.workunit.client.1.smithi053.stdout:1/482: write d7/d9/da/d11/f20 [262910,17978] 0 2024-09-21T04:35:08.410 INFO:tasks.workunit.client.1.smithi053.stdout:7/356: dwrite d0/f20 [4194304,4194304] 0 2024-09-21T04:35:08.413 INFO:tasks.workunit.client.1.smithi053.stdout:4/449: creat d1/d27/d53/d88/d91/d20/d5b/d87/f93 x:0 0 0 2024-09-21T04:35:08.417 INFO:tasks.workunit.client.1.smithi053.stdout:7/357: dwrite d0/d2b/d36/d48/f6b [0,4194304] 0 2024-09-21T04:35:08.419 INFO:tasks.workunit.client.1.smithi053.stdout:0/400: rmdir d5/d6 39 2024-09-21T04:35:08.426 INFO:tasks.workunit.client.1.smithi053.stdout:2/458: sync 2024-09-21T04:35:08.426 INFO:tasks.workunit.client.1.smithi053.stdout:8/444: sync 2024-09-21T04:35:08.439 INFO:tasks.workunit.client.1.smithi053.stdout:6/455: dwrite d2/d1b/d3d/d49/de/f85 [0,4194304] 0 2024-09-21T04:35:08.441 INFO:tasks.workunit.client.1.smithi053.stdout:6/456: dread - d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/f6a zero size 2024-09-21T04:35:08.444 INFO:tasks.workunit.client.1.smithi053.stdout:8/445: symlink d3/d5/d38/d64/d73/l99 0 2024-09-21T04:35:08.447 INFO:tasks.workunit.client.1.smithi053.stdout:9/414: dwrite d0/d1c/d52/d6d/f87 [0,4194304] 0 2024-09-21T04:35:08.449 INFO:tasks.workunit.client.1.smithi053.stdout:4/450: creat d1/d27/d53/d88/d91/d20/d73/d92/f94 x:0 0 0 2024-09-21T04:35:08.451 INFO:tasks.workunit.client.1.smithi053.stdout:4/451: stat d1/d27/d53/d88/d91/d20/d73/d92 0 2024-09-21T04:35:08.451 INFO:tasks.workunit.client.1.smithi053.stdout:3/412: getdents d16/d17/d21/d26/d49 0 2024-09-21T04:35:08.454 INFO:tasks.workunit.client.1.smithi053.stdout:7/358: link d0/fb d0/d6f/f72 0 2024-09-21T04:35:08.457 INFO:tasks.workunit.client.1.smithi053.stdout:3/413: dwrite d16/d17/d21/d8b/f66 [4194304,4194304] 0 2024-09-21T04:35:08.458 INFO:tasks.workunit.client.1.smithi053.stdout:7/359: dread d0/f20 [4194304,4194304] 0 2024-09-21T04:35:08.459 INFO:tasks.workunit.client.1.smithi053.stdout:5/446: link d0/l11 d0/dd/d30/d35/d61/l93 0 2024-09-21T04:35:08.460 INFO:tasks.workunit.client.1.smithi053.stdout:3/414: dread d16/d17/d21/d4e/d51/f54 [0,4194304] 0 2024-09-21T04:35:08.462 INFO:tasks.workunit.client.1.smithi053.stdout:1/483: creat d7/d9/da/f9b x:0 0 0 2024-09-21T04:35:08.469 INFO:tasks.workunit.client.1.smithi053.stdout:1/484: dwrite d7/d9/d43/d54/d7f/f4f [0,4194304] 0 2024-09-21T04:35:08.471 INFO:tasks.workunit.client.1.smithi053.stdout:9/415: truncate d0/d2/d9/fb 3238766 0 2024-09-21T04:35:08.472 INFO:tasks.workunit.client.1.smithi053.stdout:4/452: rmdir d1/d27/d53/d88 39 2024-09-21T04:35:08.481 INFO:tasks.workunit.client.1.smithi053.stdout:3/415: unlink d16/d1b/f40 0 2024-09-21T04:35:08.483 INFO:tasks.workunit.client.1.smithi053.stdout:5/447: mknod d0/dd/d30/d35/d61/c94 0 2024-09-21T04:35:08.485 INFO:tasks.workunit.client.1.smithi053.stdout:6/457: mknod d2/d1b/d1e/d27/d90/c95 0 2024-09-21T04:35:08.487 INFO:tasks.workunit.client.1.smithi053.stdout:8/446: symlink d3/l9a 0 2024-09-21T04:35:08.487 INFO:tasks.workunit.client.1.smithi053.stdout:8/447: fsync d3/f3a 0 2024-09-21T04:35:08.488 INFO:tasks.workunit.client.1.smithi053.stdout:1/485: mknod d7/d9/d43/d54/c9c 0 2024-09-21T04:35:08.490 INFO:tasks.workunit.client.1.smithi053.stdout:9/416: mknod d0/d2/d9/d34/c8d 0 2024-09-21T04:35:08.491 INFO:tasks.workunit.client.1.smithi053.stdout:9/417: stat d0/d1c/d52/d53/f5e 0 2024-09-21T04:35:08.493 INFO:tasks.workunit.client.1.smithi053.stdout:8/448: dwrite d3/df/d14/d16/f35 [0,4194304] 0 2024-09-21T04:35:08.493 INFO:tasks.workunit.client.1.smithi053.stdout:7/360: creat d0/d6e/f73 x:0 0 0 2024-09-21T04:35:08.495 INFO:tasks.workunit.client.1.smithi053.stdout:7/361: truncate d0/d24/f65 293174 0 2024-09-21T04:35:08.496 INFO:tasks.workunit.client.1.smithi053.stdout:6/458: write d2/d1b/d3d/d49/de/f8b [3626257,31143] 0 2024-09-21T04:35:08.498 INFO:tasks.workunit.client.1.smithi053.stdout:2/459: rename d1/d18/d1f/d22/d43/c46 to d1/d18/d1f/c9f 0 2024-09-21T04:35:08.501 INFO:tasks.workunit.client.1.smithi053.stdout:4/453: unlink d1/d27/d53/d88/d91/d20/f30 0 2024-09-21T04:35:08.505 INFO:tasks.workunit.client.1.smithi053.stdout:4/454: chown d1/d27/d53/d88/d91/d20/d4e/d51/c81 8 1 2024-09-21T04:35:08.506 INFO:tasks.workunit.client.1.smithi053.stdout:1/486: dwrite d7/d9/d43/d54/d7f/d36/f4e [0,4194304] 0 2024-09-21T04:35:08.506 INFO:tasks.workunit.client.1.smithi053.stdout:5/448: mknod d0/d8/c95 0 2024-09-21T04:35:08.510 INFO:tasks.workunit.client.1.smithi053.stdout:0/401: rename d5/de/d1c/f41 to d5/d6/d53/f87 0 2024-09-21T04:35:08.513 INFO:tasks.workunit.client.1.smithi053.stdout:3/416: dwrite d16/f1d [0,4194304] 0 2024-09-21T04:35:08.517 INFO:tasks.workunit.client.1.smithi053.stdout:8/449: symlink d3/d5/d4c/l9b 0 2024-09-21T04:35:08.518 INFO:tasks.workunit.client.1.smithi053.stdout:7/362: mkdir d0/d2b/d36/d64/d74 0 2024-09-21T04:35:08.518 INFO:tasks.workunit.client.1.smithi053.stdout:9/418: dread d0/d28/f3a [0,4194304] 0 2024-09-21T04:35:08.523 INFO:tasks.workunit.client.1.smithi053.stdout:1/487: rename d7/d9/da/f5b to d7/d9/d43/d54/d7f/d59/f9d 0 2024-09-21T04:35:08.525 INFO:tasks.workunit.client.1.smithi053.stdout:1/488: fdatasync d7/f56 0 2024-09-21T04:35:08.526 INFO:tasks.workunit.client.1.smithi053.stdout:7/363: symlink d0/d6e/l75 0 2024-09-21T04:35:08.528 INFO:tasks.workunit.client.1.smithi053.stdout:7/364: stat d0/d6/f1c 0 2024-09-21T04:35:08.528 INFO:tasks.workunit.client.1.smithi053.stdout:1/489: dread - d7/d9/d43/d54/d7f/d36/f98 zero size 2024-09-21T04:35:08.528 INFO:tasks.workunit.client.1.smithi053.stdout:8/450: dwrite d3/d5/d38/f4e [4194304,4194304] 0 2024-09-21T04:35:08.529 INFO:tasks.workunit.client.1.smithi053.stdout:2/460: creat d1/d18/d1f/d27/fa0 x:0 0 0 2024-09-21T04:35:08.532 INFO:tasks.workunit.client.1.smithi053.stdout:1/490: write d7/d9/da/f50 [400544,62086] 0 2024-09-21T04:35:08.533 INFO:tasks.workunit.client.1.smithi053.stdout:6/459: getdents d2/d1b/d3d/d49/de/d32 0 2024-09-21T04:35:08.533 INFO:tasks.workunit.client.1.smithi053.stdout:0/402: dread d5/f15 [0,4194304] 0 2024-09-21T04:35:08.534 INFO:tasks.workunit.client.1.smithi053.stdout:8/451: truncate d3/df/d14/f7f 718513 0 2024-09-21T04:35:08.536 INFO:tasks.workunit.client.1.smithi053.stdout:6/460: read d2/d1b/f5a [1500503,20468] 0 2024-09-21T04:35:08.537 INFO:tasks.workunit.client.1.smithi053.stdout:2/461: fsync d1/d18/d1f/d22/d43/f97 0 2024-09-21T04:35:08.539 INFO:tasks.workunit.client.1.smithi053.stdout:3/417: rename d16/d17/d21/d42/d43/c47 to d16/d17/d21/d26/d3e/c95 0 2024-09-21T04:35:08.539 INFO:tasks.workunit.client.1.smithi053.stdout:8/452: dread d3/f3a [0,4194304] 0 2024-09-21T04:35:08.540 INFO:tasks.workunit.client.1.smithi053.stdout:4/455: getdents d1/d27/d53/d88/d91/d20/d5b 0 2024-09-21T04:35:08.542 INFO:tasks.workunit.client.1.smithi053.stdout:7/365: rename d0/d2b/d36/l55 to d0/d2b/d36/d48/d50/l76 0 2024-09-21T04:35:08.545 INFO:tasks.workunit.client.1.smithi053.stdout:3/418: dread d16/d17/d21/f64 [0,4194304] 0 2024-09-21T04:35:08.545 INFO:tasks.workunit.client.1.smithi053.stdout:2/462: symlink d1/d18/d1f/d27/d58/d5c/d91/la1 0 2024-09-21T04:35:08.545 INFO:tasks.workunit.client.1.smithi053.stdout:5/449: dwrite d0/d8/db/f5f [0,4194304] 0 2024-09-21T04:35:08.546 INFO:tasks.workunit.client.1.smithi053.stdout:0/403: rename d5/d10/f56 to d5/d6/f88 0 2024-09-21T04:35:08.548 INFO:tasks.workunit.client.1.smithi053.stdout:0/404: stat l2 0 2024-09-21T04:35:08.548 INFO:tasks.workunit.client.1.smithi053.stdout:4/456: mkdir d1/dd/d95 0 2024-09-21T04:35:08.550 INFO:tasks.workunit.client.1.smithi053.stdout:8/453: creat d3/f9c x:0 0 0 2024-09-21T04:35:08.550 INFO:tasks.workunit.client.1.smithi053.stdout:6/461: rename d2/d1b/d1e/d2a/d4b/d77/d65 to d2/d1b/d3d/d49/de/d96 0 2024-09-21T04:35:08.551 INFO:tasks.workunit.client.1.smithi053.stdout:3/419: symlink d16/d17/d21/d4e/l96 0 2024-09-21T04:35:08.552 INFO:tasks.workunit.client.1.smithi053.stdout:2/463: mkdir d1/d18/d1f/da2 0 2024-09-21T04:35:08.552 INFO:tasks.workunit.client.1.smithi053.stdout:9/419: dwrite d0/d2/d9/fb [0,4194304] 0 2024-09-21T04:35:08.554 INFO:tasks.workunit.client.1.smithi053.stdout:2/464: readlink d1/d18/d77/l99 0 2024-09-21T04:35:08.554 INFO:tasks.workunit.client.1.smithi053.stdout:6/462: write d2/d1b/d1e/f3f [2196597,58054] 0 2024-09-21T04:35:08.556 INFO:tasks.workunit.client.1.smithi053.stdout:7/366: dread d0/f20 [4194304,4194304] 0 2024-09-21T04:35:08.558 INFO:tasks.workunit.client.1.smithi053.stdout:8/454: mkdir d3/d5/d38/d64/d9d 0 2024-09-21T04:35:08.564 INFO:tasks.workunit.client.1.smithi053.stdout:6/463: creat d2/d1b/d1e/d27/d41/f97 x:0 0 0 2024-09-21T04:35:08.565 INFO:tasks.workunit.client.1.smithi053.stdout:4/457: truncate d1/d27/d53/d88/d91/d20/d4e/f68 798011 0 2024-09-21T04:35:08.567 INFO:tasks.workunit.client.1.smithi053.stdout:6/464: chown d2/l29 6283544 1 2024-09-21T04:35:08.568 INFO:tasks.workunit.client.1.smithi053.stdout:0/405: rename d5/d6/f88 to d5/de/d42/f89 0 2024-09-21T04:35:08.569 INFO:tasks.workunit.client.1.smithi053.stdout:7/367: dwrite d0/d6/f18 [0,4194304] 0 2024-09-21T04:35:08.569 INFO:tasks.workunit.client.1.smithi053.stdout:3/420: mkdir d16/d17/d21/d26/d49/d4d/d53/d82/d97 0 2024-09-21T04:35:08.572 INFO:tasks.workunit.client.1.smithi053.stdout:6/465: read - d2/d1b/d1e/d2a/d78/f84 zero size 2024-09-21T04:35:08.573 INFO:tasks.workunit.client.1.smithi053.stdout:8/455: dwrite d3/df/d14/f7f [0,4194304] 0 2024-09-21T04:35:08.574 INFO:tasks.workunit.client.1.smithi053.stdout:9/420: rename d0/d28/d63 to d0/d1c/d21/d8e 0 2024-09-21T04:35:08.578 INFO:tasks.workunit.client.1.smithi053.stdout:0/406: mknod d5/de/d42/d78/c8a 0 2024-09-21T04:35:08.579 INFO:tasks.workunit.client.1.smithi053.stdout:8/456: fdatasync d3/d28/f79 0 2024-09-21T04:35:08.580 INFO:tasks.workunit.client.1.smithi053.stdout:7/368: symlink d0/l77 0 2024-09-21T04:35:08.581 INFO:tasks.workunit.client.1.smithi053.stdout:9/421: mkdir d0/d1a/d4c/d8f 0 2024-09-21T04:35:08.584 INFO:tasks.workunit.client.1.smithi053.stdout:4/458: dwrite d1/dd/f23 [4194304,4194304] 0 2024-09-21T04:35:08.584 INFO:tasks.workunit.client.1.smithi053.stdout:0/407: symlink d5/d6/d53/l8b 0 2024-09-21T04:35:08.585 INFO:tasks.workunit.client.1.smithi053.stdout:2/465: dwrite d1/fb [0,4194304] 0 2024-09-21T04:35:08.589 INFO:tasks.workunit.client.1.smithi053.stdout:4/459: symlink d1/d27/d33/l96 0 2024-09-21T04:35:08.590 INFO:tasks.workunit.client.1.smithi053.stdout:9/422: truncate d0/d2/fa 3024844 0 2024-09-21T04:35:08.592 INFO:tasks.workunit.client.1.smithi053.stdout:8/457: rename d3/c6f to d3/df/d14/d16/c9e 0 2024-09-21T04:35:08.593 INFO:tasks.workunit.client.1.smithi053.stdout:4/460: fdatasync d1/d27/d53/d88/f8a 0 2024-09-21T04:35:08.594 INFO:tasks.workunit.client.1.smithi053.stdout:6/466: link d2/d1b/d3d/d49/de/d32/d80/d2e/d38/l33 d2/d1b/d3d/d49/de/l98 0 2024-09-21T04:35:08.596 INFO:tasks.workunit.client.1.smithi053.stdout:3/421: dread fd [4194304,4194304] 0 2024-09-21T04:35:08.597 INFO:tasks.workunit.client.1.smithi053.stdout:3/422: fdatasync d16/d17/d21/d26/f92 0 2024-09-21T04:35:08.598 INFO:tasks.workunit.client.1.smithi053.stdout:7/369: getdents d0/d2b/d36/d48 0 2024-09-21T04:35:08.599 INFO:tasks.workunit.client.1.smithi053.stdout:8/458: creat d3/d39/f9f x:0 0 0 2024-09-21T04:35:08.600 INFO:tasks.workunit.client.1.smithi053.stdout:7/370: readlink d0/d2b/d36/d48/d5c/l71 0 2024-09-21T04:35:08.601 INFO:tasks.workunit.client.1.smithi053.stdout:4/461: creat d1/d27/d53/d88/d91/d20/d5b/f97 x:0 0 0 2024-09-21T04:35:08.604 INFO:tasks.workunit.client.1.smithi053.stdout:6/467: fdatasync d2/d1b/d1e/f4f 0 2024-09-21T04:35:08.606 INFO:tasks.workunit.client.1.smithi053.stdout:8/459: mknod d3/d28/ca0 0 2024-09-21T04:35:08.606 INFO:tasks.workunit.client.1.smithi053.stdout:8/460: chown d3/d5/d4c 56173 1 2024-09-21T04:35:08.608 INFO:tasks.workunit.client.1.smithi053.stdout:7/371: rename d0/d6e/f73 to d0/d6e/f78 0 2024-09-21T04:35:08.613 INFO:tasks.workunit.client.1.smithi053.stdout:3/423: rename d16/c44 to d16/d17/d21/d4e/c98 0 2024-09-21T04:35:08.613 INFO:tasks.workunit.client.1.smithi053.stdout:0/408: write d5/d6/f36 [311257,62120] 0 2024-09-21T04:35:08.613 INFO:tasks.workunit.client.1.smithi053.stdout:9/423: getdents d0/d1c/d25 0 2024-09-21T04:35:08.615 INFO:tasks.workunit.client.1.smithi053.stdout:0/409: dread - d5/f39 zero size 2024-09-21T04:35:08.616 INFO:tasks.workunit.client.1.smithi053.stdout:6/468: readlink d2/d1b/d1e/d2a/l40 0 2024-09-21T04:35:08.618 INFO:tasks.workunit.client.1.smithi053.stdout:8/461: dread d3/d5/d38/f3d [0,4194304] 0 2024-09-21T04:35:08.619 INFO:tasks.workunit.client.1.smithi053.stdout:8/462: chown d3/d5/d4c/d8e 1058 1 2024-09-21T04:35:08.621 INFO:tasks.workunit.client.1.smithi053.stdout:2/466: truncate d1/d18/d1f/d27/d41/d4b/f79 1586126 0 2024-09-21T04:35:08.621 INFO:tasks.workunit.client.1.smithi053.stdout:9/424: symlink d0/d1c/d52/d53/l90 0 2024-09-21T04:35:08.622 INFO:tasks.workunit.client.1.smithi053.stdout:2/467: chown d1/d18/d1f/d27/d58/d5c/d9a 34353 1 2024-09-21T04:35:08.622 INFO:tasks.workunit.client.1.smithi053.stdout:7/372: dread d0/f16 [0,4194304] 0 2024-09-21T04:35:08.623 INFO:tasks.workunit.client.1.smithi053.stdout:0/410: mkdir d5/d10/d44/d8c 0 2024-09-21T04:35:08.624 INFO:tasks.workunit.client.1.smithi053.stdout:4/462: truncate d1/d27/d53/d88/d91/f84 3191725 0 2024-09-21T04:35:08.624 INFO:tasks.workunit.client.1.smithi053.stdout:8/463: fdatasync d3/df/f81 0 2024-09-21T04:35:08.625 INFO:tasks.workunit.client.1.smithi053.stdout:3/424: write d16/d1b/f59 [5199812,99805] 0 2024-09-21T04:35:08.626 INFO:tasks.workunit.client.1.smithi053.stdout:9/425: creat d0/d1c/d21/d46/f91 x:0 0 0 2024-09-21T04:35:08.627 INFO:tasks.workunit.client.1.smithi053.stdout:7/373: mknod d0/d6/c79 0 2024-09-21T04:35:08.630 INFO:tasks.workunit.client.1.smithi053.stdout:2/468: creat d1/d18/d1f/d27/d41/d4b/d4d/fa3 x:0 0 0 2024-09-21T04:35:08.631 INFO:tasks.workunit.client.1.smithi053.stdout:2/469: chown d1/d18/d1f/d27/l4e 4931690 1 2024-09-21T04:35:08.631 INFO:tasks.workunit.client.1.smithi053.stdout:2/470: stat d1/fb 0 2024-09-21T04:35:08.632 INFO:tasks.workunit.client.1.smithi053.stdout:2/471: truncate d1/d18/d1f/d22/d2d/f86 399469 0 2024-09-21T04:35:08.633 INFO:tasks.workunit.client.1.smithi053.stdout:4/463: creat d1/d27/d53/d88/d91/d20/d4e/f98 x:0 0 0 2024-09-21T04:35:08.635 INFO:tasks.workunit.client.1.smithi053.stdout:3/425: creat d16/d17/d21/d26/d3e/f99 x:0 0 0 2024-09-21T04:35:08.636 INFO:tasks.workunit.client.1.smithi053.stdout:9/426: chown d0/c30 8865202 1 2024-09-21T04:35:08.636 INFO:tasks.workunit.client.1.smithi053.stdout:9/427: stat d0/d2/d9 0 2024-09-21T04:35:08.638 INFO:tasks.workunit.client.1.smithi053.stdout:6/469: rename d2/d1b/d3d/d49/f18 to d2/d1b/d1e/d2a/f99 0 2024-09-21T04:35:08.639 INFO:tasks.workunit.client.1.smithi053.stdout:2/472: creat d1/d18/d1f/d27/d41/d4b/d78/fa4 x:0 0 0 2024-09-21T04:35:08.640 INFO:tasks.workunit.client.1.smithi053.stdout:6/470: write d2/d1b/d1e/f3f [965261,14464] 0 2024-09-21T04:35:08.641 INFO:tasks.workunit.client.1.smithi053.stdout:4/464: symlink d1/dd/de/l99 0 2024-09-21T04:35:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.639+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5edc062740 con 0x7f5ec4074ad0 2024-09-21T04:35:08.642 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:08 smithi007 ceph-mon[27954]: pgmap v25: 65 pgs: 65 active+clean; 661 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9.3 MiB/s rd, 82 MiB/s wr, 273 op/s 2024-09-21T04:35:08.642 INFO:tasks.workunit.client.1.smithi053.stdout:3/426: creat d16/d17/d21/d26/d49/d4d/d53/f9a x:0 0 0 2024-09-21T04:35:08.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.641+0000 7f5ed97fa640 1 -- 172.21.15.7:0/4291578812 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7f5edc062740 con 0x7f5ec4074ad0 2024-09-21T04:35:08.643 INFO:tasks.workunit.client.1.smithi053.stdout:0/411: creat d5/f8d x:0 0 0 2024-09-21T04:35:08.645 INFO:tasks.workunit.client.1.smithi053.stdout:9/428: dwrite d0/d1c/d21/d46/f91 [0,4194304] 0 2024-09-21T04:35:08.645 INFO:tasks.workunit.client.1.smithi053.stdout:7/374: rename d0/d6/c79 to d0/d2b/d36/d48/d50/c7a 0 2024-09-21T04:35:08.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f5ec4074ad0 msgr2=0x7f5ec4076f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:08.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f5ec4074ad0 0x7f5ec4076f90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5edc19ab10 tx=0x7f5ed000b390 comp rx=0 tx=0).stop 2024-09-21T04:35:08.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 msgr2=0x7f5edc199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 0x7f5edc199660 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc02aa90 tx=0x7f5ecc0046c0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 shutdown_connections 2024-09-21T04:35:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f5ec4074ad0 0x7f5ec4076f90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc100b20 0x7f5edc199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 --2- 172.21.15.7:0/4291578812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5edc1001e0 0x7f5edc199660 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 >> 172.21.15.7:0/4291578812 conn(0x7f5edc0fc010 msgr2=0x7f5edc108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 shutdown_connections 2024-09-21T04:35:08.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.643+0000 7f5ee2632640 1 -- 172.21.15.7:0/4291578812 wait complete. 2024-09-21T04:35:08.648 INFO:tasks.workunit.client.1.smithi053.stdout:4/465: creat d1/d27/d53/d88/d91/d3f/f9a x:0 0 0 2024-09-21T04:35:08.648 INFO:tasks.workunit.client.1.smithi053.stdout:3/427: rmdir d16/d17/d21/d26/d2b 39 2024-09-21T04:35:08.650 INFO:tasks.workunit.client.1.smithi053.stdout:9/429: mkdir d0/d2/d9/d34/d58/d92 0 2024-09-21T04:35:08.652 INFO:tasks.workunit.client.1.smithi053.stdout:6/471: mknod d2/d1b/d1e/d27/c9a 0 2024-09-21T04:35:08.654 INFO:tasks.workunit.client.1.smithi053.stdout:0/412: mknod d5/dd/c8e 0 2024-09-21T04:35:08.655 INFO:tasks.workunit.client.1.smithi053.stdout:0/413: chown d5/de/d1c/f29 72756133 1 2024-09-21T04:35:08.659 INFO:tasks.workunit.client.1.smithi053.stdout:4/466: rename d1/d27/d53/d88/d91/d20/d5b/c82 to d1/d27/d33/c9b 0 2024-09-21T04:35:08.660 INFO:tasks.workunit.client.1.smithi053.stdout:9/430: creat d0/d1c/d52/f93 x:0 0 0 2024-09-21T04:35:08.661 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:35:08.663 INFO:tasks.workunit.client.1.smithi053.stdout:6/472: creat d2/d1b/d3d/d49/de/d32/d80/d2e/d38/f9b x:0 0 0 2024-09-21T04:35:08.664 INFO:tasks.workunit.client.1.smithi053.stdout:7/375: rmdir d0/d2b/d36/d64/d74 0 2024-09-21T04:35:08.665 INFO:tasks.workunit.client.1.smithi053.stdout:4/467: dwrite d1/d27/d53/d88/d91/d20/f6e [0,4194304] 0 2024-09-21T04:35:08.670 INFO:tasks.workunit.client.1.smithi053.stdout:3/428: rename d16/d1b/c23 to d16/d17/d21/d26/d49/d4d/c9b 0 2024-09-21T04:35:08.671 INFO:tasks.workunit.client.1.smithi053.stdout:6/473: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f8a [0,4194304] 0 2024-09-21T04:35:08.678 INFO:tasks.workunit.client.1.smithi053.stdout:6/474: mkdir d2/d1b/d3d/d49/de/d9c 0 2024-09-21T04:35:08.680 INFO:tasks.workunit.client.1.smithi053.stdout:3/429: dwrite d16/d17/d3c/f79 [0,4194304] 0 2024-09-21T04:35:08.683 INFO:tasks.workunit.client.1.smithi053.stdout:0/414: rename d5/dd/l2e to d5/de/l8f 0 2024-09-21T04:35:08.686 INFO:tasks.workunit.client.1.smithi053.stdout:9/431: rename d0/d1c/d21/d46/f66 to d0/d1a/d4c/f94 0 2024-09-21T04:35:08.689 INFO:tasks.workunit.client.1.smithi053.stdout:7/376: rename d0/d24/l26 to d0/d2b/d36/d48/l7b 0 2024-09-21T04:35:08.691 INFO:tasks.workunit.client.1.smithi053.stdout:0/415: link d5/dd/d45/f6c d5/de/d42/d78/f90 0 2024-09-21T04:35:08.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:08 smithi053 ceph-mon[35554]: pgmap v25: 65 pgs: 65 active+clean; 661 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9.3 MiB/s rd, 82 MiB/s wr, 273 op/s 2024-09-21T04:35:08.692 INFO:tasks.workunit.client.1.smithi053.stdout:9/432: unlink d0/d1a/l4d 0 2024-09-21T04:35:08.693 INFO:tasks.workunit.client.1.smithi053.stdout:4/468: rename d1/dd/c26 to d1/d27/d53/d88/d91/d3f/d74/c9c 0 2024-09-21T04:35:08.697 INFO:tasks.workunit.client.1.smithi053.stdout:4/469: chown d1/dd/de/c18 441171107 1 2024-09-21T04:35:08.697 INFO:tasks.workunit.client.1.smithi053.stdout:4/470: fsync d1/d27/d53/f78 0 2024-09-21T04:35:08.701 INFO:tasks.workunit.client.1.smithi053.stdout:9/433: dwrite d0/d2/d9/d34/f65 [0,4194304] 0 2024-09-21T04:35:08.702 INFO:tasks.workunit.client.1.smithi053.stdout:9/434: write d0/d76/f7e [583283,112475] 0 2024-09-21T04:35:08.702 INFO:tasks.workunit.client.1.smithi053.stdout:3/430: dread f9 [0,4194304] 0 2024-09-21T04:35:08.705 INFO:tasks.workunit.client.1.smithi053.stdout:3/431: creat d16/d17/d21/d60/f9c x:0 0 0 2024-09-21T04:35:08.706 INFO:tasks.workunit.client.1.smithi053.stdout:9/435: rename d0/d1c/f83 to d0/d2/d9/f95 0 2024-09-21T04:35:08.710 INFO:tasks.workunit.client.1.smithi053.stdout:3/432: unlink d16/d17/d21/d26/d2b/l2d 0 2024-09-21T04:35:08.711 INFO:tasks.workunit.client.1.smithi053.stdout:9/436: symlink d0/d2/d9/d34/d58/d92/l96 0 2024-09-21T04:35:08.712 INFO:tasks.workunit.client.1.smithi053.stdout:3/433: mkdir d16/d17/d21/d8b/d7d/d9d 0 2024-09-21T04:35:08.713 INFO:tasks.workunit.client.1.smithi053.stdout:6/475: dread d2/d1b/d3d/d49/f75 [0,4194304] 0 2024-09-21T04:35:08.714 INFO:tasks.workunit.client.1.smithi053.stdout:9/437: truncate d0/f4b 1463305 0 2024-09-21T04:35:08.714 INFO:tasks.workunit.client.1.smithi053.stdout:9/438: chown d0/d1a/d89 1 1 2024-09-21T04:35:08.721 INFO:tasks.workunit.client.1.smithi053.stdout:7/377: stat d0/d2b/d36/d48/d50/c57 0 2024-09-21T04:35:08.724 INFO:tasks.workunit.client.1.smithi053.stdout:0/416: truncate d5/de/f35 359276 0 2024-09-21T04:35:08.725 INFO:tasks.workunit.client.1.smithi053.stdout:3/434: link d16/d17/d21/c8c d16/d17/d21/d26/d49/d4d/d53/c9e 0 2024-09-21T04:35:08.726 INFO:tasks.workunit.client.1.smithi053.stdout:0/417: rmdir d5 39 2024-09-21T04:35:08.729 INFO:tasks.workunit.client.1.smithi053.stdout:4/471: dwrite d1/d27/d53/d88/d91/f84 [0,4194304] 0 2024-09-21T04:35:08.731 INFO:tasks.workunit.client.1.smithi053.stdout:3/435: rename d16/d17/d21/d42/c7a to d16/d17/d21/d26/d49/c9f 0 2024-09-21T04:35:08.732 INFO:tasks.workunit.client.1.smithi053.stdout:9/439: getdents d0/d1a/d4c 0 2024-09-21T04:35:08.733 INFO:tasks.workunit.client.1.smithi053.stdout:6/476: write d2/d1b/d3d/d49/f19 [4388215,66287] 0 2024-09-21T04:35:08.734 INFO:tasks.workunit.client.1.smithi053.stdout:0/418: fsync d5/d10/f2b 0 2024-09-21T04:35:08.735 INFO:tasks.workunit.client.1.smithi053.stdout:7/378: getdents d0/d6f 0 2024-09-21T04:35:08.737 INFO:tasks.workunit.client.1.smithi053.stdout:3/436: mkdir d16/d17/d21/d42/da0 0 2024-09-21T04:35:08.737 INFO:tasks.workunit.client.1.smithi053.stdout:0/419: dread - d5/de/d81/f86 zero size 2024-09-21T04:35:08.739 INFO:tasks.workunit.client.1.smithi053.stdout:4/472: dwrite d1/d27/d53/d88/d91/d20/d4e/d51/f7b [0,4194304] 0 2024-09-21T04:35:08.739 INFO:tasks.workunit.client.1.smithi053.stdout:6/477: unlink d2/d1b/d1e/d27/d41/f91 0 2024-09-21T04:35:08.740 INFO:tasks.workunit.client.1.smithi053.stdout:3/437: fsync d16/d17/d21/d4e/f83 0 2024-09-21T04:35:08.741 INFO:tasks.workunit.client.1.smithi053.stdout:6/478: write d2/d1b/f53 [928094,18046] 0 2024-09-21T04:35:08.742 INFO:tasks.workunit.client.1.smithi053.stdout:9/440: rename d0/d1c/d52/d53/f5e to d0/d1a/d4c/d8f/f97 0 2024-09-21T04:35:08.743 INFO:tasks.workunit.client.1.smithi053.stdout:4/473: write d1/d27/d53/d88/d91/d20/f34 [2243136,95314] 0 2024-09-21T04:35:08.744 INFO:tasks.workunit.client.1.smithi053.stdout:6/479: truncate d2/d1b/f7d 753410 0 2024-09-21T04:35:08.744 INFO:tasks.workunit.client.1.smithi053.stdout:0/420: creat d5/de/d81/f91 x:0 0 0 2024-09-21T04:35:08.746 INFO:tasks.workunit.client.1.smithi053.stdout:3/438: chown d16/d17/d21/d26/l4c 5599095 1 2024-09-21T04:35:08.746 INFO:tasks.workunit.client.1.smithi053.stdout:0/421: fsync d5/dd/f34 0 2024-09-21T04:35:08.746 INFO:tasks.workunit.client.1.smithi053.stdout:6/480: read d2/d1b/d1e/f4f [794034,114779] 0 2024-09-21T04:35:08.748 INFO:tasks.workunit.client.1.smithi053.stdout:9/441: mknod d0/d1a/d4c/d8f/c98 0 2024-09-21T04:35:08.750 INFO:tasks.workunit.client.1.smithi053.stdout:9/442: truncate d0/d28/f4a 883617 0 2024-09-21T04:35:08.753 INFO:tasks.workunit.client.1.smithi053.stdout:6/481: rmdir d2/d1b/d1e/d2a/d4b 39 2024-09-21T04:35:08.754 INFO:tasks.workunit.client.1.smithi053.stdout:3/439: creat d16/d17/d21/d26/d49/d4d/d53/fa1 x:0 0 0 2024-09-21T04:35:08.755 INFO:tasks.workunit.client.1.smithi053.stdout:4/474: dread d1/d27/d53/d88/d91/d20/d4e/f68 [0,4194304] 0 2024-09-21T04:35:08.755 INFO:tasks.workunit.client.1.smithi053.stdout:9/443: rmdir d0/d1a/d4c 39 2024-09-21T04:35:08.756 INFO:tasks.workunit.client.1.smithi053.stdout:3/440: write d16/d1b/f2f [521346,57367] 0 2024-09-21T04:35:08.756 INFO:tasks.workunit.client.1.smithi053.stdout:9/444: chown d0/d28/f4a 0 1 2024-09-21T04:35:08.760 INFO:tasks.workunit.client.1.smithi053.stdout:6/482: creat d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f9d x:0 0 0 2024-09-21T04:35:08.761 INFO:tasks.workunit.client.1.smithi053.stdout:7/379: write d0/f20 [5789422,14838] 0 2024-09-21T04:35:08.762 INFO:tasks.workunit.client.1.smithi053.stdout:0/422: rename d5/d6/d53/f77 to d5/f92 0 2024-09-21T04:35:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.760+0000 7f97f8ff6640 1 -- 172.21.15.7:0/1833561555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4100860 msgr2=0x7f97f4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.760+0000 7f97f8ff6640 1 --2- 172.21.15.7:0/1833561555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4100860 0x7f97f4100c60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009880 tx=0x7f97e402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.760+0000 7f97f8ff6640 1 -- 172.21.15.7:0/1833561555 shutdown_connections 2024-09-21T04:35:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.760+0000 7f97f8ff6640 1 --2- 172.21.15.7:0/1833561555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f4101a60 0x7f97f4101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.760+0000 7f97f8ff6640 1 --2- 172.21.15.7:0/1833561555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4100860 0x7f97f4100c60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.760+0000 7f97f8ff6640 1 -- 172.21.15.7:0/1833561555 >> 172.21.15.7:0/1833561555 conn(0x7f97f40fc010 msgr2=0x7f97f40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.761+0000 7f97f8ff6640 1 -- 172.21.15.7:0/1833561555 shutdown_connections 2024-09-21T04:35:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.761+0000 7f97f8ff6640 1 -- 172.21.15.7:0/1833561555 wait complete. 2024-09-21T04:35:08.764 INFO:tasks.workunit.client.1.smithi053.stdout:4/475: rmdir d1/d27/d33 39 2024-09-21T04:35:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97f8ff6640 1 Processor -- start 2024-09-21T04:35:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97f8ff6640 1 -- start start 2024-09-21T04:35:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97f8ff6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f4100860 0x7f97f406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97f8ff6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4101a60 0x7f97f419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97f8ff6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4109680 con 0x7f97f4101a60 2024-09-21T04:35:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97f8ff6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4109500 con 0x7f97f4100860 2024-09-21T04:35:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4101a60 0x7f97f419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4101a60 0x7f97f419c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35798/0 (socket says 172.21.15.7:35798) 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97ebfff640 1 -- 172.21.15.7:0/4191301021 learned_addr learned my addr 172.21.15.7:0/4191301021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.762+0000 7f97f259c640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f4100860 0x7f97f406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97ebfff640 1 -- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f4100860 msgr2=0x7f97f406a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97ebfff640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f4100860 0x7f97f406a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97ebfff640 1 -- 172.21.15.7:0/4191301021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f419c7b0 con 0x7f97f4101a60 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97ebfff640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4101a60 0x7f97f419c1c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00d8d0 tx=0x7f97dc00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97eb7fe640 1 -- 172.21.15.7:0/4191301021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97dc017070 con 0x7f97f4101a60 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f419caa0 con 0x7f97f4101a60 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97eb7fe640 1 -- 172.21.15.7:0/4191301021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f97dc010460 con 0x7f97f4101a60 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97eb7fe640 1 -- 172.21.15.7:0/4191301021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97dc00f5f0 con 0x7f97f4101a60 2024-09-21T04:35:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.763+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f419d030 con 0x7f97f4101a60 2024-09-21T04:35:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.765+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f4100c60 con 0x7f97f4101a60 2024-09-21T04:35:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.766+0000 7f97eb7fe640 1 -- 172.21.15.7:0/4191301021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f97dc009a30 con 0x7f97f4101a60 2024-09-21T04:35:08.769 INFO:tasks.workunit.client.1.smithi053.stdout:3/441: dwrite d16/d17/d21/d4e/f6c [0,4194304] 0 2024-09-21T04:35:08.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.767+0000 7f97eb7fe640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f97e0074ad0 0x7f97e0076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:08.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.767+0000 7f97eb7fe640 1 -- 172.21.15.7:0/4191301021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f97dc096190 con 0x7f97f4101a60 2024-09-21T04:35:08.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.767+0000 7f97f259c640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f97e0074ad0 0x7f97e0076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:08.770 INFO:tasks.workunit.client.1.smithi053.stdout:7/380: unlink d0/d24/d46/c6c 0 2024-09-21T04:35:08.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.768+0000 7f97f259c640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f97e0074ad0 0x7f97e0076f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f97e402aa10 tx=0x7f97e402a9a0 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:08.774 INFO:tasks.workunit.client.1.smithi053.stdout:9/445: dread d0/d2/d9/f43 [0,4194304] 0 2024-09-21T04:35:08.776 INFO:tasks.workunit.client.1.smithi053.stdout:3/442: dwrite f11 [0,4194304] 0 2024-09-21T04:35:08.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:08.773+0000 7f97eb7fe640 1 -- 172.21.15.7:0/4191301021 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f97dc061750 con 0x7f97f4101a60 2024-09-21T04:35:08.779 INFO:tasks.workunit.client.1.smithi053.stdout:0/423: symlink d5/dd/l93 0 2024-09-21T04:35:08.785 INFO:tasks.workunit.client.1.smithi053.stdout:3/443: fsync d16/d17/d21/d4e/d51/f54 0 2024-09-21T04:35:08.785 INFO:tasks.workunit.client.1.smithi053.stdout:6/483: link d2/d1b/d1e/d2a/f99 d2/d1b/d1e/f9e 0 2024-09-21T04:35:08.787 INFO:tasks.workunit.client.1.smithi053.stdout:6/484: write d2/d1b/d1e/d27/d41/f46 [430258,73785] 0 2024-09-21T04:35:08.788 INFO:tasks.workunit.client.1.smithi053.stdout:4/476: rename d1/d27/d53/d88/d91/d3f/d74/f7e to d1/d27/f9d 0 2024-09-21T04:35:08.790 INFO:tasks.workunit.client.1.smithi053.stdout:4/477: stat d1/f2d 0 2024-09-21T04:35:08.791 INFO:tasks.workunit.client.1.smithi053.stdout:3/444: creat d16/d17/d21/d26/d2b/fa2 x:0 0 0 2024-09-21T04:35:08.793 INFO:tasks.workunit.client.1.smithi053.stdout:6/485: mknod d2/d1b/d1e/d27/d90/c9f 0 2024-09-21T04:35:08.797 INFO:tasks.workunit.client.1.smithi053.stdout:0/424: dwrite d5/d47/f5a [0,4194304] 0 2024-09-21T04:35:08.798 INFO:tasks.workunit.client.1.smithi053.stdout:9/446: dread d0/d1c/d25/f54 [0,4194304] 0 2024-09-21T04:35:08.799 INFO:tasks.workunit.client.1.smithi053.stdout:7/381: dread d0/fd [0,4194304] 0 2024-09-21T04:35:08.799 INFO:tasks.workunit.client.1.smithi053.stdout:3/445: link d16/d17/d21/d42/l84 d16/d17/d21/d4e/la3 0 2024-09-21T04:35:08.801 INFO:tasks.workunit.client.1.smithi053.stdout:6/486: creat d2/d1b/d1e/d2a/d4b/d77/d88/fa0 x:0 0 0 2024-09-21T04:35:08.802 INFO:tasks.workunit.client.1.smithi053.stdout:0/425: unlink d5/d47/f5a 0 2024-09-21T04:35:08.804 INFO:tasks.workunit.client.1.smithi053.stdout:4/478: dwrite d1/dd/f23 [8388608,4194304] 0 2024-09-21T04:35:08.804 INFO:tasks.workunit.client.1.smithi053.stdout:9/447: creat d0/d1c/d25/f99 x:0 0 0 2024-09-21T04:35:08.806 INFO:tasks.workunit.client.1.smithi053.stdout:3/446: chown d16/d17/f1e 150141 1 2024-09-21T04:35:08.807 INFO:tasks.workunit.client.1.smithi053.stdout:1/491: sync 2024-09-21T04:35:08.808 INFO:tasks.workunit.client.1.smithi053.stdout:0/426: mkdir d5/de/d42/d78/d94 0 2024-09-21T04:35:08.808 INFO:tasks.workunit.client.1.smithi053.stdout:6/487: chown d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/d8c/f94 527250059 1 2024-09-21T04:35:08.809 INFO:tasks.workunit.client.1.smithi053.stdout:7/382: symlink d0/d2b/d36/d48/d5d/l7c 0 2024-09-21T04:35:08.813 INFO:tasks.workunit.client.1.smithi053.stdout:9/448: creat d0/d1c/d25/f9a x:0 0 0 2024-09-21T04:35:08.815 INFO:tasks.workunit.client.1.smithi053.stdout:6/488: fsync d2/d1b/d1e/f9e 0 2024-09-21T04:35:08.816 INFO:tasks.workunit.client.1.smithi053.stdout:7/383: dread d0/d2b/f43 [0,4194304] 0 2024-09-21T04:35:08.817 INFO:tasks.workunit.client.1.smithi053.stdout:1/492: rename d7/d9/da/d11/c18 to d7/d9/d43/d94/d69/c9e 0 2024-09-21T04:35:08.817 INFO:tasks.workunit.client.1.smithi053.stdout:4/479: dwrite d1/d27/d53/d88/d91/d20/f6f [0,4194304] 0 2024-09-21T04:35:08.819 INFO:tasks.workunit.client.1.smithi053.stdout:3/447: symlink d16/d17/d21/d42/da0/la4 0 2024-09-21T04:35:08.819 INFO:tasks.workunit.client.1.smithi053.stdout:0/427: dwrite d5/d47/f60 [0,4194304] 0 2024-09-21T04:35:08.821 INFO:tasks.workunit.client.1.smithi053.stdout:7/384: write d0/d2b/f41 [3061995,12616] 0 2024-09-21T04:35:08.823 INFO:tasks.workunit.client.1.smithi053.stdout:0/428: chown d5/d10/c71 0 1 2024-09-21T04:35:08.823 INFO:tasks.workunit.client.1.smithi053.stdout:6/489: mknod d2/d1d/ca1 0 2024-09-21T04:35:08.825 INFO:tasks.workunit.client.1.smithi053.stdout:1/493: mkdir d7/d9/da/d40/d64/d74/d91/d9f 0 2024-09-21T04:35:08.827 INFO:tasks.workunit.client.1.smithi053.stdout:4/480: creat d1/d27/d53/d88/d91/d20/d73/d92/f9e x:0 0 0 2024-09-21T04:35:08.827 INFO:tasks.workunit.client.1.smithi053.stdout:0/429: stat d5/d10/d4c/d4d/c7f 0 2024-09-21T04:35:08.829 INFO:tasks.workunit.client.1.smithi053.stdout:1/494: fdatasync f0 0 2024-09-21T04:35:08.829 INFO:tasks.workunit.client.1.smithi053.stdout:6/490: stat d2/d1b/d1e/d2a/d4b/f68 0 2024-09-21T04:35:08.829 INFO:tasks.workunit.client.1.smithi053.stdout:9/449: write d0/d1c/d52/f93 [2577,110727] 0 2024-09-21T04:35:08.831 INFO:tasks.workunit.client.1.smithi053.stdout:9/450: chown d0/d1c/d52/d53/f80 10 1 2024-09-21T04:35:08.831 INFO:tasks.workunit.client.1.smithi053.stdout:3/448: truncate d16/d17/d21/d4e/f83 4094804 0 2024-09-21T04:35:08.832 INFO:tasks.workunit.client.1.smithi053.stdout:4/481: mknod d1/dd/de/c9f 0 2024-09-21T04:35:08.835 INFO:tasks.workunit.client.1.smithi053.stdout:7/385: rename d0/d2b/d36/d48/f63 to d0/d24/d46/f7d 0 2024-09-21T04:35:08.837 INFO:tasks.workunit.client.1.smithi053.stdout:0/430: mkdir d5/d6/d21/d95 0 2024-09-21T04:35:08.837 INFO:tasks.workunit.client.1.smithi053.stdout:0/431: write d5/d6/f65 [3201551,30636] 0 2024-09-21T04:35:08.839 INFO:tasks.workunit.client.1.smithi053.stdout:0/432: write d5/f39 [182418,80248] 0 2024-09-21T04:35:08.839 INFO:tasks.workunit.client.1.smithi053.stdout:6/491: symlink d2/d1b/d1e/d27/d90/la2 0 2024-09-21T04:35:08.840 INFO:tasks.workunit.client.1.smithi053.stdout:9/451: rmdir d0 39 2024-09-21T04:35:08.840 INFO:tasks.workunit.client.1.smithi053.stdout:0/433: truncate d5/de/d81/f91 236259 0 2024-09-21T04:35:08.841 INFO:tasks.workunit.client.1.smithi053.stdout:3/449: mknod d16/d17/d21/d8b/d7d/d57/ca5 0 2024-09-21T04:35:08.842 INFO:tasks.workunit.client.1.smithi053.stdout:1/495: rename d7/d9/l3d to d7/d63/d96/la0 0 2024-09-21T04:35:08.843 INFO:tasks.workunit.client.1.smithi053.stdout:1/496: write d7/d9/da/f2f [190912,117306] 0 2024-09-21T04:35:08.845 INFO:tasks.workunit.client.1.smithi053.stdout:0/434: write d5/d6/d53/f87 [5237562,34744] 0 2024-09-21T04:35:08.846 INFO:tasks.workunit.client.1.smithi053.stdout:7/386: link d0/d2b/f32 d0/d24/f7e 0 2024-09-21T04:35:08.848 INFO:tasks.workunit.client.1.smithi053.stdout:3/450: rename d16/d17/d21/d4e/d51/f54 to d16/d17/d21/d26/d49/fa6 0 2024-09-21T04:35:08.850 INFO:tasks.workunit.client.1.smithi053.stdout:5/450: sync 2024-09-21T04:35:08.852 INFO:tasks.workunit.client.1.smithi053.stdout:9/452: creat d0/d28/f9b x:0 0 0 2024-09-21T04:35:08.853 INFO:tasks.workunit.client.1.smithi053.stdout:1/497: creat d7/d9/d43/d54/fa1 x:0 0 0 2024-09-21T04:35:08.853 INFO:tasks.workunit.client.1.smithi053.stdout:1/498: stat d7/d9/da/f50 0 2024-09-21T04:35:08.857 INFO:tasks.workunit.client.1.smithi053.stdout:4/482: truncate d1/d27/d53/d88/d91/d20/f6e 3010728 0 2024-09-21T04:35:08.859 INFO:tasks.workunit.client.1.smithi053.stdout:5/451: symlink d0/dd/d5c/l96 0 2024-09-21T04:35:08.860 INFO:tasks.workunit.client.1.smithi053.stdout:5/452: truncate d0/d8/db/f78 5142524 0 2024-09-21T04:35:08.860 INFO:tasks.workunit.client.1.smithi053.stdout:5/453: readlink d0/d8/db/l7e 0 2024-09-21T04:35:08.861 INFO:tasks.workunit.client.1.smithi053.stdout:5/454: fdatasync d0/dd/d30/d51/d52/f6c 0 2024-09-21T04:35:08.862 INFO:tasks.workunit.client.1.smithi053.stdout:1/499: creat d7/d9/da/d40/d64/d74/d91/fa2 x:0 0 0 2024-09-21T04:35:08.863 INFO:tasks.workunit.client.1.smithi053.stdout:7/387: creat d0/d6/d70/f7f x:0 0 0 2024-09-21T04:35:08.864 INFO:tasks.workunit.client.1.smithi053.stdout:7/388: stat d0/d6/l25 0 2024-09-21T04:35:08.865 INFO:tasks.workunit.client.1.smithi053.stdout:7/389: write d0/d6/f18 [3622343,106385] 0 2024-09-21T04:35:08.865 INFO:tasks.workunit.client.1.smithi053.stdout:0/435: write d5/d10/f2b [4768549,46387] 0 2024-09-21T04:35:08.867 INFO:tasks.workunit.client.1.smithi053.stdout:0/436: write d5/de/f5b [2638593,5965] 0 2024-09-21T04:35:08.868 INFO:tasks.workunit.client.1.smithi053.stdout:5/455: creat d0/d8/db/d68/f97 x:0 0 0 2024-09-21T04:35:08.868 INFO:tasks.workunit.client.1.smithi053.stdout:6/492: getdents d2/d1b/d3d/d49/de/d32/d80 0 2024-09-21T04:35:08.868 INFO:tasks.workunit.client.1.smithi053.stdout:1/500: rename d7/d9/d43/d94/l57 to d7/d9/d43/d54/d7f/la3 0 2024-09-21T04:35:08.869 INFO:tasks.workunit.client.1.smithi053.stdout:3/451: creat d16/d17/d21/d26/fa7 x:0 0 0 2024-09-21T04:35:08.869 INFO:tasks.workunit.client.1.smithi053.stdout:5/456: chown d0/d66/f82 235 1 2024-09-21T04:35:08.870 INFO:tasks.workunit.client.1.smithi053.stdout:7/390: creat d0/d2b/d36/f80 x:0 0 0 2024-09-21T04:35:08.871 INFO:tasks.workunit.client.1.smithi053.stdout:0/437: chown d5/c1d 0 1 2024-09-21T04:35:08.873 INFO:tasks.workunit.client.1.smithi053.stdout:1/501: fdatasync d7/d9/da/d3c/f46 0 2024-09-21T04:35:08.875 INFO:tasks.workunit.client.1.smithi053.stdout:6/493: write d2/d1b/d3d/d49/de/d32/d80/f82 [628152,129103] 0 2024-09-21T04:35:08.877 INFO:tasks.workunit.client.1.smithi053.stdout:5/457: write d0/d8/f1d [2991059,25451] 0 2024-09-21T04:35:08.877 INFO:tasks.workunit.client.1.smithi053.stdout:3/452: read d16/d17/d21/f6d [122570,46698] 0 2024-09-21T04:35:08.878 INFO:tasks.workunit.client.1.smithi053.stdout:7/391: creat d0/d6/f81 x:0 0 0 2024-09-21T04:35:08.879 INFO:tasks.workunit.client.1.smithi053.stdout:0/438: mkdir d5/d6/d21/d96 0 2024-09-21T04:35:08.879 INFO:tasks.workunit.client.1.smithi053.stdout:7/392: dread - d0/d2b/d36/f80 zero size 2024-09-21T04:35:08.880 INFO:tasks.workunit.client.1.smithi053.stdout:3/453: dread - d16/d17/d21/d26/d49/d4d/f70 zero size 2024-09-21T04:35:08.881 INFO:tasks.workunit.client.1.smithi053.stdout:7/393: truncate d0/d24/d46/f7d 40159 0 2024-09-21T04:35:08.882 INFO:tasks.workunit.client.1.smithi053.stdout:6/494: rmdir d2/d1b/d1e/d2a/d4b 39 2024-09-21T04:35:08.886 INFO:tasks.workunit.client.1.smithi053.stdout:5/458: rename d0/dd/d30/d35/d61/f88 to d0/dd/d30/d35/f98 0 2024-09-21T04:35:08.886 INFO:tasks.workunit.client.1.smithi053.stdout:4/483: link d1/dd/c1b d1/d27/ca0 0 2024-09-21T04:35:08.886 INFO:tasks.workunit.client.1.smithi053.stdout:0/439: dwrite d5/de/f2a [4194304,4194304] 0 2024-09-21T04:35:08.887 INFO:tasks.workunit.client.1.smithi053.stdout:6/495: mknod d2/d1b/d1e/d2a/d78/ca3 0 2024-09-21T04:35:08.889 INFO:tasks.workunit.client.1.smithi053.stdout:6/496: chown d2/d1d/l6d 193929 1 2024-09-21T04:35:08.891 INFO:tasks.workunit.client.1.smithi053.stdout:7/394: chown d0/d2b/d36/d48/d5d/l7c 393 1 2024-09-21T04:35:08.892 INFO:tasks.workunit.client.1.smithi053.stdout:6/497: chown d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/c60 12 1 2024-09-21T04:35:08.893 INFO:tasks.workunit.client.1.smithi053.stdout:1/502: truncate d7/d9/d43/f6e 3124158 0 2024-09-21T04:35:08.894 INFO:tasks.workunit.client.1.smithi053.stdout:4/484: creat d1/d2c/d8e/fa1 x:0 0 0 2024-09-21T04:35:08.895 INFO:tasks.workunit.client.1.smithi053.stdout:5/459: dwrite d0/d8/db/f5f [0,4194304] 0 2024-09-21T04:35:08.895 INFO:tasks.workunit.client.1.smithi053.stdout:3/454: getdents d16/d17 0 2024-09-21T04:35:08.898 INFO:tasks.workunit.client.1.smithi053.stdout:1/503: creat d7/d47/fa4 x:0 0 0 2024-09-21T04:35:08.900 INFO:tasks.workunit.client.1.smithi053.stdout:5/460: symlink d0/dd/d5c/d89/l99 0 2024-09-21T04:35:08.905 INFO:tasks.workunit.client.1.smithi053.stdout:0/440: dwrite d5/de/d42/d5e/f7a [0,4194304] 0 2024-09-21T04:35:08.905 INFO:tasks.workunit.client.1.smithi053.stdout:4/485: dread d1/d27/d53/d88/d91/f84 [0,4194304] 0 2024-09-21T04:35:08.906 INFO:tasks.workunit.client.1.smithi053.stdout:0/441: stat d5/d6/d21/c2d 0 2024-09-21T04:35:08.907 INFO:tasks.workunit.client.1.smithi053.stdout:4/486: chown d1/d27/d53/d88/d91/d3f/l48 19 1 2024-09-21T04:35:08.908 INFO:tasks.workunit.client.1.smithi053.stdout:1/504: rename d7/d9/da/d40/l55 to d7/d47/la5 0 2024-09-21T04:35:08.908 INFO:tasks.workunit.client.1.smithi053.stdout:4/487: chown d1/d27/d2e/l39 7163292 1 2024-09-21T04:35:08.909 INFO:tasks.workunit.client.1.smithi053.stdout:3/455: symlink d16/d17/d21/d8b/d7d/d57/d90/la8 0 2024-09-21T04:35:08.909 INFO:tasks.workunit.client.1.smithi053.stdout:4/488: fdatasync d1/d27/d53/d88/d91/d20/d73/f79 0 2024-09-21T04:35:08.911 INFO:tasks.workunit.client.1.smithi053.stdout:4/489: stat d1/d27/d53/d88/d91/d20/d4e/d51/c81 0 2024-09-21T04:35:08.912 INFO:tasks.workunit.client.1.smithi053.stdout:5/461: creat d0/dd/d30/d35/d61/f9a x:0 0 0 2024-09-21T04:35:08.913 INFO:tasks.workunit.client.1.smithi053.stdout:0/442: readlink d5/d6/l70 0 2024-09-21T04:35:08.917 INFO:tasks.workunit.client.1.smithi053.stdout:5/462: unlink d0/dd/f62 0 2024-09-21T04:35:08.919 INFO:tasks.workunit.client.1.smithi053.stdout:0/443: unlink d5/dd/c25 0 2024-09-21T04:35:08.920 INFO:tasks.workunit.client.1.smithi053.stdout:6/498: dread d2/d1b/d1e/f9e [4194304,4194304] 0 2024-09-21T04:35:08.921 INFO:tasks.workunit.client.1.smithi053.stdout:5/463: mknod d0/dd/d5c/d80/c9b 0 2024-09-21T04:35:08.921 INFO:tasks.workunit.client.1.smithi053.stdout:1/505: dwrite d7/d9/f1b [0,4194304] 0 2024-09-21T04:35:08.923 INFO:tasks.workunit.client.1.smithi053.stdout:1/506: write d7/d9/da/f9b [866505,34332] 0 2024-09-21T04:35:08.927 INFO:tasks.workunit.client.1.smithi053.stdout:5/464: fdatasync d0/dd/f23 0 2024-09-21T04:35:08.927 INFO:tasks.workunit.client.1.smithi053.stdout:1/507: mkdir d7/d9/d43/d54/da6 0 2024-09-21T04:35:08.929 INFO:tasks.workunit.client.1.smithi053.stdout:0/444: link d5/d6/f65 d5/f97 0 2024-09-21T04:35:08.929 INFO:tasks.workunit.client.1.smithi053.stdout:4/490: dread d1/d27/d53/d88/d91/d20/f45 [0,4194304] 0 2024-09-21T04:35:08.930 INFO:tasks.workunit.client.1.smithi053.stdout:1/508: chown d7/d9/da/d11/l42 26023873 1 2024-09-21T04:35:08.930 INFO:tasks.workunit.client.1.smithi053.stdout:6/499: mkdir d2/d1b/d1e/d2a/d4b/d77/d88/da4 0 2024-09-21T04:35:08.932 INFO:tasks.workunit.client.1.smithi053.stdout:6/500: fsync d2/d1b/d1e/d27/d41/f64 0 2024-09-21T04:35:08.932 INFO:tasks.workunit.client.1.smithi053.stdout:6/501: chown d2/d1b/d1e/d27/d90/la2 1161 1 2024-09-21T04:35:08.934 INFO:tasks.workunit.client.1.smithi053.stdout:5/465: rename d0/d8/d2a/f4a to d0/dd/d5c/d89/f9c 0 2024-09-21T04:35:08.934 INFO:tasks.workunit.client.1.smithi053.stdout:5/466: stat d0/dd/d30/d35/f98 0 2024-09-21T04:35:08.935 INFO:tasks.workunit.client.1.smithi053.stdout:5/467: readlink d0/d8/d2a/l90 0 2024-09-21T04:35:08.935 INFO:tasks.workunit.client.1.smithi053.stdout:0/445: unlink d5/de/d42/l6a 0 2024-09-21T04:35:08.936 INFO:tasks.workunit.client.1.smithi053.stdout:5/468: write d0/dd/d30/d35/f7b [339525,93745] 0 2024-09-21T04:35:08.938 INFO:tasks.workunit.client.1.smithi053.stdout:1/509: creat d7/d9/d43/d94/fa7 x:0 0 0 2024-09-21T04:35:08.942 INFO:tasks.workunit.client.1.smithi053.stdout:5/469: unlink d0/dd/d30/d83/f8f 0 2024-09-21T04:35:08.943 INFO:tasks.workunit.client.1.smithi053.stdout:1/510: dwrite d7/d9/da/d11/f33 [0,4194304] 0 2024-09-21T04:35:08.944 INFO:tasks.workunit.client.1.smithi053.stdout:6/502: symlink d2/d1b/d3d/d49/d5c/d8d/la5 0 2024-09-21T04:35:08.947 INFO:tasks.workunit.client.1.smithi053.stdout:0/446: link d5/dd/ff d5/d6/d21/d96/f98 0 2024-09-21T04:35:08.951 INFO:tasks.workunit.client.1.smithi053.stdout:4/491: getdents d1/d2c 0 2024-09-21T04:35:08.953 INFO:tasks.workunit.client.1.smithi053.stdout:1/511: rename d7/d9/da/d40/d64 to d7/da8 0 2024-09-21T04:35:08.955 INFO:tasks.workunit.client.1.smithi053.stdout:6/503: creat d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/d8c/fa6 x:0 0 0 2024-09-21T04:35:08.955 INFO:tasks.workunit.client.1.smithi053.stdout:8/464: sync 2024-09-21T04:35:08.955 INFO:tasks.workunit.client.1.smithi053.stdout:2/473: sync 2024-09-21T04:35:08.956 INFO:tasks.workunit.client.1.smithi053.stdout:0/447: rmdir d5/d47 39 2024-09-21T04:35:08.960 INFO:tasks.workunit.client.1.smithi053.stdout:2/474: mknod d1/d18/d1f/d22/d54/ca5 0 2024-09-21T04:35:08.961 INFO:tasks.workunit.client.1.smithi053.stdout:2/475: readlink d1/d18/d1f/d27/l4e 0 2024-09-21T04:35:08.961 INFO:tasks.workunit.client.1.smithi053.stdout:4/492: creat d1/dd/d95/fa2 x:0 0 0 2024-09-21T04:35:08.961 INFO:tasks.workunit.client.1.smithi053.stdout:2/476: dread - d1/d18/f96 zero size 2024-09-21T04:35:08.962 INFO:tasks.workunit.client.1.smithi053.stdout:2/477: dread - d1/d18/d77/d83/f8d zero size 2024-09-21T04:35:08.963 INFO:tasks.workunit.client.1.smithi053.stdout:0/448: mkdir d5/d10/d4c/d4d/d99 0 2024-09-21T04:35:08.964 INFO:tasks.workunit.client.1.smithi053.stdout:0/449: dread - d5/d10/d4c/f52 zero size 2024-09-21T04:35:08.965 INFO:tasks.workunit.client.1.smithi053.stdout:6/504: unlink d2/d1b/d3d/d49/de/c2d 0 2024-09-21T04:35:08.966 INFO:tasks.workunit.client.1.smithi053.stdout:2/478: rmdir d1/d18/d1f/d27/d58/d5c 39 2024-09-21T04:35:08.967 INFO:tasks.workunit.client.1.smithi053.stdout:6/505: fdatasync d2/d1b/d1e/d27/d41/f46 0 2024-09-21T04:35:08.967 INFO:tasks.workunit.client.1.smithi053.stdout:4/493: dwrite d1/d27/f7c [0,4194304] 0 2024-09-21T04:35:08.969 INFO:tasks.workunit.client.1.smithi053.stdout:0/450: stat d5/de/d42/f48 0 2024-09-21T04:35:08.975 INFO:tasks.workunit.client.1.smithi053.stdout:6/506: dwrite d2/d1b/d3d/d49/de/f8b [0,4194304] 0 2024-09-21T04:35:08.977 INFO:tasks.workunit.client.1.smithi053.stdout:6/507: write d2/d1b/d1e/d27/d41/f97 [687851,98315] 0 2024-09-21T04:35:08.980 INFO:tasks.workunit.client.1.smithi053.stdout:0/451: unlink d5/dd/d50/f63 0 2024-09-21T04:35:08.981 INFO:tasks.workunit.client.1.smithi053.stdout:4/494: dread d1/dd/de/f6d [0,4194304] 0 2024-09-21T04:35:08.983 INFO:tasks.workunit.client.1.smithi053.stdout:6/508: mknod d2/d1b/d3d/d49/de/d96/ca7 0 2024-09-21T04:35:08.985 INFO:tasks.workunit.client.1.smithi053.stdout:0/452: unlink d5/d6/d21/f3f 0 2024-09-21T04:35:08.988 INFO:tasks.workunit.client.1.smithi053.stdout:2/479: rmdir d1/d18/d1f/d27/d58/d5c/d9a 0 2024-09-21T04:35:08.990 INFO:tasks.workunit.client.1.smithi053.stdout:6/509: fdatasync d2/d1b/d3d/d49/de/d32/d80/d2e/d38/f36 0 2024-09-21T04:35:08.992 INFO:tasks.workunit.client.1.smithi053.stdout:5/470: rmdir d0/dd/d30/d83 0 2024-09-21T04:35:08.994 INFO:tasks.workunit.client.1.smithi053.stdout:4/495: creat d1/d27/d33/d41/fa3 x:0 0 0 2024-09-21T04:35:08.997 INFO:tasks.workunit.client.1.smithi053.stdout:2/480: mkdir d1/d18/d1f/d22/d43/da6 0 2024-09-21T04:35:08.997 INFO:tasks.workunit.client.1.smithi053.stdout:2/481: readlink d1/d18/l82 0 2024-09-21T04:35:08.998 INFO:tasks.workunit.client.1.smithi053.stdout:0/453: symlink d5/de/d42/d78/d94/l9a 0 2024-09-21T04:35:08.998 INFO:tasks.workunit.client.1.smithi053.stdout:0/454: fsync d5/de/d81/f91 0 2024-09-21T04:35:09.000 INFO:tasks.workunit.client.1.smithi053.stdout:5/471: creat d0/d8/d2a/f9d x:0 0 0 2024-09-21T04:35:09.001 INFO:tasks.workunit.client.1.smithi053.stdout:6/510: dread d2/d1b/d1e/d2a/d4b/f68 [0,4194304] 0 2024-09-21T04:35:09.006 INFO:tasks.workunit.client.1.smithi053.stdout:5/472: dwrite d0/f3b [0,4194304] 0 2024-09-21T04:35:09.008 INFO:tasks.workunit.client.1.smithi053.stdout:6/511: dwrite d2/d1b/d1e/d27/d41/f64 [0,4194304] 0 2024-09-21T04:35:09.009 INFO:tasks.workunit.client.1.smithi053.stdout:6/512: fdatasync d2/d1b/d1e/d27/d41/f97 0 2024-09-21T04:35:09.010 INFO:tasks.workunit.client.1.smithi053.stdout:6/513: read d2/d1d/f54 [391088,69115] 0 2024-09-21T04:35:09.012 INFO:tasks.workunit.client.1.smithi053.stdout:0/455: creat d5/d6/d21/d95/f9b x:0 0 0 2024-09-21T04:35:09.013 INFO:tasks.workunit.client.1.smithi053.stdout:0/456: truncate d5/de/d81/f86 806366 0 2024-09-21T04:35:09.017 INFO:tasks.workunit.client.1.smithi053.stdout:5/473: rename d0/l1b to d0/dd/d6d/l9e 0 2024-09-21T04:35:09.023 INFO:tasks.workunit.client.1.smithi053.stdout:2/482: link d1/d18/d77/l99 d1/d18/d1f/d27/d41/d4b/d78/la7 0 2024-09-21T04:35:09.025 INFO:tasks.workunit.client.1.smithi053.stdout:5/474: symlink d0/d8/d2a/l9f 0 2024-09-21T04:35:09.025 INFO:tasks.workunit.client.1.smithi053.stdout:1/512: link d7/d9/d43/d54/d7f/l8c d7/d47/d79/d99/la9 0 2024-09-21T04:35:09.027 INFO:tasks.workunit.client.1.smithi053.stdout:6/514: mkdir d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8 0 2024-09-21T04:35:09.028 INFO:tasks.workunit.client.1.smithi053.stdout:2/483: creat d1/d18/d1f/d9d/fa8 x:0 0 0 2024-09-21T04:35:09.029 INFO:tasks.workunit.client.1.smithi053.stdout:2/484: write d1/f8c [322190,23575] 0 2024-09-21T04:35:09.029 INFO:tasks.workunit.client.1.smithi053.stdout:2/485: readlink d1/d18/l7d 0 2024-09-21T04:35:09.031 INFO:tasks.workunit.client.1.smithi053.stdout:5/475: creat d0/dd/d30/d51/fa0 x:0 0 0 2024-09-21T04:35:09.032 INFO:tasks.workunit.client.1.smithi053.stdout:5/476: fsync d0/dd/d48/f60 0 2024-09-21T04:35:09.032 INFO:tasks.workunit.client.1.smithi053.stdout:2/486: mknod d1/d18/d1f/d9d/ca9 0 2024-09-21T04:35:09.036 INFO:tasks.workunit.client.1.smithi053.stdout:5/477: symlink d0/dd/d48/d8c/la1 0 2024-09-21T04:35:09.037 INFO:tasks.workunit.client.1.smithi053.stdout:2/487: creat d1/d18/d77/d9b/faa x:0 0 0 2024-09-21T04:35:09.039 INFO:tasks.workunit.client.1.smithi053.stdout:5/478: creat d0/dd/d5c/d80/fa2 x:0 0 0 2024-09-21T04:35:09.042 INFO:tasks.workunit.client.1.smithi053.stdout:5/479: chown d0/dd/d5c/l7d 39194 1 2024-09-21T04:35:09.045 INFO:tasks.workunit.client.1.smithi053.stdout:5/480: write d0/dd/d48/f54 [61066,60250] 0 2024-09-21T04:35:09.048 INFO:tasks.workunit.client.1.smithi053.stdout:6/515: dwrite d2/d1b/d1e/f9e [0,4194304] 0 2024-09-21T04:35:09.049 INFO:tasks.workunit.client.1.smithi053.stdout:1/513: dwrite d7/d9/da/f86 [0,4194304] 0 2024-09-21T04:35:09.049 INFO:tasks.workunit.client.1.smithi053.stdout:5/481: chown d0/dd/d5c/d89/l99 3563 1 2024-09-21T04:35:09.051 INFO:tasks.workunit.client.1.smithi053.stdout:6/516: write d2/d1b/f6b [653438,97079] 0 2024-09-21T04:35:09.051 INFO:tasks.workunit.client.1.smithi053.stdout:1/514: fdatasync f0 0 2024-09-21T04:35:09.053 INFO:tasks.workunit.client.1.smithi053.stdout:2/488: dwrite d1/d18/d1f/d27/f55 [0,4194304] 0 2024-09-21T04:35:09.053 INFO:tasks.workunit.client.1.smithi053.stdout:5/482: creat d0/dd/d67/fa3 x:0 0 0 2024-09-21T04:35:09.054 INFO:tasks.workunit.client.1.smithi053.stdout:1/515: readlink d7/d47/d79/d99/la9 0 2024-09-21T04:35:09.058 INFO:tasks.workunit.client.1.smithi053.stdout:2/489: truncate d1/d18/d1f/d22/d54/f85 777668 0 2024-09-21T04:35:09.058 INFO:tasks.workunit.client.1.smithi053.stdout:1/516: creat d7/da8/d74/d75/faa x:0 0 0 2024-09-21T04:35:09.060 INFO:tasks.workunit.client.1.smithi053.stdout:5/483: creat d0/fa4 x:0 0 0 2024-09-21T04:35:09.063 INFO:tasks.workunit.client.1.smithi053.stdout:5/484: chown d0/d8/d2a/l3c 107806005 1 2024-09-21T04:35:09.063 INFO:tasks.workunit.client.1.smithi053.stdout:2/490: truncate d1/d18/d1f/d9d/fa8 920014 0 2024-09-21T04:35:09.065 INFO:tasks.workunit.client.1.smithi053.stdout:6/517: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/d38/f9b [0,4194304] 0 2024-09-21T04:35:09.069 INFO:tasks.workunit.client.1.smithi053.stdout:5/485: creat d0/dd/d30/d35/d61/fa5 x:0 0 0 2024-09-21T04:35:09.072 INFO:tasks.workunit.client.1.smithi053.stdout:5/486: creat d0/dd/d6d/fa6 x:0 0 0 2024-09-21T04:35:09.073 INFO:tasks.workunit.client.1.smithi053.stdout:5/487: dread - d0/dd/d30/d35/d61/f91 zero size 2024-09-21T04:35:09.074 INFO:tasks.workunit.client.1.smithi053.stdout:2/491: rename d1/d18/d1f/d22/f32 to d1/d18/d1f/d27/d58/fab 0 2024-09-21T04:35:09.075 INFO:tasks.workunit.client.1.smithi053.stdout:5/488: creat d0/dd/d30/d51/d52/fa7 x:0 0 0 2024-09-21T04:35:09.076 INFO:tasks.workunit.client.1.smithi053.stdout:2/492: write d1/d18/d1f/f4a [4603797,37451] 0 2024-09-21T04:35:09.078 INFO:tasks.workunit.client.1.smithi053.stdout:2/493: write d1/f8c [451446,51795] 0 2024-09-21T04:35:09.081 INFO:tasks.workunit.client.1.smithi053.stdout:6/518: rename d2/d1b/d1e/d2a/d4b/d77/d88/fa0 to d2/d1b/d3d/d49/fa9 0 2024-09-21T04:35:09.082 INFO:tasks.workunit.client.1.smithi053.stdout:5/489: symlink d0/dd/la8 0 2024-09-21T04:35:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.095+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f97f4106010 con 0x7f97e0074ad0 2024-09-21T04:35:09.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.097+0000 7f97eb7fe640 1 -- 172.21.15.7:0/4191301021 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7f97f4106010 con 0x7f97e0074ad0 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f97e0074ad0 msgr2=0x7f97e0076f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f97e0074ad0 0x7f97e0076f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f97e402aa10 tx=0x7f97e402a9a0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4101a60 msgr2=0x7f97f419c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4101a60 0x7f97f419c1c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00d8d0 tx=0x7f97dc00dd90 comp rx=0 tx=0).stop 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 shutdown_connections 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f97e0074ad0 0x7f97e0076f90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97f4101a60 0x7f97f419c1c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 --2- 172.21.15.7:0/4191301021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f4100860 0x7f97f406a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 >> 172.21.15.7:0/4191301021 conn(0x7f97f40fc010 msgr2=0x7f97f40fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:09.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 shutdown_connections 2024-09-21T04:35:09.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.099+0000 7f97f8ff6640 1 -- 172.21.15.7:0/4191301021 wait complete. 2024-09-21T04:35:09.122 INFO:tasks.workunit.client.1.smithi053.stdout:1/517: truncate d7/d9/d43/f6d 2491143 0 2024-09-21T04:35:09.123 INFO:tasks.workunit.client.1.smithi053.stdout:2/494: mkdir d1/d18/d1f/da2/dac 0 2024-09-21T04:35:09.123 INFO:tasks.workunit.client.1.smithi053.stdout:5/490: chown d0/dd/d30/d51/d52/c85 722034449 1 2024-09-21T04:35:09.124 INFO:tasks.workunit.client.1.smithi053.stdout:5/491: write d0/dd/d67/fa3 [600832,35246] 0 2024-09-21T04:35:09.125 INFO:tasks.workunit.client.1.smithi053.stdout:2/495: symlink d1/d18/d1f/d9d/lad 0 2024-09-21T04:35:09.127 INFO:tasks.workunit.client.1.smithi053.stdout:1/518: mknod d7/d9/d43/d54/d7f/d36/d81/cab 0 2024-09-21T04:35:09.127 INFO:tasks.workunit.client.1.smithi053.stdout:2/496: truncate d1/d18/d1f/d22/d54/f85 1598672 0 2024-09-21T04:35:09.131 INFO:tasks.workunit.client.1.smithi053.stdout:5/492: dwrite d0/d8/db/f19 [4194304,4194304] 0 2024-09-21T04:35:09.131 INFO:tasks.workunit.client.1.smithi053.stdout:5/493: fsync d0/dd/f23 0 2024-09-21T04:35:09.132 INFO:tasks.workunit.client.1.smithi053.stdout:5/494: chown d0/d8/d2a/l43 12 1 2024-09-21T04:35:09.133 INFO:tasks.workunit.client.1.smithi053.stdout:5/495: write d0/dd/f3a [4091123,41299] 0 2024-09-21T04:35:09.136 INFO:tasks.workunit.client.1.smithi053.stdout:5/496: chown d0/dd/d67/l87 835 1 2024-09-21T04:35:09.137 INFO:tasks.workunit.client.1.smithi053.stdout:5/497: truncate d0/dd/d30/f6f 933088 0 2024-09-21T04:35:09.138 INFO:tasks.workunit.client.1.smithi053.stdout:1/519: truncate d7/d9/d43/d94/f51 3131394 0 2024-09-21T04:35:09.139 INFO:tasks.workunit.client.1.smithi053.stdout:1/520: fdatasync d7/d9/da/f2f 0 2024-09-21T04:35:09.140 INFO:tasks.workunit.client.1.smithi053.stdout:5/498: creat d0/dd/d48/fa9 x:0 0 0 2024-09-21T04:35:09.141 INFO:tasks.workunit.client.1.smithi053.stdout:5/499: write d0/dd/d30/d51/d52/f6c [894856,91033] 0 2024-09-21T04:35:09.150 INFO:tasks.workunit.client.1.smithi053.stdout:5/500: dwrite d0/d8/d2a/f9d [0,4194304] 0 2024-09-21T04:35:09.153 INFO:tasks.workunit.client.1.smithi053.stdout:5/501: write d0/dd/d30/d51/fa0 [616689,109337] 0 2024-09-21T04:35:09.161 INFO:tasks.workunit.client.1.smithi053.stdout:1/521: dread d7/d9/da/d11/f26 [0,4194304] 0 2024-09-21T04:35:09.162 INFO:tasks.workunit.client.1.smithi053.stdout:5/502: dwrite d0/dd/d67/fa3 [0,4194304] 0 2024-09-21T04:35:09.164 INFO:tasks.workunit.client.1.smithi053.stdout:5/503: write d0/d8/f1d [360140,101024] 0 2024-09-21T04:35:09.166 INFO:tasks.workunit.client.1.smithi053.stdout:1/522: mkdir d7/d63/d96/dac 0 2024-09-21T04:35:09.169 INFO:tasks.workunit.client.1.smithi053.stdout:9/453: sync 2024-09-21T04:35:09.169 INFO:tasks.workunit.client.1.smithi053.stdout:9/454: fsync d0/d1a/f81 0 2024-09-21T04:35:09.172 INFO:tasks.workunit.client.1.smithi053.stdout:5/504: getdents d0/d8/db 0 2024-09-21T04:35:09.172 INFO:tasks.workunit.client.1.smithi053.stdout:5/505: dread - d0/dd/d6d/fa6 zero size 2024-09-21T04:35:09.174 INFO:tasks.workunit.client.1.smithi053.stdout:9/455: unlink d0/d28/f4a 0 2024-09-21T04:35:09.176 INFO:tasks.workunit.client.1.smithi053.stdout:9/456: chown d0/d1c/d21/l33 0 1 2024-09-21T04:35:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.202+0000 7fd945717640 1 -- 172.21.15.7:0/4108563341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940102b70 msgr2=0x7fd940102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:09.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.202+0000 7fd945717640 1 --2- 172.21.15.7:0/4108563341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940102b70 0x7fd940102ff0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd934009880 tx=0x7fd93402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.203+0000 7fd945717640 1 -- 172.21.15.7:0/4108563341 shutdown_connections 2024-09-21T04:35:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.203+0000 7fd945717640 1 --2- 172.21.15.7:0/4108563341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940102b70 0x7fd940102ff0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.203+0000 7fd945717640 1 --2- 172.21.15.7:0/4108563341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd940101970 0x7fd940101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.203+0000 7fd945717640 1 -- 172.21.15.7:0/4108563341 >> 172.21.15.7:0/4108563341 conn(0x7fd9400fd140 msgr2=0x7fd9400ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.203+0000 7fd945717640 1 -- 172.21.15.7:0/4108563341 shutdown_connections 2024-09-21T04:35:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.203+0000 7fd945717640 1 -- 172.21.15.7:0/4108563341 wait complete. 2024-09-21T04:35:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.203+0000 7fd945717640 1 Processor -- start 2024-09-21T04:35:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd945717640 1 -- start start 2024-09-21T04:35:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd945717640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940101970 0x7fd940075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd945717640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd940102b70 0x7fd9400745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd945717640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd94010a7b0 con 0x7fd940101970 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd945717640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd94010a630 con 0x7fd940102b70 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd93effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940101970 0x7fd940075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd93effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940101970 0x7fd940075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35812/0 (socket says 172.21.15.7:35812) 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd93effd640 1 -- 172.21.15.7:0/3797161708 learned_addr learned my addr 172.21.15.7:0/3797161708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.204+0000 7fd93e7fc640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd940102b70 0x7fd9400745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd93effd640 1 -- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd940102b70 msgr2=0x7fd9400745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd93effd640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd940102b70 0x7fd9400745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd93effd640 1 -- 172.21.15.7:0/3797161708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd940074b20 con 0x7fd940101970 2024-09-21T04:35:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd93effd640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940101970 0x7fd940075f20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd93000d8d0 tx=0x7fd93000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd927fff640 1 -- 172.21.15.7:0/3797161708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd930017070 con 0x7fd940101970 2024-09-21T04:35:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd927fff640 1 -- 172.21.15.7:0/3797161708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd93000be10 con 0x7fd940101970 2024-09-21T04:35:09.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd940074e10 con 0x7fd940101970 2024-09-21T04:35:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd927fff640 1 -- 172.21.15.7:0/3797161708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd93000f3c0 con 0x7fd940101970 2024-09-21T04:35:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.205+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd940075370 con 0x7fd940101970 2024-09-21T04:35:09.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.207+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd940101d70 con 0x7fd940101970 2024-09-21T04:35:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.207+0000 7fd927fff640 1 -- 172.21.15.7:0/3797161708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7fd93001a070 con 0x7fd940101970 2024-09-21T04:35:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.208+0000 7fd927fff640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd928074a30 0x7fd928076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.208+0000 7fd927fff640 1 -- 172.21.15.7:0/3797161708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7fd930095ee0 con 0x7fd940101970 2024-09-21T04:35:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.208+0000 7fd93e7fc640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd928074a30 0x7fd928076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:09.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.209+0000 7fd93e7fc640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd928074a30 0x7fd928076ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd934000c00 tx=0x7fd9340058d0 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.217+0000 7fd927fff640 1 -- 172.21.15.7:0/3797161708 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fd930061410 con 0x7fd940101970 2024-09-21T04:35:09.233 INFO:tasks.workunit.client.1.smithi053.stdout:5/506: dwrite d0/dd/d5c/d80/f8e [0,4194304] 0 2024-09-21T04:35:09.234 INFO:tasks.workunit.client.1.smithi053.stdout:9/457: symlink d0/d2/d2b/l9c 0 2024-09-21T04:35:09.235 INFO:tasks.workunit.client.1.smithi053.stdout:5/507: chown d0/fa4 31917 1 2024-09-21T04:35:09.238 INFO:tasks.workunit.client.1.smithi053.stdout:1/523: dwrite d7/d9/d43/d94/f51 [0,4194304] 0 2024-09-21T04:35:09.238 INFO:tasks.workunit.client.1.smithi053.stdout:9/458: chown d0/d1a/d4c/d8f 3572662 1 2024-09-21T04:35:09.238 INFO:tasks.workunit.client.1.smithi053.stdout:1/524: chown d7/d9/d43/d54/d7f/d59/f9d 187908194 1 2024-09-21T04:35:09.240 INFO:tasks.workunit.client.1.smithi053.stdout:5/508: creat d0/dd/d48/faa x:0 0 0 2024-09-21T04:35:09.242 INFO:tasks.workunit.client.1.smithi053.stdout:9/459: chown d0/d1a/l8b 511114 1 2024-09-21T04:35:09.246 INFO:tasks.workunit.client.1.smithi053.stdout:1/525: rename d7/d9/da/d40/d58/c60 to d7/d9/da/d40/cad 0 2024-09-21T04:35:09.248 INFO:tasks.workunit.client.1.smithi053.stdout:5/509: mknod d0/d66/cab 0 2024-09-21T04:35:09.249 INFO:tasks.workunit.client.1.smithi053.stdout:9/460: fdatasync d0/d28/f3a 0 2024-09-21T04:35:09.250 INFO:tasks.workunit.client.1.smithi053.stdout:1/526: mkdir d7/d47/dae 0 2024-09-21T04:35:09.256 INFO:tasks.workunit.client.1.smithi053.stdout:1/527: rename d7/d9/da/d11/f26 to d7/d9/da/d3c/faf 0 2024-09-21T04:35:09.259 INFO:tasks.workunit.client.1.smithi053.stdout:5/510: link d0/d66/cab d0/dd/d30/d35/cac 0 2024-09-21T04:35:09.262 INFO:tasks.workunit.client.1.smithi053.stdout:5/511: rmdir d0/dd/d5c/d89 39 2024-09-21T04:35:09.263 INFO:tasks.workunit.client.1.smithi053.stdout:1/528: mkdir d7/d47/d79/d99/db0 0 2024-09-21T04:35:09.268 INFO:tasks.workunit.client.1.smithi053.stdout:5/512: dwrite d0/d8/db/d68/f8b [0,4194304] 0 2024-09-21T04:35:09.270 INFO:tasks.workunit.client.1.smithi053.stdout:1/529: truncate d7/da8/d74/d75/f88 412584 0 2024-09-21T04:35:09.272 INFO:tasks.workunit.client.1.smithi053.stdout:5/513: mkdir d0/dd/d48/d8c/dad 0 2024-09-21T04:35:09.272 INFO:tasks.workunit.client.1.smithi053.stdout:5/514: dread - d0/d8/f69 zero size 2024-09-21T04:35:09.273 INFO:tasks.workunit.client.1.smithi053.stdout:5/515: readlink d0/d8/d2a/l3c 0 2024-09-21T04:35:09.277 INFO:tasks.workunit.client.1.smithi053.stdout:7/395: sync 2024-09-21T04:35:09.277 INFO:tasks.workunit.client.1.smithi053.stdout:3/456: sync 2024-09-21T04:35:09.278 INFO:tasks.workunit.client.1.smithi053.stdout:7/396: readlink d0/d2b/l4c 0 2024-09-21T04:35:09.279 INFO:tasks.workunit.client.1.smithi053.stdout:7/397: read d0/d2b/f43 [671816,37530] 0 2024-09-21T04:35:09.283 INFO:tasks.workunit.client.1.smithi053.stdout:3/457: mkdir d16/d17/d21/d8b/d7d/d57/da9 0 2024-09-21T04:35:09.286 INFO:tasks.workunit.client.1.smithi053.stdout:7/398: mknod d0/d2b/c82 0 2024-09-21T04:35:09.291 INFO:tasks.workunit.client.1.smithi053.stdout:7/399: dwrite d0/f44 [0,4194304] 0 2024-09-21T04:35:09.294 INFO:tasks.workunit.client.1.smithi053.stdout:5/516: getdents d0/dd/d5c/d89 0 2024-09-21T04:35:09.295 INFO:tasks.workunit.client.1.smithi053.stdout:3/458: getdents d16/d17/d21/d26/d71 0 2024-09-21T04:35:09.297 INFO:tasks.workunit.client.1.smithi053.stdout:5/517: creat d0/d8/d2a/fae x:0 0 0 2024-09-21T04:35:09.300 INFO:tasks.workunit.client.1.smithi053.stdout:3/459: getdents d16/d17/d21/d26/d2b 0 2024-09-21T04:35:09.302 INFO:tasks.workunit.client.1.smithi053.stdout:5/518: getdents d0/dd/d30/d51/d52 0 2024-09-21T04:35:09.303 INFO:tasks.workunit.client.1.smithi053.stdout:3/460: creat d16/d17/d21/d26/d2b/faa x:0 0 0 2024-09-21T04:35:09.304 INFO:tasks.workunit.client.1.smithi053.stdout:5/519: readlink d0/l11 0 2024-09-21T04:35:09.304 INFO:tasks.workunit.client.1.smithi053.stdout:5/520: dread - d0/d8/f69 zero size 2024-09-21T04:35:09.305 INFO:tasks.workunit.client.1.smithi053.stdout:3/461: mknod d16/d17/d21/cab 0 2024-09-21T04:35:09.307 INFO:tasks.workunit.client.1.smithi053.stdout:3/462: chown d16/l32 196 1 2024-09-21T04:35:09.311 INFO:tasks.workunit.client.1.smithi053.stdout:3/463: rename d16/d1b/f27 to d16/d17/d21/d8b/d7d/d57/fac 0 2024-09-21T04:35:09.312 INFO:tasks.workunit.client.1.smithi053.stdout:5/521: dread d0/f2f [0,4194304] 0 2024-09-21T04:35:09.313 INFO:tasks.workunit.client.1.smithi053.stdout:3/464: dread d16/d17/d21/f64 [0,4194304] 0 2024-09-21T04:35:09.313 INFO:tasks.workunit.client.1.smithi053.stdout:5/522: rmdir d0/dd/d30/d35/d61 39 2024-09-21T04:35:09.314 INFO:tasks.workunit.client.1.smithi053.stdout:5/523: chown d0/dd/d6d/fa6 0 1 2024-09-21T04:35:09.314 INFO:tasks.workunit.client.1.smithi053.stdout:3/465: dread - d16/d17/d21/d26/d2b/fa2 zero size 2024-09-21T04:35:09.317 INFO:tasks.workunit.client.1.smithi053.stdout:5/524: getdents d0/dd/d30/d51/d52 0 2024-09-21T04:35:09.318 INFO:tasks.workunit.client.1.smithi053.stdout:5/525: write d0/dd/d30/d35/f7b [1212837,26057] 0 2024-09-21T04:35:09.318 INFO:tasks.workunit.client.1.smithi053.stdout:5/526: stat d0/dd/d5c/l96 0 2024-09-21T04:35:09.319 INFO:tasks.workunit.client.1.smithi053.stdout:5/527: dread - d0/d8/d2a/f57 zero size 2024-09-21T04:35:09.321 INFO:tasks.workunit.client.1.smithi053.stdout:5/528: mknod d0/dd/d6d/caf 0 2024-09-21T04:35:09.327 INFO:tasks.workunit.client.1.smithi053.stdout:8/465: sync 2024-09-21T04:35:09.327 INFO:tasks.workunit.client.1.smithi053.stdout:0/457: sync 2024-09-21T04:35:09.328 INFO:tasks.workunit.client.1.smithi053.stdout:4/496: sync 2024-09-21T04:35:09.328 INFO:tasks.workunit.client.1.smithi053.stdout:0/458: readlink d5/de/d42/l54 0 2024-09-21T04:35:09.329 INFO:tasks.workunit.client.1.smithi053.stdout:5/529: getdents d0/dd/d30/d35 0 2024-09-21T04:35:09.333 INFO:tasks.workunit.client.1.smithi053.stdout:5/530: rmdir d0/dd/d30/d51 39 2024-09-21T04:35:09.334 INFO:tasks.workunit.client.1.smithi053.stdout:4/497: dread d1/dd/de/f2b [0,4194304] 0 2024-09-21T04:35:09.335 INFO:tasks.workunit.client.1.smithi053.stdout:8/466: dwrite d3/df/d14/d16/f1d [0,4194304] 0 2024-09-21T04:35:09.336 INFO:tasks.workunit.client.1.smithi053.stdout:0/459: creat d5/d10/f9c x:0 0 0 2024-09-21T04:35:09.338 INFO:tasks.workunit.client.1.smithi053.stdout:5/531: creat d0/d8/db/d68/fb0 x:0 0 0 2024-09-21T04:35:09.339 INFO:tasks.workunit.client.1.smithi053.stdout:4/498: dread - d1/d27/d53/d88/d91/d20/d5b/d87/f93 zero size 2024-09-21T04:35:09.341 INFO:tasks.workunit.client.1.smithi053.stdout:0/460: creat d5/dd/d50/f9d x:0 0 0 2024-09-21T04:35:09.341 INFO:tasks.workunit.client.1.smithi053.stdout:5/532: stat d0/d8/db/f5f 0 2024-09-21T04:35:09.341 INFO:tasks.workunit.client.1.smithi053.stdout:9/461: dwrite d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:09.344 INFO:tasks.workunit.client.1.smithi053.stdout:4/499: readlink d1/d27/d53/d88/d91/d3f/l44 0 2024-09-21T04:35:09.347 INFO:tasks.workunit.client.1.smithi053.stdout:4/500: chown d1/d2c/d8e 51 1 2024-09-21T04:35:09.348 INFO:tasks.workunit.client.1.smithi053.stdout:1/530: truncate d7/d9/f1b 2474164 0 2024-09-21T04:35:09.349 INFO:tasks.workunit.client.1.smithi053.stdout:9/462: write d0/d1c/d52/d53/f80 [2778458,128896] 0 2024-09-21T04:35:09.350 INFO:tasks.workunit.client.1.smithi053.stdout:8/467: dwrite d3/d28/f79 [0,4194304] 0 2024-09-21T04:35:09.352 INFO:tasks.workunit.client.1.smithi053.stdout:5/533: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:09.352 INFO:tasks.workunit.client.1.smithi053.stdout:9/463: chown d0/d2/d4e 953894 1 2024-09-21T04:35:09.353 INFO:tasks.workunit.client.1.smithi053.stdout:0/461: mkdir d5/de/d9e 0 2024-09-21T04:35:09.357 INFO:tasks.workunit.client.1.smithi053.stdout:9/464: truncate d0/d28/f6b 467232 0 2024-09-21T04:35:09.357 INFO:tasks.workunit.client.1.smithi053.stdout:8/468: write d3/f59 [636795,43493] 0 2024-09-21T04:35:09.360 INFO:tasks.workunit.client.1.smithi053.stdout:5/534: rename d0/dd/d6d/f6e to d0/d8/db/fb1 0 2024-09-21T04:35:09.363 INFO:tasks.workunit.client.1.smithi053.stdout:4/501: dwrite d1/dd/de/f7a [4194304,4194304] 0 2024-09-21T04:35:09.365 INFO:tasks.workunit.client.1.smithi053.stdout:5/535: chown d0/dd/d67 171 1 2024-09-21T04:35:09.366 INFO:tasks.workunit.client.1.smithi053.stdout:7/400: write d0/d2b/d36/d48/f6b [282759,11629] 0 2024-09-21T04:35:09.366 INFO:tasks.workunit.client.1.smithi053.stdout:8/469: mknod d3/d5/d21/ca1 0 2024-09-21T04:35:09.368 INFO:tasks.workunit.client.1.smithi053.stdout:0/462: rename d5/de/d81 to d5/d6/d21/d96/d9f 0 2024-09-21T04:35:09.370 INFO:tasks.workunit.client.1.smithi053.stdout:3/466: dwrite d16/d17/d21/f6d [0,4194304] 0 2024-09-21T04:35:09.375 INFO:tasks.workunit.client.1.smithi053.stdout:5/536: mknod d0/d8/db/d68/cb2 0 2024-09-21T04:35:09.377 INFO:tasks.workunit.client.1.smithi053.stdout:8/470: symlink d3/d5/d1a/la2 0 2024-09-21T04:35:09.379 INFO:tasks.workunit.client.1.smithi053.stdout:0/463: creat d5/d6/d21/fa0 x:0 0 0 2024-09-21T04:35:09.381 INFO:tasks.workunit.client.1.smithi053.stdout:8/471: write d3/df/d10/f90 [419316,88726] 0 2024-09-21T04:35:09.381 INFO:tasks.workunit.client.1.smithi053.stdout:9/465: dread d0/f24 [0,4194304] 0 2024-09-21T04:35:09.382 INFO:tasks.workunit.client.1.smithi053.stdout:7/401: dwrite d0/f15 [0,4194304] 0 2024-09-21T04:35:09.383 INFO:tasks.workunit.client.1.smithi053.stdout:3/467: rename d16/d17/d21/d8b/d7d/d57/da9 to d16/d1b/dad 0 2024-09-21T04:35:09.385 INFO:tasks.workunit.client.1.smithi053.stdout:0/464: symlink d5/d10/d4c/d4d/la1 0 2024-09-21T04:35:09.386 INFO:tasks.workunit.client.1.smithi053.stdout:9/466: mknod d0/d1c/c9d 0 2024-09-21T04:35:09.387 INFO:tasks.workunit.client.1.smithi053.stdout:0/465: fdatasync d5/dd/f1a 0 2024-09-21T04:35:09.387 INFO:tasks.workunit.client.1.smithi053.stdout:5/537: mknod d0/dd/cb3 0 2024-09-21T04:35:09.389 INFO:tasks.workunit.client.1.smithi053.stdout:4/502: getdents d1/dd 0 2024-09-21T04:35:09.390 INFO:tasks.workunit.client.1.smithi053.stdout:8/472: mknod d3/d5/ca3 0 2024-09-21T04:35:09.390 INFO:tasks.workunit.client.1.smithi053.stdout:5/538: dread - d0/d8/db/f76 zero size 2024-09-21T04:35:09.393 INFO:tasks.workunit.client.1.smithi053.stdout:5/539: unlink d0/d8/d2a/l90 0 2024-09-21T04:35:09.393 INFO:tasks.workunit.client.1.smithi053.stdout:0/466: getdents d5/d10/d4c/d4d/d99 0 2024-09-21T04:35:09.394 INFO:tasks.workunit.client.1.smithi053.stdout:8/473: creat d3/d5/d38/d64/d73/d91/fa4 x:0 0 0 2024-09-21T04:35:09.395 INFO:tasks.workunit.client.1.smithi053.stdout:4/503: mkdir d1/d27/d53/d88/d91/d20/d4e/d51/da4 0 2024-09-21T04:35:09.397 INFO:tasks.workunit.client.1.smithi053.stdout:8/474: truncate d3/df/d14/d16/f98 189692 0 2024-09-21T04:35:09.398 INFO:tasks.workunit.client.1.smithi053.stdout:4/504: creat d1/dd/d95/fa5 x:0 0 0 2024-09-21T04:35:09.398 INFO:tasks.workunit.client.1.smithi053.stdout:8/475: fdatasync d3/f9c 0 2024-09-21T04:35:09.401 INFO:tasks.workunit.client.1.smithi053.stdout:8/476: dread - d3/d5/d4c/d53/f83 zero size 2024-09-21T04:35:09.402 INFO:tasks.workunit.client.1.smithi053.stdout:5/540: truncate d0/d8/db/fb1 323616 0 2024-09-21T04:35:09.402 INFO:tasks.workunit.client.1.smithi053.stdout:7/402: dread d0/f61 [0,4194304] 0 2024-09-21T04:35:09.403 INFO:tasks.workunit.client.1.smithi053.stdout:0/467: dwrite d5/fa [4194304,4194304] 0 2024-09-21T04:35:09.404 INFO:tasks.workunit.client.1.smithi053.stdout:8/477: chown d3/df/d14/d16/l44 419482774 1 2024-09-21T04:35:09.404 INFO:tasks.workunit.client.1.smithi053.stdout:0/468: stat d5/f39 0 2024-09-21T04:35:09.405 INFO:tasks.workunit.client.1.smithi053.stdout:4/505: dread d1/f42 [0,4194304] 0 2024-09-21T04:35:09.405 INFO:tasks.workunit.client.1.smithi053.stdout:7/403: write d0/d24/f5a [48515,91900] 0 2024-09-21T04:35:09.407 INFO:tasks.workunit.client.1.smithi053.stdout:4/506: write d1/fc [5556244,39112] 0 2024-09-21T04:35:09.407 INFO:tasks.workunit.client.1.smithi053.stdout:7/404: dread - d0/d6/f6a zero size 2024-09-21T04:35:09.407 INFO:tasks.workunit.client.1.smithi053.stdout:5/541: creat d0/dd/d5c/d89/fb4 x:0 0 0 2024-09-21T04:35:09.410 INFO:tasks.workunit.client.1.smithi053.stdout:5/542: chown d0/d8/d2a/l53 122919 1 2024-09-21T04:35:09.411 INFO:tasks.workunit.client.1.smithi053.stdout:8/478: readlink d3/d5/d21/l7b 0 2024-09-21T04:35:09.413 INFO:tasks.workunit.client.1.smithi053.stdout:4/507: rmdir d1/d27/d53/d88/d91/d20/d73/d92 39 2024-09-21T04:35:09.414 INFO:tasks.workunit.client.1.smithi053.stdout:4/508: fdatasync d1/dd/f1d 0 2024-09-21T04:35:09.415 INFO:tasks.workunit.client.1.smithi053.stdout:4/509: read d1/dd/f6b [47919,33696] 0 2024-09-21T04:35:09.417 INFO:tasks.workunit.client.1.smithi053.stdout:0/469: getdents d5/d6/d21 0 2024-09-21T04:35:09.421 INFO:tasks.workunit.client.1.smithi053.stdout:0/470: mknod d5/d47/ca2 0 2024-09-21T04:35:09.422 INFO:tasks.workunit.client.1.smithi053.stdout:7/405: dwrite d0/d6f/f72 [0,4194304] 0 2024-09-21T04:35:09.422 INFO:tasks.workunit.client.1.smithi053.stdout:2/497: sync 2024-09-21T04:35:09.423 INFO:tasks.workunit.client.1.smithi053.stdout:6/519: sync 2024-09-21T04:35:09.424 INFO:tasks.workunit.client.1.smithi053.stdout:4/510: dwrite d1/d27/d53/f78 [0,4194304] 0 2024-09-21T04:35:09.424 INFO:tasks.workunit.client.1.smithi053.stdout:0/471: mkdir d5/de/d1c/da3 0 2024-09-21T04:35:09.427 INFO:tasks.workunit.client.1.smithi053.stdout:6/520: mknod d2/d1b/d1e/d27/d41/caa 0 2024-09-21T04:35:09.428 INFO:tasks.workunit.client.1.smithi053.stdout:2/498: mknod d1/d18/d1f/d22/cae 0 2024-09-21T04:35:09.430 INFO:tasks.workunit.client.1.smithi053.stdout:0/472: creat d5/de/d42/fa4 x:0 0 0 2024-09-21T04:35:09.432 INFO:tasks.workunit.client.1.smithi053.stdout:4/511: mknod d1/d27/d53/d88/d91/ca6 0 2024-09-21T04:35:09.433 INFO:tasks.workunit.client.1.smithi053.stdout:6/521: creat d2/d1b/d1e/d2a/d4b/d77/d88/fab x:0 0 0 2024-09-21T04:35:09.434 INFO:tasks.workunit.client.1.smithi053.stdout:8/479: dread d3/df/f62 [0,4194304] 0 2024-09-21T04:35:09.438 INFO:tasks.workunit.client.1.smithi053.stdout:7/406: dwrite d0/d6/d70/f7f [0,4194304] 0 2024-09-21T04:35:09.440 INFO:tasks.workunit.client.1.smithi053.stdout:8/480: write d3/df/d10/f90 [549340,23093] 0 2024-09-21T04:35:09.443 INFO:tasks.workunit.client.1.smithi053.stdout:6/522: readlink d2/d1b/d3d/d49/de/l98 0 2024-09-21T04:35:09.446 INFO:tasks.workunit.client.1.smithi053.stdout:7/407: write d0/d2b/f2e [788905,97559] 0 2024-09-21T04:35:09.447 INFO:tasks.workunit.client.1.smithi053.stdout:2/499: dwrite d1/f5 [4194304,4194304] 0 2024-09-21T04:35:09.449 INFO:tasks.workunit.client.1.smithi053.stdout:7/408: dread - d0/d6/f6a zero size 2024-09-21T04:35:09.449 INFO:tasks.workunit.client.1.smithi053.stdout:8/481: link d3/df/f42 d3/d28/fa5 0 2024-09-21T04:35:09.450 INFO:tasks.workunit.client.1.smithi053.stdout:8/482: stat d3/d5/d38/d64/d73/d91 0 2024-09-21T04:35:09.451 INFO:tasks.workunit.client.1.smithi053.stdout:6/523: creat d2/d1b/d1e/d2a/d78/fac x:0 0 0 2024-09-21T04:35:09.452 INFO:tasks.workunit.client.1.smithi053.stdout:2/500: creat d1/d18/d1f/d27/d41/d4b/d4d/faf x:0 0 0 2024-09-21T04:35:09.454 INFO:tasks.workunit.client.1.smithi053.stdout:8/483: rmdir d3/d5 39 2024-09-21T04:35:09.456 INFO:tasks.workunit.client.1.smithi053.stdout:4/512: dread d1/fc [0,4194304] 0 2024-09-21T04:35:09.457 INFO:tasks.workunit.client.1.smithi053.stdout:7/409: rename d0/d2b/d36/d48/d5c/f67 to d0/d6/f83 0 2024-09-21T04:35:09.461 INFO:tasks.workunit.client.1.smithi053.stdout:2/501: dwrite d1/d18/d1f/d27/f55 [0,4194304] 0 2024-09-21T04:35:09.462 INFO:tasks.workunit.client.1.smithi053.stdout:7/410: mkdir d0/d2b/d36/d48/d50/d84 0 2024-09-21T04:35:09.463 INFO:tasks.workunit.client.1.smithi053.stdout:4/513: dread d1/d2c/f40 [8388608,4194304] 0 2024-09-21T04:35:09.465 INFO:tasks.workunit.client.1.smithi053.stdout:7/411: unlink d0/l22 0 2024-09-21T04:35:09.467 INFO:tasks.workunit.client.1.smithi053.stdout:2/502: rename d1/d18/d1f/d27/d41/d4b/d4d/f6c to d1/d18/d1f/d22/d54/fb0 0 2024-09-21T04:35:09.469 INFO:tasks.workunit.client.1.smithi053.stdout:2/503: readlink d1/d18/d1f/d9d/lad 0 2024-09-21T04:35:09.469 INFO:tasks.workunit.client.1.smithi053.stdout:8/484: dwrite d3/f4a [0,4194304] 0 2024-09-21T04:35:09.470 INFO:tasks.workunit.client.1.smithi053.stdout:4/514: symlink d1/d27/d53/d88/d91/la7 0 2024-09-21T04:35:09.471 INFO:tasks.workunit.client.1.smithi053.stdout:7/412: truncate d0/d2b/f32 1327116 0 2024-09-21T04:35:09.473 INFO:tasks.workunit.client.1.smithi053.stdout:4/515: chown d1/dd/c8c 4021573 1 2024-09-21T04:35:09.476 INFO:tasks.workunit.client.1.smithi053.stdout:4/516: creat d1/d27/d33/d41/fa8 x:0 0 0 2024-09-21T04:35:09.478 INFO:tasks.workunit.client.1.smithi053.stdout:2/504: dwrite d1/d18/d77/d9b/faa [0,4194304] 0 2024-09-21T04:35:09.479 INFO:tasks.workunit.client.1.smithi053.stdout:8/485: dwrite d3/df/d49/f55 [0,4194304] 0 2024-09-21T04:35:09.479 INFO:tasks.workunit.client.1.smithi053.stdout:4/517: write d1/d27/d53/d88/d91/d3f/f9a [1033891,89596] 0 2024-09-21T04:35:09.480 INFO:tasks.workunit.client.1.smithi053.stdout:2/505: fdatasync d1/d18/d1f/d9d/fa8 0 2024-09-21T04:35:09.485 INFO:tasks.workunit.client.1.smithi053.stdout:4/518: mknod d1/ca9 0 2024-09-21T04:35:09.487 INFO:tasks.workunit.client.1.smithi053.stdout:2/506: creat d1/d18/d1f/d27/d71/fb1 x:0 0 0 2024-09-21T04:35:09.487 INFO:tasks.workunit.client.1.smithi053.stdout:4/519: creat d1/d27/d53/d88/d91/d20/d4e/faa x:0 0 0 2024-09-21T04:35:09.489 INFO:tasks.workunit.client.1.smithi053.stdout:4/520: chown d1/f42 700315056 1 2024-09-21T04:35:09.490 INFO:tasks.workunit.client.1.smithi053.stdout:8/486: dwrite d3/d28/f31 [0,4194304] 0 2024-09-21T04:35:09.491 INFO:tasks.workunit.client.1.smithi053.stdout:2/507: creat d1/d18/d1f/d27/d41/d4b/d78/fb2 x:0 0 0 2024-09-21T04:35:09.492 INFO:tasks.workunit.client.1.smithi053.stdout:4/521: creat d1/d27/d53/d88/d91/d20/d4e/d51/fab x:0 0 0 2024-09-21T04:35:09.493 INFO:tasks.workunit.client.1.smithi053.stdout:4/522: write d1/d27/d33/f3c [1453292,95442] 0 2024-09-21T04:35:09.494 INFO:tasks.workunit.client.1.smithi053.stdout:4/523: write d1/d2c/f85 [942402,68310] 0 2024-09-21T04:35:09.498 INFO:tasks.workunit.client.1.smithi053.stdout:8/487: creat d3/d5/d4b/d5c/fa6 x:0 0 0 2024-09-21T04:35:09.499 INFO:tasks.workunit.client.1.smithi053.stdout:4/524: creat d1/d27/d33/fac x:0 0 0 2024-09-21T04:35:09.502 INFO:tasks.workunit.client.1.smithi053.stdout:2/508: rename d1/d18/d1f/d22/f45 to d1/d18/d1f/d27/d58/fb3 0 2024-09-21T04:35:09.503 INFO:tasks.workunit.client.1.smithi053.stdout:8/488: dwrite d3/d5/d38/d64/f67 [0,4194304] 0 2024-09-21T04:35:09.504 INFO:tasks.workunit.client.1.smithi053.stdout:2/509: write d1/d18/f49 [4359944,115587] 0 2024-09-21T04:35:09.505 INFO:tasks.workunit.client.1.smithi053.stdout:4/525: creat d1/d27/d53/d88/d91/d20/d5b/d87/fad x:0 0 0 2024-09-21T04:35:09.505 INFO:tasks.workunit.client.1.smithi053.stdout:4/526: chown d1/dd/de/c5a 509096 1 2024-09-21T04:35:09.510 INFO:tasks.workunit.client.1.smithi053.stdout:2/510: unlink d1/d18/d1f/d27/d41/d4b/d4d/f93 0 2024-09-21T04:35:09.512 INFO:tasks.workunit.client.1.smithi053.stdout:4/527: symlink d1/d27/d33/d41/lae 0 2024-09-21T04:35:09.519 INFO:tasks.workunit.client.1.smithi053.stdout:8/489: creat d3/d5/d4b/fa7 x:0 0 0 2024-09-21T04:35:09.524 INFO:tasks.workunit.client.1.smithi053.stdout:4/528: dwrite d1/d27/f9d [0,4194304] 0 2024-09-21T04:35:09.525 INFO:tasks.workunit.client.1.smithi053.stdout:2/511: dwrite d1/d18/d1f/d27/d41/d4b/d4d/f88 [0,4194304] 0 2024-09-21T04:35:09.526 INFO:tasks.workunit.client.1.smithi053.stdout:4/529: chown d1/d27/d53/d88/d91/c35 62120 1 2024-09-21T04:35:09.527 INFO:tasks.workunit.client.1.smithi053.stdout:4/530: chown d1/d27/d53/d88/d91/ca6 11599 1 2024-09-21T04:35:09.528 INFO:tasks.workunit.client.1.smithi053.stdout:4/531: chown d1/dd/de/c9f 3090 1 2024-09-21T04:35:09.529 INFO:tasks.workunit.client.1.smithi053.stdout:2/512: rename d1/d18/d1f/d22/d2d/f7e to d1/d18/d1f/d27/d41/d4b/fb4 0 2024-09-21T04:35:09.529 INFO:tasks.workunit.client.1.smithi053.stdout:8/490: write d3/d39/f3f [1999011,93320] 0 2024-09-21T04:35:09.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.527+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd9400626f0 con 0x7fd928074a30 2024-09-21T04:35:09.530 INFO:tasks.workunit.client.1.smithi053.stdout:2/513: write d1/d18/d1f/d22/d2d/f87 [877907,11264] 0 2024-09-21T04:35:09.531 INFO:tasks.workunit.client.1.smithi053.stdout:2/514: chown d1/d18/d1f 224 1 2024-09-21T04:35:09.532 INFO:tasks.workunit.client.1.smithi053.stdout:4/532: rename d1/l6 to d1/dd/laf 0 2024-09-21T04:35:09.533 INFO:tasks.workunit.client.1.smithi053.stdout:2/515: dread - d1/d18/d1f/d27/d41/d4b/d4d/faf zero size 2024-09-21T04:35:09.535 INFO:tasks.workunit.client.1.smithi053.stdout:8/491: symlink d3/df/d14/la8 0 2024-09-21T04:35:09.536 INFO:tasks.workunit.client.1.smithi053.stdout:4/533: mknod d1/dd/cb0 0 2024-09-21T04:35:09.536 INFO:tasks.workunit.client.1.smithi053.stdout:2/516: mkdir d1/d18/d1f/d27/d41/d4b/d4d/db5 0 2024-09-21T04:35:09.537 INFO:tasks.workunit.client.1.smithi053.stdout:8/492: write d3/f3a [1128763,23995] 0 2024-09-21T04:35:09.540 INFO:tasks.workunit.client.1.smithi053.stdout:4/534: creat d1/d27/d90/fb1 x:0 0 0 2024-09-21T04:35:09.543 INFO:tasks.workunit.client.1.smithi053.stdout:4/535: creat d1/dd/fb2 x:0 0 0 2024-09-21T04:35:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.541+0000 7fd927fff640 1 -- 172.21.15.7:0/3797161708 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fd9400626f0 con 0x7fd928074a30 2024-09-21T04:35:09.544 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:35:09.544 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (6m) 37s ago 8m 24.6M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 starting - - - - 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (7m) 33s ago 7m 5268k - 19.1.1-373-g4e45e6b4 164465d2334a 828ddad2141a 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (8m) 37s ago 8m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (7m) 33s ago 7m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (6m) 37s ago 7m 77.5M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (2m) 37s ago 2m 122M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (2m) 37s ago 2m 103M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (2m) 33s ago 2m 19.7M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (2m) 33s ago 2m 25.7M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:9283,8765,8443 running (9m) 37s ago 9m 250M - 19.1.1-373-g4e45e6b4 164465d2334a c324ee791e08 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (58s) 33s ago 7m 499M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7d35aae8ab52 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (9m) 37s ago 9m 52.9M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (7m) 33s ago 7m 43.9M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (8m) 37s ago 8m 15.8M - 1.5.0 0da6a335fe13 b4a8255a594a 2024-09-21T04:35:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (7m) 33s ago 7m 16.8M - 1.5.0 0da6a335fe13 88246b15ca5d 2024-09-21T04:35:09.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (6m) 37s ago 6m 70.3M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:35:09.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (5m) 37s ago 5m 78.0M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:35:09.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (5m) 37s ago 5m 73.5M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:35:09.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (4m) 33s ago 4m 76.8M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:35:09.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (4m) 33s ago 4m 80.3M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:35:09.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (3m) 33s ago 3m 64.5M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:35:09.546 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 starting - - - - 2024-09-21T04:35:09.546 INFO:tasks.workunit.client.1.smithi053.stdout:4/536: unlink d1/cb 0 2024-09-21T04:35:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.544+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd928074a30 msgr2=0x7fd928076ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.544+0000 7fd945717640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd928074a30 0x7fd928076ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd934000c00 tx=0x7fd9340058d0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.544+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940101970 msgr2=0x7fd940075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.544+0000 7fd945717640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940101970 0x7fd940075f20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd93000d8d0 tx=0x7fd93000dd90 comp rx=0 tx=0).stop 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.544+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 shutdown_connections 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.545+0000 7fd945717640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd928074a30 0x7fd928076ef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.545+0000 7fd945717640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd940102b70 0x7fd9400745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.545+0000 7fd945717640 1 --2- 172.21.15.7:0/3797161708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd940101970 0x7fd940075f20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.545+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 >> 172.21.15.7:0/3797161708 conn(0x7fd9400fd140 msgr2=0x7fd940106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.545+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 shutdown_connections 2024-09-21T04:35:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.545+0000 7fd945717640 1 -- 172.21.15.7:0/3797161708 wait complete. 2024-09-21T04:35:09.549 INFO:tasks.workunit.client.1.smithi053.stdout:8/493: getdents d3/d5/d38/d64 0 2024-09-21T04:35:09.549 INFO:tasks.workunit.client.1.smithi053.stdout:8/494: dread - d3/d5/d38/d64/f82 zero size 2024-09-21T04:35:09.551 INFO:tasks.workunit.client.1.smithi053.stdout:8/495: mkdir d3/d5/d38/d64/d9d/da9 0 2024-09-21T04:35:09.553 INFO:tasks.workunit.client.1.smithi053.stdout:4/537: creat d1/d27/d53/d88/d91/d20/d73/fb3 x:0 0 0 2024-09-21T04:35:09.554 INFO:tasks.workunit.client.1.smithi053.stdout:2/517: write d1/d18/d1f/d22/f5a [914602,19848] 0 2024-09-21T04:35:09.555 INFO:tasks.workunit.client.1.smithi053.stdout:2/518: chown d1/d18/d1f/da2 9 1 2024-09-21T04:35:09.556 INFO:tasks.workunit.client.1.smithi053.stdout:4/538: truncate d1/d27/d53/d88/d91/d3f/f77 406941 0 2024-09-21T04:35:09.557 INFO:tasks.workunit.client.1.smithi053.stdout:2/519: write d1/f7b [1185721,106837] 0 2024-09-21T04:35:09.558 INFO:tasks.workunit.client.1.smithi053.stdout:4/539: mkdir d1/d2c/d8e/db4 0 2024-09-21T04:35:09.559 INFO:tasks.workunit.client.1.smithi053.stdout:2/520: truncate d1/d18/d1f/d22/d43/f4f 415066 0 2024-09-21T04:35:09.566 INFO:tasks.workunit.client.1.smithi053.stdout:4/540: dwrite d1/d27/d33/fac [0,4194304] 0 2024-09-21T04:35:09.566 INFO:tasks.workunit.client.1.smithi053.stdout:1/531: sync 2024-09-21T04:35:09.566 INFO:tasks.workunit.client.1.smithi053.stdout:8/496: dread d3/d5/f19 [8388608,4194304] 0 2024-09-21T04:35:09.573 INFO:tasks.workunit.client.1.smithi053.stdout:2/521: dwrite d1/d18/d1f/d22/d43/f4f [0,4194304] 0 2024-09-21T04:35:09.576 INFO:tasks.workunit.client.1.smithi053.stdout:8/497: dread - d3/d5/d38/d64/f82 zero size 2024-09-21T04:35:09.579 INFO:tasks.workunit.client.1.smithi053.stdout:1/532: fdatasync d7/d9/d43/d54/d7f/d59/f7b 0 2024-09-21T04:35:09.587 INFO:tasks.workunit.client.1.smithi053.stdout:1/533: chown d7/c1c 13217219 1 2024-09-21T04:35:09.588 INFO:tasks.workunit.client.1.smithi053.stdout:4/541: dwrite d1/d27/d33/f3c [0,4194304] 0 2024-09-21T04:35:09.589 INFO:tasks.workunit.client.1.smithi053.stdout:6/524: dread d2/fa [0,4194304] 0 2024-09-21T04:35:09.594 INFO:tasks.workunit.client.1.smithi053.stdout:8/498: dwrite d3/d28/f79 [0,4194304] 0 2024-09-21T04:35:09.596 INFO:tasks.workunit.client.1.smithi053.stdout:6/525: rmdir d2/d1b/d3d/d49/d5c/d8d 39 2024-09-21T04:35:09.598 INFO:tasks.workunit.client.1.smithi053.stdout:1/534: getdents d7/d9/da/d40/d49 0 2024-09-21T04:35:09.602 INFO:tasks.workunit.client.1.smithi053.stdout:8/499: write d3/d5/d4c/f6d [3901590,19492] 0 2024-09-21T04:35:09.603 INFO:tasks.workunit.client.1.smithi053.stdout:4/542: dwrite d1/d27/d53/f78 [4194304,4194304] 0 2024-09-21T04:35:09.606 INFO:tasks.workunit.client.1.smithi053.stdout:1/535: unlink d7/d9/da/d11/l42 0 2024-09-21T04:35:09.607 INFO:tasks.workunit.client.1.smithi053.stdout:8/500: rmdir d3/d5/d38/d64/d9d 39 2024-09-21T04:35:09.609 INFO:tasks.workunit.client.1.smithi053.stdout:2/522: dwrite d1/d18/d1f/d27/f7a [0,4194304] 0 2024-09-21T04:35:09.615 INFO:tasks.workunit.client.1.smithi053.stdout:6/526: dwrite d2/d1d/f54 [0,4194304] 0 2024-09-21T04:35:09.617 INFO:tasks.workunit.client.1.smithi053.stdout:8/501: truncate d3/d28/f2e 2146116 0 2024-09-21T04:35:09.622 INFO:tasks.workunit.client.1.smithi053.stdout:4/543: dwrite d1/d27/d53/d88/d91/d20/d4e/faa [0,4194304] 0 2024-09-21T04:35:09.622 INFO:tasks.workunit.client.1.smithi053.stdout:6/527: mkdir d2/d1b/d3d/d49/dad 0 2024-09-21T04:35:09.623 INFO:tasks.workunit.client.1.smithi053.stdout:8/502: mknod d3/caa 0 2024-09-21T04:35:09.625 INFO:tasks.workunit.client.1.smithi053.stdout:1/536: link d7/d9/d43/d54/d7f/d59/l87 d7/da8/lb1 0 2024-09-21T04:35:09.626 INFO:tasks.workunit.client.1.smithi053.stdout:6/528: write d2/d1b/f53 [1737694,29899] 0 2024-09-21T04:35:09.628 INFO:tasks.workunit.client.1.smithi053.stdout:1/537: fdatasync d7/d47/fa4 0 2024-09-21T04:35:09.628 INFO:tasks.workunit.client.1.smithi053.stdout:2/523: dwrite d1/d18/d1f/d9d/f90 [0,4194304] 0 2024-09-21T04:35:09.631 INFO:tasks.workunit.client.1.smithi053.stdout:6/529: rmdir d2/d1b/d1e/d2a/d78 39 2024-09-21T04:35:09.633 INFO:tasks.workunit.client.1.smithi053.stdout:4/544: rename d1/dd/d95/fa5 to d1/d27/d53/d88/d91/fb5 0 2024-09-21T04:35:09.637 INFO:tasks.workunit.client.1.smithi053.stdout:8/503: rename d3/df/d10/c3e to d3/d5/d38/d64/d9d/cab 0 2024-09-21T04:35:09.638 INFO:tasks.workunit.client.1.smithi053.stdout:2/524: truncate d1/d18/d1f/d22/d43/f97 624754 0 2024-09-21T04:35:09.639 INFO:tasks.workunit.client.1.smithi053.stdout:4/545: mkdir d1/d27/d53/d88/d91/db6 0 2024-09-21T04:35:09.641 INFO:tasks.workunit.client.1.smithi053.stdout:8/504: mknod d3/d5/d21/cac 0 2024-09-21T04:35:09.641 INFO:tasks.workunit.client.1.smithi053.stdout:2/525: creat d1/d18/d1f/d22/d54/fb6 x:0 0 0 2024-09-21T04:35:09.643 INFO:tasks.workunit.client.1.smithi053.stdout:8/505: write d3/d5/f8f [861462,116168] 0 2024-09-21T04:35:09.644 INFO:tasks.workunit.client.1.smithi053.stdout:6/530: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f9d [0,4194304] 0 2024-09-21T04:35:09.649 INFO:tasks.workunit.client.1.smithi053.stdout:8/506: creat d3/d28/fad x:0 0 0 2024-09-21T04:35:09.650 INFO:tasks.workunit.client.1.smithi053.stdout:8/507: chown d3/df/d14 4844 1 2024-09-21T04:35:09.651 INFO:tasks.workunit.client.1.smithi053.stdout:4/546: link d1/dd/d95/fa2 d1/d27/d53/d88/d91/d20/d73/d92/fb7 0 2024-09-21T04:35:09.651 INFO:tasks.workunit.client.1.smithi053.stdout:4/547: stat d1/d27/f55 0 2024-09-21T04:35:09.654 INFO:tasks.workunit.client.1.smithi053.stdout:6/531: dread d2/d1b/f6b [0,4194304] 0 2024-09-21T04:35:09.654 INFO:tasks.workunit.client.1.smithi053.stdout:2/526: mkdir d1/d18/d1f/d27/d58/db7 0 2024-09-21T04:35:09.657 INFO:tasks.workunit.client.1.smithi053.stdout:4/548: rename d1/d27/d53/d88/d91/d20/d73/c7d to d1/dd/de/cb8 0 2024-09-21T04:35:09.658 INFO:tasks.workunit.client.1.smithi053.stdout:8/508: dwrite d3/d5/d4c/f6d [4194304,4194304] 0 2024-09-21T04:35:09.663 INFO:tasks.workunit.client.1.smithi053.stdout:6/532: dread d2/d1b/d1e/d2a/d4b/f68 [0,4194304] 0 2024-09-21T04:35:09.664 INFO:tasks.workunit.client.1.smithi053.stdout:8/509: fsync d3/d28/fa5 0 2024-09-21T04:35:09.670 INFO:tasks.workunit.client.1.smithi053.stdout:6/533: unlink d2/d1d/ca1 0 2024-09-21T04:35:09.671 INFO:tasks.workunit.client.1.smithi053.stdout:4/549: truncate d1/dd/de/f2b 1312877 0 2024-09-21T04:35:09.671 INFO:tasks.workunit.client.1.smithi053.stdout:8/510: creat d3/df/d10/fae x:0 0 0 2024-09-21T04:35:09.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.670+0000 7fd093a97640 1 -- 172.21.15.7:0/3015950819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c1091f0 msgr2=0x7fd08c10b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:09.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.670+0000 7fd093a97640 1 --2- 172.21.15.7:0/3015950819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c1091f0 0x7fd08c10b5e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd088009880 tx=0x7fd08802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.673 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:09 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.670+0000 7fd093a97640 1 -- 172.21.15.7:0/3015950819 shutdown_connections 2024-09-21T04:35:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.670+0000 7fd093a97640 1 --2- 172.21.15.7:0/3015950819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd08c10bb20 0x7fd08c10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.670+0000 7fd093a97640 1 --2- 172.21.15.7:0/3015950819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c1091f0 0x7fd08c10b5e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.670+0000 7fd093a97640 1 -- 172.21.15.7:0/3015950819 >> 172.21.15.7:0/3015950819 conn(0x7fd08c06f250 msgr2=0x7fd08c071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.671+0000 7fd093a97640 1 -- 172.21.15.7:0/3015950819 shutdown_connections 2024-09-21T04:35:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.671+0000 7fd093a97640 1 -- 172.21.15.7:0/3015950819 wait complete. 2024-09-21T04:35:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd093a97640 1 Processor -- start 2024-09-21T04:35:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd093a97640 1 -- start start 2024-09-21T04:35:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd093a97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd08c1091f0 0x7fd08c111060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd093a97640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c10bb20 0x7fd08c10f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd093a97640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c113040 con 0x7fd08c10bb20 2024-09-21T04:35:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd093a97640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c112ec0 con 0x7fd08c1091f0 2024-09-21T04:35:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd091032640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c10bb20 0x7fd08c10f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:09.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd091032640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c10bb20 0x7fd08c10f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35834/0 (socket says 172.21.15.7:35834) 2024-09-21T04:35:09.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.672+0000 7fd091032640 1 -- 172.21.15.7:0/3893378791 learned_addr learned my addr 172.21.15.7:0/3893378791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:09.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.673+0000 7fd091032640 1 -- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd08c1091f0 msgr2=0x7fd08c111060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.673+0000 7fd091032640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd08c1091f0 0x7fd08c111060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.673+0000 7fd091032640 1 -- 172.21.15.7:0/3893378791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c10fc60 con 0x7fd08c10bb20 2024-09-21T04:35:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.673+0000 7fd091032640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c10bb20 0x7fd08c10f720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00b3f0 tx=0x7fd07c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.673+0000 7fd082ffd640 1 -- 172.21.15.7:0/3893378791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd07c015020 con 0x7fd08c10bb20 2024-09-21T04:35:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.674+0000 7fd082ffd640 1 -- 172.21.15.7:0/3893378791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd07c011af0 con 0x7fd08c10bb20 2024-09-21T04:35:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.674+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c10ff50 con 0x7fd08c10bb20 2024-09-21T04:35:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.674+0000 7fd082ffd640 1 -- 172.21.15.7:0/3893378791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd07c010ce0 con 0x7fd08c10bb20 2024-09-21T04:35:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.674+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c110560 con 0x7fd08c10bb20 2024-09-21T04:35:09.678 INFO:tasks.workunit.client.1.smithi053.stdout:2/527: dwrite d1/d18/d1f/d27/d41/f64 [0,4194304] 0 2024-09-21T04:35:09.678 INFO:tasks.workunit.client.1.smithi053.stdout:4/550: creat d1/d27/d53/d88/fb9 x:0 0 0 2024-09-21T04:35:09.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.676+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08c1095d0 con 0x7fd08c10bb20 2024-09-21T04:35:09.678 INFO:tasks.workunit.client.1.smithi053.stdout:6/534: symlink d2/d1b/d1e/d2a/d4b/d77/d88/da4/lae 0 2024-09-21T04:35:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.676+0000 7fd082ffd640 1 -- 172.21.15.7:0/3893378791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7fd07c018020 con 0x7fd08c10bb20 2024-09-21T04:35:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.677+0000 7fd082ffd640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd078074ad0 0x7fd078076f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.677+0000 7fd082ffd640 1 -- 172.21.15.7:0/3893378791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7fd07c095d60 con 0x7fd08c10bb20 2024-09-21T04:35:09.681 INFO:tasks.workunit.client.1.smithi053.stdout:8/511: getdents d3/d5/d4c/d8e 0 2024-09-21T04:35:09.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.678+0000 7fd091833640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd078074ad0 0x7fd078076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:09.682 INFO:tasks.workunit.client.1.smithi053.stdout:8/512: fsync d3/d28/f31 0 2024-09-21T04:35:09.683 INFO:tasks.workunit.client.1.smithi053.stdout:8/513: chown d3/df/d49 59154548 1 2024-09-21T04:35:09.683 INFO:tasks.workunit.client.1.smithi053.stdout:8/514: dread - d3/d39/f97 zero size 2024-09-21T04:35:09.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.681+0000 7fd091833640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd078074ad0 0x7fd078076f90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0880096c0 tx=0x7fd088000bc0 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:09.685 INFO:tasks.workunit.client.1.smithi053.stdout:4/551: rename d1/fc to d1/d2c/d8e/db4/fba 0 2024-09-21T04:35:09.686 INFO:tasks.workunit.client.1.smithi053.stdout:8/515: write d3/d5/d4b/f7e [47999,77020] 0 2024-09-21T04:35:09.687 INFO:tasks.workunit.client.1.smithi053.stdout:4/552: chown d1/d27/d53/d88/d91/fb5 4881360 1 2024-09-21T04:35:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:09.685+0000 7fd082ffd640 1 -- 172.21.15.7:0/3893378791 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fd07c01f020 con 0x7fd08c10bb20 2024-09-21T04:35:09.688 INFO:tasks.workunit.client.1.smithi053.stdout:2/528: creat d1/d18/d1f/fb8 x:0 0 0 2024-09-21T04:35:09.688 INFO:tasks.workunit.client.1.smithi053.stdout:6/535: write d2/fa [53153,42997] 0 2024-09-21T04:35:09.690 INFO:tasks.workunit.client.1.smithi053.stdout:4/553: symlink d1/d27/d33/lbb 0 2024-09-21T04:35:09.691 INFO:tasks.workunit.client.1.smithi053.stdout:8/516: mknod d3/d5/d4c/caf 0 2024-09-21T04:35:09.692 INFO:tasks.workunit.client.1.smithi053.stdout:8/517: readlink d3/d5/l5a 0 2024-09-21T04:35:09.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:09 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:09.693 INFO:tasks.workunit.client.1.smithi053.stdout:8/518: chown d3/df/d14 1664011 1 2024-09-21T04:35:09.693 INFO:tasks.workunit.client.1.smithi053.stdout:4/554: truncate d1/d27/d53/d88/d91/fb5 160507 0 2024-09-21T04:35:09.694 INFO:tasks.workunit.client.1.smithi053.stdout:2/529: getdents d1/d18/d1f/d27 0 2024-09-21T04:35:09.694 INFO:tasks.workunit.client.1.smithi053.stdout:4/555: rmdir d1/dd/d62 39 2024-09-21T04:35:09.695 INFO:tasks.workunit.client.1.smithi053.stdout:8/519: rename d3/d28/fa5 to d3/d39/fb0 0 2024-09-21T04:35:09.696 INFO:tasks.workunit.client.1.smithi053.stdout:2/530: symlink d1/d18/d1f/d27/d41/d4b/d78/lb9 0 2024-09-21T04:35:09.698 INFO:tasks.workunit.client.1.smithi053.stdout:4/556: mknod d1/d27/d53/d88/d91/d20/d73/cbc 0 2024-09-21T04:35:09.699 INFO:tasks.workunit.client.1.smithi053.stdout:6/536: truncate d2/d1b/d1e/d2a/f99 3385738 0 2024-09-21T04:35:09.700 INFO:tasks.workunit.client.1.smithi053.stdout:6/537: fdatasync d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f9d 0 2024-09-21T04:35:09.701 INFO:tasks.workunit.client.1.smithi053.stdout:6/538: chown d2/d1b/d1e/d27/d41/f46 58929 1 2024-09-21T04:35:09.701 INFO:tasks.workunit.client.1.smithi053.stdout:4/557: creat d1/d27/d2e/d46/fbd x:0 0 0 2024-09-21T04:35:09.702 INFO:tasks.workunit.client.1.smithi053.stdout:4/558: readlink d1/d27/d2e/l69 0 2024-09-21T04:35:09.703 INFO:tasks.workunit.client.1.smithi053.stdout:2/531: creat d1/d18/d1f/fba x:0 0 0 2024-09-21T04:35:09.703 INFO:tasks.workunit.client.1.smithi053.stdout:4/559: fsync d1/d27/d33/f3c 0 2024-09-21T04:35:09.709 INFO:tasks.workunit.client.1.smithi053.stdout:2/532: stat d1/d18/d1f/d22/d43/f97 0 2024-09-21T04:35:09.711 INFO:tasks.workunit.client.1.smithi053.stdout:6/539: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/d8c/fa6 [0,4194304] 0 2024-09-21T04:35:09.711 INFO:tasks.workunit.client.1.smithi053.stdout:4/560: dwrite d1/d27/f89 [0,4194304] 0 2024-09-21T04:35:09.717 INFO:tasks.workunit.client.1.smithi053.stdout:2/533: mkdir d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb 0 2024-09-21T04:35:09.720 INFO:tasks.workunit.client.1.smithi053.stdout:8/520: dwrite d3/d39/f27 [4194304,4194304] 0 2024-09-21T04:35:09.721 INFO:tasks.workunit.client.1.smithi053.stdout:6/540: mknod d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/caf 0 2024-09-21T04:35:09.722 INFO:tasks.workunit.client.1.smithi053.stdout:2/534: dread d1/d18/d1f/d22/d43/f59 [0,4194304] 0 2024-09-21T04:35:09.723 INFO:tasks.workunit.client.1.smithi053.stdout:8/521: chown d3/df/d10/fae 551118927 1 2024-09-21T04:35:09.724 INFO:tasks.workunit.client.1.smithi053.stdout:4/561: getdents d1/d27/d53/d88/d91/d20/d73 0 2024-09-21T04:35:09.727 INFO:tasks.workunit.client.1.smithi053.stdout:4/562: dread - d1/d27/d2e/d46/fbd zero size 2024-09-21T04:35:09.729 INFO:tasks.workunit.client.1.smithi053.stdout:4/563: chown d1/d27/d2e/d46/f8d 152463 1 2024-09-21T04:35:09.731 INFO:tasks.workunit.client.1.smithi053.stdout:8/522: getdents d3/d5/d38 0 2024-09-21T04:35:09.732 INFO:tasks.workunit.client.1.smithi053.stdout:2/535: dwrite d1/d18/d1f/d27/d58/d5c/f62 [0,4194304] 0 2024-09-21T04:35:09.733 INFO:tasks.workunit.client.1.smithi053.stdout:6/541: rename d2/d1b/d3d/d49/f4c to d2/d1b/d1e/fb0 0 2024-09-21T04:35:09.734 INFO:tasks.workunit.client.1.smithi053.stdout:4/564: creat d1/d27/d53/d88/d91/d20/d5b/fbe x:0 0 0 2024-09-21T04:35:09.737 INFO:tasks.workunit.client.1.smithi053.stdout:2/536: fsync d1/d18/d1f/d27/d41/d4b/d4d/f52 0 2024-09-21T04:35:09.739 INFO:tasks.workunit.client.1.smithi053.stdout:4/565: write d1/d27/d53/d88/d91/f84 [3780701,50112] 0 2024-09-21T04:35:09.742 INFO:tasks.workunit.client.1.smithi053.stdout:2/537: unlink d1/d18/d1f/d27/d41/c76 0 2024-09-21T04:35:09.743 INFO:tasks.workunit.client.1.smithi053.stdout:6/542: creat d2/d1b/d1e/fb1 x:0 0 0 2024-09-21T04:35:09.743 INFO:tasks.workunit.client.1.smithi053.stdout:8/523: getdents d3/d5/d4c/d53/d66 0 2024-09-21T04:35:09.744 INFO:tasks.workunit.client.1.smithi053.stdout:8/524: read - d3/d5/d38/d64/f82 zero size 2024-09-21T04:35:09.745 INFO:tasks.workunit.client.1.smithi053.stdout:6/543: write d2/d1b/d3d/d49/f19 [891282,83275] 0 2024-09-21T04:35:09.745 INFO:tasks.workunit.client.1.smithi053.stdout:4/566: rename d1/d2c/c70 to d1/d27/d53/d88/d91/d20/d5b/cbf 0 2024-09-21T04:35:09.746 INFO:tasks.workunit.client.1.smithi053.stdout:8/525: readlink d3/d39/l2d 0 2024-09-21T04:35:09.746 INFO:tasks.workunit.client.1.smithi053.stdout:2/538: mknod d1/d18/d1f/d22/d2d/cbc 0 2024-09-21T04:35:09.747 INFO:tasks.workunit.client.1.smithi053.stdout:6/544: stat d2/d1b/d3d/d49/d5c/f5d 0 2024-09-21T04:35:09.747 INFO:tasks.workunit.client.1.smithi053.stdout:6/545: fsync d2/d1b/f2f 0 2024-09-21T04:35:09.749 INFO:tasks.workunit.client.1.smithi053.stdout:6/546: fdatasync d2/d1b/d3d/d49/de/d32/d80/f26 0 2024-09-21T04:35:09.751 INFO:tasks.workunit.client.1.smithi053.stdout:8/526: mkdir d3/d5/db1 0 2024-09-21T04:35:09.752 INFO:tasks.workunit.client.1.smithi053.stdout:4/567: dread d1/d27/d53/d88/d91/d3f/f9a [0,4194304] 0 2024-09-21T04:35:09.754 INFO:tasks.workunit.client.1.smithi053.stdout:6/547: creat d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/fb2 x:0 0 0 2024-09-21T04:35:09.755 INFO:tasks.workunit.client.1.smithi053.stdout:2/539: rmdir d1/d18/d1f/d27/d41/d4b/d4d/db5 0 2024-09-21T04:35:09.755 INFO:tasks.workunit.client.1.smithi053.stdout:8/527: dread d3/d5/d38/d64/f67 [0,4194304] 0 2024-09-21T04:35:09.759 INFO:tasks.workunit.client.1.smithi053.stdout:6/548: symlink d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/lb3 0 2024-09-21T04:35:09.764 INFO:tasks.workunit.client.1.smithi053.stdout:8/528: dwrite d3/d28/f79 [0,4194304] 0 2024-09-21T04:35:09.764 INFO:tasks.workunit.client.1.smithi053.stdout:6/549: write d2/d1b/d1e/d27/d41/f97 [1330544,16307] 0 2024-09-21T04:35:09.767 INFO:tasks.workunit.client.1.smithi053.stdout:2/540: link d1/d18/d1f/d9d/l95 d1/d18/d1f/d27/d41/d4b/d4d/d89/lbd 0 2024-09-21T04:35:09.768 INFO:tasks.workunit.client.1.smithi053.stdout:6/550: creat d2/d1b/d1e/d2a/d4b/fb4 x:0 0 0 2024-09-21T04:35:09.769 INFO:tasks.workunit.client.1.smithi053.stdout:4/568: getdents d1/d2c/d8e/db4 0 2024-09-21T04:35:09.771 INFO:tasks.workunit.client.1.smithi053.stdout:2/541: mknod d1/d18/d1f/d22/d54/cbe 0 2024-09-21T04:35:09.773 INFO:tasks.workunit.client.1.smithi053.stdout:4/569: unlink d1/d27/d53/d88/d91/d20/d5b/f97 0 2024-09-21T04:35:09.776 INFO:tasks.workunit.client.1.smithi053.stdout:2/542: mknod d1/d18/d1f/d22/d2d/cbf 0 2024-09-21T04:35:09.777 INFO:tasks.workunit.client.1.smithi053.stdout:6/551: truncate d2/d1b/f2f 825875 0 2024-09-21T04:35:09.778 INFO:tasks.workunit.client.1.smithi053.stdout:4/570: dwrite d1/d27/f55 [0,4194304] 0 2024-09-21T04:35:09.779 INFO:tasks.workunit.client.1.smithi053.stdout:4/571: readlink d1/dd/de/l4f 0 2024-09-21T04:35:09.780 INFO:tasks.workunit.client.1.smithi053.stdout:2/543: creat d1/d18/d1f/d27/d58/d5c/fc0 x:0 0 0 2024-09-21T04:35:09.780 INFO:tasks.workunit.client.1.smithi053.stdout:4/572: stat d1/d2c/d8e/db4/fba 0 2024-09-21T04:35:09.781 INFO:tasks.workunit.client.1.smithi053.stdout:6/552: creat d2/d1b/fb5 x:0 0 0 2024-09-21T04:35:09.782 INFO:tasks.workunit.client.1.smithi053.stdout:4/573: stat d1/d27/d53/d88/d91/d3f 0 2024-09-21T04:35:09.785 INFO:tasks.workunit.client.1.smithi053.stdout:4/574: chown d1/d27/d53/d88/d91/d20/f75 151 1 2024-09-21T04:35:09.789 INFO:tasks.workunit.client.1.smithi053.stdout:2/544: dread d1/d18/d77/d9b/faa [0,4194304] 0 2024-09-21T04:35:09.789 INFO:tasks.workunit.client.1.smithi053.stdout:8/529: dwrite d3/df/d14/d16/f29 [0,4194304] 0 2024-09-21T04:35:09.791 INFO:tasks.workunit.client.1.smithi053.stdout:4/575: dread d1/d27/d53/d88/d91/d20/d4e/f68 [0,4194304] 0 2024-09-21T04:35:09.793 INFO:tasks.workunit.client.1.smithi053.stdout:6/553: dwrite d2/d1b/d1e/d2a/d4b/d77/d88/fab [0,4194304] 0 2024-09-21T04:35:09.794 INFO:tasks.workunit.client.1.smithi053.stdout:8/530: creat d3/df/fb2 x:0 0 0 2024-09-21T04:35:09.795 INFO:tasks.workunit.client.1.smithi053.stdout:4/576: stat d1/d27/d53/d88/d91/d20/d73/d92/f94 0 2024-09-21T04:35:09.798 INFO:tasks.workunit.client.1.smithi053.stdout:2/545: dwrite d1/d18/d1f/fba [0,4194304] 0 2024-09-21T04:35:09.799 INFO:tasks.workunit.client.1.smithi053.stdout:2/546: write d1/d18/f49 [2640943,65991] 0 2024-09-21T04:35:09.803 INFO:tasks.workunit.client.1.smithi053.stdout:8/531: symlink d3/df/d14/d16/lb3 0 2024-09-21T04:35:09.803 INFO:tasks.workunit.client.1.smithi053.stdout:4/577: write d1/dd/f23 [795229,64311] 0 2024-09-21T04:35:09.808 INFO:tasks.workunit.client.1.smithi053.stdout:4/578: dread - d1/d27/d53/d88/d91/d20/d4e/d51/fab zero size 2024-09-21T04:35:09.808 INFO:tasks.workunit.client.1.smithi053.stdout:6/554: symlink d2/d1b/d1e/d27/lb6 0 2024-09-21T04:35:09.811 INFO:tasks.workunit.client.1.smithi053.stdout:2/547: mkdir d1/d18/d1f/d27/d58/d5c/d91/dc1 0 2024-09-21T04:35:09.813 INFO:tasks.workunit.client.1.smithi053.stdout:4/579: fdatasync d1/d27/d53/d88/d91/d20/d4e/faa 0 2024-09-21T04:35:09.814 INFO:tasks.workunit.client.1.smithi053.stdout:8/532: creat d3/d5/d38/d64/d9d/da9/fb4 x:0 0 0 2024-09-21T04:35:09.814 INFO:tasks.workunit.client.1.smithi053.stdout:0/473: sync 2024-09-21T04:35:09.814 INFO:tasks.workunit.client.1.smithi053.stdout:3/468: sync 2024-09-21T04:35:09.814 INFO:tasks.workunit.client.1.smithi053.stdout:5/543: sync 2024-09-21T04:35:09.814 INFO:tasks.workunit.client.1.smithi053.stdout:9/467: sync 2024-09-21T04:35:09.814 INFO:tasks.workunit.client.1.smithi053.stdout:7/413: sync 2024-09-21T04:35:09.814 INFO:tasks.workunit.client.1.smithi053.stdout:1/538: sync 2024-09-21T04:35:09.815 INFO:tasks.workunit.client.1.smithi053.stdout:6/555: readlink d2/d1b/d3d/d49/de/d32/d80/d2e/d38/l33 0 2024-09-21T04:35:09.816 INFO:tasks.workunit.client.1.smithi053.stdout:2/548: read d1/d18/d1f/d22/d43/f59 [291757,100390] 0 2024-09-21T04:35:09.818 INFO:tasks.workunit.client.1.smithi053.stdout:5/544: dread - d0/dd/d5c/d89/fb4 zero size 2024-09-21T04:35:09.820 INFO:tasks.workunit.client.1.smithi053.stdout:3/469: fdatasync d16/d17/d21/f64 0 2024-09-21T04:35:09.820 INFO:tasks.workunit.client.1.smithi053.stdout:1/539: dread - d7/d9/d43/d54/fa1 zero size 2024-09-21T04:35:09.821 INFO:tasks.workunit.client.1.smithi053.stdout:8/533: fdatasync d3/df/d14/d16/f23 0 2024-09-21T04:35:09.822 INFO:tasks.workunit.client.1.smithi053.stdout:2/549: fdatasync d1/fb 0 2024-09-21T04:35:09.823 INFO:tasks.workunit.client.1.smithi053.stdout:0/474: mknod d5/dd/ca5 0 2024-09-21T04:35:09.824 INFO:tasks.workunit.client.1.smithi053.stdout:3/470: write d16/f3d [53461,30030] 0 2024-09-21T04:35:09.824 INFO:tasks.workunit.client.1.smithi053.stdout:7/414: creat d0/d2b/d36/d48/d5c/f85 x:0 0 0 2024-09-21T04:35:09.826 INFO:tasks.workunit.client.1.smithi053.stdout:6/556: symlink d2/d1b/d3d/d49/de/d32/lb7 0 2024-09-21T04:35:09.827 INFO:tasks.workunit.client.1.smithi053.stdout:9/468: truncate d0/f4b 2348111 0 2024-09-21T04:35:09.828 INFO:tasks.workunit.client.1.smithi053.stdout:5/545: mknod d0/dd/d67/cb5 0 2024-09-21T04:35:09.830 INFO:tasks.workunit.client.1.smithi053.stdout:2/550: write d1/d18/d1f/d22/d2d/f86 [67185,64224] 0 2024-09-21T04:35:09.830 INFO:tasks.workunit.client.1.smithi053.stdout:4/580: dwrite d1/d27/f89 [4194304,4194304] 0 2024-09-21T04:35:09.831 INFO:tasks.workunit.client.1.smithi053.stdout:7/415: chown d0/f4 10225 1 2024-09-21T04:35:09.833 INFO:tasks.workunit.client.1.smithi053.stdout:0/475: rename d5/d6/l5f to d5/d6/la6 0 2024-09-21T04:35:09.836 INFO:tasks.workunit.client.1.smithi053.stdout:3/471: mknod d16/d17/d21/d8b/d7d/d57/cae 0 2024-09-21T04:35:09.836 INFO:tasks.workunit.client.1.smithi053.stdout:0/476: stat d5/dd/ff 0 2024-09-21T04:35:09.840 INFO:tasks.workunit.client.1.smithi053.stdout:5/546: mkdir d0/d8/db6 0 2024-09-21T04:35:09.844 INFO:tasks.workunit.client.1.smithi053.stdout:4/581: stat d1/d27/d53/d88/d91/d20/l21 0 2024-09-21T04:35:09.844 INFO:tasks.workunit.client.1.smithi053.stdout:0/477: creat d5/d10/d44/fa7 x:0 0 0 2024-09-21T04:35:09.844 INFO:tasks.workunit.client.1.smithi053.stdout:2/551: truncate d1/d18/d77/d83/f8d 892386 0 2024-09-21T04:35:09.845 INFO:tasks.workunit.client.1.smithi053.stdout:8/534: dwrite d3/d39/fb0 [4194304,4194304] 0 2024-09-21T04:35:09.846 INFO:tasks.workunit.client.1.smithi053.stdout:6/557: truncate d2/d1b/d3d/d49/de/d32/d80/f57 619273 0 2024-09-21T04:35:09.848 INFO:tasks.workunit.client.1.smithi053.stdout:3/472: write d16/d17/d21/d60/f9c [679372,50234] 0 2024-09-21T04:35:09.849 INFO:tasks.workunit.client.1.smithi053.stdout:4/582: fsync d1/d2c/f85 0 2024-09-21T04:35:09.851 INFO:tasks.workunit.client.1.smithi053.stdout:9/469: dwrite d0/d2/d4e/d5c/f78 [0,4194304] 0 2024-09-21T04:35:09.854 INFO:tasks.workunit.client.1.smithi053.stdout:5/547: truncate d0/dd/d48/f60 217831 0 2024-09-21T04:35:09.864 INFO:tasks.workunit.client.1.smithi053.stdout:1/540: dread d7/d63/f6a [4194304,4194304] 0 2024-09-21T04:35:09.866 INFO:tasks.workunit.client.1.smithi053.stdout:7/416: dread d0/d6f/f72 [0,4194304] 0 2024-09-21T04:35:09.870 INFO:tasks.workunit.client.1.smithi053.stdout:8/535: dread d3/d5/d4c/f8c [0,4194304] 0 2024-09-21T04:35:09.872 INFO:tasks.workunit.client.1.smithi053.stdout:1/541: mknod d7/d47/d79/d99/cb2 0 2024-09-21T04:35:09.874 INFO:tasks.workunit.client.1.smithi053.stdout:7/417: creat d0/d6f/f86 x:0 0 0 2024-09-21T04:35:09.874 INFO:tasks.workunit.client.1.smithi053.stdout:5/548: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:09.875 INFO:tasks.workunit.client.1.smithi053.stdout:4/583: creat d1/d27/d90/fc0 x:0 0 0 2024-09-21T04:35:09.876 INFO:tasks.workunit.client.1.smithi053.stdout:9/470: mknod d0/c9e 0 2024-09-21T04:35:09.876 INFO:tasks.workunit.client.1.smithi053.stdout:3/473: link d16/d17/d21/d26/f92 d16/d17/faf 0 2024-09-21T04:35:09.879 INFO:tasks.workunit.client.1.smithi053.stdout:8/536: chown d3/df/d10/fae 605 1 2024-09-21T04:35:09.880 INFO:tasks.workunit.client.1.smithi053.stdout:2/552: dwrite d1/f7b [0,4194304] 0 2024-09-21T04:35:09.880 INFO:tasks.workunit.client.1.smithi053.stdout:5/549: chown d0/d8/db/f7f 488 1 2024-09-21T04:35:09.882 INFO:tasks.workunit.client.1.smithi053.stdout:4/584: creat d1/d27/d2e/fc1 x:0 0 0 2024-09-21T04:35:09.883 INFO:tasks.workunit.client.1.smithi053.stdout:9/471: creat d0/d1c/d21/f9f x:0 0 0 2024-09-21T04:35:09.886 INFO:tasks.workunit.client.1.smithi053.stdout:9/472: fsync d0/d2/d9/d34/f56 0 2024-09-21T04:35:09.887 INFO:tasks.workunit.client.1.smithi053.stdout:1/542: truncate d7/d9/da/d11/f20 1570818 0 2024-09-21T04:35:09.891 INFO:tasks.workunit.client.1.smithi053.stdout:0/478: dread d5/de/f67 [0,4194304] 0 2024-09-21T04:35:09.892 INFO:tasks.workunit.client.1.smithi053.stdout:6/558: dwrite d2/d1b/d3d/d49/de/d32/d80/f57 [0,4194304] 0 2024-09-21T04:35:09.895 INFO:tasks.workunit.client.1.smithi053.stdout:1/543: truncate d7/d9/da/f50 662402 0 2024-09-21T04:35:09.895 INFO:tasks.workunit.client.1.smithi053.stdout:7/418: dwrite d0/f4 [0,4194304] 0 2024-09-21T04:35:09.895 INFO:tasks.workunit.client.1.smithi053.stdout:8/537: symlink d3/d5/db1/lb5 0 2024-09-21T04:35:09.895 INFO:tasks.workunit.client.1.smithi053.stdout:4/585: dread - d1/d27/d53/d88/d91/d20/d73/d92/f94 zero size 2024-09-21T04:35:09.898 INFO:tasks.workunit.client.1.smithi053.stdout:9/473: creat d0/d2/d9/d34/d58/fa0 x:0 0 0 2024-09-21T04:35:09.898 INFO:tasks.workunit.client.1.smithi053.stdout:7/419: chown d0/d2b/d36/d64 0 1 2024-09-21T04:35:09.899 INFO:tasks.workunit.client.1.smithi053.stdout:2/553: rename d1/d18/d1f/d27/d41/d4b/d4d/d89/c8f to d1/d18/d1f/d9d/cc2 0 2024-09-21T04:35:09.900 INFO:tasks.workunit.client.1.smithi053.stdout:3/474: write d16/d17/d21/f39 [181188,48301] 0 2024-09-21T04:35:09.904 INFO:tasks.workunit.client.1.smithi053.stdout:4/586: readlink d1/d27/d53/d88/d91/la7 0 2024-09-21T04:35:09.905 INFO:tasks.workunit.client.1.smithi053.stdout:6/559: readlink d2/d1b/d3d/d49/d5c/d8d/la5 0 2024-09-21T04:35:09.910 INFO:tasks.workunit.client.1.smithi053.stdout:5/550: dwrite d0/dd/d30/d35/d61/f9a [0,4194304] 0 2024-09-21T04:35:09.910 INFO:tasks.workunit.client.1.smithi053.stdout:3/475: chown d16/d17/d21/d26/d49/d4d/d53 6 1 2024-09-21T04:35:09.911 INFO:tasks.workunit.client.1.smithi053.stdout:7/420: creat d0/d2b/d36/f87 x:0 0 0 2024-09-21T04:35:09.914 INFO:tasks.workunit.client.1.smithi053.stdout:1/544: rename d7/l38 to d7/d9/da/d4c/lb3 0 2024-09-21T04:35:09.914 INFO:tasks.workunit.client.1.smithi053.stdout:2/554: mknod d1/d18/d1f/d22/cc3 0 2024-09-21T04:35:09.916 INFO:tasks.workunit.client.1.smithi053.stdout:5/551: mkdir d0/d8/db/db7 0 2024-09-21T04:35:09.916 INFO:tasks.workunit.client.1.smithi053.stdout:9/474: dread d0/d2/d4e/d5c/f78 [0,4194304] 0 2024-09-21T04:35:09.917 INFO:tasks.workunit.client.1.smithi053.stdout:0/479: dwrite d5/de/d42/f48 [0,4194304] 0 2024-09-21T04:35:09.918 INFO:tasks.workunit.client.1.smithi053.stdout:8/538: creat d3/d5/d38/d64/fb6 x:0 0 0 2024-09-21T04:35:09.919 INFO:tasks.workunit.client.1.smithi053.stdout:5/552: read - d0/dd/d48/fa9 zero size 2024-09-21T04:35:09.921 INFO:tasks.workunit.client.1.smithi053.stdout:4/587: creat d1/d27/d53/fc2 x:0 0 0 2024-09-21T04:35:09.923 INFO:tasks.workunit.client.1.smithi053.stdout:0/480: symlink d5/d10/la8 0 2024-09-21T04:35:09.924 INFO:tasks.workunit.client.1.smithi053.stdout:5/553: unlink d0/dd/d30/f6f 0 2024-09-21T04:35:09.925 INFO:tasks.workunit.client.1.smithi053.stdout:8/539: mknod d3/df/d14/cb7 0 2024-09-21T04:35:09.926 INFO:tasks.workunit.client.1.smithi053.stdout:4/588: mkdir d1/d27/d53/d88/d91/d20/d4e/d51/da4/dc3 0 2024-09-21T04:35:09.927 INFO:tasks.workunit.client.1.smithi053.stdout:3/476: dwrite d16/d17/d3c/f7e [0,4194304] 0 2024-09-21T04:35:09.928 INFO:tasks.workunit.client.1.smithi053.stdout:0/481: rename d5/de/d42/d5e/f7a to d5/d6/d53/fa9 0 2024-09-21T04:35:09.929 INFO:tasks.workunit.client.1.smithi053.stdout:8/540: symlink d3/d5/lb8 0 2024-09-21T04:35:09.930 INFO:tasks.workunit.client.1.smithi053.stdout:3/477: unlink d16/l3b 0 2024-09-21T04:35:09.933 INFO:tasks.workunit.client.1.smithi053.stdout:3/478: write d16/d1b/f1f [2935041,127756] 0 2024-09-21T04:35:09.935 INFO:tasks.workunit.client.1.smithi053.stdout:4/589: dwrite d1/d27/d2e/d46/f8d [0,4194304] 0 2024-09-21T04:35:09.937 INFO:tasks.workunit.client.1.smithi053.stdout:8/541: dread d3/d5/d4c/f6d [4194304,4194304] 0 2024-09-21T04:35:09.938 INFO:tasks.workunit.client.1.smithi053.stdout:3/479: read d16/d17/d21/d26/d49/fa6 [218663,118440] 0 2024-09-21T04:35:09.942 INFO:tasks.workunit.client.1.smithi053.stdout:3/480: mknod d16/d17/d21/d42/da0/cb0 0 2024-09-21T04:35:09.946 INFO:tasks.workunit.client.1.smithi053.stdout:3/481: rename fd to d16/d1b/dad/fb1 0 2024-09-21T04:35:09.946 INFO:tasks.workunit.client.1.smithi053.stdout:4/590: dwrite d1/d27/d53/d88/d91/d20/f45 [0,4194304] 0 2024-09-21T04:35:09.949 INFO:tasks.workunit.client.1.smithi053.stdout:4/591: unlink d1/ca9 0 2024-09-21T04:35:09.950 INFO:tasks.workunit.client.1.smithi053.stdout:8/542: dwrite d3/d5/d38/d64/fb6 [0,4194304] 0 2024-09-21T04:35:09.952 INFO:tasks.workunit.client.1.smithi053.stdout:4/592: truncate d1/d27/f86 675118 0 2024-09-21T04:35:09.954 INFO:tasks.workunit.client.1.smithi053.stdout:4/593: dread - d1/dd/de/f83 zero size 2024-09-21T04:35:09.956 INFO:tasks.workunit.client.1.smithi053.stdout:4/594: readlink d1/d27/d53/d88/d91/la7 0 2024-09-21T04:35:09.957 INFO:tasks.workunit.client.1.smithi053.stdout:3/482: dwrite d16/d17/f31 [0,4194304] 0 2024-09-21T04:35:09.960 INFO:tasks.workunit.client.1.smithi053.stdout:3/483: readlink d16/d17/d21/d26/d2b/l41 0 2024-09-21T04:35:09.962 INFO:tasks.workunit.client.1.smithi053.stdout:4/595: truncate d1/d27/d53/d88/d91/d20/d4e/f68 272097 0 2024-09-21T04:35:09.963 INFO:tasks.workunit.client.1.smithi053.stdout:3/484: creat d16/d17/d21/d4e/fb2 x:0 0 0 2024-09-21T04:35:09.965 INFO:tasks.workunit.client.1.smithi053.stdout:4/596: mknod d1/dd/d95/cc4 0 2024-09-21T04:35:09.968 INFO:tasks.workunit.client.1.smithi053.stdout:4/597: mknod d1/d27/d53/d88/d91/db6/cc5 0 2024-09-21T04:35:09.972 INFO:tasks.workunit.client.1.smithi053.stdout:3/485: dwrite d16/d17/d21/d26/d49/d4d/d53/f9a [0,4194304] 0 2024-09-21T04:35:09.974 INFO:tasks.workunit.client.1.smithi053.stdout:4/598: getdents d1/d27/d53/d88/d91/d20/d5b 0 2024-09-21T04:35:09.976 INFO:tasks.workunit.client.1.smithi053.stdout:3/486: rename d16/d1b/f24 to d16/d17/d21/d42/d43/d86/fb3 0 2024-09-21T04:35:10.001 INFO:tasks.workunit.client.1.smithi053.stdout:1/545: truncate d7/d9/da/d3c/faf 6476685 0 2024-09-21T04:35:10.004 INFO:tasks.workunit.client.1.smithi053.stdout:6/560: truncate d2/d1b/d1e/d2a/d4b/f68 2237072 0 2024-09-21T04:35:10.005 INFO:tasks.workunit.client.1.smithi053.stdout:9/475: truncate d0/d1c/f39 3869570 0 2024-09-21T04:35:10.007 INFO:tasks.workunit.client.1.smithi053.stdout:1/546: dread d7/d9/da/f50 [0,4194304] 0 2024-09-21T04:35:10.009 INFO:tasks.workunit.client.1.smithi053.stdout:9/476: mkdir d0/d1a/d89/da1 0 2024-09-21T04:35:10.009 INFO:tasks.workunit.client.1.smithi053.stdout:9/477: chown d0/d1a/f81 0 1 2024-09-21T04:35:10.010 INFO:tasks.workunit.client.1.smithi053.stdout:6/561: rename d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/d8c to d2/d1b/d1e/d2a/d4b/d77/d88/db8 0 2024-09-21T04:35:10.011 INFO:tasks.workunit.client.1.smithi053.stdout:6/562: stat d2/d1b/d1e/f3f 0 2024-09-21T04:35:10.015 INFO:tasks.workunit.client.1.smithi053.stdout:3/487: getdents d16 0 2024-09-21T04:35:10.015 INFO:tasks.workunit.client.1.smithi053.stdout:5/554: truncate d0/d8/f1f 4065759 0 2024-09-21T04:35:10.017 INFO:tasks.workunit.client.1.smithi053.stdout:0/482: truncate d5/fa 401358 0 2024-09-21T04:35:10.019 INFO:tasks.workunit.client.1.smithi053.stdout:0/483: read - d5/f92 zero size 2024-09-21T04:35:10.021 INFO:tasks.workunit.client.1.smithi053.stdout:9/478: dwrite d0/d28/f38 [0,4194304] 0 2024-09-21T04:35:10.021 INFO:tasks.workunit.client.1.smithi053.stdout:3/488: unlink d16/d17/d21/d26/d3e/c95 0 2024-09-21T04:35:10.021 INFO:tasks.workunit.client.1.smithi053.stdout:0/484: fsync d5/de/f5b 0 2024-09-21T04:35:10.022 INFO:tasks.workunit.client.1.smithi053.stdout:1/547: dread d7/d9/da/d11/f5c [0,4194304] 0 2024-09-21T04:35:10.023 INFO:tasks.workunit.client.1.smithi053.stdout:8/543: truncate d3/d5/f8f 959618 0 2024-09-21T04:35:10.026 INFO:tasks.workunit.client.1.smithi053.stdout:8/544: chown d3/df/d49/f55 26 1 2024-09-21T04:35:10.028 INFO:tasks.workunit.client.1.smithi053.stdout:5/555: dwrite d0/d8/db/d68/f97 [0,4194304] 0 2024-09-21T04:35:10.032 INFO:tasks.workunit.client.1.smithi053.stdout:0/485: mknod d5/de/d42/d5e/caa 0 2024-09-21T04:35:10.034 INFO:tasks.workunit.client.1.smithi053.stdout:4/599: write d1/d27/d53/d88/d91/d20/d73/d92/fb7 [530535,6763] 0 2024-09-21T04:35:10.038 INFO:tasks.workunit.client.1.smithi053.stdout:3/489: dread d16/d17/d3c/f79 [0,4194304] 0 2024-09-21T04:35:10.042 INFO:tasks.workunit.client.1.smithi053.stdout:5/556: dread d0/dd/f37 [0,4194304] 0 2024-09-21T04:35:10.044 INFO:tasks.workunit.client.1.smithi053.stdout:5/557: fdatasync d0/dd/d6d/fa6 0 2024-09-21T04:35:10.047 INFO:tasks.workunit.client.1.smithi053.stdout:1/548: creat d7/d47/d79/d99/db0/fb4 x:0 0 0 2024-09-21T04:35:10.052 INFO:tasks.workunit.client.1.smithi053.stdout:8/545: dread d3/d5/d21/f78 [0,4194304] 0 2024-09-21T04:35:10.054 INFO:tasks.workunit.client.1.smithi053.stdout:9/479: getdents d0/d28 0 2024-09-21T04:35:10.055 INFO:tasks.workunit.client.1.smithi053.stdout:9/480: write d0/d1c/d21/f9f [223794,26988] 0 2024-09-21T04:35:10.056 INFO:tasks.workunit.client.1.smithi053.stdout:5/558: rename d0/dd/d5c/d80/fa2 to d0/d8/db/fb8 0 2024-09-21T04:35:10.057 INFO:tasks.workunit.client.1.smithi053.stdout:7/421: sync 2024-09-21T04:35:10.058 INFO:tasks.workunit.client.1.smithi053.stdout:5/559: truncate d0/dd/d5c/d80/f8a 207566 0 2024-09-21T04:35:10.059 INFO:tasks.workunit.client.1.smithi053.stdout:0/486: creat d5/fab x:0 0 0 2024-09-21T04:35:10.060 INFO:tasks.workunit.client.1.smithi053.stdout:5/560: dread - d0/dd/d5c/d89/fb4 zero size 2024-09-21T04:35:10.061 INFO:tasks.workunit.client.1.smithi053.stdout:8/546: rename d3/d39/f6b to d3/d5/d38/d64/d9d/fb9 0 2024-09-21T04:35:10.061 INFO:tasks.workunit.client.1.smithi053.stdout:1/549: dread d7/d9/da/f86 [0,4194304] 0 2024-09-21T04:35:10.062 INFO:tasks.workunit.client.1.smithi053.stdout:3/490: rmdir d16/d17/d21/d4e/d51 0 2024-09-21T04:35:10.063 INFO:tasks.workunit.client.1.smithi053.stdout:8/547: write d3/d5/d38/d64/d73/d91/fa4 [410609,67644] 0 2024-09-21T04:35:10.064 INFO:tasks.workunit.client.1.smithi053.stdout:0/487: symlink d5/de/d42/d5e/lac 0 2024-09-21T04:35:10.065 INFO:tasks.workunit.client.1.smithi053.stdout:8/548: chown d3/d5/d4b/f7e 18 1 2024-09-21T04:35:10.066 INFO:tasks.workunit.client.1.smithi053.stdout:0/488: write d5/dd/d50/f6e [918655,92150] 0 2024-09-21T04:35:10.066 INFO:tasks.workunit.client.1.smithi053.stdout:8/549: stat d3/d4f/c58 0 2024-09-21T04:35:10.067 INFO:tasks.workunit.client.1.smithi053.stdout:8/550: chown d3/d5/d38/d64/l75 32564785 1 2024-09-21T04:35:10.070 INFO:tasks.workunit.client.1.smithi053.stdout:5/561: getdents d0/dd/d48/d8c/dad 0 2024-09-21T04:35:10.071 INFO:tasks.workunit.client.1.smithi053.stdout:9/481: link d0/d6c/c75 d0/d2/d2b/ca2 0 2024-09-21T04:35:10.071 INFO:tasks.workunit.client.1.smithi053.stdout:9/482: stat d0/d1c/d25/f54 0 2024-09-21T04:35:10.072 INFO:tasks.workunit.client.1.smithi053.stdout:9/483: stat d0/d1c/d21/f9f 0 2024-09-21T04:35:10.072 INFO:tasks.workunit.client.1.smithi053.stdout:9/484: chown d0/d1a/d4c/d8f/c98 12 1 2024-09-21T04:35:10.073 INFO:tasks.workunit.client.1.smithi053.stdout:7/422: rename d0/d24/f7e to d0/d6/d70/f88 0 2024-09-21T04:35:10.075 INFO:tasks.workunit.client.1.smithi053.stdout:8/551: mkdir d3/df/dba 0 2024-09-21T04:35:10.075 INFO:tasks.workunit.client.1.smithi053.stdout:8/552: fsync d3/df/f42 0 2024-09-21T04:35:10.076 INFO:tasks.workunit.client.1.smithi053.stdout:8/553: fsync d3/d5/f19 0 2024-09-21T04:35:10.076 INFO:tasks.workunit.client.1.smithi053.stdout:3/491: creat d16/d17/d21/d8b/d7d/d9d/fb4 x:0 0 0 2024-09-21T04:35:10.077 INFO:tasks.workunit.client.1.smithi053.stdout:8/554: stat d3/df/dba 0 2024-09-21T04:35:10.078 INFO:tasks.workunit.client.1.smithi053.stdout:8/555: chown d3/d5/l84 14220757 1 2024-09-21T04:35:10.078 INFO:tasks.workunit.client.1.smithi053.stdout:9/485: mknod d0/d6c/ca3 0 2024-09-21T04:35:10.079 INFO:tasks.workunit.client.1.smithi053.stdout:0/489: mknod d5/d10/cad 0 2024-09-21T04:35:10.081 INFO:tasks.workunit.client.1.smithi053.stdout:5/562: mkdir d0/dd/d48/d8c/dad/db9 0 2024-09-21T04:35:10.082 INFO:tasks.workunit.client.1.smithi053.stdout:5/563: dread - d0/d8/db/fb8 zero size 2024-09-21T04:35:10.082 INFO:tasks.workunit.client.1.smithi053.stdout:3/492: rename d16/d17/d21/c2e to d16/d1b/dad/cb5 0 2024-09-21T04:35:10.084 INFO:tasks.workunit.client.1.smithi053.stdout:7/423: link d0/d6/f83 d0/d2b/d36/f89 0 2024-09-21T04:35:10.084 INFO:tasks.workunit.client.1.smithi053.stdout:7/424: readlink d0/le 0 2024-09-21T04:35:10.085 INFO:tasks.workunit.client.1.smithi053.stdout:1/550: unlink d7/d9/da/d3c/faf 0 2024-09-21T04:35:10.086 INFO:tasks.workunit.client.1.smithi053.stdout:8/556: link d3/d5/d4c/f8c d3/df/d10/fbb 0 2024-09-21T04:35:10.087 INFO:tasks.workunit.client.1.smithi053.stdout:3/493: dread d16/d17/d21/f64 [0,4194304] 0 2024-09-21T04:35:10.087 INFO:tasks.workunit.client.1.smithi053.stdout:9/486: rename d0/f4 to d0/d1a/d4c/fa4 0 2024-09-21T04:35:10.089 INFO:tasks.workunit.client.1.smithi053.stdout:7/425: creat d0/d2b/d36/d48/d5c/f8a x:0 0 0 2024-09-21T04:35:10.090 INFO:tasks.workunit.client.1.smithi053.stdout:8/557: symlink d3/d5/lbc 0 2024-09-21T04:35:10.091 INFO:tasks.workunit.client.1.smithi053.stdout:3/494: unlink d16/d17/d21/d42/l7b 0 2024-09-21T04:35:10.092 INFO:tasks.workunit.client.1.smithi053.stdout:5/564: dwrite d0/dd/d48/f54 [0,4194304] 0 2024-09-21T04:35:10.093 INFO:tasks.workunit.client.1.smithi053.stdout:9/487: rename d0/d2/d9/c17 to d0/d1c/d21/d8e/ca5 0 2024-09-21T04:35:10.093 INFO:tasks.workunit.client.1.smithi053.stdout:0/490: dread d5/dd/f34 [0,4194304] 0 2024-09-21T04:35:10.095 INFO:tasks.workunit.client.1.smithi053.stdout:3/495: write d16/d17/d21/d26/fa7 [152105,69370] 0 2024-09-21T04:35:10.096 INFO:tasks.workunit.client.1.smithi053.stdout:3/496: chown d16/d17/d21/d42/da0/la4 110077881 1 2024-09-21T04:35:10.096 INFO:tasks.workunit.client.1.smithi053.stdout:8/558: mknod d3/d5/d21/cbd 0 2024-09-21T04:35:10.099 INFO:tasks.workunit.client.1.smithi053.stdout:0/491: rmdir d5/dd/d50 39 2024-09-21T04:35:10.100 INFO:tasks.workunit.client.1.smithi053.stdout:9/488: write d0/d1a/d4c/f94 [198465,125559] 0 2024-09-21T04:35:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.098+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fd08c1a7e60 con 0x7fd08c10bb20 2024-09-21T04:35:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.099+0000 7fd082ffd640 1 -- 172.21.15.7:0/3893378791 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+841 (secure 0 0 0) 0x7fd07c061380 con 0x7fd08c10bb20 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:35:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:35:10.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:35:10.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:35:10.103 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:35:10.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 13, 2024-09-21T04:35:10.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:35:10.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:35:10.103 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:35:10.103 INFO:tasks.workunit.client.1.smithi053.stdout:8/559: mknod d3/df/cbe 0 2024-09-21T04:35:10.104 INFO:tasks.workunit.client.1.smithi053.stdout:3/497: mknod d16/d17/d21/cb6 0 2024-09-21T04:35:10.104 INFO:tasks.workunit.client.1.smithi053.stdout:9/489: write d0/d1a/d4c/f86 [564925,116639] 0 2024-09-21T04:35:10.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd078074ad0 msgr2=0x7fd078076f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:10.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd078074ad0 0x7fd078076f90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0880096c0 tx=0x7fd088000bc0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c10bb20 msgr2=0x7fd08c10f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:10.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c10bb20 0x7fd08c10f720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00b3f0 tx=0x7fd07c00b8b0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 shutdown_connections 2024-09-21T04:35:10.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fd078074ad0 0x7fd078076f90 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0880096c0 tx=0x7fd088000bc0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08c10bb20 0x7fd08c10f720 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00b3f0 tx=0x7fd07c00b8b0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 --2- 172.21.15.7:0/3893378791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd08c1091f0 0x7fd08c111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.102+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 >> 172.21.15.7:0/3893378791 conn(0x7fd08c06f250 msgr2=0x7fd08c071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.103+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 shutdown_connections 2024-09-21T04:35:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.103+0000 7fd093a97640 1 -- 172.21.15.7:0/3893378791 wait complete. 2024-09-21T04:35:10.107 INFO:tasks.workunit.client.1.smithi053.stdout:3/498: dread d16/d17/d21/f64 [0,4194304] 0 2024-09-21T04:35:10.108 INFO:tasks.workunit.client.1.smithi053.stdout:9/490: symlink d0/d1c/d21/la6 0 2024-09-21T04:35:10.109 INFO:tasks.workunit.client.1.smithi053.stdout:0/492: dwrite d5/d6/d21/d96/d9f/f86 [0,4194304] 0 2024-09-21T04:35:10.111 INFO:tasks.workunit.client.1.smithi053.stdout:3/499: creat d16/d17/d21/d26/d2b/fb7 x:0 0 0 2024-09-21T04:35:10.114 INFO:tasks.workunit.client.1.smithi053.stdout:0/493: symlink d5/d6/d21/d96/lae 0 2024-09-21T04:35:10.114 INFO:tasks.workunit.client.1.smithi053.stdout:0/494: write d5/de/f3e [539066,36738] 0 2024-09-21T04:35:10.115 INFO:tasks.workunit.client.1.smithi053.stdout:7/426: rename d0/d6 to d0/d2b/d36/d48/d50/d8b 0 2024-09-21T04:35:10.117 INFO:tasks.workunit.client.1.smithi053.stdout:3/500: mknod d16/d17/d21/d26/d49/d4d/cb8 0 2024-09-21T04:35:10.119 INFO:tasks.workunit.client.1.smithi053.stdout:0/495: readlink d5/d6/l70 0 2024-09-21T04:35:10.119 INFO:tasks.workunit.client.1.smithi053.stdout:9/491: dwrite d0/d1a/d4c/d8f/f97 [0,4194304] 0 2024-09-21T04:35:10.121 INFO:tasks.workunit.client.1.smithi053.stdout:5/565: rename d0/d8/d2a/f9d to d0/dd/d48/d8c/dad/db9/fba 0 2024-09-21T04:35:10.124 INFO:tasks.workunit.client.1.smithi053.stdout:7/427: creat d0/d2b/d36/d48/d5d/f8c x:0 0 0 2024-09-21T04:35:10.126 INFO:tasks.workunit.client.1.smithi053.stdout:3/501: symlink d16/d17/d21/d26/d49/lb9 0 2024-09-21T04:35:10.131 INFO:tasks.workunit.client.1.smithi053.stdout:8/560: rename d3/df/d14/cb7 to d3/d5/d4b/d5c/d86/cbf 0 2024-09-21T04:35:10.133 INFO:tasks.workunit.client.1.smithi053.stdout:7/428: creat d0/d6e/f8d x:0 0 0 2024-09-21T04:35:10.135 INFO:tasks.workunit.client.1.smithi053.stdout:3/502: symlink d16/d17/d21/d42/lba 0 2024-09-21T04:35:10.136 INFO:tasks.workunit.client.1.smithi053.stdout:9/492: unlink d0/d1a/c36 0 2024-09-21T04:35:10.138 INFO:tasks.workunit.client.1.smithi053.stdout:7/429: symlink d0/d24/l8e 0 2024-09-21T04:35:10.139 INFO:tasks.workunit.client.1.smithi053.stdout:5/566: dwrite d0/dd/d30/d51/d52/fa7 [0,4194304] 0 2024-09-21T04:35:10.142 INFO:tasks.workunit.client.1.smithi053.stdout:8/561: creat d3/d5/d4c/d8e/fc0 x:0 0 0 2024-09-21T04:35:10.142 INFO:tasks.workunit.client.1.smithi053.stdout:3/503: mkdir d16/d17/d21/d26/d2b/dbb 0 2024-09-21T04:35:10.145 INFO:tasks.workunit.client.1.smithi053.stdout:5/567: creat d0/dd/d30/d51/d52/fbb x:0 0 0 2024-09-21T04:35:10.146 INFO:tasks.workunit.client.1.smithi053.stdout:8/562: symlink d3/d5/d4c/d8e/lc1 0 2024-09-21T04:35:10.147 INFO:tasks.workunit.client.1.smithi053.stdout:9/493: dwrite d0/d2/fe [4194304,4194304] 0 2024-09-21T04:35:10.148 INFO:tasks.workunit.client.1.smithi053.stdout:8/563: dread - d3/df/f81 zero size 2024-09-21T04:35:10.150 INFO:tasks.workunit.client.1.smithi053.stdout:0/496: write d5/de/d1c/f3d [2107950,42598] 0 2024-09-21T04:35:10.150 INFO:tasks.workunit.client.1.smithi053.stdout:7/430: link d0/d24/d46/f7d d0/d2b/d36/d48/f8f 0 2024-09-21T04:35:10.151 INFO:tasks.workunit.client.1.smithi053.stdout:0/497: write d5/f4a [1276751,97422] 0 2024-09-21T04:35:10.152 INFO:tasks.workunit.client.1.smithi053.stdout:7/431: dread - d0/d24/d46/f4a zero size 2024-09-21T04:35:10.153 INFO:tasks.workunit.client.1.smithi053.stdout:9/494: creat d0/d6c/fa7 x:0 0 0 2024-09-21T04:35:10.154 INFO:tasks.workunit.client.1.smithi053.stdout:8/564: mknod d3/d5/d4c/d53/cc2 0 2024-09-21T04:35:10.156 INFO:tasks.workunit.client.1.smithi053.stdout:9/495: truncate d0/d2/d9/f95 1204790 0 2024-09-21T04:35:10.156 INFO:tasks.workunit.client.1.smithi053.stdout:7/432: fsync d0/d2b/d36/d48/d50/d8b/f1c 0 2024-09-21T04:35:10.157 INFO:tasks.workunit.client.1.smithi053.stdout:5/568: mkdir d0/d8/db6/dbc 0 2024-09-21T04:35:10.158 INFO:tasks.workunit.client.1.smithi053.stdout:7/433: write d0/d2b/d36/f87 [951647,108962] 0 2024-09-21T04:35:10.159 INFO:tasks.workunit.client.1.smithi053.stdout:7/434: chown d0/c2d 401773944 1 2024-09-21T04:35:10.160 INFO:tasks.workunit.client.1.smithi053.stdout:3/504: link d16/c29 d16/d17/d21/d42/d43/d86/cbc 0 2024-09-21T04:35:10.161 INFO:tasks.workunit.client.1.smithi053.stdout:5/569: unlink d0/l4c 0 2024-09-21T04:35:10.162 INFO:tasks.workunit.client.1.smithi053.stdout:8/565: mkdir d3/d5/d4c/dc3 0 2024-09-21T04:35:10.163 INFO:tasks.workunit.client.1.smithi053.stdout:9/496: mknod d0/d1a/ca8 0 2024-09-21T04:35:10.166 INFO:tasks.workunit.client.1.smithi053.stdout:5/570: creat d0/dd/d5c/d89/fbd x:0 0 0 2024-09-21T04:35:10.166 INFO:tasks.workunit.client.1.smithi053.stdout:8/566: dread - d3/d28/f5b zero size 2024-09-21T04:35:10.170 INFO:tasks.workunit.client.1.smithi053.stdout:5/571: mknod d0/dd/cbe 0 2024-09-21T04:35:10.171 INFO:tasks.workunit.client.1.smithi053.stdout:3/505: getdents d16/d17/d21/d26/d2b 0 2024-09-21T04:35:10.171 INFO:tasks.workunit.client.1.smithi053.stdout:9/497: dwrite d0/d1c/d21/d46/f91 [4194304,4194304] 0 2024-09-21T04:35:10.171 INFO:tasks.workunit.client.1.smithi053.stdout:2/555: sync 2024-09-21T04:35:10.171 INFO:tasks.workunit.client.1.smithi053.stdout:6/563: sync 2024-09-21T04:35:10.171 INFO:tasks.workunit.client.1.smithi053.stdout:5/572: chown d0/d8/d2a/l50 1437 1 2024-09-21T04:35:10.173 INFO:tasks.workunit.client.1.smithi053.stdout:9/498: chown d0/d1c/d21/d8e 61425998 1 2024-09-21T04:35:10.173 INFO:tasks.workunit.client.1.smithi053.stdout:7/435: truncate d0/d2b/d36/d48/d50/d8b/d70/f7f 549371 0 2024-09-21T04:35:10.175 INFO:tasks.workunit.client.1.smithi053.stdout:3/506: chown d16/d17/d3c/l7c 0 1 2024-09-21T04:35:10.175 INFO:tasks.workunit.client.1.smithi053.stdout:6/564: dread - d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/fb2 zero size 2024-09-21T04:35:10.177 INFO:tasks.workunit.client.1.smithi053.stdout:8/567: truncate d3/d39/f2f 1284271 0 2024-09-21T04:35:10.178 INFO:tasks.workunit.client.1.smithi053.stdout:5/573: getdents d0/d8/db6/dbc 0 2024-09-21T04:35:10.180 INFO:tasks.workunit.client.1.smithi053.stdout:6/565: read d2/fb [2372790,103542] 0 2024-09-21T04:35:10.180 INFO:tasks.workunit.client.1.smithi053.stdout:8/568: chown d3/d5/d21/c5f 8 1 2024-09-21T04:35:10.180 INFO:tasks.workunit.client.1.smithi053.stdout:7/436: truncate d0/d2b/d36/d48/d50/f54 184616 0 2024-09-21T04:35:10.181 INFO:tasks.workunit.client.1.smithi053.stdout:9/499: symlink d0/d1c/d52/d6f/la9 0 2024-09-21T04:35:10.182 INFO:tasks.workunit.client.1.smithi053.stdout:3/507: mknod d16/d1b/dad/cbd 0 2024-09-21T04:35:10.183 INFO:tasks.workunit.client.1.smithi053.stdout:2/556: dread d1/d18/d1f/d27/f7a [0,4194304] 0 2024-09-21T04:35:10.184 INFO:tasks.workunit.client.1.smithi053.stdout:9/500: stat d0/d2/d9/d34/f56 0 2024-09-21T04:35:10.186 INFO:tasks.workunit.client.1.smithi053.stdout:2/557: write d1/f6d [479102,62034] 0 2024-09-21T04:35:10.186 INFO:tasks.workunit.client.1.smithi053.stdout:7/437: write d0/d6f/f72 [3992348,116670] 0 2024-09-21T04:35:10.187 INFO:tasks.workunit.client.1.smithi053.stdout:8/569: truncate d3/d5/d38/d64/d9d/fb9 303419 0 2024-09-21T04:35:10.188 INFO:tasks.workunit.client.1.smithi053.stdout:7/438: dread - d0/d2b/d36/f5e zero size 2024-09-21T04:35:10.189 INFO:tasks.workunit.client.1.smithi053.stdout:2/558: write d1/d18/d1f/d22/d54/f85 [992660,8237] 0 2024-09-21T04:35:10.190 INFO:tasks.workunit.client.1.smithi053.stdout:9/501: fdatasync d0/d1c/d25/f54 0 2024-09-21T04:35:10.191 INFO:tasks.workunit.client.1.smithi053.stdout:7/439: chown d0/f23 1 1 2024-09-21T04:35:10.192 INFO:tasks.workunit.client.1.smithi053.stdout:2/559: write d1/d18/d1f/d9d/fa8 [1100621,122226] 0 2024-09-21T04:35:10.192 INFO:tasks.workunit.client.1.smithi053.stdout:3/508: unlink d16/d17/d21/d42/l84 0 2024-09-21T04:35:10.193 INFO:tasks.workunit.client.1.smithi053.stdout:9/502: dread - d0/d2/d9/d34/d58/fa0 zero size 2024-09-21T04:35:10.194 INFO:tasks.workunit.client.1.smithi053.stdout:2/560: stat d1/d18/d1f/d27/d41/d4b/d4d/c7f 0 2024-09-21T04:35:10.194 INFO:tasks.workunit.client.1.smithi053.stdout:9/503: fdatasync d0/d1c/d25/f54 0 2024-09-21T04:35:10.199 INFO:tasks.workunit.client.1.smithi053.stdout:7/440: symlink d0/d6e/l90 0 2024-09-21T04:35:10.200 INFO:tasks.workunit.client.1.smithi053.stdout:5/574: dread d0/d8/f2b [0,4194304] 0 2024-09-21T04:35:10.200 INFO:tasks.workunit.client.1.smithi053.stdout:3/509: symlink d16/lbe 0 2024-09-21T04:35:10.201 INFO:tasks.workunit.client.1.smithi053.stdout:6/566: dwrite d2/d1b/d1e/d2a/f99 [0,4194304] 0 2024-09-21T04:35:10.202 INFO:tasks.workunit.client.1.smithi053.stdout:5/575: fsync d0/dd/d5c/d80/f8a 0 2024-09-21T04:35:10.202 INFO:tasks.workunit.client.1.smithi053.stdout:5/576: stat d0/c36 0 2024-09-21T04:35:10.204 INFO:tasks.workunit.client.1.smithi053.stdout:5/577: readlink d0/dd/d48/d8c/la1 0 2024-09-21T04:35:10.205 INFO:tasks.workunit.client.1.smithi053.stdout:9/504: mkdir d0/d1c/d52/daa 0 2024-09-21T04:35:10.208 INFO:tasks.workunit.client.1.smithi053.stdout:6/567: dread d2/d1b/d1e/d2a/d4b/d77/d88/db8/fa6 [0,4194304] 0 2024-09-21T04:35:10.209 INFO:tasks.workunit.client.1.smithi053.stdout:8/570: stat d3/d39/f2f 0 2024-09-21T04:35:10.210 INFO:tasks.workunit.client.1.smithi053.stdout:2/561: dread d1/d18/d1f/d27/d41/f42 [0,4194304] 0 2024-09-21T04:35:10.211 INFO:tasks.workunit.client.1.smithi053.stdout:2/562: chown d1/d18/d1f/d22/d54/ca5 262014 1 2024-09-21T04:35:10.216 INFO:tasks.workunit.client.1.smithi053.stdout:6/568: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/d38/f36 [0,4194304] 0 2024-09-21T04:35:10.227 INFO:tasks.workunit.client.1.smithi053.stdout:3/510: creat d16/d17/d21/d26/d49/d4d/d53/d82/fbf x:0 0 0 2024-09-21T04:35:10.229 INFO:tasks.workunit.client.1.smithi053.stdout:3/511: fdatasync d16/d17/d21/d8b/d7d/d9d/fb4 0 2024-09-21T04:35:10.230 INFO:tasks.workunit.client.1.smithi053.stdout:8/571: rmdir d3/d5/d38/d64 39 2024-09-21T04:35:10.231 INFO:tasks.workunit.client.1.smithi053.stdout:3/512: fsync d16/d17/d21/d8b/d7d/f4a 0 2024-09-21T04:35:10.231 INFO:tasks.workunit.client.1.smithi053.stdout:7/441: dwrite d0/d2b/d36/d48/f8f [0,4194304] 0 2024-09-21T04:35:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.232+0000 7f84b54c2640 1 -- 172.21.15.7:0/1118278597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 msgr2=0x7f84b0101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.232+0000 7f84b54c2640 1 --2- 172.21.15.7:0/1118278597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 0x7f84b0101f00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009880 tx=0x7f84a402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.232+0000 7f84b54c2640 1 -- 172.21.15.7:0/1118278597 shutdown_connections 2024-09-21T04:35:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.232+0000 7f84b54c2640 1 --2- 172.21.15.7:0/1118278597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 0x7f84b0101f00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.232+0000 7f84b54c2640 1 --2- 172.21.15.7:0/1118278597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b0100880 0x7f84b0100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.232+0000 7f84b54c2640 1 -- 172.21.15.7:0/1118278597 >> 172.21.15.7:0/1118278597 conn(0x7f84b00fc030 msgr2=0x7f84b00fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.232+0000 7f84b54c2640 1 -- 172.21.15.7:0/1118278597 shutdown_connections 2024-09-21T04:35:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.232+0000 7f84b54c2640 1 -- 172.21.15.7:0/1118278597 wait complete. 2024-09-21T04:35:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.233+0000 7f84b54c2640 1 Processor -- start 2024-09-21T04:35:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84b54c2640 1 -- start start 2024-09-21T04:35:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84b54c2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b0100880 0x7f84b006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:10.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84b54c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 0x7f84b019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:10.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84b54c2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84b01096a0 con 0x7f84b0101a80 2024-09-21T04:35:10.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84b54c2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84b0109520 con 0x7f84b0100880 2024-09-21T04:35:10.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b0100880 0x7f84b006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:10.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84ae7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 0x7f84b019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84ae7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 0x7f84b019c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35848/0 (socket says 172.21.15.7:35848) 2024-09-21T04:35:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.234+0000 7f84ae7fc640 1 -- 172.21.15.7:0/694021094 learned_addr learned my addr 172.21.15.7:0/694021094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.235+0000 7f84ae7fc640 1 -- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b0100880 msgr2=0x7f84b006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.235+0000 7f84ae7fc640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b0100880 0x7f84b006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.235+0000 7f84ae7fc640 1 -- 172.21.15.7:0/694021094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b019c800 con 0x7f84b0101a80 2024-09-21T04:35:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.235+0000 7f84ae7fc640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 0x7f84b019c210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009230 tx=0x7f84a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.235+0000 7f8497fff640 1 -- 172.21.15.7:0/694021094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f84a403f070 con 0x7f84b0101a80 2024-09-21T04:35:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.236+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b019ca90 con 0x7f84b0101a80 2024-09-21T04:35:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.236+0000 7f8497fff640 1 -- 172.21.15.7:0/694021094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f84a4004b80 con 0x7f84b0101a80 2024-09-21T04:35:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.236+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b019cfc0 con 0x7f84b0101a80 2024-09-21T04:35:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.236+0000 7f8497fff640 1 -- 172.21.15.7:0/694021094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f84a4035030 con 0x7f84b0101a80 2024-09-21T04:35:10.240 INFO:tasks.workunit.client.1.smithi053.stdout:5/578: truncate d0/d8/f1f 996149 0 2024-09-21T04:35:10.240 INFO:tasks.workunit.client.1.smithi053.stdout:5/579: stat d0/dd/f37 0 2024-09-21T04:35:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.237+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b0100c80 con 0x7f84b0101a80 2024-09-21T04:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.239+0000 7f8497fff640 1 -- 172.21.15.7:0/694021094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f84a4042070 con 0x7f84b0101a80 2024-09-21T04:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.240+0000 7f8497fff640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f8498074a20 0x7f8498076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.240+0000 7f8497fff640 1 -- 172.21.15.7:0/694021094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f84a40b7380 con 0x7f84b0101a80 2024-09-21T04:35:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.241+0000 7f84aeffd640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f8498074a20 0x7f8498076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.242+0000 7f84aeffd640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f8498074a20 0x7f8498076ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84b01018e0 tx=0x7f84a000b390 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:10.246 INFO:tasks.workunit.client.1.smithi053.stdout:8/572: write d3/d5/d38/d64/d73/d91/fa4 [705507,72697] 0 2024-09-21T04:35:10.248 INFO:tasks.workunit.client.1.smithi053.stdout:5/580: dwrite d0/f14 [0,4194304] 0 2024-09-21T04:35:10.249 INFO:tasks.workunit.client.1.smithi053.stdout:2/563: mknod d1/d18/d1f/d27/d58/d5c/d91/dc1/cc4 0 2024-09-21T04:35:10.249 INFO:tasks.workunit.client.1.smithi053.stdout:8/573: chown d3/d5/d4c/d8e/fc0 188578716 1 2024-09-21T04:35:10.250 INFO:tasks.workunit.client.1.smithi053.stdout:8/574: readlink d3/d5/d21/l7b 0 2024-09-21T04:35:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.248+0000 7f8497fff640 1 -- 172.21.15.7:0/694021094 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f84a40828a0 con 0x7f84b0101a80 2024-09-21T04:35:10.261 INFO:tasks.workunit.client.1.smithi053.stdout:9/505: truncate d0/d2/d9/d1f/f26 2432016 0 2024-09-21T04:35:10.262 INFO:tasks.workunit.client.1.smithi053.stdout:5/581: creat d0/d8/db/fbf x:0 0 0 2024-09-21T04:35:10.263 INFO:tasks.workunit.client.1.smithi053.stdout:2/564: rmdir d1/d18/d1f/d27/d41/d4b/d4d/d89 39 2024-09-21T04:35:10.263 INFO:tasks.workunit.client.1.smithi053.stdout:3/513: dwrite d16/d17/d21/d4e/fb2 [0,4194304] 0 2024-09-21T04:35:10.264 INFO:tasks.workunit.client.1.smithi053.stdout:9/506: dread - d0/d1c/d25/f9a zero size 2024-09-21T04:35:10.264 INFO:tasks.workunit.client.1.smithi053.stdout:8/575: mknod d3/d5/d1a/cc4 0 2024-09-21T04:35:10.264 INFO:tasks.workunit.client.1.smithi053.stdout:2/565: chown d1/d18/d1f/d27/d41 16456358 1 2024-09-21T04:35:10.267 INFO:tasks.workunit.client.1.smithi053.stdout:9/507: write d0/d1c/d21/d8e/f69 [146633,85625] 0 2024-09-21T04:35:10.267 INFO:tasks.workunit.client.1.smithi053.stdout:2/566: fsync d1/d18/d1f/d27/fa0 0 2024-09-21T04:35:10.269 INFO:tasks.workunit.client.1.smithi053.stdout:3/514: truncate d16/d17/d21/d26/d49/f77 636978 0 2024-09-21T04:35:10.270 INFO:tasks.workunit.client.1.smithi053.stdout:7/442: link d0/d2b/d36/d48/d5d/c69 d0/d6e/c91 0 2024-09-21T04:35:10.271 INFO:tasks.workunit.client.1.smithi053.stdout:8/576: rmdir d3/d39 39 2024-09-21T04:35:10.272 INFO:tasks.workunit.client.1.smithi053.stdout:7/443: write d0/d2b/d36/f87 [1745684,78456] 0 2024-09-21T04:35:10.273 INFO:tasks.workunit.client.1.smithi053.stdout:3/515: truncate d16/d17/d21/f64 716388 0 2024-09-21T04:35:10.276 INFO:tasks.workunit.client.1.smithi053.stdout:9/508: rename d0/d2/d4e/f5f to d0/d1a/d4c/fab 0 2024-09-21T04:35:10.282 INFO:tasks.workunit.client.1.smithi053.stdout:8/577: creat d3/d5/d4b/d5c/d86/fc5 x:0 0 0 2024-09-21T04:35:10.284 INFO:tasks.workunit.client.1.smithi053.stdout:7/444: symlink d0/d2b/d36/d48/d5d/l92 0 2024-09-21T04:35:10.286 INFO:tasks.workunit.client.1.smithi053.stdout:3/516: symlink d16/d17/d21/d42/lc0 0 2024-09-21T04:35:10.289 INFO:tasks.workunit.client.1.smithi053.stdout:5/582: getdents d0/d66 0 2024-09-21T04:35:10.289 INFO:tasks.workunit.client.1.smithi053.stdout:3/517: dread - d16/d17/d21/d26/d49/d4d/d53/d82/fbf zero size 2024-09-21T04:35:10.290 INFO:tasks.workunit.client.1.smithi053.stdout:2/567: chown d1/d18/d1f/d27/d41/d4b/d4d/d89/lbd 4908 1 2024-09-21T04:35:10.292 INFO:tasks.workunit.client.1.smithi053.stdout:5/583: fsync d0/dd/d5c/d89/fbd 0 2024-09-21T04:35:10.292 INFO:tasks.workunit.client.1.smithi053.stdout:8/578: dwrite d3/d28/fad [0,4194304] 0 2024-09-21T04:35:10.292 INFO:tasks.workunit.client.1.smithi053.stdout:9/509: fsync d0/d1a/d4c/fa4 0 2024-09-21T04:35:10.294 INFO:tasks.workunit.client.1.smithi053.stdout:8/579: write d3/d28/f8b [982678,102565] 0 2024-09-21T04:35:10.301 INFO:tasks.workunit.client.1.smithi053.stdout:4/600: sync 2024-09-21T04:35:10.305 INFO:tasks.workunit.client.1.smithi053.stdout:3/518: rename d16/d17/d21/d26/d49/d4d to d16/d17/d21/d8b/d7d/d57/dc1 0 2024-09-21T04:35:10.307 INFO:tasks.workunit.client.1.smithi053.stdout:7/445: link d0/d2b/d36/d48/d5c/l71 d0/d2b/d36/d48/d5d/l93 0 2024-09-21T04:35:10.309 INFO:tasks.workunit.client.1.smithi053.stdout:9/510: dwrite d0/d2/d4e/d5c/f78 [0,4194304] 0 2024-09-21T04:35:10.310 INFO:tasks.workunit.client.1.smithi053.stdout:4/601: fdatasync d1/d27/d53/d88/d91/d3f/f50 0 2024-09-21T04:35:10.312 INFO:tasks.workunit.client.1.smithi053.stdout:2/568: creat d1/d18/d1f/da2/dac/fc5 x:0 0 0 2024-09-21T04:35:10.315 INFO:tasks.workunit.client.1.smithi053.stdout:2/569: truncate d1/d18/d1f/d27/d58/d5c/fc0 4304 0 2024-09-21T04:35:10.316 INFO:tasks.workunit.client.1.smithi053.stdout:2/570: chown d1/d18/d1f/d27/d41/d4b/d78 1419587 1 2024-09-21T04:35:10.323 INFO:tasks.workunit.client.1.smithi053.stdout:9/511: symlink d0/d1c/d21/d46/lac 0 2024-09-21T04:35:10.324 INFO:tasks.workunit.client.1.smithi053.stdout:4/602: rename d1/dd/c3b to d1/d27/d53/d88/d91/d20/d4e/d51/da4/cc6 0 2024-09-21T04:35:10.326 INFO:tasks.workunit.client.1.smithi053.stdout:9/512: read d0/d2/d9/d34/f56 [3394988,21990] 0 2024-09-21T04:35:10.327 INFO:tasks.workunit.client.1.smithi053.stdout:3/519: mknod d16/d17/d21/d26/d2b/dbb/cc2 0 2024-09-21T04:35:10.329 INFO:tasks.workunit.client.1.smithi053.stdout:3/520: write d16/d17/d21/d8b/d7d/d57/dc1/d53/f9a [1595673,110818] 0 2024-09-21T04:35:10.330 INFO:tasks.workunit.client.1.smithi053.stdout:5/584: truncate d0/dd/d48/d8c/dad/db9/fba 414109 0 2024-09-21T04:35:10.335 INFO:tasks.workunit.client.1.smithi053.stdout:7/446: dread d0/d2b/f41 [0,4194304] 0 2024-09-21T04:35:10.336 INFO:tasks.workunit.client.1.smithi053.stdout:7/447: readlink d0/l77 0 2024-09-21T04:35:10.337 INFO:tasks.workunit.client.1.smithi053.stdout:2/571: dwrite d1/d18/f5b [0,4194304] 0 2024-09-21T04:35:10.337 INFO:tasks.workunit.client.1.smithi053.stdout:3/521: dread d16/d17/d21/d8b/d7d/f85 [0,4194304] 0 2024-09-21T04:35:10.340 INFO:tasks.workunit.client.1.smithi053.stdout:3/522: read f9 [1999732,105463] 0 2024-09-21T04:35:10.341 INFO:tasks.workunit.client.1.smithi053.stdout:5/585: dwrite d0/d8/d2a/fae [0,4194304] 0 2024-09-21T04:35:10.344 INFO:tasks.workunit.client.1.smithi053.stdout:0/498: sync 2024-09-21T04:35:10.344 INFO:tasks.workunit.client.1.smithi053.stdout:1/551: sync 2024-09-21T04:35:10.344 INFO:tasks.workunit.client.1.smithi053.stdout:6/569: sync 2024-09-21T04:35:10.345 INFO:tasks.workunit.client.1.smithi053.stdout:0/499: stat d5/c40 0 2024-09-21T04:35:10.345 INFO:tasks.workunit.client.1.smithi053.stdout:0/500: readlink l2 0 2024-09-21T04:35:10.349 INFO:tasks.workunit.client.1.smithi053.stdout:8/580: truncate d3/d39/f3f 4028374 0 2024-09-21T04:35:10.350 INFO:tasks.workunit.client.1.smithi053.stdout:8/581: dread - d3/d5/d4b/d5c/fa6 zero size 2024-09-21T04:35:10.351 INFO:tasks.workunit.client.1.smithi053.stdout:8/582: dread - d3/df/d10/fae zero size 2024-09-21T04:35:10.351 INFO:tasks.workunit.client.1.smithi053.stdout:7/448: rename d0/d2b/d36/d48/d50/d8b/f29 to d0/d2b/d36/f94 0 2024-09-21T04:35:10.352 INFO:tasks.workunit.client.1.smithi053.stdout:8/583: chown d3/df/d14/d16 69 1 2024-09-21T04:35:10.353 INFO:tasks.workunit.client.1.smithi053.stdout:8/584: chown d3/d5/d38/d64/f82 66269475 1 2024-09-21T04:35:10.354 INFO:tasks.workunit.client.1.smithi053.stdout:5/586: mknod d0/d8/db/d68/cc0 0 2024-09-21T04:35:10.354 INFO:tasks.workunit.client.1.smithi053.stdout:8/585: dread - d3/d39/f9f zero size 2024-09-21T04:35:10.355 INFO:tasks.workunit.client.1.smithi053.stdout:7/449: dread d0/d2b/d36/f3c [0,4194304] 0 2024-09-21T04:35:10.356 INFO:tasks.workunit.client.1.smithi053.stdout:4/603: getdents d1/d27/d53/d88/d91/db6 0 2024-09-21T04:35:10.357 INFO:tasks.workunit.client.1.smithi053.stdout:4/604: chown d1/d27/d33/l96 5462 1 2024-09-21T04:35:10.359 INFO:tasks.workunit.client.1.smithi053.stdout:2/572: mknod d1/d18/d1f/d27/d58/cc6 0 2024-09-21T04:35:10.361 INFO:tasks.workunit.client.1.smithi053.stdout:6/570: unlink d2/d1b/d1e/d2a/d4b/fb4 0 2024-09-21T04:35:10.371 INFO:tasks.workunit.client.1.smithi053.stdout:3/523: rename d16/d17/d21/d4e to d16/d1b/dc3 0 2024-09-21T04:35:10.373 INFO:tasks.workunit.client.1.smithi053.stdout:2/573: dread d1/d18/d1f/d27/d58/fb3 [0,4194304] 0 2024-09-21T04:35:10.374 INFO:tasks.workunit.client.1.smithi053.stdout:5/587: creat d0/dd/d30/d51/d52/fc1 x:0 0 0 2024-09-21T04:35:10.403 INFO:tasks.workunit.client.1.smithi053.stdout:4/605: creat d1/dd/de/fc7 x:0 0 0 2024-09-21T04:35:10.403 INFO:tasks.workunit.client.1.smithi053.stdout:0/501: creat d5/de/d9e/faf x:0 0 0 2024-09-21T04:35:10.403 INFO:tasks.workunit.client.1.smithi053.stdout:8/586: symlink d3/d5/d4b/lc6 0 2024-09-21T04:35:10.404 INFO:tasks.workunit.client.1.smithi053.stdout:7/450: write d0/d2b/d36/d48/d50/d8b/f83 [881075,84563] 0 2024-09-21T04:35:10.405 INFO:tasks.workunit.client.1.smithi053.stdout:3/524: dwrite d16/d17/d21/d8b/d7d/d57/dc1/d53/fa1 [0,4194304] 0 2024-09-21T04:35:10.406 INFO:tasks.workunit.client.1.smithi053.stdout:5/588: unlink d0/dd/d30/d51/d52/c73 0 2024-09-21T04:35:10.407 INFO:tasks.workunit.client.1.smithi053.stdout:2/574: creat d1/d18/d1f/d27/d41/d4b/d78/fc7 x:0 0 0 2024-09-21T04:35:10.408 INFO:tasks.workunit.client.1.smithi053.stdout:0/502: rmdir d5/d6/d21/d95 39 2024-09-21T04:35:10.411 INFO:tasks.workunit.client.1.smithi053.stdout:8/587: truncate d3/df/d49/f61 399895 0 2024-09-21T04:35:10.412 INFO:tasks.workunit.client.1.smithi053.stdout:1/552: getdents d7/da8/d74 0 2024-09-21T04:35:10.413 INFO:tasks.workunit.client.1.smithi053.stdout:8/588: stat d3/d5/d4c/d53/c6e 0 2024-09-21T04:35:10.414 INFO:tasks.workunit.client.1.smithi053.stdout:7/451: mkdir d0/d2b/d36/d48/d95 0 2024-09-21T04:35:10.415 INFO:tasks.workunit.client.1.smithi053.stdout:4/606: dwrite d1/f4 [0,4194304] 0 2024-09-21T04:35:10.416 INFO:tasks.workunit.client.1.smithi053.stdout:3/525: write d16/d17/faf [857506,86227] 0 2024-09-21T04:35:10.416 INFO:tasks.workunit.client.1.smithi053.stdout:5/589: symlink d0/dd/d5c/d92/lc2 0 2024-09-21T04:35:10.417 INFO:tasks.workunit.client.1.smithi053.stdout:7/452: write d0/f23 [3583907,10303] 0 2024-09-21T04:35:10.418 INFO:tasks.workunit.client.1.smithi053.stdout:1/553: symlink d7/d9/d43/d94/d69/lb5 0 2024-09-21T04:35:10.421 INFO:tasks.workunit.client.1.smithi053.stdout:4/607: rmdir d1 39 2024-09-21T04:35:10.422 INFO:tasks.workunit.client.1.smithi053.stdout:5/590: mkdir d0/dd/d48/dc3 0 2024-09-21T04:35:10.423 INFO:tasks.workunit.client.1.smithi053.stdout:8/589: truncate d3/d5/d38/d64/fb6 4495908 0 2024-09-21T04:35:10.424 INFO:tasks.workunit.client.1.smithi053.stdout:0/503: getdents d5/de/d42/d78/d94 0 2024-09-21T04:35:10.426 INFO:tasks.workunit.client.1.smithi053.stdout:7/453: creat d0/d2b/d36/d48/d50/f96 x:0 0 0 2024-09-21T04:35:10.426 INFO:tasks.workunit.client.1.smithi053.stdout:3/526: write d16/d17/d21/d26/d2b/faa [250952,88302] 0 2024-09-21T04:35:10.426 INFO:tasks.workunit.client.1.smithi053.stdout:2/575: dwrite d1/d18/d1f/d27/d41/d4b/d78/fc7 [0,4194304] 0 2024-09-21T04:35:10.428 INFO:tasks.workunit.client.1.smithi053.stdout:5/591: write d0/dd/d30/d35/f64 [1562164,113652] 0 2024-09-21T04:35:10.432 INFO:tasks.workunit.client.1.smithi053.stdout:0/504: symlink d5/d10/d44/lb0 0 2024-09-21T04:35:10.433 INFO:tasks.workunit.client.1.smithi053.stdout:8/590: rename d3/df/d10/fae to d3/d5/db1/fc7 0 2024-09-21T04:35:10.434 INFO:tasks.workunit.client.1.smithi053.stdout:2/576: creat d1/d18/d77/fc8 x:0 0 0 2024-09-21T04:35:10.435 INFO:tasks.workunit.client.1.smithi053.stdout:0/505: write d5/f8d [634600,128777] 0 2024-09-21T04:35:10.435 INFO:tasks.workunit.client.1.smithi053.stdout:8/591: readlink d3/d5/d4b/lc6 0 2024-09-21T04:35:10.437 INFO:tasks.workunit.client.1.smithi053.stdout:1/554: truncate d7/d9/da/f2f 1590025 0 2024-09-21T04:35:10.438 INFO:tasks.workunit.client.1.smithi053.stdout:0/506: chown d5/de/d42 4 1 2024-09-21T04:35:10.440 INFO:tasks.workunit.client.1.smithi053.stdout:9/513: sync 2024-09-21T04:35:10.441 INFO:tasks.workunit.client.1.smithi053.stdout:7/454: dread d0/d2b/f41 [0,4194304] 0 2024-09-21T04:35:10.444 INFO:tasks.workunit.client.1.smithi053.stdout:3/527: dwrite d16/d17/d21/d26/d2b/faa [0,4194304] 0 2024-09-21T04:35:10.447 INFO:tasks.workunit.client.1.smithi053.stdout:1/555: creat d7/d9/d43/d54/d7f/fb6 x:0 0 0 2024-09-21T04:35:10.456 INFO:tasks.workunit.client.1.smithi053.stdout:1/556: chown d7/d9/da/c17 30396 1 2024-09-21T04:35:10.456 INFO:tasks.workunit.client.1.smithi053.stdout:3/528: dread - d16/d17/d21/d8b/d7d/d57/dc1/f70 zero size 2024-09-21T04:35:10.460 INFO:tasks.workunit.client.1.smithi053.stdout:2/577: dwrite d1/f6d [0,4194304] 0 2024-09-21T04:35:10.464 INFO:tasks.workunit.client.1.smithi053.stdout:8/592: dread d3/f4a [0,4194304] 0 2024-09-21T04:35:10.465 INFO:tasks.workunit.client.1.smithi053.stdout:5/592: dwrite d0/dd/d30/d35/f98 [0,4194304] 0 2024-09-21T04:35:10.465 INFO:tasks.workunit.client.1.smithi053.stdout:4/608: dwrite d1/d27/d53/d88/d91/fb5 [0,4194304] 0 2024-09-21T04:35:10.465 INFO:tasks.workunit.client.1.smithi053.stdout:3/529: readlink d16/d17/d3c/l80 0 2024-09-21T04:35:10.468 INFO:tasks.workunit.client.1.smithi053.stdout:4/609: dread - d1/d27/d2e/f61 zero size 2024-09-21T04:35:10.469 INFO:tasks.workunit.client.1.smithi053.stdout:9/514: link d0/df/l47 d0/d76/lad 0 2024-09-21T04:35:10.469 INFO:tasks.workunit.client.1.smithi053.stdout:7/455: creat d0/d2b/f97 x:0 0 0 2024-09-21T04:35:10.470 INFO:tasks.workunit.client.1.smithi053.stdout:8/593: mknod d3/d5/db1/cc8 0 2024-09-21T04:35:10.470 INFO:tasks.workunit.client.1.smithi053.stdout:3/530: fsync d16/d17/f31 0 2024-09-21T04:35:10.470 INFO:tasks.workunit.client.1.smithi053.stdout:0/507: dread d5/f97 [0,4194304] 0 2024-09-21T04:35:10.471 INFO:tasks.workunit.client.1.smithi053.stdout:5/593: write d0/d8/d2a/fae [4212104,94052] 0 2024-09-21T04:35:10.473 INFO:tasks.workunit.client.1.smithi053.stdout:8/594: dread - d3/d39/f97 zero size 2024-09-21T04:35:10.474 INFO:tasks.workunit.client.1.smithi053.stdout:2/578: creat d1/d18/d1f/d27/d58/d5c/d91/fc9 x:0 0 0 2024-09-21T04:35:10.479 INFO:tasks.workunit.client.1.smithi053.stdout:0/508: mkdir d5/d10/d4c/db1 0 2024-09-21T04:35:10.482 INFO:tasks.workunit.client.1.smithi053.stdout:8/595: creat d3/d4f/fc9 x:0 0 0 2024-09-21T04:35:10.482 INFO:tasks.workunit.client.1.smithi053.stdout:1/557: link d7/d47/d79/d99/cb2 d7/d9/d43/d54/d7f/d59/cb7 0 2024-09-21T04:35:10.482 INFO:tasks.workunit.client.1.smithi053.stdout:0/509: fsync d5/de/d1c/f3d 0 2024-09-21T04:35:10.483 INFO:tasks.workunit.client.1.smithi053.stdout:2/579: creat d1/d18/d1f/da2/dac/fca x:0 0 0 2024-09-21T04:35:10.483 INFO:tasks.workunit.client.1.smithi053.stdout:7/456: dwrite d0/fb [4194304,4194304] 0 2024-09-21T04:35:10.484 INFO:tasks.workunit.client.1.smithi053.stdout:5/594: mknod d0/dd/d30/d51/cc4 0 2024-09-21T04:35:10.485 INFO:tasks.workunit.client.1.smithi053.stdout:8/596: chown d3/df/fb2 50488852 1 2024-09-21T04:35:10.485 INFO:tasks.workunit.client.1.smithi053.stdout:3/531: rename d16/d17/d21/d26/c55 to d16/d17/d21/cc4 0 2024-09-21T04:35:10.487 INFO:tasks.workunit.client.1.smithi053.stdout:0/510: mknod d5/d10/d44/cb2 0 2024-09-21T04:35:10.488 INFO:tasks.workunit.client.1.smithi053.stdout:7/457: write d0/d24/f4d [1098418,60498] 0 2024-09-21T04:35:10.488 INFO:tasks.workunit.client.1.smithi053.stdout:8/597: chown d3/d5/d1a 21303 1 2024-09-21T04:35:10.491 INFO:tasks.workunit.client.1.smithi053.stdout:0/511: symlink d5/d6/d53/lb3 0 2024-09-21T04:35:10.494 INFO:tasks.workunit.client.1.smithi053.stdout:5/595: rmdir d0/dd/d48/dc3 0 2024-09-21T04:35:10.498 INFO:tasks.workunit.client.1.smithi053.stdout:5/596: readlink d0/dd/d5c/l96 0 2024-09-21T04:35:10.499 INFO:tasks.workunit.client.1.smithi053.stdout:8/598: link d3/d28/f68 d3/df/dba/fca 0 2024-09-21T04:35:10.500 INFO:tasks.workunit.client.1.smithi053.stdout:2/580: dwrite d1/d18/d1f/d9d/f90 [4194304,4194304] 0 2024-09-21T04:35:10.501 INFO:tasks.workunit.client.1.smithi053.stdout:4/610: dread d1/d27/f37 [0,4194304] 0 2024-09-21T04:35:10.501 INFO:tasks.workunit.client.1.smithi053.stdout:0/512: mknod d5/d6/d21/cb4 0 2024-09-21T04:35:10.501 INFO:tasks.workunit.client.1.smithi053.stdout:1/558: dwrite d7/d63/d6b/f8d [0,4194304] 0 2024-09-21T04:35:10.503 INFO:tasks.workunit.client.1.smithi053.stdout:2/581: fsync d1/d18/f49 0 2024-09-21T04:35:10.506 INFO:tasks.workunit.client.1.smithi053.stdout:7/458: rename d0/d2b/d36/d48/d50/d8b/f6a to d0/d2b/d36/d48/d50/d84/f98 0 2024-09-21T04:35:10.507 INFO:tasks.workunit.client.1.smithi053.stdout:9/515: truncate d0/d2/d9/fb 2112622 0 2024-09-21T04:35:10.507 INFO:tasks.workunit.client.1.smithi053.stdout:5/597: mkdir d0/dd/d5c/d89/dc5 0 2024-09-21T04:35:10.508 INFO:tasks.workunit.client.1.smithi053.stdout:2/582: chown d1/d18/d1f/d22/d43/f59 42 1 2024-09-21T04:35:10.509 INFO:tasks.workunit.client.1.smithi053.stdout:6/571: sync 2024-09-21T04:35:10.510 INFO:tasks.workunit.client.1.smithi053.stdout:9/516: read - d0/df/f72 zero size 2024-09-21T04:35:10.511 INFO:tasks.workunit.client.1.smithi053.stdout:3/532: link d16/d17/d21/d8b/d7d/d57/dc1/c87 d16/d17/d21/d26/d2b/cc5 0 2024-09-21T04:35:10.511 INFO:tasks.workunit.client.1.smithi053.stdout:7/459: write d0/d2b/d36/d48/d50/f96 [56172,114390] 0 2024-09-21T04:35:10.514 INFO:tasks.workunit.client.1.smithi053.stdout:5/598: write d0/dd/d30/d51/d52/fa7 [3712447,119559] 0 2024-09-21T04:35:10.515 INFO:tasks.workunit.client.1.smithi053.stdout:1/559: symlink d7/d9/da/d40/lb8 0 2024-09-21T04:35:10.516 INFO:tasks.workunit.client.1.smithi053.stdout:8/599: dwrite d3/df/d14/d16/f98 [0,4194304] 0 2024-09-21T04:35:10.517 INFO:tasks.workunit.client.1.smithi053.stdout:5/599: chown d0/d8/f69 1280 1 2024-09-21T04:35:10.522 INFO:tasks.workunit.client.1.smithi053.stdout:0/513: rename d5/d10/d4c/d4d to d5/d6/db5 0 2024-09-21T04:35:10.522 INFO:tasks.workunit.client.1.smithi053.stdout:3/533: chown d16/d17/f1e 367572 1 2024-09-21T04:35:10.522 INFO:tasks.workunit.client.1.smithi053.stdout:6/572: mknod d2/d1b/d3d/d49/d5c/d8d/cb9 0 2024-09-21T04:35:10.523 INFO:tasks.workunit.client.1.smithi053.stdout:9/517: symlink d0/d2/d9/d34/d58/d92/lae 0 2024-09-21T04:35:10.523 INFO:tasks.workunit.client.1.smithi053.stdout:3/534: fsync d16/d17/f31 0 2024-09-21T04:35:10.526 INFO:tasks.workunit.client.1.smithi053.stdout:8/600: rmdir d3/d5/d38/d64/d73 39 2024-09-21T04:35:10.526 INFO:tasks.workunit.client.1.smithi053.stdout:1/560: dread d7/d9/da/d11/f20 [0,4194304] 0 2024-09-21T04:35:10.528 INFO:tasks.workunit.client.1.smithi053.stdout:8/601: chown d3/df/f42 175 1 2024-09-21T04:35:10.528 INFO:tasks.workunit.client.1.smithi053.stdout:5/600: dwrite d0/dd/d30/d51/d52/f59 [0,4194304] 0 2024-09-21T04:35:10.532 INFO:tasks.workunit.client.1.smithi053.stdout:5/601: write d0/d66/f82 [1719487,16914] 0 2024-09-21T04:35:10.534 INFO:tasks.workunit.client.1.smithi053.stdout:6/573: rmdir d2/d1b/d1e/d2a 39 2024-09-21T04:35:10.534 INFO:tasks.workunit.client.1.smithi053.stdout:4/611: dread d1/d27/d53/d88/d91/f84 [0,4194304] 0 2024-09-21T04:35:10.535 INFO:tasks.workunit.client.1.smithi053.stdout:5/602: read d0/dd/f3a [3648801,117618] 0 2024-09-21T04:35:10.536 INFO:tasks.workunit.client.1.smithi053.stdout:9/518: rename d0/d2/d9/d34/f65 to d0/df/faf 0 2024-09-21T04:35:10.537 INFO:tasks.workunit.client.1.smithi053.stdout:6/574: read d2/d1b/d3d/d49/de/d96/f7b [454269,97124] 0 2024-09-21T04:35:10.537 INFO:tasks.workunit.client.1.smithi053.stdout:3/535: mkdir d16/d17/d21/d8b/d7d/dc6 0 2024-09-21T04:35:10.538 INFO:tasks.workunit.client.1.smithi053.stdout:6/575: stat d2/d1b/d3d 0 2024-09-21T04:35:10.538 INFO:tasks.workunit.client.1.smithi053.stdout:9/519: write d0/d2/d9/d34/f56 [2845115,104134] 0 2024-09-21T04:35:10.539 INFO:tasks.workunit.client.1.smithi053.stdout:1/561: creat d7/d9/d43/d94/fb9 x:0 0 0 2024-09-21T04:35:10.542 INFO:tasks.workunit.client.1.smithi053.stdout:1/562: truncate d7/da8/d74/d91/fa2 36275 0 2024-09-21T04:35:10.542 INFO:tasks.workunit.client.1.smithi053.stdout:2/583: link d1/d18/d1f/d9d/l95 d1/d18/d77/lcb 0 2024-09-21T04:35:10.548 INFO:tasks.workunit.client.1.smithi053.stdout:5/603: creat d0/dd/d48/d8c/fc6 x:0 0 0 2024-09-21T04:35:10.549 INFO:tasks.workunit.client.1.smithi053.stdout:3/536: creat d16/d17/d21/d42/d43/d86/fc7 x:0 0 0 2024-09-21T04:35:10.550 INFO:tasks.workunit.client.1.smithi053.stdout:0/514: creat d5/de/fb6 x:0 0 0 2024-09-21T04:35:10.552 INFO:tasks.workunit.client.1.smithi053.stdout:6/576: rename d2/d1b/d3d/d49/de/d32/d80/d2e/d38/f36 to d2/fba 0 2024-09-21T04:35:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.550+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f84b019d330 con 0x7f8498074a20 2024-09-21T04:35:10.553 INFO:tasks.workunit.client.1.smithi053.stdout:5/604: dread d0/dd/d30/d35/f98 [0,4194304] 0 2024-09-21T04:35:10.553 INFO:tasks.workunit.client.1.smithi053.stdout:1/563: rmdir d7/d9/da/d40/d58 39 2024-09-21T04:35:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.552+0000 7f8497fff640 1 -- 172.21.15.7:0/694021094 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7f84b019d330 con 0x7f8498074a20 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "1/2 daemons upgraded", 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:35:10.555 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:35:10.556 INFO:tasks.workunit.client.1.smithi053.stdout:6/577: dread d2/d1b/f7d [0,4194304] 0 2024-09-21T04:35:10.556 INFO:tasks.workunit.client.1.smithi053.stdout:5/605: fdatasync d0/dd/d5c/d80/f8e 0 2024-09-21T04:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f8498074a20 msgr2=0x7f8498076ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f8498074a20 0x7f8498076ee0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84b01018e0 tx=0x7f84a000b390 comp rx=0 tx=0).stop 2024-09-21T04:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 msgr2=0x7f84b019c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 0x7f84b019c210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009230 tx=0x7f84a40046c0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 shutdown_connections 2024-09-21T04:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f8498074a20 0x7f8498076ee0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84b0101a80 0x7f84b019c210 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 --2- 172.21.15.7:0/694021094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b0100880 0x7f84b006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 >> 172.21.15.7:0/694021094 conn(0x7f84b00fc030 msgr2=0x7f84b00fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 shutdown_connections 2024-09-21T04:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:10.555+0000 7f84b54c2640 1 -- 172.21.15.7:0/694021094 wait complete. 2024-09-21T04:35:10.559 INFO:tasks.workunit.client.1.smithi053.stdout:3/537: creat d16/d17/d21/fc8 x:0 0 0 2024-09-21T04:35:10.559 INFO:tasks.workunit.client.1.smithi053.stdout:0/515: dread d5/d6/db5/f4e [0,4194304] 0 2024-09-21T04:35:10.567 INFO:tasks.workunit.client.1.smithi053.stdout:1/564: mkdir d7/d63/d6b/dba 0 2024-09-21T04:35:10.569 INFO:tasks.workunit.client.1.smithi053.stdout:5/606: mknod d0/dd/d6d/cc7 0 2024-09-21T04:35:10.572 INFO:tasks.workunit.client.1.smithi053.stdout:3/538: chown d16/d1b/dc3/f83 180 1 2024-09-21T04:35:10.573 INFO:tasks.workunit.client.1.smithi053.stdout:0/516: unlink d5/d6/f1b 0 2024-09-21T04:35:10.573 INFO:tasks.workunit.client.1.smithi053.stdout:4/612: dwrite d1/dd/de/f2b [0,4194304] 0 2024-09-21T04:35:10.575 INFO:tasks.workunit.client.1.smithi053.stdout:8/602: getdents d3/df/d14/d16 0 2024-09-21T04:35:10.576 INFO:tasks.workunit.client.1.smithi053.stdout:9/520: truncate d0/d1c/d21/d8e/f69 2642202 0 2024-09-21T04:35:10.576 INFO:tasks.workunit.client.1.smithi053.stdout:4/613: dread - d1/d27/d53/d88/d91/d20/d4e/f98 zero size 2024-09-21T04:35:10.578 INFO:tasks.workunit.client.1.smithi053.stdout:3/539: truncate d16/d17/d21/d8b/d7d/d57/dc1/f70 308399 0 2024-09-21T04:35:10.579 INFO:tasks.workunit.client.1.smithi053.stdout:3/540: fdatasync d16/d1b/f1f 0 2024-09-21T04:35:10.580 INFO:tasks.workunit.client.1.smithi053.stdout:5/607: unlink d0/dd/d5c/d80/c9b 0 2024-09-21T04:35:10.581 INFO:tasks.workunit.client.1.smithi053.stdout:0/517: dread d5/d6/d21/d96/d9f/f86 [0,4194304] 0 2024-09-21T04:35:10.582 INFO:tasks.workunit.client.1.smithi053.stdout:5/608: write d0/dd/d30/d51/d52/f56 [531377,125484] 0 2024-09-21T04:35:10.584 INFO:tasks.workunit.client.1.smithi053.stdout:5/609: write d0/dd/d6d/fa6 [625591,56955] 0 2024-09-21T04:35:10.584 INFO:tasks.workunit.client.1.smithi053.stdout:9/521: creat d0/d1c/fb0 x:0 0 0 2024-09-21T04:35:10.586 INFO:tasks.workunit.client.1.smithi053.stdout:6/578: rename d2/d1b/d1e/d2a/d78 to d2/d1d/dbb 0 2024-09-21T04:35:10.587 INFO:tasks.workunit.client.1.smithi053.stdout:6/579: write d2/d1b/d1e/fb1 [551558,64947] 0 2024-09-21T04:35:10.588 INFO:tasks.workunit.client.1.smithi053.stdout:4/614: mknod d1/d27/d53/d88/d91/d20/d4e/d51/cc8 0 2024-09-21T04:35:10.590 INFO:tasks.workunit.client.1.smithi053.stdout:3/541: mknod d16/d17/d21/d60/cc9 0 2024-09-21T04:35:10.591 INFO:tasks.workunit.client.1.smithi053.stdout:2/584: getdents d1/d18/d1f/d27 0 2024-09-21T04:35:10.593 INFO:tasks.workunit.client.1.smithi053.stdout:0/518: symlink d5/d47/lb7 0 2024-09-21T04:35:10.593 INFO:tasks.workunit.client.1.smithi053.stdout:0/519: chown d5/d10/c71 0 1 2024-09-21T04:35:10.593 INFO:tasks.workunit.client.1.smithi053.stdout:9/522: rmdir d0/d1c/d21 39 2024-09-21T04:35:10.594 INFO:tasks.workunit.client.1.smithi053.stdout:5/610: mkdir d0/d8/d2a/dc8 0 2024-09-21T04:35:10.594 INFO:tasks.workunit.client.1.smithi053.stdout:9/523: write d0/d1c/d25/f99 [163649,130898] 0 2024-09-21T04:35:10.595 INFO:tasks.workunit.client.1.smithi053.stdout:9/524: fdatasync d0/d28/f38 0 2024-09-21T04:35:10.596 INFO:tasks.workunit.client.1.smithi053.stdout:1/565: rename d7/d47/d79/d99/la9 to d7/d63/d96/dac/lbb 0 2024-09-21T04:35:10.600 INFO:tasks.workunit.client.1.smithi053.stdout:2/585: creat d1/d18/d1f/d27/d58/d5c/fcc x:0 0 0 2024-09-21T04:35:10.600 INFO:tasks.workunit.client.1.smithi053.stdout:3/542: chown d16/d17/l25 7 1 2024-09-21T04:35:10.601 INFO:tasks.workunit.client.1.smithi053.stdout:0/520: fdatasync d5/f92 0 2024-09-21T04:35:10.604 INFO:tasks.workunit.client.1.smithi053.stdout:5/611: rename d0/dd/d67/l8d to d0/dd/d48/lc9 0 2024-09-21T04:35:10.604 INFO:tasks.workunit.client.1.smithi053.stdout:0/521: read - d5/d6/f61 zero size 2024-09-21T04:35:10.606 INFO:tasks.workunit.client.1.smithi053.stdout:6/580: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/f55 [0,4194304] 0 2024-09-21T04:35:10.607 INFO:tasks.workunit.client.1.smithi053.stdout:1/566: dwrite d7/da8/d74/d91/f9a [0,4194304] 0 2024-09-21T04:35:10.608 INFO:tasks.workunit.client.1.smithi053.stdout:5/612: dread - d0/d8/d2a/f57 zero size 2024-09-21T04:35:10.608 INFO:tasks.workunit.client.1.smithi053.stdout:2/586: mkdir d1/d18/d1f/d22/d2d/dcd 0 2024-09-21T04:35:10.608 INFO:tasks.workunit.client.1.smithi053.stdout:4/615: link d1/dd/l56 d1/d27/d2e/lc9 0 2024-09-21T04:35:10.609 INFO:tasks.workunit.client.1.smithi053.stdout:0/522: chown d5/dd/d45/f6c 79 1 2024-09-21T04:35:10.610 INFO:tasks.workunit.client.1.smithi053.stdout:3/543: fsync d16/d17/d21/d42/d43/d86/fb3 0 2024-09-21T04:35:10.614 INFO:tasks.workunit.client.1.smithi053.stdout:1/567: write d7/da8/d74/d91/fa2 [232724,117060] 0 2024-09-21T04:35:10.614 INFO:tasks.workunit.client.1.smithi053.stdout:0/523: truncate d5/f39 353380 0 2024-09-21T04:35:10.614 INFO:tasks.workunit.client.1.smithi053.stdout:4/616: unlink d1/d27/d33/l96 0 2024-09-21T04:35:10.615 INFO:tasks.workunit.client.1.smithi053.stdout:3/544: rmdir d16/d17/d21/d42/da0 39 2024-09-21T04:35:10.616 INFO:tasks.workunit.client.1.smithi053.stdout:9/525: dread d0/d2/d9/fb [0,4194304] 0 2024-09-21T04:35:10.617 INFO:tasks.workunit.client.1.smithi053.stdout:3/545: chown d16/d17/d21/d8b/d7d/d57/dc1/d53/fa1 78 1 2024-09-21T04:35:10.617 INFO:tasks.workunit.client.1.smithi053.stdout:5/613: rmdir d0/dd/d48/d8c/dad/db9 39 2024-09-21T04:35:10.618 INFO:tasks.workunit.client.1.smithi053.stdout:6/581: rename d2/d1b/f6b to d2/d1b/d3d/d49/d5c/d8d/fbc 0 2024-09-21T04:35:10.618 INFO:tasks.workunit.client.1.smithi053.stdout:5/614: fsync d0/f14 0 2024-09-21T04:35:10.621 INFO:tasks.workunit.client.1.smithi053.stdout:5/615: write d0/dd/d48/f60 [173692,60902] 0 2024-09-21T04:35:10.621 INFO:tasks.workunit.client.1.smithi053.stdout:4/617: symlink d1/dd/d95/lca 0 2024-09-21T04:35:10.622 INFO:tasks.workunit.client.1.smithi053.stdout:6/582: write d2/d1b/d3d/d49/de/d32/d80/f26 [3250233,95419] 0 2024-09-21T04:35:10.622 INFO:tasks.workunit.client.1.smithi053.stdout:9/526: mkdir d0/d1c/d52/d53/db1 0 2024-09-21T04:35:10.624 INFO:tasks.workunit.client.1.smithi053.stdout:2/587: symlink d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/lce 0 2024-09-21T04:35:10.624 INFO:tasks.workunit.client.1.smithi053.stdout:5/616: readlink d0/d8/d2a/l9f 0 2024-09-21T04:35:10.627 INFO:tasks.workunit.client.1.smithi053.stdout:4/618: mknod d1/d27/d33/d41/ccb 0 2024-09-21T04:35:10.627 INFO:tasks.workunit.client.1.smithi053.stdout:1/568: write d7/d24/f41 [3838633,72763] 0 2024-09-21T04:35:10.627 INFO:tasks.workunit.client.1.smithi053.stdout:9/527: mkdir d0/d2/d9/d34/db2 0 2024-09-21T04:35:10.629 INFO:tasks.workunit.client.1.smithi053.stdout:3/546: chown d16/d17/d21/d8b/d7d/d57/dc1/d53/c9e 8976880 1 2024-09-21T04:35:10.632 INFO:tasks.workunit.client.1.smithi053.stdout:0/524: dwrite d5/d6/d21/d95/f9b [0,4194304] 0 2024-09-21T04:35:10.632 INFO:tasks.workunit.client.1.smithi053.stdout:9/528: readlink d0/d2/d9/d34/d58/d92/l96 0 2024-09-21T04:35:10.633 INFO:tasks.workunit.client.1.smithi053.stdout:5/617: chown d0/dd/d30/d35/cac 911946 1 2024-09-21T04:35:10.633 INFO:tasks.workunit.client.1.smithi053.stdout:6/583: creat d2/d1b/d3d/d49/de/d9c/fbd x:0 0 0 2024-09-21T04:35:10.635 INFO:tasks.workunit.client.1.smithi053.stdout:1/569: creat d7/d9/da/d3c/fbc x:0 0 0 2024-09-21T04:35:10.637 INFO:tasks.workunit.client.1.smithi053.stdout:4/619: symlink d1/d27/d53/d88/d91/d20/d4e/d51/lcc 0 2024-09-21T04:35:10.637 INFO:tasks.workunit.client.1.smithi053.stdout:6/584: fsync d2/d1b/d3d/d49/de/d32/d80/f44 0 2024-09-21T04:35:10.638 INFO:tasks.workunit.client.1.smithi053.stdout:0/525: creat d5/de/d42/d78/fb8 x:0 0 0 2024-09-21T04:35:10.642 INFO:tasks.workunit.client.1.smithi053.stdout:3/547: symlink d16/d17/d21/d42/da0/lca 0 2024-09-21T04:35:10.644 INFO:tasks.workunit.client.1.smithi053.stdout:1/570: mknod d7/d9/d43/d54/d7f/d59/cbd 0 2024-09-21T04:35:10.645 INFO:tasks.workunit.client.1.smithi053.stdout:2/588: dread d1/d18/d1f/d27/d41/f64 [0,4194304] 0 2024-09-21T04:35:10.646 INFO:tasks.workunit.client.1.smithi053.stdout:3/548: symlink d16/d1b/dad/lcb 0 2024-09-21T04:35:10.647 INFO:tasks.workunit.client.1.smithi053.stdout:5/618: creat d0/d8/db/db7/fca x:0 0 0 2024-09-21T04:35:10.648 INFO:tasks.workunit.client.1.smithi053.stdout:4/620: dwrite d1/d27/d33/d41/fa3 [0,4194304] 0 2024-09-21T04:35:10.649 INFO:tasks.workunit.client.1.smithi053.stdout:1/571: chown d7/d9/da/d11/f5c 1288 1 2024-09-21T04:35:10.652 INFO:tasks.workunit.client.1.smithi053.stdout:4/621: write d1/d27/d53/fc2 [453311,41395] 0 2024-09-21T04:35:10.652 INFO:tasks.workunit.client.1.smithi053.stdout:6/585: getdents d2/d1b/d3d 0 2024-09-21T04:35:10.655 INFO:tasks.workunit.client.1.smithi053.stdout:0/526: dread d5/fa [0,4194304] 0 2024-09-21T04:35:10.655 INFO:tasks.workunit.client.1.smithi053.stdout:4/622: dread - d1/dd/de/f83 zero size 2024-09-21T04:35:10.656 INFO:tasks.workunit.client.1.smithi053.stdout:9/529: dread d0/d28/f38 [0,4194304] 0 2024-09-21T04:35:10.656 INFO:tasks.workunit.client.1.smithi053.stdout:5/619: symlink d0/dd/d30/d35/lcb 0 2024-09-21T04:35:10.656 INFO:tasks.workunit.client.1.smithi053.stdout:3/549: creat d16/d17/d21/d60/fcc x:0 0 0 2024-09-21T04:35:10.658 INFO:tasks.workunit.client.1.smithi053.stdout:5/620: stat d0/d8/db/db7/fca 0 2024-09-21T04:35:10.659 INFO:tasks.workunit.client.1.smithi053.stdout:1/572: creat d7/d9/d43/d54/d7f/d36/d81/d95/fbe x:0 0 0 2024-09-21T04:35:10.659 INFO:tasks.workunit.client.1.smithi053.stdout:9/530: readlink d0/d2/d9/d34/l7d 0 2024-09-21T04:35:10.659 INFO:tasks.workunit.client.1.smithi053.stdout:2/589: dread d1/d18/d1f/d27/f55 [0,4194304] 0 2024-09-21T04:35:10.659 INFO:tasks.workunit.client.1.smithi053.stdout:6/586: mkdir d2/d1b/d1e/dbe 0 2024-09-21T04:35:10.660 INFO:tasks.workunit.client.1.smithi053.stdout:1/573: fsync d7/d9/da/f9b 0 2024-09-21T04:35:10.660 INFO:tasks.workunit.client.1.smithi053.stdout:4/623: unlink d1/d27/d33/d41/fa8 0 2024-09-21T04:35:10.662 INFO:tasks.workunit.client.1.smithi053.stdout:4/624: chown d1/dd/f23 392 1 2024-09-21T04:35:10.662 INFO:tasks.workunit.client.1.smithi053.stdout:0/527: creat d5/d10/fb9 x:0 0 0 2024-09-21T04:35:10.663 INFO:tasks.workunit.client.1.smithi053.stdout:3/550: mknod d16/d1b/dc3/ccd 0 2024-09-21T04:35:10.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:10 smithi053 ceph-mon[35554]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:10.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:10 smithi053 ceph-mon[35554]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:10.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:10 smithi053 ceph-mon[35554]: pgmap v26: 65 pgs: 65 active+clean; 815 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 108 MiB/s wr, 320 op/s 2024-09-21T04:35:10.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:10 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3893378791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:35:10.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:10 smithi007 ceph-mon[27954]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:10.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:10 smithi007 ceph-mon[27954]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:10.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:10 smithi007 ceph-mon[27954]: pgmap v26: 65 pgs: 65 active+clean; 815 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 108 MiB/s wr, 320 op/s 2024-09-21T04:35:10.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:10 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3893378791' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:35:10.747 INFO:tasks.workunit.client.1.smithi053.stdout:0/528: readlink d5/d47/lb7 0 2024-09-21T04:35:10.747 INFO:tasks.workunit.client.1.smithi053.stdout:1/574: mknod d7/d47/cbf 0 2024-09-21T04:35:10.747 INFO:tasks.workunit.client.1.smithi053.stdout:5/621: dread d0/dd/d30/d51/d52/f56 [0,4194304] 0 2024-09-21T04:35:10.747 INFO:tasks.workunit.client.1.smithi053.stdout:4/625: fsync d1/d27/d53/d88/d91/d20/d4e/f98 0 2024-09-21T04:35:10.748 INFO:tasks.workunit.client.1.smithi053.stdout:2/590: mkdir d1/d18/d1f/d22/d2d/dcd/dcf 0 2024-09-21T04:35:10.750 INFO:tasks.workunit.client.1.smithi053.stdout:3/551: write d16/d17/d21/d60/f9c [514030,124815] 0 2024-09-21T04:35:10.752 INFO:tasks.workunit.client.1.smithi053.stdout:5/622: mkdir d0/dd/d5c/dcc 0 2024-09-21T04:35:10.756 INFO:tasks.workunit.client.1.smithi053.stdout:4/626: creat d1/dd/fcd x:0 0 0 2024-09-21T04:35:10.759 INFO:tasks.workunit.client.1.smithi053.stdout:2/591: getdents d1 0 2024-09-21T04:35:10.759 INFO:tasks.workunit.client.1.smithi053.stdout:0/529: dwrite d5/f4a [0,4194304] 0 2024-09-21T04:35:10.760 INFO:tasks.workunit.client.1.smithi053.stdout:4/627: write d1/d27/d53/f78 [7677904,66420] 0 2024-09-21T04:35:10.762 INFO:tasks.workunit.client.1.smithi053.stdout:6/587: dread d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f8a [0,4194304] 0 2024-09-21T04:35:10.763 INFO:tasks.workunit.client.1.smithi053.stdout:5/623: dread d0/dd/d5c/f7a [0,4194304] 0 2024-09-21T04:35:10.772 INFO:tasks.workunit.client.1.smithi053.stdout:5/624: read - d0/d8/db/db7/fca zero size 2024-09-21T04:35:10.772 INFO:tasks.workunit.client.1.smithi053.stdout:3/552: dread d16/d17/d21/d8b/d7d/d57/dc1/d53/fa1 [0,4194304] 0 2024-09-21T04:35:10.773 INFO:tasks.workunit.client.1.smithi053.stdout:6/588: readlink d2/d1b/d3d/d49/de/d32/d80/d2e/d38/l52 0 2024-09-21T04:35:10.775 INFO:tasks.workunit.client.1.smithi053.stdout:2/592: rename d1/d18/d1f/d27/d41/d4b/d78 to d1/d18/d1f/d27/d58/d5c/d91/dd0 0 2024-09-21T04:35:10.778 INFO:tasks.workunit.client.1.smithi053.stdout:1/575: dwrite d7/d9/da/f2f [0,4194304] 0 2024-09-21T04:35:10.783 INFO:tasks.workunit.client.1.smithi053.stdout:5/625: dread d0/dd/d30/d51/fa0 [0,4194304] 0 2024-09-21T04:35:10.786 INFO:tasks.workunit.client.1.smithi053.stdout:1/576: truncate d7/d9/da/d4c/f7e 2077660 0 2024-09-21T04:35:10.786 INFO:tasks.workunit.client.1.smithi053.stdout:0/530: dwrite d5/de/f67 [0,4194304] 0 2024-09-21T04:35:10.789 INFO:tasks.workunit.client.1.smithi053.stdout:1/577: fsync d7/da8/d74/d75/faa 0 2024-09-21T04:35:10.789 INFO:tasks.workunit.client.1.smithi053.stdout:6/589: dread d2/d1b/d1e/d2a/f4d [0,4194304] 0 2024-09-21T04:35:10.790 INFO:tasks.workunit.client.1.smithi053.stdout:3/553: dwrite d16/d17/d21/d26/f92 [0,4194304] 0 2024-09-21T04:35:10.791 INFO:tasks.workunit.client.1.smithi053.stdout:7/460: sync 2024-09-21T04:35:10.793 INFO:tasks.workunit.client.1.smithi053.stdout:6/590: read d2/d1b/d1e/d2a/d4b/d77/d88/db8/fa6 [3074955,12163] 0 2024-09-21T04:35:10.795 INFO:tasks.workunit.client.1.smithi053.stdout:3/554: chown d16/d17/d21/d42/lba 57047 1 2024-09-21T04:35:10.803 INFO:tasks.workunit.client.1.smithi053.stdout:1/578: dwrite d7/d9/d43/d94/fa7 [0,4194304] 0 2024-09-21T04:35:10.804 INFO:tasks.workunit.client.1.smithi053.stdout:1/579: chown d7/d9/l3e 3 1 2024-09-21T04:35:10.808 INFO:tasks.workunit.client.1.smithi053.stdout:6/591: dwrite d2/d1d/dbb/f84 [0,4194304] 0 2024-09-21T04:35:10.824 INFO:tasks.workunit.client.1.smithi053.stdout:7/461: rename d0/d24/d46/c62 to d0/d2b/d36/d64/c99 0 2024-09-21T04:35:10.828 INFO:tasks.workunit.client.1.smithi053.stdout:4/628: truncate d1/dd/de/f2b 2271813 0 2024-09-21T04:35:10.829 INFO:tasks.workunit.client.1.smithi053.stdout:4/629: chown d1/d27/d53/d88/d91/d20/d4e/d51/lcc 8042127 1 2024-09-21T04:35:10.830 INFO:tasks.workunit.client.1.smithi053.stdout:1/580: unlink f0 0 2024-09-21T04:35:10.830 INFO:tasks.workunit.client.1.smithi053.stdout:1/581: chown d7/d9/c8f 341 1 2024-09-21T04:35:10.832 INFO:tasks.workunit.client.1.smithi053.stdout:1/582: read d7/d9/da/d40/d49/f5e [3245784,75660] 0 2024-09-21T04:35:10.835 INFO:tasks.workunit.client.1.smithi053.stdout:6/592: dread d2/d1b/d1e/d2a/d4b/f58 [0,4194304] 0 2024-09-21T04:35:10.838 INFO:tasks.workunit.client.1.smithi053.stdout:0/531: symlink d5/lba 0 2024-09-21T04:35:10.844 INFO:tasks.workunit.client.1.smithi053.stdout:0/532: dwrite d5/d6/d21/d95/f9b [0,4194304] 0 2024-09-21T04:35:10.845 INFO:tasks.workunit.client.1.smithi053.stdout:0/533: chown d5/f4a 203 1 2024-09-21T04:35:10.847 INFO:tasks.workunit.client.1.smithi053.stdout:1/583: creat d7/d63/d96/dac/fc0 x:0 0 0 2024-09-21T04:35:10.847 INFO:tasks.workunit.client.1.smithi053.stdout:0/534: read d5/de/d1c/f7e [627022,68059] 0 2024-09-21T04:35:10.848 INFO:tasks.workunit.client.1.smithi053.stdout:0/535: chown d5/de/d1c 251794995 1 2024-09-21T04:35:10.857 INFO:tasks.workunit.client.1.smithi053.stdout:1/584: mkdir d7/d47/d79/dc1 0 2024-09-21T04:35:10.858 INFO:tasks.workunit.client.1.smithi053.stdout:1/585: chown d7/d9/da/d40/l53 0 1 2024-09-21T04:35:10.858 INFO:tasks.workunit.client.1.smithi053.stdout:2/593: link d1/d18/d1f/d22/d2d/c3e d1/d18/d1f/d9d/cd1 0 2024-09-21T04:35:10.860 INFO:tasks.workunit.client.1.smithi053.stdout:2/594: truncate d1/d18/d1f/d27/fa0 949764 0 2024-09-21T04:35:10.861 INFO:tasks.workunit.client.1.smithi053.stdout:0/536: mkdir d5/d6/d21/d96/dbb 0 2024-09-21T04:35:10.866 INFO:tasks.workunit.client.1.smithi053.stdout:2/595: symlink d1/d18/d77/d9b/ld2 0 2024-09-21T04:35:10.866 INFO:tasks.workunit.client.1.smithi053.stdout:2/596: chown d1/d18/d1f/c98 302 1 2024-09-21T04:35:10.874 INFO:tasks.workunit.client.1.smithi053.stdout:1/586: link d7/c1f d7/da8/d74/d91/d9f/cc2 0 2024-09-21T04:35:10.877 INFO:tasks.workunit.client.1.smithi053.stdout:2/597: dwrite d1/d18/d1f/d27/d41/f64 [4194304,4194304] 0 2024-09-21T04:35:10.879 INFO:tasks.workunit.client.1.smithi053.stdout:1/587: mkdir d7/d9/d43/d54/d7f/d59/dc3 0 2024-09-21T04:35:10.882 INFO:tasks.workunit.client.1.smithi053.stdout:2/598: symlink d1/d18/d1f/d22/d43/ld3 0 2024-09-21T04:35:10.885 INFO:tasks.workunit.client.1.smithi053.stdout:8/603: sync 2024-09-21T04:35:10.886 INFO:tasks.workunit.client.1.smithi053.stdout:2/599: creat d1/d18/d1f/d27/d58/d5c/fd4 x:0 0 0 2024-09-21T04:35:10.898 INFO:tasks.workunit.client.1.smithi053.stdout:1/588: creat d7/da8/fc4 x:0 0 0 2024-09-21T04:35:10.903 INFO:tasks.workunit.client.1.smithi053.stdout:1/589: dwrite d7/da8/d74/d91/f9a [0,4194304] 0 2024-09-21T04:35:10.907 INFO:tasks.workunit.client.1.smithi053.stdout:1/590: truncate d7/d9/da/d3c/f65 490068 0 2024-09-21T04:35:10.913 INFO:tasks.workunit.client.1.smithi053.stdout:1/591: rmdir d7/d63/d6b/dba 0 2024-09-21T04:35:10.913 INFO:tasks.workunit.client.1.smithi053.stdout:1/592: chown d7/da8/d74/d75/faa 1618651 1 2024-09-21T04:35:10.916 INFO:tasks.workunit.client.1.smithi053.stdout:1/593: symlink d7/d9/d43/d54/d7f/d36/d81/lc5 0 2024-09-21T04:35:10.918 INFO:tasks.workunit.client.1.smithi053.stdout:1/594: write d7/d9/da/d4c/f7e [2980408,46925] 0 2024-09-21T04:35:10.919 INFO:tasks.workunit.client.1.smithi053.stdout:1/595: fdatasync d7/d9/d43/d94/f51 0 2024-09-21T04:35:10.920 INFO:tasks.workunit.client.0.smithi007.stdout:0/5: sync 2024-09-21T04:35:10.920 INFO:tasks.workunit.client.0.smithi007.stdout:0/6: dwrite - no filename 2024-09-21T04:35:10.920 INFO:tasks.workunit.client.0.smithi007.stdout:0/7: truncate - no filename 2024-09-21T04:35:10.920 INFO:tasks.workunit.client.0.smithi007.stdout:0/8: chown . 1361739313 1 2024-09-21T04:35:10.923 INFO:tasks.workunit.client.0.smithi007.stdout:0/9: creat f0 x:0 0 0 2024-09-21T04:35:10.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/10: creat f1 x:0 0 0 2024-09-21T04:35:10.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/11: stat f0 0 2024-09-21T04:35:10.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/12: fdatasync f0 0 2024-09-21T04:35:10.926 INFO:tasks.workunit.client.0.smithi007.stdout:0/13: chown f1 482857387 1 2024-09-21T04:35:10.929 INFO:tasks.workunit.client.0.smithi007.stdout:0/14: mknod c2 0 2024-09-21T04:35:10.930 INFO:tasks.workunit.client.0.smithi007.stdout:0/15: stat f1 0 2024-09-21T04:35:10.930 INFO:tasks.workunit.client.0.smithi007.stdout:0/16: symlink l3 0 2024-09-21T04:35:10.931 INFO:tasks.workunit.client.0.smithi007.stdout:0/17: chown c2 194074914 1 2024-09-21T04:35:10.931 INFO:tasks.workunit.client.0.smithi007.stdout:0/18: dread - f0 zero size 2024-09-21T04:35:10.932 INFO:tasks.workunit.client.0.smithi007.stdout:0/19: write f0 [460218,111950] 0 2024-09-21T04:35:10.934 INFO:tasks.workunit.client.1.smithi053.stdout:5/626: truncate d0/d8/db/d68/f8b 2892516 0 2024-09-21T04:35:10.935 INFO:tasks.workunit.client.0.smithi007.stdout:0/20: rename f0 to f4 0 2024-09-21T04:35:10.936 INFO:tasks.workunit.client.1.smithi053.stdout:5/627: rmdir d0/dd/d48/d8c 39 2024-09-21T04:35:10.937 INFO:tasks.workunit.client.1.smithi053.stdout:2/600: symlink d1/ld5 0 2024-09-21T04:35:10.937 INFO:tasks.workunit.client.0.smithi007.stdout:0/21: mknod c5 0 2024-09-21T04:35:10.938 INFO:tasks.workunit.client.0.smithi007.stdout:0/22: chown c5 12 1 2024-09-21T04:35:10.940 INFO:tasks.workunit.client.1.smithi053.stdout:7/462: dwrite d0/f49 [0,4194304] 0 2024-09-21T04:35:10.940 INFO:tasks.workunit.client.0.smithi007.stdout:0/23: mkdir d6 0 2024-09-21T04:35:10.940 INFO:tasks.workunit.client.1.smithi053.stdout:7/463: stat d0/d6f/f72 0 2024-09-21T04:35:10.941 INFO:tasks.workunit.client.1.smithi053.stdout:7/464: truncate d0/f49 4782903 0 2024-09-21T04:35:10.942 INFO:tasks.workunit.client.1.smithi053.stdout:2/601: creat d1/d18/d1f/d22/d43/fd6 x:0 0 0 2024-09-21T04:35:10.944 INFO:tasks.workunit.client.1.smithi053.stdout:7/465: symlink d0/d2b/d36/d64/l9a 0 2024-09-21T04:35:10.945 INFO:tasks.workunit.client.0.smithi007.stdout:0/24: creat d6/f7 x:0 0 0 2024-09-21T04:35:10.948 INFO:tasks.workunit.client.1.smithi053.stdout:7/466: fsync d0/d2b/d36/d48/d50/d8b/d70/f88 0 2024-09-21T04:35:10.952 INFO:tasks.workunit.client.1.smithi053.stdout:7/467: symlink d0/d2b/d36/d48/d95/l9b 0 2024-09-21T04:35:10.952 INFO:tasks.workunit.client.0.smithi007.stdout:0/25: dwrite f1 [0,4194304] 0 2024-09-21T04:35:10.954 INFO:tasks.workunit.client.1.smithi053.stdout:7/468: write d0/d2b/d36/f94 [88879,62360] 0 2024-09-21T04:35:10.955 INFO:tasks.workunit.client.1.smithi053.stdout:9/531: sync 2024-09-21T04:35:10.956 INFO:tasks.workunit.client.0.smithi007.stdout:0/26: mknod d6/c8 0 2024-09-21T04:35:10.959 INFO:tasks.workunit.client.0.smithi007.stdout:0/27: write f4 [1602140,877] 0 2024-09-21T04:35:10.960 INFO:tasks.workunit.client.1.smithi053.stdout:9/532: readlink d0/df/l47 0 2024-09-21T04:35:10.961 INFO:tasks.workunit.client.0.smithi007.stdout:0/28: creat d6/f9 x:0 0 0 2024-09-21T04:35:10.963 INFO:tasks.workunit.client.0.smithi007.stdout:0/29: mknod d6/ca 0 2024-09-21T04:35:10.963 INFO:tasks.workunit.client.0.smithi007.stdout:0/30: readlink l3 0 2024-09-21T04:35:10.965 INFO:tasks.workunit.client.1.smithi053.stdout:9/533: getdents d0/d2 0 2024-09-21T04:35:10.965 INFO:tasks.workunit.client.0.smithi007.stdout:0/31: mknod d6/cb 0 2024-09-21T04:35:10.966 INFO:tasks.workunit.client.0.smithi007.stdout:0/32: write d6/f7 [516558,124406] 0 2024-09-21T04:35:10.967 INFO:tasks.workunit.client.1.smithi053.stdout:9/534: symlink d0/d1c/d52/lb3 0 2024-09-21T04:35:10.969 INFO:tasks.workunit.client.1.smithi053.stdout:9/535: write d0/d1c/d25/f54 [4450970,91559] 0 2024-09-21T04:35:10.969 INFO:tasks.workunit.client.0.smithi007.stdout:0/33: rename d6/f7 to d6/fc 0 2024-09-21T04:35:10.970 INFO:tasks.workunit.client.0.smithi007.stdout:0/34: fdatasync f1 0 2024-09-21T04:35:10.972 INFO:tasks.workunit.client.1.smithi053.stdout:4/630: creat d1/d27/fce x:0 0 0 2024-09-21T04:35:10.973 INFO:tasks.workunit.client.1.smithi053.stdout:7/469: dread d0/f23 [0,4194304] 0 2024-09-21T04:35:10.973 INFO:tasks.workunit.client.1.smithi053.stdout:4/631: chown d1/d27/f55 104989 1 2024-09-21T04:35:10.974 INFO:tasks.workunit.client.0.smithi007.stdout:0/35: unlink d6/cb 0 2024-09-21T04:35:10.974 INFO:tasks.workunit.client.1.smithi053.stdout:9/536: symlink d0/d76/d79/lb4 0 2024-09-21T04:35:10.974 INFO:tasks.workunit.client.1.smithi053.stdout:8/604: write d3/df/f81 [638964,59626] 0 2024-09-21T04:35:10.975 INFO:tasks.workunit.client.0.smithi007.stdout:0/36: rmdir d6 39 2024-09-21T04:35:10.975 INFO:tasks.workunit.client.1.smithi053.stdout:8/605: read - d3/d39/f97 zero size 2024-09-21T04:35:10.975 INFO:tasks.workunit.client.0.smithi007.stdout:0/37: write f4 [2500984,120157] 0 2024-09-21T04:35:10.976 INFO:tasks.workunit.client.1.smithi053.stdout:8/606: stat d3/d28/f7c 0 2024-09-21T04:35:10.977 INFO:tasks.workunit.client.0.smithi007.stdout:2/14: sync 2024-09-21T04:35:10.977 INFO:tasks.workunit.client.0.smithi007.stdout:9/7: sync 2024-09-21T04:35:10.977 INFO:tasks.workunit.client.1.smithi053.stdout:8/607: readlink d3/df/d14/d16/l44 0 2024-09-21T04:35:10.977 INFO:tasks.workunit.client.1.smithi053.stdout:8/608: readlink d3/d5/d38/d64/l75 0 2024-09-21T04:35:10.978 INFO:tasks.workunit.client.1.smithi053.stdout:8/609: stat d3/d5/d4c/d53/d66/l69 0 2024-09-21T04:35:10.978 INFO:tasks.workunit.client.0.smithi007.stdout:9/8: write f0 [72295,93846] 0 2024-09-21T04:35:10.979 INFO:tasks.workunit.client.1.smithi053.stdout:9/537: mkdir d0/d1c/d52/db5 0 2024-09-21T04:35:10.981 INFO:tasks.workunit.client.0.smithi007.stdout:2/15: rename d0/d1 to d0/d2 0 2024-09-21T04:35:10.982 INFO:tasks.workunit.client.0.smithi007.stdout:2/16: write - no filename 2024-09-21T04:35:10.983 INFO:tasks.workunit.client.0.smithi007.stdout:9/9: symlink l2 0 2024-09-21T04:35:10.984 INFO:tasks.workunit.client.1.smithi053.stdout:9/538: write d0/d2/f55 [623546,33262] 0 2024-09-21T04:35:10.984 INFO:tasks.workunit.client.1.smithi053.stdout:0/537: rename d5/d10/d44 to d5/d10/d4c/db1/dbc 0 2024-09-21T04:35:10.986 INFO:tasks.workunit.client.1.smithi053.stdout:7/470: link d0/d2b/d36/d48/d50/d8b/f81 d0/d2b/d36/d48/d50/d84/f9c 0 2024-09-21T04:35:10.987 INFO:tasks.workunit.client.0.smithi007.stdout:2/17: creat d0/f3 x:0 0 0 2024-09-21T04:35:10.988 INFO:tasks.workunit.client.0.smithi007.stdout:2/18: chown d0/f3 1911951543 1 2024-09-21T04:35:10.988 INFO:tasks.workunit.client.0.smithi007.stdout:2/19: read - d0/f3 zero size 2024-09-21T04:35:10.990 INFO:tasks.workunit.client.0.smithi007.stdout:2/20: write d0/f3 [540277,108999] 0 2024-09-21T04:35:10.990 INFO:tasks.workunit.client.0.smithi007.stdout:9/10: rename l2 to l3 0 2024-09-21T04:35:10.990 INFO:tasks.workunit.client.0.smithi007.stdout:2/21: chown d0 675151 1 2024-09-21T04:35:10.990 INFO:tasks.workunit.client.0.smithi007.stdout:2/22: chown d0 983612 1 2024-09-21T04:35:10.992 INFO:tasks.workunit.client.1.smithi053.stdout:7/471: dwrite d0/d24/d46/f4a [0,4194304] 0 2024-09-21T04:35:10.993 INFO:tasks.workunit.client.0.smithi007.stdout:9/11: mknod c4 0 2024-09-21T04:35:10.995 INFO:tasks.workunit.client.1.smithi053.stdout:7/472: write d0/d2b/d36/f94 [1538569,20171] 0 2024-09-21T04:35:10.997 INFO:tasks.workunit.client.1.smithi053.stdout:7/473: write d0/d2b/d36/d48/d50/d8b/f1c [4826851,51269] 0 2024-09-21T04:35:10.998 INFO:tasks.workunit.client.0.smithi007.stdout:2/23: getdents d0/d2 0 2024-09-21T04:35:10.998 INFO:tasks.workunit.client.0.smithi007.stdout:2/24: readlink - no filename 2024-09-21T04:35:10.998 INFO:tasks.workunit.client.0.smithi007.stdout:9/12: dread f0 [0,4194304] 0 2024-09-21T04:35:10.998 INFO:tasks.workunit.client.1.smithi053.stdout:9/539: dwrite d0/d1c/d21/d46/f91 [0,4194304] 0 2024-09-21T04:35:11.000 INFO:tasks.workunit.client.1.smithi053.stdout:7/474: write d0/f4 [1823375,87925] 0 2024-09-21T04:35:11.001 INFO:tasks.workunit.client.1.smithi053.stdout:9/540: truncate d0/d1c/fb0 978612 0 2024-09-21T04:35:11.002 INFO:tasks.workunit.client.1.smithi053.stdout:1/596: rename d7/l6c to d7/d47/d79/lc6 0 2024-09-21T04:35:11.004 INFO:tasks.workunit.client.0.smithi007.stdout:2/25: rmdir d0 39 2024-09-21T04:35:11.006 INFO:tasks.workunit.client.0.smithi007.stdout:9/13: fsync f0 0 2024-09-21T04:35:11.008 INFO:tasks.workunit.client.1.smithi053.stdout:8/610: link d3/df/d14/c50 d3/df/dba/ccb 0 2024-09-21T04:35:11.008 INFO:tasks.workunit.client.1.smithi053.stdout:8/611: stat d3/d5/d38/f3d 0 2024-09-21T04:35:11.009 INFO:tasks.workunit.client.1.smithi053.stdout:8/612: dread - d3/d5/d4b/d5c/fa6 zero size 2024-09-21T04:35:11.012 INFO:tasks.workunit.client.0.smithi007.stdout:9/14: rename l1 to l5 0 2024-09-21T04:35:11.014 INFO:tasks.workunit.client.1.smithi053.stdout:7/475: mknod d0/d6f/c9d 0 2024-09-21T04:35:11.017 INFO:tasks.workunit.client.0.smithi007.stdout:2/26: dwrite d0/f3 [0,4194304] 0 2024-09-21T04:35:11.023 INFO:tasks.workunit.client.0.smithi007.stdout:9/15: dwrite f0 [0,4194304] 0 2024-09-21T04:35:11.024 INFO:tasks.workunit.client.0.smithi007.stdout:2/27: symlink d0/d2/l4 0 2024-09-21T04:35:11.025 INFO:tasks.workunit.client.0.smithi007.stdout:2/28: chown d0/f3 482 1 2024-09-21T04:35:11.026 INFO:tasks.workunit.client.1.smithi053.stdout:7/476: read d0/d2b/d36/d48/d50/d8b/ff [263930,34118] 0 2024-09-21T04:35:11.028 INFO:tasks.workunit.client.0.smithi007.stdout:9/16: creat f6 x:0 0 0 2024-09-21T04:35:11.029 INFO:tasks.workunit.client.0.smithi007.stdout:9/17: fdatasync f6 0 2024-09-21T04:35:11.030 INFO:tasks.workunit.client.0.smithi007.stdout:2/29: creat d0/d2/f5 x:0 0 0 2024-09-21T04:35:11.033 INFO:tasks.workunit.client.1.smithi053.stdout:7/477: dwrite d0/f44 [0,4194304] 0 2024-09-21T04:35:11.034 INFO:tasks.workunit.client.0.smithi007.stdout:2/30: creat d0/d2/f6 x:0 0 0 2024-09-21T04:35:11.035 INFO:tasks.workunit.client.0.smithi007.stdout:2/31: readlink d0/d2/l4 0 2024-09-21T04:35:11.036 INFO:tasks.workunit.client.0.smithi007.stdout:2/32: write d0/f3 [1440839,101430] 0 2024-09-21T04:35:11.037 INFO:tasks.workunit.client.0.smithi007.stdout:2/33: write d0/d2/f5 [684176,88233] 0 2024-09-21T04:35:11.039 INFO:tasks.workunit.client.0.smithi007.stdout:2/34: chown d0/d2/f6 4 1 2024-09-21T04:35:11.040 INFO:tasks.workunit.client.1.smithi053.stdout:5/628: rename d0/dd/d30/d51/d52/f6b to d0/dd/fcd 0 2024-09-21T04:35:11.041 INFO:tasks.workunit.client.1.smithi053.stdout:5/629: chown d0/d8/d2a/l34 594 1 2024-09-21T04:35:11.042 INFO:tasks.workunit.client.1.smithi053.stdout:7/478: mkdir d0/d2b/d36/d64/d9e 0 2024-09-21T04:35:11.044 INFO:tasks.workunit.client.0.smithi007.stdout:2/35: creat d0/f7 x:0 0 0 2024-09-21T04:35:11.046 INFO:tasks.workunit.client.0.smithi007.stdout:2/36: read d0/d2/f5 [527944,36019] 0 2024-09-21T04:35:11.047 INFO:tasks.workunit.client.1.smithi053.stdout:0/538: rename d5/d10/d4c/f52 to d5/d6/d21/d96/fbd 0 2024-09-21T04:35:11.049 INFO:tasks.workunit.client.1.smithi053.stdout:5/630: read d0/d8/db/fb1 [93006,126007] 0 2024-09-21T04:35:11.050 INFO:tasks.workunit.client.1.smithi053.stdout:7/479: mkdir d0/d2b/d36/d48/d5d/d9f 0 2024-09-21T04:35:11.050 INFO:tasks.workunit.client.1.smithi053.stdout:7/480: dread - d0/d2b/d36/f5e zero size 2024-09-21T04:35:11.051 INFO:tasks.workunit.client.1.smithi053.stdout:7/481: dread - d0/d2b/d36/f80 zero size 2024-09-21T04:35:11.056 INFO:tasks.workunit.client.1.smithi053.stdout:8/613: rename d3/d5/d38/d64/fb6 to d3/d5/d4c/dc3/fcc 0 2024-09-21T04:35:11.057 INFO:tasks.workunit.client.1.smithi053.stdout:7/482: dwrite d0/f15 [4194304,4194304] 0 2024-09-21T04:35:11.057 INFO:tasks.workunit.client.0.smithi007.stdout:2/37: rename d0/d2/f5 to d0/f8 0 2024-09-21T04:35:11.060 INFO:tasks.workunit.client.0.smithi007.stdout:2/38: rmdir d0 39 2024-09-21T04:35:11.066 INFO:tasks.workunit.client.1.smithi053.stdout:6/593: sync 2024-09-21T04:35:11.067 INFO:tasks.workunit.client.1.smithi053.stdout:3/555: sync 2024-09-21T04:35:11.068 INFO:tasks.workunit.client.1.smithi053.stdout:6/594: write d2/d1b/d1e/d27/d41/f64 [3086249,3292] 0 2024-09-21T04:35:11.069 INFO:tasks.workunit.client.1.smithi053.stdout:7/483: symlink d0/d2b/d36/d48/d5c/la0 0 2024-09-21T04:35:11.070 INFO:tasks.workunit.client.1.smithi053.stdout:6/595: write d2/d1b/d3d/d49/f8f [984059,81839] 0 2024-09-21T04:35:11.070 INFO:tasks.workunit.client.1.smithi053.stdout:7/484: chown d0/d2b/d36/f5e 31658 1 2024-09-21T04:35:11.074 INFO:tasks.workunit.client.1.smithi053.stdout:8/614: truncate d3/d28/f31 2928054 0 2024-09-21T04:35:11.076 INFO:tasks.workunit.client.1.smithi053.stdout:3/556: creat d16/d17/d21/d42/d43/d86/fce x:0 0 0 2024-09-21T04:35:11.080 INFO:tasks.workunit.client.0.smithi007.stdout:9/18: fsync f6 0 2024-09-21T04:35:11.081 INFO:tasks.workunit.client.1.smithi053.stdout:8/615: rmdir d3/df/d10 39 2024-09-21T04:35:11.083 INFO:tasks.workunit.client.0.smithi007.stdout:9/19: creat f7 x:0 0 0 2024-09-21T04:35:11.083 INFO:tasks.workunit.client.0.smithi007.stdout:2/39: fsync d0/d2/f6 0 2024-09-21T04:35:11.083 INFO:tasks.workunit.client.1.smithi053.stdout:9/541: dwrite d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:11.084 INFO:tasks.workunit.client.1.smithi053.stdout:3/557: fdatasync d16/d1b/f75 0 2024-09-21T04:35:11.084 INFO:tasks.workunit.client.0.smithi007.stdout:9/20: write f7 [90386,12504] 0 2024-09-21T04:35:11.084 INFO:tasks.workunit.client.0.smithi007.stdout:2/40: dread - d0/f7 zero size 2024-09-21T04:35:11.085 INFO:tasks.workunit.client.0.smithi007.stdout:9/21: write f0 [780092,14793] 0 2024-09-21T04:35:11.085 INFO:tasks.workunit.client.0.smithi007.stdout:2/41: readlink d0/d2/l4 0 2024-09-21T04:35:11.085 INFO:tasks.workunit.client.1.smithi053.stdout:6/596: dread d2/f8 [0,4194304] 0 2024-09-21T04:35:11.085 INFO:tasks.workunit.client.0.smithi007.stdout:2/42: chown d0 7 1 2024-09-21T04:35:11.091 INFO:tasks.workunit.client.0.smithi007.stdout:2/43: mknod d0/d2/c9 0 2024-09-21T04:35:11.093 INFO:tasks.workunit.client.1.smithi053.stdout:9/542: creat d0/d1c/d52/fb6 x:0 0 0 2024-09-21T04:35:11.095 INFO:tasks.workunit.client.0.smithi007.stdout:9/22: unlink l5 0 2024-09-21T04:35:11.095 INFO:tasks.workunit.client.0.smithi007.stdout:2/44: dread d0/f3 [0,4194304] 0 2024-09-21T04:35:11.095 INFO:tasks.workunit.client.0.smithi007.stdout:2/45: dread - d0/f7 zero size 2024-09-21T04:35:11.096 INFO:tasks.workunit.client.0.smithi007.stdout:2/46: write d0/f3 [4502943,20379] 0 2024-09-21T04:35:11.097 INFO:tasks.workunit.client.0.smithi007.stdout:2/47: write d0/f7 [873350,103654] 0 2024-09-21T04:35:11.098 INFO:tasks.workunit.client.1.smithi053.stdout:3/558: mknod d16/d17/d21/d42/d43/d86/ccf 0 2024-09-21T04:35:11.100 INFO:tasks.workunit.client.0.smithi007.stdout:9/23: unlink f0 0 2024-09-21T04:35:11.101 INFO:tasks.workunit.client.1.smithi053.stdout:0/539: truncate d5/d10/f2b 22056 0 2024-09-21T04:35:11.102 INFO:tasks.workunit.client.1.smithi053.stdout:1/597: mkdir d7/d63/d96/dc7 0 2024-09-21T04:35:11.104 INFO:tasks.workunit.client.1.smithi053.stdout:5/631: getdents d0/dd/d48/d8c/dad 0 2024-09-21T04:35:11.105 INFO:tasks.workunit.client.0.smithi007.stdout:2/48: rmdir d0 39 2024-09-21T04:35:11.107 INFO:tasks.workunit.client.1.smithi053.stdout:6/597: creat d2/d1b/d1e/d2a/d4b/d77/d88/db8/fbf x:0 0 0 2024-09-21T04:35:11.109 INFO:tasks.workunit.client.1.smithi053.stdout:4/632: write d1/dd/de/f2b [2829327,61391] 0 2024-09-21T04:35:11.109 INFO:tasks.workunit.client.1.smithi053.stdout:6/598: fsync d2/d1b/d3d/d49/de/d32/d80/d2e/f55 0 2024-09-21T04:35:11.109 INFO:tasks.workunit.client.0.smithi007.stdout:9/24: dwrite f7 [0,4194304] 0 2024-09-21T04:35:11.112 INFO:tasks.workunit.client.1.smithi053.stdout:8/616: truncate d3/df/f62 2424205 0 2024-09-21T04:35:11.113 INFO:tasks.workunit.client.1.smithi053.stdout:8/617: dread - d3/d5/d38/d64/f82 zero size 2024-09-21T04:35:11.113 INFO:tasks.workunit.client.1.smithi053.stdout:9/543: readlink d0/d1c/d52/d6f/l70 0 2024-09-21T04:35:11.115 INFO:tasks.workunit.client.1.smithi053.stdout:7/485: getdents d0/d6e 0 2024-09-21T04:35:11.115 INFO:tasks.workunit.client.0.smithi007.stdout:2/49: rmdir d0 39 2024-09-21T04:35:11.117 INFO:tasks.workunit.client.1.smithi053.stdout:2/602: sync 2024-09-21T04:35:11.118 INFO:tasks.workunit.client.0.smithi007.stdout:9/25: dwrite f6 [0,4194304] 0 2024-09-21T04:35:11.120 INFO:tasks.workunit.client.1.smithi053.stdout:5/632: dread d0/d8/db/d68/f97 [0,4194304] 0 2024-09-21T04:35:11.122 INFO:tasks.workunit.client.1.smithi053.stdout:1/598: write d7/d9/da/d4c/f7e [244609,108343] 0 2024-09-21T04:35:11.124 INFO:tasks.workunit.client.0.smithi007.stdout:9/26: rename c4 to c8 0 2024-09-21T04:35:11.127 INFO:tasks.workunit.client.0.smithi007.stdout:2/50: symlink d0/d2/la 0 2024-09-21T04:35:11.130 INFO:tasks.workunit.client.0.smithi007.stdout:9/27: symlink l9 0 2024-09-21T04:35:11.134 INFO:tasks.workunit.client.1.smithi053.stdout:1/599: dread d7/d9/da/f86 [0,4194304] 0 2024-09-21T04:35:11.134 INFO:tasks.workunit.client.0.smithi007.stdout:2/51: rmdir d0/d2 39 2024-09-21T04:35:11.135 INFO:tasks.workunit.client.0.smithi007.stdout:9/28: creat fa x:0 0 0 2024-09-21T04:35:11.136 INFO:tasks.workunit.client.1.smithi053.stdout:2/603: dread d1/d18/f67 [0,4194304] 0 2024-09-21T04:35:11.137 INFO:tasks.workunit.client.1.smithi053.stdout:2/604: stat d1/d18/d1f/d27/f7a 0 2024-09-21T04:35:11.138 INFO:tasks.workunit.client.1.smithi053.stdout:6/599: symlink d2/d1b/d3d/d49/de/d32/d80/lc0 0 2024-09-21T04:35:11.138 INFO:tasks.workunit.client.1.smithi053.stdout:0/540: write d5/de/d1c/f7e [197918,103641] 0 2024-09-21T04:35:11.140 INFO:tasks.workunit.client.1.smithi053.stdout:8/618: mkdir d3/d5/d4b/d5c/d86/dcd 0 2024-09-21T04:35:11.141 INFO:tasks.workunit.client.1.smithi053.stdout:8/619: stat d3/d5/d4c/f8c 0 2024-09-21T04:35:11.142 INFO:tasks.workunit.client.1.smithi053.stdout:8/620: dread d3/df/d49/f61 [0,4194304] 0 2024-09-21T04:35:11.142 INFO:tasks.workunit.client.1.smithi053.stdout:9/544: creat d0/d1a/d89/fb7 x:0 0 0 2024-09-21T04:35:11.143 INFO:tasks.workunit.client.1.smithi053.stdout:9/545: readlink d0/d1c/d52/lb3 0 2024-09-21T04:35:11.149 INFO:tasks.workunit.client.1.smithi053.stdout:4/633: symlink d1/d27/lcf 0 2024-09-21T04:35:11.149 INFO:tasks.workunit.client.0.smithi007.stdout:9/29: write f7 [4560498,103141] 0 2024-09-21T04:35:11.149 INFO:tasks.workunit.client.1.smithi053.stdout:4/634: write d1/d27/d33/d41/fa3 [1367872,92203] 0 2024-09-21T04:35:11.150 INFO:tasks.workunit.client.1.smithi053.stdout:2/605: creat d1/d18/d1f/d27/d41/d4b/d4d/d89/fd7 x:0 0 0 2024-09-21T04:35:11.154 INFO:tasks.workunit.client.0.smithi007.stdout:9/30: dread f7 [0,4194304] 0 2024-09-21T04:35:11.154 INFO:tasks.workunit.client.0.smithi007.stdout:9/31: fsync fa 0 2024-09-21T04:35:11.155 INFO:tasks.workunit.client.0.smithi007.stdout:9/32: fsync f6 0 2024-09-21T04:35:11.156 INFO:tasks.workunit.client.1.smithi053.stdout:0/541: dread d5/de/f35 [0,4194304] 0 2024-09-21T04:35:11.160 INFO:tasks.workunit.client.0.smithi007.stdout:9/33: symlink lb 0 2024-09-21T04:35:11.161 INFO:tasks.workunit.client.1.smithi053.stdout:8/621: creat d3/d5/d4c/d8e/fce x:0 0 0 2024-09-21T04:35:11.162 INFO:tasks.workunit.client.1.smithi053.stdout:8/622: fsync d3/d28/f8b 0 2024-09-21T04:35:11.163 INFO:tasks.workunit.client.1.smithi053.stdout:5/633: creat d0/dd/d5c/d89/dc5/fce x:0 0 0 2024-09-21T04:35:11.164 INFO:tasks.workunit.client.1.smithi053.stdout:1/600: symlink d7/d63/d6b/lc8 0 2024-09-21T04:35:11.167 INFO:tasks.workunit.client.1.smithi053.stdout:6/600: mkdir d2/d1b/d1e/dc1 0 2024-09-21T04:35:11.168 INFO:tasks.workunit.client.0.smithi007.stdout:9/34: dwrite f7 [0,4194304] 0 2024-09-21T04:35:11.168 INFO:tasks.workunit.client.0.smithi007.stdout:9/35: fsync fa 0 2024-09-21T04:35:11.169 INFO:tasks.workunit.client.0.smithi007.stdout:9/36: write f6 [4986010,40527] 0 2024-09-21T04:35:11.170 INFO:tasks.workunit.client.1.smithi053.stdout:4/635: dwrite d1/d27/f37 [4194304,4194304] 0 2024-09-21T04:35:11.170 INFO:tasks.workunit.client.1.smithi053.stdout:0/542: creat d5/d6/d53/fbe x:0 0 0 2024-09-21T04:35:11.171 INFO:tasks.workunit.client.1.smithi053.stdout:7/486: truncate d0/f49 414257 0 2024-09-21T04:35:11.172 INFO:tasks.workunit.client.1.smithi053.stdout:0/543: fdatasync d5/d6/d53/f87 0 2024-09-21T04:35:11.173 INFO:tasks.workunit.client.1.smithi053.stdout:4/636: write d1/dd/d95/fa2 [837865,129961] 0 2024-09-21T04:35:11.173 INFO:tasks.workunit.client.1.smithi053.stdout:7/487: chown d0/fb 0 1 2024-09-21T04:35:11.173 INFO:tasks.workunit.client.0.smithi007.stdout:9/37: stat l3 0 2024-09-21T04:35:11.174 INFO:tasks.workunit.client.1.smithi053.stdout:0/544: write d5/d10/fb9 [734662,99153] 0 2024-09-21T04:35:11.175 INFO:tasks.workunit.client.1.smithi053.stdout:3/559: getdents d16/d17/d3c 0 2024-09-21T04:35:11.178 INFO:tasks.workunit.client.1.smithi053.stdout:0/545: chown d5/d10/d4c/db1/dbc/fa7 3833 1 2024-09-21T04:35:11.179 INFO:tasks.workunit.client.0.smithi007.stdout:9/38: symlink lc 0 2024-09-21T04:35:11.180 INFO:tasks.workunit.client.0.smithi007.stdout:9/39: write f7 [3959944,16141] 0 2024-09-21T04:35:11.181 INFO:tasks.workunit.client.0.smithi007.stdout:9/40: truncate f7 4796110 0 2024-09-21T04:35:11.182 INFO:tasks.workunit.client.1.smithi053.stdout:5/634: rename d0/d8/db/f19 to d0/dd/d48/d8c/dad/db9/fcf 0 2024-09-21T04:35:11.184 INFO:tasks.workunit.client.1.smithi053.stdout:3/560: dwrite f11 [0,4194304] 0 2024-09-21T04:35:11.185 INFO:tasks.workunit.client.1.smithi053.stdout:2/606: mknod d1/d18/d1f/d27/d58/db7/cd8 0 2024-09-21T04:35:11.185 INFO:tasks.workunit.client.1.smithi053.stdout:5/635: write d0/dd/d48/f60 [370066,111517] 0 2024-09-21T04:35:11.186 INFO:tasks.workunit.client.0.smithi007.stdout:9/41: mknod cd 0 2024-09-21T04:35:11.187 INFO:tasks.workunit.client.1.smithi053.stdout:6/601: mknod d2/d1b/d3d/d49/de/d32/d80/cc2 0 2024-09-21T04:35:11.191 INFO:tasks.workunit.client.1.smithi053.stdout:9/546: write d0/f4b [2658295,47769] 0 2024-09-21T04:35:11.192 INFO:tasks.workunit.client.0.smithi007.stdout:9/42: creat fe x:0 0 0 2024-09-21T04:35:11.194 INFO:tasks.workunit.client.1.smithi053.stdout:4/637: symlink d1/d27/d53/d88/ld0 0 2024-09-21T04:35:11.195 INFO:tasks.workunit.client.1.smithi053.stdout:5/636: dwrite d0/dd/d6d/fa6 [0,4194304] 0 2024-09-21T04:35:11.195 INFO:tasks.workunit.client.0.smithi007.stdout:9/43: rename cd to cf 0 2024-09-21T04:35:11.196 INFO:tasks.workunit.client.1.smithi053.stdout:5/637: stat d0/dd/d5c/l96 0 2024-09-21T04:35:11.198 INFO:tasks.workunit.client.1.smithi053.stdout:8/623: creat d3/df/d10/fcf x:0 0 0 2024-09-21T04:35:11.198 INFO:tasks.workunit.client.1.smithi053.stdout:8/624: chown d3/d5/d38 258371512 1 2024-09-21T04:35:11.199 INFO:tasks.workunit.client.1.smithi053.stdout:8/625: dread - d3/d5/d4c/d8e/fc0 zero size 2024-09-21T04:35:11.204 INFO:tasks.workunit.client.1.smithi053.stdout:7/488: rename d0/d2b/d36/f94 to d0/d2b/d36/d48/d50/d8b/fa1 0 2024-09-21T04:35:11.205 INFO:tasks.workunit.client.1.smithi053.stdout:1/601: truncate d7/d9/f1b 792625 0 2024-09-21T04:35:11.208 INFO:tasks.workunit.client.1.smithi053.stdout:8/626: dwrite d3/df/f42 [4194304,4194304] 0 2024-09-21T04:35:11.208 INFO:tasks.workunit.client.1.smithi053.stdout:0/546: dwrite d5/de/d42/d78/f6f [4194304,4194304] 0 2024-09-21T04:35:11.209 INFO:tasks.workunit.client.1.smithi053.stdout:2/607: creat d1/d18/d1f/d27/d58/d5c/d91/dc1/fd9 x:0 0 0 2024-09-21T04:35:11.214 INFO:tasks.workunit.client.1.smithi053.stdout:4/638: unlink d1/d27/lcf 0 2024-09-21T04:35:11.214 INFO:tasks.workunit.client.1.smithi053.stdout:3/561: dwrite d16/d1b/dc3/f6c [0,4194304] 0 2024-09-21T04:35:11.216 INFO:tasks.workunit.client.1.smithi053.stdout:3/562: write d16/d17/d21/fc8 [6484,73152] 0 2024-09-21T04:35:11.216 INFO:tasks.workunit.client.1.smithi053.stdout:5/638: rmdir d0/d8/db 39 2024-09-21T04:35:11.218 INFO:tasks.workunit.client.1.smithi053.stdout:5/639: write d0/dd/d5c/d80/f8a [520882,73993] 0 2024-09-21T04:35:11.223 INFO:tasks.workunit.client.1.smithi053.stdout:7/489: unlink d0/c47 0 2024-09-21T04:35:11.224 INFO:tasks.workunit.client.1.smithi053.stdout:4/639: dwrite d1/d27/d53/d88/d91/d20/d5b/d87/f93 [0,4194304] 0 2024-09-21T04:35:11.229 INFO:tasks.workunit.client.1.smithi053.stdout:0/547: mknod d5/dd/d45/cbf 0 2024-09-21T04:35:11.230 INFO:tasks.workunit.client.1.smithi053.stdout:9/547: mkdir d0/d1c/db8 0 2024-09-21T04:35:11.234 INFO:tasks.workunit.client.1.smithi053.stdout:3/563: symlink d16/d17/d21/d8b/d7d/d57/dc1/d53/ld0 0 2024-09-21T04:35:11.235 INFO:tasks.workunit.client.0.smithi007.stdout:9/44: truncate f7 4243724 0 2024-09-21T04:35:11.235 INFO:tasks.workunit.client.0.smithi007.stdout:9/45: fsync fa 0 2024-09-21T04:35:11.235 INFO:tasks.workunit.client.1.smithi053.stdout:3/564: write d16/d1b/dc3/f6c [3376555,62440] 0 2024-09-21T04:35:11.241 INFO:tasks.workunit.client.0.smithi007.stdout:9/46: unlink lb 0 2024-09-21T04:35:11.241 INFO:tasks.workunit.client.0.smithi007.stdout:9/47: chown fa 0 1 2024-09-21T04:35:11.241 INFO:tasks.workunit.client.1.smithi053.stdout:3/565: dwrite d16/d17/d21/d26/f92 [0,4194304] 0 2024-09-21T04:35:11.244 INFO:tasks.workunit.client.1.smithi053.stdout:1/602: unlink d7/d47/d79/d99/db0/fb4 0 2024-09-21T04:35:11.247 INFO:tasks.workunit.client.0.smithi007.stdout:9/48: creat f10 x:0 0 0 2024-09-21T04:35:11.247 INFO:tasks.workunit.client.0.smithi007.stdout:9/49: readlink l3 0 2024-09-21T04:35:11.250 INFO:tasks.workunit.client.1.smithi053.stdout:8/627: mkdir d3/d5/d4c/dd0 0 2024-09-21T04:35:11.250 INFO:tasks.workunit.client.1.smithi053.stdout:0/548: symlink d5/d6/d21/d95/lc0 0 2024-09-21T04:35:11.251 INFO:tasks.workunit.client.0.smithi007.stdout:9/50: creat f11 x:0 0 0 2024-09-21T04:35:11.253 INFO:tasks.workunit.client.1.smithi053.stdout:2/608: mknod d1/d18/d1f/d22/cda 0 2024-09-21T04:35:11.256 INFO:tasks.workunit.client.1.smithi053.stdout:6/602: rename d2/d1d/l6d to d2/d1b/d3d/d49/lc3 0 2024-09-21T04:35:11.258 INFO:tasks.workunit.client.0.smithi007.stdout:9/51: rename c8 to c12 0 2024-09-21T04:35:11.261 INFO:tasks.workunit.client.0.smithi007.stdout:9/52: mkdir d13 0 2024-09-21T04:35:11.262 INFO:tasks.workunit.client.1.smithi053.stdout:7/490: mkdir d0/d2b/d36/d48/da2 0 2024-09-21T04:35:11.264 INFO:tasks.workunit.client.1.smithi053.stdout:4/640: mknod d1/d27/d53/cd1 0 2024-09-21T04:35:11.267 INFO:tasks.workunit.client.1.smithi053.stdout:0/549: symlink d5/d6/db5/lc1 0 2024-09-21T04:35:11.267 INFO:tasks.workunit.client.1.smithi053.stdout:9/548: mknod d0/d1a/d89/da1/cb9 0 2024-09-21T04:35:11.268 INFO:tasks.workunit.client.1.smithi053.stdout:2/609: mkdir d1/d18/d1f/d27/d41/d4b/d4d/d89/ddb 0 2024-09-21T04:35:11.268 INFO:tasks.workunit.client.1.smithi053.stdout:0/550: chown d5/de/d1c/f3d 1313733 1 2024-09-21T04:35:11.269 INFO:tasks.workunit.client.1.smithi053.stdout:3/566: rename d16/d1b/dad/cbd to d16/d17/d21/d26/d3e/cd1 0 2024-09-21T04:35:11.269 INFO:tasks.workunit.client.1.smithi053.stdout:5/640: link d0/dd/d48/l72 d0/d8/db6/dbc/ld0 0 2024-09-21T04:35:11.270 INFO:tasks.workunit.client.1.smithi053.stdout:4/641: symlink d1/d2c/ld2 0 2024-09-21T04:35:11.271 INFO:tasks.workunit.client.1.smithi053.stdout:0/551: fsync d5/f4a 0 2024-09-21T04:35:11.271 INFO:tasks.workunit.client.1.smithi053.stdout:2/610: readlink d1/d18/d1f/d27/l4e 0 2024-09-21T04:35:11.271 INFO:tasks.workunit.client.1.smithi053.stdout:1/603: dwrite d7/d9/d43/d54/d7f/d59/f9d [0,4194304] 0 2024-09-21T04:35:11.273 INFO:tasks.workunit.client.1.smithi053.stdout:9/549: creat d0/d1c/d21/d8e/fba x:0 0 0 2024-09-21T04:35:11.273 INFO:tasks.workunit.client.1.smithi053.stdout:7/491: fsync d0/d2b/f32 0 2024-09-21T04:35:11.273 INFO:tasks.workunit.client.1.smithi053.stdout:8/628: chown d3/d5/d38/d64/d73/d91/c95 14762989 1 2024-09-21T04:35:11.274 INFO:tasks.workunit.client.1.smithi053.stdout:6/603: creat d2/d1b/d1e/fc4 x:0 0 0 2024-09-21T04:35:11.277 INFO:tasks.workunit.client.1.smithi053.stdout:1/604: write d7/da8/f83 [689425,83061] 0 2024-09-21T04:35:11.279 INFO:tasks.workunit.client.1.smithi053.stdout:3/567: symlink d16/d17/d21/d26/ld2 0 2024-09-21T04:35:11.280 INFO:tasks.workunit.client.1.smithi053.stdout:9/550: creat d0/d1a/d89/fbb x:0 0 0 2024-09-21T04:35:11.281 INFO:tasks.workunit.client.1.smithi053.stdout:2/611: mkdir d1/d18/d1f/d27/d41/d4b/ddc 0 2024-09-21T04:35:11.283 INFO:tasks.workunit.client.1.smithi053.stdout:4/642: creat d1/d27/d53/d88/d91/d3f/d74/fd3 x:0 0 0 2024-09-21T04:35:11.284 INFO:tasks.workunit.client.1.smithi053.stdout:5/641: truncate d0/d8/db/f29 6687239 0 2024-09-21T04:35:11.290 INFO:tasks.workunit.client.1.smithi053.stdout:2/612: creat d1/fdd x:0 0 0 2024-09-21T04:35:11.290 INFO:tasks.workunit.client.1.smithi053.stdout:7/492: dwrite d0/d2b/d36/d48/f6b [4194304,4194304] 0 2024-09-21T04:35:11.291 INFO:tasks.workunit.client.1.smithi053.stdout:8/629: dwrite d3/d5/d38/d64/d73/d91/fa4 [0,4194304] 0 2024-09-21T04:35:11.291 INFO:tasks.workunit.client.1.smithi053.stdout:6/604: mknod d2/d1b/d3d/d49/de/d32/d80/d2e/cc5 0 2024-09-21T04:35:11.293 INFO:tasks.workunit.client.1.smithi053.stdout:8/630: stat d3/d5/f13 0 2024-09-21T04:35:11.293 INFO:tasks.workunit.client.1.smithi053.stdout:2/613: write d1/d18/d1f/f8a [485489,121833] 0 2024-09-21T04:35:11.296 INFO:tasks.workunit.client.1.smithi053.stdout:4/643: unlink d1/d27/d53/d88/f8a 0 2024-09-21T04:35:11.296 INFO:tasks.workunit.client.1.smithi053.stdout:4/644: readlink d1/d2c/ld2 0 2024-09-21T04:35:11.297 INFO:tasks.workunit.client.1.smithi053.stdout:4/645: stat d1/dd/d95/lca 0 2024-09-21T04:35:11.299 INFO:tasks.workunit.client.1.smithi053.stdout:5/642: symlink d0/dd/d67/ld1 0 2024-09-21T04:35:11.301 INFO:tasks.workunit.client.1.smithi053.stdout:1/605: write d7/d63/f6a [3572323,18664] 0 2024-09-21T04:35:11.301 INFO:tasks.workunit.client.1.smithi053.stdout:5/643: write d0/d8/d2a/f57 [957989,52131] 0 2024-09-21T04:35:11.305 INFO:tasks.workunit.client.1.smithi053.stdout:6/605: symlink d2/d1d/dbb/lc6 0 2024-09-21T04:35:11.305 INFO:tasks.workunit.client.1.smithi053.stdout:6/606: fsync d2/fa 0 2024-09-21T04:35:11.306 INFO:tasks.workunit.client.1.smithi053.stdout:7/493: symlink d0/d2b/d36/d48/d50/d8b/la3 0 2024-09-21T04:35:11.308 INFO:tasks.workunit.client.1.smithi053.stdout:8/631: fdatasync d3/d5/d38/f3d 0 2024-09-21T04:35:11.309 INFO:tasks.workunit.client.1.smithi053.stdout:9/551: link d0/d1a/d89/fb7 d0/d1c/d52/daa/fbc 0 2024-09-21T04:35:11.309 INFO:tasks.workunit.client.1.smithi053.stdout:0/552: truncate d5/dd/d50/f6e 224880 0 2024-09-21T04:35:11.309 INFO:tasks.workunit.client.1.smithi053.stdout:3/568: link d16/d17/d21/d26/d2b/fb7 d16/d17/d21/d26/d3e/fd3 0 2024-09-21T04:35:11.311 INFO:tasks.workunit.client.1.smithi053.stdout:3/569: fdatasync d16/d17/d3c/f7e 0 2024-09-21T04:35:11.311 INFO:tasks.workunit.client.1.smithi053.stdout:5/644: unlink d0/d8/db/fb8 0 2024-09-21T04:35:11.312 INFO:tasks.workunit.client.1.smithi053.stdout:0/553: write d5/d10/d4c/db1/dbc/fa7 [1004260,86104] 0 2024-09-21T04:35:11.314 INFO:tasks.workunit.client.1.smithi053.stdout:6/607: mknod d2/d1d/cc7 0 2024-09-21T04:35:11.314 INFO:tasks.workunit.client.0.smithi007.stdout:4/54: sync 2024-09-21T04:35:11.314 INFO:tasks.workunit.client.0.smithi007.stdout:8/35: sync 2024-09-21T04:35:11.314 INFO:tasks.workunit.client.0.smithi007.stdout:5/87: sync 2024-09-21T04:35:11.315 INFO:tasks.workunit.client.0.smithi007.stdout:3/20: sync 2024-09-21T04:35:11.315 INFO:tasks.workunit.client.0.smithi007.stdout:6/21: sync 2024-09-21T04:35:11.315 INFO:tasks.workunit.client.0.smithi007.stdout:6/22: dread - no filename 2024-09-21T04:35:11.315 INFO:tasks.workunit.client.0.smithi007.stdout:6/23: fdatasync - no filename 2024-09-21T04:35:11.315 INFO:tasks.workunit.client.0.smithi007.stdout:1/108: sync 2024-09-21T04:35:11.315 INFO:tasks.workunit.client.0.smithi007.stdout:7/65: sync 2024-09-21T04:35:11.315 INFO:tasks.workunit.client.1.smithi053.stdout:2/614: mknod d1/d18/d1f/d27/d41/d4b/d4d/cde 0 2024-09-21T04:35:11.315 INFO:tasks.workunit.client.1.smithi053.stdout:9/552: mknod d0/d1c/d52/d6f/cbd 0 2024-09-21T04:35:11.316 INFO:tasks.workunit.client.0.smithi007.stdout:7/66: fdatasync d3/d6/fa 0 2024-09-21T04:35:11.317 INFO:tasks.workunit.client.1.smithi053.stdout:5/645: chown d0/d8/db/d68/c7c 14873507 1 2024-09-21T04:35:11.318 INFO:tasks.workunit.client.0.smithi007.stdout:3/21: symlink l5 0 2024-09-21T04:35:11.320 INFO:tasks.workunit.client.0.smithi007.stdout:3/22: readlink l5 0 2024-09-21T04:35:11.320 INFO:tasks.workunit.client.1.smithi053.stdout:8/632: dwrite d3/df/f42 [0,4194304] 0 2024-09-21T04:35:11.321 INFO:tasks.workunit.client.1.smithi053.stdout:3/570: rename d16/d17/d21/c8c to d16/d17/d21/d8b/cd4 0 2024-09-21T04:35:11.321 INFO:tasks.workunit.client.1.smithi053.stdout:5/646: read d0/f3b [3809515,4494] 0 2024-09-21T04:35:11.321 INFO:tasks.workunit.client.1.smithi053.stdout:4/646: dwrite d1/dd/de/f6d [0,4194304] 0 2024-09-21T04:35:11.321 INFO:tasks.workunit.client.0.smithi007.stdout:4/55: fdatasync d2/d7/f10 0 2024-09-21T04:35:11.323 INFO:tasks.workunit.client.1.smithi053.stdout:4/647: stat d1/d27/d33/d41/lae 0 2024-09-21T04:35:11.323 INFO:tasks.workunit.client.0.smithi007.stdout:1/109: dwrite d5/fc [0,4194304] 0 2024-09-21T04:35:11.324 INFO:tasks.workunit.client.1.smithi053.stdout:0/554: creat d5/d6/d21/d96/fc2 x:0 0 0 2024-09-21T04:35:11.324 INFO:tasks.workunit.client.1.smithi053.stdout:3/571: read d16/d17/d21/d42/d43/d86/fb3 [3859339,112534] 0 2024-09-21T04:35:11.324 INFO:tasks.workunit.client.1.smithi053.stdout:4/648: chown d1/l5 6353649 1 2024-09-21T04:35:11.325 INFO:tasks.workunit.client.0.smithi007.stdout:1/110: write f4 [1757423,70657] 0 2024-09-21T04:35:11.326 INFO:tasks.workunit.client.0.smithi007.stdout:1/111: write f4 [2381343,113755] 0 2024-09-21T04:35:11.327 INFO:tasks.workunit.client.0.smithi007.stdout:1/112: chown d5/f1f 25 1 2024-09-21T04:35:11.333 INFO:tasks.workunit.client.0.smithi007.stdout:8/36: rename d0/l2 to d0/d3/d8/lc 0 2024-09-21T04:35:11.339 INFO:tasks.workunit.client.0.smithi007.stdout:7/67: creat d3/d6/d13/f17 x:0 0 0 2024-09-21T04:35:11.339 INFO:tasks.workunit.client.0.smithi007.stdout:7/68: write d3/f15 [716560,110077] 0 2024-09-21T04:35:11.340 INFO:tasks.workunit.client.0.smithi007.stdout:7/69: read d3/f5 [967349,102435] 0 2024-09-21T04:35:11.341 INFO:tasks.workunit.client.1.smithi053.stdout:0/555: truncate d5/fab 372234 0 2024-09-21T04:35:11.341 INFO:tasks.workunit.client.0.smithi007.stdout:6/24: symlink d3/l4 0 2024-09-21T04:35:11.342 INFO:tasks.workunit.client.0.smithi007.stdout:7/70: write d3/d6/fa [937943,124824] 0 2024-09-21T04:35:11.344 INFO:tasks.workunit.client.1.smithi053.stdout:2/615: mknod d1/d18/d1f/d27/d71/cdf 0 2024-09-21T04:35:11.344 INFO:tasks.workunit.client.0.smithi007.stdout:3/23: creat f6 x:0 0 0 2024-09-21T04:35:11.345 INFO:tasks.workunit.client.1.smithi053.stdout:5/647: creat d0/dd/d5c/d80/fd2 x:0 0 0 2024-09-21T04:35:11.347 INFO:tasks.workunit.client.1.smithi053.stdout:1/606: truncate d7/d9/da/d11/f5c 2425877 0 2024-09-21T04:35:11.350 INFO:tasks.workunit.client.1.smithi053.stdout:4/649: creat d1/d27/d33/fd4 x:0 0 0 2024-09-21T04:35:11.352 INFO:tasks.workunit.client.1.smithi053.stdout:5/648: dread d0/d8/db/fb1 [0,4194304] 0 2024-09-21T04:35:11.352 INFO:tasks.workunit.client.1.smithi053.stdout:6/608: dread d2/d1b/d3d/d49/de/f85 [0,4194304] 0 2024-09-21T04:35:11.352 INFO:tasks.workunit.client.1.smithi053.stdout:0/556: readlink d5/de/d1c/l43 0 2024-09-21T04:35:11.353 INFO:tasks.workunit.client.0.smithi007.stdout:3/24: dwrite f6 [0,4194304] 0 2024-09-21T04:35:11.355 INFO:tasks.workunit.client.1.smithi053.stdout:8/633: getdents d3/d5/d4b/d5c/d86/dcd 0 2024-09-21T04:35:11.356 INFO:tasks.workunit.client.1.smithi053.stdout:9/553: dread d0/d2/f13 [4194304,4194304] 0 2024-09-21T04:35:11.356 INFO:tasks.workunit.client.1.smithi053.stdout:3/572: creat d16/d17/d21/d26/d3e/fd5 x:0 0 0 2024-09-21T04:35:11.357 INFO:tasks.workunit.client.0.smithi007.stdout:8/37: truncate d0/d3/d8/fa 3852536 0 2024-09-21T04:35:11.357 INFO:tasks.workunit.client.0.smithi007.stdout:8/38: chown d0/c7 106 1 2024-09-21T04:35:11.358 INFO:tasks.workunit.client.1.smithi053.stdout:8/634: truncate d3/d5/d4b/d5c/d86/fc5 209322 0 2024-09-21T04:35:11.358 INFO:tasks.workunit.client.0.smithi007.stdout:5/88: creat d2/f18 x:0 0 0 2024-09-21T04:35:11.359 INFO:tasks.workunit.client.1.smithi053.stdout:8/635: chown d3/d5/d21/cac 16671 1 2024-09-21T04:35:11.360 INFO:tasks.workunit.client.0.smithi007.stdout:6/25: creat d3/f5 x:0 0 0 2024-09-21T04:35:11.369 INFO:tasks.workunit.client.0.smithi007.stdout:3/25: mknod c7 0 2024-09-21T04:35:11.372 INFO:tasks.workunit.client.0.smithi007.stdout:1/113: creat d5/d6/d21/d23/f24 x:0 0 0 2024-09-21T04:35:11.373 INFO:tasks.workunit.client.0.smithi007.stdout:4/56: dwrite d2/d7/f8 [0,4194304] 0 2024-09-21T04:35:11.376 INFO:tasks.workunit.client.0.smithi007.stdout:5/89: fdatasync d2/dd/f14 0 2024-09-21T04:35:11.381 INFO:tasks.workunit.client.0.smithi007.stdout:7/71: rename d3/d6/d13/f17 to d3/d6/d14/f18 0 2024-09-21T04:35:11.382 INFO:tasks.workunit.client.0.smithi007.stdout:3/26: creat f8 x:0 0 0 2024-09-21T04:35:11.385 INFO:tasks.workunit.client.0.smithi007.stdout:1/114: write f1 [3500991,86368] 0 2024-09-21T04:35:11.387 INFO:tasks.workunit.client.0.smithi007.stdout:1/115: read d5/fc [3741743,714] 0 2024-09-21T04:35:11.387 INFO:tasks.workunit.client.0.smithi007.stdout:3/27: dread f6 [0,4194304] 0 2024-09-21T04:35:11.390 INFO:tasks.workunit.client.0.smithi007.stdout:6/26: link d3/f5 d3/f6 0 2024-09-21T04:35:11.391 INFO:tasks.workunit.client.0.smithi007.stdout:7/72: symlink d3/d6/d14/l19 0 2024-09-21T04:35:11.392 INFO:tasks.workunit.client.0.smithi007.stdout:7/73: write d3/d6/f8 [3582929,49532] 0 2024-09-21T04:35:11.394 INFO:tasks.workunit.client.0.smithi007.stdout:7/74: truncate d3/d6/d14/f18 468401 0 2024-09-21T04:35:11.397 INFO:tasks.workunit.client.0.smithi007.stdout:3/28: unlink c4 0 2024-09-21T04:35:11.399 INFO:tasks.workunit.client.0.smithi007.stdout:1/116: creat d5/f25 x:0 0 0 2024-09-21T04:35:11.401 INFO:tasks.workunit.client.0.smithi007.stdout:1/117: dread - d5/d6/d21/d23/f24 zero size 2024-09-21T04:35:11.403 INFO:tasks.workunit.client.0.smithi007.stdout:7/75: mkdir d3/d1a 0 2024-09-21T04:35:11.404 INFO:tasks.workunit.client.0.smithi007.stdout:6/27: write d3/f6 [445601,64240] 0 2024-09-21T04:35:11.406 INFO:tasks.workunit.client.0.smithi007.stdout:3/29: dwrite f8 [0,4194304] 0 2024-09-21T04:35:11.407 INFO:tasks.workunit.client.0.smithi007.stdout:5/90: creat d2/f19 x:0 0 0 2024-09-21T04:35:11.408 INFO:tasks.workunit.client.0.smithi007.stdout:6/28: read d3/f5 [29036,71627] 0 2024-09-21T04:35:11.410 INFO:tasks.workunit.client.0.smithi007.stdout:4/57: link d2/d7/ce d2/d13/c16 0 2024-09-21T04:35:11.411 INFO:tasks.workunit.client.0.smithi007.stdout:8/39: read d0/d3/d8/fa [2833990,84667] 0 2024-09-21T04:35:11.413 INFO:tasks.workunit.client.0.smithi007.stdout:7/76: creat d3/d6/d14/f1b x:0 0 0 2024-09-21T04:35:11.416 INFO:tasks.workunit.client.0.smithi007.stdout:7/77: dread d3/f5 [4194304,4194304] 0 2024-09-21T04:35:11.417 INFO:tasks.workunit.client.0.smithi007.stdout:6/29: dread d3/f6 [0,4194304] 0 2024-09-21T04:35:11.419 INFO:tasks.workunit.client.0.smithi007.stdout:6/30: dread d3/f5 [0,4194304] 0 2024-09-21T04:35:11.421 INFO:tasks.workunit.client.0.smithi007.stdout:5/91: rename d2/c7 to d2/dd/c1a 0 2024-09-21T04:35:11.422 INFO:tasks.workunit.client.0.smithi007.stdout:7/78: fsync d3/d6/d14/f1b 0 2024-09-21T04:35:11.422 INFO:tasks.workunit.client.0.smithi007.stdout:1/118: dwrite d5/fc [0,4194304] 0 2024-09-21T04:35:11.423 INFO:tasks.workunit.client.0.smithi007.stdout:5/92: write d2/dd/f16 [5160045,21595] 0 2024-09-21T04:35:11.424 INFO:tasks.workunit.client.0.smithi007.stdout:8/40: write d0/f6 [4232464,63295] 0 2024-09-21T04:35:11.424 INFO:tasks.workunit.client.0.smithi007.stdout:7/79: dread - d3/d6/d14/f1b zero size 2024-09-21T04:35:11.425 INFO:tasks.workunit.client.0.smithi007.stdout:8/41: chown d0/d3/d8 5384981 1 2024-09-21T04:35:11.426 INFO:tasks.workunit.client.0.smithi007.stdout:6/31: mknod d3/c7 0 2024-09-21T04:35:11.426 INFO:tasks.workunit.client.0.smithi007.stdout:7/80: readlink d3/d6/d14/l19 0 2024-09-21T04:35:11.427 INFO:tasks.workunit.client.0.smithi007.stdout:7/81: truncate d3/d6/fa 1685173 0 2024-09-21T04:35:11.427 INFO:tasks.workunit.client.0.smithi007.stdout:1/119: creat d5/f26 x:0 0 0 2024-09-21T04:35:11.429 INFO:tasks.workunit.client.0.smithi007.stdout:5/93: read d2/fc [3970467,111555] 0 2024-09-21T04:35:11.431 INFO:tasks.workunit.client.0.smithi007.stdout:1/120: write d5/f15 [3561,20218] 0 2024-09-21T04:35:11.431 INFO:tasks.workunit.client.0.smithi007.stdout:4/58: truncate d2/d7/ff 542120 0 2024-09-21T04:35:11.432 INFO:tasks.workunit.client.0.smithi007.stdout:7/82: symlink d3/d6/d10/l1c 0 2024-09-21T04:35:11.433 INFO:tasks.workunit.client.1.smithi053.stdout:2/616: mknod d1/d18/d1f/d27/d58/d5c/d91/dc1/ce0 0 2024-09-21T04:35:11.433 INFO:tasks.workunit.client.0.smithi007.stdout:5/94: write d2/dd/f14 [2584876,61965] 0 2024-09-21T04:35:11.434 INFO:tasks.workunit.client.0.smithi007.stdout:4/59: dread - d2/f12 zero size 2024-09-21T04:35:11.435 INFO:tasks.workunit.client.0.smithi007.stdout:5/95: chown d2 22711 1 2024-09-21T04:35:11.435 INFO:tasks.workunit.client.1.smithi053.stdout:4/650: mkdir d1/d27/d53/d88/d91/d20/d5b/d87/dd5 0 2024-09-21T04:35:11.436 INFO:tasks.workunit.client.1.smithi053.stdout:5/649: symlink d0/dd/d5c/d80/ld3 0 2024-09-21T04:35:11.436 INFO:tasks.workunit.client.0.smithi007.stdout:1/121: creat d5/f27 x:0 0 0 2024-09-21T04:35:11.436 INFO:tasks.workunit.client.0.smithi007.stdout:4/60: readlink d2/d7/l11 0 2024-09-21T04:35:11.437 INFO:tasks.workunit.client.0.smithi007.stdout:8/42: dwrite d0/f6 [0,4194304] 0 2024-09-21T04:35:11.439 INFO:tasks.workunit.client.0.smithi007.stdout:7/83: rmdir d3/d6/d10 39 2024-09-21T04:35:11.439 INFO:tasks.workunit.client.0.smithi007.stdout:4/61: truncate d2/f12 145265 0 2024-09-21T04:35:11.442 INFO:tasks.workunit.client.1.smithi053.stdout:3/573: dwrite d16/d17/d21/d26/f92 [0,4194304] 0 2024-09-21T04:35:11.442 INFO:tasks.workunit.client.0.smithi007.stdout:8/43: readlink d0/d3/d8/lc 0 2024-09-21T04:35:11.443 INFO:tasks.workunit.client.1.smithi053.stdout:9/554: mknod d0/d1c/cbe 0 2024-09-21T04:35:11.443 INFO:tasks.workunit.client.1.smithi053.stdout:0/557: dread d5/de/d1c/f7e [0,4194304] 0 2024-09-21T04:35:11.444 INFO:tasks.workunit.client.1.smithi053.stdout:5/650: dread d0/dd/d30/d51/d52/f56 [0,4194304] 0 2024-09-21T04:35:11.446 INFO:tasks.workunit.client.0.smithi007.stdout:4/62: mkdir d2/d17 0 2024-09-21T04:35:11.448 INFO:tasks.workunit.client.1.smithi053.stdout:8/636: write d3/df/dba/fca [159896,25772] 0 2024-09-21T04:35:11.449 INFO:tasks.workunit.client.0.smithi007.stdout:1/122: write d5/f1a [910214,36335] 0 2024-09-21T04:35:11.449 INFO:tasks.workunit.client.1.smithi053.stdout:5/651: write d0/d8/f69 [804599,124221] 0 2024-09-21T04:35:11.450 INFO:tasks.workunit.client.1.smithi053.stdout:2/617: dwrite d1/d18/d1f/d27/d41/f64 [0,4194304] 0 2024-09-21T04:35:11.455 INFO:tasks.workunit.client.1.smithi053.stdout:3/574: unlink d16/d17/d21/fc8 0 2024-09-21T04:35:11.456 INFO:tasks.workunit.client.1.smithi053.stdout:4/651: dread d1/f42 [0,4194304] 0 2024-09-21T04:35:11.458 INFO:tasks.workunit.client.1.smithi053.stdout:1/607: getdents d7/d9/d43/d54/d7f/d36/d81/d95 0 2024-09-21T04:35:11.458 INFO:tasks.workunit.client.1.smithi053.stdout:9/555: mknod d0/d2/d9/d34/d58/cbf 0 2024-09-21T04:35:11.459 INFO:tasks.workunit.client.1.smithi053.stdout:3/575: stat d16/d17/d21/d26/d49/lb9 0 2024-09-21T04:35:11.460 INFO:tasks.workunit.client.1.smithi053.stdout:0/558: dwrite d5/d6/d53/f87 [0,4194304] 0 2024-09-21T04:35:11.460 INFO:tasks.workunit.client.1.smithi053.stdout:4/652: fdatasync d1/d27/d90/fb1 0 2024-09-21T04:35:11.460 INFO:tasks.workunit.client.1.smithi053.stdout:1/608: stat d7/d9/da/d11/f33 0 2024-09-21T04:35:11.462 INFO:tasks.workunit.client.0.smithi007.stdout:4/63: rename d2/d13 to d2/d17/d18 0 2024-09-21T04:35:11.463 INFO:tasks.workunit.client.1.smithi053.stdout:5/652: symlink d0/d8/d2a/ld4 0 2024-09-21T04:35:11.463 INFO:tasks.workunit.client.1.smithi053.stdout:4/653: stat d1/l5 0 2024-09-21T04:35:11.464 INFO:tasks.workunit.client.1.smithi053.stdout:3/576: write d16/d17/d21/d26/d3e/f99 [179094,80302] 0 2024-09-21T04:35:11.464 INFO:tasks.workunit.client.1.smithi053.stdout:0/559: chown d5/dd/f34 1379254 1 2024-09-21T04:35:11.465 INFO:tasks.workunit.client.0.smithi007.stdout:5/96: truncate d2/dd/f14 3412934 0 2024-09-21T04:35:11.466 INFO:tasks.workunit.client.1.smithi053.stdout:1/609: write d7/d9/da/d11/f48 [371739,84482] 0 2024-09-21T04:35:11.468 INFO:tasks.workunit.client.0.smithi007.stdout:7/84: dwrite d3/d6/d14/f18 [0,4194304] 0 2024-09-21T04:35:11.468 INFO:tasks.workunit.client.1.smithi053.stdout:5/653: dread d0/d8/db/fb1 [0,4194304] 0 2024-09-21T04:35:11.468 INFO:tasks.workunit.client.1.smithi053.stdout:4/654: dread - d1/d27/d53/d88/d91/d3f/d74/f8b zero size 2024-09-21T04:35:11.470 INFO:tasks.workunit.client.0.smithi007.stdout:1/123: creat d5/f28 x:0 0 0 2024-09-21T04:35:11.471 INFO:tasks.workunit.client.0.smithi007.stdout:8/44: write d0/d3/d8/fb [108675,122731] 0 2024-09-21T04:35:11.472 INFO:tasks.workunit.client.1.smithi053.stdout:9/556: dwrite d0/d1c/d25/f54 [0,4194304] 0 2024-09-21T04:35:11.477 INFO:tasks.workunit.client.0.smithi007.stdout:4/64: creat d2/d7/f19 x:0 0 0 2024-09-21T04:35:11.479 INFO:tasks.workunit.client.1.smithi053.stdout:5/654: symlink d0/dd/d6d/ld5 0 2024-09-21T04:35:11.481 INFO:tasks.workunit.client.0.smithi007.stdout:1/124: mknod d5/d6/c29 0 2024-09-21T04:35:11.481 INFO:tasks.workunit.client.1.smithi053.stdout:2/618: creat d1/d18/d1f/d27/d41/d4b/d4d/d89/ddb/fe1 x:0 0 0 2024-09-21T04:35:11.482 INFO:tasks.workunit.client.1.smithi053.stdout:4/655: rename d1/dd/de/c5a to d1/d27/d53/d88/d91/d20/d5b/d87/cd6 0 2024-09-21T04:35:11.484 INFO:tasks.workunit.client.0.smithi007.stdout:8/45: creat d0/d3/d8/fd x:0 0 0 2024-09-21T04:35:11.487 INFO:tasks.workunit.client.1.smithi053.stdout:2/619: dwrite d1/d18/d1f/d27/d71/fb1 [0,4194304] 0 2024-09-21T04:35:11.487 INFO:tasks.workunit.client.1.smithi053.stdout:2/620: dread - d1/d18/f96 zero size 2024-09-21T04:35:11.489 INFO:tasks.workunit.client.0.smithi007.stdout:4/65: mknod d2/d7/c1a 0 2024-09-21T04:35:11.489 INFO:tasks.workunit.client.0.smithi007.stdout:4/66: chown d2/d7/f10 0 1 2024-09-21T04:35:11.490 INFO:tasks.workunit.client.1.smithi053.stdout:0/560: mknod d5/d6/d21/d96/dbb/cc3 0 2024-09-21T04:35:11.492 INFO:tasks.workunit.client.1.smithi053.stdout:1/610: creat d7/d47/d79/dc1/fc9 x:0 0 0 2024-09-21T04:35:11.494 INFO:tasks.workunit.client.1.smithi053.stdout:5/655: readlink d0/d8/l27 0 2024-09-21T04:35:11.495 INFO:tasks.workunit.client.0.smithi007.stdout:1/125: creat d5/d6/d21/d23/f2a x:0 0 0 2024-09-21T04:35:11.496 INFO:tasks.workunit.client.1.smithi053.stdout:4/656: fdatasync d1/d27/d53/d88/d91/d3f/f50 0 2024-09-21T04:35:11.497 INFO:tasks.workunit.client.0.smithi007.stdout:8/46: mkdir d0/d3/d8/de 0 2024-09-21T04:35:11.499 INFO:tasks.workunit.client.1.smithi053.stdout:4/657: write d1/d27/d2e/d46/f8d [4459487,105196] 0 2024-09-21T04:35:11.501 INFO:tasks.workunit.client.1.smithi053.stdout:5/656: dwrite d0/dd/d30/d51/d52/f59 [0,4194304] 0 2024-09-21T04:35:11.501 INFO:tasks.workunit.client.0.smithi007.stdout:5/97: rename d2/f18 to d2/f1b 0 2024-09-21T04:35:11.502 INFO:tasks.workunit.client.1.smithi053.stdout:8/637: link d3/fa d3/d5/db1/fd1 0 2024-09-21T04:35:11.502 INFO:tasks.workunit.client.1.smithi053.stdout:8/638: write d3/f85 [3016367,57351] 0 2024-09-21T04:35:11.503 INFO:tasks.workunit.client.1.smithi053.stdout:8/639: dread - d3/d4f/fc9 zero size 2024-09-21T04:35:11.505 INFO:tasks.workunit.client.1.smithi053.stdout:8/640: read d3/d5/f13 [2872591,98628] 0 2024-09-21T04:35:11.506 INFO:tasks.workunit.client.1.smithi053.stdout:8/641: write d3/d5/d4c/d53/f83 [11622,77453] 0 2024-09-21T04:35:11.507 INFO:tasks.workunit.client.0.smithi007.stdout:1/126: mkdir d5/d6/d21/d2b 0 2024-09-21T04:35:11.510 INFO:tasks.workunit.client.0.smithi007.stdout:7/85: truncate d3/d6/d14/f18 1162541 0 2024-09-21T04:35:11.514 INFO:tasks.workunit.client.0.smithi007.stdout:1/127: dwrite d5/fc [0,4194304] 0 2024-09-21T04:35:11.516 INFO:tasks.workunit.client.0.smithi007.stdout:1/128: write d5/f27 [522076,79736] 0 2024-09-21T04:35:11.516 INFO:tasks.workunit.client.0.smithi007.stdout:7/86: dread d3/f15 [0,4194304] 0 2024-09-21T04:35:11.519 INFO:tasks.workunit.client.1.smithi053.stdout:9/557: truncate d0/d2/d4e/d5c/f78 1429716 0 2024-09-21T04:35:11.520 INFO:tasks.workunit.client.1.smithi053.stdout:9/558: write d0/f4b [2521173,33516] 0 2024-09-21T04:35:11.524 INFO:tasks.workunit.client.0.smithi007.stdout:4/67: truncate d2/d7/f8 3549015 0 2024-09-21T04:35:11.525 INFO:tasks.workunit.client.0.smithi007.stdout:1/129: rmdir d5 39 2024-09-21T04:35:11.525 INFO:tasks.workunit.client.0.smithi007.stdout:1/130: chown f3 1559 1 2024-09-21T04:35:11.531 INFO:tasks.workunit.client.1.smithi053.stdout:8/642: mknod d3/d5/d4b/d5c/cd2 0 2024-09-21T04:35:11.532 INFO:tasks.workunit.client.1.smithi053.stdout:4/658: write d1/f2d [4933648,100033] 0 2024-09-21T04:35:11.532 INFO:tasks.workunit.client.1.smithi053.stdout:2/621: symlink d1/d18/d1f/d27/d41/d4b/ddc/le2 0 2024-09-21T04:35:11.533 INFO:tasks.workunit.client.1.smithi053.stdout:4/659: write d1/d27/d33/f3c [2331801,121122] 0 2024-09-21T04:35:11.534 INFO:tasks.workunit.client.0.smithi007.stdout:4/68: creat d2/d17/f1b x:0 0 0 2024-09-21T04:35:11.536 INFO:tasks.workunit.client.1.smithi053.stdout:4/660: dread - d1/d27/d53/d88/d91/d20/d5b/d87/fad zero size 2024-09-21T04:35:11.536 INFO:tasks.workunit.client.0.smithi007.stdout:7/87: mknod d3/d6/d10/c1d 0 2024-09-21T04:35:11.537 INFO:tasks.workunit.client.0.smithi007.stdout:7/88: chown d3/d6/d14/l19 6 1 2024-09-21T04:35:11.538 INFO:tasks.workunit.client.0.smithi007.stdout:7/89: stat d3/d6/d10/l1c 0 2024-09-21T04:35:11.538 INFO:tasks.workunit.client.1.smithi053.stdout:1/611: link d7/d9/d43/d54/d7f/d36/f4e d7/d9/d43/d54/d7f/d59/dc3/fca 0 2024-09-21T04:35:11.538 INFO:tasks.workunit.client.0.smithi007.stdout:7/90: chown d3/d6 0 1 2024-09-21T04:35:11.539 INFO:tasks.workunit.client.1.smithi053.stdout:1/612: chown d7/da8/d74/d91/f9a 19 1 2024-09-21T04:35:11.541 INFO:tasks.workunit.client.0.smithi007.stdout:4/69: unlink d2/c4 0 2024-09-21T04:35:11.542 INFO:tasks.workunit.client.0.smithi007.stdout:5/98: link d2/l11 d2/l1c 0 2024-09-21T04:35:11.544 INFO:tasks.workunit.client.0.smithi007.stdout:7/91: symlink d3/d6/d13/l1e 0 2024-09-21T04:35:11.546 INFO:tasks.workunit.client.1.smithi053.stdout:5/657: getdents d0/d66 0 2024-09-21T04:35:11.547 INFO:tasks.workunit.client.0.smithi007.stdout:5/99: mkdir d2/dd/d1d 0 2024-09-21T04:35:11.550 INFO:tasks.workunit.client.0.smithi007.stdout:7/92: dread d3/d6/f7 [0,4194304] 0 2024-09-21T04:35:11.551 INFO:tasks.workunit.client.1.smithi053.stdout:5/658: dread d0/d8/d2a/f57 [0,4194304] 0 2024-09-21T04:35:11.551 INFO:tasks.workunit.client.1.smithi053.stdout:8/643: rmdir d3/d5/d21 39 2024-09-21T04:35:11.551 INFO:tasks.workunit.client.1.smithi053.stdout:5/659: truncate d0/d8/db/f76 341114 0 2024-09-21T04:35:11.552 INFO:tasks.workunit.client.1.smithi053.stdout:2/622: mkdir d1/d18/d1f/d27/d71/de3 0 2024-09-21T04:35:11.554 INFO:tasks.workunit.client.1.smithi053.stdout:0/561: creat d5/d6/d21/fc4 x:0 0 0 2024-09-21T04:35:11.554 INFO:tasks.workunit.client.1.smithi053.stdout:7/494: sync 2024-09-21T04:35:11.556 INFO:tasks.workunit.client.1.smithi053.stdout:4/661: mkdir d1/d27/d53/d88/d91/d20/d73/d92/dd7 0 2024-09-21T04:35:11.557 INFO:tasks.workunit.client.1.smithi053.stdout:5/660: dwrite d0/d8/f1d [0,4194304] 0 2024-09-21T04:35:11.557 INFO:tasks.workunit.client.0.smithi007.stdout:7/93: creat d3/d6/d13/f1f x:0 0 0 2024-09-21T04:35:11.558 INFO:tasks.workunit.client.1.smithi053.stdout:4/662: read d1/d27/f37 [878696,127742] 0 2024-09-21T04:35:11.559 INFO:tasks.workunit.client.1.smithi053.stdout:1/613: symlink d7/da8/d74/lcb 0 2024-09-21T04:35:11.560 INFO:tasks.workunit.client.1.smithi053.stdout:9/559: mknod d0/d1a/cc0 0 2024-09-21T04:35:11.561 INFO:tasks.workunit.client.0.smithi007.stdout:5/100: mkdir d2/d1e 0 2024-09-21T04:35:11.562 INFO:tasks.workunit.client.1.smithi053.stdout:1/614: truncate d7/d47/fa4 717943 0 2024-09-21T04:35:11.563 INFO:tasks.workunit.client.0.smithi007.stdout:7/94: mkdir d3/d6/d14/d20 0 2024-09-21T04:35:11.564 INFO:tasks.workunit.client.0.smithi007.stdout:4/70: write d2/d7/ff [553584,43174] 0 2024-09-21T04:35:11.565 INFO:tasks.workunit.client.0.smithi007.stdout:4/71: rename d2/d17 to d2/d17/d18/d1c 22 2024-09-21T04:35:11.565 INFO:tasks.workunit.client.1.smithi053.stdout:8/644: rmdir d3/d5/d4c/d53/d66 39 2024-09-21T04:35:11.566 INFO:tasks.workunit.client.0.smithi007.stdout:7/95: dread d3/d6/d14/f18 [0,4194304] 0 2024-09-21T04:35:11.567 INFO:tasks.workunit.client.0.smithi007.stdout:7/96: chown d3/d6/d13/f1f 643742822 1 2024-09-21T04:35:11.567 INFO:tasks.workunit.client.0.smithi007.stdout:5/101: creat d2/dd/d1d/f1f x:0 0 0 2024-09-21T04:35:11.567 INFO:tasks.workunit.client.1.smithi053.stdout:9/560: dread d0/d1c/d21/d46/f91 [0,4194304] 0 2024-09-21T04:35:11.568 INFO:tasks.workunit.client.1.smithi053.stdout:8/645: write d3/d28/f79 [852215,24236] 0 2024-09-21T04:35:11.568 INFO:tasks.workunit.client.1.smithi053.stdout:2/623: dread d1/d18/d1f/d22/d43/f4f [0,4194304] 0 2024-09-21T04:35:11.570 INFO:tasks.workunit.client.1.smithi053.stdout:8/646: truncate d3/d5/d38/f47 1233792 0 2024-09-21T04:35:11.571 INFO:tasks.workunit.client.1.smithi053.stdout:7/495: creat d0/d2b/d36/d48/d50/fa4 x:0 0 0 2024-09-21T04:35:11.572 INFO:tasks.workunit.client.0.smithi007.stdout:4/72: link d2/d7/f19 d2/f1d 0 2024-09-21T04:35:11.575 INFO:tasks.workunit.client.0.smithi007.stdout:5/102: getdents d2/d1e 0 2024-09-21T04:35:11.577 INFO:tasks.workunit.client.0.smithi007.stdout:4/73: creat d2/d17/d18/f1e x:0 0 0 2024-09-21T04:35:11.579 INFO:tasks.workunit.client.0.smithi007.stdout:5/103: mkdir d2/dd/d1d/d20 0 2024-09-21T04:35:11.581 INFO:tasks.workunit.client.0.smithi007.stdout:4/74: write d2/d7/f19 [813187,125174] 0 2024-09-21T04:35:11.584 INFO:tasks.workunit.client.0.smithi007.stdout:4/75: creat d2/d17/d18/f1f x:0 0 0 2024-09-21T04:35:11.585 INFO:tasks.workunit.client.1.smithi053.stdout:0/562: symlink d5/d6/db5/d99/lc5 0 2024-09-21T04:35:11.585 INFO:tasks.workunit.client.0.smithi007.stdout:5/104: dread d2/dd/f14 [0,4194304] 0 2024-09-21T04:35:11.590 INFO:tasks.workunit.client.1.smithi053.stdout:8/647: creat d3/df/d10/fd3 x:0 0 0 2024-09-21T04:35:11.591 INFO:tasks.workunit.client.1.smithi053.stdout:8/648: write d3/d28/f8b [1884309,59894] 0 2024-09-21T04:35:11.592 INFO:tasks.workunit.client.1.smithi053.stdout:2/624: dwrite d1/d18/d1f/d22/d43/f4f [0,4194304] 0 2024-09-21T04:35:11.594 INFO:tasks.workunit.client.1.smithi053.stdout:5/661: symlink d0/dd/ld6 0 2024-09-21T04:35:11.595 INFO:tasks.workunit.client.0.smithi007.stdout:7/97: dwrite d3/f5 [4194304,4194304] 0 2024-09-21T04:35:11.595 INFO:tasks.workunit.client.1.smithi053.stdout:4/663: mknod d1/d27/d53/d88/d91/d20/d5b/d87/dd5/cd8 0 2024-09-21T04:35:11.597 INFO:tasks.workunit.client.0.smithi007.stdout:4/76: dwrite d2/d7/ff [0,4194304] 0 2024-09-21T04:35:11.599 INFO:tasks.workunit.client.1.smithi053.stdout:1/615: link d7/d47/d79/dc1/fc9 d7/d9/d43/d54/d7f/fcc 0 2024-09-21T04:35:11.600 INFO:tasks.workunit.client.1.smithi053.stdout:0/563: symlink d5/d6/d21/d96/dbb/lc6 0 2024-09-21T04:35:11.601 INFO:tasks.workunit.client.0.smithi007.stdout:5/105: dwrite d2/dd/f16 [0,4194304] 0 2024-09-21T04:35:11.605 INFO:tasks.workunit.client.0.smithi007.stdout:4/77: creat d2/d17/f20 x:0 0 0 2024-09-21T04:35:11.606 INFO:tasks.workunit.client.1.smithi053.stdout:7/496: link d0/d2b/d36/f80 d0/d2b/d36/d48/d50/fa5 0 2024-09-21T04:35:11.608 INFO:tasks.workunit.client.0.smithi007.stdout:7/98: symlink d3/d6/l21 0 2024-09-21T04:35:11.611 INFO:tasks.workunit.client.0.smithi007.stdout:4/78: unlink d2/f3 0 2024-09-21T04:35:11.614 INFO:tasks.workunit.client.0.smithi007.stdout:7/99: rename d3/d6/l21 to d3/l22 0 2024-09-21T04:35:11.615 INFO:tasks.workunit.client.0.smithi007.stdout:5/106: dwrite d2/dd/d1d/f1f [0,4194304] 0 2024-09-21T04:35:11.616 INFO:tasks.workunit.client.1.smithi053.stdout:1/616: mknod d7/d9/d43/d54/d7f/d36/d81/d95/ccd 0 2024-09-21T04:35:11.617 INFO:tasks.workunit.client.1.smithi053.stdout:4/664: read d1/d27/d53/d88/d91/d3f/f50 [390680,27585] 0 2024-09-21T04:35:11.618 INFO:tasks.workunit.client.0.smithi007.stdout:4/79: symlink d2/l21 0 2024-09-21T04:35:11.618 INFO:tasks.workunit.client.0.smithi007.stdout:7/100: write d3/d6/f8 [1936054,79228] 0 2024-09-21T04:35:11.618 INFO:tasks.workunit.client.1.smithi053.stdout:4/665: readlink d1/d27/d53/d88/d91/d3f/l48 0 2024-09-21T04:35:11.619 INFO:tasks.workunit.client.0.smithi007.stdout:7/101: chown d3/d6/d10 30379530 1 2024-09-21T04:35:11.620 INFO:tasks.workunit.client.0.smithi007.stdout:5/107: mknod d2/dd/c21 0 2024-09-21T04:35:11.620 INFO:tasks.workunit.client.1.smithi053.stdout:9/561: link d0/d1c/cbe d0/d2/d2b/cc1 0 2024-09-21T04:35:11.621 INFO:tasks.workunit.client.0.smithi007.stdout:7/102: dread - d3/d6/d14/f1b zero size 2024-09-21T04:35:11.624 INFO:tasks.workunit.client.1.smithi053.stdout:8/649: dread d3/df/d14/d16/f35 [0,4194304] 0 2024-09-21T04:35:11.624 INFO:tasks.workunit.client.1.smithi053.stdout:2/625: fdatasync d1/d18/d1f/d27/d41/d4b/fb4 0 2024-09-21T04:35:11.625 INFO:tasks.workunit.client.1.smithi053.stdout:8/650: fdatasync d3/f85 0 2024-09-21T04:35:11.629 INFO:tasks.workunit.client.1.smithi053.stdout:0/564: link d5/d6/d21/d96/fc2 d5/dd/d50/fc7 0 2024-09-21T04:35:11.630 INFO:tasks.workunit.client.0.smithi007.stdout:5/108: creat d2/d1e/f22 x:0 0 0 2024-09-21T04:35:11.630 INFO:tasks.workunit.client.0.smithi007.stdout:5/109: stat d2 0 2024-09-21T04:35:11.631 INFO:tasks.workunit.client.0.smithi007.stdout:5/110: dread - d2/d1e/f22 zero size 2024-09-21T04:35:11.631 INFO:tasks.workunit.client.1.smithi053.stdout:4/666: symlink d1/d27/d90/ld9 0 2024-09-21T04:35:11.632 INFO:tasks.workunit.client.1.smithi053.stdout:4/667: fdatasync d1/d27/f7c 0 2024-09-21T04:35:11.634 INFO:tasks.workunit.client.1.smithi053.stdout:9/562: unlink d0/d2/d9/d34/d58/c5d 0 2024-09-21T04:35:11.636 INFO:tasks.workunit.client.0.smithi007.stdout:5/111: write d2/f8 [1600620,130673] 0 2024-09-21T04:35:11.637 INFO:tasks.workunit.client.0.smithi007.stdout:7/103: link d3/d6/c12 d3/d6/d13/c23 0 2024-09-21T04:35:11.641 INFO:tasks.workunit.client.0.smithi007.stdout:5/112: mkdir d2/dd/d23 0 2024-09-21T04:35:11.642 INFO:tasks.workunit.client.0.smithi007.stdout:7/104: dread d3/d6/f8 [0,4194304] 0 2024-09-21T04:35:11.643 INFO:tasks.workunit.client.1.smithi053.stdout:2/626: symlink d1/d18/d1f/d9d/le4 0 2024-09-21T04:35:11.646 INFO:tasks.workunit.client.1.smithi053.stdout:8/651: rename d3/l46 to d3/d5/d4c/dc3/ld4 0 2024-09-21T04:35:11.649 INFO:tasks.workunit.client.1.smithi053.stdout:1/617: unlink d7/d47/d79/d99/cb2 0 2024-09-21T04:35:11.650 INFO:tasks.workunit.client.1.smithi053.stdout:5/662: dwrite d0/dd/d30/d51/fa0 [0,4194304] 0 2024-09-21T04:35:11.651 INFO:tasks.workunit.client.0.smithi007.stdout:7/105: creat d3/d6/d10/f24 x:0 0 0 2024-09-21T04:35:11.652 INFO:tasks.workunit.client.1.smithi053.stdout:0/565: mknod d5/d6/d21/d96/cc8 0 2024-09-21T04:35:11.653 INFO:tasks.workunit.client.1.smithi053.stdout:0/566: write d5/d6/d53/fa9 [4618680,33444] 0 2024-09-21T04:35:11.656 INFO:tasks.workunit.client.1.smithi053.stdout:7/497: write d0/d2b/d36/d48/d50/d8b/f42 [684187,19315] 0 2024-09-21T04:35:11.657 INFO:tasks.workunit.client.1.smithi053.stdout:4/668: mknod d1/d2c/d8e/db4/cda 0 2024-09-21T04:35:11.657 INFO:tasks.workunit.client.0.smithi007.stdout:4/80: write d2/f6 [835798,78099] 0 2024-09-21T04:35:11.658 INFO:tasks.workunit.client.1.smithi053.stdout:3/577: sync 2024-09-21T04:35:11.658 INFO:tasks.workunit.client.1.smithi053.stdout:6/609: sync 2024-09-21T04:35:11.659 INFO:tasks.workunit.client.1.smithi053.stdout:9/563: rmdir d0/d76/d79 39 2024-09-21T04:35:11.661 INFO:tasks.workunit.client.0.smithi007.stdout:7/106: rename d3/f15 to d3/d1a/f25 0 2024-09-21T04:35:11.661 INFO:tasks.workunit.client.0.smithi007.stdout:7/107: stat d3/d6/fa 0 2024-09-21T04:35:11.662 INFO:tasks.workunit.client.0.smithi007.stdout:7/108: dread - d3/d6/d10/f24 zero size 2024-09-21T04:35:11.663 INFO:tasks.workunit.client.1.smithi053.stdout:2/627: mknod d1/d18/d1f/d27/d58/d5c/d91/dc1/ce5 0 2024-09-21T04:35:11.664 INFO:tasks.workunit.client.0.smithi007.stdout:4/81: creat d2/d7/f22 x:0 0 0 2024-09-21T04:35:11.666 INFO:tasks.workunit.client.1.smithi053.stdout:8/652: read d3/d5/d38/d64/f67 [2265128,90311] 0 2024-09-21T04:35:11.667 INFO:tasks.workunit.client.1.smithi053.stdout:3/578: dwrite d16/d1b/f75 [0,4194304] 0 2024-09-21T04:35:11.669 INFO:tasks.workunit.client.0.smithi007.stdout:7/109: creat d3/d6/d14/f26 x:0 0 0 2024-09-21T04:35:11.669 INFO:tasks.workunit.client.1.smithi053.stdout:6/610: dwrite d2/d1b/d3d/d49/f42 [4194304,4194304] 0 2024-09-21T04:35:11.670 INFO:tasks.workunit.client.1.smithi053.stdout:1/618: mkdir d7/da8/d74/dce 0 2024-09-21T04:35:11.673 INFO:tasks.workunit.client.1.smithi053.stdout:5/663: unlink d0/d8/db/d68/f97 0 2024-09-21T04:35:11.679 INFO:tasks.workunit.client.1.smithi053.stdout:3/579: dread d16/d17/f1e [0,4194304] 0 2024-09-21T04:35:11.680 INFO:tasks.workunit.client.1.smithi053.stdout:2/628: dread d1/d18/d1f/d22/d54/f68 [0,4194304] 0 2024-09-21T04:35:11.682 INFO:tasks.workunit.client.1.smithi053.stdout:0/567: unlink d5/l23 0 2024-09-21T04:35:11.685 INFO:tasks.workunit.client.0.smithi007.stdout:5/113: truncate d2/f8 1126531 0 2024-09-21T04:35:11.686 INFO:tasks.workunit.client.1.smithi053.stdout:7/498: truncate d0/d2b/d36/d48/d50/d8b/f8 4397199 0 2024-09-21T04:35:11.687 INFO:tasks.workunit.client.1.smithi053.stdout:9/564: rmdir d0/d2/d4e 39 2024-09-21T04:35:11.688 INFO:tasks.workunit.client.1.smithi053.stdout:3/580: dwrite d16/d17/d21/d42/d43/d86/fb3 [0,4194304] 0 2024-09-21T04:35:11.689 INFO:tasks.workunit.client.0.smithi007.stdout:5/114: creat d2/d1e/f24 x:0 0 0 2024-09-21T04:35:11.690 INFO:tasks.workunit.client.1.smithi053.stdout:0/568: dwrite d5/de/f67 [0,4194304] 0 2024-09-21T04:35:11.690 INFO:tasks.workunit.client.0.smithi007.stdout:5/115: write d2/d1e/f22 [703298,2495] 0 2024-09-21T04:35:11.692 INFO:tasks.workunit.client.0.smithi007.stdout:5/116: write d2/dd/d1d/f1f [1511725,73249] 0 2024-09-21T04:35:11.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:11 smithi053 ceph-mon[35554]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:11.693 INFO:tasks.workunit.client.0.smithi007.stdout:5/117: rename d2 to d2/dd/d25 22 2024-09-21T04:35:11.694 INFO:tasks.workunit.client.0.smithi007.stdout:7/110: getdents d3/d6/d13 0 2024-09-21T04:35:11.694 INFO:tasks.workunit.client.0.smithi007.stdout:5/118: fdatasync d2/f19 0 2024-09-21T04:35:11.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:11 smithi007 ceph-mon[27954]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:11.695 INFO:tasks.workunit.client.0.smithi007.stdout:5/119: chown d2/dd/c21 119 1 2024-09-21T04:35:11.697 INFO:tasks.workunit.client.1.smithi053.stdout:8/653: creat d3/d28/fd5 x:0 0 0 2024-09-21T04:35:11.705 INFO:tasks.workunit.client.0.smithi007.stdout:4/82: truncate d2/f1d 391984 0 2024-09-21T04:35:11.706 INFO:tasks.workunit.client.0.smithi007.stdout:7/111: dread d3/d6/f8 [0,4194304] 0 2024-09-21T04:35:11.707 INFO:tasks.workunit.client.0.smithi007.stdout:7/112: dread - d3/d6/d13/f1f zero size 2024-09-21T04:35:11.708 INFO:tasks.workunit.client.1.smithi053.stdout:2/629: creat d1/d18/d77/d83/fe6 x:0 0 0 2024-09-21T04:35:11.710 INFO:tasks.workunit.client.1.smithi053.stdout:1/619: dread d7/d9/f1b [0,4194304] 0 2024-09-21T04:35:11.710 INFO:tasks.workunit.client.0.smithi007.stdout:5/120: symlink d2/dd/d23/l26 0 2024-09-21T04:35:11.711 INFO:tasks.workunit.client.1.smithi053.stdout:8/654: dread d3/d5/f19 [4194304,4194304] 0 2024-09-21T04:35:11.711 INFO:tasks.workunit.client.1.smithi053.stdout:1/620: readlink d7/d9/da/d40/d49/d77/l84 0 2024-09-21T04:35:11.712 INFO:tasks.workunit.client.1.smithi053.stdout:8/655: dread - d3/d39/f97 zero size 2024-09-21T04:35:11.713 INFO:tasks.workunit.client.1.smithi053.stdout:1/621: truncate d7/d9/d43/d94/fb9 278452 0 2024-09-21T04:35:11.714 INFO:tasks.workunit.client.1.smithi053.stdout:4/669: mkdir d1/ddb 0 2024-09-21T04:35:11.718 INFO:tasks.workunit.client.0.smithi007.stdout:5/121: dwrite d2/d1e/f22 [0,4194304] 0 2024-09-21T04:35:11.719 INFO:tasks.workunit.client.0.smithi007.stdout:5/122: read d2/dd/f16 [2429246,11987] 0 2024-09-21T04:35:11.720 INFO:tasks.workunit.client.1.smithi053.stdout:7/499: dread d0/f49 [0,4194304] 0 2024-09-21T04:35:11.722 INFO:tasks.workunit.client.0.smithi007.stdout:4/83: truncate d2/d7/f8 2936172 0 2024-09-21T04:35:11.728 INFO:tasks.workunit.client.1.smithi053.stdout:6/611: dwrite d2/d1b/d3d/d49/de/d96/f7b [0,4194304] 0 2024-09-21T04:35:11.731 INFO:tasks.workunit.client.1.smithi053.stdout:3/581: creat d16/d17/d21/d60/fd6 x:0 0 0 2024-09-21T04:35:11.731 INFO:tasks.workunit.client.1.smithi053.stdout:7/500: dread d0/d2b/d36/d48/d50/d8b/fa1 [0,4194304] 0 2024-09-21T04:35:11.735 INFO:tasks.workunit.client.1.smithi053.stdout:7/501: dread d0/d2b/d36/d48/d50/f96 [0,4194304] 0 2024-09-21T04:35:11.739 INFO:tasks.workunit.client.1.smithi053.stdout:3/582: dwrite d16/d17/d3c/f7e [0,4194304] 0 2024-09-21T04:35:11.739 INFO:tasks.workunit.client.0.smithi007.stdout:5/123: mkdir d2/d27 0 2024-09-21T04:35:11.740 INFO:tasks.workunit.client.0.smithi007.stdout:7/113: truncate d3/d6/d14/f18 1159362 0 2024-09-21T04:35:11.743 INFO:tasks.workunit.client.0.smithi007.stdout:5/124: write d2/f1b [969955,43523] 0 2024-09-21T04:35:11.744 INFO:tasks.workunit.client.1.smithi053.stdout:9/565: dwrite d0/d2/fa [0,4194304] 0 2024-09-21T04:35:11.744 INFO:tasks.workunit.client.1.smithi053.stdout:1/622: mknod d7/da8/d74/d75/ccf 0 2024-09-21T04:35:11.745 INFO:tasks.workunit.client.0.smithi007.stdout:7/114: rmdir d3 39 2024-09-21T04:35:11.748 INFO:tasks.workunit.client.1.smithi053.stdout:5/664: write d0/dd/d30/d35/f98 [1975939,4884] 0 2024-09-21T04:35:11.748 INFO:tasks.workunit.client.1.smithi053.stdout:4/670: creat d1/d27/d53/d88/d91/db6/fdc x:0 0 0 2024-09-21T04:35:11.750 INFO:tasks.workunit.client.1.smithi053.stdout:2/630: dwrite d1/d18/d77/d9b/faa [0,4194304] 0 2024-09-21T04:35:11.760 INFO:tasks.workunit.client.1.smithi053.stdout:3/583: dwrite d16/d17/d21/d8b/d7d/d57/fac [4194304,4194304] 0 2024-09-21T04:35:11.764 INFO:tasks.workunit.client.0.smithi007.stdout:7/115: dwrite d3/d6/d14/f26 [0,4194304] 0 2024-09-21T04:35:11.764 INFO:tasks.workunit.client.1.smithi053.stdout:6/612: symlink d2/d1b/d3d/d49/dad/lc8 0 2024-09-21T04:35:11.765 INFO:tasks.workunit.client.1.smithi053.stdout:3/584: readlink d16/d17/d21/d26/d2b/l3f 0 2024-09-21T04:35:11.766 INFO:tasks.workunit.client.1.smithi053.stdout:1/623: dwrite d7/d9/da/d11/f33 [4194304,4194304] 0 2024-09-21T04:35:11.766 INFO:tasks.workunit.client.0.smithi007.stdout:7/116: truncate d3/d6/d13/f1f 885628 0 2024-09-21T04:35:11.768 INFO:tasks.workunit.client.1.smithi053.stdout:0/569: creat d5/d6/d21/fc9 x:0 0 0 2024-09-21T04:35:11.768 INFO:tasks.workunit.client.0.smithi007.stdout:4/84: dread d2/f1d [0,4194304] 0 2024-09-21T04:35:11.775 INFO:tasks.workunit.client.0.smithi007.stdout:4/85: dread - d2/d17/d18/f1e zero size 2024-09-21T04:35:11.775 INFO:tasks.workunit.client.0.smithi007.stdout:5/125: dwrite d2/dd/f16 [4194304,4194304] 0 2024-09-21T04:35:11.775 INFO:tasks.workunit.client.1.smithi053.stdout:7/502: dwrite d0/d2b/d36/d48/f53 [0,4194304] 0 2024-09-21T04:35:11.775 INFO:tasks.workunit.client.1.smithi053.stdout:5/665: creat d0/dd/d5c/d80/fd7 x:0 0 0 2024-09-21T04:35:11.777 INFO:tasks.workunit.client.1.smithi053.stdout:0/570: truncate d5/dd/d50/fc7 122306 0 2024-09-21T04:35:11.778 INFO:tasks.workunit.client.1.smithi053.stdout:9/566: write d0/d2/d9/f19 [1504647,49708] 0 2024-09-21T04:35:11.781 INFO:tasks.workunit.client.0.smithi007.stdout:7/117: rename d3/d6/c12 to d3/d6/d13/c27 0 2024-09-21T04:35:11.783 INFO:tasks.workunit.client.1.smithi053.stdout:6/613: write d2/fd [5239967,72726] 0 2024-09-21T04:35:11.785 INFO:tasks.workunit.client.0.smithi007.stdout:4/86: dwrite d2/d17/d18/f1e [0,4194304] 0 2024-09-21T04:35:11.785 INFO:tasks.workunit.client.0.smithi007.stdout:7/118: read d3/d6/d14/f26 [677775,94793] 0 2024-09-21T04:35:11.797 INFO:tasks.workunit.client.0.smithi007.stdout:4/87: dread d2/fd [0,4194304] 0 2024-09-21T04:35:11.801 INFO:tasks.workunit.client.0.smithi007.stdout:5/126: write d2/f8 [2058135,125610] 0 2024-09-21T04:35:11.803 INFO:tasks.workunit.client.0.smithi007.stdout:4/88: fsync d2/d7/f8 0 2024-09-21T04:35:11.805 INFO:tasks.workunit.client.0.smithi007.stdout:7/119: creat d3/d6/f28 x:0 0 0 2024-09-21T04:35:11.810 INFO:tasks.workunit.client.1.smithi053.stdout:6/614: dread d2/d1b/d3d/d49/de/d32/d80/f26 [0,4194304] 0 2024-09-21T04:35:11.811 INFO:tasks.workunit.client.1.smithi053.stdout:2/631: creat d1/d18/d1f/d22/d43/da6/fe7 x:0 0 0 2024-09-21T04:35:11.816 INFO:tasks.workunit.client.0.smithi007.stdout:4/89: dwrite d2/d7/fa [0,4194304] 0 2024-09-21T04:35:11.816 INFO:tasks.workunit.client.1.smithi053.stdout:9/567: mknod d0/d1c/d52/d6f/cc2 0 2024-09-21T04:35:11.818 INFO:tasks.workunit.client.1.smithi053.stdout:1/624: dread d7/d9/da/d11/f5c [0,4194304] 0 2024-09-21T04:35:11.819 INFO:tasks.workunit.client.1.smithi053.stdout:6/615: mkdir d2/d1b/d1e/d27/d41/dc9 0 2024-09-21T04:35:11.819 INFO:tasks.workunit.client.1.smithi053.stdout:8/656: link d3/df/d14/d16/l44 d3/d5/d21/ld6 0 2024-09-21T04:35:11.820 INFO:tasks.workunit.client.0.smithi007.stdout:7/120: dwrite d3/d6/f28 [0,4194304] 0 2024-09-21T04:35:11.821 INFO:tasks.workunit.client.0.smithi007.stdout:4/90: creat d2/f23 x:0 0 0 2024-09-21T04:35:11.821 INFO:tasks.workunit.client.1.smithi053.stdout:1/625: fsync d7/d9/d43/d94/fa7 0 2024-09-21T04:35:11.822 INFO:tasks.workunit.client.1.smithi053.stdout:2/632: mkdir d1/d18/d1f/d27/d58/db7/de8 0 2024-09-21T04:35:11.822 INFO:tasks.workunit.client.1.smithi053.stdout:6/616: fsync d2/d1b/d1e/d2a/f7c 0 2024-09-21T04:35:11.823 INFO:tasks.workunit.client.1.smithi053.stdout:5/666: dwrite d0/dd/d30/d35/d61/fa5 [0,4194304] 0 2024-09-21T04:35:11.825 INFO:tasks.workunit.client.1.smithi053.stdout:4/671: rename d1/d27/d53/d88/d91/d20/d4e/d51/lcc to d1/d27/d53/d88/d91/ldd 0 2024-09-21T04:35:11.825 INFO:tasks.workunit.client.1.smithi053.stdout:6/617: stat d2/d1b/d1e/d27/d41/c47 0 2024-09-21T04:35:11.827 INFO:tasks.workunit.client.1.smithi053.stdout:4/672: chown d1/d27/d53/d88 52453171 1 2024-09-21T04:35:11.827 INFO:tasks.workunit.client.1.smithi053.stdout:5/667: write d0/dd/d5c/d80/f8e [1954388,32743] 0 2024-09-21T04:35:11.828 INFO:tasks.workunit.client.0.smithi007.stdout:7/121: creat d3/d6/d13/f29 x:0 0 0 2024-09-21T04:35:11.828 INFO:tasks.workunit.client.0.smithi007.stdout:5/127: dwrite d2/dd/f14 [0,4194304] 0 2024-09-21T04:35:11.835 INFO:tasks.workunit.client.1.smithi053.stdout:5/668: dwrite d0/dd/f23 [4194304,4194304] 0 2024-09-21T04:35:11.837 INFO:tasks.workunit.client.0.smithi007.stdout:5/128: unlink d2/l3 0 2024-09-21T04:35:11.839 INFO:tasks.workunit.client.1.smithi053.stdout:2/633: mkdir d1/d18/d1f/d27/d41/d4b/d4d/d89/ddb/de9 0 2024-09-21T04:35:11.840 INFO:tasks.workunit.client.1.smithi053.stdout:3/585: rename d16/d17/d21/d8b/d7d/d57/ca5 to d16/d17/d21/d8b/cd7 0 2024-09-21T04:35:11.842 INFO:tasks.workunit.client.1.smithi053.stdout:6/618: creat d2/d1b/d1e/d2a/d4b/d77/d88/fca x:0 0 0 2024-09-21T04:35:11.843 INFO:tasks.workunit.client.1.smithi053.stdout:4/673: mknod d1/d27/d53/d88/d91/db6/cde 0 2024-09-21T04:35:11.843 INFO:tasks.workunit.client.1.smithi053.stdout:6/619: fsync d2/d1b/f53 0 2024-09-21T04:35:11.845 INFO:tasks.workunit.client.1.smithi053.stdout:4/674: fdatasync d1/d27/d53/d88/d91/d3f/d74/fd3 0 2024-09-21T04:35:11.846 INFO:tasks.workunit.client.0.smithi007.stdout:5/129: dwrite d2/fc [0,4194304] 0 2024-09-21T04:35:11.847 INFO:tasks.workunit.client.1.smithi053.stdout:5/669: creat d0/dd/d48/d8c/dad/fd8 x:0 0 0 2024-09-21T04:35:11.848 INFO:tasks.workunit.client.1.smithi053.stdout:5/670: write d0/dd/d5c/d80/fd2 [443296,72530] 0 2024-09-21T04:35:11.848 INFO:tasks.workunit.client.1.smithi053.stdout:9/568: creat d0/fc3 x:0 0 0 2024-09-21T04:35:11.850 INFO:tasks.workunit.client.1.smithi053.stdout:5/671: fsync d0/d8/db/f78 0 2024-09-21T04:35:11.853 INFO:tasks.workunit.client.0.smithi007.stdout:5/130: dread d2/f1b [0,4194304] 0 2024-09-21T04:35:11.856 INFO:tasks.workunit.client.0.smithi007.stdout:5/131: dread d2/f8 [0,4194304] 0 2024-09-21T04:35:11.857 INFO:tasks.workunit.client.1.smithi053.stdout:0/571: rename d5/d6 to d5/d10/d4c/db1/dbc/dca 0 2024-09-21T04:35:11.859 INFO:tasks.workunit.client.1.smithi053.stdout:3/586: symlink d16/d17/d21/d8b/ld8 0 2024-09-21T04:35:11.860 INFO:tasks.workunit.client.1.smithi053.stdout:3/587: chown d16/d17/d21/d26/d2b 205 1 2024-09-21T04:35:11.861 INFO:tasks.workunit.client.0.smithi007.stdout:7/122: write d3/f5 [7789729,83552] 0 2024-09-21T04:35:11.862 INFO:tasks.workunit.client.0.smithi007.stdout:5/132: mkdir d2/dd/d28 0 2024-09-21T04:35:11.862 INFO:tasks.workunit.client.0.smithi007.stdout:5/133: stat d2/dd 0 2024-09-21T04:35:11.864 INFO:tasks.workunit.client.1.smithi053.stdout:6/620: rmdir d2/d1b/d3d/d49/de/d96 39 2024-09-21T04:35:11.864 INFO:tasks.workunit.client.1.smithi053.stdout:0/572: dwrite d5/f8d [0,4194304] 0 2024-09-21T04:35:11.865 INFO:tasks.workunit.client.1.smithi053.stdout:0/573: fsync d5/de/f5b 0 2024-09-21T04:35:11.866 INFO:tasks.workunit.client.0.smithi007.stdout:7/123: symlink d3/d6/d14/l2a 0 2024-09-21T04:35:11.867 INFO:tasks.workunit.client.1.smithi053.stdout:3/588: dread d16/d17/d21/d60/f9c [0,4194304] 0 2024-09-21T04:35:11.868 INFO:tasks.workunit.client.0.smithi007.stdout:5/134: symlink d2/dd/d23/l29 0 2024-09-21T04:35:11.869 INFO:tasks.workunit.client.1.smithi053.stdout:3/589: stat d16/d17/d21/d60/cc9 0 2024-09-21T04:35:11.872 INFO:tasks.workunit.client.1.smithi053.stdout:8/657: dwrite d3/d39/f3f [0,4194304] 0 2024-09-21T04:35:11.880 INFO:tasks.workunit.client.0.smithi007.stdout:7/124: creat d3/d6/f2b x:0 0 0 2024-09-21T04:35:11.881 INFO:tasks.workunit.client.1.smithi053.stdout:4/675: rename d1/d27/d53/d88/d91/d3f/l48 to d1/d27/d2e/d46/ldf 0 2024-09-21T04:35:11.883 INFO:tasks.workunit.client.0.smithi007.stdout:7/125: creat d3/d6/d13/f2c x:0 0 0 2024-09-21T04:35:11.885 INFO:tasks.workunit.client.1.smithi053.stdout:0/574: unlink d5/de/d42/c7b 0 2024-09-21T04:35:11.887 INFO:tasks.workunit.client.1.smithi053.stdout:9/569: mknod d0/d1c/d52/daa/cc4 0 2024-09-21T04:35:11.888 INFO:tasks.workunit.client.1.smithi053.stdout:9/570: chown d0/d1c/d21/c84 1724 1 2024-09-21T04:35:11.888 INFO:tasks.workunit.client.1.smithi053.stdout:3/590: mknod d16/d17/d21/d8b/cd9 0 2024-09-21T04:35:11.888 INFO:tasks.workunit.client.0.smithi007.stdout:5/135: dwrite d2/f1b [0,4194304] 0 2024-09-21T04:35:11.889 INFO:tasks.workunit.client.1.smithi053.stdout:5/672: mknod d0/d8/db6/dbc/cd9 0 2024-09-21T04:35:11.890 INFO:tasks.workunit.client.1.smithi053.stdout:8/658: unlink d3/df/d14/d16/f35 0 2024-09-21T04:35:11.890 INFO:tasks.workunit.client.1.smithi053.stdout:1/626: link d7/da8/d74/d91/d9f/cc2 d7/da8/d74/d91/cd0 0 2024-09-21T04:35:11.891 INFO:tasks.workunit.client.0.smithi007.stdout:5/136: read d2/f8 [2045164,15611] 0 2024-09-21T04:35:11.891 INFO:tasks.workunit.client.0.smithi007.stdout:7/126: creat d3/d6/f2d x:0 0 0 2024-09-21T04:35:11.892 INFO:tasks.workunit.client.1.smithi053.stdout:4/676: mkdir d1/d27/d33/d41/de0 0 2024-09-21T04:35:11.893 INFO:tasks.workunit.client.0.smithi007.stdout:5/137: mkdir d2/dd/d23/d2a 0 2024-09-21T04:35:11.895 INFO:tasks.workunit.client.1.smithi053.stdout:2/634: dwrite d1/d18/d1f/d27/d41/d4b/f79 [0,4194304] 0 2024-09-21T04:35:11.895 INFO:tasks.workunit.client.1.smithi053.stdout:8/659: chown d3/df/d49/f61 1 1 2024-09-21T04:35:11.897 INFO:tasks.workunit.client.0.smithi007.stdout:7/127: mkdir d3/d6/d2e 0 2024-09-21T04:35:11.897 INFO:tasks.workunit.client.1.smithi053.stdout:8/660: fdatasync d3/d28/f68 0 2024-09-21T04:35:11.901 INFO:tasks.workunit.client.1.smithi053.stdout:6/621: creat d2/d1b/d1e/fcb x:0 0 0 2024-09-21T04:35:11.902 INFO:tasks.workunit.client.0.smithi007.stdout:7/128: write d3/d6/d13/f29 [355030,114774] 0 2024-09-21T04:35:11.903 INFO:tasks.workunit.client.1.smithi053.stdout:3/591: dwrite d16/d17/d21/f6d [0,4194304] 0 2024-09-21T04:35:11.903 INFO:tasks.workunit.client.1.smithi053.stdout:4/677: mkdir d1/d27/d53/d88/d91/d3f/d74/de1 0 2024-09-21T04:35:11.904 INFO:tasks.workunit.client.0.smithi007.stdout:7/129: dread - d3/d6/d14/f1b zero size 2024-09-21T04:35:11.905 INFO:tasks.workunit.client.1.smithi053.stdout:4/678: chown d1/l25 127599694 1 2024-09-21T04:35:11.906 INFO:tasks.workunit.client.0.smithi007.stdout:5/138: dwrite d2/d1e/f24 [0,4194304] 0 2024-09-21T04:35:11.906 INFO:tasks.workunit.client.1.smithi053.stdout:6/622: write d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f9d [4021897,7178] 0 2024-09-21T04:35:11.907 INFO:tasks.workunit.client.0.smithi007.stdout:5/139: read d2/dd/d1d/f1f [2761957,92820] 0 2024-09-21T04:35:11.908 INFO:tasks.workunit.client.1.smithi053.stdout:8/661: mknod d3/df/d10/cd7 0 2024-09-21T04:35:11.911 INFO:tasks.workunit.client.1.smithi053.stdout:9/571: dread d0/d2/fe [0,4194304] 0 2024-09-21T04:35:11.911 INFO:tasks.workunit.client.1.smithi053.stdout:3/592: dread d16/d17/d21/d42/d43/d86/fb3 [0,4194304] 0 2024-09-21T04:35:11.912 INFO:tasks.workunit.client.1.smithi053.stdout:1/627: link d7/d9/d43/d54/d7f/d36/f97 d7/d9/d43/d94/fd1 0 2024-09-21T04:35:11.916 INFO:tasks.workunit.client.0.smithi007.stdout:7/130: dread d3/d1a/f25 [0,4194304] 0 2024-09-21T04:35:11.919 INFO:tasks.workunit.client.1.smithi053.stdout:5/673: write d0/dd/d48/d8c/dad/db9/fcf [7566516,66791] 0 2024-09-21T04:35:11.920 INFO:tasks.workunit.client.1.smithi053.stdout:5/674: chown d0/d66/cab 58 1 2024-09-21T04:35:11.920 INFO:tasks.workunit.client.0.smithi007.stdout:5/140: mknod d2/dd/d1d/c2b 0 2024-09-21T04:35:11.924 INFO:tasks.workunit.client.1.smithi053.stdout:7/503: sync 2024-09-21T04:35:11.926 INFO:tasks.workunit.client.1.smithi053.stdout:0/575: dread d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f86 [0,4194304] 0 2024-09-21T04:35:11.926 INFO:tasks.workunit.client.1.smithi053.stdout:2/635: link d1/d18/f96 d1/d18/d1f/da2/dac/fea 0 2024-09-21T04:35:11.928 INFO:tasks.workunit.client.0.smithi007.stdout:7/131: dread d3/d6/d14/f18 [0,4194304] 0 2024-09-21T04:35:11.928 INFO:tasks.workunit.client.1.smithi053.stdout:2/636: write d1/d18/d1f/d27/d58/d5c/d91/dd0/f80 [462750,41699] 0 2024-09-21T04:35:11.930 INFO:tasks.workunit.client.1.smithi053.stdout:1/628: fsync d7/d24/f41 0 2024-09-21T04:35:11.930 INFO:tasks.workunit.client.1.smithi053.stdout:0/576: dread d5/d10/fb9 [0,4194304] 0 2024-09-21T04:35:11.931 INFO:tasks.workunit.client.0.smithi007.stdout:5/141: mknod d2/dd/c2c 0 2024-09-21T04:35:11.932 INFO:tasks.workunit.client.1.smithi053.stdout:9/572: mknod d0/d1a/d4c/d8f/cc5 0 2024-09-21T04:35:11.934 INFO:tasks.workunit.client.1.smithi053.stdout:3/593: creat d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fda x:0 0 0 2024-09-21T04:35:11.934 INFO:tasks.workunit.client.0.smithi007.stdout:5/142: truncate d2/f8 650756 0 2024-09-21T04:35:11.935 INFO:tasks.workunit.client.0.smithi007.stdout:5/143: chown d2/dd/d23 63939 1 2024-09-21T04:35:11.935 INFO:tasks.workunit.client.0.smithi007.stdout:5/144: stat d2/dd/d1d 0 2024-09-21T04:35:11.936 INFO:tasks.workunit.client.1.smithi053.stdout:5/675: mknod d0/d8/db6/dbc/cda 0 2024-09-21T04:35:11.937 INFO:tasks.workunit.client.1.smithi053.stdout:5/676: truncate d0/dd/d48/d8c/fc6 270819 0 2024-09-21T04:35:11.938 INFO:tasks.workunit.client.0.smithi007.stdout:7/132: link d3/d6/d14/f1b d3/d6/f2f 0 2024-09-21T04:35:11.940 INFO:tasks.workunit.client.1.smithi053.stdout:6/623: write d2/d1b/d3d/f6c [725417,22906] 0 2024-09-21T04:35:11.941 INFO:tasks.workunit.client.0.smithi007.stdout:7/133: write d3/d6/f2f [563592,65613] 0 2024-09-21T04:35:11.942 INFO:tasks.workunit.client.0.smithi007.stdout:7/134: write d3/d6/f2d [236890,110453] 0 2024-09-21T04:35:11.943 INFO:tasks.workunit.client.1.smithi053.stdout:7/504: rename d0/f15 to d0/d2b/d36/d48/d5d/d9f/fa6 0 2024-09-21T04:35:11.944 INFO:tasks.workunit.client.0.smithi007.stdout:7/135: write d3/d6/f2f [667489,36070] 0 2024-09-21T04:35:11.945 INFO:tasks.workunit.client.0.smithi007.stdout:5/145: creat d2/dd/d23/d2a/f2d x:0 0 0 2024-09-21T04:35:11.947 INFO:tasks.workunit.client.1.smithi053.stdout:6/624: dwrite d2/d1b/d1e/fcb [0,4194304] 0 2024-09-21T04:35:11.949 INFO:tasks.workunit.client.0.smithi007.stdout:7/136: read d3/d6/f7 [3502899,98314] 0 2024-09-21T04:35:11.952 INFO:tasks.workunit.client.1.smithi053.stdout:7/505: dwrite d0/d2b/d36/d48/f53 [0,4194304] 0 2024-09-21T04:35:11.953 INFO:tasks.workunit.client.1.smithi053.stdout:7/506: chown d0/d2b/d36/d48/d50/d8b/c2f 273 1 2024-09-21T04:35:11.953 INFO:tasks.workunit.client.1.smithi053.stdout:2/637: mkdir d1/d18/d1f/d22/d2d/deb 0 2024-09-21T04:35:11.954 INFO:tasks.workunit.client.0.smithi007.stdout:7/137: dread d3/d6/f2f [0,4194304] 0 2024-09-21T04:35:11.958 INFO:tasks.workunit.client.0.smithi007.stdout:5/146: mknod d2/dd/d28/c2e 0 2024-09-21T04:35:11.958 INFO:tasks.workunit.client.1.smithi053.stdout:4/679: truncate d1/d27/d2e/d46/f8d 1229974 0 2024-09-21T04:35:11.959 INFO:tasks.workunit.client.0.smithi007.stdout:5/147: write d2/d1e/f24 [2303913,50375] 0 2024-09-21T04:35:11.966 INFO:tasks.workunit.client.1.smithi053.stdout:3/594: creat d16/d17/d21/d26/d3e/fdb x:0 0 0 2024-09-21T04:35:11.968 INFO:tasks.workunit.client.1.smithi053.stdout:8/662: truncate d3/df/d14/d16/f29 3389109 0 2024-09-21T04:35:11.969 INFO:tasks.workunit.client.0.smithi007.stdout:5/148: creat d2/dd/d23/d2a/f2f x:0 0 0 2024-09-21T04:35:11.971 INFO:tasks.workunit.client.1.smithi053.stdout:0/577: rename d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/cc3 to d5/dd/d45/ccb 0 2024-09-21T04:35:11.975 INFO:tasks.workunit.client.0.smithi007.stdout:5/149: readlink d2/l11 0 2024-09-21T04:35:11.976 INFO:tasks.workunit.client.0.smithi007.stdout:7/138: write d3/d6/f7 [1385645,123286] 0 2024-09-21T04:35:11.977 INFO:tasks.workunit.client.0.smithi007.stdout:5/150: write d2/dd/d23/d2a/f2d [950998,80883] 0 2024-09-21T04:35:11.979 INFO:tasks.workunit.client.1.smithi053.stdout:6/625: unlink d2/fba 0 2024-09-21T04:35:11.980 INFO:tasks.workunit.client.1.smithi053.stdout:7/507: mkdir d0/d2b/d36/d64/da7 0 2024-09-21T04:35:11.981 INFO:tasks.workunit.client.1.smithi053.stdout:1/629: rmdir d7/d9/d43/d54/d7f/d59 39 2024-09-21T04:35:11.982 INFO:tasks.workunit.client.0.smithi007.stdout:7/139: fsync d3/d6/f2f 0 2024-09-21T04:35:11.983 INFO:tasks.workunit.client.0.smithi007.stdout:7/140: chown d3/d6/d14/f18 50952 1 2024-09-21T04:35:11.984 INFO:tasks.workunit.client.0.smithi007.stdout:5/151: dwrite d2/f1b [0,4194304] 0 2024-09-21T04:35:11.987 INFO:tasks.workunit.client.1.smithi053.stdout:9/573: creat d0/d2/d4e/fc6 x:0 0 0 2024-09-21T04:35:11.992 INFO:tasks.workunit.client.1.smithi053.stdout:5/677: rename d0/dd/d30 to d0/dd/d5c/d92/ddb 0 2024-09-21T04:35:11.996 INFO:tasks.workunit.client.1.smithi053.stdout:4/680: getdents d1/d27/d33/d41/de0 0 2024-09-21T04:35:11.998 INFO:tasks.workunit.client.0.smithi007.stdout:5/152: getdents d2/d1e 0 2024-09-21T04:35:11.998 INFO:tasks.workunit.client.1.smithi053.stdout:7/508: dread d0/d24/d46/f5f [0,4194304] 0 2024-09-21T04:35:11.998 INFO:tasks.workunit.client.1.smithi053.stdout:3/595: mkdir d16/d17/ddc 0 2024-09-21T04:35:11.999 INFO:tasks.workunit.client.1.smithi053.stdout:5/678: dread d0/d8/f69 [0,4194304] 0 2024-09-21T04:35:11.999 INFO:tasks.workunit.client.1.smithi053.stdout:4/681: write d1/d27/d53/d88/d91/d20/d5b/d87/fad [988976,84324] 0 2024-09-21T04:35:11.999 INFO:tasks.workunit.client.1.smithi053.stdout:9/574: symlink d0/d2/d4e/lc7 0 2024-09-21T04:35:12.000 INFO:tasks.workunit.client.0.smithi007.stdout:5/153: mknod d2/dd/d23/d2a/c30 0 2024-09-21T04:35:12.001 INFO:tasks.workunit.client.1.smithi053.stdout:0/578: symlink d5/d10/lcc 0 2024-09-21T04:35:12.002 INFO:tasks.workunit.client.1.smithi053.stdout:2/638: rename d1/d18/l82 to d1/d18/d1f/d27/d58/d5c/d91/dd0/lec 0 2024-09-21T04:35:12.004 INFO:tasks.workunit.client.1.smithi053.stdout:2/639: stat d1/d18/d1f/d27/d41/d4b/d4d/cde 0 2024-09-21T04:35:12.005 INFO:tasks.workunit.client.1.smithi053.stdout:9/575: dread d0/d76/f7e [0,4194304] 0 2024-09-21T04:35:12.005 INFO:tasks.workunit.client.1.smithi053.stdout:2/640: readlink d1/d18/l7d 0 2024-09-21T04:35:12.007 INFO:tasks.workunit.client.1.smithi053.stdout:8/663: creat d3/df/d14/fd8 x:0 0 0 2024-09-21T04:35:12.007 INFO:tasks.workunit.client.1.smithi053.stdout:5/679: dwrite d0/d66/f82 [0,4194304] 0 2024-09-21T04:35:12.007 INFO:tasks.workunit.client.0.smithi007.stdout:5/154: link d2/dd/c21 d2/c31 0 2024-09-21T04:35:12.008 INFO:tasks.workunit.client.1.smithi053.stdout:3/596: creat d16/d17/d21/d8b/d7d/d57/fdd x:0 0 0 2024-09-21T04:35:12.009 INFO:tasks.workunit.client.1.smithi053.stdout:4/682: symlink d1/d27/d53/d88/d91/d20/d5b/d87/le2 0 2024-09-21T04:35:12.009 INFO:tasks.workunit.client.0.smithi007.stdout:5/155: mknod d2/dd/c32 0 2024-09-21T04:35:12.010 INFO:tasks.workunit.client.1.smithi053.stdout:3/597: read d16/d17/d21/d8b/d7d/d57/fac [4796527,117658] 0 2024-09-21T04:35:12.011 INFO:tasks.workunit.client.1.smithi053.stdout:1/630: link d7/da8/d74/lcb d7/d9/d43/d94/ld2 0 2024-09-21T04:35:12.012 INFO:tasks.workunit.client.0.smithi007.stdout:5/156: creat d2/dd/d1d/f33 x:0 0 0 2024-09-21T04:35:12.012 INFO:tasks.workunit.client.0.smithi007.stdout:5/157: read - d2/f19 zero size 2024-09-21T04:35:12.014 INFO:tasks.workunit.client.1.smithi053.stdout:7/509: fsync d0/d2b/d36/d48/d50/d8b/f1b 0 2024-09-21T04:35:12.014 INFO:tasks.workunit.client.1.smithi053.stdout:1/631: chown d7/d9/da/d3c 1 1 2024-09-21T04:35:12.015 INFO:tasks.workunit.client.0.smithi007.stdout:5/158: unlink d2/l1c 0 2024-09-21T04:35:12.017 INFO:tasks.workunit.client.1.smithi053.stdout:2/641: fdatasync d1/d18/d1f/da2/dac/fea 0 2024-09-21T04:35:12.017 INFO:tasks.workunit.client.1.smithi053.stdout:8/664: rmdir d3/df/dba 39 2024-09-21T04:35:12.019 INFO:tasks.workunit.client.0.smithi007.stdout:5/159: symlink d2/dd/d23/l34 0 2024-09-21T04:35:12.020 INFO:tasks.workunit.client.1.smithi053.stdout:5/680: readlink d0/d8/db/d68/l70 0 2024-09-21T04:35:12.020 INFO:tasks.workunit.client.0.smithi007.stdout:5/160: write d2/dd/d23/d2a/f2d [1739154,80128] 0 2024-09-21T04:35:12.026 INFO:tasks.workunit.client.0.smithi007.stdout:5/161: creat d2/dd/d1d/d20/f35 x:0 0 0 2024-09-21T04:35:12.027 INFO:tasks.workunit.client.0.smithi007.stdout:5/162: symlink d2/dd/d23/l36 0 2024-09-21T04:35:12.028 INFO:tasks.workunit.client.0.smithi007.stdout:5/163: chown d2/dd/d23/d2a/f2d 326048 1 2024-09-21T04:35:12.029 INFO:tasks.workunit.client.1.smithi053.stdout:3/598: dwrite d16/d17/d21/d8b/d7d/d57/dc1/d53/fa1 [4194304,4194304] 0 2024-09-21T04:35:12.030 INFO:tasks.workunit.client.1.smithi053.stdout:7/510: chown d0/d2b/d36/d48/d50/l66 1039 1 2024-09-21T04:35:12.031 INFO:tasks.workunit.client.1.smithi053.stdout:1/632: creat d7/d9/d43/d54/d7f/d36/fd3 x:0 0 0 2024-09-21T04:35:12.034 INFO:tasks.workunit.client.1.smithi053.stdout:2/642: creat d1/d18/d77/d9b/fed x:0 0 0 2024-09-21T04:35:12.036 INFO:tasks.workunit.client.1.smithi053.stdout:1/633: write d7/d9/da/d3c/fbc [812378,110779] 0 2024-09-21T04:35:12.037 INFO:tasks.workunit.client.1.smithi053.stdout:5/681: rename d0/d8/c47 to d0/d8/d2a/cdc 0 2024-09-21T04:35:12.038 INFO:tasks.workunit.client.1.smithi053.stdout:4/683: mknod d1/d27/d53/d88/d91/d3f/d74/de1/ce3 0 2024-09-21T04:35:12.039 INFO:tasks.workunit.client.0.smithi007.stdout:5/164: getdents d2 0 2024-09-21T04:35:12.041 INFO:tasks.workunit.client.1.smithi053.stdout:1/634: dread - d7/da8/fc4 zero size 2024-09-21T04:35:12.042 INFO:tasks.workunit.client.0.smithi007.stdout:5/165: unlink d2/dd/d1d/f33 0 2024-09-21T04:35:12.044 INFO:tasks.workunit.client.1.smithi053.stdout:3/599: dwrite d16/d17/d21/d26/f92 [0,4194304] 0 2024-09-21T04:35:12.044 INFO:tasks.workunit.client.1.smithi053.stdout:1/635: chown d7/d24 0 1 2024-09-21T04:35:12.044 INFO:tasks.workunit.client.1.smithi053.stdout:0/579: dwrite d5/dd/d50/fc7 [0,4194304] 0 2024-09-21T04:35:12.044 INFO:tasks.workunit.client.1.smithi053.stdout:9/576: truncate d0/d1c/f39 2998486 0 2024-09-21T04:35:12.046 INFO:tasks.workunit.client.1.smithi053.stdout:7/511: creat d0/d2b/fa8 x:0 0 0 2024-09-21T04:35:12.047 INFO:tasks.workunit.client.1.smithi053.stdout:2/643: rename d1/d18/d1f/d27/d58/db7/de8 to d1/d18/d1f/d22/d2d/dcd/dee 0 2024-09-21T04:35:12.048 INFO:tasks.workunit.client.1.smithi053.stdout:1/636: write d7/d9/d43/d94/f61 [4647344,60020] 0 2024-09-21T04:35:12.051 INFO:tasks.workunit.client.1.smithi053.stdout:1/637: fsync d7/da8/d74/d91/f9a 0 2024-09-21T04:35:12.052 INFO:tasks.workunit.client.0.smithi007.stdout:0/38: sync 2024-09-21T04:35:12.052 INFO:tasks.workunit.client.1.smithi053.stdout:2/644: truncate d1/f8c 1532016 0 2024-09-21T04:35:12.053 INFO:tasks.workunit.client.0.smithi007.stdout:5/166: getdents d2/dd/d23 0 2024-09-21T04:35:12.059 INFO:tasks.workunit.client.1.smithi053.stdout:5/682: rename d0/d8/db/d68/fb0 to d0/dd/d5c/d89/fdd 0 2024-09-21T04:35:12.060 INFO:tasks.workunit.client.0.smithi007.stdout:0/39: creat d6/fd x:0 0 0 2024-09-21T04:35:12.061 INFO:tasks.workunit.client.0.smithi007.stdout:5/167: dread d2/dd/d23/d2a/f2d [0,4194304] 0 2024-09-21T04:35:12.062 INFO:tasks.workunit.client.1.smithi053.stdout:7/512: creat d0/d2b/d36/fa9 x:0 0 0 2024-09-21T04:35:12.064 INFO:tasks.workunit.client.1.smithi053.stdout:5/683: fsync d0/d8/d2a/fae 0 2024-09-21T04:35:12.064 INFO:tasks.workunit.client.0.smithi007.stdout:0/40: rename c5 to d6/ce 0 2024-09-21T04:35:12.065 INFO:tasks.workunit.client.1.smithi053.stdout:3/600: dwrite d16/d17/d21/d8b/d7d/f85 [0,4194304] 0 2024-09-21T04:35:12.065 INFO:tasks.workunit.client.1.smithi053.stdout:1/638: creat d7/d63/d96/dac/fd4 x:0 0 0 2024-09-21T04:35:12.066 INFO:tasks.workunit.client.0.smithi007.stdout:5/168: creat d2/d1e/f37 x:0 0 0 2024-09-21T04:35:12.066 INFO:tasks.workunit.client.1.smithi053.stdout:4/684: link d1/c3d d1/d27/d53/d88/d91/d20/d4e/ce4 0 2024-09-21T04:35:12.067 INFO:tasks.workunit.client.0.smithi007.stdout:0/41: mknod d6/cf 0 2024-09-21T04:35:12.067 INFO:tasks.workunit.client.0.smithi007.stdout:5/169: dread - d2/dd/d23/d2a/f2f zero size 2024-09-21T04:35:12.067 INFO:tasks.workunit.client.1.smithi053.stdout:0/580: dread d5/dd/f34 [0,4194304] 0 2024-09-21T04:35:12.068 INFO:tasks.workunit.client.0.smithi007.stdout:5/170: truncate d2/dd/f16 8733503 0 2024-09-21T04:35:12.068 INFO:tasks.workunit.client.1.smithi053.stdout:2/645: dwrite d1/d18/d1f/d27/d58/d5c/fd4 [0,4194304] 0 2024-09-21T04:35:12.069 INFO:tasks.workunit.client.0.smithi007.stdout:5/171: fdatasync d2/d1e/f24 0 2024-09-21T04:35:12.071 INFO:tasks.workunit.client.1.smithi053.stdout:0/581: dread - d5/de/d42/d78/fb8 zero size 2024-09-21T04:35:12.071 INFO:tasks.workunit.client.0.smithi007.stdout:0/42: mknod d6/c10 0 2024-09-21T04:35:12.077 INFO:tasks.workunit.client.0.smithi007.stdout:5/172: fsync d2/dd/d23/d2a/f2d 0 2024-09-21T04:35:12.077 INFO:tasks.workunit.client.1.smithi053.stdout:7/513: dwrite d0/fb [4194304,4194304] 0 2024-09-21T04:35:12.077 INFO:tasks.workunit.client.1.smithi053.stdout:0/582: write d5/d10/d4c/db1/dbc/dca/d21/d96/fc2 [652262,606] 0 2024-09-21T04:35:12.080 INFO:tasks.workunit.client.0.smithi007.stdout:5/173: dread d2/dd/d23/d2a/f2d [0,4194304] 0 2024-09-21T04:35:12.081 INFO:tasks.workunit.client.1.smithi053.stdout:3/601: write d16/f1d [4095383,128803] 0 2024-09-21T04:35:12.081 INFO:tasks.workunit.client.1.smithi053.stdout:1/639: dwrite d7/d63/d96/dac/fd4 [0,4194304] 0 2024-09-21T04:35:12.085 INFO:tasks.workunit.client.1.smithi053.stdout:4/685: rmdir d1/d27/d53/d88/d91/d3f/d74 39 2024-09-21T04:35:12.087 INFO:tasks.workunit.client.0.smithi007.stdout:5/174: mknod d2/dd/d23/c38 0 2024-09-21T04:35:12.087 INFO:tasks.workunit.client.1.smithi053.stdout:9/577: truncate d0/f24 189967 0 2024-09-21T04:35:12.089 INFO:tasks.workunit.client.0.smithi007.stdout:0/43: dwrite d6/fc [0,4194304] 0 2024-09-21T04:35:12.091 INFO:tasks.workunit.client.1.smithi053.stdout:7/514: dread d0/d24/d46/f5f [0,4194304] 0 2024-09-21T04:35:12.095 INFO:tasks.workunit.client.1.smithi053.stdout:5/684: unlink d0/d8/db6/dbc/ld0 0 2024-09-21T04:35:12.097 INFO:tasks.workunit.client.1.smithi053.stdout:5/685: chown d0/dd/d5c/d92/ddb/d51/d52/l58 59635 1 2024-09-21T04:35:12.098 INFO:tasks.workunit.client.0.smithi007.stdout:0/44: mknod d6/c11 0 2024-09-21T04:35:12.101 INFO:tasks.workunit.client.0.smithi007.stdout:5/175: dwrite d2/fc [4194304,4194304] 0 2024-09-21T04:35:12.102 INFO:tasks.workunit.client.1.smithi053.stdout:3/602: write d16/f3d [1194099,12632] 0 2024-09-21T04:35:12.102 INFO:tasks.workunit.client.1.smithi053.stdout:1/640: symlink d7/d9/d43/d54/d7f/d36/d81/d95/ld5 0 2024-09-21T04:35:12.103 INFO:tasks.workunit.client.0.smithi007.stdout:5/176: read d2/dd/f14 [1472551,72588] 0 2024-09-21T04:35:12.105 INFO:tasks.workunit.client.0.smithi007.stdout:0/45: rename d6/fc to d6/f12 0 2024-09-21T04:35:12.107 INFO:tasks.workunit.client.0.smithi007.stdout:0/46: write f4 [2604685,45104] 0 2024-09-21T04:35:12.112 INFO:tasks.workunit.client.1.smithi053.stdout:7/515: mknod d0/d24/d46/caa 0 2024-09-21T04:35:12.114 INFO:tasks.workunit.client.0.smithi007.stdout:5/177: dwrite d2/fc [0,4194304] 0 2024-09-21T04:35:12.118 INFO:tasks.workunit.client.1.smithi053.stdout:5/686: mknod d0/dd/d5c/d89/cde 0 2024-09-21T04:35:12.120 INFO:tasks.workunit.client.0.smithi007.stdout:0/47: dwrite d6/fd [0,4194304] 0 2024-09-21T04:35:12.121 INFO:tasks.workunit.client.1.smithi053.stdout:4/686: mknod d1/dd/d62/ce5 0 2024-09-21T04:35:12.122 INFO:tasks.workunit.client.1.smithi053.stdout:4/687: dread - d1/d27/d2e/d46/fbd zero size 2024-09-21T04:35:12.123 INFO:tasks.workunit.client.1.smithi053.stdout:4/688: stat d1/d27/d2e/f61 0 2024-09-21T04:35:12.129 INFO:tasks.workunit.client.0.smithi007.stdout:0/48: dread f4 [0,4194304] 0 2024-09-21T04:35:12.134 INFO:tasks.workunit.client.0.smithi007.stdout:0/49: creat d6/f13 x:0 0 0 2024-09-21T04:35:12.139 INFO:tasks.workunit.client.1.smithi053.stdout:9/578: truncate d0/d1a/d4c/fab 114461 0 2024-09-21T04:35:12.140 INFO:tasks.workunit.client.0.smithi007.stdout:0/50: rmdir d6 39 2024-09-21T04:35:12.142 INFO:tasks.workunit.client.0.smithi007.stdout:5/178: write d2/dd/d1d/f1f [1455583,59835] 0 2024-09-21T04:35:12.143 INFO:tasks.workunit.client.0.smithi007.stdout:0/51: fsync d6/f9 0 2024-09-21T04:35:12.144 INFO:tasks.workunit.client.0.smithi007.stdout:0/52: write d6/f9 [926592,82283] 0 2024-09-21T04:35:12.145 INFO:tasks.workunit.client.1.smithi053.stdout:2/646: getdents d1/d18/d1f/d27/d41/d4b/d4d/d89 0 2024-09-21T04:35:12.145 INFO:tasks.workunit.client.1.smithi053.stdout:9/579: dwrite d0/d1a/d89/fbb [0,4194304] 0 2024-09-21T04:35:12.147 INFO:tasks.workunit.client.0.smithi007.stdout:5/179: creat d2/dd/d1d/f39 x:0 0 0 2024-09-21T04:35:12.147 INFO:tasks.workunit.client.1.smithi053.stdout:3/603: symlink d16/d17/d21/d42/lde 0 2024-09-21T04:35:12.148 INFO:tasks.workunit.client.0.smithi007.stdout:5/180: chown d2/dd/d28 70 1 2024-09-21T04:35:12.148 INFO:tasks.workunit.client.1.smithi053.stdout:0/583: link d5/de/d42/d78/l5c d5/d47/lcd 0 2024-09-21T04:35:12.149 INFO:tasks.workunit.client.1.smithi053.stdout:7/516: symlink d0/d2b/d36/d64/d9e/lab 0 2024-09-21T04:35:12.150 INFO:tasks.workunit.client.1.smithi053.stdout:1/641: dwrite d7/d9/da/d40/d49/f5e [0,4194304] 0 2024-09-21T04:35:12.152 INFO:tasks.workunit.client.0.smithi007.stdout:0/53: dwrite d6/f9 [0,4194304] 0 2024-09-21T04:35:12.153 INFO:tasks.workunit.client.0.smithi007.stdout:0/54: chown c2 24559332 1 2024-09-21T04:35:12.154 INFO:tasks.workunit.client.0.smithi007.stdout:5/181: mkdir d2/dd/d1d/d3a 0 2024-09-21T04:35:12.155 INFO:tasks.workunit.client.1.smithi053.stdout:2/647: creat d1/d18/d77/fef x:0 0 0 2024-09-21T04:35:12.156 INFO:tasks.workunit.client.0.smithi007.stdout:2/52: sync 2024-09-21T04:35:12.156 INFO:tasks.workunit.client.0.smithi007.stdout:9/53: sync 2024-09-21T04:35:12.157 INFO:tasks.workunit.client.1.smithi053.stdout:9/580: unlink d0/d1c/d21/d46/l68 0 2024-09-21T04:35:12.157 INFO:tasks.workunit.client.0.smithi007.stdout:9/54: chown f6 1317 1 2024-09-21T04:35:12.158 INFO:tasks.workunit.client.0.smithi007.stdout:0/55: creat d6/f14 x:0 0 0 2024-09-21T04:35:12.158 INFO:tasks.workunit.client.0.smithi007.stdout:9/55: write fa [615652,85732] 0 2024-09-21T04:35:12.158 INFO:tasks.workunit.client.0.smithi007.stdout:2/53: read d0/f8 [8594,87477] 0 2024-09-21T04:35:12.160 INFO:tasks.workunit.client.0.smithi007.stdout:5/182: mknod d2/dd/d28/c3b 0 2024-09-21T04:35:12.160 INFO:tasks.workunit.client.1.smithi053.stdout:5/687: link d0/dd/d48/d8c/la1 d0/dd/d48/d8c/ldf 0 2024-09-21T04:35:12.161 INFO:tasks.workunit.client.0.smithi007.stdout:9/56: write f6 [1408010,54836] 0 2024-09-21T04:35:12.162 INFO:tasks.workunit.client.0.smithi007.stdout:2/54: read d0/f3 [4193560,46126] 0 2024-09-21T04:35:12.163 INFO:tasks.workunit.client.1.smithi053.stdout:1/642: creat d7/d47/fd6 x:0 0 0 2024-09-21T04:35:12.165 INFO:tasks.workunit.client.1.smithi053.stdout:0/584: rmdir d5/d10/d4c/db1/dbc/dca/d21/d96 39 2024-09-21T04:35:12.167 INFO:tasks.workunit.client.1.smithi053.stdout:0/585: write d5/de/d42/d78/f6f [2478616,122972] 0 2024-09-21T04:35:12.168 INFO:tasks.workunit.client.0.smithi007.stdout:0/56: rename l3 to d6/l15 0 2024-09-21T04:35:12.169 INFO:tasks.workunit.client.1.smithi053.stdout:0/586: write d5/de/d42/d78/f6f [6640491,26486] 0 2024-09-21T04:35:12.172 INFO:tasks.workunit.client.1.smithi053.stdout:3/604: mknod d16/d17/cdf 0 2024-09-21T04:35:12.172 INFO:tasks.workunit.client.0.smithi007.stdout:2/55: write d0/d2/f6 [1036957,19058] 0 2024-09-21T04:35:12.173 INFO:tasks.workunit.client.0.smithi007.stdout:5/183: rename d2/c15 to d2/dd/d23/d2a/c3c 0 2024-09-21T04:35:12.175 INFO:tasks.workunit.client.1.smithi053.stdout:5/688: rename d0/d8/db/cf to d0/d66/ce0 0 2024-09-21T04:35:12.175 INFO:tasks.workunit.client.1.smithi053.stdout:3/605: dread - d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fbf zero size 2024-09-21T04:35:12.175 INFO:tasks.workunit.client.0.smithi007.stdout:9/57: mknod d13/c14 0 2024-09-21T04:35:12.175 INFO:tasks.workunit.client.0.smithi007.stdout:2/56: fsync d0/d2/f6 0 2024-09-21T04:35:12.176 INFO:tasks.workunit.client.1.smithi053.stdout:2/648: dwrite d1/d18/d1f/d27/d58/fb3 [0,4194304] 0 2024-09-21T04:35:12.177 INFO:tasks.workunit.client.1.smithi053.stdout:1/643: symlink d7/d9/da/d4c/ld7 0 2024-09-21T04:35:12.177 INFO:tasks.workunit.client.1.smithi053.stdout:3/606: write d16/d17/faf [4544921,27745] 0 2024-09-21T04:35:12.178 INFO:tasks.workunit.client.1.smithi053.stdout:2/649: fsync d1/d18/d1f/d27/d71/fb1 0 2024-09-21T04:35:12.183 INFO:tasks.workunit.client.0.smithi007.stdout:2/57: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:12.185 INFO:tasks.workunit.client.1.smithi053.stdout:4/689: getdents d1/d27 0 2024-09-21T04:35:12.187 INFO:tasks.workunit.client.0.smithi007.stdout:9/58: link f11 d13/f15 0 2024-09-21T04:35:12.188 INFO:tasks.workunit.client.1.smithi053.stdout:9/581: symlink d0/d2/lc8 0 2024-09-21T04:35:12.189 INFO:tasks.workunit.client.0.smithi007.stdout:2/58: mknod d0/d2/cb 0 2024-09-21T04:35:12.190 INFO:tasks.workunit.client.0.smithi007.stdout:2/59: chown d0/d2/f6 2460 1 2024-09-21T04:35:12.191 INFO:tasks.workunit.client.1.smithi053.stdout:0/587: rename d5/d10/f9c to d5/d10/d4c/db1/dbc/dca/db5/d99/fce 0 2024-09-21T04:35:12.193 INFO:tasks.workunit.client.0.smithi007.stdout:9/59: creat d13/f16 x:0 0 0 2024-09-21T04:35:12.194 INFO:tasks.workunit.client.1.smithi053.stdout:5/689: truncate d0/d8/f2b 1720055 0 2024-09-21T04:35:12.197 INFO:tasks.workunit.client.1.smithi053.stdout:4/690: dread d1/d27/d53/d88/d91/d20/f45 [0,4194304] 0 2024-09-21T04:35:12.197 INFO:tasks.workunit.client.1.smithi053.stdout:4/691: stat d1/d27/d53/d88/d91/d3f/f77 0 2024-09-21T04:35:12.198 INFO:tasks.workunit.client.0.smithi007.stdout:2/60: write d0/f8 [392524,50248] 0 2024-09-21T04:35:12.199 INFO:tasks.workunit.client.1.smithi053.stdout:1/644: symlink d7/d47/d79/dc1/ld8 0 2024-09-21T04:35:12.200 INFO:tasks.workunit.client.0.smithi007.stdout:2/61: chown d0/f8 6651 1 2024-09-21T04:35:12.201 INFO:tasks.workunit.client.1.smithi053.stdout:1/645: write d7/d9/da/d40/d49/d77/f90 [890536,35898] 0 2024-09-21T04:35:12.203 INFO:tasks.workunit.client.0.smithi007.stdout:9/60: dwrite fa [0,4194304] 0 2024-09-21T04:35:12.210 INFO:tasks.workunit.client.1.smithi053.stdout:3/607: rename d16/d17/d21/d8b/d7d/f4a to d16/d17/d21/d26/d49/fe0 0 2024-09-21T04:35:12.213 INFO:tasks.workunit.client.1.smithi053.stdout:0/588: creat d5/d10/d4c/db1/dbc/fcf x:0 0 0 2024-09-21T04:35:12.215 INFO:tasks.workunit.client.1.smithi053.stdout:5/690: creat d0/dd/d5c/d89/fe1 x:0 0 0 2024-09-21T04:35:12.218 INFO:tasks.workunit.client.1.smithi053.stdout:2/650: creat d1/d18/d1f/d27/d41/d4b/d4d/d89/ddb/de9/ff0 x:0 0 0 2024-09-21T04:35:12.219 INFO:tasks.workunit.client.1.smithi053.stdout:0/589: dwrite d5/de/d42/d78/fb8 [0,4194304] 0 2024-09-21T04:35:12.219 INFO:tasks.workunit.client.0.smithi007.stdout:2/62: creat d0/d2/fc x:0 0 0 2024-09-21T04:35:12.220 INFO:tasks.workunit.client.1.smithi053.stdout:1/646: rmdir d7/d9/d43/d54/d7f 39 2024-09-21T04:35:12.221 INFO:tasks.workunit.client.1.smithi053.stdout:0/590: read d5/dd/d45/f6c [149835,7901] 0 2024-09-21T04:35:12.221 INFO:tasks.workunit.client.1.smithi053.stdout:2/651: write d1/d18/d1f/d27/d41/d4b/d4d/d89/ddb/de9/ff0 [598641,49047] 0 2024-09-21T04:35:12.224 INFO:tasks.workunit.client.0.smithi007.stdout:2/63: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:12.228 INFO:tasks.workunit.client.1.smithi053.stdout:3/608: creat d16/d17/d21/d26/d2b/fe1 x:0 0 0 2024-09-21T04:35:12.230 INFO:tasks.workunit.client.0.smithi007.stdout:5/184: mkdir d2/d27/d3d 0 2024-09-21T04:35:12.235 INFO:tasks.workunit.client.0.smithi007.stdout:5/185: dread - d2/dd/d23/d2a/f2f zero size 2024-09-21T04:35:12.237 INFO:tasks.workunit.client.0.smithi007.stdout:2/64: link d0/d2/fc d0/fd 0 2024-09-21T04:35:12.239 INFO:tasks.workunit.client.0.smithi007.stdout:5/186: creat d2/dd/f3e x:0 0 0 2024-09-21T04:35:12.239 INFO:tasks.workunit.client.1.smithi053.stdout:1/647: rmdir d7/da8/d74/d75 39 2024-09-21T04:35:12.240 INFO:tasks.workunit.client.0.smithi007.stdout:2/65: symlink d0/d2/le 0 2024-09-21T04:35:12.242 INFO:tasks.workunit.client.1.smithi053.stdout:2/652: mknod d1/d18/cf1 0 2024-09-21T04:35:12.244 INFO:tasks.workunit.client.0.smithi007.stdout:5/187: write d2/fc [7985260,61512] 0 2024-09-21T04:35:12.244 INFO:tasks.workunit.client.1.smithi053.stdout:0/591: mkdir d5/de/d42/d78/dd0 0 2024-09-21T04:35:12.246 INFO:tasks.workunit.client.1.smithi053.stdout:9/582: creat d0/d76/fc9 x:0 0 0 2024-09-21T04:35:12.246 INFO:tasks.workunit.client.1.smithi053.stdout:8/665: sync 2024-09-21T04:35:12.246 INFO:tasks.workunit.client.1.smithi053.stdout:6/626: sync 2024-09-21T04:35:12.247 INFO:tasks.workunit.client.0.smithi007.stdout:2/66: dwrite d0/d2/f6 [0,4194304] 0 2024-09-21T04:35:12.248 INFO:tasks.workunit.client.1.smithi053.stdout:3/609: mknod d16/d17/d21/ce2 0 2024-09-21T04:35:12.248 INFO:tasks.workunit.client.1.smithi053.stdout:8/666: dread d3/d5/d38/d64/d9d/fb9 [0,4194304] 0 2024-09-21T04:35:12.250 INFO:tasks.workunit.client.0.smithi007.stdout:5/188: creat d2/dd/d1d/f3f x:0 0 0 2024-09-21T04:35:12.251 INFO:tasks.workunit.client.0.smithi007.stdout:5/189: chown d2/dd/d28/c2e 33548840 1 2024-09-21T04:35:12.252 INFO:tasks.workunit.client.1.smithi053.stdout:4/692: creat d1/d27/d53/d88/d91/d3f/fe6 x:0 0 0 2024-09-21T04:35:12.255 INFO:tasks.workunit.client.1.smithi053.stdout:2/653: mkdir d1/df2 0 2024-09-21T04:35:12.259 INFO:tasks.workunit.client.0.smithi007.stdout:5/190: mknod d2/d27/d3d/c40 0 2024-09-21T04:35:12.260 INFO:tasks.workunit.client.1.smithi053.stdout:9/583: rename d0/d2/d9/fc to d0/d2/d4e/fca 0 2024-09-21T04:35:12.261 INFO:tasks.workunit.client.0.smithi007.stdout:5/191: dread d2/dd/d23/d2a/f2d [0,4194304] 0 2024-09-21T04:35:12.262 INFO:tasks.workunit.client.1.smithi053.stdout:9/584: write d0/d28/f3a [764812,115774] 0 2024-09-21T04:35:12.262 INFO:tasks.workunit.client.1.smithi053.stdout:3/610: mknod d16/d1b/ce3 0 2024-09-21T04:35:12.264 INFO:tasks.workunit.client.1.smithi053.stdout:5/691: creat d0/dd/d5c/d92/ddb/d51/fe2 x:0 0 0 2024-09-21T04:35:12.265 INFO:tasks.workunit.client.1.smithi053.stdout:5/692: stat d0/dd/d5c/dcc 0 2024-09-21T04:35:12.266 INFO:tasks.workunit.client.1.smithi053.stdout:4/693: unlink d1/dd/d95/cc4 0 2024-09-21T04:35:12.267 INFO:tasks.workunit.client.0.smithi007.stdout:2/67: write d0/d2/fc [1044524,79278] 0 2024-09-21T04:35:12.268 INFO:tasks.workunit.client.1.smithi053.stdout:2/654: rmdir d1/d18/d1f/d22/d43 39 2024-09-21T04:35:12.268 INFO:tasks.workunit.client.1.smithi053.stdout:4/694: write d1/d27/d53/fc2 [266884,122182] 0 2024-09-21T04:35:12.268 INFO:tasks.workunit.client.1.smithi053.stdout:0/592: symlink d5/de/ld1 0 2024-09-21T04:35:12.270 INFO:tasks.workunit.client.1.smithi053.stdout:0/593: write d5/de/f3e [1094043,124664] 0 2024-09-21T04:35:12.271 INFO:tasks.workunit.client.1.smithi053.stdout:6/627: rename d2/d1b/d1e/d27/d41/f64 to d2/d1b/d1e/d2a/d4b/d77/d88/da4/fcc 0 2024-09-21T04:35:12.271 INFO:tasks.workunit.client.1.smithi053.stdout:6/628: stat d2/d1d/f54 0 2024-09-21T04:35:12.273 INFO:tasks.workunit.client.0.smithi007.stdout:5/192: dwrite d2/d1e/f24 [0,4194304] 0 2024-09-21T04:35:12.275 INFO:tasks.workunit.client.1.smithi053.stdout:1/648: creat d7/d9/da/fd9 x:0 0 0 2024-09-21T04:35:12.277 INFO:tasks.workunit.client.1.smithi053.stdout:6/629: dwrite d2/d1b/d3d/d49/f42 [4194304,4194304] 0 2024-09-21T04:35:12.282 INFO:tasks.workunit.client.0.smithi007.stdout:5/193: unlink d2/cb 0 2024-09-21T04:35:12.286 INFO:tasks.workunit.client.1.smithi053.stdout:2/655: mknod d1/d18/d1f/d27/d58/d5c/d91/cf3 0 2024-09-21T04:35:12.286 INFO:tasks.workunit.client.1.smithi053.stdout:2/656: chown d1/d18/d1f/d22/f5a 5138 1 2024-09-21T04:35:12.289 INFO:tasks.workunit.client.1.smithi053.stdout:7/517: sync 2024-09-21T04:35:12.291 INFO:tasks.workunit.client.1.smithi053.stdout:7/518: write d0/f20 [5630662,45471] 0 2024-09-21T04:35:12.291 INFO:tasks.workunit.client.1.smithi053.stdout:8/667: rename d3/d5/d4b/d5c/cd2 to d3/d5/d4b/d5c/d86/dcd/cd9 0 2024-09-21T04:35:12.293 INFO:tasks.workunit.client.0.smithi007.stdout:2/68: truncate d0/d2/fc 1112450 0 2024-09-21T04:35:12.294 INFO:tasks.workunit.client.0.smithi007.stdout:5/194: symlink d2/dd/d1d/d3a/l41 0 2024-09-21T04:35:12.296 INFO:tasks.workunit.client.0.smithi007.stdout:2/69: rmdir d0 39 2024-09-21T04:35:12.298 INFO:tasks.workunit.client.0.smithi007.stdout:5/195: unlink d2/dd/d1d/d20/f35 0 2024-09-21T04:35:12.299 INFO:tasks.workunit.client.1.smithi053.stdout:9/585: dwrite d0/d76/f7e [0,4194304] 0 2024-09-21T04:35:12.300 INFO:tasks.workunit.client.1.smithi053.stdout:8/668: dwrite d3/d5/d4c/d8e/fce [0,4194304] 0 2024-09-21T04:35:12.300 INFO:tasks.workunit.client.1.smithi053.stdout:9/586: readlink d0/l5a 0 2024-09-21T04:35:12.302 INFO:tasks.workunit.client.0.smithi007.stdout:2/70: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:12.307 INFO:tasks.workunit.client.1.smithi053.stdout:0/594: dwrite d5/de/f35 [0,4194304] 0 2024-09-21T04:35:12.310 INFO:tasks.workunit.client.1.smithi053.stdout:0/595: dread - d5/d10/d4c/db1/dbc/dca/d21/fc9 zero size 2024-09-21T04:35:12.310 INFO:tasks.workunit.client.1.smithi053.stdout:3/611: truncate d16/d1b/dad/fb1 3907591 0 2024-09-21T04:35:12.313 INFO:tasks.workunit.client.1.smithi053.stdout:5/693: write d0/d8/db/f29 [6356503,32339] 0 2024-09-21T04:35:12.314 INFO:tasks.workunit.client.0.smithi007.stdout:5/196: dwrite d2/d1e/f24 [4194304,4194304] 0 2024-09-21T04:35:12.318 INFO:tasks.workunit.client.1.smithi053.stdout:7/519: mkdir d0/d2b/d36/d48/d5c/dac 0 2024-09-21T04:35:12.318 INFO:tasks.workunit.client.1.smithi053.stdout:6/630: dread d2/d1b/d1e/d2a/d4b/f68 [0,4194304] 0 2024-09-21T04:35:12.319 INFO:tasks.workunit.client.1.smithi053.stdout:7/520: chown d0/d2b/d36/f3c 1855684 1 2024-09-21T04:35:12.321 INFO:tasks.workunit.client.1.smithi053.stdout:7/521: fdatasync d0/d2b/d36/d48/d50/f54 0 2024-09-21T04:35:12.329 INFO:tasks.workunit.client.1.smithi053.stdout:0/596: mkdir d5/de/d9e/dd2 0 2024-09-21T04:35:12.329 INFO:tasks.workunit.client.0.smithi007.stdout:2/71: truncate d0/d2/f6 1905051 0 2024-09-21T04:35:12.330 INFO:tasks.workunit.client.1.smithi053.stdout:0/597: write d5/d10/d4c/db1/dbc/dca/d53/fbe [675745,16250] 0 2024-09-21T04:35:12.330 INFO:tasks.workunit.client.1.smithi053.stdout:2/657: symlink d1/d18/d1f/d27/d58/lf4 0 2024-09-21T04:35:12.331 INFO:tasks.workunit.client.1.smithi053.stdout:0/598: readlink d5/d10/d4c/l7d 0 2024-09-21T04:35:12.331 INFO:tasks.workunit.client.1.smithi053.stdout:2/658: read - d1/d18/d77/d9b/fed zero size 2024-09-21T04:35:12.332 INFO:tasks.workunit.client.1.smithi053.stdout:3/612: mknod d16/d17/d21/d26/d49/ce4 0 2024-09-21T04:35:12.332 INFO:tasks.workunit.client.0.smithi007.stdout:2/72: rmdir d0 39 2024-09-21T04:35:12.333 INFO:tasks.workunit.client.1.smithi053.stdout:3/613: stat d16/d17/d21/d8b/d7d/d57/cae 0 2024-09-21T04:35:12.333 INFO:tasks.workunit.client.1.smithi053.stdout:5/694: mknod d0/d66/ce3 0 2024-09-21T04:35:12.333 INFO:tasks.workunit.client.1.smithi053.stdout:6/631: rmdir d2/d1b/d3d/d49/de/d32 39 2024-09-21T04:35:12.334 INFO:tasks.workunit.client.1.smithi053.stdout:5/695: write d0/fa4 [794008,93111] 0 2024-09-21T04:35:12.334 INFO:tasks.workunit.client.1.smithi053.stdout:5/696: stat d0/dd/d48/f54 0 2024-09-21T04:35:12.336 INFO:tasks.workunit.client.1.smithi053.stdout:9/587: mkdir d0/d1c/dcb 0 2024-09-21T04:35:12.336 INFO:tasks.workunit.client.1.smithi053.stdout:6/632: truncate d2/d1d/f7e 659843 0 2024-09-21T04:35:12.338 INFO:tasks.workunit.client.1.smithi053.stdout:4/695: link d1/dd/c1b d1/d2c/d8e/ce7 0 2024-09-21T04:35:12.340 INFO:tasks.workunit.client.0.smithi007.stdout:2/73: creat d0/ff x:0 0 0 2024-09-21T04:35:12.341 INFO:tasks.workunit.client.1.smithi053.stdout:1/649: dwrite d7/d9/d43/f6d [0,4194304] 0 2024-09-21T04:35:12.341 INFO:tasks.workunit.client.0.smithi007.stdout:2/74: fdatasync d0/ff 0 2024-09-21T04:35:12.341 INFO:tasks.workunit.client.0.smithi007.stdout:5/197: dwrite d2/f8 [0,4194304] 0 2024-09-21T04:35:12.345 INFO:tasks.workunit.client.1.smithi053.stdout:9/588: dwrite d0/d2/d4e/fc6 [0,4194304] 0 2024-09-21T04:35:12.347 INFO:tasks.workunit.client.1.smithi053.stdout:8/669: rename d3/df/d14/d16/c9e to d3/cda 0 2024-09-21T04:35:12.354 INFO:tasks.workunit.client.0.smithi007.stdout:5/198: mkdir d2/d27/d42 0 2024-09-21T04:35:12.355 INFO:tasks.workunit.client.1.smithi053.stdout:0/599: write d5/dd/ff [4267530,2552] 0 2024-09-21T04:35:12.356 INFO:tasks.workunit.client.0.smithi007.stdout:2/75: mknod d0/c10 0 2024-09-21T04:35:12.359 INFO:tasks.workunit.client.0.smithi007.stdout:2/76: truncate d0/f7 57892 0 2024-09-21T04:35:12.361 INFO:tasks.workunit.client.1.smithi053.stdout:1/650: read - d7/da8/d74/d75/faa zero size 2024-09-21T04:35:12.361 INFO:tasks.workunit.client.1.smithi053.stdout:6/633: write d2/d1b/f6f [925645,130854] 0 2024-09-21T04:35:12.363 INFO:tasks.workunit.client.1.smithi053.stdout:1/651: write d7/d63/f6a [8637932,110283] 0 2024-09-21T04:35:12.366 INFO:tasks.workunit.client.1.smithi053.stdout:8/670: mkdir d3/d5/d4c/dc3/ddb 0 2024-09-21T04:35:12.367 INFO:tasks.workunit.client.1.smithi053.stdout:2/659: rename d1/d18/d1f/d22/d43/ld3 to d1/d18/d1f/d22/d2d/lf5 0 2024-09-21T04:35:12.369 INFO:tasks.workunit.client.1.smithi053.stdout:7/522: link d0/d2b/d36/d48/d50/d8b/ff d0/d2b/d36/d64/d9e/fad 0 2024-09-21T04:35:12.369 INFO:tasks.workunit.client.1.smithi053.stdout:9/589: dwrite d0/d2/f13 [0,4194304] 0 2024-09-21T04:35:12.369 INFO:tasks.workunit.client.1.smithi053.stdout:2/660: write d1/d18/d77/d9b/fed [728014,44683] 0 2024-09-21T04:35:12.372 INFO:tasks.workunit.client.0.smithi007.stdout:5/199: truncate d2/d1e/f22 3049467 0 2024-09-21T04:35:12.372 INFO:tasks.workunit.client.1.smithi053.stdout:9/590: truncate d0/d1a/f81 671066 0 2024-09-21T04:35:12.373 INFO:tasks.workunit.client.1.smithi053.stdout:4/696: mknod d1/ddb/ce8 0 2024-09-21T04:35:12.373 INFO:tasks.workunit.client.1.smithi053.stdout:7/523: fsync d0/d2b/d36/d48/f6b 0 2024-09-21T04:35:12.373 INFO:tasks.workunit.client.1.smithi053.stdout:4/697: readlink d1/d2c/ld2 0 2024-09-21T04:35:12.373 INFO:tasks.workunit.client.1.smithi053.stdout:2/661: stat d1/d18/d1f/d27/d41/d4b 0 2024-09-21T04:35:12.373 INFO:tasks.workunit.client.1.smithi053.stdout:0/600: mknod d5/de/d42/cd3 0 2024-09-21T04:35:12.374 INFO:tasks.workunit.client.0.smithi007.stdout:5/200: write d2/dd/d1d/f1f [1850492,113520] 0 2024-09-21T04:35:12.376 INFO:tasks.workunit.client.1.smithi053.stdout:7/524: write d0/d2b/d36/d48/d5d/f8c [389057,27574] 0 2024-09-21T04:35:12.381 INFO:tasks.workunit.client.1.smithi053.stdout:8/671: mkdir d3/d28/ddc 0 2024-09-21T04:35:12.381 INFO:tasks.workunit.client.0.smithi007.stdout:5/201: write d2/f1b [2865103,9348] 0 2024-09-21T04:35:12.385 INFO:tasks.workunit.client.0.smithi007.stdout:2/77: dwrite d0/d2/fc [0,4194304] 0 2024-09-21T04:35:12.386 INFO:tasks.workunit.client.1.smithi053.stdout:3/614: getdents d16/d17 0 2024-09-21T04:35:12.387 INFO:tasks.workunit.client.1.smithi053.stdout:0/601: mknod d5/de/d9e/cd4 0 2024-09-21T04:35:12.391 INFO:tasks.workunit.client.1.smithi053.stdout:2/662: creat d1/d18/d1f/d22/d54/ff6 x:0 0 0 2024-09-21T04:35:12.391 INFO:tasks.workunit.client.1.smithi053.stdout:2/663: stat d1/d18/d1f/fb8 0 2024-09-21T04:35:12.392 INFO:tasks.workunit.client.1.smithi053.stdout:7/525: fdatasync d0/f59 0 2024-09-21T04:35:12.394 INFO:tasks.workunit.client.0.smithi007.stdout:5/202: creat d2/f43 x:0 0 0 2024-09-21T04:35:12.395 INFO:tasks.workunit.client.1.smithi053.stdout:6/634: mkdir d2/d1b/d3d/d49/de/d32/d80/dcd 0 2024-09-21T04:35:12.396 INFO:tasks.workunit.client.0.smithi007.stdout:2/78: mkdir d0/d11 0 2024-09-21T04:35:12.397 INFO:tasks.workunit.client.1.smithi053.stdout:1/652: symlink d7/d47/dae/lda 0 2024-09-21T04:35:12.399 INFO:tasks.workunit.client.1.smithi053.stdout:2/664: dwrite d1/d18/d1f/d27/d71/fb1 [0,4194304] 0 2024-09-21T04:35:12.401 INFO:tasks.workunit.client.1.smithi053.stdout:3/615: symlink d16/d17/d21/d8b/d7d/d57/dc1/d53/le5 0 2024-09-21T04:35:12.405 INFO:tasks.workunit.client.1.smithi053.stdout:0/602: rename d5/de/d42/d5e to d5/d10/d4c/db1/dbc/dca/db5/dd5 0 2024-09-21T04:35:12.408 INFO:tasks.workunit.client.1.smithi053.stdout:7/526: creat d0/d2b/d36/fae x:0 0 0 2024-09-21T04:35:12.409 INFO:tasks.workunit.client.1.smithi053.stdout:3/616: dwrite d16/f3d [0,4194304] 0 2024-09-21T04:35:12.413 INFO:tasks.workunit.client.0.smithi007.stdout:5/203: truncate d2/fc 1363016 0 2024-09-21T04:35:12.413 INFO:tasks.workunit.client.1.smithi053.stdout:2/665: truncate d1/d18/d1f/d27/d41/f42 265787 0 2024-09-21T04:35:12.414 INFO:tasks.workunit.client.1.smithi053.stdout:2/666: chown d1/d18/d1f/d27/d41/d4b/d4d/cde 11045 1 2024-09-21T04:35:12.415 INFO:tasks.workunit.client.1.smithi053.stdout:2/667: chown d1/d18/d1f/d27/d41/d4b/d4d/c6b 5388892 1 2024-09-21T04:35:12.417 INFO:tasks.workunit.client.1.smithi053.stdout:0/603: creat d5/d10/d4c/db1/dbc/dca/db5/fd6 x:0 0 0 2024-09-21T04:35:12.419 INFO:tasks.workunit.client.0.smithi007.stdout:2/79: dwrite d0/f7 [0,4194304] 0 2024-09-21T04:35:12.419 INFO:tasks.workunit.client.1.smithi053.stdout:6/635: symlink d2/d1b/d3d/d49/de/d32/d80/d2e/lce 0 2024-09-21T04:35:12.420 INFO:tasks.workunit.client.1.smithi053.stdout:3/617: mknod d16/d17/d21/d8b/d7d/ce6 0 2024-09-21T04:35:12.421 INFO:tasks.workunit.client.1.smithi053.stdout:3/618: chown d16/d17/d21/d60/fd6 5583233 1 2024-09-21T04:35:12.422 INFO:tasks.workunit.client.0.smithi007.stdout:2/80: truncate d0/ff 649228 0 2024-09-21T04:35:12.422 INFO:tasks.workunit.client.1.smithi053.stdout:7/527: creat d0/d2b/d36/d48/d50/d8b/d70/faf x:0 0 0 2024-09-21T04:35:12.423 INFO:tasks.workunit.client.0.smithi007.stdout:2/81: chown d0/f8 423 1 2024-09-21T04:35:12.423 INFO:tasks.workunit.client.0.smithi007.stdout:5/204: dwrite d2/f1b [0,4194304] 0 2024-09-21T04:35:12.424 INFO:tasks.workunit.client.1.smithi053.stdout:4/698: getdents d1/d2c 0 2024-09-21T04:35:12.424 INFO:tasks.workunit.client.1.smithi053.stdout:7/528: write d0/d2b/d36/d48/d5c/f8a [568294,25126] 0 2024-09-21T04:35:12.427 INFO:tasks.workunit.client.1.smithi053.stdout:4/699: read d1/d27/d53/d88/d91/d3f/f72 [252031,22291] 0 2024-09-21T04:35:12.429 INFO:tasks.workunit.client.1.smithi053.stdout:7/529: write d0/d24/d46/f7d [3487087,83432] 0 2024-09-21T04:35:12.431 INFO:tasks.workunit.client.1.smithi053.stdout:9/591: getdents d0/d1c/d52/daa 0 2024-09-21T04:35:12.431 INFO:tasks.workunit.client.0.smithi007.stdout:5/205: mkdir d2/dd/d1d/d3a/d44 0 2024-09-21T04:35:12.431 INFO:tasks.workunit.client.1.smithi053.stdout:7/530: fsync d0/d24/f65 0 2024-09-21T04:35:12.433 INFO:tasks.workunit.client.0.smithi007.stdout:5/206: chown d2/dd/d1d/d3a/d44 0 1 2024-09-21T04:35:12.435 INFO:tasks.workunit.client.1.smithi053.stdout:0/604: write d5/d10/d4c/db1/dbc/dca/f65 [1731518,23082] 0 2024-09-21T04:35:12.436 INFO:tasks.workunit.client.1.smithi053.stdout:6/636: creat d2/d1b/d1e/d2a/d4b/d77/d88/da4/fcf x:0 0 0 2024-09-21T04:35:12.437 INFO:tasks.workunit.client.0.smithi007.stdout:2/82: creat d0/f12 x:0 0 0 2024-09-21T04:35:12.438 INFO:tasks.workunit.client.0.smithi007.stdout:2/83: write d0/f8 [814259,114676] 0 2024-09-21T04:35:12.440 INFO:tasks.workunit.client.1.smithi053.stdout:1/653: truncate d7/d9/d43/d94/fb9 167162 0 2024-09-21T04:35:12.440 INFO:tasks.workunit.client.1.smithi053.stdout:4/700: dwrite d1/d27/d53/d88/d91/d20/d73/d92/f94 [0,4194304] 0 2024-09-21T04:35:12.440 INFO:tasks.workunit.client.1.smithi053.stdout:0/605: fdatasync d5/d10/d4c/db1/dbc/dca/d21/d95/f9b 0 2024-09-21T04:35:12.442 INFO:tasks.workunit.client.0.smithi007.stdout:2/84: fdatasync d0/f8 0 2024-09-21T04:35:12.443 INFO:tasks.workunit.client.0.smithi007.stdout:3/30: sync 2024-09-21T04:35:12.443 INFO:tasks.workunit.client.0.smithi007.stdout:5/207: mknod d2/d27/d42/c45 0 2024-09-21T04:35:12.443 INFO:tasks.workunit.client.1.smithi053.stdout:3/619: dread d16/d1b/f2f [0,4194304] 0 2024-09-21T04:35:12.444 INFO:tasks.workunit.client.0.smithi007.stdout:5/208: stat d2/dd/d23/d2a 0 2024-09-21T04:35:12.445 INFO:tasks.workunit.client.0.smithi007.stdout:5/209: dread - d2/dd/d1d/f39 zero size 2024-09-21T04:35:12.449 INFO:tasks.workunit.client.0.smithi007.stdout:3/31: mkdir d9 0 2024-09-21T04:35:12.452 INFO:tasks.workunit.client.0.smithi007.stdout:5/210: creat d2/dd/d23/f46 x:0 0 0 2024-09-21T04:35:12.452 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:12 smithi053 ceph-mon[35554]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:12.453 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:12 smithi053 ceph-mon[35554]: pgmap v27: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 36 MiB/s rd, 172 MiB/s wr, 386 op/s 2024-09-21T04:35:12.453 INFO:tasks.workunit.client.1.smithi053.stdout:6/637: mknod d2/d1b/d3d/d49/de/d9c/cd0 0 2024-09-21T04:35:12.454 INFO:tasks.workunit.client.1.smithi053.stdout:7/531: dwrite d0/d2b/f2e [0,4194304] 0 2024-09-21T04:35:12.455 INFO:tasks.workunit.client.1.smithi053.stdout:7/532: dread - d0/d2b/d36/d48/d5c/f85 zero size 2024-09-21T04:35:12.456 INFO:tasks.workunit.client.1.smithi053.stdout:7/533: chown d0/d2b/d36/d48/d5d/d9f 15 1 2024-09-21T04:35:12.466 INFO:tasks.workunit.client.0.smithi007.stdout:5/211: truncate d2/fc 417104 0 2024-09-21T04:35:12.466 INFO:tasks.workunit.client.1.smithi053.stdout:2/668: link d1/d18/d1f/d27/d71/l72 d1/d18/lf7 0 2024-09-21T04:35:12.468 INFO:tasks.workunit.client.0.smithi007.stdout:3/32: rmdir d9 0 2024-09-21T04:35:12.468 INFO:tasks.workunit.client.0.smithi007.stdout:3/33: readlink l5 0 2024-09-21T04:35:12.468 INFO:tasks.workunit.client.0.smithi007.stdout:2/85: link d0/d2/cb d0/c13 0 2024-09-21T04:35:12.469 INFO:tasks.workunit.client.1.smithi053.stdout:3/620: dwrite f9 [4194304,4194304] 0 2024-09-21T04:35:12.471 INFO:tasks.workunit.client.0.smithi007.stdout:2/86: symlink d0/d2/l14 0 2024-09-21T04:35:12.474 INFO:tasks.workunit.client.1.smithi053.stdout:4/701: rename d1/l7 to d1/d2c/d8e/le9 0 2024-09-21T04:35:12.475 INFO:tasks.workunit.client.0.smithi007.stdout:5/212: stat d2/fc 0 2024-09-21T04:35:12.476 INFO:tasks.workunit.client.0.smithi007.stdout:3/34: dwrite f6 [0,4194304] 0 2024-09-21T04:35:12.476 INFO:tasks.workunit.client.0.smithi007.stdout:5/213: truncate d2/dd/f16 9175487 0 2024-09-21T04:35:12.477 INFO:tasks.workunit.client.0.smithi007.stdout:2/87: rename d0/d2/c9 to d0/d2/c15 0 2024-09-21T04:35:12.478 INFO:tasks.workunit.client.0.smithi007.stdout:3/35: write f8 [2487652,76182] 0 2024-09-21T04:35:12.478 INFO:tasks.workunit.client.0.smithi007.stdout:5/214: readlink d2/dd/d23/l36 0 2024-09-21T04:35:12.479 INFO:tasks.workunit.client.1.smithi053.stdout:9/592: creat d0/d2/d9/fcc x:0 0 0 2024-09-21T04:35:12.480 INFO:tasks.workunit.client.0.smithi007.stdout:5/215: read d2/dd/d23/d2a/f2d [97316,70205] 0 2024-09-21T04:35:12.480 INFO:tasks.workunit.client.1.smithi053.stdout:9/593: dread d0/d1a/f81 [0,4194304] 0 2024-09-21T04:35:12.481 INFO:tasks.workunit.client.0.smithi007.stdout:2/88: rmdir d0/d2 39 2024-09-21T04:35:12.484 INFO:tasks.workunit.client.0.smithi007.stdout:3/36: rename f6 to fa 0 2024-09-21T04:35:12.485 INFO:tasks.workunit.client.1.smithi053.stdout:3/621: mknod d16/d17/d21/d42/d43/ce7 0 2024-09-21T04:35:12.485 INFO:tasks.workunit.client.0.smithi007.stdout:5/216: fsync d2/dd/f14 0 2024-09-21T04:35:12.489 INFO:tasks.workunit.client.1.smithi053.stdout:1/654: link d7/d47/l92 d7/d47/ldb 0 2024-09-21T04:35:12.489 INFO:tasks.workunit.client.1.smithi053.stdout:9/594: dwrite d0/d28/f6b [0,4194304] 0 2024-09-21T04:35:12.490 INFO:tasks.workunit.client.0.smithi007.stdout:2/89: dwrite d0/f8 [0,4194304] 0 2024-09-21T04:35:12.492 INFO:tasks.workunit.client.1.smithi053.stdout:3/622: truncate d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fda 16727 0 2024-09-21T04:35:12.493 INFO:tasks.workunit.client.0.smithi007.stdout:7/141: sync 2024-09-21T04:35:12.493 INFO:tasks.workunit.client.0.smithi007.stdout:8/47: sync 2024-09-21T04:35:12.493 INFO:tasks.workunit.client.0.smithi007.stdout:6/32: sync 2024-09-21T04:35:12.493 INFO:tasks.workunit.client.0.smithi007.stdout:4/91: sync 2024-09-21T04:35:12.493 INFO:tasks.workunit.client.0.smithi007.stdout:0/57: sync 2024-09-21T04:35:12.494 INFO:tasks.workunit.client.0.smithi007.stdout:9/61: sync 2024-09-21T04:35:12.494 INFO:tasks.workunit.client.0.smithi007.stdout:1/131: sync 2024-09-21T04:35:12.494 INFO:tasks.workunit.client.1.smithi053.stdout:3/623: dread - d16/d17/d21/d26/d2b/fa2 zero size 2024-09-21T04:35:12.499 INFO:tasks.workunit.client.0.smithi007.stdout:0/58: write d6/f9 [3269505,782] 0 2024-09-21T04:35:12.501 INFO:tasks.workunit.client.0.smithi007.stdout:0/59: fdatasync d6/fd 0 2024-09-21T04:35:12.502 INFO:tasks.workunit.client.0.smithi007.stdout:5/217: rename d2/f8 to d2/dd/f47 0 2024-09-21T04:35:12.507 INFO:tasks.workunit.client.0.smithi007.stdout:7/142: dwrite d3/d6/d10/f24 [0,4194304] 0 2024-09-21T04:35:12.507 INFO:tasks.workunit.client.1.smithi053.stdout:6/638: creat d2/d1b/d3d/d49/de/fd1 x:0 0 0 2024-09-21T04:35:12.512 INFO:tasks.workunit.client.0.smithi007.stdout:9/62: symlink d13/l17 0 2024-09-21T04:35:12.513 INFO:tasks.workunit.client.0.smithi007.stdout:8/48: rmdir d0/d3/d8 39 2024-09-21T04:35:12.514 INFO:tasks.workunit.client.0.smithi007.stdout:3/37: truncate fa 1344196 0 2024-09-21T04:35:12.515 INFO:tasks.workunit.client.0.smithi007.stdout:6/33: write d3/f5 [669383,125893] 0 2024-09-21T04:35:12.519 INFO:tasks.workunit.client.0.smithi007.stdout:4/92: chown d2/d7/c15 19631 1 2024-09-21T04:35:12.519 INFO:tasks.workunit.client.1.smithi053.stdout:7/534: dwrite d0/d2b/d36/d48/d50/d8b/d70/f7f [0,4194304] 0 2024-09-21T04:35:12.527 INFO:tasks.workunit.client.0.smithi007.stdout:0/60: mkdir d6/d16 0 2024-09-21T04:35:12.527 INFO:tasks.workunit.client.0.smithi007.stdout:0/61: chown f4 212 1 2024-09-21T04:35:12.528 INFO:tasks.workunit.client.1.smithi053.stdout:9/595: mkdir d0/d1a/dcd 0 2024-09-21T04:35:12.529 INFO:tasks.workunit.client.0.smithi007.stdout:0/62: write d6/f14 [354638,100818] 0 2024-09-21T04:35:12.529 INFO:tasks.workunit.client.1.smithi053.stdout:2/669: getdents d1/d18/d1f/d27/d41/d4b/d4d/d89 0 2024-09-21T04:35:12.531 INFO:tasks.workunit.client.0.smithi007.stdout:7/143: symlink d3/d1a/l30 0 2024-09-21T04:35:12.532 INFO:tasks.workunit.client.1.smithi053.stdout:2/670: write d1/d18/d77/fc8 [274308,1088] 0 2024-09-21T04:35:12.532 INFO:tasks.workunit.client.1.smithi053.stdout:3/624: dwrite d16/d17/d21/d42/d43/d86/fb3 [0,4194304] 0 2024-09-21T04:35:12.534 INFO:tasks.workunit.client.1.smithi053.stdout:1/655: creat d7/d63/d96/dc7/fdc x:0 0 0 2024-09-21T04:35:12.534 INFO:tasks.workunit.client.1.smithi053.stdout:2/671: write d1/d18/d77/d83/f8d [1686738,115109] 0 2024-09-21T04:35:12.535 INFO:tasks.workunit.client.0.smithi007.stdout:1/132: unlink d5/d6/l11 0 2024-09-21T04:35:12.536 INFO:tasks.workunit.client.1.smithi053.stdout:1/656: chown d7/da8/d74/d91/fa2 759 1 2024-09-21T04:35:12.536 INFO:tasks.workunit.client.0.smithi007.stdout:9/63: mkdir d13/d18 0 2024-09-21T04:35:12.536 INFO:tasks.workunit.client.1.smithi053.stdout:7/535: write d0/d2b/d36/d48/d50/d8b/fa1 [110722,58782] 0 2024-09-21T04:35:12.537 INFO:tasks.workunit.client.0.smithi007.stdout:9/64: dread - f10 zero size 2024-09-21T04:35:12.539 INFO:tasks.workunit.client.1.smithi053.stdout:6/639: getdents d2/d1b/d3d/d49/de/d32/d80/dcd 0 2024-09-21T04:35:12.541 INFO:tasks.workunit.client.0.smithi007.stdout:9/65: write f6 [791070,2862] 0 2024-09-21T04:35:12.541 INFO:tasks.workunit.client.1.smithi053.stdout:4/702: getdents d1/ddb 0 2024-09-21T04:35:12.541 INFO:tasks.workunit.client.0.smithi007.stdout:7/144: dwrite d3/d6/f2f [0,4194304] 0 2024-09-21T04:35:12.542 INFO:tasks.workunit.client.0.smithi007.stdout:6/34: creat d3/f8 x:0 0 0 2024-09-21T04:35:12.542 INFO:tasks.workunit.client.1.smithi053.stdout:4/703: write d1/d27/d33/fac [2599454,13956] 0 2024-09-21T04:35:12.543 INFO:tasks.workunit.client.0.smithi007.stdout:3/38: dread fa [0,4194304] 0 2024-09-21T04:35:12.545 INFO:tasks.workunit.client.0.smithi007.stdout:4/93: rename d2/d7/f19 to d2/d7/f24 0 2024-09-21T04:35:12.546 INFO:tasks.workunit.client.0.smithi007.stdout:6/35: write d3/f6 [147890,43073] 0 2024-09-21T04:35:12.546 INFO:tasks.workunit.client.1.smithi053.stdout:3/625: symlink d16/d1b/dad/le8 0 2024-09-21T04:35:12.547 INFO:tasks.workunit.client.1.smithi053.stdout:3/626: stat d16/d17/d21/d8b/d7d/d57/d90 0 2024-09-21T04:35:12.548 INFO:tasks.workunit.client.0.smithi007.stdout:0/63: unlink d6/f9 0 2024-09-21T04:35:12.550 INFO:tasks.workunit.client.1.smithi053.stdout:7/536: readlink d0/d2b/d36/d48/d50/l76 0 2024-09-21T04:35:12.551 INFO:tasks.workunit.client.0.smithi007.stdout:1/133: dwrite d5/d6/f8 [0,4194304] 0 2024-09-21T04:35:12.551 INFO:tasks.workunit.client.0.smithi007.stdout:6/36: fdatasync d3/f5 0 2024-09-21T04:35:12.553 INFO:tasks.workunit.client.0.smithi007.stdout:1/134: write f1 [5027923,149] 0 2024-09-21T04:35:12.554 INFO:tasks.workunit.client.0.smithi007.stdout:8/49: unlink d0/d3/d8/fd 0 2024-09-21T04:35:12.557 INFO:tasks.workunit.client.0.smithi007.stdout:1/135: chown d5/d6/d21/d2b 0 1 2024-09-21T04:35:12.557 INFO:tasks.workunit.client.1.smithi053.stdout:9/596: creat d0/d1c/d52/d53/db1/fce x:0 0 0 2024-09-21T04:35:12.558 INFO:tasks.workunit.client.0.smithi007.stdout:1/136: fdatasync d5/fc 0 2024-09-21T04:35:12.559 INFO:tasks.workunit.client.1.smithi053.stdout:9/597: chown d0/d1c/d25/f9a 465 1 2024-09-21T04:35:12.560 INFO:tasks.workunit.client.0.smithi007.stdout:0/64: dwrite d6/fd [0,4194304] 0 2024-09-21T04:35:12.560 INFO:tasks.workunit.client.0.smithi007.stdout:3/39: creat fb x:0 0 0 2024-09-21T04:35:12.561 INFO:tasks.workunit.client.0.smithi007.stdout:0/65: dread - d6/f13 zero size 2024-09-21T04:35:12.562 INFO:tasks.workunit.client.0.smithi007.stdout:0/66: dread - d6/f13 zero size 2024-09-21T04:35:12.562 INFO:tasks.workunit.client.0.smithi007.stdout:4/94: dread d2/f6 [0,4194304] 0 2024-09-21T04:35:12.563 INFO:tasks.workunit.client.1.smithi053.stdout:2/672: link d1/d18/f49 d1/d18/d1f/da2/ff8 0 2024-09-21T04:35:12.564 INFO:tasks.workunit.client.0.smithi007.stdout:9/66: rename l9 to d13/d18/l19 0 2024-09-21T04:35:12.565 INFO:tasks.workunit.client.1.smithi053.stdout:5/697: sync 2024-09-21T04:35:12.565 INFO:tasks.workunit.client.0.smithi007.stdout:9/67: dread - f10 zero size 2024-09-21T04:35:12.566 INFO:tasks.workunit.client.1.smithi053.stdout:3/627: mknod d16/d17/d21/d8b/ce9 0 2024-09-21T04:35:12.567 INFO:tasks.workunit.client.1.smithi053.stdout:9/598: dwrite d0/d1c/d52/fb6 [0,4194304] 0 2024-09-21T04:35:12.569 INFO:tasks.workunit.client.1.smithi053.stdout:1/657: unlink d7/d9/d43/d54/d7f/l62 0 2024-09-21T04:35:12.570 INFO:tasks.workunit.client.1.smithi053.stdout:7/537: unlink d0/d2b/d36/f3d 0 2024-09-21T04:35:12.577 INFO:tasks.workunit.client.1.smithi053.stdout:9/599: dwrite d0/d76/fc9 [0,4194304] 0 2024-09-21T04:35:12.577 INFO:tasks.workunit.client.1.smithi053.stdout:2/673: creat d1/d18/d77/d83/ff9 x:0 0 0 2024-09-21T04:35:12.579 INFO:tasks.workunit.client.1.smithi053.stdout:1/658: mknod d7/d9/da/d3c/cdd 0 2024-09-21T04:35:12.580 INFO:tasks.workunit.client.0.smithi007.stdout:8/50: unlink d0/d3/d8/fb 0 2024-09-21T04:35:12.582 INFO:tasks.workunit.client.0.smithi007.stdout:1/137: creat d5/d6/d21/f2c x:0 0 0 2024-09-21T04:35:12.583 INFO:tasks.workunit.client.1.smithi053.stdout:7/538: chown d0/d2b/d36/d48/d50/d8b/f8 6 1 2024-09-21T04:35:12.583 INFO:tasks.workunit.client.1.smithi053.stdout:9/600: read d0/d1a/f2c [1816408,50263] 0 2024-09-21T04:35:12.583 INFO:tasks.workunit.client.1.smithi053.stdout:6/640: dwrite d2/d1b/d3d/d49/de/f85 [0,4194304] 0 2024-09-21T04:35:12.584 INFO:tasks.workunit.client.0.smithi007.stdout:3/40: mkdir dc 0 2024-09-21T04:35:12.584 INFO:tasks.workunit.client.1.smithi053.stdout:2/674: truncate d1/d18/d77/d9b/fed 854713 0 2024-09-21T04:35:12.584 INFO:tasks.workunit.client.1.smithi053.stdout:7/539: readlink d0/d24/l27 0 2024-09-21T04:35:12.587 INFO:tasks.workunit.client.1.smithi053.stdout:2/675: fsync d1/d18/d1f/d22/d2d/f87 0 2024-09-21T04:35:12.588 INFO:tasks.workunit.client.1.smithi053.stdout:5/698: mkdir d0/dd/de4 0 2024-09-21T04:35:12.590 INFO:tasks.workunit.client.1.smithi053.stdout:4/704: rename d1/dd/de/c4c to d1/cea 0 2024-09-21T04:35:12.591 INFO:tasks.workunit.client.0.smithi007.stdout:9/68: creat d13/d18/f1a x:0 0 0 2024-09-21T04:35:12.593 INFO:tasks.workunit.client.1.smithi053.stdout:1/659: creat d7/d47/d79/fde x:0 0 0 2024-09-21T04:35:12.595 INFO:tasks.workunit.client.0.smithi007.stdout:5/218: getdents d2/dd/d1d 0 2024-09-21T04:35:12.597 INFO:tasks.workunit.client.0.smithi007.stdout:7/145: symlink d3/d6/d2e/l31 0 2024-09-21T04:35:12.600 INFO:tasks.workunit.client.1.smithi053.stdout:7/540: truncate d0/f16 4712214 0 2024-09-21T04:35:12.600 INFO:tasks.workunit.client.0.smithi007.stdout:9/69: dwrite fe [0,4194304] 0 2024-09-21T04:35:12.602 INFO:tasks.workunit.client.0.smithi007.stdout:2/90: getdents d0/d2 0 2024-09-21T04:35:12.603 INFO:tasks.workunit.client.1.smithi053.stdout:2/676: creat d1/d18/d1f/d22/d2d/ffa x:0 0 0 2024-09-21T04:35:12.605 INFO:tasks.workunit.client.1.smithi053.stdout:5/699: mknod d0/dd/d5c/d89/dc5/ce5 0 2024-09-21T04:35:12.605 INFO:tasks.workunit.client.1.smithi053.stdout:6/641: dread d2/d1b/d1e/d2a/d4b/d77/d88/da4/fcc [0,4194304] 0 2024-09-21T04:35:12.608 INFO:tasks.workunit.client.1.smithi053.stdout:3/628: rename d16/d17/d21/d8b/cd9 to d16/d17/d21/d26/d2b/cea 0 2024-09-21T04:35:12.608 INFO:tasks.workunit.client.0.smithi007.stdout:1/138: read - d5/f12 zero size 2024-09-21T04:35:12.612 INFO:tasks.workunit.client.1.smithi053.stdout:0/606: sync 2024-09-21T04:35:12.612 INFO:tasks.workunit.client.1.smithi053.stdout:8/672: sync 2024-09-21T04:35:12.612 INFO:tasks.workunit.client.0.smithi007.stdout:0/67: rename d6/ce to d6/d16/c17 0 2024-09-21T04:35:12.613 INFO:tasks.workunit.client.1.smithi053.stdout:6/642: dwrite d2/d1b/d3d/d49/de/d32/d80/f4e [4194304,4194304] 0 2024-09-21T04:35:12.613 INFO:tasks.workunit.client.1.smithi053.stdout:8/673: chown d3/d28/fad 64893 1 2024-09-21T04:35:12.614 INFO:tasks.workunit.client.0.smithi007.stdout:5/219: chown d2/c9 55879803 1 2024-09-21T04:35:12.616 INFO:tasks.workunit.client.1.smithi053.stdout:7/541: symlink d0/d2b/d36/d48/d50/d8b/d70/lb0 0 2024-09-21T04:35:12.617 INFO:tasks.workunit.client.0.smithi007.stdout:8/51: symlink d0/d3/d8/de/lf 0 2024-09-21T04:35:12.618 INFO:tasks.workunit.client.1.smithi053.stdout:0/607: dwrite d5/f97 [0,4194304] 0 2024-09-21T04:35:12.619 INFO:tasks.workunit.client.0.smithi007.stdout:1/139: readlink d5/l1e 0 2024-09-21T04:35:12.622 INFO:tasks.workunit.client.0.smithi007.stdout:3/41: symlink dc/ld 0 2024-09-21T04:35:12.624 INFO:tasks.workunit.client.1.smithi053.stdout:4/705: symlink d1/d27/d53/d88/d91/d20/d4e/d51/da4/dc3/leb 0 2024-09-21T04:35:12.625 INFO:tasks.workunit.client.1.smithi053.stdout:0/608: dread d5/f39 [0,4194304] 0 2024-09-21T04:35:12.628 INFO:tasks.workunit.client.0.smithi007.stdout:4/95: rename d2/d7/f22 to d2/f25 0 2024-09-21T04:35:12.630 INFO:tasks.workunit.client.0.smithi007.stdout:0/68: mkdir d6/d16/d18 0 2024-09-21T04:35:12.630 INFO:tasks.workunit.client.0.smithi007.stdout:4/96: read d2/f12 [1450051,72924] 0 2024-09-21T04:35:12.632 INFO:tasks.workunit.client.0.smithi007.stdout:3/42: dwrite fb [0,4194304] 0 2024-09-21T04:35:12.633 INFO:tasks.workunit.client.1.smithi053.stdout:4/706: dwrite d1/d27/d53/d88/d91/d20/d5b/d87/fad [0,4194304] 0 2024-09-21T04:35:12.633 INFO:tasks.workunit.client.1.smithi053.stdout:4/707: dread - d1/dd/fb2 zero size 2024-09-21T04:35:12.634 INFO:tasks.workunit.client.1.smithi053.stdout:6/643: creat d2/d1b/d3d/d49/de/d32/fd2 x:0 0 0 2024-09-21T04:35:12.636 INFO:tasks.workunit.client.1.smithi053.stdout:9/601: truncate d0/d1a/d4c/d8f/f97 2688779 0 2024-09-21T04:35:12.637 INFO:tasks.workunit.client.1.smithi053.stdout:2/677: creat d1/d18/d1f/d22/d2d/dcd/dcf/ffb x:0 0 0 2024-09-21T04:35:12.638 INFO:tasks.workunit.client.1.smithi053.stdout:9/602: chown d0/d2/d4e 43089 1 2024-09-21T04:35:12.638 INFO:tasks.workunit.client.1.smithi053.stdout:6/644: chown d2/d1b/d3d/d49/f42 8 1 2024-09-21T04:35:12.640 INFO:tasks.workunit.client.1.smithi053.stdout:7/542: mkdir d0/d2b/d36/d48/d50/d84/db1 0 2024-09-21T04:35:12.645 INFO:tasks.workunit.client.1.smithi053.stdout:1/660: write d7/d63/f7d [322469,128362] 0 2024-09-21T04:35:12.648 INFO:tasks.workunit.client.0.smithi007.stdout:7/146: rename d3/d6/f8 to d3/d6/d14/f32 0 2024-09-21T04:35:12.648 INFO:tasks.workunit.client.1.smithi053.stdout:6/645: dwrite d2/d1b/fb5 [0,4194304] 0 2024-09-21T04:35:12.651 INFO:tasks.workunit.client.1.smithi053.stdout:6/646: write d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f9d [2708923,68311] 0 2024-09-21T04:35:12.653 INFO:tasks.workunit.client.1.smithi053.stdout:4/708: dread d1/d27/d53/d88/d91/d20/f6f [0,4194304] 0 2024-09-21T04:35:12.654 INFO:tasks.workunit.client.0.smithi007.stdout:2/91: truncate d0/f7 994752 0 2024-09-21T04:35:12.656 INFO:tasks.workunit.client.0.smithi007.stdout:2/92: write d0/d2/fc [4709984,65199] 0 2024-09-21T04:35:12.657 INFO:tasks.workunit.client.1.smithi053.stdout:8/674: mkdir d3/d5/d4b/ddd 0 2024-09-21T04:35:12.658 INFO:tasks.workunit.client.1.smithi053.stdout:2/678: dread d1/d18/f67 [0,4194304] 0 2024-09-21T04:35:12.662 INFO:tasks.workunit.client.1.smithi053.stdout:9/603: fdatasync d0/d2/d9/d1f/f4f 0 2024-09-21T04:35:12.662 INFO:tasks.workunit.client.1.smithi053.stdout:4/709: dwrite d1/dd/de/fc7 [0,4194304] 0 2024-09-21T04:35:12.663 INFO:tasks.workunit.client.1.smithi053.stdout:4/710: readlink d1/l5 0 2024-09-21T04:35:12.663 INFO:tasks.workunit.client.1.smithi053.stdout:9/604: write d0/d1c/d52/f93 [856777,18296] 0 2024-09-21T04:35:12.664 INFO:tasks.workunit.client.1.smithi053.stdout:4/711: chown d1/d2c/ld2 0 1 2024-09-21T04:35:12.665 INFO:tasks.workunit.client.1.smithi053.stdout:4/712: fsync d1/d27/d2e/fc1 0 2024-09-21T04:35:12.666 INFO:tasks.workunit.client.1.smithi053.stdout:9/605: chown d0/d2/d9/f7a 58891 1 2024-09-21T04:35:12.667 INFO:tasks.workunit.client.0.smithi007.stdout:5/220: truncate d2/f1b 2735031 0 2024-09-21T04:35:12.668 INFO:tasks.workunit.client.1.smithi053.stdout:4/713: chown d1/d27/d53/d88/d91/db6/cde 458084073 1 2024-09-21T04:35:12.671 INFO:tasks.workunit.client.0.smithi007.stdout:7/147: stat d3/l22 0 2024-09-21T04:35:12.672 INFO:tasks.workunit.client.1.smithi053.stdout:1/661: readlink d7/d63/d96/dac/lbb 0 2024-09-21T04:35:12.673 INFO:tasks.workunit.client.0.smithi007.stdout:9/70: rename lc to d13/l1b 0 2024-09-21T04:35:12.674 INFO:tasks.workunit.client.0.smithi007.stdout:9/71: chown d13 13335 1 2024-09-21T04:35:12.675 INFO:tasks.workunit.client.1.smithi053.stdout:4/714: dwrite d1/d27/d33/d41/fa3 [4194304,4194304] 0 2024-09-21T04:35:12.677 INFO:tasks.workunit.client.0.smithi007.stdout:7/148: dread d3/d6/f7 [0,4194304] 0 2024-09-21T04:35:12.678 INFO:tasks.workunit.client.0.smithi007.stdout:1/140: truncate d5/d6/f20 2709177 0 2024-09-21T04:35:12.680 INFO:tasks.workunit.client.0.smithi007.stdout:4/97: dwrite d2/d7/f10 [0,4194304] 0 2024-09-21T04:35:12.681 INFO:tasks.workunit.client.0.smithi007.stdout:2/93: creat d0/d2/f16 x:0 0 0 2024-09-21T04:35:12.683 INFO:tasks.workunit.client.1.smithi053.stdout:4/715: dwrite d1/d27/d53/fc2 [0,4194304] 0 2024-09-21T04:35:12.684 INFO:tasks.workunit.client.1.smithi053.stdout:4/716: chown d1/dd/d95/fa2 63502426 1 2024-09-21T04:35:12.685 INFO:tasks.workunit.client.1.smithi053.stdout:8/675: mkdir d3/df/d14/d16/dde 0 2024-09-21T04:35:12.692 INFO:tasks.workunit.client.0.smithi007.stdout:3/43: link fa dc/fe 0 2024-09-21T04:35:12.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:12 smithi007 ceph-mon[27954]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:12.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:12 smithi007 ceph-mon[27954]: pgmap v27: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 36 MiB/s rd, 172 MiB/s wr, 386 op/s 2024-09-21T04:35:12.695 INFO:tasks.workunit.client.1.smithi053.stdout:6/647: dread d2/d1b/f3a [0,4194304] 0 2024-09-21T04:35:12.701 INFO:tasks.workunit.client.1.smithi053.stdout:6/648: dwrite d2/d1b/d3d/f6c [0,4194304] 0 2024-09-21T04:35:12.704 INFO:tasks.workunit.client.1.smithi053.stdout:7/543: link d0/f4 d0/d6e/fb2 0 2024-09-21T04:35:12.706 INFO:tasks.workunit.client.1.smithi053.stdout:3/629: link d16/l32 d16/d17/d21/d8b/leb 0 2024-09-21T04:35:12.707 INFO:tasks.workunit.client.1.smithi053.stdout:6/649: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f9d [0,4194304] 0 2024-09-21T04:35:12.715 INFO:tasks.workunit.client.1.smithi053.stdout:0/609: creat d5/de/fd7 x:0 0 0 2024-09-21T04:35:12.716 INFO:tasks.workunit.client.1.smithi053.stdout:2/679: dwrite d1/d18/d1f/d27/d41/d4b/d4d/f52 [0,4194304] 0 2024-09-21T04:35:12.717 INFO:tasks.workunit.client.1.smithi053.stdout:2/680: chown d1/d18/l25 0 1 2024-09-21T04:35:12.718 INFO:tasks.workunit.client.1.smithi053.stdout:2/681: dread - d1/d18/d1f/da2/dac/fc5 zero size 2024-09-21T04:35:12.719 INFO:tasks.workunit.client.0.smithi007.stdout:5/221: creat d2/dd/d1d/d3a/d44/f48 x:0 0 0 2024-09-21T04:35:12.719 INFO:tasks.workunit.client.0.smithi007.stdout:4/98: dwrite d2/d7/fa [0,4194304] 0 2024-09-21T04:35:12.720 INFO:tasks.workunit.client.1.smithi053.stdout:2/682: fdatasync d1/d18/d1f/d9d/f90 0 2024-09-21T04:35:12.723 INFO:tasks.workunit.client.0.smithi007.stdout:4/99: fsync d2/d17/d18/f1e 0 2024-09-21T04:35:12.723 INFO:tasks.workunit.client.1.smithi053.stdout:2/683: truncate d1/d18/d1f/d22/d2d/f87 1701460 0 2024-09-21T04:35:12.724 INFO:tasks.workunit.client.0.smithi007.stdout:9/72: dwrite d13/f15 [0,4194304] 0 2024-09-21T04:35:12.724 INFO:tasks.workunit.client.1.smithi053.stdout:8/676: rename d3/d28/f79 to d3/d5/d38/d64/d9d/fdf 0 2024-09-21T04:35:12.729 INFO:tasks.workunit.client.0.smithi007.stdout:4/100: dread d2/d7/f10 [0,4194304] 0 2024-09-21T04:35:12.734 INFO:tasks.workunit.client.0.smithi007.stdout:2/94: truncate d0/f3 3627481 0 2024-09-21T04:35:12.735 INFO:tasks.workunit.client.1.smithi053.stdout:4/717: write d1/d27/d53/d88/d91/d20/f45 [1936500,44310] 0 2024-09-21T04:35:12.736 INFO:tasks.workunit.client.1.smithi053.stdout:4/718: fdatasync d1/d27/d2e/fc1 0 2024-09-21T04:35:12.736 INFO:tasks.workunit.client.0.smithi007.stdout:0/69: getdents d6/d16 0 2024-09-21T04:35:12.737 INFO:tasks.workunit.client.1.smithi053.stdout:6/650: creat d2/d1b/d1e/d27/d90/fd3 x:0 0 0 2024-09-21T04:35:12.739 INFO:tasks.workunit.client.1.smithi053.stdout:1/662: mknod d7/cdf 0 2024-09-21T04:35:12.739 INFO:tasks.workunit.client.0.smithi007.stdout:4/101: dwrite d2/f23 [0,4194304] 0 2024-09-21T04:35:12.740 INFO:tasks.workunit.client.1.smithi053.stdout:0/610: rmdir d5/d10/d4c/db1/dbc 39 2024-09-21T04:35:12.740 INFO:tasks.workunit.client.0.smithi007.stdout:4/102: fdatasync d2/d17/d18/f1f 0 2024-09-21T04:35:12.746 INFO:tasks.workunit.client.0.smithi007.stdout:7/149: truncate d3/d6/d14/f26 2121261 0 2024-09-21T04:35:12.754 INFO:tasks.workunit.client.1.smithi053.stdout:7/544: rename d0/d6e/f8d to d0/d24/fb3 0 2024-09-21T04:35:12.757 INFO:tasks.workunit.client.1.smithi053.stdout:2/684: rename d1/d18/d1f/d27/d41/d4b/d4d/d89/ddb to d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc 0 2024-09-21T04:35:12.760 INFO:tasks.workunit.client.1.smithi053.stdout:9/606: creat d0/fcf x:0 0 0 2024-09-21T04:35:12.762 INFO:tasks.workunit.client.0.smithi007.stdout:2/95: rename d0/d2/la to d0/d2/l17 0 2024-09-21T04:35:12.762 INFO:tasks.workunit.client.1.smithi053.stdout:0/611: dread d5/dd/ff [0,4194304] 0 2024-09-21T04:35:12.763 INFO:tasks.workunit.client.1.smithi053.stdout:3/630: mkdir d16/d17/d21/d8b/d7d/dc6/dec 0 2024-09-21T04:35:12.763 INFO:tasks.workunit.client.1.smithi053.stdout:0/612: chown d5/d47/l66 921 1 2024-09-21T04:35:12.763 INFO:tasks.workunit.client.0.smithi007.stdout:3/44: link dc/ld dc/lf 0 2024-09-21T04:35:12.765 INFO:tasks.workunit.client.0.smithi007.stdout:3/45: read fb [989020,122520] 0 2024-09-21T04:35:12.767 INFO:tasks.workunit.client.0.smithi007.stdout:5/222: dwrite d2/dd/f14 [0,4194304] 0 2024-09-21T04:35:12.769 INFO:tasks.workunit.client.1.smithi053.stdout:1/663: chown d7/c1c 3910895 1 2024-09-21T04:35:12.770 INFO:tasks.workunit.client.0.smithi007.stdout:4/103: creat d2/d17/f26 x:0 0 0 2024-09-21T04:35:12.776 INFO:tasks.workunit.client.0.smithi007.stdout:0/70: dwrite d6/f12 [0,4194304] 0 2024-09-21T04:35:12.778 INFO:tasks.workunit.client.0.smithi007.stdout:0/71: truncate d6/f13 383554 0 2024-09-21T04:35:12.780 INFO:tasks.workunit.client.1.smithi053.stdout:2/685: creat d1/d18/d1f/d27/d41/d4b/ddc/ffd x:0 0 0 2024-09-21T04:35:12.783 INFO:tasks.workunit.client.0.smithi007.stdout:9/73: truncate fe 330180 0 2024-09-21T04:35:12.785 INFO:tasks.workunit.client.1.smithi053.stdout:0/613: mkdir d5/de/d9e/dd8 0 2024-09-21T04:35:12.785 INFO:tasks.workunit.client.1.smithi053.stdout:7/545: write d0/d2b/d36/d48/d50/f96 [1111946,8459] 0 2024-09-21T04:35:12.786 INFO:tasks.workunit.client.1.smithi053.stdout:7/546: stat d0/f13 0 2024-09-21T04:35:12.788 INFO:tasks.workunit.client.0.smithi007.stdout:9/74: dread d13/f15 [0,4194304] 0 2024-09-21T04:35:12.789 INFO:tasks.workunit.client.1.smithi053.stdout:2/686: dread d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/ff0 [0,4194304] 0 2024-09-21T04:35:12.789 INFO:tasks.workunit.client.0.smithi007.stdout:3/46: creat dc/f10 x:0 0 0 2024-09-21T04:35:12.790 INFO:tasks.workunit.client.0.smithi007.stdout:3/47: dread - dc/f10 zero size 2024-09-21T04:35:12.791 INFO:tasks.workunit.client.0.smithi007.stdout:5/223: rename d2/dd/d1d/d3a to d2/dd/d23/d49 0 2024-09-21T04:35:12.791 INFO:tasks.workunit.client.0.smithi007.stdout:1/141: read d5/d6/f20 [352681,59353] 0 2024-09-21T04:35:12.792 INFO:tasks.workunit.client.0.smithi007.stdout:4/104: truncate d2/f6 1490641 0 2024-09-21T04:35:12.793 INFO:tasks.workunit.client.0.smithi007.stdout:1/142: write d5/f15 [646304,125552] 0 2024-09-21T04:35:12.800 INFO:tasks.workunit.client.0.smithi007.stdout:1/143: dwrite d5/f27 [0,4194304] 0 2024-09-21T04:35:12.808 INFO:tasks.workunit.client.0.smithi007.stdout:9/75: creat d13/d18/f1c x:0 0 0 2024-09-21T04:35:12.810 INFO:tasks.workunit.client.0.smithi007.stdout:3/48: mknod dc/c11 0 2024-09-21T04:35:12.810 INFO:tasks.workunit.client.0.smithi007.stdout:3/49: chown f8 124424 1 2024-09-21T04:35:12.812 INFO:tasks.workunit.client.0.smithi007.stdout:5/224: dread d2/fc [0,4194304] 0 2024-09-21T04:35:12.818 INFO:tasks.workunit.client.0.smithi007.stdout:2/96: link d0/d2/f16 d0/f18 0 2024-09-21T04:35:12.822 INFO:tasks.workunit.client.0.smithi007.stdout:2/97: dread d0/fd [0,4194304] 0 2024-09-21T04:35:12.823 INFO:tasks.workunit.client.0.smithi007.stdout:9/76: rename d13/d18/f1c to d13/f1d 0 2024-09-21T04:35:12.823 INFO:tasks.workunit.client.0.smithi007.stdout:9/77: stat f10 0 2024-09-21T04:35:12.824 INFO:tasks.workunit.client.0.smithi007.stdout:3/50: rmdir dc 39 2024-09-21T04:35:12.828 INFO:tasks.workunit.client.0.smithi007.stdout:0/72: link d6/d16/c17 d6/d16/d18/c19 0 2024-09-21T04:35:12.828 INFO:tasks.workunit.client.0.smithi007.stdout:3/51: dread f8 [0,4194304] 0 2024-09-21T04:35:12.837 INFO:tasks.workunit.client.0.smithi007.stdout:5/225: rename d2/dd/d23/f46 to d2/dd/d23/d49/f4a 0 2024-09-21T04:35:12.839 INFO:tasks.workunit.client.0.smithi007.stdout:1/144: dwrite d5/d6/f8 [0,4194304] 0 2024-09-21T04:35:12.840 INFO:tasks.workunit.client.0.smithi007.stdout:4/105: link f0 d2/f27 0 2024-09-21T04:35:12.841 INFO:tasks.workunit.client.0.smithi007.stdout:1/145: read d5/d6/fa [88401,71546] 0 2024-09-21T04:35:12.841 INFO:tasks.workunit.client.1.smithi053.stdout:8/677: creat d3/d5/d38/fe0 x:0 0 0 2024-09-21T04:35:12.842 INFO:tasks.workunit.client.0.smithi007.stdout:3/52: write fb [3582885,14616] 0 2024-09-21T04:35:12.845 INFO:tasks.workunit.client.1.smithi053.stdout:9/607: symlink d0/ld0 0 2024-09-21T04:35:12.850 INFO:tasks.workunit.client.0.smithi007.stdout:3/53: dwrite fb [0,4194304] 0 2024-09-21T04:35:12.853 INFO:tasks.workunit.client.0.smithi007.stdout:5/226: creat d2/dd/d1d/f4b x:0 0 0 2024-09-21T04:35:12.858 INFO:tasks.workunit.client.0.smithi007.stdout:4/106: write d2/d7/f10 [3324628,97478] 0 2024-09-21T04:35:12.859 INFO:tasks.workunit.client.0.smithi007.stdout:3/54: dwrite fb [4194304,4194304] 0 2024-09-21T04:35:12.860 INFO:tasks.workunit.client.0.smithi007.stdout:3/55: chown fb 8 1 2024-09-21T04:35:12.861 INFO:tasks.workunit.client.1.smithi053.stdout:8/678: dread - d3/df/d14/f60 zero size 2024-09-21T04:35:12.861 INFO:tasks.workunit.client.1.smithi053.stdout:8/679: readlink d3/l9a 0 2024-09-21T04:35:12.863 INFO:tasks.workunit.client.1.smithi053.stdout:9/608: symlink d0/d1a/d4c/d8f/ld1 0 2024-09-21T04:35:12.864 INFO:tasks.workunit.client.1.smithi053.stdout:3/631: truncate d16/d1b/dc3/f6c 1480800 0 2024-09-21T04:35:12.867 INFO:tasks.workunit.client.0.smithi007.stdout:9/78: write fe [100321,30239] 0 2024-09-21T04:35:12.869 INFO:tasks.workunit.client.1.smithi053.stdout:6/651: getdents d2/d1b/d1e/d2a/d4b/d77 0 2024-09-21T04:35:12.869 INFO:tasks.workunit.client.1.smithi053.stdout:3/632: dread d16/d17/d21/f39 [0,4194304] 0 2024-09-21T04:35:12.869 INFO:tasks.workunit.client.0.smithi007.stdout:2/98: creat d0/f19 x:0 0 0 2024-09-21T04:35:12.869 INFO:tasks.workunit.client.0.smithi007.stdout:2/99: dread - d0/d2/f16 zero size 2024-09-21T04:35:12.870 INFO:tasks.workunit.client.0.smithi007.stdout:2/100: stat d0/ff 0 2024-09-21T04:35:12.871 INFO:tasks.workunit.client.1.smithi053.stdout:7/547: rename d0/d2b/d36/c60 to d0/d2b/d36/d48/d5d/d9f/cb4 0 2024-09-21T04:35:12.872 INFO:tasks.workunit.client.1.smithi053.stdout:7/548: fsync d0/d2b/fa8 0 2024-09-21T04:35:12.872 INFO:tasks.workunit.client.0.smithi007.stdout:5/227: creat d2/dd/d28/f4c x:0 0 0 2024-09-21T04:35:12.874 INFO:tasks.workunit.client.1.smithi053.stdout:8/680: mknod d3/d5/d4b/d5c/d86/ce1 0 2024-09-21T04:35:12.876 INFO:tasks.workunit.client.1.smithi053.stdout:9/609: unlink d0/d6c/ca3 0 2024-09-21T04:35:12.879 INFO:tasks.workunit.client.1.smithi053.stdout:0/614: creat d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/fd9 x:0 0 0 2024-09-21T04:35:12.880 INFO:tasks.workunit.client.0.smithi007.stdout:1/146: creat d5/d6/d21/d2b/f2d x:0 0 0 2024-09-21T04:35:12.881 INFO:tasks.workunit.client.0.smithi007.stdout:1/147: fdatasync d5/f26 0 2024-09-21T04:35:12.883 INFO:tasks.workunit.client.1.smithi053.stdout:3/633: rename d16/d17/d21/d26/f92 to d16/d17/d21/d26/d3e/fed 0 2024-09-21T04:35:12.884 INFO:tasks.workunit.client.1.smithi053.stdout:3/634: readlink d16/d17/d21/d8b/leb 0 2024-09-21T04:35:12.884 INFO:tasks.workunit.client.0.smithi007.stdout:9/79: stat d13/d18/l19 0 2024-09-21T04:35:12.885 INFO:tasks.workunit.client.1.smithi053.stdout:7/549: rmdir d0/d2b/d36/d48/d50 39 2024-09-21T04:35:12.886 INFO:tasks.workunit.client.0.smithi007.stdout:9/80: chown d13/c14 0 1 2024-09-21T04:35:12.889 INFO:tasks.workunit.client.0.smithi007.stdout:1/148: dwrite d5/f15 [0,4194304] 0 2024-09-21T04:35:12.890 INFO:tasks.workunit.client.0.smithi007.stdout:1/149: chown d5/d6/fa 2221957 1 2024-09-21T04:35:12.890 INFO:tasks.workunit.client.0.smithi007.stdout:1/150: read - d5/f28 zero size 2024-09-21T04:35:12.895 INFO:tasks.workunit.client.1.smithi053.stdout:6/652: creat d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8/fd4 x:0 0 0 2024-09-21T04:35:12.897 INFO:tasks.workunit.client.0.smithi007.stdout:0/73: getdents d6/d16/d18 0 2024-09-21T04:35:12.900 INFO:tasks.workunit.client.1.smithi053.stdout:5/700: sync 2024-09-21T04:35:12.903 INFO:tasks.workunit.client.1.smithi053.stdout:0/615: dread d5/de/d1c/f29 [0,4194304] 0 2024-09-21T04:35:12.903 INFO:tasks.workunit.client.1.smithi053.stdout:7/550: creat d0/d2b/d36/d64/fb5 x:0 0 0 2024-09-21T04:35:12.904 INFO:tasks.workunit.client.0.smithi007.stdout:2/101: creat d0/d11/f1a x:0 0 0 2024-09-21T04:35:12.904 INFO:tasks.workunit.client.1.smithi053.stdout:1/664: getdents d7/d63/d96 0 2024-09-21T04:35:12.905 INFO:tasks.workunit.client.0.smithi007.stdout:9/81: dread f7 [0,4194304] 0 2024-09-21T04:35:12.906 INFO:tasks.workunit.client.0.smithi007.stdout:9/82: write d13/d18/f1a [14811,35816] 0 2024-09-21T04:35:12.909 INFO:tasks.workunit.client.0.smithi007.stdout:1/151: symlink d5/l2e 0 2024-09-21T04:35:12.909 INFO:tasks.workunit.client.0.smithi007.stdout:1/152: chown d5/l1e 18993 1 2024-09-21T04:35:12.910 INFO:tasks.workunit.client.1.smithi053.stdout:9/610: mknod d0/d1c/d52/d6d/cd2 0 2024-09-21T04:35:12.912 INFO:tasks.workunit.client.1.smithi053.stdout:6/653: mkdir d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/dd5 0 2024-09-21T04:35:12.912 INFO:tasks.workunit.client.0.smithi007.stdout:3/56: write dc/fe [1717168,14639] 0 2024-09-21T04:35:12.913 INFO:tasks.workunit.client.0.smithi007.stdout:3/57: dread - dc/f10 zero size 2024-09-21T04:35:12.913 INFO:tasks.workunit.client.1.smithi053.stdout:3/635: symlink d16/d17/ddc/lee 0 2024-09-21T04:35:12.915 INFO:tasks.workunit.client.1.smithi053.stdout:3/636: dread d16/d17/d21/d8b/d7d/d57/dc1/f70 [0,4194304] 0 2024-09-21T04:35:12.915 INFO:tasks.workunit.client.1.smithi053.stdout:0/616: fsync d5/f39 0 2024-09-21T04:35:12.915 INFO:tasks.workunit.client.1.smithi053.stdout:3/637: stat d16/d17/d21/d42/c5c 0 2024-09-21T04:35:12.917 INFO:tasks.workunit.client.0.smithi007.stdout:1/153: mkdir d5/d2f 0 2024-09-21T04:35:12.917 INFO:tasks.workunit.client.1.smithi053.stdout:7/551: symlink d0/d24/d46/lb6 0 2024-09-21T04:35:12.920 INFO:tasks.workunit.client.0.smithi007.stdout:4/107: getdents d2 0 2024-09-21T04:35:12.921 INFO:tasks.workunit.client.0.smithi007.stdout:4/108: chown d2/f12 119026496 1 2024-09-21T04:35:12.921 INFO:tasks.workunit.client.0.smithi007.stdout:9/83: dwrite d13/f15 [0,4194304] 0 2024-09-21T04:35:12.922 INFO:tasks.workunit.client.1.smithi053.stdout:0/617: dwrite d5/d10/d4c/db1/dbc/dca/f61 [0,4194304] 0 2024-09-21T04:35:12.922 INFO:tasks.workunit.client.0.smithi007.stdout:3/58: symlink dc/l12 0 2024-09-21T04:35:12.923 INFO:tasks.workunit.client.0.smithi007.stdout:4/109: truncate d2/d17/d18/f1f 149580 0 2024-09-21T04:35:12.923 INFO:tasks.workunit.client.1.smithi053.stdout:8/681: creat d3/d5/d21/fe2 x:0 0 0 2024-09-21T04:35:12.925 INFO:tasks.workunit.client.0.smithi007.stdout:5/228: getdents d2/dd/d23 0 2024-09-21T04:35:12.928 INFO:tasks.workunit.client.0.smithi007.stdout:5/229: fdatasync d2/dd/d1d/f4b 0 2024-09-21T04:35:12.930 INFO:tasks.workunit.client.1.smithi053.stdout:9/611: rename d0/d2/d9/l85 to d0/d1c/d52/d6d/ld3 0 2024-09-21T04:35:12.933 INFO:tasks.workunit.client.1.smithi053.stdout:6/654: creat d2/d1b/d3d/fd6 x:0 0 0 2024-09-21T04:35:12.934 INFO:tasks.workunit.client.0.smithi007.stdout:3/59: dwrite dc/fe [0,4194304] 0 2024-09-21T04:35:12.939 INFO:tasks.workunit.client.0.smithi007.stdout:9/84: link f10 d13/f1e 0 2024-09-21T04:35:12.941 INFO:tasks.workunit.client.0.smithi007.stdout:4/110: unlink f0 0 2024-09-21T04:35:12.941 INFO:tasks.workunit.client.0.smithi007.stdout:5/230: dwrite d2/dd/d23/d49/d44/f48 [0,4194304] 0 2024-09-21T04:35:12.942 INFO:tasks.workunit.client.1.smithi053.stdout:7/552: creat d0/d2b/d36/d64/fb7 x:0 0 0 2024-09-21T04:35:12.942 INFO:tasks.workunit.client.0.smithi007.stdout:4/111: fdatasync d2/f23 0 2024-09-21T04:35:12.944 INFO:tasks.workunit.client.0.smithi007.stdout:4/112: dread - d2/d17/f26 zero size 2024-09-21T04:35:12.944 INFO:tasks.workunit.client.0.smithi007.stdout:1/154: link d5/f1a d5/d6/d21/f30 0 2024-09-21T04:35:12.945 INFO:tasks.workunit.client.1.smithi053.stdout:1/665: readlink d7/d47/d79/lc6 0 2024-09-21T04:35:12.946 INFO:tasks.workunit.client.1.smithi053.stdout:1/666: dread - d7/d47/d79/fde zero size 2024-09-21T04:35:12.947 INFO:tasks.workunit.client.0.smithi007.stdout:9/85: dread f11 [0,4194304] 0 2024-09-21T04:35:12.949 INFO:tasks.workunit.client.1.smithi053.stdout:0/618: unlink l2 0 2024-09-21T04:35:12.952 INFO:tasks.workunit.client.0.smithi007.stdout:9/86: dread f6 [0,4194304] 0 2024-09-21T04:35:12.954 INFO:tasks.workunit.client.1.smithi053.stdout:9/612: mknod d0/d1c/d52/cd4 0 2024-09-21T04:35:12.955 INFO:tasks.workunit.client.1.smithi053.stdout:9/613: write d0/d28/f9b [129997,46893] 0 2024-09-21T04:35:12.957 INFO:tasks.workunit.client.1.smithi053.stdout:6/655: symlink d2/d1b/ld7 0 2024-09-21T04:35:12.959 INFO:tasks.workunit.client.0.smithi007.stdout:1/155: rename d5/f12 to d5/d6/f31 0 2024-09-21T04:35:12.960 INFO:tasks.workunit.client.1.smithi053.stdout:9/614: dwrite d0/d2/f13 [0,4194304] 0 2024-09-21T04:35:12.966 INFO:tasks.workunit.client.1.smithi053.stdout:2/687: sync 2024-09-21T04:35:12.966 INFO:tasks.workunit.client.1.smithi053.stdout:4/719: sync 2024-09-21T04:35:12.967 INFO:tasks.workunit.client.1.smithi053.stdout:2/688: write d1/d18/d1f/fba [3555728,125199] 0 2024-09-21T04:35:12.967 INFO:tasks.workunit.client.1.smithi053.stdout:3/638: mknod d16/d1b/dc3/cef 0 2024-09-21T04:35:12.970 INFO:tasks.workunit.client.0.smithi007.stdout:9/87: creat d13/d18/f1f x:0 0 0 2024-09-21T04:35:12.972 INFO:tasks.workunit.client.1.smithi053.stdout:7/553: read d0/d2b/d36/d48/d50/f96 [663639,79226] 0 2024-09-21T04:35:12.973 INFO:tasks.workunit.client.1.smithi053.stdout:1/667: symlink d7/d47/d79/dc1/le0 0 2024-09-21T04:35:12.974 INFO:tasks.workunit.client.1.smithi053.stdout:1/668: chown d7/d47/d79/dc1/fc9 2833555 1 2024-09-21T04:35:12.974 INFO:tasks.workunit.client.0.smithi007.stdout:4/113: write d2/d7/f8 [950969,65619] 0 2024-09-21T04:35:12.975 INFO:tasks.workunit.client.1.smithi053.stdout:0/619: mkdir d5/d10/d4c/db1/dbc/dca/db5/dda 0 2024-09-21T04:35:12.976 INFO:tasks.workunit.client.1.smithi053.stdout:0/620: chown d5/dd/d50/fc7 6697 1 2024-09-21T04:35:12.976 INFO:tasks.workunit.client.1.smithi053.stdout:0/621: fdatasync d5/dd/f1a 0 2024-09-21T04:35:12.977 INFO:tasks.workunit.client.0.smithi007.stdout:9/88: dwrite fa [0,4194304] 0 2024-09-21T04:35:12.978 INFO:tasks.workunit.client.1.smithi053.stdout:8/682: unlink d3/d5/d4c/dc3/ld4 0 2024-09-21T04:35:12.978 INFO:tasks.workunit.client.0.smithi007.stdout:9/89: chown l3 274 1 2024-09-21T04:35:12.981 INFO:tasks.workunit.client.0.smithi007.stdout:1/156: dread d5/f17 [0,4194304] 0 2024-09-21T04:35:12.983 INFO:tasks.workunit.client.1.smithi053.stdout:5/701: getdents d0/dd/d5c/d92/ddb/d35 0 2024-09-21T04:35:12.984 INFO:tasks.workunit.client.0.smithi007.stdout:1/157: dread d5/f27 [0,4194304] 0 2024-09-21T04:35:12.985 INFO:tasks.workunit.client.1.smithi053.stdout:5/702: fsync d0/dd/d5c/d92/ddb/d51/d52/f59 0 2024-09-21T04:35:12.986 INFO:tasks.workunit.client.0.smithi007.stdout:9/90: write f11 [1286061,81560] 0 2024-09-21T04:35:12.986 INFO:tasks.workunit.client.0.smithi007.stdout:1/158: truncate d5/d6/d21/d23/f2a 496042 0 2024-09-21T04:35:12.990 INFO:tasks.workunit.client.1.smithi053.stdout:9/615: creat d0/d1c/d52/daa/fd5 x:0 0 0 2024-09-21T04:35:12.991 INFO:tasks.workunit.client.1.smithi053.stdout:9/616: write d0/d2/d9/d34/f59 [5105597,8404] 0 2024-09-21T04:35:12.991 INFO:tasks.workunit.client.0.smithi007.stdout:5/231: getdents d2/d27 0 2024-09-21T04:35:12.993 INFO:tasks.workunit.client.0.smithi007.stdout:5/232: readlink d2/dd/l10 0 2024-09-21T04:35:12.996 INFO:tasks.workunit.client.1.smithi053.stdout:4/720: mkdir d1/d27/d33/dec 0 2024-09-21T04:35:12.996 INFO:tasks.workunit.client.0.smithi007.stdout:5/233: stat d2/dd/d23/d49/f4a 0 2024-09-21T04:35:12.996 INFO:tasks.workunit.client.0.smithi007.stdout:1/159: dwrite d5/f26 [0,4194304] 0 2024-09-21T04:35:12.997 INFO:tasks.workunit.client.0.smithi007.stdout:5/234: read d2/dd/d23/d2a/f2d [1071471,26235] 0 2024-09-21T04:35:12.998 INFO:tasks.workunit.client.0.smithi007.stdout:9/91: fsync f6 0 2024-09-21T04:35:12.998 INFO:tasks.workunit.client.0.smithi007.stdout:9/92: stat d13/f1e 0 2024-09-21T04:35:12.999 INFO:tasks.workunit.client.0.smithi007.stdout:9/93: dread - d13/f1d zero size 2024-09-21T04:35:13.000 INFO:tasks.workunit.client.0.smithi007.stdout:9/94: chown f11 580641 1 2024-09-21T04:35:13.006 INFO:tasks.workunit.client.0.smithi007.stdout:1/160: mknod d5/d6/d21/d23/c32 0 2024-09-21T04:35:13.007 INFO:tasks.workunit.client.0.smithi007.stdout:5/235: dwrite d2/dd/d1d/f4b [0,4194304] 0 2024-09-21T04:35:13.007 INFO:tasks.workunit.client.1.smithi053.stdout:7/554: fsync d0/d2b/d36/d48/d50/d84/f98 0 2024-09-21T04:35:13.008 INFO:tasks.workunit.client.1.smithi053.stdout:7/555: readlink d0/d2b/d36/d64/l9a 0 2024-09-21T04:35:13.009 INFO:tasks.workunit.client.1.smithi053.stdout:7/556: chown d0/d24/f4d 48671 1 2024-09-21T04:35:13.010 INFO:tasks.workunit.client.1.smithi053.stdout:7/557: chown d0/d2b/d36/d48/d50/d8b/c2f 6592 1 2024-09-21T04:35:13.016 INFO:tasks.workunit.client.0.smithi007.stdout:4/114: read d2/f23 [2003587,14756] 0 2024-09-21T04:35:13.016 INFO:tasks.workunit.client.0.smithi007.stdout:1/161: rmdir d5 39 2024-09-21T04:35:13.017 INFO:tasks.workunit.client.1.smithi053.stdout:6/656: write d2/d1b/d3d/d49/de/d32/d80/d2e/f79 [302303,111643] 0 2024-09-21T04:35:13.017 INFO:tasks.workunit.client.0.smithi007.stdout:5/236: creat d2/dd/d28/f4d x:0 0 0 2024-09-21T04:35:13.020 INFO:tasks.workunit.client.1.smithi053.stdout:5/703: mkdir d0/dd/d5c/d92/de6 0 2024-09-21T04:35:13.020 INFO:tasks.workunit.client.1.smithi053.stdout:8/683: dread d3/fa [0,4194304] 0 2024-09-21T04:35:13.021 INFO:tasks.workunit.client.1.smithi053.stdout:8/684: chown d3/d5/l84 357234 1 2024-09-21T04:35:13.021 INFO:tasks.workunit.client.1.smithi053.stdout:8/685: readlink d3/d5/d38/l6c 0 2024-09-21T04:35:13.023 INFO:tasks.workunit.client.1.smithi053.stdout:9/617: creat d0/d28/fd6 x:0 0 0 2024-09-21T04:35:13.034 INFO:tasks.workunit.client.0.smithi007.stdout:9/95: dwrite f10 [0,4194304] 0 2024-09-21T04:35:13.034 INFO:tasks.workunit.client.0.smithi007.stdout:5/237: dwrite d2/dd/f14 [0,4194304] 0 2024-09-21T04:35:13.035 INFO:tasks.workunit.client.1.smithi053.stdout:7/558: mkdir d0/d2b/d36/d48/d50/d84/db8 0 2024-09-21T04:35:13.036 INFO:tasks.workunit.client.1.smithi053.stdout:0/622: getdents d5/d10/d4c/db1/dbc/d8c 0 2024-09-21T04:35:13.036 INFO:tasks.workunit.client.0.smithi007.stdout:4/115: getdents d2 0 2024-09-21T04:35:13.037 INFO:tasks.workunit.client.0.smithi007.stdout:9/96: creat d13/f20 x:0 0 0 2024-09-21T04:35:13.039 INFO:tasks.workunit.client.0.smithi007.stdout:5/238: creat d2/d27/d3d/f4e x:0 0 0 2024-09-21T04:35:13.041 INFO:tasks.workunit.client.0.smithi007.stdout:5/239: write d2/dd/f16 [738587,59642] 0 2024-09-21T04:35:13.043 INFO:tasks.workunit.client.1.smithi053.stdout:5/704: write d0/dd/fcd [608794,27492] 0 2024-09-21T04:35:13.045 INFO:tasks.workunit.client.0.smithi007.stdout:4/116: read d2/d7/f24 [223698,119352] 0 2024-09-21T04:35:13.046 INFO:tasks.workunit.client.0.smithi007.stdout:9/97: creat d13/f21 x:0 0 0 2024-09-21T04:35:13.047 INFO:tasks.workunit.client.1.smithi053.stdout:4/721: creat d1/d27/d53/d88/d91/d3f/d74/de1/fed x:0 0 0 2024-09-21T04:35:13.048 INFO:tasks.workunit.client.1.smithi053.stdout:4/722: write d1/d27/d53/d88/fb9 [212208,85689] 0 2024-09-21T04:35:13.049 INFO:tasks.workunit.client.0.smithi007.stdout:5/240: dread d2/dd/d1d/f4b [0,4194304] 0 2024-09-21T04:35:13.050 INFO:tasks.workunit.client.1.smithi053.stdout:3/639: rename d16/d17/d21/d60/fcc to d16/d17/d21/d42/ff0 0 2024-09-21T04:35:13.050 INFO:tasks.workunit.client.0.smithi007.stdout:1/162: dwrite d5/d6/fa [0,4194304] 0 2024-09-21T04:35:13.051 INFO:tasks.workunit.client.1.smithi053.stdout:3/640: write d16/f3d [3185508,44607] 0 2024-09-21T04:35:13.051 INFO:tasks.workunit.client.0.smithi007.stdout:5/241: read d2/dd/d1d/f1f [1939949,33210] 0 2024-09-21T04:35:13.052 INFO:tasks.workunit.client.0.smithi007.stdout:1/163: dread - d5/d6/d21/d23/f24 zero size 2024-09-21T04:35:13.053 INFO:tasks.workunit.client.1.smithi053.stdout:3/641: write d16/d17/d21/d8b/d7d/d57/dc1/d53/fa1 [4107660,80415] 0 2024-09-21T04:35:13.054 INFO:tasks.workunit.client.1.smithi053.stdout:3/642: read d16/d17/faf [1193049,116739] 0 2024-09-21T04:35:13.054 INFO:tasks.workunit.client.0.smithi007.stdout:4/117: mknod d2/d17/d18/c28 0 2024-09-21T04:35:13.059 INFO:tasks.workunit.client.0.smithi007.stdout:9/98: symlink d13/d18/l22 0 2024-09-21T04:35:13.061 INFO:tasks.workunit.client.1.smithi053.stdout:4/723: dread d1/d27/f55 [4194304,4194304] 0 2024-09-21T04:35:13.064 INFO:tasks.workunit.client.0.smithi007.stdout:5/242: mknod d2/dd/d1d/d20/c4f 0 2024-09-21T04:35:13.066 INFO:tasks.workunit.client.0.smithi007.stdout:4/118: mkdir d2/d17/d29 0 2024-09-21T04:35:13.070 INFO:tasks.workunit.client.1.smithi053.stdout:2/689: getdents d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9 0 2024-09-21T04:35:13.073 INFO:tasks.workunit.client.1.smithi053.stdout:1/669: dread d7/d9/d43/d54/d7f/d59/f9d [0,4194304] 0 2024-09-21T04:35:13.073 INFO:tasks.workunit.client.0.smithi007.stdout:5/243: write d2/dd/d1d/f1f [4162053,109491] 0 2024-09-21T04:35:13.075 INFO:tasks.workunit.client.0.smithi007.stdout:4/119: creat d2/d17/f2a x:0 0 0 2024-09-21T04:35:13.076 INFO:tasks.workunit.client.1.smithi053.stdout:3/643: mkdir d16/d17/d21/d26/d71/df1 0 2024-09-21T04:35:13.077 INFO:tasks.workunit.client.0.smithi007.stdout:1/164: dwrite d5/f27 [0,4194304] 0 2024-09-21T04:35:13.078 INFO:tasks.workunit.client.1.smithi053.stdout:3/644: write d16/d17/d21/d42/d43/d86/fc7 [1022048,106755] 0 2024-09-21T04:35:13.080 INFO:tasks.workunit.client.1.smithi053.stdout:0/623: symlink d5/dd/ldb 0 2024-09-21T04:35:13.086 INFO:tasks.workunit.client.0.smithi007.stdout:4/120: rename d2/d17/f20 to d2/d17/d18/f2b 0 2024-09-21T04:35:13.087 INFO:tasks.workunit.client.0.smithi007.stdout:4/121: chown d2/d7/fa 129899 1 2024-09-21T04:35:13.087 INFO:tasks.workunit.client.0.smithi007.stdout:4/122: chown d2/d17/d18/c28 34 1 2024-09-21T04:35:13.088 INFO:tasks.workunit.client.0.smithi007.stdout:1/165: mknod d5/c33 0 2024-09-21T04:35:13.088 INFO:tasks.workunit.client.0.smithi007.stdout:9/99: truncate f11 507062 0 2024-09-21T04:35:13.089 INFO:tasks.workunit.client.0.smithi007.stdout:9/100: chown d13/c14 11642582 1 2024-09-21T04:35:13.090 INFO:tasks.workunit.client.1.smithi053.stdout:3/645: dread d16/d1b/f59 [0,4194304] 0 2024-09-21T04:35:13.090 INFO:tasks.workunit.client.1.smithi053.stdout:4/724: unlink d1/d2c/c5c 0 2024-09-21T04:35:13.091 INFO:tasks.workunit.client.0.smithi007.stdout:9/101: write d13/f1e [2149653,5141] 0 2024-09-21T04:35:13.091 INFO:tasks.workunit.client.1.smithi053.stdout:8/686: link d3/d5/d38/d64/f67 d3/d5/d38/d64/d9d/da9/fe3 0 2024-09-21T04:35:13.092 INFO:tasks.workunit.client.1.smithi053.stdout:8/687: chown d3/d39/f9f 83 1 2024-09-21T04:35:13.093 INFO:tasks.workunit.client.0.smithi007.stdout:4/123: mknod d2/d17/c2c 0 2024-09-21T04:35:13.094 INFO:tasks.workunit.client.0.smithi007.stdout:4/124: fsync d2/d17/d18/f1f 0 2024-09-21T04:35:13.094 INFO:tasks.workunit.client.1.smithi053.stdout:8/688: truncate d3/d5/d4c/d53/f83 1136524 0 2024-09-21T04:35:13.094 INFO:tasks.workunit.client.1.smithi053.stdout:2/690: unlink d1/d18/c44 0 2024-09-21T04:35:13.097 INFO:tasks.workunit.client.0.smithi007.stdout:5/244: creat d2/f50 x:0 0 0 2024-09-21T04:35:13.098 INFO:tasks.workunit.client.1.smithi053.stdout:0/624: dread d5/de/d42/d78/f6f [4194304,4194304] 0 2024-09-21T04:35:13.099 INFO:tasks.workunit.client.0.smithi007.stdout:9/102: rmdir d13 39 2024-09-21T04:35:13.099 INFO:tasks.workunit.client.1.smithi053.stdout:0/625: write d5/de/fb6 [941840,85441] 0 2024-09-21T04:35:13.106 INFO:tasks.workunit.client.1.smithi053.stdout:6/657: getdents d2/d1b/d3d/d49/de/d32/d80 0 2024-09-21T04:35:13.106 INFO:tasks.workunit.client.1.smithi053.stdout:6/658: write d2/d1b/d3d/d49/de/fd1 [706215,81144] 0 2024-09-21T04:35:13.107 INFO:tasks.workunit.client.0.smithi007.stdout:4/125: creat d2/d17/d18/f2d x:0 0 0 2024-09-21T04:35:13.108 INFO:tasks.workunit.client.1.smithi053.stdout:5/705: creat d0/dd/d48/fe7 x:0 0 0 2024-09-21T04:35:13.109 INFO:tasks.workunit.client.0.smithi007.stdout:1/166: truncate d5/fc 1533229 0 2024-09-21T04:35:13.111 INFO:tasks.workunit.client.0.smithi007.stdout:5/245: rmdir d2/d27/d42 39 2024-09-21T04:35:13.112 INFO:tasks.workunit.client.1.smithi053.stdout:4/725: mkdir d1/d27/d53/d88/d91/d20/dee 0 2024-09-21T04:35:13.113 INFO:tasks.workunit.client.0.smithi007.stdout:1/167: dread d5/d6/fa [0,4194304] 0 2024-09-21T04:35:13.115 INFO:tasks.workunit.client.0.smithi007.stdout:4/126: write d2/f25 [948344,90678] 0 2024-09-21T04:35:13.116 INFO:tasks.workunit.client.1.smithi053.stdout:7/559: getdents d0/d2b/d36/d48/d95 0 2024-09-21T04:35:13.118 INFO:tasks.workunit.client.1.smithi053.stdout:2/691: symlink d1/d18/d1f/d22/d54/lfe 0 2024-09-21T04:35:13.119 INFO:tasks.workunit.client.1.smithi053.stdout:3/646: dread d16/d1b/dad/fb1 [0,4194304] 0 2024-09-21T04:35:13.120 INFO:tasks.workunit.client.0.smithi007.stdout:4/127: dread d2/d17/d18/f1e [0,4194304] 0 2024-09-21T04:35:13.120 INFO:tasks.workunit.client.1.smithi053.stdout:3/647: chown d16/d1b/dad/lcb 6672 1 2024-09-21T04:35:13.120 INFO:tasks.workunit.client.1.smithi053.stdout:1/670: creat d7/d24/fe1 x:0 0 0 2024-09-21T04:35:13.122 INFO:tasks.workunit.client.0.smithi007.stdout:9/103: rename d13/f16 to d13/f23 0 2024-09-21T04:35:13.124 INFO:tasks.workunit.client.1.smithi053.stdout:8/689: write d3/d5/d38/d64/f67 [13835,34428] 0 2024-09-21T04:35:13.125 INFO:tasks.workunit.client.1.smithi053.stdout:8/690: fdatasync d3/df/fb2 0 2024-09-21T04:35:13.127 INFO:tasks.workunit.client.1.smithi053.stdout:6/659: unlink d2/d1b/d3d/d49/f8f 0 2024-09-21T04:35:13.129 INFO:tasks.workunit.client.1.smithi053.stdout:6/660: truncate d2/d1b/d3d/d49/de/d9c/fbd 826543 0 2024-09-21T04:35:13.130 INFO:tasks.workunit.client.1.smithi053.stdout:4/726: symlink d1/dd/de/lef 0 2024-09-21T04:35:13.131 INFO:tasks.workunit.client.1.smithi053.stdout:4/727: write d1/d27/d33/fac [1167699,89705] 0 2024-09-21T04:35:13.131 INFO:tasks.workunit.client.1.smithi053.stdout:7/560: unlink d0/d2b/d36/fae 0 2024-09-21T04:35:13.133 INFO:tasks.workunit.client.0.smithi007.stdout:9/104: creat d13/d18/f24 x:0 0 0 2024-09-21T04:35:13.134 INFO:tasks.workunit.client.0.smithi007.stdout:9/105: dread - d13/d18/f24 zero size 2024-09-21T04:35:13.134 INFO:tasks.workunit.client.1.smithi053.stdout:2/692: creat d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/fff x:0 0 0 2024-09-21T04:35:13.136 INFO:tasks.workunit.client.0.smithi007.stdout:1/168: creat d5/d2f/f34 x:0 0 0 2024-09-21T04:35:13.137 INFO:tasks.workunit.client.1.smithi053.stdout:3/648: creat d16/d17/d21/d8b/ff2 x:0 0 0 2024-09-21T04:35:13.139 INFO:tasks.workunit.client.0.smithi007.stdout:5/246: rename d2/f43 to d2/f51 0 2024-09-21T04:35:13.143 INFO:tasks.workunit.client.1.smithi053.stdout:0/626: symlink d5/de/d9e/dd8/ldc 0 2024-09-21T04:35:13.144 INFO:tasks.workunit.client.1.smithi053.stdout:8/691: creat d3/d5/db1/fe4 x:0 0 0 2024-09-21T04:35:13.147 INFO:tasks.workunit.client.0.smithi007.stdout:1/169: creat d5/d6/d21/d23/f35 x:0 0 0 2024-09-21T04:35:13.148 INFO:tasks.workunit.client.0.smithi007.stdout:4/128: dwrite d2/fd [0,4194304] 0 2024-09-21T04:35:13.155 INFO:tasks.workunit.client.0.smithi007.stdout:5/247: symlink d2/dd/d23/d49/d44/l52 0 2024-09-21T04:35:13.157 INFO:tasks.workunit.client.0.smithi007.stdout:5/248: write d2/d1e/f24 [2010744,90801] 0 2024-09-21T04:35:13.159 INFO:tasks.workunit.client.0.smithi007.stdout:5/249: stat d2/dd/d23/l34 0 2024-09-21T04:35:13.159 INFO:tasks.workunit.client.0.smithi007.stdout:4/129: dwrite d2/d7/fa [4194304,4194304] 0 2024-09-21T04:35:13.160 INFO:tasks.workunit.client.0.smithi007.stdout:6/37: sync 2024-09-21T04:35:13.162 INFO:tasks.workunit.client.0.smithi007.stdout:5/250: write d2/dd/d23/d2a/f2f [672752,14236] 0 2024-09-21T04:35:13.164 INFO:tasks.workunit.client.0.smithi007.stdout:5/251: write d2/dd/d1d/f39 [47014,106369] 0 2024-09-21T04:35:13.169 INFO:tasks.workunit.client.0.smithi007.stdout:4/130: mkdir d2/d17/d2e 0 2024-09-21T04:35:13.171 INFO:tasks.workunit.client.0.smithi007.stdout:4/131: chown d2/d17/f2a 24198502 1 2024-09-21T04:35:13.171 INFO:tasks.workunit.client.0.smithi007.stdout:1/170: dwrite d5/d6/d21/f30 [4194304,4194304] 0 2024-09-21T04:35:13.173 INFO:tasks.workunit.client.0.smithi007.stdout:6/38: mknod d3/c9 0 2024-09-21T04:35:13.174 INFO:tasks.workunit.client.0.smithi007.stdout:5/252: truncate d2/dd/d1d/f4b 1416247 0 2024-09-21T04:35:13.175 INFO:tasks.workunit.client.0.smithi007.stdout:6/39: write d3/f5 [82896,112545] 0 2024-09-21T04:35:13.177 INFO:tasks.workunit.client.0.smithi007.stdout:9/106: link f11 d13/d18/f25 0 2024-09-21T04:35:13.177 INFO:tasks.workunit.client.1.smithi053.stdout:9/618: sync 2024-09-21T04:35:13.179 INFO:tasks.workunit.client.0.smithi007.stdout:1/171: unlink f1 0 2024-09-21T04:35:13.180 INFO:tasks.workunit.client.0.smithi007.stdout:1/172: stat d5/f15 0 2024-09-21T04:35:13.180 INFO:tasks.workunit.client.1.smithi053.stdout:4/728: creat d1/dd/d62/ff0 x:0 0 0 2024-09-21T04:35:13.180 INFO:tasks.workunit.client.1.smithi053.stdout:7/561: mknod d0/d2b/d36/d48/d5c/cb9 0 2024-09-21T04:35:13.181 INFO:tasks.workunit.client.0.smithi007.stdout:4/132: dread d2/f6 [0,4194304] 0 2024-09-21T04:35:13.181 INFO:tasks.workunit.client.1.smithi053.stdout:1/671: dwrite d7/d9/da/f50 [0,4194304] 0 2024-09-21T04:35:13.183 INFO:tasks.workunit.client.0.smithi007.stdout:6/40: unlink d3/c9 0 2024-09-21T04:35:13.183 INFO:tasks.workunit.client.1.smithi053.stdout:3/649: creat d16/d17/d21/d8b/d7d/ff3 x:0 0 0 2024-09-21T04:35:13.186 INFO:tasks.workunit.client.0.smithi007.stdout:6/41: write d3/f6 [67409,90960] 0 2024-09-21T04:35:13.186 INFO:tasks.workunit.client.0.smithi007.stdout:6/42: rename d3 to d3/da 22 2024-09-21T04:35:13.187 INFO:tasks.workunit.client.1.smithi053.stdout:6/661: write d2/d1b/f7d [896148,29273] 0 2024-09-21T04:35:13.187 INFO:tasks.workunit.client.1.smithi053.stdout:5/706: write d0/d8/f1f [622419,97931] 0 2024-09-21T04:35:13.187 INFO:tasks.workunit.client.1.smithi053.stdout:1/672: chown d7/d9/d43/d94/f51 3030871 1 2024-09-21T04:35:13.190 INFO:tasks.workunit.client.0.smithi007.stdout:5/253: mkdir d2/d27/d42/d53 0 2024-09-21T04:35:13.193 INFO:tasks.workunit.client.0.smithi007.stdout:1/173: dwrite d5/d6/d21/d23/f2a [0,4194304] 0 2024-09-21T04:35:13.193 INFO:tasks.workunit.client.1.smithi053.stdout:5/707: write d0/dd/d48/d8c/fc6 [900643,57448] 0 2024-09-21T04:35:13.195 INFO:tasks.workunit.client.1.smithi053.stdout:9/619: dwrite d0/d1a/d4c/f94 [0,4194304] 0 2024-09-21T04:35:13.196 INFO:tasks.workunit.client.1.smithi053.stdout:5/708: stat d0/dd/d5c/d89/fb4 0 2024-09-21T04:35:13.197 INFO:tasks.workunit.client.1.smithi053.stdout:3/650: dwrite d16/d17/d21/d8b/f66 [4194304,4194304] 0 2024-09-21T04:35:13.203 INFO:tasks.workunit.client.0.smithi007.stdout:9/107: dread d13/f15 [0,4194304] 0 2024-09-21T04:35:13.206 INFO:tasks.workunit.client.1.smithi053.stdout:4/729: rename d1/d2c to d1/d27/d53/d88/d91/d20/d4e/d51/da4/df1 0 2024-09-21T04:35:13.206 INFO:tasks.workunit.client.1.smithi053.stdout:1/673: mkdir d7/d63/d96/de2 0 2024-09-21T04:35:13.207 INFO:tasks.workunit.client.0.smithi007.stdout:6/43: mkdir d3/db 0 2024-09-21T04:35:13.207 INFO:tasks.workunit.client.0.smithi007.stdout:6/44: dread - d3/f8 zero size 2024-09-21T04:35:13.207 INFO:tasks.workunit.client.0.smithi007.stdout:6/45: stat d3/f8 0 2024-09-21T04:35:13.208 INFO:tasks.workunit.client.1.smithi053.stdout:4/730: write d1/d27/d90/fb1 [117386,47676] 0 2024-09-21T04:35:13.210 INFO:tasks.workunit.client.1.smithi053.stdout:4/731: write d1/d27/d53/fc2 [3621047,106595] 0 2024-09-21T04:35:13.210 INFO:tasks.workunit.client.1.smithi053.stdout:0/627: creat d5/d10/d4c/db1/fdd x:0 0 0 2024-09-21T04:35:13.211 INFO:tasks.workunit.client.0.smithi007.stdout:5/254: unlink d2/c9 0 2024-09-21T04:35:13.215 INFO:tasks.workunit.client.1.smithi053.stdout:8/692: creat d3/d5/d38/d64/fe5 x:0 0 0 2024-09-21T04:35:13.217 INFO:tasks.workunit.client.1.smithi053.stdout:5/709: mkdir d0/dd/d5c/d92/de8 0 2024-09-21T04:35:13.218 INFO:tasks.workunit.client.0.smithi007.stdout:1/174: dread d5/fe [0,4194304] 0 2024-09-21T04:35:13.218 INFO:tasks.workunit.client.1.smithi053.stdout:8/693: read d3/d39/f3f [3738768,77696] 0 2024-09-21T04:35:13.219 INFO:tasks.workunit.client.0.smithi007.stdout:5/255: dwrite d2/d1e/f24 [8388608,4194304] 0 2024-09-21T04:35:13.219 INFO:tasks.workunit.client.0.smithi007.stdout:1/175: chown d5/d6/d21/d2b 123524 1 2024-09-21T04:35:13.220 INFO:tasks.workunit.client.1.smithi053.stdout:8/694: write d3/d5/d38/d64/d9d/da9/fe3 [1764912,99722] 0 2024-09-21T04:35:13.221 INFO:tasks.workunit.client.0.smithi007.stdout:1/176: chown d5/d6/d21/d23 10 1 2024-09-21T04:35:13.221 INFO:tasks.workunit.client.1.smithi053.stdout:3/651: mknod d16/d17/d3c/cf4 0 2024-09-21T04:35:13.223 INFO:tasks.workunit.client.1.smithi053.stdout:3/652: write d16/d17/d21/d8b/d7d/d57/dc1/d53/f9a [450340,43638] 0 2024-09-21T04:35:13.226 INFO:tasks.workunit.client.0.smithi007.stdout:1/177: dread d5/f15 [0,4194304] 0 2024-09-21T04:35:13.227 INFO:tasks.workunit.client.0.smithi007.stdout:1/178: chown d5/f17 1317 1 2024-09-21T04:35:13.229 INFO:tasks.workunit.client.1.smithi053.stdout:7/562: write d0/d2b/f43 [1725351,75436] 0 2024-09-21T04:35:13.230 INFO:tasks.workunit.client.1.smithi053.stdout:4/732: creat d1/dd/d62/ff2 x:0 0 0 2024-09-21T04:35:13.232 INFO:tasks.workunit.client.0.smithi007.stdout:5/256: mkdir d2/dd/d23/d49/d44/d54 0 2024-09-21T04:35:13.233 INFO:tasks.workunit.client.1.smithi053.stdout:1/674: dwrite d7/f56 [0,4194304] 0 2024-09-21T04:35:13.237 INFO:tasks.workunit.client.1.smithi053.stdout:1/675: write d7/d47/d79/fde [104342,95184] 0 2024-09-21T04:35:13.237 INFO:tasks.workunit.client.0.smithi007.stdout:4/133: write d2/f27 [2237107,51749] 0 2024-09-21T04:35:13.245 INFO:tasks.workunit.client.1.smithi053.stdout:5/710: unlink d0/dd/d5c/d80/fd2 0 2024-09-21T04:35:13.245 INFO:tasks.workunit.client.0.smithi007.stdout:4/134: dread d2/f25 [0,4194304] 0 2024-09-21T04:35:13.246 INFO:tasks.workunit.client.0.smithi007.stdout:4/135: chown d2/d17/d29 30 1 2024-09-21T04:35:13.249 INFO:tasks.workunit.client.1.smithi053.stdout:8/695: rename d3/d5/d4c/d8e/fc0 to d3/df/d49/fe6 0 2024-09-21T04:35:13.251 INFO:tasks.workunit.client.0.smithi007.stdout:6/46: rmdir d3/db 0 2024-09-21T04:35:13.252 INFO:tasks.workunit.client.1.smithi053.stdout:9/620: write d0/f24 [911086,25532] 0 2024-09-21T04:35:13.253 INFO:tasks.workunit.client.1.smithi053.stdout:9/621: chown d0/fcf 1676811 1 2024-09-21T04:35:13.254 INFO:tasks.workunit.client.1.smithi053.stdout:3/653: unlink d16/d17/d21/d42/da0/lca 0 2024-09-21T04:35:13.255 INFO:tasks.workunit.client.1.smithi053.stdout:4/733: dread d1/d27/f9d [0,4194304] 0 2024-09-21T04:35:13.256 INFO:tasks.workunit.client.0.smithi007.stdout:7/150: sync 2024-09-21T04:35:13.256 INFO:tasks.workunit.client.0.smithi007.stdout:8/52: sync 2024-09-21T04:35:13.258 INFO:tasks.workunit.client.1.smithi053.stdout:7/563: write d0/d24/fb3 [247508,101423] 0 2024-09-21T04:35:13.258 INFO:tasks.workunit.client.0.smithi007.stdout:9/108: getdents d13 0 2024-09-21T04:35:13.259 INFO:tasks.workunit.client.1.smithi053.stdout:9/622: dread d0/d2/d9/f95 [0,4194304] 0 2024-09-21T04:35:13.259 INFO:tasks.workunit.client.0.smithi007.stdout:4/136: chown d2/d17/d18/c16 6961940 1 2024-09-21T04:35:13.260 INFO:tasks.workunit.client.1.smithi053.stdout:0/628: mknod d5/de/d1c/da3/cde 0 2024-09-21T04:35:13.262 INFO:tasks.workunit.client.1.smithi053.stdout:6/662: creat d2/d1b/d3d/d49/de/fd8 x:0 0 0 2024-09-21T04:35:13.266 INFO:tasks.workunit.client.1.smithi053.stdout:6/663: write d2/fd [3288099,103080] 0 2024-09-21T04:35:13.266 INFO:tasks.workunit.client.0.smithi007.stdout:9/109: dwrite d13/f20 [0,4194304] 0 2024-09-21T04:35:13.267 INFO:tasks.workunit.client.1.smithi053.stdout:3/654: dwrite d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fda [0,4194304] 0 2024-09-21T04:35:13.267 INFO:tasks.workunit.client.0.smithi007.stdout:9/110: fsync d13/f1e 0 2024-09-21T04:35:13.271 INFO:tasks.workunit.client.0.smithi007.stdout:5/257: write d2/d1e/f22 [2458281,7262] 0 2024-09-21T04:35:13.276 INFO:tasks.workunit.client.1.smithi053.stdout:5/711: creat d0/dd/d67/fe9 x:0 0 0 2024-09-21T04:35:13.276 INFO:tasks.workunit.client.0.smithi007.stdout:4/137: mknod d2/d17/c2f 0 2024-09-21T04:35:13.276 INFO:tasks.workunit.client.0.smithi007.stdout:6/47: link d3/f5 d3/fc 0 2024-09-21T04:35:13.276 INFO:tasks.workunit.client.0.smithi007.stdout:9/111: dwrite d13/f21 [0,4194304] 0 2024-09-21T04:35:13.277 INFO:tasks.workunit.client.1.smithi053.stdout:9/623: dwrite d0/d1c/d25/f54 [0,4194304] 0 2024-09-21T04:35:13.278 INFO:tasks.workunit.client.0.smithi007.stdout:1/179: mknod d5/d6/c36 0 2024-09-21T04:35:13.279 INFO:tasks.workunit.client.1.smithi053.stdout:7/564: dwrite d0/d2b/d36/d48/d50/d8b/f42 [0,4194304] 0 2024-09-21T04:35:13.279 INFO:tasks.workunit.client.1.smithi053.stdout:4/734: write d1/d27/f31 [4742973,28272] 0 2024-09-21T04:35:13.282 INFO:tasks.workunit.client.0.smithi007.stdout:7/151: fdatasync d3/d6/d14/f26 0 2024-09-21T04:35:13.285 INFO:tasks.workunit.client.0.smithi007.stdout:4/138: mknod d2/c30 0 2024-09-21T04:35:13.286 INFO:tasks.workunit.client.0.smithi007.stdout:4/139: chown d2/d17/f26 47571370 1 2024-09-21T04:35:13.288 INFO:tasks.workunit.client.0.smithi007.stdout:6/48: symlink d3/ld 0 2024-09-21T04:35:13.288 INFO:tasks.workunit.client.0.smithi007.stdout:4/140: chown d2/d7/f10 45349 1 2024-09-21T04:35:13.289 INFO:tasks.workunit.client.1.smithi053.stdout:3/655: mkdir d16/d17/d21/d8b/d7d/d57/df5 0 2024-09-21T04:35:13.289 INFO:tasks.workunit.client.0.smithi007.stdout:6/49: chown d3/f6 325 1 2024-09-21T04:35:13.290 INFO:tasks.workunit.client.0.smithi007.stdout:9/112: creat d13/d18/f26 x:0 0 0 2024-09-21T04:35:13.290 INFO:tasks.workunit.client.1.smithi053.stdout:8/696: mkdir d3/d5/d4c/dc3/ddb/de7 0 2024-09-21T04:35:13.291 INFO:tasks.workunit.client.0.smithi007.stdout:6/50: write d3/f8 [68061,294] 0 2024-09-21T04:35:13.293 INFO:tasks.workunit.client.1.smithi053.stdout:2/693: sync 2024-09-21T04:35:13.294 INFO:tasks.workunit.client.0.smithi007.stdout:1/180: truncate d5/d6/d21/d2b/f2d 178185 0 2024-09-21T04:35:13.295 INFO:tasks.workunit.client.1.smithi053.stdout:9/624: dwrite d0/d28/fd6 [0,4194304] 0 2024-09-21T04:35:13.295 INFO:tasks.workunit.client.1.smithi053.stdout:6/664: dread d2/d1b/f7d [0,4194304] 0 2024-09-21T04:35:13.296 INFO:tasks.workunit.client.1.smithi053.stdout:0/629: dread d5/dd/f34 [0,4194304] 0 2024-09-21T04:35:13.298 INFO:tasks.workunit.client.1.smithi053.stdout:2/694: dread - d1/d18/d1f/d27/d41/d4b/d4d/d89/fd7 zero size 2024-09-21T04:35:13.301 INFO:tasks.workunit.client.1.smithi053.stdout:2/695: fsync d1/d18/d1f/d27/d58/d5c/fd4 0 2024-09-21T04:35:13.303 INFO:tasks.workunit.client.0.smithi007.stdout:1/181: dwrite d5/f28 [0,4194304] 0 2024-09-21T04:35:13.304 INFO:tasks.workunit.client.0.smithi007.stdout:1/182: read - d5/f25 zero size 2024-09-21T04:35:13.304 INFO:tasks.workunit.client.1.smithi053.stdout:1/676: rename d7/d9/da/f9b to d7/d9/d43/d54/d7f/fe3 0 2024-09-21T04:35:13.307 INFO:tasks.workunit.client.1.smithi053.stdout:3/656: rmdir d16/d17/ddc 39 2024-09-21T04:35:13.308 INFO:tasks.workunit.client.0.smithi007.stdout:8/53: getdents d0 0 2024-09-21T04:35:13.309 INFO:tasks.workunit.client.1.smithi053.stdout:0/630: dwrite d5/d10/d4c/db1/dbc/dca/d53/fa9 [4194304,4194304] 0 2024-09-21T04:35:13.313 INFO:tasks.workunit.client.1.smithi053.stdout:8/697: unlink d3/d5/d1a/l20 0 2024-09-21T04:35:13.313 INFO:tasks.workunit.client.0.smithi007.stdout:5/258: rename d2/dd/f47 to d2/f55 0 2024-09-21T04:35:13.314 INFO:tasks.workunit.client.1.smithi053.stdout:9/625: dread d0/d2/d9/f19 [0,4194304] 0 2024-09-21T04:35:13.315 INFO:tasks.workunit.client.1.smithi053.stdout:8/698: chown d3/d5/d21/cbd 1 1 2024-09-21T04:35:13.316 INFO:tasks.workunit.client.1.smithi053.stdout:9/626: write d0/f4b [2297010,3412] 0 2024-09-21T04:35:13.316 INFO:tasks.workunit.client.0.smithi007.stdout:5/259: write d2/dd/d28/f4d [592890,3193] 0 2024-09-21T04:35:13.317 INFO:tasks.workunit.client.0.smithi007.stdout:1/183: dwrite d5/d6/d21/d23/f24 [0,4194304] 0 2024-09-21T04:35:13.317 INFO:tasks.workunit.client.1.smithi053.stdout:8/699: write d3/d5/d38/d64/d73/d91/fa4 [848907,121668] 0 2024-09-21T04:35:13.320 INFO:tasks.workunit.client.0.smithi007.stdout:4/141: link d2/f23 d2/d7/f31 0 2024-09-21T04:35:13.321 INFO:tasks.workunit.client.1.smithi053.stdout:7/565: write d0/fd [1131782,88000] 0 2024-09-21T04:35:13.321 INFO:tasks.workunit.client.0.smithi007.stdout:4/142: fdatasync d2/d17/d18/f2d 0 2024-09-21T04:35:13.324 INFO:tasks.workunit.client.0.smithi007.stdout:1/184: dread d5/d6/d21/d23/f2a [0,4194304] 0 2024-09-21T04:35:13.325 INFO:tasks.workunit.client.0.smithi007.stdout:8/54: creat d0/d3/d8/f10 x:0 0 0 2024-09-21T04:35:13.326 INFO:tasks.workunit.client.0.smithi007.stdout:1/185: chown d5/d2f/f34 187502224 1 2024-09-21T04:35:13.328 INFO:tasks.workunit.client.1.smithi053.stdout:5/712: truncate d0/d8/f1f 957012 0 2024-09-21T04:35:13.330 INFO:tasks.workunit.client.0.smithi007.stdout:8/55: write d0/d3/d8/f10 [666582,68301] 0 2024-09-21T04:35:13.331 INFO:tasks.workunit.client.0.smithi007.stdout:7/152: rename d3/d1a to d3/d6/d14/d20/d33 0 2024-09-21T04:35:13.331 INFO:tasks.workunit.client.0.smithi007.stdout:6/51: dwrite d3/fc [0,4194304] 0 2024-09-21T04:35:13.334 INFO:tasks.workunit.client.1.smithi053.stdout:2/696: creat d1/d18/d1f/d22/d43/da6/f100 x:0 0 0 2024-09-21T04:35:13.337 INFO:tasks.workunit.client.1.smithi053.stdout:6/665: rename d2/d1b/d3d/d49/f19 to d2/d1b/d1e/d2a/d4b/fd9 0 2024-09-21T04:35:13.338 INFO:tasks.workunit.client.0.smithi007.stdout:9/113: getdents d13 0 2024-09-21T04:35:13.338 INFO:tasks.workunit.client.0.smithi007.stdout:9/114: truncate d13/f1d 9493 0 2024-09-21T04:35:13.340 INFO:tasks.workunit.client.0.smithi007.stdout:4/143: symlink d2/d17/l32 0 2024-09-21T04:35:13.340 INFO:tasks.workunit.client.0.smithi007.stdout:9/115: dread d13/f15 [0,4194304] 0 2024-09-21T04:35:13.343 INFO:tasks.workunit.client.1.smithi053.stdout:0/631: readlink d5/de/d42/d78/l5c 0 2024-09-21T04:35:13.344 INFO:tasks.workunit.client.1.smithi053.stdout:6/666: dwrite d2/d1b/d1e/fcb [0,4194304] 0 2024-09-21T04:35:13.345 INFO:tasks.workunit.client.1.smithi053.stdout:0/632: write d5/dd/d50/f9d [227722,79934] 0 2024-09-21T04:35:13.346 INFO:tasks.workunit.client.1.smithi053.stdout:6/667: write d2/d1b/d3d/d49/de/d32/d80/f57 [3012657,56904] 0 2024-09-21T04:35:13.348 INFO:tasks.workunit.client.1.smithi053.stdout:0/633: write d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f91 [1051572,128527] 0 2024-09-21T04:35:13.349 INFO:tasks.workunit.client.1.smithi053.stdout:0/634: dread - d5/de/f82 zero size 2024-09-21T04:35:13.350 INFO:tasks.workunit.client.1.smithi053.stdout:0/635: dread - d5/de/d42/fa4 zero size 2024-09-21T04:35:13.351 INFO:tasks.workunit.client.0.smithi007.stdout:6/52: mkdir d3/de 0 2024-09-21T04:35:13.352 INFO:tasks.workunit.client.1.smithi053.stdout:0/636: chown d5/d10/d4c/db1/dbc/dca/db5/fd6 4165 1 2024-09-21T04:35:13.352 INFO:tasks.workunit.client.0.smithi007.stdout:6/53: stat d3/ld 0 2024-09-21T04:35:13.352 INFO:tasks.workunit.client.1.smithi053.stdout:4/735: creat d1/d27/d53/d88/d91/ff3 x:0 0 0 2024-09-21T04:35:13.353 INFO:tasks.workunit.client.0.smithi007.stdout:7/153: write d3/d6/d14/f32 [1437865,50436] 0 2024-09-21T04:35:13.354 INFO:tasks.workunit.client.1.smithi053.stdout:4/736: read - d1/d27/d53/d88/d91/d3f/d74/de1/fed zero size 2024-09-21T04:35:13.354 INFO:tasks.workunit.client.0.smithi007.stdout:5/260: unlink d2/dd/c1a 0 2024-09-21T04:35:13.356 INFO:tasks.workunit.client.0.smithi007.stdout:5/261: write d2/d27/d3d/f4e [576893,101069] 0 2024-09-21T04:35:13.358 INFO:tasks.workunit.client.0.smithi007.stdout:4/144: fsync d2/f1d 0 2024-09-21T04:35:13.360 INFO:tasks.workunit.client.0.smithi007.stdout:4/145: dread d2/f1d [0,4194304] 0 2024-09-21T04:35:13.361 INFO:tasks.workunit.client.1.smithi053.stdout:4/737: dwrite d1/dd/fb2 [0,4194304] 0 2024-09-21T04:35:13.362 INFO:tasks.workunit.client.0.smithi007.stdout:7/154: dwrite d3/f5 [4194304,4194304] 0 2024-09-21T04:35:13.364 INFO:tasks.workunit.client.0.smithi007.stdout:9/116: creat d13/d18/f27 x:0 0 0 2024-09-21T04:35:13.366 INFO:tasks.workunit.client.1.smithi053.stdout:9/627: write d0/d1a/f81 [87564,21402] 0 2024-09-21T04:35:13.368 INFO:tasks.workunit.client.1.smithi053.stdout:7/566: rename d0/le to d0/d2b/d36/d48/d95/lba 0 2024-09-21T04:35:13.370 INFO:tasks.workunit.client.1.smithi053.stdout:7/567: truncate d0/d2b/fa8 559859 0 2024-09-21T04:35:13.373 INFO:tasks.workunit.client.0.smithi007.stdout:8/56: dwrite d0/d3/d8/f9 [4194304,4194304] 0 2024-09-21T04:35:13.373 INFO:tasks.workunit.client.0.smithi007.stdout:4/146: dwrite d2/d17/d18/f1f [0,4194304] 0 2024-09-21T04:35:13.377 INFO:tasks.workunit.client.0.smithi007.stdout:5/262: rmdir d2/dd/d23/d2a 39 2024-09-21T04:35:13.383 INFO:tasks.workunit.client.0.smithi007.stdout:7/155: creat d3/d6/d10/f34 x:0 0 0 2024-09-21T04:35:13.383 INFO:tasks.workunit.client.0.smithi007.stdout:7/156: stat d3/d6/d14/l2a 0 2024-09-21T04:35:13.385 INFO:tasks.workunit.client.1.smithi053.stdout:1/677: truncate d7/d9/da/d11/f48 253772 0 2024-09-21T04:35:13.387 INFO:tasks.workunit.client.1.smithi053.stdout:7/568: dread d0/fb [4194304,4194304] 0 2024-09-21T04:35:13.388 INFO:tasks.workunit.client.1.smithi053.stdout:6/668: rmdir d2/d1d 39 2024-09-21T04:35:13.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/117: rename d13/d18/f24 to d13/f28 0 2024-09-21T04:35:13.391 INFO:tasks.workunit.client.1.smithi053.stdout:3/657: truncate d16/f1d 2536571 0 2024-09-21T04:35:13.394 INFO:tasks.workunit.client.0.smithi007.stdout:6/54: symlink d3/de/lf 0 2024-09-21T04:35:13.394 INFO:tasks.workunit.client.0.smithi007.stdout:9/118: dread f10 [0,4194304] 0 2024-09-21T04:35:13.396 INFO:tasks.workunit.client.1.smithi053.stdout:0/637: write d5/de/d1c/f7e [1500458,105302] 0 2024-09-21T04:35:13.397 INFO:tasks.workunit.client.0.smithi007.stdout:9/119: dread f11 [0,4194304] 0 2024-09-21T04:35:13.398 INFO:tasks.workunit.client.0.smithi007.stdout:1/186: truncate d5/d6/fa 2113854 0 2024-09-21T04:35:13.398 INFO:tasks.workunit.client.0.smithi007.stdout:4/147: fdatasync d2/f6 0 2024-09-21T04:35:13.399 INFO:tasks.workunit.client.0.smithi007.stdout:6/55: dread d3/fc [0,4194304] 0 2024-09-21T04:35:13.400 INFO:tasks.workunit.client.0.smithi007.stdout:4/148: read - d2/d17/f26 zero size 2024-09-21T04:35:13.401 INFO:tasks.workunit.client.0.smithi007.stdout:4/149: stat d2/d7/c15 0 2024-09-21T04:35:13.402 INFO:tasks.workunit.client.0.smithi007.stdout:0/74: sync 2024-09-21T04:35:13.403 INFO:tasks.workunit.client.0.smithi007.stdout:5/263: mkdir d2/d27/d3d/d56 0 2024-09-21T04:35:13.403 INFO:tasks.workunit.client.0.smithi007.stdout:0/75: chown d6/f13 3018 1 2024-09-21T04:35:13.407 INFO:tasks.workunit.client.1.smithi053.stdout:4/738: symlink d1/d27/d53/d88/d91/d20/d73/d92/lf4 0 2024-09-21T04:35:13.408 INFO:tasks.workunit.client.0.smithi007.stdout:7/157: creat d3/d6/d14/d20/f35 x:0 0 0 2024-09-21T04:35:13.409 INFO:tasks.workunit.client.0.smithi007.stdout:7/158: chown d3/d6/cb 1864 1 2024-09-21T04:35:13.412 INFO:tasks.workunit.client.1.smithi053.stdout:2/697: rename d1/d18/d1f/d9d/c6f to d1/d18/d1f/d27/d58/d5c/c101 0 2024-09-21T04:35:13.416 INFO:tasks.workunit.client.0.smithi007.stdout:7/159: dwrite d3/d6/fa [0,4194304] 0 2024-09-21T04:35:13.416 INFO:tasks.workunit.client.0.smithi007.stdout:9/120: fdatasync f7 0 2024-09-21T04:35:13.417 INFO:tasks.workunit.client.0.smithi007.stdout:9/121: truncate d13/f1e 4862112 0 2024-09-21T04:35:13.418 INFO:tasks.workunit.client.1.smithi053.stdout:7/569: symlink d0/d2b/d36/d48/d95/lbb 0 2024-09-21T04:35:13.419 INFO:tasks.workunit.client.0.smithi007.stdout:1/187: mknod d5/d2f/c37 0 2024-09-21T04:35:13.424 INFO:tasks.workunit.client.1.smithi053.stdout:8/700: creat d3/df/fe8 x:0 0 0 2024-09-21T04:35:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:6/56: creat d3/de/f10 x:0 0 0 2024-09-21T04:35:13.431 INFO:tasks.workunit.client.1.smithi053.stdout:5/713: rmdir d0/dd/d5c/dcc 0 2024-09-21T04:35:13.431 INFO:tasks.workunit.client.1.smithi053.stdout:5/714: chown d0/dd/d48/f60 1576 1 2024-09-21T04:35:13.433 INFO:tasks.workunit.client.1.smithi053.stdout:0/638: dread d5/d10/fb9 [0,4194304] 0 2024-09-21T04:35:13.434 INFO:tasks.workunit.client.0.smithi007.stdout:8/57: truncate d0/f6 2928202 0 2024-09-21T04:35:13.436 INFO:tasks.workunit.client.0.smithi007.stdout:5/264: write d2/f19 [586168,35493] 0 2024-09-21T04:35:13.436 INFO:tasks.workunit.client.0.smithi007.stdout:8/58: fdatasync d0/d3/d8/f10 0 2024-09-21T04:35:13.436 INFO:tasks.workunit.client.0.smithi007.stdout:5/265: stat d2/dd/le 0 2024-09-21T04:35:13.437 INFO:tasks.workunit.client.1.smithi053.stdout:4/739: rename d1/d27/d53/d88/d91 to d1/dd/de/df5 0 2024-09-21T04:35:13.438 INFO:tasks.workunit.client.0.smithi007.stdout:0/76: symlink d6/d16/l1a 0 2024-09-21T04:35:13.442 INFO:tasks.workunit.client.1.smithi053.stdout:2/698: unlink d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/lce 0 2024-09-21T04:35:13.444 INFO:tasks.workunit.client.1.smithi053.stdout:2/699: dread - d1/d18/d1f/d22/d43/da6/f100 zero size 2024-09-21T04:35:13.445 INFO:tasks.workunit.client.1.smithi053.stdout:4/740: dwrite d1/dd/de/df5/d3f/f77 [0,4194304] 0 2024-09-21T04:35:13.445 INFO:tasks.workunit.client.0.smithi007.stdout:1/188: mknod d5/d6/d21/d23/c38 0 2024-09-21T04:35:13.446 INFO:tasks.workunit.client.1.smithi053.stdout:6/669: mknod d2/d1b/d3d/d49/de/cda 0 2024-09-21T04:35:13.447 INFO:tasks.workunit.client.1.smithi053.stdout:5/715: dread d0/d8/db/f7f [0,4194304] 0 2024-09-21T04:35:13.447 INFO:tasks.workunit.client.0.smithi007.stdout:1/189: dread d5/fe [0,4194304] 0 2024-09-21T04:35:13.448 INFO:tasks.workunit.client.1.smithi053.stdout:4/741: read - d1/d27/d2e/f61 zero size 2024-09-21T04:35:13.448 INFO:tasks.workunit.client.0.smithi007.stdout:1/190: write d5/f28 [4322733,48316] 0 2024-09-21T04:35:13.449 INFO:tasks.workunit.client.1.smithi053.stdout:3/658: read - d16/d17/d21/d42/d43/d86/fce zero size 2024-09-21T04:35:13.450 INFO:tasks.workunit.client.0.smithi007.stdout:1/191: truncate d5/d2f/f34 936258 0 2024-09-21T04:35:13.450 INFO:tasks.workunit.client.0.smithi007.stdout:1/192: chown d5/d6/d21/d23/f35 1704 1 2024-09-21T04:35:13.451 INFO:tasks.workunit.client.1.smithi053.stdout:8/701: creat d3/d4f/fe9 x:0 0 0 2024-09-21T04:35:13.451 INFO:tasks.workunit.client.0.smithi007.stdout:1/193: readlink d5/l1e 0 2024-09-21T04:35:13.452 INFO:tasks.workunit.client.0.smithi007.stdout:1/194: write d5/f26 [142764,98] 0 2024-09-21T04:35:13.452 INFO:tasks.workunit.client.1.smithi053.stdout:9/628: sync 2024-09-21T04:35:13.457 INFO:tasks.workunit.client.1.smithi053.stdout:3/659: dwrite d16/d17/d21/d8b/d7d/d57/dc1/d53/fa1 [8388608,4194304] 0 2024-09-21T04:35:13.458 INFO:tasks.workunit.client.0.smithi007.stdout:4/150: link d2/d17/f26 d2/d17/d2e/f33 0 2024-09-21T04:35:13.460 INFO:tasks.workunit.client.1.smithi053.stdout:0/639: mkdir d5/d10/d4c/ddf 0 2024-09-21T04:35:13.463 INFO:tasks.workunit.client.1.smithi053.stdout:1/678: truncate d7/d9/d43/d94/f61 356123 0 2024-09-21T04:35:13.464 INFO:tasks.workunit.client.1.smithi053.stdout:1/679: dread - d7/d9/d43/d54/fa1 zero size 2024-09-21T04:35:13.466 INFO:tasks.workunit.client.1.smithi053.stdout:0/640: dwrite d5/de/f3e [0,4194304] 0 2024-09-21T04:35:13.468 INFO:tasks.workunit.client.1.smithi053.stdout:5/716: rmdir d0/dd/d48/d8c 39 2024-09-21T04:35:13.471 INFO:tasks.workunit.client.1.smithi053.stdout:4/742: fsync d1/dd/de/df5/d20/d4e/d51/da4/df1/f7f 0 2024-09-21T04:35:13.476 INFO:tasks.workunit.client.0.smithi007.stdout:6/57: link d3/f8 d3/de/f11 0 2024-09-21T04:35:13.477 INFO:tasks.workunit.client.0.smithi007.stdout:4/151: mknod d2/d7/c34 0 2024-09-21T04:35:13.478 INFO:tasks.workunit.client.1.smithi053.stdout:3/660: dread d16/d17/d21/d60/f9c [0,4194304] 0 2024-09-21T04:35:13.479 INFO:tasks.workunit.client.0.smithi007.stdout:8/59: mknod d0/d3/c11 0 2024-09-21T04:35:13.484 INFO:tasks.workunit.client.1.smithi053.stdout:0/641: rmdir d5/de/d9e 39 2024-09-21T04:35:13.485 INFO:tasks.workunit.client.0.smithi007.stdout:9/122: truncate fa 2573121 0 2024-09-21T04:35:13.485 INFO:tasks.workunit.client.1.smithi053.stdout:0/642: write d5/de/f3e [638867,16765] 0 2024-09-21T04:35:13.487 INFO:tasks.workunit.client.1.smithi053.stdout:0/643: dread - d5/d10/d4c/db1/dbc/dca/db5/d99/fce zero size 2024-09-21T04:35:13.489 INFO:tasks.workunit.client.0.smithi007.stdout:6/58: mkdir d3/de/d12 0 2024-09-21T04:35:13.490 INFO:tasks.workunit.client.0.smithi007.stdout:4/152: symlink d2/d17/l35 0 2024-09-21T04:35:13.491 INFO:tasks.workunit.client.1.smithi053.stdout:7/570: rename d0/d2b/d36/d48/d50/d8b/l39 to d0/d2b/lbc 0 2024-09-21T04:35:13.493 INFO:tasks.workunit.client.0.smithi007.stdout:8/60: creat d0/d3/d8/f12 x:0 0 0 2024-09-21T04:35:13.493 INFO:tasks.workunit.client.0.smithi007.stdout:0/77: truncate d6/f13 57395 0 2024-09-21T04:35:13.494 INFO:tasks.workunit.client.0.smithi007.stdout:8/61: dread - d0/d3/d8/f12 zero size 2024-09-21T04:35:13.494 INFO:tasks.workunit.client.1.smithi053.stdout:3/661: rmdir d16/d17/d21/d8b/d7d 39 2024-09-21T04:35:13.495 INFO:tasks.workunit.client.0.smithi007.stdout:9/123: truncate f11 1282469 0 2024-09-21T04:35:13.495 INFO:tasks.workunit.client.1.smithi053.stdout:3/662: chown d16/d17/d21/d42/d43/d86 322 1 2024-09-21T04:35:13.497 INFO:tasks.workunit.client.1.smithi053.stdout:7/571: dwrite d0/d2b/d36/d48/d50/d8b/f9 [4194304,4194304] 0 2024-09-21T04:35:13.499 INFO:tasks.workunit.client.0.smithi007.stdout:6/59: mknod d3/c13 0 2024-09-21T04:35:13.499 INFO:tasks.workunit.client.0.smithi007.stdout:8/62: dread d0/d3/d8/f9 [4194304,4194304] 0 2024-09-21T04:35:13.503 INFO:tasks.workunit.client.0.smithi007.stdout:4/153: unlink d2/d17/d18/f2b 0 2024-09-21T04:35:13.504 INFO:tasks.workunit.client.1.smithi053.stdout:5/717: mknod d0/dd/d48/cea 0 2024-09-21T04:35:13.506 INFO:tasks.workunit.client.1.smithi053.stdout:5/718: chown d0/dd/d5c/d92/ddb/d51/d52/c85 3 1 2024-09-21T04:35:13.512 INFO:tasks.workunit.client.1.smithi053.stdout:1/680: truncate d7/d9/da/f86 3170502 0 2024-09-21T04:35:13.512 INFO:tasks.workunit.client.1.smithi053.stdout:0/644: write d5/de/d9e/faf [536950,28158] 0 2024-09-21T04:35:13.513 INFO:tasks.workunit.client.0.smithi007.stdout:9/124: truncate f7 395479 0 2024-09-21T04:35:13.516 INFO:tasks.workunit.client.0.smithi007.stdout:1/195: link d5/f1a d5/f39 0 2024-09-21T04:35:13.517 INFO:tasks.workunit.client.0.smithi007.stdout:1/196: write d5/d6/d21/d2b/f2d [322305,66633] 0 2024-09-21T04:35:13.518 INFO:tasks.workunit.client.1.smithi053.stdout:5/719: dread d0/f2f [0,4194304] 0 2024-09-21T04:35:13.518 INFO:tasks.workunit.client.0.smithi007.stdout:1/197: chown d5/d6/d21/d23/c38 3561 1 2024-09-21T04:35:13.519 INFO:tasks.workunit.client.1.smithi053.stdout:0/645: dwrite d5/d10/d4c/db1/fdd [0,4194304] 0 2024-09-21T04:35:13.521 INFO:tasks.workunit.client.0.smithi007.stdout:8/63: symlink d0/l13 0 2024-09-21T04:35:13.524 INFO:tasks.workunit.client.0.smithi007.stdout:4/154: mknod d2/d17/d18/c36 0 2024-09-21T04:35:13.525 INFO:tasks.workunit.client.1.smithi053.stdout:3/663: fdatasync d16/d17/d21/d26/d49/f77 0 2024-09-21T04:35:13.525 INFO:tasks.workunit.client.0.smithi007.stdout:1/198: dwrite d5/f28 [0,4194304] 0 2024-09-21T04:35:13.527 INFO:tasks.workunit.client.1.smithi053.stdout:6/670: getdents d2/d1b/d3d/d49/d5c 0 2024-09-21T04:35:13.531 INFO:tasks.workunit.client.1.smithi053.stdout:7/572: unlink d0/d2b/c82 0 2024-09-21T04:35:13.532 INFO:tasks.workunit.client.1.smithi053.stdout:8/702: getdents d3/df/d10 0 2024-09-21T04:35:13.534 INFO:tasks.workunit.client.0.smithi007.stdout:4/155: dwrite d2/d17/d18/f2d [0,4194304] 0 2024-09-21T04:35:13.535 INFO:tasks.workunit.client.1.smithi053.stdout:9/629: link d0/d1c/f39 d0/d1c/d21/d46/fd7 0 2024-09-21T04:35:13.536 INFO:tasks.workunit.client.0.smithi007.stdout:9/125: mknod d13/c29 0 2024-09-21T04:35:13.538 INFO:tasks.workunit.client.0.smithi007.stdout:6/60: creat d3/de/d12/f14 x:0 0 0 2024-09-21T04:35:13.541 INFO:tasks.workunit.client.1.smithi053.stdout:2/700: rename d1/cd to d1/d18/d1f/d27/d41/d4b/c102 0 2024-09-21T04:35:13.541 INFO:tasks.workunit.client.1.smithi053.stdout:8/703: dwrite d3/df/fb2 [0,4194304] 0 2024-09-21T04:35:13.543 INFO:tasks.workunit.client.1.smithi053.stdout:2/701: chown d1/d18/d77/d9b 123813368 1 2024-09-21T04:35:13.548 INFO:tasks.workunit.client.1.smithi053.stdout:9/630: dwrite d0/d2/f55 [0,4194304] 0 2024-09-21T04:35:13.548 INFO:tasks.workunit.client.0.smithi007.stdout:5/266: link d2/dd/d23/d2a/c3c d2/c57 0 2024-09-21T04:35:13.552 INFO:tasks.workunit.client.1.smithi053.stdout:1/681: truncate d7/d9/da/d3c/f46 2489896 0 2024-09-21T04:35:13.552 INFO:tasks.workunit.client.0.smithi007.stdout:0/78: link d6/c10 d6/d16/c1b 0 2024-09-21T04:35:13.556 INFO:tasks.workunit.client.1.smithi053.stdout:2/702: dwrite d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/fff [0,4194304] 0 2024-09-21T04:35:13.558 INFO:tasks.workunit.client.0.smithi007.stdout:5/267: dread d2/f19 [0,4194304] 0 2024-09-21T04:35:13.559 INFO:tasks.workunit.client.0.smithi007.stdout:5/268: fsync d2/d1e/f37 0 2024-09-21T04:35:13.560 INFO:tasks.workunit.client.0.smithi007.stdout:4/156: symlink d2/d7/l37 0 2024-09-21T04:35:13.563 INFO:tasks.workunit.client.1.smithi053.stdout:4/743: rename d1/dd/de/df5/d20/d4e/f98 to d1/dd/de/df5/d20/dee/ff6 0 2024-09-21T04:35:13.567 INFO:tasks.workunit.client.0.smithi007.stdout:6/61: creat d3/f15 x:0 0 0 2024-09-21T04:35:13.567 INFO:tasks.workunit.client.1.smithi053.stdout:2/703: dwrite d1/d18/d1f/d27/d58/d5c/d91/dd0/fa4 [0,4194304] 0 2024-09-21T04:35:13.568 INFO:tasks.workunit.client.0.smithi007.stdout:6/62: dread - d3/f15 zero size 2024-09-21T04:35:13.568 INFO:tasks.workunit.client.1.smithi053.stdout:4/744: truncate d1/dd/d62/ff2 227097 0 2024-09-21T04:35:13.569 INFO:tasks.workunit.client.0.smithi007.stdout:1/199: link d5/f15 d5/d2f/f3a 0 2024-09-21T04:35:13.572 INFO:tasks.workunit.client.1.smithi053.stdout:8/704: rmdir d3/d5/d38/d64/d73/d91 39 2024-09-21T04:35:13.575 INFO:tasks.workunit.client.1.smithi053.stdout:7/573: mkdir d0/d2b/d36/d48/d5c/dac/dbd 0 2024-09-21T04:35:13.575 INFO:tasks.workunit.client.0.smithi007.stdout:1/200: dread d5/f28 [0,4194304] 0 2024-09-21T04:35:13.578 INFO:tasks.workunit.client.0.smithi007.stdout:5/269: creat d2/d1e/f58 x:0 0 0 2024-09-21T04:35:13.580 INFO:tasks.workunit.client.1.smithi053.stdout:8/705: dwrite d3/d4f/fe9 [0,4194304] 0 2024-09-21T04:35:13.580 INFO:tasks.workunit.client.0.smithi007.stdout:4/157: creat d2/f38 x:0 0 0 2024-09-21T04:35:13.581 INFO:tasks.workunit.client.1.smithi053.stdout:8/706: fsync d3/d5/d4c/d53/f83 0 2024-09-21T04:35:13.583 INFO:tasks.workunit.client.0.smithi007.stdout:0/79: dwrite f4 [0,4194304] 0 2024-09-21T04:35:13.583 INFO:tasks.workunit.client.1.smithi053.stdout:5/720: write d0/d8/f69 [1915328,40610] 0 2024-09-21T04:35:13.583 INFO:tasks.workunit.client.1.smithi053.stdout:6/671: write d2/d1b/d3d/d49/de/f7f [1366717,95266] 0 2024-09-21T04:35:13.585 INFO:tasks.workunit.client.0.smithi007.stdout:0/80: fsync d6/f14 0 2024-09-21T04:35:13.587 INFO:tasks.workunit.client.1.smithi053.stdout:1/682: creat d7/d47/d79/d99/fe4 x:0 0 0 2024-09-21T04:35:13.587 INFO:tasks.workunit.client.0.smithi007.stdout:1/201: dwrite d5/d6/d21/f2c [0,4194304] 0 2024-09-21T04:35:13.588 INFO:tasks.workunit.client.0.smithi007.stdout:6/63: mknod d3/de/c16 0 2024-09-21T04:35:13.598 INFO:tasks.workunit.client.1.smithi053.stdout:0/646: creat d5/d10/d4c/db1/dbc/dca/d21/fe0 x:0 0 0 2024-09-21T04:35:13.601 INFO:tasks.workunit.client.1.smithi053.stdout:4/745: unlink d1/dd/f1d 0 2024-09-21T04:35:13.603 INFO:tasks.workunit.client.1.smithi053.stdout:2/704: chown d1/d18/d77/lcb 0 1 2024-09-21T04:35:13.604 INFO:tasks.workunit.client.0.smithi007.stdout:8/64: symlink d0/d3/d8/l14 0 2024-09-21T04:35:13.605 INFO:tasks.workunit.client.1.smithi053.stdout:4/746: read d1/dd/de/df5/d20/d4e/d51/da4/df1/f40 [11398530,54916] 0 2024-09-21T04:35:13.608 INFO:tasks.workunit.client.0.smithi007.stdout:0/81: mknod d6/c1c 0 2024-09-21T04:35:13.608 INFO:tasks.workunit.client.0.smithi007.stdout:3/60: sync 2024-09-21T04:35:13.609 INFO:tasks.workunit.client.0.smithi007.stdout:2/102: sync 2024-09-21T04:35:13.609 INFO:tasks.workunit.client.0.smithi007.stdout:3/61: read - dc/f10 zero size 2024-09-21T04:35:13.618 INFO:tasks.workunit.client.0.smithi007.stdout:4/158: write d2/f23 [486877,67249] 0 2024-09-21T04:35:13.618 INFO:tasks.workunit.client.0.smithi007.stdout:4/159: chown d2/d7 53306983 1 2024-09-21T04:35:13.624 INFO:tasks.workunit.client.0.smithi007.stdout:0/82: rmdir d6/d16/d18 39 2024-09-21T04:35:13.624 INFO:tasks.workunit.client.1.smithi053.stdout:7/574: write d0/d2b/d36/d48/d50/d8b/f8 [1661497,79887] 0 2024-09-21T04:35:13.625 INFO:tasks.workunit.client.0.smithi007.stdout:4/160: dwrite d2/d7/ff [0,4194304] 0 2024-09-21T04:35:13.625 INFO:tasks.workunit.client.1.smithi053.stdout:7/575: fsync d0/d24/f5a 0 2024-09-21T04:35:13.626 INFO:tasks.workunit.client.1.smithi053.stdout:7/576: dread - d0/d2b/d36/d48/d50/fa5 zero size 2024-09-21T04:35:13.627 INFO:tasks.workunit.client.0.smithi007.stdout:4/161: write d2/d7/fa [1057840,52617] 0 2024-09-21T04:35:13.627 INFO:tasks.workunit.client.0.smithi007.stdout:3/62: rmdir dc 39 2024-09-21T04:35:13.627 INFO:tasks.workunit.client.0.smithi007.stdout:4/162: fsync d2/f27 0 2024-09-21T04:35:13.628 INFO:tasks.workunit.client.1.smithi053.stdout:7/577: fdatasync d0/d2b/d36/d48/d5c/f85 0 2024-09-21T04:35:13.628 INFO:tasks.workunit.client.0.smithi007.stdout:3/63: write fb [4247451,3525] 0 2024-09-21T04:35:13.630 INFO:tasks.workunit.client.0.smithi007.stdout:7/160: sync 2024-09-21T04:35:13.630 INFO:tasks.workunit.client.1.smithi053.stdout:4/747: mknod d1/d27/d33/d41/cf7 0 2024-09-21T04:35:13.631 INFO:tasks.workunit.client.0.smithi007.stdout:2/103: truncate d0/d2/fc 862952 0 2024-09-21T04:35:13.631 INFO:tasks.workunit.client.0.smithi007.stdout:2/104: readlink d0/d2/l14 0 2024-09-21T04:35:13.634 INFO:tasks.workunit.client.1.smithi053.stdout:0/647: write d5/d10/fb9 [1598761,86158] 0 2024-09-21T04:35:13.635 INFO:tasks.workunit.client.1.smithi053.stdout:8/707: truncate d3/d5/d38/f3d 802345 0 2024-09-21T04:35:13.638 INFO:tasks.workunit.client.1.smithi053.stdout:5/721: mkdir d0/dd/d48/d8c/dad/db9/deb 0 2024-09-21T04:35:13.639 INFO:tasks.workunit.client.0.smithi007.stdout:5/270: dwrite d2/f1b [0,4194304] 0 2024-09-21T04:35:13.639 INFO:tasks.workunit.client.1.smithi053.stdout:5/722: chown d0/dd/d5c/d92/ddb/d35/d61 2 1 2024-09-21T04:35:13.640 INFO:tasks.workunit.client.0.smithi007.stdout:5/271: readlink d2/dd/d23/l29 0 2024-09-21T04:35:13.641 INFO:tasks.workunit.client.1.smithi053.stdout:3/664: rename d16/d17/d21/d26/d49/fa6 to d16/d17/ff6 0 2024-09-21T04:35:13.645 INFO:tasks.workunit.client.1.smithi053.stdout:3/665: truncate d16/d17/d21/d26/fa7 374741 0 2024-09-21T04:35:13.646 INFO:tasks.workunit.client.1.smithi053.stdout:1/683: dwrite d7/d9/da/d3c/f46 [0,4194304] 0 2024-09-21T04:35:13.649 INFO:tasks.workunit.client.0.smithi007.stdout:9/126: dwrite f7 [0,4194304] 0 2024-09-21T04:35:13.650 INFO:tasks.workunit.client.1.smithi053.stdout:3/666: write d16/d17/d21/d42/d43/d86/fb3 [5068709,99089] 0 2024-09-21T04:35:13.651 INFO:tasks.workunit.client.0.smithi007.stdout:5/272: dwrite d2/dd/d1d/f39 [0,4194304] 0 2024-09-21T04:35:13.663 INFO:tasks.workunit.client.1.smithi053.stdout:4/748: mkdir d1/d27/df8 0 2024-09-21T04:35:13.665 INFO:tasks.workunit.client.1.smithi053.stdout:0/648: dread d5/de/d42/f48 [0,4194304] 0 2024-09-21T04:35:13.665 INFO:tasks.workunit.client.0.smithi007.stdout:1/202: link d5/d6/c36 d5/d6/d21/d2b/c3b 0 2024-09-21T04:35:13.665 INFO:tasks.workunit.client.0.smithi007.stdout:1/203: stat d5/c13 0 2024-09-21T04:35:13.668 INFO:tasks.workunit.client.0.smithi007.stdout:1/204: dread d5/fe [0,4194304] 0 2024-09-21T04:35:13.668 INFO:tasks.workunit.client.1.smithi053.stdout:4/749: dread - d1/dd/de/df5/d20/d73/d92/f9e zero size 2024-09-21T04:35:13.668 INFO:tasks.workunit.client.0.smithi007.stdout:8/65: creat d0/f15 x:0 0 0 2024-09-21T04:35:13.670 INFO:tasks.workunit.client.1.smithi053.stdout:0/649: truncate d5/d10/d4c/db1/dbc/dca/d21/fc4 493928 0 2024-09-21T04:35:13.671 INFO:tasks.workunit.client.1.smithi053.stdout:5/723: fdatasync d0/f3b 0 2024-09-21T04:35:13.673 INFO:tasks.workunit.client.0.smithi007.stdout:0/83: creat d6/d16/d18/f1d x:0 0 0 2024-09-21T04:35:13.674 INFO:tasks.workunit.client.0.smithi007.stdout:0/84: write f1 [3869790,104681] 0 2024-09-21T04:35:13.674 INFO:tasks.workunit.client.1.smithi053.stdout:8/708: dwrite d3/d5/d4c/dc3/fcc [0,4194304] 0 2024-09-21T04:35:13.675 INFO:tasks.workunit.client.1.smithi053.stdout:1/684: chown d7/da8/f83 64616195 1 2024-09-21T04:35:13.676 INFO:tasks.workunit.client.1.smithi053.stdout:8/709: write d3/d5/d4b/f7e [1127741,25952] 0 2024-09-21T04:35:13.676 INFO:tasks.workunit.client.1.smithi053.stdout:1/685: chown d7/d63/d96/dac/fd4 3 1 2024-09-21T04:35:13.677 INFO:tasks.workunit.client.0.smithi007.stdout:9/127: rmdir d13/d18 39 2024-09-21T04:35:13.679 INFO:tasks.workunit.client.0.smithi007.stdout:6/64: getdents d3 0 2024-09-21T04:35:13.683 INFO:tasks.workunit.client.0.smithi007.stdout:3/64: rename f8 to dc/f13 0 2024-09-21T04:35:13.687 INFO:tasks.workunit.client.0.smithi007.stdout:2/105: fdatasync d0/d2/fc 0 2024-09-21T04:35:13.689 INFO:tasks.workunit.client.0.smithi007.stdout:8/66: truncate d0/d3/d8/f10 806786 0 2024-09-21T04:35:13.690 INFO:tasks.workunit.client.0.smithi007.stdout:1/205: mkdir d5/d6/d21/d3c 0 2024-09-21T04:35:13.691 INFO:tasks.workunit.client.1.smithi053.stdout:0/650: symlink d5/de/d1c/da3/le1 0 2024-09-21T04:35:13.692 INFO:tasks.workunit.client.0.smithi007.stdout:4/163: truncate d2/d7/f8 1076959 0 2024-09-21T04:35:13.692 INFO:tasks.workunit.client.1.smithi053.stdout:0/651: write d5/f97 [782809,99238] 0 2024-09-21T04:35:13.696 INFO:tasks.workunit.client.0.smithi007.stdout:5/273: truncate d2/dd/d1d/f4b 224822 0 2024-09-21T04:35:13.698 INFO:tasks.workunit.client.1.smithi053.stdout:1/686: stat d7/d9/d43/d54/d7f/d59/dc3/fca 0 2024-09-21T04:35:13.701 INFO:tasks.workunit.client.0.smithi007.stdout:7/161: link d3/d6/d10/c1d d3/d6/d14/d20/c36 0 2024-09-21T04:35:13.701 INFO:tasks.workunit.client.0.smithi007.stdout:6/65: dwrite d3/fc [0,4194304] 0 2024-09-21T04:35:13.702 INFO:tasks.workunit.client.1.smithi053.stdout:2/705: getdents d1/d18/d1f/d27/d41 0 2024-09-21T04:35:13.704 INFO:tasks.workunit.client.1.smithi053.stdout:7/578: rmdir d0/d2b/d36/d48/d50/d84/db1 0 2024-09-21T04:35:13.710 INFO:tasks.workunit.client.1.smithi053.stdout:9/631: rename d0/df/l11 to d0/d2/d9/ld8 0 2024-09-21T04:35:13.710 INFO:tasks.workunit.client.0.smithi007.stdout:0/85: truncate d6/f13 243623 0 2024-09-21T04:35:13.711 INFO:tasks.workunit.client.0.smithi007.stdout:0/86: write f1 [2882266,16259] 0 2024-09-21T04:35:13.713 INFO:tasks.workunit.client.0.smithi007.stdout:9/128: mkdir d13/d2a 0 2024-09-21T04:35:13.714 INFO:tasks.workunit.client.1.smithi053.stdout:3/667: dwrite d16/d17/ff6 [0,4194304] 0 2024-09-21T04:35:13.715 INFO:tasks.workunit.client.0.smithi007.stdout:4/164: write d2/f6 [1218754,3184] 0 2024-09-21T04:35:13.716 INFO:tasks.workunit.client.0.smithi007.stdout:5/274: rename d2/dd/d1d/f3f to d2/dd/d23/d49/d44/d54/f59 0 2024-09-21T04:35:13.717 INFO:tasks.workunit.client.1.smithi053.stdout:0/652: symlink d5/d10/d4c/db1/dbc/dca/db5/d99/le2 0 2024-09-21T04:35:13.720 INFO:tasks.workunit.client.1.smithi053.stdout:8/710: symlink d3/d5/d21/lea 0 2024-09-21T04:35:13.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/162: creat d3/d6/d14/d20/f37 x:0 0 0 2024-09-21T04:35:13.722 INFO:tasks.workunit.client.0.smithi007.stdout:7/163: read - d3/d6/f2b zero size 2024-09-21T04:35:13.723 INFO:tasks.workunit.client.0.smithi007.stdout:7/164: dread - d3/d6/d14/d20/f35 zero size 2024-09-21T04:35:13.725 INFO:tasks.workunit.client.1.smithi053.stdout:5/724: dwrite d0/d8/f1f [0,4194304] 0 2024-09-21T04:35:13.727 INFO:tasks.workunit.client.1.smithi053.stdout:5/725: truncate d0/d8/db/fbf 626238 0 2024-09-21T04:35:13.728 INFO:tasks.workunit.client.1.smithi053.stdout:2/706: unlink d1/d18/d1f/d27/d58/d5c/d91/dc1/fd9 0 2024-09-21T04:35:13.728 INFO:tasks.workunit.client.0.smithi007.stdout:3/65: write dc/f13 [4221325,97356] 0 2024-09-21T04:35:13.729 INFO:tasks.workunit.client.0.smithi007.stdout:3/66: write fb [8279760,58051] 0 2024-09-21T04:35:13.729 INFO:tasks.workunit.client.1.smithi053.stdout:4/750: dread d1/d27/f89 [4194304,4194304] 0 2024-09-21T04:35:13.730 INFO:tasks.workunit.client.1.smithi053.stdout:7/579: creat d0/d2b/d36/d48/d95/fbe x:0 0 0 2024-09-21T04:35:13.731 INFO:tasks.workunit.client.0.smithi007.stdout:8/67: mkdir d0/d3/d16 0 2024-09-21T04:35:13.736 INFO:tasks.workunit.client.1.smithi053.stdout:6/672: rename d2/d1b/ld7 to d2/d1b/d3d/d49/de/d32/d80/dcd/ldb 0 2024-09-21T04:35:13.737 INFO:tasks.workunit.client.0.smithi007.stdout:2/106: dwrite d0/f3 [0,4194304] 0 2024-09-21T04:35:13.741 INFO:tasks.workunit.client.1.smithi053.stdout:7/580: dwrite d0/d2b/d36/d48/d50/d84/f98 [0,4194304] 0 2024-09-21T04:35:13.742 INFO:tasks.workunit.client.1.smithi053.stdout:0/653: fdatasync d5/de/d42/d78/f6f 0 2024-09-21T04:35:13.743 INFO:tasks.workunit.client.1.smithi053.stdout:7/581: fsync d0/d2b/d36/d48/d50/d8b/f42 0 2024-09-21T04:35:13.743 INFO:tasks.workunit.client.1.smithi053.stdout:1/687: truncate d7/d9/d43/d54/d7f/d59/dc3/fca 203645 0 2024-09-21T04:35:13.743 INFO:tasks.workunit.client.1.smithi053.stdout:0/654: chown d5/de/l27 0 1 2024-09-21T04:35:13.744 INFO:tasks.workunit.client.0.smithi007.stdout:4/165: mknod d2/d7/c39 0 2024-09-21T04:35:13.745 INFO:tasks.workunit.client.1.smithi053.stdout:8/711: truncate d3/d39/f87 319078 0 2024-09-21T04:35:13.745 INFO:tasks.workunit.client.0.smithi007.stdout:4/166: write d2/d17/f2a [589109,18660] 0 2024-09-21T04:35:13.746 INFO:tasks.workunit.client.1.smithi053.stdout:8/712: dread - d3/d5/d21/fe2 zero size 2024-09-21T04:35:13.746 INFO:tasks.workunit.client.0.smithi007.stdout:5/275: unlink d2/dd/d23/d49/d44/f48 0 2024-09-21T04:35:13.747 INFO:tasks.workunit.client.1.smithi053.stdout:8/713: fdatasync d3/df/d49/f55 0 2024-09-21T04:35:13.747 INFO:tasks.workunit.client.1.smithi053.stdout:0/655: write d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/fd9 [331674,1332] 0 2024-09-21T04:35:13.748 INFO:tasks.workunit.client.0.smithi007.stdout:4/167: write d2/d17/d18/f1f [4477820,118193] 0 2024-09-21T04:35:13.748 INFO:tasks.workunit.client.0.smithi007.stdout:4/168: stat d2/d7/ce 0 2024-09-21T04:35:13.752 INFO:tasks.workunit.client.1.smithi053.stdout:4/751: creat d1/dd/de/df5/d20/d4e/d51/da4/ff9 x:0 0 0 2024-09-21T04:35:13.755 INFO:tasks.workunit.client.1.smithi053.stdout:4/752: read d1/dd/de/df5/d20/f34 [4110001,34019] 0 2024-09-21T04:35:13.755 INFO:tasks.workunit.client.0.smithi007.stdout:4/169: dwrite d2/d17/d18/f1e [0,4194304] 0 2024-09-21T04:35:13.756 INFO:tasks.workunit.client.0.smithi007.stdout:7/165: rename d3/d6/f2b to d3/d6/d2e/f38 0 2024-09-21T04:35:13.758 INFO:tasks.workunit.client.1.smithi053.stdout:2/707: creat d1/d18/d1f/d22/d2d/dcd/dcf/f103 x:0 0 0 2024-09-21T04:35:13.759 INFO:tasks.workunit.client.1.smithi053.stdout:2/708: write d1/d18/f5b [4298959,60394] 0 2024-09-21T04:35:13.763 INFO:tasks.workunit.client.1.smithi053.stdout:9/632: creat d0/d1c/dcb/fd9 x:0 0 0 2024-09-21T04:35:13.765 INFO:tasks.workunit.client.1.smithi053.stdout:5/726: rename d0/dd/d67/fa3 to d0/dd/d5c/d92/de8/fec 0 2024-09-21T04:35:13.767 INFO:tasks.workunit.client.1.smithi053.stdout:6/673: chown d2/d1b/d1e/d2a/l86 319337746 1 2024-09-21T04:35:13.767 INFO:tasks.workunit.client.1.smithi053.stdout:5/727: stat d0/dd/d48/d8c/dad/db9 0 2024-09-21T04:35:13.768 INFO:tasks.workunit.client.1.smithi053.stdout:6/674: read d2/d1b/d1e/fcb [3938828,13572] 0 2024-09-21T04:35:13.769 INFO:tasks.workunit.client.0.smithi007.stdout:8/68: creat d0/d3/d8/de/f17 x:0 0 0 2024-09-21T04:35:13.771 INFO:tasks.workunit.client.1.smithi053.stdout:3/668: symlink d16/d17/d21/d8b/d7d/d57/lf7 0 2024-09-21T04:35:13.771 INFO:tasks.workunit.client.0.smithi007.stdout:2/107: fdatasync d0/d2/f16 0 2024-09-21T04:35:13.780 INFO:tasks.workunit.client.0.smithi007.stdout:3/67: dwrite dc/fe [4194304,4194304] 0 2024-09-21T04:35:13.781 INFO:tasks.workunit.client.0.smithi007.stdout:3/68: readlink dc/lf 0 2024-09-21T04:35:13.781 INFO:tasks.workunit.client.0.smithi007.stdout:3/69: fsync fb 0 2024-09-21T04:35:13.783 INFO:tasks.workunit.client.0.smithi007.stdout:3/70: write dc/f10 [438035,75204] 0 2024-09-21T04:35:13.784 INFO:tasks.workunit.client.0.smithi007.stdout:7/166: rmdir d3 39 2024-09-21T04:35:13.785 INFO:tasks.workunit.client.1.smithi053.stdout:8/714: symlink d3/d28/leb 0 2024-09-21T04:35:13.785 INFO:tasks.workunit.client.0.smithi007.stdout:4/170: dread d2/f1d [0,4194304] 0 2024-09-21T04:35:13.786 INFO:tasks.workunit.client.1.smithi053.stdout:4/753: fsync d1/dd/de/df5/d20/dee/ff6 0 2024-09-21T04:35:13.786 INFO:tasks.workunit.client.0.smithi007.stdout:1/206: getdents d5 0 2024-09-21T04:35:13.787 INFO:tasks.workunit.client.1.smithi053.stdout:8/715: stat d3/df/d14/la8 0 2024-09-21T04:35:13.787 INFO:tasks.workunit.client.0.smithi007.stdout:8/69: creat d0/d3/d8/de/f18 x:0 0 0 2024-09-21T04:35:13.788 INFO:tasks.workunit.client.0.smithi007.stdout:4/171: write d2/d17/d18/f2d [1060889,121549] 0 2024-09-21T04:35:13.789 INFO:tasks.workunit.client.1.smithi053.stdout:2/709: creat d1/d18/d1f/d22/d43/f104 x:0 0 0 2024-09-21T04:35:13.790 INFO:tasks.workunit.client.0.smithi007.stdout:2/108: symlink d0/d11/l1b 0 2024-09-21T04:35:13.792 INFO:tasks.workunit.client.0.smithi007.stdout:0/87: creat d6/d16/f1e x:0 0 0 2024-09-21T04:35:13.795 INFO:tasks.workunit.client.0.smithi007.stdout:4/172: dread d2/d7/ff [0,4194304] 0 2024-09-21T04:35:13.796 INFO:tasks.workunit.client.1.smithi053.stdout:4/754: dwrite d1/dd/de/df5/d3f/d74/f8b [0,4194304] 0 2024-09-21T04:35:13.797 INFO:tasks.workunit.client.0.smithi007.stdout:8/70: dread d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:13.798 INFO:tasks.workunit.client.1.smithi053.stdout:8/716: dread d3/df/dba/fca [0,4194304] 0 2024-09-21T04:35:13.798 INFO:tasks.workunit.client.1.smithi053.stdout:7/582: rename d0/d2b/d36/d48 to d0/d6f/dbf 0 2024-09-21T04:35:13.798 INFO:tasks.workunit.client.0.smithi007.stdout:4/173: stat d2/l21 0 2024-09-21T04:35:13.799 INFO:tasks.workunit.client.0.smithi007.stdout:8/71: readlink d0/d3/d8/lc 0 2024-09-21T04:35:13.799 INFO:tasks.workunit.client.0.smithi007.stdout:6/66: getdents d3/de 0 2024-09-21T04:35:13.800 INFO:tasks.workunit.client.0.smithi007.stdout:4/174: write d2/d17/f1b [889822,51935] 0 2024-09-21T04:35:13.801 INFO:tasks.workunit.client.1.smithi053.stdout:5/728: mknod d0/dd/d5c/d92/ddb/d35/ced 0 2024-09-21T04:35:13.801 INFO:tasks.workunit.client.0.smithi007.stdout:8/72: fdatasync d0/d3/d8/de/f18 0 2024-09-21T04:35:13.802 INFO:tasks.workunit.client.1.smithi053.stdout:5/729: chown d0/dd/d67/ld1 16444 1 2024-09-21T04:35:13.803 INFO:tasks.workunit.client.0.smithi007.stdout:0/88: dwrite d6/f12 [0,4194304] 0 2024-09-21T04:35:13.804 INFO:tasks.workunit.client.0.smithi007.stdout:7/167: stat d3/d6/d2e/l31 0 2024-09-21T04:35:13.805 INFO:tasks.workunit.client.0.smithi007.stdout:8/73: dread d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:13.810 INFO:tasks.workunit.client.0.smithi007.stdout:1/207: mknod d5/d6/d21/d2b/c3d 0 2024-09-21T04:35:13.812 INFO:tasks.workunit.client.1.smithi053.stdout:1/688: mkdir d7/d9/da/d11/de5 0 2024-09-21T04:35:13.815 INFO:tasks.workunit.client.0.smithi007.stdout:8/74: dwrite d0/d3/d8/f12 [0,4194304] 0 2024-09-21T04:35:13.817 INFO:tasks.workunit.client.1.smithi053.stdout:3/669: write d16/d17/d21/d8b/d7d/d57/dc1/f70 [322685,25706] 0 2024-09-21T04:35:13.822 INFO:tasks.workunit.client.1.smithi053.stdout:1/689: dwrite d7/d9/d43/d54/d7f/d36/fd3 [0,4194304] 0 2024-09-21T04:35:13.822 INFO:tasks.workunit.client.0.smithi007.stdout:5/276: link d2/dd/d23/d49/d44/d54/f59 d2/f5a 0 2024-09-21T04:35:13.825 INFO:tasks.workunit.client.0.smithi007.stdout:6/67: creat d3/de/d12/f17 x:0 0 0 2024-09-21T04:35:13.826 INFO:tasks.workunit.client.0.smithi007.stdout:6/68: write d3/de/f10 [256463,111609] 0 2024-09-21T04:35:13.828 INFO:tasks.workunit.client.0.smithi007.stdout:0/89: symlink d6/d16/d18/l1f 0 2024-09-21T04:35:13.829 INFO:tasks.workunit.client.1.smithi053.stdout:3/670: dwrite d16/d17/d21/d8b/d7d/d57/dc1/f70 [0,4194304] 0 2024-09-21T04:35:13.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/90: write d6/fd [1923258,51632] 0 2024-09-21T04:35:13.832 INFO:tasks.workunit.client.0.smithi007.stdout:1/208: rename f4 to d5/f3e 0 2024-09-21T04:35:13.832 INFO:tasks.workunit.client.0.smithi007.stdout:1/209: readlink d5/l1e 0 2024-09-21T04:35:13.832 INFO:tasks.workunit.client.0.smithi007.stdout:1/210: dread - d5/f25 zero size 2024-09-21T04:35:13.834 INFO:tasks.workunit.client.0.smithi007.stdout:1/211: dread d5/d2f/f34 [0,4194304] 0 2024-09-21T04:35:13.837 INFO:tasks.workunit.client.0.smithi007.stdout:8/75: rmdir d0/d3 39 2024-09-21T04:35:13.837 INFO:tasks.workunit.client.0.smithi007.stdout:1/212: dread d5/fe [0,4194304] 0 2024-09-21T04:35:13.840 INFO:tasks.workunit.client.0.smithi007.stdout:1/213: stat d5/d6/d21/d23/f24 0 2024-09-21T04:35:13.840 INFO:tasks.workunit.client.0.smithi007.stdout:5/277: write d2/f55 [62275,124847] 0 2024-09-21T04:35:13.842 INFO:tasks.workunit.client.0.smithi007.stdout:5/278: write d2/dd/d1d/f1f [2099635,107850] 0 2024-09-21T04:35:13.845 INFO:tasks.workunit.client.0.smithi007.stdout:6/69: creat d3/f18 x:0 0 0 2024-09-21T04:35:13.846 INFO:tasks.workunit.client.0.smithi007.stdout:6/70: dread - d3/de/d12/f14 zero size 2024-09-21T04:35:13.847 INFO:tasks.workunit.client.0.smithi007.stdout:5/279: dread d2/f1b [0,4194304] 0 2024-09-21T04:35:13.853 INFO:tasks.workunit.client.1.smithi053.stdout:1/690: dread d7/d63/f6a [0,4194304] 0 2024-09-21T04:35:13.855 INFO:tasks.workunit.client.0.smithi007.stdout:5/280: dwrite d2/dd/d1d/f39 [0,4194304] 0 2024-09-21T04:35:13.858 INFO:tasks.workunit.client.1.smithi053.stdout:9/633: creat d0/d1c/d52/db5/fda x:0 0 0 2024-09-21T04:35:13.861 INFO:tasks.workunit.client.1.smithi053.stdout:2/710: creat d1/d18/d1f/da2/dac/f105 x:0 0 0 2024-09-21T04:35:13.862 INFO:tasks.workunit.client.0.smithi007.stdout:2/109: creat d0/f1c x:0 0 0 2024-09-21T04:35:13.862 INFO:tasks.workunit.client.0.smithi007.stdout:7/168: write d3/d6/f7 [1970555,62270] 0 2024-09-21T04:35:13.863 INFO:tasks.workunit.client.0.smithi007.stdout:2/110: dread - d0/f1c zero size 2024-09-21T04:35:13.867 INFO:tasks.workunit.client.0.smithi007.stdout:1/214: creat d5/d6/d21/f3f x:0 0 0 2024-09-21T04:35:13.868 INFO:tasks.workunit.client.1.smithi053.stdout:4/755: mknod d1/dd/d62/cfa 0 2024-09-21T04:35:13.872 INFO:tasks.workunit.client.0.smithi007.stdout:3/71: getdents dc 0 2024-09-21T04:35:13.872 INFO:tasks.workunit.client.0.smithi007.stdout:2/111: dwrite d0/fd [0,4194304] 0 2024-09-21T04:35:13.877 INFO:tasks.workunit.client.0.smithi007.stdout:6/71: mkdir d3/de/d12/d19 0 2024-09-21T04:35:13.879 INFO:tasks.workunit.client.1.smithi053.stdout:6/675: symlink d2/ldc 0 2024-09-21T04:35:13.880 INFO:tasks.workunit.client.0.smithi007.stdout:2/112: dread d0/fd [0,4194304] 0 2024-09-21T04:35:13.883 INFO:tasks.workunit.client.0.smithi007.stdout:5/281: creat d2/d1e/f5b x:0 0 0 2024-09-21T04:35:13.884 INFO:tasks.workunit.client.0.smithi007.stdout:5/282: truncate d2/f50 189072 0 2024-09-21T04:35:13.888 INFO:tasks.workunit.client.0.smithi007.stdout:8/76: symlink d0/l19 0 2024-09-21T04:35:13.890 INFO:tasks.workunit.client.0.smithi007.stdout:7/169: creat d3/d6/d13/f39 x:0 0 0 2024-09-21T04:35:13.896 INFO:tasks.workunit.client.1.smithi053.stdout:9/634: symlink d0/d1a/d89/da1/ldb 0 2024-09-21T04:35:13.897 INFO:tasks.workunit.client.1.smithi053.stdout:9/635: readlink d0/d1c/d52/d53/l90 0 2024-09-21T04:35:13.898 INFO:tasks.workunit.client.1.smithi053.stdout:2/711: mkdir d1/d18/d1f/d22/d2d/d106 0 2024-09-21T04:35:13.900 INFO:tasks.workunit.client.0.smithi007.stdout:6/72: unlink d3/f5 0 2024-09-21T04:35:13.900 INFO:tasks.workunit.client.0.smithi007.stdout:6/73: dread - d3/f18 zero size 2024-09-21T04:35:13.901 INFO:tasks.workunit.client.1.smithi053.stdout:4/756: mknod d1/dd/de/df5/d20/d73/cfb 0 2024-09-21T04:35:13.903 INFO:tasks.workunit.client.1.smithi053.stdout:4/757: fsync d1/dd/de/df5/d20/d4e/d51/da4/df1/d8e/db4/fba 0 2024-09-21T04:35:13.908 INFO:tasks.workunit.client.0.smithi007.stdout:8/77: chown d0/f6 1 1 2024-09-21T04:35:13.908 INFO:tasks.workunit.client.0.smithi007.stdout:2/113: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:13.911 INFO:tasks.workunit.client.0.smithi007.stdout:7/170: symlink d3/d6/d10/l3a 0 2024-09-21T04:35:13.912 INFO:tasks.workunit.client.0.smithi007.stdout:2/114: dread d0/fd [0,4194304] 0 2024-09-21T04:35:13.913 INFO:tasks.workunit.client.0.smithi007.stdout:7/171: write d3/d6/d13/f2c [318636,96338] 0 2024-09-21T04:35:13.913 INFO:tasks.workunit.client.1.smithi053.stdout:7/583: truncate d0/d24/d46/f7d 3068754 0 2024-09-21T04:35:13.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/72: getdents dc 0 2024-09-21T04:35:13.917 INFO:tasks.workunit.client.1.smithi053.stdout:0/656: getdents d5/dd/d45 0 2024-09-21T04:35:13.920 INFO:tasks.workunit.client.1.smithi053.stdout:9/636: mknod d0/df/cdc 0 2024-09-21T04:35:13.921 INFO:tasks.workunit.client.0.smithi007.stdout:5/283: dwrite d2/f19 [0,4194304] 0 2024-09-21T04:35:13.921 INFO:tasks.workunit.client.0.smithi007.stdout:0/91: getdents d6 0 2024-09-21T04:35:13.922 INFO:tasks.workunit.client.1.smithi053.stdout:2/712: creat d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/f107 x:0 0 0 2024-09-21T04:35:13.923 INFO:tasks.workunit.client.0.smithi007.stdout:5/284: write d2/d1e/f22 [1672433,82264] 0 2024-09-21T04:35:13.927 INFO:tasks.workunit.client.1.smithi053.stdout:9/637: dwrite d0/d2/d9/fcc [0,4194304] 0 2024-09-21T04:35:13.930 INFO:tasks.workunit.client.1.smithi053.stdout:5/730: creat d0/dd/d5c/d92/ddb/fee x:0 0 0 2024-09-21T04:35:13.935 INFO:tasks.workunit.client.1.smithi053.stdout:6/676: link d2/d1b/d1e/d2a/d4b/d77/d88/fab d2/d1b/d3d/d49/de/d32/d80/dcd/fdd 0 2024-09-21T04:35:13.936 INFO:tasks.workunit.client.1.smithi053.stdout:6/677: chown d2/d1b/d1e/d27/d41 264 1 2024-09-21T04:35:13.937 INFO:tasks.workunit.client.0.smithi007.stdout:3/73: rmdir dc 39 2024-09-21T04:35:13.937 INFO:tasks.workunit.client.1.smithi053.stdout:6/678: chown d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/dd5 111949714 1 2024-09-21T04:35:13.938 INFO:tasks.workunit.client.0.smithi007.stdout:3/74: chown fb 638458127 1 2024-09-21T04:35:13.939 INFO:tasks.workunit.client.1.smithi053.stdout:7/584: mkdir d0/d2b/d36/d64/dc0 0 2024-09-21T04:35:13.941 INFO:tasks.workunit.client.1.smithi053.stdout:3/671: link d16/d17/d21/d26/c69 d16/d17/d21/d8b/d7d/d57/dc1/d53/cf8 0 2024-09-21T04:35:13.942 INFO:tasks.workunit.client.1.smithi053.stdout:3/672: write d16/d17/d21/d8b/d7d/d57/fdd [400997,30374] 0 2024-09-21T04:35:13.944 INFO:tasks.workunit.client.1.smithi053.stdout:3/673: chown d16/d17/d21/d8b/d7d/d57/dc1/d53/fa1 0 1 2024-09-21T04:35:13.944 INFO:tasks.workunit.client.1.smithi053.stdout:0/657: rename d5/d47/ca2 to d5/de/d9e/dd2/ce3 0 2024-09-21T04:35:13.945 INFO:tasks.workunit.client.0.smithi007.stdout:2/115: write d0/d2/f6 [1721110,113569] 0 2024-09-21T04:35:13.946 INFO:tasks.workunit.client.0.smithi007.stdout:2/116: chown d0/d2/c15 40259 1 2024-09-21T04:35:13.946 INFO:tasks.workunit.client.1.smithi053.stdout:1/691: link d7/d9/d43/d54/d7f/d36/d81/d95/ccd d7/d9/da/d3c/ce6 0 2024-09-21T04:35:13.947 INFO:tasks.workunit.client.0.smithi007.stdout:2/117: write d0/d11/f1a [468711,50375] 0 2024-09-21T04:35:13.948 INFO:tasks.workunit.client.0.smithi007.stdout:0/92: fdatasync d6/f13 0 2024-09-21T04:35:13.948 INFO:tasks.workunit.client.1.smithi053.stdout:1/692: dread - d7/d9/da/fd9 zero size 2024-09-21T04:35:13.950 INFO:tasks.workunit.client.0.smithi007.stdout:8/78: mknod d0/c1a 0 2024-09-21T04:35:13.951 INFO:tasks.workunit.client.0.smithi007.stdout:8/79: chown d0/f6 497 1 2024-09-21T04:35:13.952 INFO:tasks.workunit.client.1.smithi053.stdout:3/674: dwrite d16/d17/d21/d8b/d7d/f85 [4194304,4194304] 0 2024-09-21T04:35:13.953 INFO:tasks.workunit.client.0.smithi007.stdout:5/285: symlink d2/dd/d1d/l5c 0 2024-09-21T04:35:13.956 INFO:tasks.workunit.client.1.smithi053.stdout:8/717: link d3/d5/l15 d3/d5/d4c/d53/d66/lec 0 2024-09-21T04:35:13.958 INFO:tasks.workunit.client.0.smithi007.stdout:7/172: dwrite d3/d6/d14/d20/d33/f25 [0,4194304] 0 2024-09-21T04:35:13.963 INFO:tasks.workunit.client.1.smithi053.stdout:7/585: fdatasync d0/d6f/f72 0 2024-09-21T04:35:13.963 INFO:tasks.workunit.client.0.smithi007.stdout:3/75: dread dc/f10 [0,4194304] 0 2024-09-21T04:35:13.965 INFO:tasks.workunit.client.0.smithi007.stdout:3/76: write dc/f13 [2101883,60958] 0 2024-09-21T04:35:13.966 INFO:tasks.workunit.client.0.smithi007.stdout:0/93: creat d6/f20 x:0 0 0 2024-09-21T04:35:13.967 INFO:tasks.workunit.client.0.smithi007.stdout:3/77: write fa [2341950,66554] 0 2024-09-21T04:35:13.968 INFO:tasks.workunit.client.0.smithi007.stdout:8/80: chown d0/d3/d16 352 1 2024-09-21T04:35:13.969 INFO:tasks.workunit.client.0.smithi007.stdout:8/81: fsync d0/d3/d8/de/f18 0 2024-09-21T04:35:13.970 INFO:tasks.workunit.client.1.smithi053.stdout:9/638: rename d0/d2/d9/d34/c8d to d0/d2/d4e/d5c/cdd 0 2024-09-21T04:35:13.971 INFO:tasks.workunit.client.1.smithi053.stdout:0/658: mknod d5/d10/d4c/db1/dbc/dca/db5/ce4 0 2024-09-21T04:35:13.975 INFO:tasks.workunit.client.1.smithi053.stdout:1/693: creat d7/d47/d79/d99/db0/fe7 x:0 0 0 2024-09-21T04:35:13.975 INFO:tasks.workunit.client.1.smithi053.stdout:0/659: dread d5/de/d9e/faf [0,4194304] 0 2024-09-21T04:35:13.977 INFO:tasks.workunit.client.0.smithi007.stdout:6/74: getdents d3/de/d12 0 2024-09-21T04:35:13.984 INFO:tasks.workunit.client.0.smithi007.stdout:7/173: dwrite d3/d6/d14/d20/d33/f25 [4194304,4194304] 0 2024-09-21T04:35:13.984 INFO:tasks.workunit.client.1.smithi053.stdout:6/679: fdatasync d2/d1d/f50 0 2024-09-21T04:35:13.985 INFO:tasks.workunit.client.0.smithi007.stdout:7/174: chown d3/d6/d2e/f38 35566097 1 2024-09-21T04:35:13.986 INFO:tasks.workunit.client.1.smithi053.stdout:3/675: write fa [4885277,51816] 0 2024-09-21T04:35:13.986 INFO:tasks.workunit.client.0.smithi007.stdout:0/94: dread d6/f13 [0,4194304] 0 2024-09-21T04:35:13.988 INFO:tasks.workunit.client.1.smithi053.stdout:3/676: dread - d16/d17/d21/d8b/d7d/ff3 zero size 2024-09-21T04:35:13.989 INFO:tasks.workunit.client.0.smithi007.stdout:8/82: rmdir d0/d3/d8 39 2024-09-21T04:35:13.989 INFO:tasks.workunit.client.1.smithi053.stdout:7/586: mkdir d0/d6f/dbf/d50/d8b/d70/dc1 0 2024-09-21T04:35:13.992 INFO:tasks.workunit.client.1.smithi053.stdout:9/639: mknod d0/d2/d4e/d5c/cde 0 2024-09-21T04:35:13.992 INFO:tasks.workunit.client.1.smithi053.stdout:4/758: link d1/f2d d1/d27/d53/ffc 0 2024-09-21T04:35:13.993 INFO:tasks.workunit.client.1.smithi053.stdout:6/680: dwrite d2/d1b/d1e/d2a/f7c [0,4194304] 0 2024-09-21T04:35:13.994 INFO:tasks.workunit.client.1.smithi053.stdout:0/660: unlink d5/d10/l20 0 2024-09-21T04:35:13.995 INFO:tasks.workunit.client.1.smithi053.stdout:0/661: dread - d5/de/fd7 zero size 2024-09-21T04:35:13.996 INFO:tasks.workunit.client.1.smithi053.stdout:1/694: readlink d7/d9/d43/d54/d7f/l82 0 2024-09-21T04:35:13.998 INFO:tasks.workunit.client.0.smithi007.stdout:7/175: symlink d3/d6/d13/l3b 0 2024-09-21T04:35:13.999 INFO:tasks.workunit.client.0.smithi007.stdout:0/95: rename d6/d16 to d6/d16/d21 22 2024-09-21T04:35:14.000 INFO:tasks.workunit.client.0.smithi007.stdout:0/96: dread d6/f13 [0,4194304] 0 2024-09-21T04:35:14.002 INFO:tasks.workunit.client.1.smithi053.stdout:5/731: mkdir d0/dd/d5c/d92/ddb/d35/d61/def 0 2024-09-21T04:35:14.002 INFO:tasks.workunit.client.1.smithi053.stdout:4/759: dread d1/dd/de/df5/d20/d4e/d51/da4/df1/f85 [0,4194304] 0 2024-09-21T04:35:14.003 INFO:tasks.workunit.client.1.smithi053.stdout:0/662: dwrite d5/d10/d4c/db1/dbc/dca/f65 [0,4194304] 0 2024-09-21T04:35:14.003 INFO:tasks.workunit.client.0.smithi007.stdout:0/97: truncate d6/d16/f1e 737925 0 2024-09-21T04:35:14.005 INFO:tasks.workunit.client.1.smithi053.stdout:0/663: readlink d5/d10/d4c/db1/dbc/dca/d53/lb3 0 2024-09-21T04:35:14.005 INFO:tasks.workunit.client.0.smithi007.stdout:7/176: creat d3/d6/d13/f3c x:0 0 0 2024-09-21T04:35:14.010 INFO:tasks.workunit.client.0.smithi007.stdout:3/78: truncate fa 225321 0 2024-09-21T04:35:14.012 INFO:tasks.workunit.client.0.smithi007.stdout:5/286: getdents d2/dd/d23/d49 0 2024-09-21T04:35:14.013 INFO:tasks.workunit.client.1.smithi053.stdout:8/718: truncate d3/df/fb2 349879 0 2024-09-21T04:35:14.016 INFO:tasks.workunit.client.0.smithi007.stdout:7/177: symlink d3/d6/d14/l3d 0 2024-09-21T04:35:14.017 INFO:tasks.workunit.client.1.smithi053.stdout:7/587: write d0/f49 [1250127,61978] 0 2024-09-21T04:35:14.018 INFO:tasks.workunit.client.1.smithi053.stdout:2/713: getdents d1/d18/d1f/d22/d2d 0 2024-09-21T04:35:14.020 INFO:tasks.workunit.client.1.smithi053.stdout:5/732: creat d0/d8/db/d68/ff0 x:0 0 0 2024-09-21T04:35:14.021 INFO:tasks.workunit.client.1.smithi053.stdout:4/760: creat d1/dd/d62/ffd x:0 0 0 2024-09-21T04:35:14.022 INFO:tasks.workunit.client.1.smithi053.stdout:5/733: write d0/dd/f23 [621243,60578] 0 2024-09-21T04:35:14.026 INFO:tasks.workunit.client.1.smithi053.stdout:0/664: fdatasync d5/d10/d4c/db1/dbc/dca/d21/f33 0 2024-09-21T04:35:14.030 INFO:tasks.workunit.client.1.smithi053.stdout:3/677: rename d16/d17/d21/d8b/cd7 to d16/d17/d21/d8b/d7d/dc6/dec/cf9 0 2024-09-21T04:35:14.032 INFO:tasks.workunit.client.0.smithi007.stdout:5/287: mkdir d2/dd/d23/d5d 0 2024-09-21T04:35:14.033 INFO:tasks.workunit.client.1.smithi053.stdout:9/640: mkdir d0/d2/ddf 0 2024-09-21T04:35:14.033 INFO:tasks.workunit.client.0.smithi007.stdout:3/79: dwrite dc/f10 [0,4194304] 0 2024-09-21T04:35:14.035 INFO:tasks.workunit.client.1.smithi053.stdout:7/588: dread d0/d24/d46/f5f [0,4194304] 0 2024-09-21T04:35:14.035 INFO:tasks.workunit.client.0.smithi007.stdout:3/80: chown dc/f13 153279 1 2024-09-21T04:35:14.035 INFO:tasks.workunit.client.0.smithi007.stdout:0/98: rename d6/d16/f1e to d6/d16/f22 0 2024-09-21T04:35:14.038 INFO:tasks.workunit.client.0.smithi007.stdout:8/83: getdents d0/d3/d8 0 2024-09-21T04:35:14.042 INFO:tasks.workunit.client.1.smithi053.stdout:9/641: dwrite d0/d1c/d25/f9a [0,4194304] 0 2024-09-21T04:35:14.045 INFO:tasks.workunit.client.1.smithi053.stdout:0/665: chown d5/de/d9e/dd2/ce3 153434 1 2024-09-21T04:35:14.045 INFO:tasks.workunit.client.0.smithi007.stdout:5/288: rename d2 to d2/dd/d23/d5e 22 2024-09-21T04:35:14.055 INFO:tasks.workunit.client.0.smithi007.stdout:3/81: write dc/f10 [4581565,1322] 0 2024-09-21T04:35:14.056 INFO:tasks.workunit.client.1.smithi053.stdout:7/589: mknod d0/d24/d46/cc2 0 2024-09-21T04:35:14.056 INFO:tasks.workunit.client.1.smithi053.stdout:4/761: dread d1/d27/f55 [0,4194304] 0 2024-09-21T04:35:14.057 INFO:tasks.workunit.client.0.smithi007.stdout:7/178: link d3/d6/d13/l3b d3/d6/d2e/l3e 0 2024-09-21T04:35:14.057 INFO:tasks.workunit.client.1.smithi053.stdout:4/762: chown d1/d27/f89 181037597 1 2024-09-21T04:35:14.059 INFO:tasks.workunit.client.0.smithi007.stdout:8/84: rmdir d0/d3 39 2024-09-21T04:35:14.062 INFO:tasks.workunit.client.0.smithi007.stdout:5/289: chown d2/c57 24564 1 2024-09-21T04:35:14.064 INFO:tasks.workunit.client.0.smithi007.stdout:3/82: creat dc/f14 x:0 0 0 2024-09-21T04:35:14.068 INFO:tasks.workunit.client.0.smithi007.stdout:0/99: dwrite d6/f13 [0,4194304] 0 2024-09-21T04:35:14.068 INFO:tasks.workunit.client.1.smithi053.stdout:3/678: creat d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/d97/ffa x:0 0 0 2024-09-21T04:35:14.069 INFO:tasks.workunit.client.1.smithi053.stdout:3/679: fsync d16/d17/d21/d60/fd6 0 2024-09-21T04:35:14.071 INFO:tasks.workunit.client.1.smithi053.stdout:7/590: unlink d0/d6f/dbf/d50/c56 0 2024-09-21T04:35:14.072 INFO:tasks.workunit.client.1.smithi053.stdout:3/680: dread d16/d17/d21/f64 [0,4194304] 0 2024-09-21T04:35:14.074 INFO:tasks.workunit.client.1.smithi053.stdout:4/763: unlink d1/dd/de/df5/d20/d4e/d51/da4/ff9 0 2024-09-21T04:35:14.075 INFO:tasks.workunit.client.1.smithi053.stdout:7/591: chown d0/d2b/d36/d64/d9e/fad 465 1 2024-09-21T04:35:14.075 INFO:tasks.workunit.client.1.smithi053.stdout:4/764: chown d1/d27 326736 1 2024-09-21T04:35:14.075 INFO:tasks.workunit.client.0.smithi007.stdout:5/290: dwrite d2/d27/d3d/f4e [0,4194304] 0 2024-09-21T04:35:14.076 INFO:tasks.workunit.client.0.smithi007.stdout:3/83: rmdir dc 39 2024-09-21T04:35:14.078 INFO:tasks.workunit.client.1.smithi053.stdout:0/666: dwrite d5/de/d9e/faf [0,4194304] 0 2024-09-21T04:35:14.079 INFO:tasks.workunit.client.0.smithi007.stdout:7/179: link d3/d6/d14/d20/f35 d3/d6/d14/d20/f3f 0 2024-09-21T04:35:14.079 INFO:tasks.workunit.client.0.smithi007.stdout:5/291: dread - d2/d1e/f5b zero size 2024-09-21T04:35:14.081 INFO:tasks.workunit.client.1.smithi053.stdout:9/642: symlink d0/d2/ddf/le0 0 2024-09-21T04:35:14.083 INFO:tasks.workunit.client.0.smithi007.stdout:0/100: dwrite d6/f12 [4194304,4194304] 0 2024-09-21T04:35:14.083 INFO:tasks.workunit.client.0.smithi007.stdout:8/85: dwrite d0/d3/d8/de/f18 [0,4194304] 0 2024-09-21T04:35:14.084 INFO:tasks.workunit.client.1.smithi053.stdout:9/643: read d0/d1a/d4c/f86 [215626,77231] 0 2024-09-21T04:35:14.084 INFO:tasks.workunit.client.0.smithi007.stdout:5/292: chown d2/dd/d23/d2a/c30 21178 1 2024-09-21T04:35:14.085 INFO:tasks.workunit.client.0.smithi007.stdout:5/293: dread - d2/f51 zero size 2024-09-21T04:35:14.086 INFO:tasks.workunit.client.1.smithi053.stdout:6/681: rename d2/d1b/d1e/d2a/d4b/d77/d88/db8 to d2/d1b/d3d/d49/dde 0 2024-09-21T04:35:14.089 INFO:tasks.workunit.client.1.smithi053.stdout:9/644: read d0/d2/d9/fb [1243185,2313] 0 2024-09-21T04:35:14.091 INFO:tasks.workunit.client.0.smithi007.stdout:3/84: creat dc/f15 x:0 0 0 2024-09-21T04:35:14.091 INFO:tasks.workunit.client.1.smithi053.stdout:3/681: mkdir d16/d17/d21/d42/da0/dfb 0 2024-09-21T04:35:14.092 INFO:tasks.workunit.client.0.smithi007.stdout:0/101: symlink d6/d16/d18/l23 0 2024-09-21T04:35:14.094 INFO:tasks.workunit.client.0.smithi007.stdout:5/294: rmdir d2/dd/d23/d49/d44 39 2024-09-21T04:35:14.095 INFO:tasks.workunit.client.1.smithi053.stdout:7/592: mknod d0/d2b/d36/d64/cc3 0 2024-09-21T04:35:14.096 INFO:tasks.workunit.client.1.smithi053.stdout:5/734: link d0/dd/d5c/d92/ddb/d35/c41 d0/dd/d67/cf1 0 2024-09-21T04:35:14.096 INFO:tasks.workunit.client.1.smithi053.stdout:7/593: chown d0/d2b/d36 169 1 2024-09-21T04:35:14.101 INFO:tasks.workunit.client.0.smithi007.stdout:5/295: readlink d2/l11 0 2024-09-21T04:35:14.102 INFO:tasks.workunit.client.1.smithi053.stdout:1/695: rename d7/d9/c2e to d7/d9/da/d40/d49/ce8 0 2024-09-21T04:35:14.104 INFO:tasks.workunit.client.1.smithi053.stdout:6/682: creat d2/d1b/d1e/d2a/d4b/d77/d88/da4/fdf x:0 0 0 2024-09-21T04:35:14.105 INFO:tasks.workunit.client.0.smithi007.stdout:8/86: dread d0/d3/d8/fa [0,4194304] 0 2024-09-21T04:35:14.105 INFO:tasks.workunit.client.0.smithi007.stdout:8/87: chown d0/d3/d8/de/f18 8917 1 2024-09-21T04:35:14.106 INFO:tasks.workunit.client.0.smithi007.stdout:3/85: truncate dc/fe 605938 0 2024-09-21T04:35:14.107 INFO:tasks.workunit.client.0.smithi007.stdout:3/86: write fb [3010361,59059] 0 2024-09-21T04:35:14.112 INFO:tasks.workunit.client.0.smithi007.stdout:3/87: dread dc/f10 [0,4194304] 0 2024-09-21T04:35:14.113 INFO:tasks.workunit.client.1.smithi053.stdout:3/682: creat d16/d17/d21/d42/da0/ffc x:0 0 0 2024-09-21T04:35:14.115 INFO:tasks.workunit.client.1.smithi053.stdout:9/645: dread d0/d2/fa [0,4194304] 0 2024-09-21T04:35:14.116 INFO:tasks.workunit.client.1.smithi053.stdout:9/646: write d0/d1c/d25/f54 [1519749,80335] 0 2024-09-21T04:35:14.119 INFO:tasks.workunit.client.1.smithi053.stdout:0/667: fsync d5/d10/f2b 0 2024-09-21T04:35:14.120 INFO:tasks.workunit.client.0.smithi007.stdout:7/180: dwrite d3/d6/d14/f26 [0,4194304] 0 2024-09-21T04:35:14.122 INFO:tasks.workunit.client.0.smithi007.stdout:8/88: creat d0/d3/d16/f1b x:0 0 0 2024-09-21T04:35:14.123 INFO:tasks.workunit.client.0.smithi007.stdout:3/88: mknod dc/c16 0 2024-09-21T04:35:14.123 INFO:tasks.workunit.client.0.smithi007.stdout:8/89: chown d0/d3/d8/l14 269886310 1 2024-09-21T04:35:14.124 INFO:tasks.workunit.client.0.smithi007.stdout:3/89: chown dc/f13 7919205 1 2024-09-21T04:35:14.125 INFO:tasks.workunit.client.1.smithi053.stdout:1/696: rename d7/d9/d43/d54/d7f/d36/d81/d95/fbe to d7/d47/d79/d99/fe9 0 2024-09-21T04:35:14.129 INFO:tasks.workunit.client.1.smithi053.stdout:6/683: fsync d2/d1b/f3a 0 2024-09-21T04:35:14.132 INFO:tasks.workunit.client.0.smithi007.stdout:8/90: dwrite d0/f15 [0,4194304] 0 2024-09-21T04:35:14.133 INFO:tasks.workunit.client.1.smithi053.stdout:3/683: fdatasync d16/d17/d21/d42/ff0 0 2024-09-21T04:35:14.136 INFO:tasks.workunit.client.0.smithi007.stdout:5/296: truncate d2/dd/d1d/f1f 209804 0 2024-09-21T04:35:14.137 INFO:tasks.workunit.client.1.smithi053.stdout:7/594: write d0/d2b/f32 [1590301,39839] 0 2024-09-21T04:35:14.145 INFO:tasks.workunit.client.0.smithi007.stdout:5/297: dwrite d2/dd/d1d/f39 [0,4194304] 0 2024-09-21T04:35:14.147 INFO:tasks.workunit.client.0.smithi007.stdout:7/181: dread d3/d6/d10/f24 [0,4194304] 0 2024-09-21T04:35:14.148 INFO:tasks.workunit.client.1.smithi053.stdout:9/647: creat d0/d1a/d89/da1/fe1 x:0 0 0 2024-09-21T04:35:14.151 INFO:tasks.workunit.client.1.smithi053.stdout:0/668: rmdir d5/d10/d4c/db1/dbc/dca/d53 39 2024-09-21T04:35:14.157 INFO:tasks.workunit.client.1.smithi053.stdout:9/648: dwrite d0/d6c/fa7 [0,4194304] 0 2024-09-21T04:35:14.158 INFO:tasks.workunit.client.1.smithi053.stdout:1/697: dread d7/d9/da/d11/f33 [0,4194304] 0 2024-09-21T04:35:14.159 INFO:tasks.workunit.client.0.smithi007.stdout:8/91: write d0/f6 [2134863,65399] 0 2024-09-21T04:35:14.159 INFO:tasks.workunit.client.1.smithi053.stdout:9/649: chown d0/d1c/d21/f9f 136 1 2024-09-21T04:35:14.160 INFO:tasks.workunit.client.0.smithi007.stdout:8/92: readlink d0/l13 0 2024-09-21T04:35:14.161 INFO:tasks.workunit.client.0.smithi007.stdout:8/93: chown d0/f6 205861 1 2024-09-21T04:35:14.163 INFO:tasks.workunit.client.1.smithi053.stdout:6/684: rename d2/d1b/d3d/d49/dde to d2/d1d/de0 0 2024-09-21T04:35:14.163 INFO:tasks.workunit.client.0.smithi007.stdout:3/90: dwrite fa [0,4194304] 0 2024-09-21T04:35:14.164 INFO:tasks.workunit.client.0.smithi007.stdout:8/94: write d0/d3/d8/f12 [1383996,97405] 0 2024-09-21T04:35:14.164 INFO:tasks.workunit.client.0.smithi007.stdout:3/91: chown l5 90 1 2024-09-21T04:35:14.168 INFO:tasks.workunit.client.0.smithi007.stdout:5/298: symlink d2/dd/d23/d49/l5f 0 2024-09-21T04:35:14.169 INFO:tasks.workunit.client.0.smithi007.stdout:5/299: write d2/d1e/f22 [243484,2287] 0 2024-09-21T04:35:14.171 INFO:tasks.workunit.client.0.smithi007.stdout:5/300: write d2/dd/f14 [4681605,42126] 0 2024-09-21T04:35:14.172 INFO:tasks.workunit.client.0.smithi007.stdout:5/301: dread - d2/dd/f3e zero size 2024-09-21T04:35:14.173 INFO:tasks.workunit.client.0.smithi007.stdout:7/182: mkdir d3/d6/d40 0 2024-09-21T04:35:14.173 INFO:tasks.workunit.client.1.smithi053.stdout:7/595: symlink d0/d2b/d36/lc4 0 2024-09-21T04:35:14.174 INFO:tasks.workunit.client.1.smithi053.stdout:7/596: fdatasync d0/f44 0 2024-09-21T04:35:14.176 INFO:tasks.workunit.client.0.smithi007.stdout:3/92: creat dc/f17 x:0 0 0 2024-09-21T04:35:14.177 INFO:tasks.workunit.client.0.smithi007.stdout:3/93: read - dc/f14 zero size 2024-09-21T04:35:14.177 INFO:tasks.workunit.client.0.smithi007.stdout:3/94: readlink dc/ld 0 2024-09-21T04:35:14.178 INFO:tasks.workunit.client.1.smithi053.stdout:0/669: mkdir d5/d10/d4c/db1/dbc/dca/db5/dd5/de5 0 2024-09-21T04:35:14.178 INFO:tasks.workunit.client.0.smithi007.stdout:8/95: symlink d0/d3/d8/de/l1c 0 2024-09-21T04:35:14.179 INFO:tasks.workunit.client.0.smithi007.stdout:8/96: chown d0/d3/d8/lc 335 1 2024-09-21T04:35:14.179 INFO:tasks.workunit.client.1.smithi053.stdout:8/719: sync 2024-09-21T04:35:14.179 INFO:tasks.workunit.client.1.smithi053.stdout:4/765: sync 2024-09-21T04:35:14.180 INFO:tasks.workunit.client.1.smithi053.stdout:2/714: sync 2024-09-21T04:35:14.180 INFO:tasks.workunit.client.1.smithi053.stdout:1/698: symlink d7/d9/da/d40/d49/lea 0 2024-09-21T04:35:14.181 INFO:tasks.workunit.client.0.smithi007.stdout:8/97: write d0/d3/d8/de/f17 [845135,74109] 0 2024-09-21T04:35:14.181 INFO:tasks.workunit.client.1.smithi053.stdout:2/715: readlink d1/d18/lf7 0 2024-09-21T04:35:14.182 INFO:tasks.workunit.client.0.smithi007.stdout:5/302: symlink d2/dd/d28/l60 0 2024-09-21T04:35:14.182 INFO:tasks.workunit.client.1.smithi053.stdout:1/699: write d7/d47/d79/d99/db0/fe7 [1019369,83557] 0 2024-09-21T04:35:14.184 INFO:tasks.workunit.client.0.smithi007.stdout:8/98: write d0/d3/d8/fa [3501715,13890] 0 2024-09-21T04:35:14.184 INFO:tasks.workunit.client.0.smithi007.stdout:7/183: creat d3/d6/d14/d20/d33/f41 x:0 0 0 2024-09-21T04:35:14.185 INFO:tasks.workunit.client.1.smithi053.stdout:6/685: creat d2/d1b/d3d/fe1 x:0 0 0 2024-09-21T04:35:14.188 INFO:tasks.workunit.client.0.smithi007.stdout:3/95: mkdir dc/d18 0 2024-09-21T04:35:14.190 INFO:tasks.workunit.client.1.smithi053.stdout:3/684: creat d16/d17/d21/d8b/d7d/d57/df5/ffd x:0 0 0 2024-09-21T04:35:14.194 INFO:tasks.workunit.client.0.smithi007.stdout:7/184: symlink d3/d6/d13/l42 0 2024-09-21T04:35:14.195 INFO:tasks.workunit.client.1.smithi053.stdout:7/597: chown d0/d2b/f41 10325966 1 2024-09-21T04:35:14.197 INFO:tasks.workunit.client.0.smithi007.stdout:3/96: symlink dc/l19 0 2024-09-21T04:35:14.198 INFO:tasks.workunit.client.0.smithi007.stdout:3/97: chown dc/lf 4657776 1 2024-09-21T04:35:14.199 INFO:tasks.workunit.client.0.smithi007.stdout:5/303: mkdir d2/dd/d23/d5d/d61 0 2024-09-21T04:35:14.199 INFO:tasks.workunit.client.1.smithi053.stdout:8/720: creat d3/d39/fed x:0 0 0 2024-09-21T04:35:14.200 INFO:tasks.workunit.client.0.smithi007.stdout:5/304: stat d2/dd/d23/l29 0 2024-09-21T04:35:14.201 INFO:tasks.workunit.client.0.smithi007.stdout:7/185: mkdir d3/d6/d10/d43 0 2024-09-21T04:35:14.201 INFO:tasks.workunit.client.1.smithi053.stdout:4/766: chown d1/d27/f86 213 1 2024-09-21T04:35:14.201 INFO:tasks.workunit.client.0.smithi007.stdout:7/186: readlink d3/d6/d13/l3b 0 2024-09-21T04:35:14.202 INFO:tasks.workunit.client.0.smithi007.stdout:8/99: dread d0/d3/d8/f9 [0,4194304] 0 2024-09-21T04:35:14.204 INFO:tasks.workunit.client.0.smithi007.stdout:5/305: creat d2/d27/d42/f62 x:0 0 0 2024-09-21T04:35:14.205 INFO:tasks.workunit.client.0.smithi007.stdout:7/187: write d3/d6/d10/f24 [4223519,117210] 0 2024-09-21T04:35:14.206 INFO:tasks.workunit.client.1.smithi053.stdout:9/650: truncate d0/d1a/d4c/d8f/f97 2437761 0 2024-09-21T04:35:14.209 INFO:tasks.workunit.client.0.smithi007.stdout:8/100: rmdir d0/d3/d8/de 39 2024-09-21T04:35:14.209 INFO:tasks.workunit.client.0.smithi007.stdout:7/188: fdatasync d3/d6/d13/f29 0 2024-09-21T04:35:14.210 INFO:tasks.workunit.client.0.smithi007.stdout:8/101: chown d0/l13 4056 1 2024-09-21T04:35:14.213 INFO:tasks.workunit.client.1.smithi053.stdout:5/735: link d0/dd/d48/d8c/la1 d0/dd/d5c/d92/ddb/lf2 0 2024-09-21T04:35:14.217 INFO:tasks.workunit.client.0.smithi007.stdout:7/189: dwrite d3/d6/d14/d20/f37 [0,4194304] 0 2024-09-21T04:35:14.219 INFO:tasks.workunit.client.1.smithi053.stdout:1/700: dread d7/d9/d43/d54/d7f/d59/f7b [0,4194304] 0 2024-09-21T04:35:14.220 INFO:tasks.workunit.client.0.smithi007.stdout:8/102: write d0/d3/d8/de/f17 [77364,105999] 0 2024-09-21T04:35:14.220 INFO:tasks.workunit.client.1.smithi053.stdout:1/701: write d7/d47/d79/d99/fe4 [526502,4791] 0 2024-09-21T04:35:14.226 INFO:tasks.workunit.client.1.smithi053.stdout:4/767: unlink d1/dd/de/lef 0 2024-09-21T04:35:14.227 INFO:tasks.workunit.client.1.smithi053.stdout:4/768: chown d1/dd/d62/cfa 473457 1 2024-09-21T04:35:14.227 INFO:tasks.workunit.client.0.smithi007.stdout:7/190: mknod d3/d6/d10/c44 0 2024-09-21T04:35:14.227 INFO:tasks.workunit.client.1.smithi053.stdout:4/769: dread - d1/dd/de/df5/ff3 zero size 2024-09-21T04:35:14.230 INFO:tasks.workunit.client.1.smithi053.stdout:6/686: symlink d2/d1b/d1e/d27/d41/dc9/le2 0 2024-09-21T04:35:14.231 INFO:tasks.workunit.client.0.smithi007.stdout:8/103: fdatasync d0/d3/d8/f9 0 2024-09-21T04:35:14.233 INFO:tasks.workunit.client.1.smithi053.stdout:2/716: dwrite d1/d18/f49 [4194304,4194304] 0 2024-09-21T04:35:14.234 INFO:tasks.workunit.client.0.smithi007.stdout:3/98: dwrite dc/f10 [4194304,4194304] 0 2024-09-21T04:35:14.234 INFO:tasks.workunit.client.1.smithi053.stdout:9/651: rename d0/d2/c57 to d0/d2/d9/d34/ce2 0 2024-09-21T04:35:14.238 INFO:tasks.workunit.client.1.smithi053.stdout:0/670: truncate d5/f8d 2419260 0 2024-09-21T04:35:14.239 INFO:tasks.workunit.client.1.smithi053.stdout:3/685: dwrite d16/d17/d21/d26/d3e/fd3 [0,4194304] 0 2024-09-21T04:35:14.241 INFO:tasks.workunit.client.0.smithi007.stdout:7/191: unlink d3/d6/d13/l3b 0 2024-09-21T04:35:14.248 INFO:tasks.workunit.client.1.smithi053.stdout:1/702: creat d7/d9/da/d40/d49/d77/feb x:0 0 0 2024-09-21T04:35:14.249 INFO:tasks.workunit.client.1.smithi053.stdout:8/721: creat d3/df/d14/d16/dde/fee x:0 0 0 2024-09-21T04:35:14.249 INFO:tasks.workunit.client.0.smithi007.stdout:5/306: link d2/dd/d23/d49/d44/d54/f59 d2/dd/d23/d2a/f63 0 2024-09-21T04:35:14.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/192: rename d3/d6/d14/f32 to d3/d6/d14/f45 0 2024-09-21T04:35:14.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/193: fdatasync d3/d6/d14/f26 0 2024-09-21T04:35:14.259 INFO:tasks.workunit.client.1.smithi053.stdout:9/652: mkdir d0/d1a/d89/de3 0 2024-09-21T04:35:14.259 INFO:tasks.workunit.client.0.smithi007.stdout:3/99: mknod dc/d18/c1a 0 2024-09-21T04:35:14.261 INFO:tasks.workunit.client.0.smithi007.stdout:3/100: write dc/f10 [1765297,55095] 0 2024-09-21T04:35:14.261 INFO:tasks.workunit.client.1.smithi053.stdout:7/598: write d0/f16 [2420804,6268] 0 2024-09-21T04:35:14.265 INFO:tasks.workunit.client.1.smithi053.stdout:3/686: mkdir d16/d17/d21/d8b/d7d/d57/d90/dfe 0 2024-09-21T04:35:14.265 INFO:tasks.workunit.client.1.smithi053.stdout:5/736: dwrite d0/d8/d2a/f57 [0,4194304] 0 2024-09-21T04:35:14.267 INFO:tasks.workunit.client.0.smithi007.stdout:7/194: mknod d3/d6/d14/d20/d33/c46 0 2024-09-21T04:35:14.270 INFO:tasks.workunit.client.0.smithi007.stdout:8/104: rename d0/d3/d8/f12 to d0/d3/d8/f1d 0 2024-09-21T04:35:14.271 INFO:tasks.workunit.client.1.smithi053.stdout:9/653: dwrite d0/d28/fd6 [4194304,4194304] 0 2024-09-21T04:35:14.271 INFO:tasks.workunit.client.1.smithi053.stdout:1/703: rmdir d7/da8/d74/d75 39 2024-09-21T04:35:14.272 INFO:tasks.workunit.client.1.smithi053.stdout:3/687: write d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fda [3091852,64105] 0 2024-09-21T04:35:14.275 INFO:tasks.workunit.client.1.smithi053.stdout:2/717: dwrite d1/d18/d1f/d27/f6a [0,4194304] 0 2024-09-21T04:35:14.276 INFO:tasks.workunit.client.1.smithi053.stdout:1/704: chown d7/d9/d43/d54/d7f/d59/f9d 20 1 2024-09-21T04:35:14.277 INFO:tasks.workunit.client.1.smithi053.stdout:1/705: chown d7/d9/d43/d54/c9c 10690974 1 2024-09-21T04:35:14.277 INFO:tasks.workunit.client.1.smithi053.stdout:6/687: symlink d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/dd5/le3 0 2024-09-21T04:35:14.278 INFO:tasks.workunit.client.0.smithi007.stdout:3/101: symlink dc/d18/l1b 0 2024-09-21T04:35:14.279 INFO:tasks.workunit.client.0.smithi007.stdout:3/102: dread - dc/f14 zero size 2024-09-21T04:35:14.280 INFO:tasks.workunit.client.0.smithi007.stdout:5/307: fsync d2/dd/d23/d49/d44/d54/f59 0 2024-09-21T04:35:14.287 INFO:tasks.workunit.client.0.smithi007.stdout:5/308: dwrite d2/dd/f16 [0,4194304] 0 2024-09-21T04:35:14.288 INFO:tasks.workunit.client.0.smithi007.stdout:5/309: chown d2/d1e 44 1 2024-09-21T04:35:14.292 INFO:tasks.workunit.client.0.smithi007.stdout:3/103: creat dc/f1c x:0 0 0 2024-09-21T04:35:14.297 INFO:tasks.workunit.client.0.smithi007.stdout:7/195: write d3/d6/d14/f18 [760417,104526] 0 2024-09-21T04:35:14.301 INFO:tasks.workunit.client.1.smithi053.stdout:7/599: dread d0/d6f/dbf/d50/f96 [0,4194304] 0 2024-09-21T04:35:14.302 INFO:tasks.workunit.client.0.smithi007.stdout:5/310: creat d2/dd/d28/f64 x:0 0 0 2024-09-21T04:35:14.304 INFO:tasks.workunit.client.0.smithi007.stdout:3/104: unlink dc/d18/c1a 0 2024-09-21T04:35:14.307 INFO:tasks.workunit.client.0.smithi007.stdout:7/196: creat d3/d6/d13/f47 x:0 0 0 2024-09-21T04:35:14.307 INFO:tasks.workunit.client.0.smithi007.stdout:8/105: dwrite d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:14.309 INFO:tasks.workunit.client.0.smithi007.stdout:5/311: fsync d2/dd/d23/d2a/f63 0 2024-09-21T04:35:14.310 INFO:tasks.workunit.client.0.smithi007.stdout:8/106: write d0/d3/d8/de/f17 [430548,38905] 0 2024-09-21T04:35:14.311 INFO:tasks.workunit.client.1.smithi053.stdout:8/722: creat d3/d5/d4c/dc3/ddb/de7/fef x:0 0 0 2024-09-21T04:35:14.315 INFO:tasks.workunit.client.0.smithi007.stdout:3/105: rename dc/f17 to dc/d18/f1d 0 2024-09-21T04:35:14.316 INFO:tasks.workunit.client.1.smithi053.stdout:0/671: link d5/dd/ff d5/d10/d4c/ddf/fe6 0 2024-09-21T04:35:14.317 INFO:tasks.workunit.client.0.smithi007.stdout:8/107: dread d0/d3/d8/de/f17 [0,4194304] 0 2024-09-21T04:35:14.319 INFO:tasks.workunit.client.1.smithi053.stdout:9/654: dwrite d0/d1a/d89/fb7 [0,4194304] 0 2024-09-21T04:35:14.322 INFO:tasks.workunit.client.1.smithi053.stdout:7/600: symlink d0/d24/lc5 0 2024-09-21T04:35:14.325 INFO:tasks.workunit.client.0.smithi007.stdout:8/108: dwrite d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:14.325 INFO:tasks.workunit.client.0.smithi007.stdout:5/312: dread d2/f55 [0,4194304] 0 2024-09-21T04:35:14.325 INFO:tasks.workunit.client.1.smithi053.stdout:3/688: symlink d16/d17/d21/d42/lff 0 2024-09-21T04:35:14.326 INFO:tasks.workunit.client.0.smithi007.stdout:8/109: stat d0/d3/d8/fa 0 2024-09-21T04:35:14.330 INFO:tasks.workunit.client.1.smithi053.stdout:8/723: dread - d3/d28/f65 zero size 2024-09-21T04:35:14.331 INFO:tasks.workunit.client.0.smithi007.stdout:7/197: creat d3/d6/d40/f48 x:0 0 0 2024-09-21T04:35:14.335 INFO:tasks.workunit.client.1.smithi053.stdout:9/655: symlink d0/d2/d2b/le4 0 2024-09-21T04:35:14.335 INFO:tasks.workunit.client.0.smithi007.stdout:8/110: dwrite d0/d3/d8/fa [0,4194304] 0 2024-09-21T04:35:14.336 INFO:tasks.workunit.client.1.smithi053.stdout:7/601: symlink d0/d6f/dbf/d5d/lc6 0 2024-09-21T04:35:14.337 INFO:tasks.workunit.client.1.smithi053.stdout:8/724: dwrite d3/d39/f27 [0,4194304] 0 2024-09-21T04:35:14.340 INFO:tasks.workunit.client.1.smithi053.stdout:9/656: truncate d0/d2/d9/d34/d58/fa0 301634 0 2024-09-21T04:35:14.343 INFO:tasks.workunit.client.0.smithi007.stdout:8/111: dwrite d0/d3/d8/fa [0,4194304] 0 2024-09-21T04:35:14.344 INFO:tasks.workunit.client.0.smithi007.stdout:7/198: creat d3/d6/d14/d20/d33/f49 x:0 0 0 2024-09-21T04:35:14.345 INFO:tasks.workunit.client.1.smithi053.stdout:2/718: write d1/d18/d1f/d27/d58/fab [1531819,85380] 0 2024-09-21T04:35:14.349 INFO:tasks.workunit.client.1.smithi053.stdout:7/602: dwrite d0/d6f/dbf/d50/d8b/f42 [0,4194304] 0 2024-09-21T04:35:14.352 INFO:tasks.workunit.client.0.smithi007.stdout:9/129: sync 2024-09-21T04:35:14.352 INFO:tasks.workunit.client.0.smithi007.stdout:4/175: sync 2024-09-21T04:35:14.352 INFO:tasks.workunit.client.0.smithi007.stdout:1/215: sync 2024-09-21T04:35:14.354 INFO:tasks.workunit.client.1.smithi053.stdout:9/657: creat d0/d1c/d21/d46/fe5 x:0 0 0 2024-09-21T04:35:14.355 INFO:tasks.workunit.client.1.smithi053.stdout:1/706: dwrite d7/d9/d43/f6e [0,4194304] 0 2024-09-21T04:35:14.355 INFO:tasks.workunit.client.0.smithi007.stdout:8/112: symlink d0/d3/d16/l1e 0 2024-09-21T04:35:14.357 INFO:tasks.workunit.client.1.smithi053.stdout:6/688: getdents d2/d1b/d3d/d49/de/d32/d80/d2e/d5b 0 2024-09-21T04:35:14.357 INFO:tasks.workunit.client.1.smithi053.stdout:3/689: link d16/d17/d21/d42/d43/c93 d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/c100 0 2024-09-21T04:35:14.358 INFO:tasks.workunit.client.0.smithi007.stdout:7/199: symlink d3/d6/d13/l4a 0 2024-09-21T04:35:14.358 INFO:tasks.workunit.client.1.smithi053.stdout:2/719: dread d1/d18/d1f/d22/d2d/f86 [0,4194304] 0 2024-09-21T04:35:14.361 INFO:tasks.workunit.client.0.smithi007.stdout:8/113: dread d0/d3/d8/f9 [4194304,4194304] 0 2024-09-21T04:35:14.361 INFO:tasks.workunit.client.1.smithi053.stdout:6/689: chown d2/d1b/f7d 7917 1 2024-09-21T04:35:14.362 INFO:tasks.workunit.client.1.smithi053.stdout:1/707: write d7/d9/da/d3c/f46 [3491979,85417] 0 2024-09-21T04:35:14.362 INFO:tasks.workunit.client.1.smithi053.stdout:9/658: read d0/d1c/d52/d53/f80 [430261,69274] 0 2024-09-21T04:35:14.362 INFO:tasks.workunit.client.0.smithi007.stdout:8/114: truncate d0/f15 4948845 0 2024-09-21T04:35:14.363 INFO:tasks.workunit.client.0.smithi007.stdout:4/176: rmdir d2/d17/d18 39 2024-09-21T04:35:14.364 INFO:tasks.workunit.client.0.smithi007.stdout:8/115: chown d0/d3/d8/de/lf 2949217 1 2024-09-21T04:35:14.365 INFO:tasks.workunit.client.0.smithi007.stdout:8/116: stat d0/d3/d8/f10 0 2024-09-21T04:35:14.365 INFO:tasks.workunit.client.0.smithi007.stdout:1/216: rmdir d5/d6/d21 39 2024-09-21T04:35:14.366 INFO:tasks.workunit.client.0.smithi007.stdout:8/117: fdatasync d0/d3/d8/f10 0 2024-09-21T04:35:14.367 INFO:tasks.workunit.client.1.smithi053.stdout:6/690: read d2/d1b/f2f [3153487,112471] 0 2024-09-21T04:35:14.368 INFO:tasks.workunit.client.0.smithi007.stdout:5/313: truncate d2/d27/d3d/f4e 1428588 0 2024-09-21T04:35:14.368 INFO:tasks.workunit.client.0.smithi007.stdout:3/106: getdents dc/d18 0 2024-09-21T04:35:14.371 INFO:tasks.workunit.client.0.smithi007.stdout:7/200: unlink d3/d6/d13/f1f 0 2024-09-21T04:35:14.372 INFO:tasks.workunit.client.1.smithi053.stdout:1/708: dread d7/d9/da/d3c/fbc [0,4194304] 0 2024-09-21T04:35:14.373 INFO:tasks.workunit.client.0.smithi007.stdout:8/118: dread d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:14.374 INFO:tasks.workunit.client.0.smithi007.stdout:9/130: truncate f11 2309803 0 2024-09-21T04:35:14.375 INFO:tasks.workunit.client.1.smithi053.stdout:1/709: write d7/d24/f41 [225590,91334] 0 2024-09-21T04:35:14.376 INFO:tasks.workunit.client.0.smithi007.stdout:4/177: rename d2/d17/f1b to d2/d17/d29/f3a 0 2024-09-21T04:35:14.376 INFO:tasks.workunit.client.1.smithi053.stdout:3/690: dwrite d16/d17/d21/d26/d3e/fd3 [0,4194304] 0 2024-09-21T04:35:14.379 INFO:tasks.workunit.client.1.smithi053.stdout:7/603: dread d0/f16 [0,4194304] 0 2024-09-21T04:35:14.381 INFO:tasks.workunit.client.0.smithi007.stdout:7/201: dwrite d3/d6/f28 [4194304,4194304] 0 2024-09-21T04:35:14.382 INFO:tasks.workunit.client.0.smithi007.stdout:8/119: dread d0/f6 [0,4194304] 0 2024-09-21T04:35:14.382 INFO:tasks.workunit.client.1.smithi053.stdout:2/720: dwrite d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/f107 [0,4194304] 0 2024-09-21T04:35:14.385 INFO:tasks.workunit.client.1.smithi053.stdout:9/659: fsync d0/d1c/d52/d53/f80 0 2024-09-21T04:35:14.386 INFO:tasks.workunit.client.0.smithi007.stdout:7/202: dread d3/f5 [4194304,4194304] 0 2024-09-21T04:35:14.387 INFO:tasks.workunit.client.1.smithi053.stdout:6/691: truncate d2/d1b/d3d/d49/d5c/f5d 722365 0 2024-09-21T04:35:14.389 INFO:tasks.workunit.client.0.smithi007.stdout:3/107: symlink dc/d18/l1e 0 2024-09-21T04:35:14.398 INFO:tasks.workunit.client.0.smithi007.stdout:7/203: dwrite d3/d6/f2d [0,4194304] 0 2024-09-21T04:35:14.403 INFO:tasks.workunit.client.0.smithi007.stdout:7/204: write d3/d6/f7 [460189,93739] 0 2024-09-21T04:35:14.404 INFO:tasks.workunit.client.1.smithi053.stdout:8/725: rename d3/d5/d4b/d5c/d86/dcd/cd9 to d3/d5/d4c/cf0 0 2024-09-21T04:35:14.405 INFO:tasks.workunit.client.0.smithi007.stdout:5/314: write d2/f5a [774040,47712] 0 2024-09-21T04:35:14.406 INFO:tasks.workunit.client.1.smithi053.stdout:6/692: dread d2/d1b/d3d/d49/f42 [0,4194304] 0 2024-09-21T04:35:14.407 INFO:tasks.workunit.client.0.smithi007.stdout:4/178: truncate d2/d7/ff 1214397 0 2024-09-21T04:35:14.407 INFO:tasks.workunit.client.0.smithi007.stdout:7/205: write d3/d6/d13/f47 [374678,130376] 0 2024-09-21T04:35:14.407 INFO:tasks.workunit.client.1.smithi053.stdout:6/693: readlink d2/d1b/d1e/d27/d90/la2 0 2024-09-21T04:35:14.409 INFO:tasks.workunit.client.1.smithi053.stdout:6/694: chown d2/d1b/d3d/d49/f42 752381 1 2024-09-21T04:35:14.410 INFO:tasks.workunit.client.0.smithi007.stdout:1/217: dwrite d5/d2f/f34 [0,4194304] 0 2024-09-21T04:35:14.412 INFO:tasks.workunit.client.0.smithi007.stdout:4/179: chown d2/d17/l32 618 1 2024-09-21T04:35:14.414 INFO:tasks.workunit.client.1.smithi053.stdout:9/660: mkdir d0/d2/d9/d34/d58/d92/de6 0 2024-09-21T04:35:14.415 INFO:tasks.workunit.client.0.smithi007.stdout:8/120: unlink d0/l19 0 2024-09-21T04:35:14.417 INFO:tasks.workunit.client.0.smithi007.stdout:3/108: rmdir dc 39 2024-09-21T04:35:14.417 INFO:tasks.workunit.client.0.smithi007.stdout:4/180: dread d2/d7/fa [4194304,4194304] 0 2024-09-21T04:35:14.418 INFO:tasks.workunit.client.0.smithi007.stdout:5/315: dread d2/dd/d23/d2a/f2f [0,4194304] 0 2024-09-21T04:35:14.421 INFO:tasks.workunit.client.1.smithi053.stdout:1/710: rename d7/c1c to d7/d47/d79/dc1/cec 0 2024-09-21T04:35:14.421 INFO:tasks.workunit.client.1.smithi053.stdout:7/604: write d0/d6e/fb2 [1213256,76597] 0 2024-09-21T04:35:14.425 INFO:tasks.workunit.client.1.smithi053.stdout:8/726: creat d3/d5/d38/d64/d9d/da9/ff1 x:0 0 0 2024-09-21T04:35:14.425 INFO:tasks.workunit.client.0.smithi007.stdout:9/131: truncate fa 3207009 0 2024-09-21T04:35:14.431 INFO:tasks.workunit.client.1.smithi053.stdout:8/727: dwrite d3/d28/f68 [0,4194304] 0 2024-09-21T04:35:14.431 INFO:tasks.workunit.client.0.smithi007.stdout:7/206: mkdir d3/d6/d13/d4b 0 2024-09-21T04:35:14.431 INFO:tasks.workunit.client.0.smithi007.stdout:7/207: dread - d3/d6/d13/f3c zero size 2024-09-21T04:35:14.433 INFO:tasks.workunit.client.1.smithi053.stdout:6/695: symlink d2/d1b/d1e/d2a/d4b/d77/d88/le4 0 2024-09-21T04:35:14.439 INFO:tasks.workunit.client.1.smithi053.stdout:3/691: truncate d16/d17/d21/d8b/d7d/d57/dc1/d53/f9a 1239365 0 2024-09-21T04:35:14.441 INFO:tasks.workunit.client.1.smithi053.stdout:2/721: creat d1/d18/d1f/d27/d58/f108 x:0 0 0 2024-09-21T04:35:14.442 INFO:tasks.workunit.client.1.smithi053.stdout:1/711: rmdir d7/d9/d43/d54/d7f/d36/d81/d95 39 2024-09-21T04:35:14.447 INFO:tasks.workunit.client.0.smithi007.stdout:5/316: rmdir d2/d1e 39 2024-09-21T04:35:14.452 INFO:tasks.workunit.client.1.smithi053.stdout:6/696: truncate d2/d1b/d3d/d49/f42 2240985 0 2024-09-21T04:35:14.453 INFO:tasks.workunit.client.0.smithi007.stdout:7/208: creat d3/f4c x:0 0 0 2024-09-21T04:35:14.453 INFO:tasks.workunit.client.0.smithi007.stdout:7/209: dread - d3/d6/d14/d20/d33/f49 zero size 2024-09-21T04:35:14.454 INFO:tasks.workunit.client.1.smithi053.stdout:3/692: symlink d16/d17/d21/l101 0 2024-09-21T04:35:14.454 INFO:tasks.workunit.client.0.smithi007.stdout:2/118: sync 2024-09-21T04:35:14.455 INFO:tasks.workunit.client.1.smithi053.stdout:2/722: dread d1/d18/d1f/d27/d58/fb3 [0,4194304] 0 2024-09-21T04:35:14.456 INFO:tasks.workunit.client.0.smithi007.stdout:2/119: truncate d0/f19 525060 0 2024-09-21T04:35:14.457 INFO:tasks.workunit.client.0.smithi007.stdout:8/121: write d0/d3/d8/fa [1964069,67037] 0 2024-09-21T04:35:14.466 INFO:tasks.workunit.client.1.smithi053.stdout:9/661: rename d0/d1c/d52/d53/db1 to d0/d2/d9/de7 0 2024-09-21T04:35:14.467 INFO:tasks.workunit.client.1.smithi053.stdout:6/697: stat d2/d1b/d3d/d49/de/d96/f7b 0 2024-09-21T04:35:14.468 INFO:tasks.workunit.client.0.smithi007.stdout:1/218: dread d5/d6/d21/f30 [0,4194304] 0 2024-09-21T04:35:14.468 INFO:tasks.workunit.client.0.smithi007.stdout:3/109: mknod dc/d18/c1f 0 2024-09-21T04:35:14.469 INFO:tasks.workunit.client.0.smithi007.stdout:3/110: chown dc/f10 206911646 1 2024-09-21T04:35:14.470 INFO:tasks.workunit.client.1.smithi053.stdout:2/723: mknod d1/d18/d1f/d22/d2d/c109 0 2024-09-21T04:35:14.470 INFO:tasks.workunit.client.1.smithi053.stdout:7/605: truncate d0/d6f/dbf/d50/d8b/f1c 56589 0 2024-09-21T04:35:14.471 INFO:tasks.workunit.client.1.smithi053.stdout:2/724: read - d1/d18/d1f/da2/dac/fea zero size 2024-09-21T04:35:14.471 INFO:tasks.workunit.client.0.smithi007.stdout:5/317: write d2/d1e/f5b [733062,12545] 0 2024-09-21T04:35:14.472 INFO:tasks.workunit.client.0.smithi007.stdout:9/132: chown d13/d18/f25 891140787 1 2024-09-21T04:35:14.473 INFO:tasks.workunit.client.0.smithi007.stdout:9/133: write f10 [5401827,29025] 0 2024-09-21T04:35:14.474 INFO:tasks.workunit.client.1.smithi053.stdout:1/712: link d7/d63/f7d d7/d63/d96/dc7/fed 0 2024-09-21T04:35:14.475 INFO:tasks.workunit.client.1.smithi053.stdout:1/713: chown d7/d9/d43/d54/d7f/d36/f98 15865 1 2024-09-21T04:35:14.475 INFO:tasks.workunit.client.0.smithi007.stdout:9/134: chown d13/d18 1403 1 2024-09-21T04:35:14.477 INFO:tasks.workunit.client.1.smithi053.stdout:8/728: rename d3/d5/d4b/d5c/d86/ce1 to d3/df/d14/d16/dde/cf2 0 2024-09-21T04:35:14.479 INFO:tasks.workunit.client.0.smithi007.stdout:5/318: dwrite d2/d1e/f22 [0,4194304] 0 2024-09-21T04:35:14.480 INFO:tasks.workunit.client.0.smithi007.stdout:3/111: unlink dc/d18/l1e 0 2024-09-21T04:35:14.480 INFO:tasks.workunit.client.1.smithi053.stdout:2/725: chown d1/d18/d77/l99 18941 1 2024-09-21T04:35:14.482 INFO:tasks.workunit.client.1.smithi053.stdout:3/693: creat d16/d17/d21/d26/f102 x:0 0 0 2024-09-21T04:35:14.484 INFO:tasks.workunit.client.1.smithi053.stdout:3/694: write d16/d17/d21/d26/d3e/fd5 [628412,63169] 0 2024-09-21T04:35:14.484 INFO:tasks.workunit.client.0.smithi007.stdout:7/210: mknod d3/d6/c4d 0 2024-09-21T04:35:14.485 INFO:tasks.workunit.client.1.smithi053.stdout:9/662: rename d0/d1c/d52/d6f/l70 to d0/d2/ddf/le8 0 2024-09-21T04:35:14.487 INFO:tasks.workunit.client.1.smithi053.stdout:8/729: readlink d3/df/d14/d16/l44 0 2024-09-21T04:35:14.490 INFO:tasks.workunit.client.1.smithi053.stdout:1/714: symlink d7/d9/da/d11/de5/lee 0 2024-09-21T04:35:14.493 INFO:tasks.workunit.client.1.smithi053.stdout:7/606: rmdir d0/d6f/dbf/d5c/dac/dbd 0 2024-09-21T04:35:14.495 INFO:tasks.workunit.client.1.smithi053.stdout:2/726: creat d1/d18/d1f/d22/d2d/deb/f10a x:0 0 0 2024-09-21T04:35:14.496 INFO:tasks.workunit.client.1.smithi053.stdout:2/727: readlink d1/l23 0 2024-09-21T04:35:14.497 INFO:tasks.workunit.client.1.smithi053.stdout:2/728: stat d1/d18/d1f/d27/d41 0 2024-09-21T04:35:14.498 INFO:tasks.workunit.client.1.smithi053.stdout:9/663: link d0/d1c/d25/f99 d0/d28/fe9 0 2024-09-21T04:35:14.501 INFO:tasks.workunit.client.1.smithi053.stdout:9/664: dread d0/d1c/fb0 [0,4194304] 0 2024-09-21T04:35:14.502 INFO:tasks.workunit.client.1.smithi053.stdout:7/607: rmdir d0/d6f/dbf/d50/d8b 39 2024-09-21T04:35:14.502 INFO:tasks.workunit.client.1.smithi053.stdout:9/665: readlink d0/d1c/d52/lb3 0 2024-09-21T04:35:14.503 INFO:tasks.workunit.client.1.smithi053.stdout:3/695: dread d16/d17/d3c/f7e [0,4194304] 0 2024-09-21T04:35:14.507 INFO:tasks.workunit.client.1.smithi053.stdout:1/715: fsync d7/d9/da/f86 0 2024-09-21T04:35:14.508 INFO:tasks.workunit.client.1.smithi053.stdout:1/716: dread - d7/d9/d43/d54/d7f/fcc zero size 2024-09-21T04:35:14.510 INFO:tasks.workunit.client.1.smithi053.stdout:9/666: fdatasync d0/d1a/f2c 0 2024-09-21T04:35:14.514 INFO:tasks.workunit.client.1.smithi053.stdout:3/696: dread d16/d17/d21/d8b/d7d/d57/f6b [0,4194304] 0 2024-09-21T04:35:14.514 INFO:tasks.workunit.client.1.smithi053.stdout:2/729: dread d1/d18/d1f/d27/d41/d4b/f79 [0,4194304] 0 2024-09-21T04:35:14.514 INFO:tasks.workunit.client.1.smithi053.stdout:4/770: sync 2024-09-21T04:35:14.515 INFO:tasks.workunit.client.1.smithi053.stdout:3/697: stat d16/d17/d3c 0 2024-09-21T04:35:14.516 INFO:tasks.workunit.client.1.smithi053.stdout:8/730: link d3/df/d14/d16/c36 d3/df/d14/cf3 0 2024-09-21T04:35:14.517 INFO:tasks.workunit.client.1.smithi053.stdout:9/667: mkdir d0/d1a/d89/da1/dea 0 2024-09-21T04:35:14.519 INFO:tasks.workunit.client.1.smithi053.stdout:7/608: dwrite d0/f16 [0,4194304] 0 2024-09-21T04:35:14.519 INFO:tasks.workunit.client.1.smithi053.stdout:1/717: dwrite d7/d9/d43/f6e [0,4194304] 0 2024-09-21T04:35:14.521 INFO:tasks.workunit.client.1.smithi053.stdout:3/698: creat d16/d17/d3c/f103 x:0 0 0 2024-09-21T04:35:14.522 INFO:tasks.workunit.client.1.smithi053.stdout:9/668: write d0/fc3 [689175,79826] 0 2024-09-21T04:35:14.524 INFO:tasks.workunit.client.1.smithi053.stdout:7/609: fdatasync d0/d2b/d36/f89 0 2024-09-21T04:35:14.526 INFO:tasks.workunit.client.1.smithi053.stdout:2/730: truncate d1/d18/d1f/d22/d43/f4f 4924931 0 2024-09-21T04:35:14.535 INFO:tasks.workunit.client.1.smithi053.stdout:8/731: dwrite d3/f3a [0,4194304] 0 2024-09-21T04:35:14.539 INFO:tasks.workunit.client.1.smithi053.stdout:3/699: rename d16/d17/cdf to d16/d1b/dad/c104 0 2024-09-21T04:35:14.539 INFO:tasks.workunit.client.1.smithi053.stdout:2/731: unlink d1/d18/d1f/d27/d58/db7/cd8 0 2024-09-21T04:35:14.540 INFO:tasks.workunit.client.1.smithi053.stdout:1/718: creat d7/d9/fef x:0 0 0 2024-09-21T04:35:14.540 INFO:tasks.workunit.client.1.smithi053.stdout:4/771: dwrite d1/d27/d33/d41/fa3 [0,4194304] 0 2024-09-21T04:35:14.540 INFO:tasks.workunit.client.1.smithi053.stdout:8/732: read d3/f85 [2314738,45829] 0 2024-09-21T04:35:14.545 INFO:tasks.workunit.client.1.smithi053.stdout:8/733: read - d3/df/d49/fe6 zero size 2024-09-21T04:35:14.547 INFO:tasks.workunit.client.1.smithi053.stdout:4/772: mknod d1/dd/de/df5/d20/dee/cfe 0 2024-09-21T04:35:14.549 INFO:tasks.workunit.client.1.smithi053.stdout:4/773: fdatasync d1/dd/de/df5/db6/fdc 0 2024-09-21T04:35:14.549 INFO:tasks.workunit.client.1.smithi053.stdout:3/700: dread d16/d17/d21/d26/d3e/f99 [0,4194304] 0 2024-09-21T04:35:14.550 INFO:tasks.workunit.client.1.smithi053.stdout:1/719: link d7/d9/da/d40/d49/lea d7/d63/d96/dac/lf0 0 2024-09-21T04:35:14.551 INFO:tasks.workunit.client.1.smithi053.stdout:9/669: dread d0/d1c/d25/f54 [0,4194304] 0 2024-09-21T04:35:14.552 INFO:tasks.workunit.client.1.smithi053.stdout:8/734: link d3/d5/db1/cc8 d3/d5/cf4 0 2024-09-21T04:35:14.553 INFO:tasks.workunit.client.1.smithi053.stdout:3/701: rmdir d16/d17/d21/d26/d49 39 2024-09-21T04:35:14.555 INFO:tasks.workunit.client.1.smithi053.stdout:8/735: dread - d3/d28/f65 zero size 2024-09-21T04:35:14.555 INFO:tasks.workunit.client.1.smithi053.stdout:2/732: dread d1/d18/d1f/d22/d54/fb0 [0,4194304] 0 2024-09-21T04:35:14.556 INFO:tasks.workunit.client.1.smithi053.stdout:7/610: truncate d0/d6f/dbf/d50/d8b/f42 2581826 0 2024-09-21T04:35:14.561 INFO:tasks.workunit.client.1.smithi053.stdout:7/611: write d0/d6f/dbf/d50/fa4 [70207,126457] 0 2024-09-21T04:35:14.561 INFO:tasks.workunit.client.1.smithi053.stdout:4/774: dwrite d1/dd/d62/ff0 [0,4194304] 0 2024-09-21T04:35:14.564 INFO:tasks.workunit.client.1.smithi053.stdout:7/612: fdatasync d0/d24/f5a 0 2024-09-21T04:35:14.570 INFO:tasks.workunit.client.1.smithi053.stdout:1/720: link d7/d9/da/d11/f48 d7/d9/d43/d54/d7f/d36/d81/ff1 0 2024-09-21T04:35:14.570 INFO:tasks.workunit.client.1.smithi053.stdout:9/670: dwrite d0/df/faf [0,4194304] 0 2024-09-21T04:35:14.572 INFO:tasks.workunit.client.1.smithi053.stdout:3/702: link d16/d17/d3c/f7e d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/d97/f105 0 2024-09-21T04:35:14.573 INFO:tasks.workunit.client.1.smithi053.stdout:2/733: dwrite d1/d18/d1f/d22/f5a [0,4194304] 0 2024-09-21T04:35:14.575 INFO:tasks.workunit.client.1.smithi053.stdout:4/775: rmdir d1/d27/d2e 39 2024-09-21T04:35:14.577 INFO:tasks.workunit.client.1.smithi053.stdout:7/613: mkdir d0/d6f/dbf/d50/dc7 0 2024-09-21T04:35:14.577 INFO:tasks.workunit.client.1.smithi053.stdout:1/721: unlink d7/d9/d43/d54/d7f/f4f 0 2024-09-21T04:35:14.578 INFO:tasks.workunit.client.1.smithi053.stdout:5/737: sync 2024-09-21T04:35:14.581 INFO:tasks.workunit.client.1.smithi053.stdout:3/703: mknod d16/d17/d21/d8b/d7d/d9d/c106 0 2024-09-21T04:35:14.585 INFO:tasks.workunit.client.1.smithi053.stdout:7/614: dread - d0/d6f/dbf/d50/d84/f9c zero size 2024-09-21T04:35:14.587 INFO:tasks.workunit.client.1.smithi053.stdout:8/736: dwrite d3/df/d14/d16/f1d [4194304,4194304] 0 2024-09-21T04:35:14.589 INFO:tasks.workunit.client.1.smithi053.stdout:9/671: getdents d0/d2/d9/d34/d58/d92 0 2024-09-21T04:35:14.590 INFO:tasks.workunit.client.1.smithi053.stdout:7/615: chown d0/d2b/d36/f5e 158900 1 2024-09-21T04:35:14.591 INFO:tasks.workunit.client.1.smithi053.stdout:4/776: rename d1/dd/f6b to d1/dd/de/df5/d20/d4e/fff 0 2024-09-21T04:35:14.592 INFO:tasks.workunit.client.1.smithi053.stdout:3/704: creat d16/d17/d21/d8b/d7d/d57/f107 x:0 0 0 2024-09-21T04:35:14.593 INFO:tasks.workunit.client.1.smithi053.stdout:8/737: creat d3/d5/d4c/d8e/ff5 x:0 0 0 2024-09-21T04:35:14.593 INFO:tasks.workunit.client.1.smithi053.stdout:4/777: write d1/f4 [502236,87221] 0 2024-09-21T04:35:14.595 INFO:tasks.workunit.client.1.smithi053.stdout:2/734: dwrite d1/d18/d1f/d27/d41/f64 [8388608,4194304] 0 2024-09-21T04:35:14.596 INFO:tasks.workunit.client.0.smithi007.stdout:2/120: mknod d0/c1d 0 2024-09-21T04:35:14.597 INFO:tasks.workunit.client.0.smithi007.stdout:4/181: getdents d2/d17 0 2024-09-21T04:35:14.598 INFO:tasks.workunit.client.1.smithi053.stdout:9/672: creat d0/d1c/db8/feb x:0 0 0 2024-09-21T04:35:14.598 INFO:tasks.workunit.client.0.smithi007.stdout:5/319: dwrite d2/d1e/f58 [0,4194304] 0 2024-09-21T04:35:14.598 INFO:tasks.workunit.client.0.smithi007.stdout:8/122: truncate d0/d3/d8/f1d 3356508 0 2024-09-21T04:35:14.599 INFO:tasks.workunit.client.0.smithi007.stdout:2/121: read d0/d2/fc [1953097,45332] 0 2024-09-21T04:35:14.600 INFO:tasks.workunit.client.1.smithi053.stdout:7/616: creat d0/d6f/dbf/d50/d84/fc8 x:0 0 0 2024-09-21T04:35:14.600 INFO:tasks.workunit.client.0.smithi007.stdout:7/211: mknod d3/d6/d40/c4e 0 2024-09-21T04:35:14.602 INFO:tasks.workunit.client.0.smithi007.stdout:1/219: rename d5/f1a to d5/d6/f40 0 2024-09-21T04:35:14.602 INFO:tasks.workunit.client.1.smithi053.stdout:7/617: readlink d0/d24/l8e 0 2024-09-21T04:35:14.603 INFO:tasks.workunit.client.1.smithi053.stdout:1/722: write d7/d9/d43/d94/fd1 [674034,8029] 0 2024-09-21T04:35:14.606 INFO:tasks.workunit.client.0.smithi007.stdout:2/122: chown d0/d11/f1a 2 1 2024-09-21T04:35:14.607 INFO:tasks.workunit.client.1.smithi053.stdout:5/738: dread d0/dd/d6d/fa6 [0,4194304] 0 2024-09-21T04:35:14.607 INFO:tasks.workunit.client.1.smithi053.stdout:1/723: chown d7/d47/d79/dc1 9198934 1 2024-09-21T04:35:14.608 INFO:tasks.workunit.client.1.smithi053.stdout:3/705: dwrite d16/d1b/dc3/fb2 [0,4194304] 0 2024-09-21T04:35:14.611 INFO:tasks.workunit.client.0.smithi007.stdout:8/123: dread d0/d3/d8/de/f18 [0,4194304] 0 2024-09-21T04:35:14.611 INFO:tasks.workunit.client.1.smithi053.stdout:9/673: dwrite d0/d28/f9b [0,4194304] 0 2024-09-21T04:35:14.612 INFO:tasks.workunit.client.1.smithi053.stdout:0/672: sync 2024-09-21T04:35:14.612 INFO:tasks.workunit.client.1.smithi053.stdout:6/698: sync 2024-09-21T04:35:14.614 INFO:tasks.workunit.client.0.smithi007.stdout:2/123: chown d0/d2/cb 3512445 1 2024-09-21T04:35:14.614 INFO:tasks.workunit.client.0.smithi007.stdout:7/212: mkdir d3/d6/d10/d4f 0 2024-09-21T04:35:14.615 INFO:tasks.workunit.client.1.smithi053.stdout:1/724: dread d7/d47/fa4 [0,4194304] 0 2024-09-21T04:35:14.615 INFO:tasks.workunit.client.1.smithi053.stdout:4/778: mkdir d1/dd/de/df5/d20/d4e/d51/da4/dc3/d100 0 2024-09-21T04:35:14.615 INFO:tasks.workunit.client.0.smithi007.stdout:2/124: stat d0/c1d 0 2024-09-21T04:35:14.618 INFO:tasks.workunit.client.1.smithi053.stdout:7/618: creat d0/d6f/dbf/d95/fc9 x:0 0 0 2024-09-21T04:35:14.618 INFO:tasks.workunit.client.1.smithi053.stdout:5/739: truncate d0/dd/d48/d8c/dad/fd8 511506 0 2024-09-21T04:35:14.619 INFO:tasks.workunit.client.1.smithi053.stdout:4/779: stat d1/d27/d33/d41/c57 0 2024-09-21T04:35:14.620 INFO:tasks.workunit.client.1.smithi053.stdout:1/725: fdatasync d7/d9/d43/d94/fa7 0 2024-09-21T04:35:14.620 INFO:tasks.workunit.client.1.smithi053.stdout:5/740: fdatasync d0/dd/d48/fe7 0 2024-09-21T04:35:14.621 INFO:tasks.workunit.client.0.smithi007.stdout:5/320: dwrite d2/dd/d23/d49/d44/d54/f59 [0,4194304] 0 2024-09-21T04:35:14.622 INFO:tasks.workunit.client.1.smithi053.stdout:4/780: write d1/dd/de/df5/d3f/f72 [260321,116023] 0 2024-09-21T04:35:14.623 INFO:tasks.workunit.client.1.smithi053.stdout:5/741: chown d0/dd/d5c/d80/ld3 610728 1 2024-09-21T04:35:14.623 INFO:tasks.workunit.client.0.smithi007.stdout:3/112: rename l5 to dc/l20 0 2024-09-21T04:35:14.623 INFO:tasks.workunit.client.1.smithi053.stdout:8/738: truncate d3/df/d14/d16/f23 5607161 0 2024-09-21T04:35:14.623 INFO:tasks.workunit.client.1.smithi053.stdout:3/706: dread d16/d1b/dad/fb1 [0,4194304] 0 2024-09-21T04:35:14.624 INFO:tasks.workunit.client.0.smithi007.stdout:3/113: fsync dc/f1c 0 2024-09-21T04:35:14.624 INFO:tasks.workunit.client.1.smithi053.stdout:5/742: chown d0/d8/f31 59311157 1 2024-09-21T04:35:14.624 INFO:tasks.workunit.client.1.smithi053.stdout:4/781: readlink d1/dd/de/df5/l29 0 2024-09-21T04:35:14.625 INFO:tasks.workunit.client.1.smithi053.stdout:9/674: creat d0/d1c/d52/d6d/fec x:0 0 0 2024-09-21T04:35:14.626 INFO:tasks.workunit.client.0.smithi007.stdout:1/220: mknod d5/d6/d21/d3c/c41 0 2024-09-21T04:35:14.626 INFO:tasks.workunit.client.1.smithi053.stdout:5/743: dread - d0/d8/db/d68/ff0 zero size 2024-09-21T04:35:14.628 INFO:tasks.workunit.client.0.smithi007.stdout:8/124: rmdir d0/d3/d8 39 2024-09-21T04:35:14.628 INFO:tasks.workunit.client.1.smithi053.stdout:9/675: write d0/d1c/d52/daa/fbc [5220618,120462] 0 2024-09-21T04:35:14.629 INFO:tasks.workunit.client.0.smithi007.stdout:8/125: rename d0/d3 to d0/d3/d16/d1f 22 2024-09-21T04:35:14.630 INFO:tasks.workunit.client.0.smithi007.stdout:2/125: truncate d0/f7 1964829 0 2024-09-21T04:35:14.630 INFO:tasks.workunit.client.0.smithi007.stdout:2/126: stat d0 0 2024-09-21T04:35:14.635 INFO:tasks.workunit.client.1.smithi053.stdout:8/739: dwrite d3/d5/d38/f70 [0,4194304] 0 2024-09-21T04:35:14.637 INFO:tasks.workunit.client.1.smithi053.stdout:8/740: truncate d3/df/d10/fcf 940656 0 2024-09-21T04:35:14.637 INFO:tasks.workunit.client.1.smithi053.stdout:8/741: readlink d3/df/d14/d16/lb3 0 2024-09-21T04:35:14.638 INFO:tasks.workunit.client.0.smithi007.stdout:3/114: creat dc/f21 x:0 0 0 2024-09-21T04:35:14.639 INFO:tasks.workunit.client.1.smithi053.stdout:1/726: creat d7/d47/ff2 x:0 0 0 2024-09-21T04:35:14.641 INFO:tasks.workunit.client.0.smithi007.stdout:1/221: mkdir d5/d6/d21/d23/d42 0 2024-09-21T04:35:14.646 INFO:tasks.workunit.client.0.smithi007.stdout:3/115: dwrite dc/f13 [0,4194304] 0 2024-09-21T04:35:14.646 INFO:tasks.workunit.client.0.smithi007.stdout:2/127: dread d0/f8 [0,4194304] 0 2024-09-21T04:35:14.647 INFO:tasks.workunit.client.0.smithi007.stdout:2/128: chown d0/d11 12176 1 2024-09-21T04:35:14.651 INFO:tasks.workunit.client.1.smithi053.stdout:2/735: getdents d1/d18 0 2024-09-21T04:35:14.654 INFO:tasks.workunit.client.1.smithi053.stdout:2/736: read d1/d18/d1f/d22/d43/f4f [4751791,37262] 0 2024-09-21T04:35:14.654 INFO:tasks.workunit.client.0.smithi007.stdout:6/75: sync 2024-09-21T04:35:14.654 INFO:tasks.workunit.client.0.smithi007.stdout:0/102: sync 2024-09-21T04:35:14.655 INFO:tasks.workunit.client.1.smithi053.stdout:5/744: rmdir d0/d8/db6/dbc 39 2024-09-21T04:35:14.655 INFO:tasks.workunit.client.0.smithi007.stdout:3/116: dwrite dc/f1c [0,4194304] 0 2024-09-21T04:35:14.657 INFO:tasks.workunit.client.0.smithi007.stdout:3/117: readlink dc/d18/l1b 0 2024-09-21T04:35:14.657 INFO:tasks.workunit.client.0.smithi007.stdout:3/118: write dc/f21 [950238,55391] 0 2024-09-21T04:35:14.658 INFO:tasks.workunit.client.1.smithi053.stdout:2/737: write d1/d18/d1f/d22/d2d/deb/f10a [113435,22469] 0 2024-09-21T04:35:14.658 INFO:tasks.workunit.client.1.smithi053.stdout:5/745: write d0/dd/d48/fe7 [161399,60765] 0 2024-09-21T04:35:14.660 INFO:tasks.workunit.client.1.smithi053.stdout:5/746: read d0/d8/d2a/fae [664645,41242] 0 2024-09-21T04:35:14.666 INFO:tasks.workunit.client.1.smithi053.stdout:7/619: mkdir d0/dca 0 2024-09-21T04:35:14.666 INFO:tasks.workunit.client.0.smithi007.stdout:2/129: fdatasync d0/fd 0 2024-09-21T04:35:14.668 INFO:tasks.workunit.client.1.smithi053.stdout:8/742: mkdir d3/d5/d4c/d8e/df6 0 2024-09-21T04:35:14.668 INFO:tasks.workunit.client.0.smithi007.stdout:7/213: link d3/d6/d14/d20/f35 d3/d6/d14/d20/d33/f50 0 2024-09-21T04:35:14.670 INFO:tasks.workunit.client.0.smithi007.stdout:4/182: getdents d2/d17 0 2024-09-21T04:35:14.671 INFO:tasks.workunit.client.1.smithi053.stdout:3/707: symlink d16/d17/ddc/l108 0 2024-09-21T04:35:14.672 INFO:tasks.workunit.client.1.smithi053.stdout:6/699: write d2/d1b/d3d/d49/f42 [1878051,92462] 0 2024-09-21T04:35:14.672 INFO:tasks.workunit.client.0.smithi007.stdout:4/183: write d2/f38 [268155,115375] 0 2024-09-21T04:35:14.673 INFO:tasks.workunit.client.0.smithi007.stdout:1/222: write d5/d2f/f3a [3630887,75761] 0 2024-09-21T04:35:14.674 INFO:tasks.workunit.client.1.smithi053.stdout:3/708: write d16/d17/d21/d26/fa7 [429715,31251] 0 2024-09-21T04:35:14.678 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:14 smithi053 ceph-mon[35554]: pgmap v28: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 36 MiB/s rd, 168 MiB/s wr, 323 op/s 2024-09-21T04:35:14.678 INFO:tasks.workunit.client.1.smithi053.stdout:2/738: rename d1/d18/d1f/d27/d58/d5c/d91/dd0/l8b to d1/d18/d1f/d22/d2d/deb/l10b 0 2024-09-21T04:35:14.681 INFO:tasks.workunit.client.1.smithi053.stdout:5/747: dread d0/dd/d5c/d92/ddb/d35/d61/fa5 [0,4194304] 0 2024-09-21T04:35:14.687 INFO:tasks.workunit.client.1.smithi053.stdout:9/676: truncate d0/d1c/d52/daa/fbc 2327793 0 2024-09-21T04:35:14.688 INFO:tasks.workunit.client.1.smithi053.stdout:9/677: fdatasync d0/d2/d9/de7/fce 0 2024-09-21T04:35:14.689 INFO:tasks.workunit.client.1.smithi053.stdout:8/743: dread d3/d5/d38/d64/d9d/da9/fe3 [0,4194304] 0 2024-09-21T04:35:14.689 INFO:tasks.workunit.client.1.smithi053.stdout:6/700: symlink d2/d1b/d3d/d49/de/d32/d80/le5 0 2024-09-21T04:35:14.690 INFO:tasks.workunit.client.1.smithi053.stdout:6/701: stat d2/d1b/d3d/d49/de/f7f 0 2024-09-21T04:35:14.692 INFO:tasks.workunit.client.1.smithi053.stdout:2/739: creat d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/f10c x:0 0 0 2024-09-21T04:35:14.693 INFO:tasks.workunit.client.1.smithi053.stdout:7/620: creat d0/d2b/d36/d64/dc0/fcb x:0 0 0 2024-09-21T04:35:14.693 INFO:tasks.workunit.client.1.smithi053.stdout:2/740: read d1/d18/d1f/d22/d54/fb0 [1933884,46043] 0 2024-09-21T04:35:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:14 smithi007 ceph-mon[27954]: pgmap v28: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 36 MiB/s rd, 168 MiB/s wr, 323 op/s 2024-09-21T04:35:14.694 INFO:tasks.workunit.client.1.smithi053.stdout:9/678: mknod d0/d6c/ced 0 2024-09-21T04:35:14.694 INFO:tasks.workunit.client.1.smithi053.stdout:3/709: symlink d16/d17/d21/d26/l109 0 2024-09-21T04:35:14.695 INFO:tasks.workunit.client.1.smithi053.stdout:8/744: symlink d3/d5/d4b/d5c/d86/lf7 0 2024-09-21T04:35:14.696 INFO:tasks.workunit.client.1.smithi053.stdout:8/745: chown d3/d39 36043 1 2024-09-21T04:35:14.697 INFO:tasks.workunit.client.1.smithi053.stdout:0/673: getdents d5/d10/d4c/db1/dbc/dca/d53 0 2024-09-21T04:35:14.698 INFO:tasks.workunit.client.1.smithi053.stdout:1/727: getdents d7/d63/d96/dc7 0 2024-09-21T04:35:14.699 INFO:tasks.workunit.client.1.smithi053.stdout:6/702: dwrite d2/d1b/d1e/d2a/f99 [0,4194304] 0 2024-09-21T04:35:14.702 INFO:tasks.workunit.client.1.smithi053.stdout:7/621: creat d0/d24/fcc x:0 0 0 2024-09-21T04:35:14.702 INFO:tasks.workunit.client.1.smithi053.stdout:0/674: read d5/dd/f58 [1163675,91228] 0 2024-09-21T04:35:14.703 INFO:tasks.workunit.client.1.smithi053.stdout:2/741: symlink d1/d18/d1f/d22/d2d/dcd/l10d 0 2024-09-21T04:35:14.704 INFO:tasks.workunit.client.1.smithi053.stdout:5/748: rename d0/dd/d5c/d92/ddb/c4d to d0/d8/cf3 0 2024-09-21T04:35:14.707 INFO:tasks.workunit.client.1.smithi053.stdout:8/746: truncate d3/d5/d4b/d5c/d86/fc5 760581 0 2024-09-21T04:35:14.707 INFO:tasks.workunit.client.1.smithi053.stdout:9/679: getdents d0/d1a/dcd 0 2024-09-21T04:35:14.707 INFO:tasks.workunit.client.1.smithi053.stdout:0/675: write d5/de/d1c/f7e [4563606,34050] 0 2024-09-21T04:35:14.708 INFO:tasks.workunit.client.1.smithi053.stdout:6/703: mknod d2/d1b/d3d/d49/de/d9c/ce6 0 2024-09-21T04:35:14.709 INFO:tasks.workunit.client.1.smithi053.stdout:4/782: truncate d1/dd/de/df5/d20/d73/d92/fb7 103341 0 2024-09-21T04:35:14.710 INFO:tasks.workunit.client.1.smithi053.stdout:3/710: dwrite d16/d17/d21/d42/d43/d86/fc7 [0,4194304] 0 2024-09-21T04:35:14.710 INFO:tasks.workunit.client.1.smithi053.stdout:8/747: write d3/d4f/fe9 [32270,11567] 0 2024-09-21T04:35:14.711 INFO:tasks.workunit.client.1.smithi053.stdout:9/680: chown d0/d1c/d21/c84 403 1 2024-09-21T04:35:14.712 INFO:tasks.workunit.client.1.smithi053.stdout:5/749: unlink d0/d8/f69 0 2024-09-21T04:35:14.712 INFO:tasks.workunit.client.1.smithi053.stdout:4/783: chown d1/d27/f86 2803890 1 2024-09-21T04:35:14.713 INFO:tasks.workunit.client.1.smithi053.stdout:9/681: chown d0/d28/f9b 47916 1 2024-09-21T04:35:14.714 INFO:tasks.workunit.client.1.smithi053.stdout:0/676: chown d5/de/d9e/dd8/ldc 31787 1 2024-09-21T04:35:14.716 INFO:tasks.workunit.client.1.smithi053.stdout:5/750: dread - d0/dd/d5c/d89/fbd zero size 2024-09-21T04:35:14.716 INFO:tasks.workunit.client.1.smithi053.stdout:3/711: fdatasync d16/d17/d21/d26/d2b/fb7 0 2024-09-21T04:35:14.716 INFO:tasks.workunit.client.1.smithi053.stdout:2/742: creat d1/d18/d1f/d27/d58/d5c/d91/dc1/f10e x:0 0 0 2024-09-21T04:35:14.719 INFO:tasks.workunit.client.1.smithi053.stdout:5/751: readlink d0/dd/d5c/d92/ddb/d35/d61/l81 0 2024-09-21T04:35:14.719 INFO:tasks.workunit.client.1.smithi053.stdout:9/682: creat d0/df/fee x:0 0 0 2024-09-21T04:35:14.721 INFO:tasks.workunit.client.1.smithi053.stdout:5/752: readlink d0/d8/d2a/l53 0 2024-09-21T04:35:14.721 INFO:tasks.workunit.client.1.smithi053.stdout:3/712: creat d16/d17/ddc/f10a x:0 0 0 2024-09-21T04:35:14.726 INFO:tasks.workunit.client.1.smithi053.stdout:9/683: creat d0/d1c/d21/d8e/fef x:0 0 0 2024-09-21T04:35:14.726 INFO:tasks.workunit.client.1.smithi053.stdout:4/784: dwrite d1/d27/d90/fc0 [0,4194304] 0 2024-09-21T04:35:14.728 INFO:tasks.workunit.client.1.smithi053.stdout:2/743: rename d1/d18/d1f/d22/d54/f68 to d1/d18/d1f/d27/d58/d5c/d91/dd0/f10f 0 2024-09-21T04:35:14.732 INFO:tasks.workunit.client.1.smithi053.stdout:8/748: creat d3/d5/d38/d64/ff8 x:0 0 0 2024-09-21T04:35:14.732 INFO:tasks.workunit.client.1.smithi053.stdout:1/728: dwrite d7/d9/d43/d94/fb9 [0,4194304] 0 2024-09-21T04:35:14.735 INFO:tasks.workunit.client.1.smithi053.stdout:0/677: rmdir d5/de/d42/d78/dd0 0 2024-09-21T04:35:14.736 INFO:tasks.workunit.client.0.smithi007.stdout:0/103: symlink d6/l24 0 2024-09-21T04:35:14.736 INFO:tasks.workunit.client.1.smithi053.stdout:2/744: write d1/d18/d1f/d27/d58/f108 [959264,6854] 0 2024-09-21T04:35:14.737 INFO:tasks.workunit.client.0.smithi007.stdout:7/214: mkdir d3/d6/d10/d51 0 2024-09-21T04:35:14.738 INFO:tasks.workunit.client.0.smithi007.stdout:5/321: dwrite d2/d27/d3d/f4e [0,4194304] 0 2024-09-21T04:35:14.739 INFO:tasks.workunit.client.0.smithi007.stdout:8/126: write d0/d3/d8/f1d [3310977,28033] 0 2024-09-21T04:35:14.740 INFO:tasks.workunit.client.0.smithi007.stdout:4/184: rename d2/l21 to d2/d17/l3b 0 2024-09-21T04:35:14.740 INFO:tasks.workunit.client.1.smithi053.stdout:9/684: truncate d0/d1a/d4c/fa4 1105258 0 2024-09-21T04:35:14.741 INFO:tasks.workunit.client.0.smithi007.stdout:1/223: rmdir d5/d2f 39 2024-09-21T04:35:14.741 INFO:tasks.workunit.client.0.smithi007.stdout:7/215: chown d3/d6/d10/l1c 867426083 1 2024-09-21T04:35:14.742 INFO:tasks.workunit.client.0.smithi007.stdout:4/185: stat d2/d17 0 2024-09-21T04:35:14.744 INFO:tasks.workunit.client.0.smithi007.stdout:6/76: truncate d3/de/d12/f17 457020 0 2024-09-21T04:35:14.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/186: mkdir d2/d3c 0 2024-09-21T04:35:14.748 INFO:tasks.workunit.client.0.smithi007.stdout:6/77: fsync d3/de/d12/f14 0 2024-09-21T04:35:14.750 INFO:tasks.workunit.client.0.smithi007.stdout:0/104: rename d6/f20 to d6/d16/f25 0 2024-09-21T04:35:14.750 INFO:tasks.workunit.client.0.smithi007.stdout:7/216: write d3/d6/d14/f45 [7500566,119192] 0 2024-09-21T04:35:14.751 INFO:tasks.workunit.client.1.smithi053.stdout:2/745: write d1/d18/d1f/d27/d41/d4b/d4d/f88 [4445724,68091] 0 2024-09-21T04:35:14.751 INFO:tasks.workunit.client.1.smithi053.stdout:8/749: dwrite d3/d5/d21/fe2 [0,4194304] 0 2024-09-21T04:35:14.753 INFO:tasks.workunit.client.1.smithi053.stdout:0/678: readlink d5/d10/d4c/db1/dbc/l80 0 2024-09-21T04:35:14.754 INFO:tasks.workunit.client.0.smithi007.stdout:8/127: dwrite d0/d3/d8/f1d [0,4194304] 0 2024-09-21T04:35:14.756 INFO:tasks.workunit.client.0.smithi007.stdout:0/105: write d6/f12 [33523,33650] 0 2024-09-21T04:35:14.756 INFO:tasks.workunit.client.0.smithi007.stdout:4/187: mknod d2/d17/d2e/c3d 0 2024-09-21T04:35:14.757 INFO:tasks.workunit.client.0.smithi007.stdout:1/224: dwrite d5/d6/d21/f2c [0,4194304] 0 2024-09-21T04:35:14.758 INFO:tasks.workunit.client.1.smithi053.stdout:2/746: fsync d1/d18/d77/d83/f8d 0 2024-09-21T04:35:14.759 INFO:tasks.workunit.client.0.smithi007.stdout:6/78: creat d3/de/f1a x:0 0 0 2024-09-21T04:35:14.759 INFO:tasks.workunit.client.1.smithi053.stdout:3/713: dread d16/d1b/f59 [0,4194304] 0 2024-09-21T04:35:14.759 INFO:tasks.workunit.client.0.smithi007.stdout:8/128: stat d0/d3/d8/lc 0 2024-09-21T04:35:14.759 INFO:tasks.workunit.client.1.smithi053.stdout:0/679: dread - d5/de/fd7 zero size 2024-09-21T04:35:14.760 INFO:tasks.workunit.client.1.smithi053.stdout:8/750: write d3/df/f81 [1284909,52464] 0 2024-09-21T04:35:14.760 INFO:tasks.workunit.client.1.smithi053.stdout:4/785: rename d1/d27/d53/d88/ld0 to d1/l101 0 2024-09-21T04:35:14.764 INFO:tasks.workunit.client.1.smithi053.stdout:9/685: dwrite d0/d28/f38 [4194304,4194304] 0 2024-09-21T04:35:14.765 INFO:tasks.workunit.client.0.smithi007.stdout:5/322: rename d2/d1e/f22 to d2/dd/d23/d5d/d61/f65 0 2024-09-21T04:35:14.766 INFO:tasks.workunit.client.1.smithi053.stdout:1/729: dread d7/d9/da/d3c/f46 [0,4194304] 0 2024-09-21T04:35:14.767 INFO:tasks.workunit.client.1.smithi053.stdout:9/686: write d0/f24 [1265721,68953] 0 2024-09-21T04:35:14.767 INFO:tasks.workunit.client.1.smithi053.stdout:0/680: fsync d5/d10/d4c/db1/dbc/dca/d21/fc9 0 2024-09-21T04:35:14.768 INFO:tasks.workunit.client.0.smithi007.stdout:0/106: dwrite d6/f12 [4194304,4194304] 0 2024-09-21T04:35:14.768 INFO:tasks.workunit.client.1.smithi053.stdout:5/753: getdents d0 0 2024-09-21T04:35:14.769 INFO:tasks.workunit.client.1.smithi053.stdout:2/747: fsync d1/d18/d1f/d27/d41/d4b/d4d/faf 0 2024-09-21T04:35:14.779 INFO:tasks.workunit.client.1.smithi053.stdout:9/687: dwrite d0/d1c/d21/d8e/fba [0,4194304] 0 2024-09-21T04:35:14.781 INFO:tasks.workunit.client.1.smithi053.stdout:4/786: fsync d1/dd/de/df5/d20/f34 0 2024-09-21T04:35:14.782 INFO:tasks.workunit.client.1.smithi053.stdout:5/754: chown d0/dd/d5c/d92/ddb/d35/c41 47831096 1 2024-09-21T04:35:14.784 INFO:tasks.workunit.client.1.smithi053.stdout:5/755: write d0/dd/d5c/d89/fe1 [431012,68164] 0 2024-09-21T04:35:14.784 INFO:tasks.workunit.client.1.smithi053.stdout:4/787: read d1/dd/de/f6d [3110961,68439] 0 2024-09-21T04:35:14.793 INFO:tasks.workunit.client.1.smithi053.stdout:8/751: truncate d3/df/d14/d16/f23 1236540 0 2024-09-21T04:35:14.793 INFO:tasks.workunit.client.0.smithi007.stdout:6/79: link d3/de/d12/f17 d3/de/d12/d19/f1b 0 2024-09-21T04:35:14.794 INFO:tasks.workunit.client.1.smithi053.stdout:9/688: mknod d0/d2/d2b/cf0 0 2024-09-21T04:35:14.796 INFO:tasks.workunit.client.0.smithi007.stdout:0/107: symlink d6/d16/l26 0 2024-09-21T04:35:14.796 INFO:tasks.workunit.client.0.smithi007.stdout:4/188: write d2/d17/f26 [568926,4038] 0 2024-09-21T04:35:14.799 INFO:tasks.workunit.client.0.smithi007.stdout:4/189: write d2/f23 [2573197,8387] 0 2024-09-21T04:35:14.799 INFO:tasks.workunit.client.1.smithi053.stdout:2/748: dwrite d1/d18/d1f/d27/d58/d5c/d91/dd0/f10f [4194304,4194304] 0 2024-09-21T04:35:14.800 INFO:tasks.workunit.client.0.smithi007.stdout:8/129: rename d0/f6 to d0/f20 0 2024-09-21T04:35:14.802 INFO:tasks.workunit.client.0.smithi007.stdout:1/225: dwrite d5/f28 [0,4194304] 0 2024-09-21T04:35:14.802 INFO:tasks.workunit.client.1.smithi053.stdout:5/756: dwrite d0/dd/d5c/d92/ddb/d51/d52/fa7 [0,4194304] 0 2024-09-21T04:35:14.804 INFO:tasks.workunit.client.1.smithi053.stdout:5/757: fsync d0/dd/d67/fe9 0 2024-09-21T04:35:14.806 INFO:tasks.workunit.client.0.smithi007.stdout:5/323: link d2/f19 d2/f66 0 2024-09-21T04:35:14.807 INFO:tasks.workunit.client.0.smithi007.stdout:7/217: truncate d3/d6/fa 4145328 0 2024-09-21T04:35:14.809 INFO:tasks.workunit.client.0.smithi007.stdout:7/218: read d3/d6/f2d [708905,84139] 0 2024-09-21T04:35:14.810 INFO:tasks.workunit.client.1.smithi053.stdout:3/714: dwrite d16/d1b/dad/fb1 [0,4194304] 0 2024-09-21T04:35:14.810 INFO:tasks.workunit.client.0.smithi007.stdout:7/219: dread - d3/f4c zero size 2024-09-21T04:35:14.815 INFO:tasks.workunit.client.0.smithi007.stdout:5/324: dwrite d2/d1e/f24 [4194304,4194304] 0 2024-09-21T04:35:14.817 INFO:tasks.workunit.client.0.smithi007.stdout:0/108: mkdir d6/d16/d18/d27 0 2024-09-21T04:35:14.819 INFO:tasks.workunit.client.0.smithi007.stdout:6/80: creat d3/de/d12/d19/f1c x:0 0 0 2024-09-21T04:35:14.820 INFO:tasks.workunit.client.0.smithi007.stdout:6/81: chown d3 22 1 2024-09-21T04:35:14.822 INFO:tasks.workunit.client.0.smithi007.stdout:6/82: chown d3/de/d12/d19 523790257 1 2024-09-21T04:35:14.822 INFO:tasks.workunit.client.0.smithi007.stdout:7/220: dwrite d3/d6/d14/d20/f37 [0,4194304] 0 2024-09-21T04:35:14.824 INFO:tasks.workunit.client.0.smithi007.stdout:1/226: creat d5/d6/d21/d3c/f43 x:0 0 0 2024-09-21T04:35:14.825 INFO:tasks.workunit.client.0.smithi007.stdout:7/221: write d3/d6/f2d [2893268,49652] 0 2024-09-21T04:35:14.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/109: dwrite f1 [0,4194304] 0 2024-09-21T04:35:14.840 INFO:tasks.workunit.client.0.smithi007.stdout:0/110: dwrite d6/f14 [0,4194304] 0 2024-09-21T04:35:14.845 INFO:tasks.workunit.client.0.smithi007.stdout:5/325: rename d2/dd/d23/d49/d44/l52 to d2/dd/l67 0 2024-09-21T04:35:14.846 INFO:tasks.workunit.client.0.smithi007.stdout:6/83: rename d3/de/d12 to d3/de/d12/d1d 22 2024-09-21T04:35:14.846 INFO:tasks.workunit.client.0.smithi007.stdout:6/84: stat d3/de/d12 0 2024-09-21T04:35:14.847 INFO:tasks.workunit.client.0.smithi007.stdout:1/227: fsync d5/f3e 0 2024-09-21T04:35:14.849 INFO:tasks.workunit.client.0.smithi007.stdout:7/222: mknod d3/d6/d10/c52 0 2024-09-21T04:35:14.856 INFO:tasks.workunit.client.0.smithi007.stdout:7/223: dwrite d3/d6/d10/f24 [0,4194304] 0 2024-09-21T04:35:14.856 INFO:tasks.workunit.client.0.smithi007.stdout:8/130: link d0/d3/d8/f9 d0/d3/d16/f21 0 2024-09-21T04:35:14.863 INFO:tasks.workunit.client.0.smithi007.stdout:7/224: dwrite d3/d6/f28 [0,4194304] 0 2024-09-21T04:35:14.869 INFO:tasks.workunit.client.0.smithi007.stdout:7/225: dread d3/d6/d14/d20/d33/f25 [0,4194304] 0 2024-09-21T04:35:14.872 INFO:tasks.workunit.client.0.smithi007.stdout:5/326: creat d2/d27/d3d/d56/f68 x:0 0 0 2024-09-21T04:35:14.877 INFO:tasks.workunit.client.0.smithi007.stdout:7/226: symlink d3/d6/d13/l53 0 2024-09-21T04:35:14.878 INFO:tasks.workunit.client.0.smithi007.stdout:7/227: stat d3/d6/d10/f34 0 2024-09-21T04:35:14.881 INFO:tasks.workunit.client.0.smithi007.stdout:5/327: creat d2/d27/d3d/f69 x:0 0 0 2024-09-21T04:35:14.882 INFO:tasks.workunit.client.1.smithi053.stdout:9/689: fsync d0/df/f48 0 2024-09-21T04:35:14.883 INFO:tasks.workunit.client.1.smithi053.stdout:4/788: dwrite d1/d27/d2e/d46/fbd [0,4194304] 0 2024-09-21T04:35:14.884 INFO:tasks.workunit.client.1.smithi053.stdout:9/690: write d0/d2/d9/d34/f56 [2763801,90809] 0 2024-09-21T04:35:14.886 INFO:tasks.workunit.client.1.smithi053.stdout:4/789: write d1/dd/de/df5/d3f/d74/f8b [409652,80482] 0 2024-09-21T04:35:14.889 INFO:tasks.workunit.client.1.smithi053.stdout:2/749: rmdir d1/d18/d1f/d27/d58/d5c/d91/dd0 39 2024-09-21T04:35:14.891 INFO:tasks.workunit.client.0.smithi007.stdout:0/111: rename d6/f14 to d6/d16/f28 0 2024-09-21T04:35:14.891 INFO:tasks.workunit.client.1.smithi053.stdout:5/758: write d0/dd/d5c/f7a [1795639,25097] 0 2024-09-21T04:35:14.892 INFO:tasks.workunit.client.0.smithi007.stdout:5/328: rmdir d2/dd/d23/d2a 39 2024-09-21T04:35:14.897 INFO:tasks.workunit.client.1.smithi053.stdout:0/681: creat d5/d10/d4c/db1/dbc/dca/d21/fe7 x:0 0 0 2024-09-21T04:35:14.897 INFO:tasks.workunit.client.1.smithi053.stdout:0/682: chown d5/d10/lcc 589702375 1 2024-09-21T04:35:14.899 INFO:tasks.workunit.client.0.smithi007.stdout:6/85: rename d3/f8 to d3/f1e 0 2024-09-21T04:35:14.899 INFO:tasks.workunit.client.1.smithi053.stdout:0/683: dread d5/d10/d4c/db1/dbc/dca/d21/fc4 [0,4194304] 0 2024-09-21T04:35:14.901 INFO:tasks.workunit.client.0.smithi007.stdout:7/228: mknod d3/d6/d10/d43/c54 0 2024-09-21T04:35:14.902 INFO:tasks.workunit.client.0.smithi007.stdout:0/112: dwrite d6/f12 [4194304,4194304] 0 2024-09-21T04:35:14.902 INFO:tasks.workunit.client.1.smithi053.stdout:8/752: mkdir d3/d28/ddc/df9 0 2024-09-21T04:35:14.902 INFO:tasks.workunit.client.1.smithi053.stdout:1/730: getdents d7/d63/d6b 0 2024-09-21T04:35:14.903 INFO:tasks.workunit.client.1.smithi053.stdout:1/731: dread - d7/d63/d6b/f8a zero size 2024-09-21T04:35:14.903 INFO:tasks.workunit.client.0.smithi007.stdout:0/113: chown f1 2650 1 2024-09-21T04:35:14.904 INFO:tasks.workunit.client.0.smithi007.stdout:7/229: fsync d3/d6/d14/d20/d33/f41 0 2024-09-21T04:35:14.904 INFO:tasks.workunit.client.0.smithi007.stdout:0/114: chown d6/d16/d18/d27 9 1 2024-09-21T04:35:14.905 INFO:tasks.workunit.client.0.smithi007.stdout:1/228: getdents d5/d2f 0 2024-09-21T04:35:14.906 INFO:tasks.workunit.client.0.smithi007.stdout:7/230: dread - d3/d6/d14/d20/d33/f41 zero size 2024-09-21T04:35:14.907 INFO:tasks.workunit.client.0.smithi007.stdout:0/115: write d6/fd [26997,103062] 0 2024-09-21T04:35:14.909 INFO:tasks.workunit.client.0.smithi007.stdout:6/86: dread - d3/f15 zero size 2024-09-21T04:35:14.918 INFO:tasks.workunit.client.0.smithi007.stdout:0/116: write d6/d16/f22 [421688,23781] 0 2024-09-21T04:35:14.920 INFO:tasks.workunit.client.0.smithi007.stdout:5/329: creat d2/dd/d23/d49/f6a x:0 0 0 2024-09-21T04:35:14.923 INFO:tasks.workunit.client.0.smithi007.stdout:6/87: truncate d3/f6 4017667 0 2024-09-21T04:35:14.924 INFO:tasks.workunit.client.1.smithi053.stdout:2/750: symlink d1/d18/d1f/d22/d2d/deb/l110 0 2024-09-21T04:35:14.925 INFO:tasks.workunit.client.1.smithi053.stdout:3/715: mknod d16/d17/c10b 0 2024-09-21T04:35:14.925 INFO:tasks.workunit.client.1.smithi053.stdout:5/759: chown d0/d8/f2b 112 1 2024-09-21T04:35:14.925 INFO:tasks.workunit.client.1.smithi053.stdout:0/684: rmdir d5/de 39 2024-09-21T04:35:14.926 INFO:tasks.workunit.client.1.smithi053.stdout:8/753: fdatasync d3/d39/f3f 0 2024-09-21T04:35:14.926 INFO:tasks.workunit.client.0.smithi007.stdout:5/330: dread d2/dd/f16 [0,4194304] 0 2024-09-21T04:35:14.927 INFO:tasks.workunit.client.1.smithi053.stdout:8/754: dread - d3/d28/fd5 zero size 2024-09-21T04:35:14.927 INFO:tasks.workunit.client.1.smithi053.stdout:1/732: creat d7/d47/d79/ff3 x:0 0 0 2024-09-21T04:35:14.930 INFO:tasks.workunit.client.0.smithi007.stdout:0/117: getdents d6/d16 0 2024-09-21T04:35:14.931 INFO:tasks.workunit.client.1.smithi053.stdout:3/716: dread d16/d17/d21/d26/d2b/fb7 [0,4194304] 0 2024-09-21T04:35:14.931 INFO:tasks.workunit.client.0.smithi007.stdout:1/229: write d5/d6/fa [1033043,95119] 0 2024-09-21T04:35:14.935 INFO:tasks.workunit.client.0.smithi007.stdout:5/331: chown d2/dd/l67 4 1 2024-09-21T04:35:14.935 INFO:tasks.workunit.client.0.smithi007.stdout:6/88: dwrite d3/f18 [0,4194304] 0 2024-09-21T04:35:14.937 INFO:tasks.workunit.client.0.smithi007.stdout:1/230: mknod d5/d2f/c44 0 2024-09-21T04:35:14.938 INFO:tasks.workunit.client.1.smithi053.stdout:0/685: symlink d5/d10/d4c/db1/dbc/dca/db5/dd5/le8 0 2024-09-21T04:35:14.938 INFO:tasks.workunit.client.1.smithi053.stdout:2/751: dwrite d1/d18/f67 [0,4194304] 0 2024-09-21T04:35:14.939 INFO:tasks.workunit.client.0.smithi007.stdout:7/231: dread d3/d6/fa [0,4194304] 0 2024-09-21T04:35:14.939 INFO:tasks.workunit.client.1.smithi053.stdout:1/733: mkdir d7/d9/da/d4c/df4 0 2024-09-21T04:35:14.941 INFO:tasks.workunit.client.0.smithi007.stdout:7/232: fsync d3/d6/d14/f45 0 2024-09-21T04:35:14.941 INFO:tasks.workunit.client.0.smithi007.stdout:1/231: dread d5/fc [0,4194304] 0 2024-09-21T04:35:14.942 INFO:tasks.workunit.client.0.smithi007.stdout:5/332: rmdir d2/dd/d23 39 2024-09-21T04:35:14.944 INFO:tasks.workunit.client.0.smithi007.stdout:1/232: write d5/f26 [2815226,104359] 0 2024-09-21T04:35:14.944 INFO:tasks.workunit.client.0.smithi007.stdout:0/118: rename d6/d16/d18/f1d to d6/d16/d18/d27/f29 0 2024-09-21T04:35:14.945 INFO:tasks.workunit.client.1.smithi053.stdout:3/717: rmdir d16/d17/d21/d42/da0 39 2024-09-21T04:35:14.948 INFO:tasks.workunit.client.0.smithi007.stdout:6/89: creat d3/de/d12/f1f x:0 0 0 2024-09-21T04:35:14.949 INFO:tasks.workunit.client.0.smithi007.stdout:6/90: chown d3/de/d12/f17 8 1 2024-09-21T04:35:14.950 INFO:tasks.workunit.client.1.smithi053.stdout:2/752: creat d1/d18/d1f/d22/d2d/deb/f111 x:0 0 0 2024-09-21T04:35:14.951 INFO:tasks.workunit.client.1.smithi053.stdout:9/691: dwrite d0/d1a/d4c/d8f/f97 [0,4194304] 0 2024-09-21T04:35:14.951 INFO:tasks.workunit.client.0.smithi007.stdout:6/91: fsync d3/de/f10 0 2024-09-21T04:35:14.952 INFO:tasks.workunit.client.1.smithi053.stdout:2/753: write d1/d18/d1f/d27/fa0 [671439,69611] 0 2024-09-21T04:35:14.954 INFO:tasks.workunit.client.1.smithi053.stdout:9/692: chown d0/d1c/d21/d46/f91 3 1 2024-09-21T04:35:14.955 INFO:tasks.workunit.client.0.smithi007.stdout:0/119: dwrite d6/fd [0,4194304] 0 2024-09-21T04:35:14.957 INFO:tasks.workunit.client.0.smithi007.stdout:7/233: symlink d3/d6/d14/l55 0 2024-09-21T04:35:14.957 INFO:tasks.workunit.client.0.smithi007.stdout:1/233: dread d5/d6/d21/d2b/f2d [0,4194304] 0 2024-09-21T04:35:14.958 INFO:tasks.workunit.client.1.smithi053.stdout:3/718: symlink d16/d17/d21/d8b/d7d/d57/d90/l10c 0 2024-09-21T04:35:14.958 INFO:tasks.workunit.client.0.smithi007.stdout:7/234: chown d3/d6/d13/f47 52767899 1 2024-09-21T04:35:14.961 INFO:tasks.workunit.client.0.smithi007.stdout:5/333: write d2/f51 [502267,68905] 0 2024-09-21T04:35:14.964 INFO:tasks.workunit.client.1.smithi053.stdout:2/754: dread - d1/d18/d1f/fb8 zero size 2024-09-21T04:35:14.969 INFO:tasks.workunit.client.1.smithi053.stdout:0/686: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/f98 [0,4194304] 0 2024-09-21T04:35:14.970 INFO:tasks.workunit.client.0.smithi007.stdout:6/92: readlink d3/l4 0 2024-09-21T04:35:14.971 INFO:tasks.workunit.client.1.smithi053.stdout:2/755: mkdir d1/d18/d1f/d27/d58/db7/d112 0 2024-09-21T04:35:14.973 INFO:tasks.workunit.client.1.smithi053.stdout:7/622: sync 2024-09-21T04:35:14.974 INFO:tasks.workunit.client.1.smithi053.stdout:2/756: read d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/ff0 [383982,84649] 0 2024-09-21T04:35:14.975 INFO:tasks.workunit.client.1.smithi053.stdout:9/693: dread d0/d2/d9/fcc [0,4194304] 0 2024-09-21T04:35:14.975 INFO:tasks.workunit.client.0.smithi007.stdout:0/120: creat d6/d16/f2a x:0 0 0 2024-09-21T04:35:14.975 INFO:tasks.workunit.client.0.smithi007.stdout:6/93: dread d3/f18 [0,4194304] 0 2024-09-21T04:35:14.979 INFO:tasks.workunit.client.0.smithi007.stdout:0/121: chown d6 317 1 2024-09-21T04:35:14.979 INFO:tasks.workunit.client.1.smithi053.stdout:0/687: chown d5/de/d42/d78/c8a 176335 1 2024-09-21T04:35:14.980 INFO:tasks.workunit.client.0.smithi007.stdout:1/234: dread d5/f26 [0,4194304] 0 2024-09-21T04:35:14.982 INFO:tasks.workunit.client.0.smithi007.stdout:7/235: mknod d3/d6/d10/c56 0 2024-09-21T04:35:14.983 INFO:tasks.workunit.client.1.smithi053.stdout:2/757: write d1/d18/d1f/d22/d43/f104 [1304,114697] 0 2024-09-21T04:35:14.991 INFO:tasks.workunit.client.0.smithi007.stdout:6/94: rename d3/de/f11 to d3/de/d12/d19/f20 0 2024-09-21T04:35:14.991 INFO:tasks.workunit.client.0.smithi007.stdout:5/334: rename d2 to d2/dd/d23/d49/d44/d6b 22 2024-09-21T04:35:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:6/95: truncate d3/de/d12/d19/f1c 711141 0 2024-09-21T04:35:14.995 INFO:tasks.workunit.client.0.smithi007.stdout:6/96: dread - d3/de/d12/f14 zero size 2024-09-21T04:35:14.997 INFO:tasks.workunit.client.0.smithi007.stdout:5/335: dread d2/f1b [0,4194304] 0 2024-09-21T04:35:14.998 INFO:tasks.workunit.client.0.smithi007.stdout:1/235: unlink d5/d6/d21/d23/f35 0 2024-09-21T04:35:14.999 INFO:tasks.workunit.client.0.smithi007.stdout:1/236: chown d5/fe 288177 1 2024-09-21T04:35:15.003 INFO:tasks.workunit.client.0.smithi007.stdout:5/336: rmdir d2/d27/d3d/d56 39 2024-09-21T04:35:15.004 INFO:tasks.workunit.client.0.smithi007.stdout:6/97: truncate d3/de/d12/d19/f1b 473864 0 2024-09-21T04:35:15.005 INFO:tasks.workunit.client.0.smithi007.stdout:7/236: creat d3/d6/d13/d4b/f57 x:0 0 0 2024-09-21T04:35:15.005 INFO:tasks.workunit.client.0.smithi007.stdout:1/237: rmdir d5/d6/d21/d3c 39 2024-09-21T04:35:15.006 INFO:tasks.workunit.client.0.smithi007.stdout:7/237: chown d3/d6/cb 116 1 2024-09-21T04:35:15.006 INFO:tasks.workunit.client.0.smithi007.stdout:0/122: creat d6/d16/d18/f2b x:0 0 0 2024-09-21T04:35:15.007 INFO:tasks.workunit.client.0.smithi007.stdout:1/238: write d5/d6/d21/f30 [746359,113369] 0 2024-09-21T04:35:15.008 INFO:tasks.workunit.client.0.smithi007.stdout:0/123: chown c2 259636 1 2024-09-21T04:35:15.011 INFO:tasks.workunit.client.0.smithi007.stdout:5/337: link d2/dd/f3e d2/d1e/f6c 0 2024-09-21T04:35:15.012 INFO:tasks.workunit.client.0.smithi007.stdout:7/238: mkdir d3/d6/d14/d20/d58 0 2024-09-21T04:35:15.015 INFO:tasks.workunit.client.0.smithi007.stdout:6/98: getdents d3/de 0 2024-09-21T04:35:15.020 INFO:tasks.workunit.client.0.smithi007.stdout:6/99: creat d3/de/d12/f21 x:0 0 0 2024-09-21T04:35:15.022 INFO:tasks.workunit.client.0.smithi007.stdout:1/239: dwrite d5/d6/d21/f3f [0,4194304] 0 2024-09-21T04:35:15.025 INFO:tasks.workunit.client.0.smithi007.stdout:0/124: dwrite d6/fd [0,4194304] 0 2024-09-21T04:35:15.025 INFO:tasks.workunit.client.0.smithi007.stdout:6/100: symlink d3/de/l22 0 2024-09-21T04:35:15.026 INFO:tasks.workunit.client.0.smithi007.stdout:1/240: write d5/d2f/f34 [2214504,80503] 0 2024-09-21T04:35:15.028 INFO:tasks.workunit.client.0.smithi007.stdout:5/338: dwrite d2/f1b [0,4194304] 0 2024-09-21T04:35:15.028 INFO:tasks.workunit.client.0.smithi007.stdout:6/101: fsync d3/de/d12/d19/f1c 0 2024-09-21T04:35:15.033 INFO:tasks.workunit.client.0.smithi007.stdout:5/339: creat d2/dd/d1d/d20/f6d x:0 0 0 2024-09-21T04:35:15.036 INFO:tasks.workunit.client.0.smithi007.stdout:1/241: mkdir d5/d45 0 2024-09-21T04:35:15.037 INFO:tasks.workunit.client.0.smithi007.stdout:7/239: dwrite d3/d6/d14/d20/d33/f50 [0,4194304] 0 2024-09-21T04:35:15.040 INFO:tasks.workunit.client.0.smithi007.stdout:5/340: creat d2/dd/d1d/f6e x:0 0 0 2024-09-21T04:35:15.041 INFO:tasks.workunit.client.1.smithi053.stdout:9/694: dread d0/d2/f55 [0,4194304] 0 2024-09-21T04:35:15.041 INFO:tasks.workunit.client.1.smithi053.stdout:3/719: creat d16/d17/f10d x:0 0 0 2024-09-21T04:35:15.043 INFO:tasks.workunit.client.0.smithi007.stdout:1/242: rename f3 to d5/d45/f46 0 2024-09-21T04:35:15.043 INFO:tasks.workunit.client.1.smithi053.stdout:7/623: mkdir d0/dcd 0 2024-09-21T04:35:15.044 INFO:tasks.workunit.client.0.smithi007.stdout:0/125: truncate d6/fd 957982 0 2024-09-21T04:35:15.046 INFO:tasks.workunit.client.0.smithi007.stdout:7/240: symlink d3/d6/d13/l59 0 2024-09-21T04:35:15.046 INFO:tasks.workunit.client.0.smithi007.stdout:5/341: dread d2/d1e/f24 [4194304,4194304] 0 2024-09-21T04:35:15.050 INFO:tasks.workunit.client.1.smithi053.stdout:1/734: dwrite d7/d9/d43/d54/d7f/d36/f4e [0,4194304] 0 2024-09-21T04:35:15.052 INFO:tasks.workunit.client.0.smithi007.stdout:0/126: truncate d6/d16/f25 55376 0 2024-09-21T04:35:15.053 INFO:tasks.workunit.client.0.smithi007.stdout:7/241: symlink d3/d6/d14/d20/l5a 0 2024-09-21T04:35:15.053 INFO:tasks.workunit.client.1.smithi053.stdout:2/758: dwrite d1/d18/f2a [4194304,4194304] 0 2024-09-21T04:35:15.053 INFO:tasks.workunit.client.1.smithi053.stdout:7/624: dread d0/d24/f5a [0,4194304] 0 2024-09-21T04:35:15.055 INFO:tasks.workunit.client.1.smithi053.stdout:2/759: write d1/d18/d1f/d27/d58/d5c/fc0 [537001,13910] 0 2024-09-21T04:35:15.058 INFO:tasks.workunit.client.1.smithi053.stdout:7/625: read d0/d6f/dbf/d50/fa4 [66906,12692] 0 2024-09-21T04:35:15.059 INFO:tasks.workunit.client.1.smithi053.stdout:6/704: sync 2024-09-21T04:35:15.059 INFO:tasks.workunit.client.1.smithi053.stdout:2/760: write d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/fe1 [486483,80971] 0 2024-09-21T04:35:15.060 INFO:tasks.workunit.client.1.smithi053.stdout:7/626: dread - d0/d6f/dbf/d50/d8b/f81 zero size 2024-09-21T04:35:15.062 INFO:tasks.workunit.client.0.smithi007.stdout:5/342: mkdir d2/dd/d23/d49/d6f 0 2024-09-21T04:35:15.063 INFO:tasks.workunit.client.0.smithi007.stdout:5/343: chown d2/dd/d23/d49/f6a 7932418 1 2024-09-21T04:35:15.064 INFO:tasks.workunit.client.0.smithi007.stdout:5/344: fsync d2/d27/d42/f62 0 2024-09-21T04:35:15.066 INFO:tasks.workunit.client.1.smithi053.stdout:0/688: write d5/de/d42/f89 [643179,17043] 0 2024-09-21T04:35:15.067 INFO:tasks.workunit.client.0.smithi007.stdout:1/243: creat d5/d6/d21/f47 x:0 0 0 2024-09-21T04:35:15.071 INFO:tasks.workunit.client.0.smithi007.stdout:0/127: rename d6/c10 to d6/d16/d18/d27/c2c 0 2024-09-21T04:35:15.072 INFO:tasks.workunit.client.1.smithi053.stdout:0/689: dwrite d5/de/d42/d78/fb8 [0,4194304] 0 2024-09-21T04:35:15.072 INFO:tasks.workunit.client.0.smithi007.stdout:7/242: mknod d3/d6/d14/d20/d58/c5b 0 2024-09-21T04:35:15.073 INFO:tasks.workunit.client.1.smithi053.stdout:1/735: truncate d7/d9/da/d3c/f65 475708 0 2024-09-21T04:35:15.074 INFO:tasks.workunit.client.1.smithi053.stdout:9/695: getdents d0/d1c/d52/daa 0 2024-09-21T04:35:15.075 INFO:tasks.workunit.client.1.smithi053.stdout:1/736: chown d7/d9/d43/d54/d7f/fb6 322117049 1 2024-09-21T04:35:15.076 INFO:tasks.workunit.client.1.smithi053.stdout:3/720: truncate d16/d17/d21/d26/d2b/faa 689240 0 2024-09-21T04:35:15.078 INFO:tasks.workunit.client.0.smithi007.stdout:0/128: rename d6/d16/d18/l23 to d6/d16/d18/l2d 0 2024-09-21T04:35:15.079 INFO:tasks.workunit.client.0.smithi007.stdout:0/129: stat d6/fd 0 2024-09-21T04:35:15.079 INFO:tasks.workunit.client.1.smithi053.stdout:6/705: chown d2/d1b/d3d/d49/de/d32/d80/d2e/c8e 231333167 1 2024-09-21T04:35:15.083 INFO:tasks.workunit.client.0.smithi007.stdout:0/130: mkdir d6/d16/d18/d2e 0 2024-09-21T04:35:15.085 INFO:tasks.workunit.client.0.smithi007.stdout:1/244: creat d5/f48 x:0 0 0 2024-09-21T04:35:15.087 INFO:tasks.workunit.client.0.smithi007.stdout:5/345: getdents d2/d27/d42 0 2024-09-21T04:35:15.088 INFO:tasks.workunit.client.0.smithi007.stdout:5/346: chown d2/dd/d28/f4d 115269 1 2024-09-21T04:35:15.091 INFO:tasks.workunit.client.0.smithi007.stdout:1/245: chown d5/d6/c9 41027125 1 2024-09-21T04:35:15.092 INFO:tasks.workunit.client.0.smithi007.stdout:1/246: chown d5/c22 1080 1 2024-09-21T04:35:15.093 INFO:tasks.workunit.client.1.smithi053.stdout:7/627: write d0/d6f/dbf/d50/fa5 [392730,81846] 0 2024-09-21T04:35:15.094 INFO:tasks.workunit.client.1.smithi053.stdout:7/628: read d0/d6f/dbf/d50/f54 [154142,119563] 0 2024-09-21T04:35:15.095 INFO:tasks.workunit.client.1.smithi053.stdout:7/629: fsync d0/d6f/dbf/d50/d8b/d70/f88 0 2024-09-21T04:35:15.098 INFO:tasks.workunit.client.1.smithi053.stdout:0/690: write d5/d10/d4c/db1/dbc/dca/d21/f33 [607267,9957] 0 2024-09-21T04:35:15.099 INFO:tasks.workunit.client.1.smithi053.stdout:0/691: write d5/de/d9e/faf [3695099,120276] 0 2024-09-21T04:35:15.099 INFO:tasks.workunit.client.1.smithi053.stdout:1/737: symlink d7/d9/da/d40/d58/lf5 0 2024-09-21T04:35:15.100 INFO:tasks.workunit.client.0.smithi007.stdout:0/131: getdents d6/d16 0 2024-09-21T04:35:15.101 INFO:tasks.workunit.client.1.smithi053.stdout:0/692: chown d5/de/d42/c75 1167 1 2024-09-21T04:35:15.101 INFO:tasks.workunit.client.1.smithi053.stdout:1/738: dread - d7/d63/d96/dc7/fdc zero size 2024-09-21T04:35:15.101 INFO:tasks.workunit.client.0.smithi007.stdout:1/247: mknod d5/d6/d21/d23/d42/c49 0 2024-09-21T04:35:15.102 INFO:tasks.workunit.client.1.smithi053.stdout:1/739: truncate d7/d9/d43/d54/d7f/d36/f97 767554 0 2024-09-21T04:35:15.103 INFO:tasks.workunit.client.1.smithi053.stdout:1/740: chown d7/d9/da/fd9 3314 1 2024-09-21T04:35:15.103 INFO:tasks.workunit.client.0.smithi007.stdout:0/132: rename d6/c11 to d6/d16/c2f 0 2024-09-21T04:35:15.104 INFO:tasks.workunit.client.1.smithi053.stdout:1/741: write d7/d9/da/d4c/f7e [2399715,53486] 0 2024-09-21T04:35:15.105 INFO:tasks.workunit.client.0.smithi007.stdout:1/248: unlink d5/d2f/f34 0 2024-09-21T04:35:15.106 INFO:tasks.workunit.client.0.smithi007.stdout:0/133: mknod d6/c30 0 2024-09-21T04:35:15.109 INFO:tasks.workunit.client.1.smithi053.stdout:2/761: getdents d1/d18 0 2024-09-21T04:35:15.111 INFO:tasks.workunit.client.1.smithi053.stdout:1/742: mkdir d7/d9/da/d4c/df4/df6 0 2024-09-21T04:35:15.112 INFO:tasks.workunit.client.1.smithi053.stdout:0/693: creat d5/d10/d4c/db1/fe9 x:0 0 0 2024-09-21T04:35:15.112 INFO:tasks.workunit.client.1.smithi053.stdout:0/694: fdatasync d5/de/fd7 0 2024-09-21T04:35:15.123 INFO:tasks.workunit.client.1.smithi053.stdout:1/743: dread d7/d9/d43/d54/d7f/d36/fd3 [0,4194304] 0 2024-09-21T04:35:15.128 INFO:tasks.workunit.client.1.smithi053.stdout:1/744: dwrite d7/f56 [0,4194304] 0 2024-09-21T04:35:15.129 INFO:tasks.workunit.client.1.smithi053.stdout:4/790: sync 2024-09-21T04:35:15.129 INFO:tasks.workunit.client.1.smithi053.stdout:5/760: sync 2024-09-21T04:35:15.129 INFO:tasks.workunit.client.1.smithi053.stdout:8/755: sync 2024-09-21T04:35:15.133 INFO:tasks.workunit.client.1.smithi053.stdout:4/791: creat d1/dd/de/df5/d3f/d74/f102 x:0 0 0 2024-09-21T04:35:15.135 INFO:tasks.workunit.client.1.smithi053.stdout:8/756: creat d3/df/d14/ffa x:0 0 0 2024-09-21T04:35:15.136 INFO:tasks.workunit.client.1.smithi053.stdout:4/792: symlink d1/d27/df8/l103 0 2024-09-21T04:35:15.137 INFO:tasks.workunit.client.1.smithi053.stdout:4/793: chown d1/d27/d2e/fc1 39 1 2024-09-21T04:35:15.139 INFO:tasks.workunit.client.0.smithi007.stdout:9/135: sync 2024-09-21T04:35:15.143 INFO:tasks.workunit.client.0.smithi007.stdout:9/136: mknod d13/d2a/c2b 0 2024-09-21T04:35:15.144 INFO:tasks.workunit.client.0.smithi007.stdout:9/137: rename d13/d2a to d13/d2a/d2c 22 2024-09-21T04:35:15.147 INFO:tasks.workunit.client.0.smithi007.stdout:9/138: creat d13/d2a/f2d x:0 0 0 2024-09-21T04:35:15.148 INFO:tasks.workunit.client.0.smithi007.stdout:5/347: write d2/fc [1190486,91087] 0 2024-09-21T04:35:15.148 INFO:tasks.workunit.client.0.smithi007.stdout:9/139: write d13/f20 [4086648,40294] 0 2024-09-21T04:35:15.150 INFO:tasks.workunit.client.1.smithi053.stdout:8/757: link d3/d39/f3f d3/d5/d38/d64/d73/d91/ffb 0 2024-09-21T04:35:15.151 INFO:tasks.workunit.client.1.smithi053.stdout:9/696: write d0/d1c/d21/d46/fd7 [1226377,93514] 0 2024-09-21T04:35:15.152 INFO:tasks.workunit.client.1.smithi053.stdout:1/745: dread d7/d9/da/d40/d49/f5e [0,4194304] 0 2024-09-21T04:35:15.153 INFO:tasks.workunit.client.1.smithi053.stdout:9/697: truncate d0/d28/f6b 4742542 0 2024-09-21T04:35:15.153 INFO:tasks.workunit.client.1.smithi053.stdout:6/706: truncate d2/fd 3650646 0 2024-09-21T04:35:15.155 INFO:tasks.workunit.client.1.smithi053.stdout:8/758: stat d3/d5/cf4 0 2024-09-21T04:35:15.157 INFO:tasks.workunit.client.0.smithi007.stdout:1/249: dread d5/d6/d21/d23/f2a [0,4194304] 0 2024-09-21T04:35:15.159 INFO:tasks.workunit.client.0.smithi007.stdout:9/140: rename l3 to d13/d18/l2e 0 2024-09-21T04:35:15.162 INFO:tasks.workunit.client.1.smithi053.stdout:9/698: creat d0/d1a/d4c/d8f/ff1 x:0 0 0 2024-09-21T04:35:15.165 INFO:tasks.workunit.client.0.smithi007.stdout:9/141: dwrite f7 [0,4194304] 0 2024-09-21T04:35:15.166 INFO:tasks.workunit.client.0.smithi007.stdout:9/142: write f10 [122142,123494] 0 2024-09-21T04:35:15.169 INFO:tasks.workunit.client.1.smithi053.stdout:7/630: dwrite d0/d6f/dbf/d50/f96 [0,4194304] 0 2024-09-21T04:35:15.170 INFO:tasks.workunit.client.1.smithi053.stdout:6/707: mkdir d2/d1b/d3d/d49/de/d32/d80/d2e/de7 0 2024-09-21T04:35:15.171 INFO:tasks.workunit.client.1.smithi053.stdout:7/631: stat d0/f59 0 2024-09-21T04:35:15.171 INFO:tasks.workunit.client.1.smithi053.stdout:9/699: dwrite d0/d1c/db8/feb [0,4194304] 0 2024-09-21T04:35:15.172 INFO:tasks.workunit.client.0.smithi007.stdout:5/348: creat d2/f70 x:0 0 0 2024-09-21T04:35:15.173 INFO:tasks.workunit.client.0.smithi007.stdout:5/349: chown d2/dd/d23/l34 3375 1 2024-09-21T04:35:15.173 INFO:tasks.workunit.client.1.smithi053.stdout:2/762: truncate d1/d18/f2a 1874516 0 2024-09-21T04:35:15.176 INFO:tasks.workunit.client.1.smithi053.stdout:0/695: write d5/de/d42/d78/f6f [5450983,87305] 0 2024-09-21T04:35:15.177 INFO:tasks.workunit.client.1.smithi053.stdout:1/746: dread d7/d9/da/d11/f33 [0,4194304] 0 2024-09-21T04:35:15.184 INFO:tasks.workunit.client.1.smithi053.stdout:0/696: dwrite d5/de/d9e/faf [0,4194304] 0 2024-09-21T04:35:15.186 INFO:tasks.workunit.client.1.smithi053.stdout:4/794: write d1/d27/d2e/d46/f54 [2386045,20650] 0 2024-09-21T04:35:15.186 INFO:tasks.workunit.client.1.smithi053.stdout:9/700: symlink d0/d2/ddf/lf2 0 2024-09-21T04:35:15.187 INFO:tasks.workunit.client.1.smithi053.stdout:7/632: creat d0/d6f/dbf/d5c/dac/fce x:0 0 0 2024-09-21T04:35:15.189 INFO:tasks.workunit.client.0.smithi007.stdout:0/134: mknod d6/d16/d18/d2e/c31 0 2024-09-21T04:35:15.190 INFO:tasks.workunit.client.1.smithi053.stdout:9/701: chown d0/d2/d9/f43 1 1 2024-09-21T04:35:15.190 INFO:tasks.workunit.client.0.smithi007.stdout:0/135: dread - d6/d16/d18/f2b zero size 2024-09-21T04:35:15.192 INFO:tasks.workunit.client.1.smithi053.stdout:2/763: creat d1/d18/d1f/d27/d58/d5c/f113 x:0 0 0 2024-09-21T04:35:15.193 INFO:tasks.workunit.client.0.smithi007.stdout:1/250: rename d5/f3e to d5/d6/f4a 0 2024-09-21T04:35:15.197 INFO:tasks.workunit.client.0.smithi007.stdout:0/136: symlink d6/l32 0 2024-09-21T04:35:15.197 INFO:tasks.workunit.client.1.smithi053.stdout:6/708: dwrite d2/d1b/d1e/d2a/d4b/d77/d88/da4/fcc [0,4194304] 0 2024-09-21T04:35:15.200 INFO:tasks.workunit.client.1.smithi053.stdout:3/721: sync 2024-09-21T04:35:15.202 INFO:tasks.workunit.client.1.smithi053.stdout:3/722: write d16/d17/d21/d26/f102 [564808,117612] 0 2024-09-21T04:35:15.203 INFO:tasks.workunit.client.0.smithi007.stdout:5/350: dread d2/dd/d23/d2a/f2d [0,4194304] 0 2024-09-21T04:35:15.204 INFO:tasks.workunit.client.0.smithi007.stdout:5/351: dread - d2/d1e/f37 zero size 2024-09-21T04:35:15.205 INFO:tasks.workunit.client.0.smithi007.stdout:5/352: write d2/f1b [3206551,66636] 0 2024-09-21T04:35:15.205 INFO:tasks.workunit.client.0.smithi007.stdout:1/251: mkdir d5/d6/d21/d2b/d4b 0 2024-09-21T04:35:15.206 INFO:tasks.workunit.client.0.smithi007.stdout:1/252: readlink d5/l1e 0 2024-09-21T04:35:15.212 INFO:tasks.workunit.client.1.smithi053.stdout:2/764: chown d1/c15 363462639 1 2024-09-21T04:35:15.212 INFO:tasks.workunit.client.0.smithi007.stdout:9/143: readlink d13/d18/l2e 0 2024-09-21T04:35:15.213 INFO:tasks.workunit.client.0.smithi007.stdout:0/137: creat d6/d16/d18/d27/f33 x:0 0 0 2024-09-21T04:35:15.217 INFO:tasks.workunit.client.0.smithi007.stdout:9/144: dread f7 [0,4194304] 0 2024-09-21T04:35:15.218 INFO:tasks.workunit.client.0.smithi007.stdout:1/253: creat d5/d6/d21/d23/d42/f4c x:0 0 0 2024-09-21T04:35:15.218 INFO:tasks.workunit.client.0.smithi007.stdout:9/145: chown d13/d18/l19 20580 1 2024-09-21T04:35:15.220 INFO:tasks.workunit.client.1.smithi053.stdout:6/709: rename d2/ldc to d2/d1b/d3d/d49/de/d9c/le8 0 2024-09-21T04:35:15.220 INFO:tasks.workunit.client.1.smithi053.stdout:8/759: write d3/d5/db1/fd1 [1938040,39522] 0 2024-09-21T04:35:15.221 INFO:tasks.workunit.client.1.smithi053.stdout:4/795: unlink d1/dd/de/df5/d20/d4e/d51/da4/df1/d8e/le9 0 2024-09-21T04:35:15.222 INFO:tasks.workunit.client.1.smithi053.stdout:8/760: readlink d3/d5/d4c/d53/d66/l69 0 2024-09-21T04:35:15.226 INFO:tasks.workunit.client.0.smithi007.stdout:5/353: dread d2/f66 [0,4194304] 0 2024-09-21T04:35:15.227 INFO:tasks.workunit.client.0.smithi007.stdout:5/354: chown d2/dd/d23/d2a/c3c 201 1 2024-09-21T04:35:15.227 INFO:tasks.workunit.client.0.smithi007.stdout:5/355: chown d2/dd/d23/l26 3950447 1 2024-09-21T04:35:15.228 INFO:tasks.workunit.client.1.smithi053.stdout:6/710: dwrite d2/d1b/d3d/d49/de/fd8 [0,4194304] 0 2024-09-21T04:35:15.239 INFO:tasks.workunit.client.1.smithi053.stdout:3/723: symlink d16/d17/d21/l10e 0 2024-09-21T04:35:15.241 INFO:tasks.workunit.client.1.smithi053.stdout:7/633: symlink d0/d6f/dbf/d50/d84/db8/lcf 0 2024-09-21T04:35:15.241 INFO:tasks.workunit.client.0.smithi007.stdout:9/146: rmdir d13/d2a 39 2024-09-21T04:35:15.242 INFO:tasks.workunit.client.0.smithi007.stdout:9/147: fdatasync f6 0 2024-09-21T04:35:15.246 INFO:tasks.workunit.client.1.smithi053.stdout:9/702: getdents d0/d2/d9/d34/db2 0 2024-09-21T04:35:15.246 INFO:tasks.workunit.client.0.smithi007.stdout:5/356: creat d2/dd/d1d/f71 x:0 0 0 2024-09-21T04:35:15.247 INFO:tasks.workunit.client.1.smithi053.stdout:8/761: dread d3/df/d10/f48 [0,4194304] 0 2024-09-21T04:35:15.247 INFO:tasks.workunit.client.0.smithi007.stdout:5/357: chown d2/d1e/f37 15 1 2024-09-21T04:35:15.247 INFO:tasks.workunit.client.1.smithi053.stdout:7/634: dwrite d0/d2b/f2e [0,4194304] 0 2024-09-21T04:35:15.248 INFO:tasks.workunit.client.1.smithi053.stdout:9/703: readlink d0/d28/l49 0 2024-09-21T04:35:15.249 INFO:tasks.workunit.client.1.smithi053.stdout:4/796: unlink d1/dd/l10 0 2024-09-21T04:35:15.254 INFO:tasks.workunit.client.0.smithi007.stdout:5/358: dwrite d2/dd/d23/d2a/f2d [0,4194304] 0 2024-09-21T04:35:15.254 INFO:tasks.workunit.client.1.smithi053.stdout:6/711: truncate d2/d1b/d1e/fcb 2743299 0 2024-09-21T04:35:15.255 INFO:tasks.workunit.client.1.smithi053.stdout:9/704: dwrite d0/d1a/f2c [8388608,4194304] 0 2024-09-21T04:35:15.256 INFO:tasks.workunit.client.0.smithi007.stdout:5/359: truncate d2/dd/d1d/d20/f6d 606527 0 2024-09-21T04:35:15.262 INFO:tasks.workunit.client.0.smithi007.stdout:0/138: getdents d6 0 2024-09-21T04:35:15.264 INFO:tasks.workunit.client.1.smithi053.stdout:8/762: mknod d3/d5/d4c/d8e/cfc 0 2024-09-21T04:35:15.266 INFO:tasks.workunit.client.1.smithi053.stdout:7/635: rename d0/d6f/dbf/d5d/l92 to d0/d2b/d36/d64/dc0/ld0 0 2024-09-21T04:35:15.269 INFO:tasks.workunit.client.1.smithi053.stdout:7/636: dread - d0/d2b/d36/d64/dc0/fcb zero size 2024-09-21T04:35:15.269 INFO:tasks.workunit.client.1.smithi053.stdout:4/797: mknod d1/dd/de/df5/d3f/d74/de1/c104 0 2024-09-21T04:35:15.269 INFO:tasks.workunit.client.1.smithi053.stdout:6/712: rmdir d2/d1b/d3d/d49/de/d96 39 2024-09-21T04:35:15.270 INFO:tasks.workunit.client.1.smithi053.stdout:0/697: write d5/dd/d50/f6e [444002,3509] 0 2024-09-21T04:35:15.271 INFO:tasks.workunit.client.0.smithi007.stdout:9/148: getdents d13/d18 0 2024-09-21T04:35:15.271 INFO:tasks.workunit.client.0.smithi007.stdout:9/149: chown d13/c29 4834103 1 2024-09-21T04:35:15.274 INFO:tasks.workunit.client.1.smithi053.stdout:6/713: fdatasync d2/d1b/d3d/d49/de/d32/fd2 0 2024-09-21T04:35:15.275 INFO:tasks.workunit.client.1.smithi053.stdout:8/763: dwrite d3/d39/f97 [0,4194304] 0 2024-09-21T04:35:15.276 INFO:tasks.workunit.client.0.smithi007.stdout:9/150: creat d13/d18/f2f x:0 0 0 2024-09-21T04:35:15.278 INFO:tasks.workunit.client.1.smithi053.stdout:8/764: write d3/d5/d38/d64/ff8 [261221,58998] 0 2024-09-21T04:35:15.279 INFO:tasks.workunit.client.1.smithi053.stdout:4/798: dwrite d1/dd/de/df5/d20/f75 [0,4194304] 0 2024-09-21T04:35:15.280 INFO:tasks.workunit.client.0.smithi007.stdout:0/139: rename d6/c8 to d6/d16/c34 0 2024-09-21T04:35:15.285 INFO:tasks.workunit.client.1.smithi053.stdout:5/761: sync 2024-09-21T04:35:15.285 INFO:tasks.workunit.client.0.smithi007.stdout:0/140: chown d6/d16/d18/d2e/c31 20002 1 2024-09-21T04:35:15.286 INFO:tasks.workunit.client.1.smithi053.stdout:5/762: fsync d0/dd/d5c/d89/fbd 0 2024-09-21T04:35:15.287 INFO:tasks.workunit.client.1.smithi053.stdout:3/724: creat d16/f10f x:0 0 0 2024-09-21T04:35:15.289 INFO:tasks.workunit.client.0.smithi007.stdout:5/360: rename d2/d27/d3d/d56/f68 to d2/d27/d42/f72 0 2024-09-21T04:35:15.290 INFO:tasks.workunit.client.1.smithi053.stdout:3/725: truncate d16/d17/d21/d8b/d7d/d57/f107 495508 0 2024-09-21T04:35:15.290 INFO:tasks.workunit.client.0.smithi007.stdout:5/361: truncate d2/dd/f3e 601558 0 2024-09-21T04:35:15.292 INFO:tasks.workunit.client.1.smithi053.stdout:0/698: creat d5/d10/d4c/db1/dbc/dca/d21/d96/fea x:0 0 0 2024-09-21T04:35:15.293 INFO:tasks.workunit.client.0.smithi007.stdout:9/151: mknod d13/d2a/c30 0 2024-09-21T04:35:15.294 INFO:tasks.workunit.client.1.smithi053.stdout:2/765: getdents d1/d18/d1f/da2 0 2024-09-21T04:35:15.294 INFO:tasks.workunit.client.1.smithi053.stdout:9/705: dread d0/f15 [0,4194304] 0 2024-09-21T04:35:15.295 INFO:tasks.workunit.client.0.smithi007.stdout:0/141: stat d6/d16/l26 0 2024-09-21T04:35:15.299 INFO:tasks.workunit.client.0.smithi007.stdout:5/362: rmdir d2/d1e 39 2024-09-21T04:35:15.302 INFO:tasks.workunit.client.1.smithi053.stdout:8/765: write d3/d5/d38/d64/d9d/da9/fe3 [4456624,116825] 0 2024-09-21T04:35:15.303 INFO:tasks.workunit.client.0.smithi007.stdout:0/142: fdatasync d6/fd 0 2024-09-21T04:35:15.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/363: dread d2/dd/f16 [4194304,4194304] 0 2024-09-21T04:35:15.309 INFO:tasks.workunit.client.1.smithi053.stdout:6/714: write d2/d1b/d3d/d49/d5c/f74 [922175,124093] 0 2024-09-21T04:35:15.310 INFO:tasks.workunit.client.0.smithi007.stdout:5/364: read d2/d1e/f24 [2642521,78785] 0 2024-09-21T04:35:15.312 INFO:tasks.workunit.client.1.smithi053.stdout:2/766: creat d1/d18/d1f/d27/d41/f114 x:0 0 0 2024-09-21T04:35:15.314 INFO:tasks.workunit.client.0.smithi007.stdout:9/152: getdents d13 0 2024-09-21T04:35:15.319 INFO:tasks.workunit.client.0.smithi007.stdout:5/365: dwrite d2/f5a [0,4194304] 0 2024-09-21T04:35:15.319 INFO:tasks.workunit.client.0.smithi007.stdout:0/143: dread d6/f13 [0,4194304] 0 2024-09-21T04:35:15.322 INFO:tasks.workunit.client.0.smithi007.stdout:5/366: truncate d2/d27/d42/f62 124340 0 2024-09-21T04:35:15.324 INFO:tasks.workunit.client.0.smithi007.stdout:0/144: dread d6/f12 [4194304,4194304] 0 2024-09-21T04:35:15.326 INFO:tasks.workunit.client.0.smithi007.stdout:5/367: creat d2/d27/d42/d53/f73 x:0 0 0 2024-09-21T04:35:15.327 INFO:tasks.workunit.client.0.smithi007.stdout:0/145: write d6/d16/f25 [509719,97328] 0 2024-09-21T04:35:15.327 INFO:tasks.workunit.client.0.smithi007.stdout:5/368: dread - d2/dd/d28/f64 zero size 2024-09-21T04:35:15.328 INFO:tasks.workunit.client.0.smithi007.stdout:0/146: chown d6/d16/d18/f2b 5271 1 2024-09-21T04:35:15.330 INFO:tasks.workunit.client.0.smithi007.stdout:9/153: dread f10 [0,4194304] 0 2024-09-21T04:35:15.332 INFO:tasks.workunit.client.0.smithi007.stdout:9/154: write d13/f21 [4441213,6794] 0 2024-09-21T04:35:15.334 INFO:tasks.workunit.client.0.smithi007.stdout:9/155: readlink d13/d18/l22 0 2024-09-21T04:35:15.336 INFO:tasks.workunit.client.0.smithi007.stdout:0/147: dwrite d6/fd [0,4194304] 0 2024-09-21T04:35:15.339 INFO:tasks.workunit.client.0.smithi007.stdout:0/148: creat d6/d16/d18/d27/f35 x:0 0 0 2024-09-21T04:35:15.342 INFO:tasks.workunit.client.0.smithi007.stdout:0/149: read - d6/d16/d18/d27/f29 zero size 2024-09-21T04:35:15.345 INFO:tasks.workunit.client.0.smithi007.stdout:5/369: dwrite d2/dd/d1d/f1f [0,4194304] 0 2024-09-21T04:35:15.347 INFO:tasks.workunit.client.0.smithi007.stdout:0/150: symlink d6/d16/d18/d27/l36 0 2024-09-21T04:35:15.348 INFO:tasks.workunit.client.0.smithi007.stdout:9/156: dwrite f7 [0,4194304] 0 2024-09-21T04:35:15.351 INFO:tasks.workunit.client.0.smithi007.stdout:5/370: unlink d2/dd/d23/d49/l41 0 2024-09-21T04:35:15.351 INFO:tasks.workunit.client.0.smithi007.stdout:9/157: stat d13/d2a/c30 0 2024-09-21T04:35:15.351 INFO:tasks.workunit.client.0.smithi007.stdout:5/371: stat d2/f50 0 2024-09-21T04:35:15.351 INFO:tasks.workunit.client.0.smithi007.stdout:0/151: write d6/d16/d18/d27/f33 [838349,42539] 0 2024-09-21T04:35:15.353 INFO:tasks.workunit.client.0.smithi007.stdout:9/158: creat d13/d2a/f31 x:0 0 0 2024-09-21T04:35:15.353 INFO:tasks.workunit.client.0.smithi007.stdout:5/372: fsync d2/d1e/f24 0 2024-09-21T04:35:15.356 INFO:tasks.workunit.client.0.smithi007.stdout:9/159: creat d13/d2a/f32 x:0 0 0 2024-09-21T04:35:15.358 INFO:tasks.workunit.client.0.smithi007.stdout:5/373: unlink d2/d27/d3d/f4e 0 2024-09-21T04:35:15.359 INFO:tasks.workunit.client.0.smithi007.stdout:5/374: chown d2/dd/f16 1 1 2024-09-21T04:35:15.359 INFO:tasks.workunit.client.0.smithi007.stdout:5/375: dread - d2/dd/d1d/f71 zero size 2024-09-21T04:35:15.359 INFO:tasks.workunit.client.0.smithi007.stdout:0/152: dread d6/d16/d18/d27/f33 [0,4194304] 0 2024-09-21T04:35:15.361 INFO:tasks.workunit.client.0.smithi007.stdout:9/160: rename d13/d18/l19 to d13/d2a/l33 0 2024-09-21T04:35:15.361 INFO:tasks.workunit.client.0.smithi007.stdout:0/153: rmdir d6 39 2024-09-21T04:35:15.361 INFO:tasks.workunit.client.0.smithi007.stdout:9/161: fdatasync d13/d18/f2f 0 2024-09-21T04:35:15.362 INFO:tasks.workunit.client.0.smithi007.stdout:9/162: fsync d13/d18/f26 0 2024-09-21T04:35:15.363 INFO:tasks.workunit.client.0.smithi007.stdout:5/376: creat d2/dd/d23/d5d/d61/f74 x:0 0 0 2024-09-21T04:35:15.365 INFO:tasks.workunit.client.0.smithi007.stdout:9/163: creat d13/d18/f34 x:0 0 0 2024-09-21T04:35:15.366 INFO:tasks.workunit.client.0.smithi007.stdout:0/154: creat d6/d16/d18/d2e/f37 x:0 0 0 2024-09-21T04:35:15.369 INFO:tasks.workunit.client.0.smithi007.stdout:5/377: dread d2/d1e/f24 [8388608,4194304] 0 2024-09-21T04:35:15.373 INFO:tasks.workunit.client.0.smithi007.stdout:9/164: dwrite d13/d18/f1f [0,4194304] 0 2024-09-21T04:35:15.373 INFO:tasks.workunit.client.0.smithi007.stdout:5/378: creat d2/dd/d23/d2a/f75 x:0 0 0 2024-09-21T04:35:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:0/155: dread f1 [0,4194304] 0 2024-09-21T04:35:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:9/165: creat d13/d18/f35 x:0 0 0 2024-09-21T04:35:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:0/156: creat d6/f38 x:0 0 0 2024-09-21T04:35:15.378 INFO:tasks.workunit.client.0.smithi007.stdout:5/379: mknod d2/dd/d23/d49/d44/d54/c76 0 2024-09-21T04:35:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:5/380: fsync d2/d1e/f5b 0 2024-09-21T04:35:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:5/381: fsync d2/f50 0 2024-09-21T04:35:15.380 INFO:tasks.workunit.client.0.smithi007.stdout:9/166: creat d13/d18/f36 x:0 0 0 2024-09-21T04:35:15.384 INFO:tasks.workunit.client.1.smithi053.stdout:5/763: mkdir d0/d8/db6/dbc/df4 0 2024-09-21T04:35:15.384 INFO:tasks.workunit.client.1.smithi053.stdout:3/726: mkdir d16/d17/d21/d8b/d110 0 2024-09-21T04:35:15.384 INFO:tasks.workunit.client.1.smithi053.stdout:8/766: write d3/d39/f40 [5024895,19794] 0 2024-09-21T04:35:15.386 INFO:tasks.workunit.client.1.smithi053.stdout:6/715: write d2/d1d/de0/fa6 [1340287,126784] 0 2024-09-21T04:35:15.387 INFO:tasks.workunit.client.0.smithi007.stdout:5/382: dread d2/dd/d28/f4d [0,4194304] 0 2024-09-21T04:35:15.388 INFO:tasks.workunit.client.1.smithi053.stdout:2/767: mkdir d1/d18/d1f/d27/d58/db7/d112/d115 0 2024-09-21T04:35:15.389 INFO:tasks.workunit.client.1.smithi053.stdout:3/727: creat d16/d1b/f111 x:0 0 0 2024-09-21T04:35:15.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/167: dwrite d13/d18/f35 [0,4194304] 0 2024-09-21T04:35:15.390 INFO:tasks.workunit.client.1.smithi053.stdout:3/728: write d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fbf [175866,48046] 0 2024-09-21T04:35:15.391 INFO:tasks.workunit.client.1.smithi053.stdout:5/764: mkdir d0/d8/d2a/df5 0 2024-09-21T04:35:15.391 INFO:tasks.workunit.client.1.smithi053.stdout:0/699: getdents d5/d10/d4c/db1/dbc/dca/d21/d95 0 2024-09-21T04:35:15.391 INFO:tasks.workunit.client.0.smithi007.stdout:5/383: truncate d2/dd/d23/d49/f4a 130376 0 2024-09-21T04:35:15.394 INFO:tasks.workunit.client.0.smithi007.stdout:9/168: symlink d13/l37 0 2024-09-21T04:35:15.395 INFO:tasks.workunit.client.1.smithi053.stdout:6/716: fsync d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f8a 0 2024-09-21T04:35:15.395 INFO:tasks.workunit.client.1.smithi053.stdout:3/729: truncate d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fbf 610977 0 2024-09-21T04:35:15.395 INFO:tasks.workunit.client.0.smithi007.stdout:5/384: write d2/dd/d23/d49/f4a [1100033,108454] 0 2024-09-21T04:35:15.396 INFO:tasks.workunit.client.1.smithi053.stdout:6/717: write d2/d1b/d3d/d49/de/d32/d80/f4e [2781043,8531] 0 2024-09-21T04:35:15.398 INFO:tasks.workunit.client.0.smithi007.stdout:9/169: unlink f11 0 2024-09-21T04:35:15.399 INFO:tasks.workunit.client.1.smithi053.stdout:3/730: dread d16/d17/d21/d26/d3e/f99 [0,4194304] 0 2024-09-21T04:35:15.400 INFO:tasks.workunit.client.0.smithi007.stdout:5/385: mknod d2/dd/d23/d49/d44/c77 0 2024-09-21T04:35:15.400 INFO:tasks.workunit.client.0.smithi007.stdout:0/157: write d6/d16/f28 [1343689,113697] 0 2024-09-21T04:35:15.401 INFO:tasks.workunit.client.0.smithi007.stdout:5/386: readlink d2/dd/d23/l34 0 2024-09-21T04:35:15.401 INFO:tasks.workunit.client.1.smithi053.stdout:8/767: symlink d3/d5/d38/lfd 0 2024-09-21T04:35:15.402 INFO:tasks.workunit.client.0.smithi007.stdout:5/387: read - d2/d27/d42/d53/f73 zero size 2024-09-21T04:35:15.403 INFO:tasks.workunit.client.1.smithi053.stdout:5/765: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:35:15.404 INFO:tasks.workunit.client.0.smithi007.stdout:0/158: mkdir d6/d16/d18/d39 0 2024-09-21T04:35:15.405 INFO:tasks.workunit.client.0.smithi007.stdout:3/119: sync 2024-09-21T04:35:15.405 INFO:tasks.workunit.client.0.smithi007.stdout:3/120: stat dc/f1c 0 2024-09-21T04:35:15.406 INFO:tasks.workunit.client.1.smithi053.stdout:0/700: creat d5/d10/d4c/db1/dbc/dca/d21/d96/feb x:0 0 0 2024-09-21T04:35:15.406 INFO:tasks.workunit.client.0.smithi007.stdout:5/388: symlink d2/d1e/l78 0 2024-09-21T04:35:15.408 INFO:tasks.workunit.client.1.smithi053.stdout:6/718: dwrite d2/d1b/f53 [0,4194304] 0 2024-09-21T04:35:15.408 INFO:tasks.workunit.client.1.smithi053.stdout:2/768: dwrite d1/d18/d1f/d22/d54/fb0 [0,4194304] 0 2024-09-21T04:35:15.408 INFO:tasks.workunit.client.0.smithi007.stdout:3/121: read fb [1527534,96260] 0 2024-09-21T04:35:15.413 INFO:tasks.workunit.client.1.smithi053.stdout:3/731: creat d16/d17/ddc/f112 x:0 0 0 2024-09-21T04:35:15.414 INFO:tasks.workunit.client.0.smithi007.stdout:0/159: dwrite d6/d16/d18/d27/f33 [0,4194304] 0 2024-09-21T04:35:15.417 INFO:tasks.workunit.client.1.smithi053.stdout:4/799: link d1/d27/f7c d1/dd/de/df5/f105 0 2024-09-21T04:35:15.427 INFO:tasks.workunit.client.0.smithi007.stdout:5/389: truncate d2/dd/d1d/f4b 1073601 0 2024-09-21T04:35:15.428 INFO:tasks.workunit.client.1.smithi053.stdout:4/800: mkdir d1/d27/d2e/d106 0 2024-09-21T04:35:15.429 INFO:tasks.workunit.client.1.smithi053.stdout:4/801: dread - d1/dd/de/df5/d20/dee/ff6 zero size 2024-09-21T04:35:15.430 INFO:tasks.workunit.client.1.smithi053.stdout:4/802: readlink d1/dd/de/df5/d20/d4e/d51/da4/df1/l6a 0 2024-09-21T04:35:15.432 INFO:tasks.workunit.client.0.smithi007.stdout:5/390: unlink d2/dd/d28/f4c 0 2024-09-21T04:35:15.433 INFO:tasks.workunit.client.1.smithi053.stdout:3/732: truncate d16/d17/d3c/f7e 1610201 0 2024-09-21T04:35:15.434 INFO:tasks.workunit.client.0.smithi007.stdout:5/391: write d2/dd/d23/d2a/f2d [4594191,100042] 0 2024-09-21T04:35:15.435 INFO:tasks.workunit.client.1.smithi053.stdout:3/733: write d16/d17/ddc/f112 [743503,53527] 0 2024-09-21T04:35:15.437 INFO:tasks.workunit.client.1.smithi053.stdout:3/734: stat d16/d17/d21/d8b/d7d/d57/dc1/d53 0 2024-09-21T04:35:15.438 INFO:tasks.workunit.client.1.smithi053.stdout:3/735: fsync d16/d1b/f111 0 2024-09-21T04:35:15.438 INFO:tasks.workunit.client.1.smithi053.stdout:2/769: dwrite d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/f10c [0,4194304] 0 2024-09-21T04:35:15.439 INFO:tasks.workunit.client.1.smithi053.stdout:0/701: write d5/d10/d4c/db1/dbc/dca/d21/fc4 [211784,53830] 0 2024-09-21T04:35:15.440 INFO:tasks.workunit.client.1.smithi053.stdout:3/736: write d16/d1b/dc3/fb2 [2453331,96300] 0 2024-09-21T04:35:15.441 INFO:tasks.workunit.client.0.smithi007.stdout:9/170: mknod d13/c38 0 2024-09-21T04:35:15.441 INFO:tasks.workunit.client.1.smithi053.stdout:0/702: chown d5/dd/d45 467 1 2024-09-21T04:35:15.442 INFO:tasks.workunit.client.0.smithi007.stdout:3/122: getdents dc/d18 0 2024-09-21T04:35:15.444 INFO:tasks.workunit.client.1.smithi053.stdout:8/768: truncate d3/f4a 3144272 0 2024-09-21T04:35:15.445 INFO:tasks.workunit.client.0.smithi007.stdout:5/392: symlink d2/d27/d42/l79 0 2024-09-21T04:35:15.449 INFO:tasks.workunit.client.0.smithi007.stdout:0/160: getdents d6/d16/d18 0 2024-09-21T04:35:15.449 INFO:tasks.workunit.client.1.smithi053.stdout:4/803: write d1/dd/de/df5/d3f/f50 [1964081,52661] 0 2024-09-21T04:35:15.450 INFO:tasks.workunit.client.0.smithi007.stdout:5/393: dread d2/dd/d1d/f1f [0,4194304] 0 2024-09-21T04:35:15.450 INFO:tasks.workunit.client.1.smithi053.stdout:4/804: write d1/d27/d2e/d46/f54 [2727251,7647] 0 2024-09-21T04:35:15.451 INFO:tasks.workunit.client.0.smithi007.stdout:5/394: write d2/dd/d28/f64 [871262,58506] 0 2024-09-21T04:35:15.453 INFO:tasks.workunit.client.1.smithi053.stdout:8/769: dread d3/df/d49/f61 [0,4194304] 0 2024-09-21T04:35:15.453 INFO:tasks.workunit.client.0.smithi007.stdout:3/123: mkdir dc/d22 0 2024-09-21T04:35:15.454 INFO:tasks.workunit.client.1.smithi053.stdout:8/770: fsync d3/d28/fd5 0 2024-09-21T04:35:15.454 INFO:tasks.workunit.client.1.smithi053.stdout:2/770: creat d1/d18/d1f/d22/d2d/d106/f116 x:0 0 0 2024-09-21T04:35:15.457 INFO:tasks.workunit.client.1.smithi053.stdout:8/771: write d3/df/d10/fd3 [372858,84036] 0 2024-09-21T04:35:15.457 INFO:tasks.workunit.client.0.smithi007.stdout:3/124: dread dc/f13 [0,4194304] 0 2024-09-21T04:35:15.457 INFO:tasks.workunit.client.1.smithi053.stdout:4/805: dwrite d1/d27/d33/f3c [0,4194304] 0 2024-09-21T04:35:15.461 INFO:tasks.workunit.client.0.smithi007.stdout:0/161: creat d6/d16/d18/d27/f3a x:0 0 0 2024-09-21T04:35:15.463 INFO:tasks.workunit.client.0.smithi007.stdout:9/171: mknod d13/c39 0 2024-09-21T04:35:15.465 INFO:tasks.workunit.client.0.smithi007.stdout:3/125: dwrite dc/f14 [0,4194304] 0 2024-09-21T04:35:15.466 INFO:tasks.workunit.client.0.smithi007.stdout:5/395: creat d2/dd/d23/d2a/f7a x:0 0 0 2024-09-21T04:35:15.468 INFO:tasks.workunit.client.0.smithi007.stdout:0/162: dread f1 [0,4194304] 0 2024-09-21T04:35:15.468 INFO:tasks.workunit.client.0.smithi007.stdout:3/126: truncate dc/d18/f1d 694644 0 2024-09-21T04:35:15.469 INFO:tasks.workunit.client.0.smithi007.stdout:3/127: chown dc/f13 6482 1 2024-09-21T04:35:15.470 INFO:tasks.workunit.client.0.smithi007.stdout:3/128: write dc/f15 [337225,71330] 0 2024-09-21T04:35:15.472 INFO:tasks.workunit.client.0.smithi007.stdout:9/172: fsync d13/d2a/f32 0 2024-09-21T04:35:15.473 INFO:tasks.workunit.client.0.smithi007.stdout:3/129: dread dc/d18/f1d [0,4194304] 0 2024-09-21T04:35:15.473 INFO:tasks.workunit.client.1.smithi053.stdout:3/737: dwrite d16/d1b/dc3/f6c [0,4194304] 0 2024-09-21T04:35:15.475 INFO:tasks.workunit.client.1.smithi053.stdout:2/771: unlink d1/d18/d1f/d27/d41/d4b/c102 0 2024-09-21T04:35:15.475 INFO:tasks.workunit.client.1.smithi053.stdout:8/772: dread d3/d39/f27 [0,4194304] 0 2024-09-21T04:35:15.476 INFO:tasks.workunit.client.1.smithi053.stdout:3/738: fdatasync d16/d17/d21/d26/d2b/fa2 0 2024-09-21T04:35:15.481 INFO:tasks.workunit.client.0.smithi007.stdout:9/173: dwrite d13/d2a/f32 [0,4194304] 0 2024-09-21T04:35:15.482 INFO:tasks.workunit.client.0.smithi007.stdout:0/163: rename d6/d16/d18/f2b to d6/d16/d18/d39/f3b 0 2024-09-21T04:35:15.485 INFO:tasks.workunit.client.1.smithi053.stdout:4/806: getdents d1/ddb 0 2024-09-21T04:35:15.487 INFO:tasks.workunit.client.0.smithi007.stdout:3/130: readlink dc/l20 0 2024-09-21T04:35:15.490 INFO:tasks.workunit.client.1.smithi053.stdout:3/739: creat d16/d17/d21/d26/d2b/f113 x:0 0 0 2024-09-21T04:35:15.496 INFO:tasks.workunit.client.1.smithi053.stdout:2/772: mkdir d1/d18/d1f/d27/d58/d5c/d91/dd0/d117 0 2024-09-21T04:35:15.497 INFO:tasks.workunit.client.0.smithi007.stdout:5/396: dwrite d2/dd/f3e [0,4194304] 0 2024-09-21T04:35:15.498 INFO:tasks.workunit.client.0.smithi007.stdout:5/397: rename d2/dd/d23/d49 to d2/dd/d23/d49/d6f/d7b 22 2024-09-21T04:35:15.499 INFO:tasks.workunit.client.1.smithi053.stdout:2/773: truncate d1/d18/d1f/d27/d58/d5c/f113 10574 0 2024-09-21T04:35:15.500 INFO:tasks.workunit.client.0.smithi007.stdout:0/164: mknod d6/d16/d18/c3c 0 2024-09-21T04:35:15.502 INFO:tasks.workunit.client.0.smithi007.stdout:3/131: creat dc/f23 x:0 0 0 2024-09-21T04:35:15.506 INFO:tasks.workunit.client.0.smithi007.stdout:9/174: mknod d13/d18/c3a 0 2024-09-21T04:35:15.506 INFO:tasks.workunit.client.0.smithi007.stdout:9/175: dread - d13/d2a/f2d zero size 2024-09-21T04:35:15.506 INFO:tasks.workunit.client.1.smithi053.stdout:2/774: chown d1/d18/d1f/d27/d58/c6e 11004 1 2024-09-21T04:35:15.506 INFO:tasks.workunit.client.1.smithi053.stdout:8/773: link d3/d5/db1/lb5 d3/df/d14/d16/lfe 0 2024-09-21T04:35:15.507 INFO:tasks.workunit.client.1.smithi053.stdout:3/740: dread d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/d97/f105 [0,4194304] 0 2024-09-21T04:35:15.508 INFO:tasks.workunit.client.1.smithi053.stdout:3/741: chown d16/d1b/dad 1 1 2024-09-21T04:35:15.508 INFO:tasks.workunit.client.0.smithi007.stdout:5/398: mknod d2/d27/d42/d53/c7c 0 2024-09-21T04:35:15.509 INFO:tasks.workunit.client.0.smithi007.stdout:0/165: mkdir d6/d3d 0 2024-09-21T04:35:15.509 INFO:tasks.workunit.client.0.smithi007.stdout:5/399: chown d2/fc 64 1 2024-09-21T04:35:15.510 INFO:tasks.workunit.client.0.smithi007.stdout:3/132: rename dc/fe to dc/d22/f24 0 2024-09-21T04:35:15.511 INFO:tasks.workunit.client.1.smithi053.stdout:4/807: dread d1/dd/de/df5/d20/d4e/d51/f7b [0,4194304] 0 2024-09-21T04:35:15.511 INFO:tasks.workunit.client.0.smithi007.stdout:3/133: truncate fa 4198897 0 2024-09-21T04:35:15.511 INFO:tasks.workunit.client.1.smithi053.stdout:8/774: mkdir d3/d5/d38/d64/d9d/dff 0 2024-09-21T04:35:15.512 INFO:tasks.workunit.client.0.smithi007.stdout:9/176: mknod d13/d2a/c3b 0 2024-09-21T04:35:15.513 INFO:tasks.workunit.client.1.smithi053.stdout:3/742: creat d16/d17/d3c/f114 x:0 0 0 2024-09-21T04:35:15.516 INFO:tasks.workunit.client.0.smithi007.stdout:9/177: dread d13/d18/f35 [0,4194304] 0 2024-09-21T04:35:15.519 INFO:tasks.workunit.client.1.smithi053.stdout:4/808: dread d1/d27/f31 [0,4194304] 0 2024-09-21T04:35:15.519 INFO:tasks.workunit.client.1.smithi053.stdout:4/809: fsync d1/d27/f31 0 2024-09-21T04:35:15.526 INFO:tasks.workunit.client.1.smithi053.stdout:3/743: link d16/d17/d21/d26/d3e/fd3 d16/d17/d21/d8b/d7d/d9d/f115 0 2024-09-21T04:35:15.528 INFO:tasks.workunit.client.0.smithi007.stdout:5/400: symlink d2/d27/l7d 0 2024-09-21T04:35:15.529 INFO:tasks.workunit.client.1.smithi053.stdout:4/810: rename d1/dd/de/df5/d20/d4e/d51/da4/df1/d8e/db4/cda to d1/d27/d53/d88/c107 0 2024-09-21T04:35:15.531 INFO:tasks.workunit.client.1.smithi053.stdout:8/775: dwrite d3/d5/d38/f4e [4194304,4194304] 0 2024-09-21T04:35:15.531 INFO:tasks.workunit.client.0.smithi007.stdout:5/401: dread d2/f51 [0,4194304] 0 2024-09-21T04:35:15.532 INFO:tasks.workunit.client.0.smithi007.stdout:5/402: fdatasync d2/dd/d1d/d20/f6d 0 2024-09-21T04:35:15.532 INFO:tasks.workunit.client.1.smithi053.stdout:4/811: truncate d1/dd/de/df5/db6/fdc 755443 0 2024-09-21T04:35:15.532 INFO:tasks.workunit.client.0.smithi007.stdout:0/166: link d6/d16/d18/d2e/c31 d6/c3e 0 2024-09-21T04:35:15.533 INFO:tasks.workunit.client.0.smithi007.stdout:5/403: fdatasync d2/dd/d23/d49/f6a 0 2024-09-21T04:35:15.535 INFO:tasks.workunit.client.0.smithi007.stdout:9/178: link d13/f28 d13/d18/f3c 0 2024-09-21T04:35:15.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/167: fsync d6/d16/f22 0 2024-09-21T04:35:15.538 INFO:tasks.workunit.client.1.smithi053.stdout:8/776: truncate d3/d5/f19 1364803 0 2024-09-21T04:35:15.538 INFO:tasks.workunit.client.0.smithi007.stdout:3/134: getdents dc 0 2024-09-21T04:35:15.538 INFO:tasks.workunit.client.0.smithi007.stdout:9/179: write d13/d18/f36 [530969,27468] 0 2024-09-21T04:35:15.539 INFO:tasks.workunit.client.0.smithi007.stdout:3/135: chown c7 4976579 1 2024-09-21T04:35:15.540 INFO:tasks.workunit.client.1.smithi053.stdout:3/744: dwrite d16/d17/d21/d26/d49/f77 [0,4194304] 0 2024-09-21T04:35:15.541 INFO:tasks.workunit.client.0.smithi007.stdout:3/136: readlink dc/l12 0 2024-09-21T04:35:15.541 INFO:tasks.workunit.client.1.smithi053.stdout:4/812: dwrite d1/d27/d33/fd4 [0,4194304] 0 2024-09-21T04:35:15.543 INFO:tasks.workunit.client.0.smithi007.stdout:0/168: mkdir d6/d16/d18/d2e/d3f 0 2024-09-21T04:35:15.544 INFO:tasks.workunit.client.0.smithi007.stdout:5/404: dwrite d2/d1e/f5b [0,4194304] 0 2024-09-21T04:35:15.545 INFO:tasks.workunit.client.0.smithi007.stdout:5/405: readlink d2/d27/d42/l79 0 2024-09-21T04:35:15.547 INFO:tasks.workunit.client.0.smithi007.stdout:3/137: creat dc/d22/f25 x:0 0 0 2024-09-21T04:35:15.549 INFO:tasks.workunit.client.1.smithi053.stdout:8/777: truncate d3/df/d14/d16/f29 2916615 0 2024-09-21T04:35:15.553 INFO:tasks.workunit.client.1.smithi053.stdout:3/745: creat d16/d17/d21/d42/d43/d86/f116 x:0 0 0 2024-09-21T04:35:15.553 INFO:tasks.workunit.client.0.smithi007.stdout:0/169: dwrite d6/d16/f2a [0,4194304] 0 2024-09-21T04:35:15.555 INFO:tasks.workunit.client.0.smithi007.stdout:0/170: truncate d6/d16/f22 772613 0 2024-09-21T04:35:15.559 INFO:tasks.workunit.client.1.smithi053.stdout:8/778: fdatasync d3/df/d10/fbb 0 2024-09-21T04:35:15.560 INFO:tasks.workunit.client.0.smithi007.stdout:0/171: dread f1 [0,4194304] 0 2024-09-21T04:35:15.561 INFO:tasks.workunit.client.0.smithi007.stdout:5/406: symlink d2/l7e 0 2024-09-21T04:35:15.561 INFO:tasks.workunit.client.1.smithi053.stdout:8/779: write d3/df/d10/fcf [1788927,34240] 0 2024-09-21T04:35:15.563 INFO:tasks.workunit.client.1.smithi053.stdout:3/746: creat d16/d1b/dad/f117 x:0 0 0 2024-09-21T04:35:15.564 INFO:tasks.workunit.client.1.smithi053.stdout:3/747: truncate d16/d17/ddc/f10a 930233 0 2024-09-21T04:35:15.564 INFO:tasks.workunit.client.1.smithi053.stdout:4/813: link d1/dd/de/df5/d3f/d74/fd3 d1/dd/de/df5/d20/d4e/d51/da4/dc3/d100/f108 0 2024-09-21T04:35:15.565 INFO:tasks.workunit.client.0.smithi007.stdout:0/172: unlink d6/d16/d18/l2d 0 2024-09-21T04:35:15.570 INFO:tasks.workunit.client.0.smithi007.stdout:0/173: unlink d6/c30 0 2024-09-21T04:35:15.571 INFO:tasks.workunit.client.1.smithi053.stdout:4/814: dwrite d1/d27/d2e/d46/f54 [0,4194304] 0 2024-09-21T04:35:15.574 INFO:tasks.workunit.client.0.smithi007.stdout:5/407: link d2/d1e/f58 d2/dd/d23/d49/d6f/f7f 0 2024-09-21T04:35:15.576 INFO:tasks.workunit.client.0.smithi007.stdout:0/174: mknod d6/d16/d18/d39/c40 0 2024-09-21T04:35:15.576 INFO:tasks.workunit.client.0.smithi007.stdout:5/408: stat d2/dd/d1d/d20/c4f 0 2024-09-21T04:35:15.578 INFO:tasks.workunit.client.0.smithi007.stdout:3/138: dwrite dc/f13 [0,4194304] 0 2024-09-21T04:35:15.582 INFO:tasks.workunit.client.1.smithi053.stdout:1/747: sync 2024-09-21T04:35:15.582 INFO:tasks.workunit.client.0.smithi007.stdout:0/175: mknod d6/d16/d18/d27/c41 0 2024-09-21T04:35:15.583 INFO:tasks.workunit.client.1.smithi053.stdout:1/748: stat d7/da8/d74/d91/cd0 0 2024-09-21T04:35:15.583 INFO:tasks.workunit.client.1.smithi053.stdout:7/637: sync 2024-09-21T04:35:15.583 INFO:tasks.workunit.client.1.smithi053.stdout:1/749: readlink d7/d63/d96/la0 0 2024-09-21T04:35:15.584 INFO:tasks.workunit.client.1.smithi053.stdout:3/748: fdatasync d16/d17/d21/d8b/d7d/d57/dc1/d53/f9a 0 2024-09-21T04:35:15.586 INFO:tasks.workunit.client.0.smithi007.stdout:3/139: write fa [3038348,112589] 0 2024-09-21T04:35:15.587 INFO:tasks.workunit.client.1.smithi053.stdout:3/749: chown d16/d17/d21/d8b/d7d/d57/f6b 3032359 1 2024-09-21T04:35:15.588 INFO:tasks.workunit.client.0.smithi007.stdout:0/176: mknod d6/d16/d18/c42 0 2024-09-21T04:35:15.591 INFO:tasks.workunit.client.0.smithi007.stdout:0/177: truncate d6/d16/f22 1440505 0 2024-09-21T04:35:15.591 INFO:tasks.workunit.client.0.smithi007.stdout:3/140: unlink dc/d18/f1d 0 2024-09-21T04:35:15.592 INFO:tasks.workunit.client.1.smithi053.stdout:7/638: dwrite d0/d2b/d36/f89 [0,4194304] 0 2024-09-21T04:35:15.593 INFO:tasks.workunit.client.1.smithi053.stdout:8/780: getdents d3/df/d49 0 2024-09-21T04:35:15.594 INFO:tasks.workunit.client.1.smithi053.stdout:4/815: dread d1/dd/de/f6d [0,4194304] 0 2024-09-21T04:35:15.595 INFO:tasks.workunit.client.0.smithi007.stdout:5/409: dread d2/f55 [0,4194304] 0 2024-09-21T04:35:15.595 INFO:tasks.workunit.client.1.smithi053.stdout:7/639: truncate d0/f49 1339210 0 2024-09-21T04:35:15.596 INFO:tasks.workunit.client.0.smithi007.stdout:3/141: mkdir dc/d18/d26 0 2024-09-21T04:35:15.596 INFO:tasks.workunit.client.1.smithi053.stdout:4/816: truncate d1/d27/f86 1454952 0 2024-09-21T04:35:15.600 INFO:tasks.workunit.client.1.smithi053.stdout:3/750: symlink d16/d17/d21/d8b/d7d/d57/df5/l118 0 2024-09-21T04:35:15.601 INFO:tasks.workunit.client.1.smithi053.stdout:1/750: dread d7/d9/da/d11/f20 [0,4194304] 0 2024-09-21T04:35:15.603 INFO:tasks.workunit.client.1.smithi053.stdout:8/781: mkdir d3/df/d49/d100 0 2024-09-21T04:35:15.604 INFO:tasks.workunit.client.0.smithi007.stdout:3/142: dwrite dc/f10 [4194304,4194304] 0 2024-09-21T04:35:15.605 INFO:tasks.workunit.client.0.smithi007.stdout:3/143: write dc/f10 [6055787,5495] 0 2024-09-21T04:35:15.605 INFO:tasks.workunit.client.0.smithi007.stdout:3/144: chown dc/f15 2 1 2024-09-21T04:35:15.607 INFO:tasks.workunit.client.0.smithi007.stdout:0/178: link d6/d16/f25 d6/d16/d18/d2e/d3f/f43 0 2024-09-21T04:35:15.607 INFO:tasks.workunit.client.0.smithi007.stdout:3/145: write dc/d22/f24 [371691,98924] 0 2024-09-21T04:35:15.609 INFO:tasks.workunit.client.1.smithi053.stdout:4/817: creat d1/dd/de/df5/d20/d4e/d51/da4/dc3/f109 x:0 0 0 2024-09-21T04:35:15.615 INFO:tasks.workunit.client.0.smithi007.stdout:3/146: dwrite dc/f21 [0,4194304] 0 2024-09-21T04:35:15.618 INFO:tasks.workunit.client.1.smithi053.stdout:1/751: rename d7/d9/da/l3a to d7/d47/d79/d99/lf7 0 2024-09-21T04:35:15.619 INFO:tasks.workunit.client.1.smithi053.stdout:1/752: fsync d7/d9/d43/f6d 0 2024-09-21T04:35:15.620 INFO:tasks.workunit.client.1.smithi053.stdout:1/753: write d7/d9/d43/d54/d7f/d36/f97 [1509995,100250] 0 2024-09-21T04:35:15.621 INFO:tasks.workunit.client.0.smithi007.stdout:5/410: rename d2/dd/d1d/f71 to d2/dd/d23/d49/f80 0 2024-09-21T04:35:15.623 INFO:tasks.workunit.client.0.smithi007.stdout:0/179: symlink d6/d3d/l44 0 2024-09-21T04:35:15.625 INFO:tasks.workunit.client.0.smithi007.stdout:3/147: mknod dc/c27 0 2024-09-21T04:35:15.627 INFO:tasks.workunit.client.1.smithi053.stdout:4/818: mknod d1/dd/c10a 0 2024-09-21T04:35:15.629 INFO:tasks.workunit.client.1.smithi053.stdout:7/640: write d0/d6f/dbf/d50/d8b/f42 [1460489,92764] 0 2024-09-21T04:35:15.632 INFO:tasks.workunit.client.0.smithi007.stdout:3/148: dwrite dc/f10 [0,4194304] 0 2024-09-21T04:35:15.633 INFO:tasks.workunit.client.0.smithi007.stdout:3/149: dread - dc/d22/f25 zero size 2024-09-21T04:35:15.639 INFO:tasks.workunit.client.1.smithi053.stdout:7/641: creat d0/d2b/d36/d64/fd1 x:0 0 0 2024-09-21T04:35:15.640 INFO:tasks.workunit.client.0.smithi007.stdout:5/411: truncate d2/dd/d28/f64 759330 0 2024-09-21T04:35:15.644 INFO:tasks.workunit.client.1.smithi053.stdout:8/782: dwrite d3/d5/f8f [0,4194304] 0 2024-09-21T04:35:15.644 INFO:tasks.workunit.client.0.smithi007.stdout:5/412: dread d2/dd/f16 [4194304,4194304] 0 2024-09-21T04:35:15.645 INFO:tasks.workunit.client.0.smithi007.stdout:5/413: chown d2/fc 187992422 1 2024-09-21T04:35:15.646 INFO:tasks.workunit.client.1.smithi053.stdout:4/819: dread d1/dd/de/df5/d20/d73/d92/f94 [0,4194304] 0 2024-09-21T04:35:15.646 INFO:tasks.workunit.client.1.smithi053.stdout:1/754: dread d7/d63/f6a [0,4194304] 0 2024-09-21T04:35:15.650 INFO:tasks.workunit.client.0.smithi007.stdout:5/414: dread d2/dd/f16 [4194304,4194304] 0 2024-09-21T04:35:15.653 INFO:tasks.workunit.client.1.smithi053.stdout:4/820: fsync d1/d27/f9d 0 2024-09-21T04:35:15.656 INFO:tasks.workunit.client.0.smithi007.stdout:5/415: rename d2/f1b to d2/d27/d3d/d56/f81 0 2024-09-21T04:35:15.656 INFO:tasks.workunit.client.1.smithi053.stdout:7/642: dread d0/f20 [0,4194304] 0 2024-09-21T04:35:15.657 INFO:tasks.workunit.client.1.smithi053.stdout:8/783: write d3/d39/f3f [5146193,116064] 0 2024-09-21T04:35:15.658 INFO:tasks.workunit.client.1.smithi053.stdout:8/784: stat d3/d5/d4c/d8e/fce 0 2024-09-21T04:35:15.659 INFO:tasks.workunit.client.1.smithi053.stdout:8/785: fdatasync d3/d5/d38/fe0 0 2024-09-21T04:35:15.659 INFO:tasks.workunit.client.1.smithi053.stdout:7/643: unlink d0/l77 0 2024-09-21T04:35:15.660 INFO:tasks.workunit.client.0.smithi007.stdout:5/416: creat d2/dd/d1d/d20/f82 x:0 0 0 2024-09-21T04:35:15.661 INFO:tasks.workunit.client.1.smithi053.stdout:1/755: link d7/d9/d43/d54/d7f/c5f d7/d63/d96/cf8 0 2024-09-21T04:35:15.661 INFO:tasks.workunit.client.0.smithi007.stdout:5/417: chown d2/dd/d23/d2a/f63 23244 1 2024-09-21T04:35:15.662 INFO:tasks.workunit.client.1.smithi053.stdout:1/756: fdatasync d7/d47/d79/d99/db0/fe7 0 2024-09-21T04:35:15.664 INFO:tasks.workunit.client.1.smithi053.stdout:4/821: rename d1/d27/d33 to d1/dd/de/df5/d20/d73/d10b 0 2024-09-21T04:35:15.665 INFO:tasks.workunit.client.0.smithi007.stdout:2/130: sync 2024-09-21T04:35:15.665 INFO:tasks.workunit.client.0.smithi007.stdout:2/131: readlink d0/d11/l1b 0 2024-09-21T04:35:15.666 INFO:tasks.workunit.client.0.smithi007.stdout:5/418: mkdir d2/d27/d42/d83 0 2024-09-21T04:35:15.666 INFO:tasks.workunit.client.0.smithi007.stdout:2/132: chown d0/c1d 13 1 2024-09-21T04:35:15.668 INFO:tasks.workunit.client.1.smithi053.stdout:1/757: dwrite d7/da8/f83 [0,4194304] 0 2024-09-21T04:35:15.669 INFO:tasks.workunit.client.1.smithi053.stdout:7/644: rename d0/d2b/f2e to d0/d6f/fd2 0 2024-09-21T04:35:15.673 INFO:tasks.workunit.client.0.smithi007.stdout:2/133: dwrite d0/f3 [0,4194304] 0 2024-09-21T04:35:15.673 INFO:tasks.workunit.client.1.smithi053.stdout:4/822: creat d1/dd/de/df5/d3f/d74/de1/f10c x:0 0 0 2024-09-21T04:35:15.675 INFO:tasks.workunit.client.0.smithi007.stdout:5/419: creat d2/dd/d23/d49/d44/f84 x:0 0 0 2024-09-21T04:35:15.677 INFO:tasks.workunit.client.0.smithi007.stdout:2/134: fdatasync d0/f8 0 2024-09-21T04:35:15.682 INFO:tasks.workunit.client.0.smithi007.stdout:2/135: dread d0/f8 [0,4194304] 0 2024-09-21T04:35:15.683 INFO:tasks.workunit.client.1.smithi053.stdout:1/758: mknod d7/d9/d43/d54/d7f/d36/cf9 0 2024-09-21T04:35:15.684 INFO:tasks.workunit.client.1.smithi053.stdout:7/645: rmdir d0/d2b/d36 39 2024-09-21T04:35:15.686 INFO:tasks.workunit.client.0.smithi007.stdout:2/136: dread d0/f8 [0,4194304] 0 2024-09-21T04:35:15.686 INFO:tasks.workunit.client.1.smithi053.stdout:4/823: creat d1/d27/d2e/f10d x:0 0 0 2024-09-21T04:35:15.689 INFO:tasks.workunit.client.0.smithi007.stdout:5/420: link d2/dd/d23/d2a/f7a d2/d27/d3d/f85 0 2024-09-21T04:35:15.689 INFO:tasks.workunit.client.1.smithi053.stdout:1/759: dread d7/d9/da/d4c/f7e [0,4194304] 0 2024-09-21T04:35:15.690 INFO:tasks.workunit.client.1.smithi053.stdout:8/786: dwrite d3/df/fb2 [0,4194304] 0 2024-09-21T04:35:15.691 INFO:tasks.workunit.client.0.smithi007.stdout:2/137: fdatasync d0/f18 0 2024-09-21T04:35:15.692 INFO:tasks.workunit.client.0.smithi007.stdout:5/421: symlink d2/d1e/l86 0 2024-09-21T04:35:15.695 INFO:tasks.workunit.client.0.smithi007.stdout:2/138: dread d0/f8 [0,4194304] 0 2024-09-21T04:35:15.696 INFO:tasks.workunit.client.1.smithi053.stdout:7/646: mknod d0/d6f/dbf/da2/cd3 0 2024-09-21T04:35:15.697 INFO:tasks.workunit.client.1.smithi053.stdout:1/760: creat d7/d9/da/d4c/ffa x:0 0 0 2024-09-21T04:35:15.699 INFO:tasks.workunit.client.1.smithi053.stdout:1/761: truncate d7/d9/da/f2f 4981552 0 2024-09-21T04:35:15.700 INFO:tasks.workunit.client.1.smithi053.stdout:4/824: dread d1/dd/de/df5/d20/d5b/d87/fad [0,4194304] 0 2024-09-21T04:35:15.700 INFO:tasks.workunit.client.1.smithi053.stdout:4/825: stat d1/dd/de/df5/d20/d4e/d51/fab 0 2024-09-21T04:35:15.703 INFO:tasks.workunit.client.0.smithi007.stdout:5/422: creat d2/d27/f87 x:0 0 0 2024-09-21T04:35:15.703 INFO:tasks.workunit.client.0.smithi007.stdout:2/139: dwrite d0/f3 [0,4194304] 0 2024-09-21T04:35:15.705 INFO:tasks.workunit.client.1.smithi053.stdout:1/762: stat d7/d9/da/d3c/f65 0 2024-09-21T04:35:15.708 INFO:tasks.workunit.client.0.smithi007.stdout:2/140: rename d0/d2/l14 to d0/d11/l1e 0 2024-09-21T04:35:15.709 INFO:tasks.workunit.client.0.smithi007.stdout:2/141: chown d0/d2/fc 122233 1 2024-09-21T04:35:15.711 INFO:tasks.workunit.client.0.smithi007.stdout:5/423: rename d2/d27/d42/f72 to d2/dd/d23/d49/d44/d54/f88 0 2024-09-21T04:35:15.712 INFO:tasks.workunit.client.1.smithi053.stdout:4/826: mkdir d1/d27/d53/d10e 0 2024-09-21T04:35:15.713 INFO:tasks.workunit.client.1.smithi053.stdout:1/763: write d7/d9/d43/d54/d7f/fe3 [395482,97157] 0 2024-09-21T04:35:15.713 INFO:tasks.workunit.client.1.smithi053.stdout:8/787: getdents d3/df/d14/d16 0 2024-09-21T04:35:15.714 INFO:tasks.workunit.client.1.smithi053.stdout:4/827: readlink d1/d27/d2e/lc9 0 2024-09-21T04:35:15.715 INFO:tasks.workunit.client.1.smithi053.stdout:1/764: readlink d7/d9/d43/d54/d7f/d36/d81/lc5 0 2024-09-21T04:35:15.717 INFO:tasks.workunit.client.0.smithi007.stdout:2/142: dwrite d0/d2/fc [0,4194304] 0 2024-09-21T04:35:15.719 INFO:tasks.workunit.client.0.smithi007.stdout:2/143: chown d0/d2/fc 8177 1 2024-09-21T04:35:15.720 INFO:tasks.workunit.client.1.smithi053.stdout:8/788: dwrite d3/f3a [0,4194304] 0 2024-09-21T04:35:15.721 INFO:tasks.workunit.client.1.smithi053.stdout:7/647: write d0/d6f/dbf/f8f [995493,126216] 0 2024-09-21T04:35:15.722 INFO:tasks.workunit.client.0.smithi007.stdout:5/424: dwrite d2/dd/d1d/d20/f6d [0,4194304] 0 2024-09-21T04:35:15.723 INFO:tasks.workunit.client.1.smithi053.stdout:4/828: dwrite d1/dd/de/df5/d20/d5b/d87/f93 [0,4194304] 0 2024-09-21T04:35:15.724 INFO:tasks.workunit.client.1.smithi053.stdout:9/706: sync 2024-09-21T04:35:15.724 INFO:tasks.workunit.client.1.smithi053.stdout:5/766: sync 2024-09-21T04:35:15.725 INFO:tasks.workunit.client.1.smithi053.stdout:7/648: write d0/d2b/f32 [1200906,122493] 0 2024-09-21T04:35:15.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/144: symlink d0/d2/l1f 0 2024-09-21T04:35:15.730 INFO:tasks.workunit.client.1.smithi053.stdout:8/789: creat d3/d5/db1/f101 x:0 0 0 2024-09-21T04:35:15.731 INFO:tasks.workunit.client.0.smithi007.stdout:5/425: rmdir d2/dd/d23/d49/d6f 39 2024-09-21T04:35:15.732 INFO:tasks.workunit.client.0.smithi007.stdout:2/145: unlink d0/f12 0 2024-09-21T04:35:15.733 INFO:tasks.workunit.client.0.smithi007.stdout:2/146: fsync d0/f1c 0 2024-09-21T04:35:15.734 INFO:tasks.workunit.client.0.smithi007.stdout:2/147: stat d0/d2/l1f 0 2024-09-21T04:35:15.735 INFO:tasks.workunit.client.0.smithi007.stdout:2/148: read - d0/f1c zero size 2024-09-21T04:35:15.736 INFO:tasks.workunit.client.1.smithi053.stdout:9/707: symlink d0/d1c/d52/daa/lf3 0 2024-09-21T04:35:15.737 INFO:tasks.workunit.client.1.smithi053.stdout:4/829: dread - d1/dd/de/df5/d20/d5b/fbe zero size 2024-09-21T04:35:15.737 INFO:tasks.workunit.client.1.smithi053.stdout:9/708: chown d0/d2/d9/f95 262596333 1 2024-09-21T04:35:15.737 INFO:tasks.workunit.client.1.smithi053.stdout:4/830: readlink d1/dd/de/df5/d3f/l5f 0 2024-09-21T04:35:15.737 INFO:tasks.workunit.client.0.smithi007.stdout:2/149: dread d0/f19 [0,4194304] 0 2024-09-21T04:35:15.738 INFO:tasks.workunit.client.0.smithi007.stdout:5/426: creat d2/dd/d23/d5d/f89 x:0 0 0 2024-09-21T04:35:15.739 INFO:tasks.workunit.client.1.smithi053.stdout:5/767: creat d0/dd/d5c/d92/ddb/d51/d52/ff6 x:0 0 0 2024-09-21T04:35:15.739 INFO:tasks.workunit.client.1.smithi053.stdout:1/765: write d7/d9/d43/d54/d7f/d36/d81/ff1 [62592,65406] 0 2024-09-21T04:35:15.741 INFO:tasks.workunit.client.1.smithi053.stdout:8/790: rename d3/d28/ddc to d3/d4f/d102 0 2024-09-21T04:35:15.742 INFO:tasks.workunit.client.0.smithi007.stdout:5/427: symlink d2/d1e/l8a 0 2024-09-21T04:35:15.744 INFO:tasks.workunit.client.1.smithi053.stdout:9/709: dread d0/fc3 [0,4194304] 0 2024-09-21T04:35:15.746 INFO:tasks.workunit.client.1.smithi053.stdout:9/710: readlink d0/d1a/l22 0 2024-09-21T04:35:15.746 INFO:tasks.workunit.client.1.smithi053.stdout:8/791: rmdir d3/d5/d4c/d53 39 2024-09-21T04:35:15.747 INFO:tasks.workunit.client.0.smithi007.stdout:5/428: symlink d2/l8b 0 2024-09-21T04:35:15.748 INFO:tasks.workunit.client.1.smithi053.stdout:1/766: dwrite d7/d47/ff2 [0,4194304] 0 2024-09-21T04:35:15.749 INFO:tasks.workunit.client.1.smithi053.stdout:1/767: write d7/d9/da/d4c/ffa [135988,33626] 0 2024-09-21T04:35:15.750 INFO:tasks.workunit.client.0.smithi007.stdout:2/150: getdents d0/d2 0 2024-09-21T04:35:15.751 INFO:tasks.workunit.client.0.smithi007.stdout:2/151: write d0/ff [1399647,124555] 0 2024-09-21T04:35:15.754 INFO:tasks.workunit.client.0.smithi007.stdout:5/429: dread d2/fc [0,4194304] 0 2024-09-21T04:35:15.756 INFO:tasks.workunit.client.1.smithi053.stdout:5/768: mkdir d0/dd/d48/d8c/dad/db9/deb/df7 0 2024-09-21T04:35:15.761 INFO:tasks.workunit.client.1.smithi053.stdout:9/711: creat d0/d1a/d89/ff4 x:0 0 0 2024-09-21T04:35:15.761 INFO:tasks.workunit.client.0.smithi007.stdout:2/152: dread d0/d2/f6 [0,4194304] 0 2024-09-21T04:35:15.762 INFO:tasks.workunit.client.1.smithi053.stdout:9/712: dread - d0/df/f72 zero size 2024-09-21T04:35:15.763 INFO:tasks.workunit.client.1.smithi053.stdout:8/792: truncate d3/f59 1159118 0 2024-09-21T04:35:15.765 INFO:tasks.workunit.client.1.smithi053.stdout:7/649: link d0/d6f/dbf/d50/d8b/l25 d0/d6f/dbf/ld4 0 2024-09-21T04:35:15.765 INFO:tasks.workunit.client.0.smithi007.stdout:2/153: dread d0/ff [0,4194304] 0 2024-09-21T04:35:15.767 INFO:tasks.workunit.client.1.smithi053.stdout:1/768: symlink d7/d9/d43/d54/d7f/d36/lfb 0 2024-09-21T04:35:15.770 INFO:tasks.workunit.client.0.smithi007.stdout:5/430: link d2/dd/f14 d2/dd/d28/f8c 0 2024-09-21T04:35:15.770 INFO:tasks.workunit.client.0.smithi007.stdout:5/431: dread - d2/dd/d23/d5d/f89 zero size 2024-09-21T04:35:15.772 INFO:tasks.workunit.client.1.smithi053.stdout:1/769: dwrite d7/d63/d6b/f8d [0,4194304] 0 2024-09-21T04:35:15.772 INFO:tasks.workunit.client.1.smithi053.stdout:9/713: unlink d0/d1c/d21/d46/fd7 0 2024-09-21T04:35:15.773 INFO:tasks.workunit.client.1.smithi053.stdout:7/650: truncate d0/fb 3534449 0 2024-09-21T04:35:15.773 INFO:tasks.workunit.client.1.smithi053.stdout:8/793: write d3/d5/d38/d64/d9d/fb9 [1312213,35929] 0 2024-09-21T04:35:15.774 INFO:tasks.workunit.client.0.smithi007.stdout:2/154: rename d0/ff to d0/d11/f20 0 2024-09-21T04:35:15.775 INFO:tasks.workunit.client.1.smithi053.stdout:7/651: chown d0/d24/f5a 387 1 2024-09-21T04:35:15.775 INFO:tasks.workunit.client.1.smithi053.stdout:8/794: dread - d3/d5/db1/fc7 zero size 2024-09-21T04:35:15.775 INFO:tasks.workunit.client.0.smithi007.stdout:2/155: truncate d0/d2/f6 2019730 0 2024-09-21T04:35:15.776 INFO:tasks.workunit.client.0.smithi007.stdout:5/432: fsync d2/dd/d23/d2a/f2f 0 2024-09-21T04:35:15.778 INFO:tasks.workunit.client.0.smithi007.stdout:5/433: write d2/dd/d23/d2a/f2d [2507795,62537] 0 2024-09-21T04:35:15.780 INFO:tasks.workunit.client.0.smithi007.stdout:2/156: dread d0/d2/f6 [0,4194304] 0 2024-09-21T04:35:15.782 INFO:tasks.workunit.client.1.smithi053.stdout:8/795: dwrite d3/d28/fd5 [0,4194304] 0 2024-09-21T04:35:15.783 INFO:tasks.workunit.client.1.smithi053.stdout:8/796: chown d3/d5/d38/fe0 3 1 2024-09-21T04:35:15.784 INFO:tasks.workunit.client.0.smithi007.stdout:5/434: dread d2/dd/f16 [4194304,4194304] 0 2024-09-21T04:35:15.785 INFO:tasks.workunit.client.1.smithi053.stdout:6/719: sync 2024-09-21T04:35:15.821 INFO:tasks.workunit.client.1.smithi053.stdout:4/831: getdents d1/dd/de/df5/d20/d4e/d51/da4 0 2024-09-21T04:35:15.829 INFO:tasks.workunit.client.1.smithi053.stdout:1/770: fsync d7/da8/d74/d75/faa 0 2024-09-21T04:35:15.830 INFO:tasks.workunit.client.1.smithi053.stdout:1/771: chown d7/da8/d74/c8b 6 1 2024-09-21T04:35:15.830 INFO:tasks.workunit.client.1.smithi053.stdout:1/772: dread - d7/d9/da/fd9 zero size 2024-09-21T04:35:15.835 INFO:tasks.workunit.client.0.smithi007.stdout:2/157: unlink d0/d11/f1a 0 2024-09-21T04:35:15.840 INFO:tasks.workunit.client.1.smithi053.stdout:8/797: mkdir d3/df/d49/d103 0 2024-09-21T04:35:15.847 INFO:tasks.workunit.client.1.smithi053.stdout:6/720: rename d2/d1b/d1e/dbe to d2/d1b/d3d/de9 0 2024-09-21T04:35:15.853 INFO:tasks.workunit.client.1.smithi053.stdout:6/721: dwrite d2/d1b/d1e/d27/d90/fd3 [0,4194304] 0 2024-09-21T04:35:15.864 INFO:tasks.workunit.client.0.smithi007.stdout:4/190: sync 2024-09-21T04:35:15.872 INFO:tasks.workunit.client.0.smithi007.stdout:4/191: dwrite d2/d17/f2a [0,4194304] 0 2024-09-21T04:35:15.875 INFO:tasks.workunit.client.1.smithi053.stdout:2/775: sync 2024-09-21T04:35:15.875 INFO:tasks.workunit.client.1.smithi053.stdout:3/751: sync 2024-09-21T04:35:15.875 INFO:tasks.workunit.client.1.smithi053.stdout:0/703: sync 2024-09-21T04:35:15.876 INFO:tasks.workunit.client.1.smithi053.stdout:2/776: fdatasync d1/d18/d1f/da2/ff8 0 2024-09-21T04:35:15.877 INFO:tasks.workunit.client.1.smithi053.stdout:2/777: fsync d1/d18/d1f/d27/d58/d5c/fc0 0 2024-09-21T04:35:15.879 INFO:tasks.workunit.client.1.smithi053.stdout:2/778: truncate d1/d18/d1f/d27/d58/f108 1835347 0 2024-09-21T04:35:15.880 INFO:tasks.workunit.client.0.smithi007.stdout:4/192: unlink d2/d7/l37 0 2024-09-21T04:35:15.880 INFO:tasks.workunit.client.0.smithi007.stdout:4/193: chown d2/f12 852 1 2024-09-21T04:35:15.888 INFO:tasks.workunit.client.0.smithi007.stdout:4/194: dwrite d2/f38 [0,4194304] 0 2024-09-21T04:35:15.906 INFO:tasks.workunit.client.0.smithi007.stdout:4/195: dread d2/d7/f8 [0,4194304] 0 2024-09-21T04:35:15.912 INFO:tasks.workunit.client.0.smithi007.stdout:4/196: creat d2/d17/d2e/f3e x:0 0 0 2024-09-21T04:35:15.920 INFO:tasks.workunit.client.0.smithi007.stdout:4/197: link d2/f38 d2/d3c/f3f 0 2024-09-21T04:35:15.926 INFO:tasks.workunit.client.0.smithi007.stdout:4/198: dwrite d2/fd [0,4194304] 0 2024-09-21T04:35:15.934 INFO:tasks.workunit.client.0.smithi007.stdout:4/199: symlink d2/d17/d2e/l40 0 2024-09-21T04:35:15.941 INFO:tasks.workunit.client.1.smithi053.stdout:5/769: creat d0/dd/d48/d8c/dad/ff8 x:0 0 0 2024-09-21T04:35:15.942 INFO:tasks.workunit.client.1.smithi053.stdout:6/722: symlink d2/d1b/d3d/d49/dad/lea 0 2024-09-21T04:35:15.943 INFO:tasks.workunit.client.0.smithi007.stdout:4/200: dwrite d2/d7/f31 [4194304,4194304] 0 2024-09-21T04:35:15.943 INFO:tasks.workunit.client.1.smithi053.stdout:0/704: fsync d5/de/d42/f48 0 2024-09-21T04:35:15.946 INFO:tasks.workunit.client.1.smithi053.stdout:3/752: creat d16/d17/d21/d42/d43/d86/f119 x:0 0 0 2024-09-21T04:35:15.947 INFO:tasks.workunit.client.0.smithi007.stdout:4/201: creat d2/d17/d18/f41 x:0 0 0 2024-09-21T04:35:15.950 INFO:tasks.workunit.client.0.smithi007.stdout:4/202: mknod d2/c42 0 2024-09-21T04:35:15.952 INFO:tasks.workunit.client.1.smithi053.stdout:6/723: dwrite d2/d1b/d3d/d49/de/d9c/fbd [0,4194304] 0 2024-09-21T04:35:15.954 INFO:tasks.workunit.client.0.smithi007.stdout:4/203: rename d2/d7/f31 to d2/f43 0 2024-09-21T04:35:15.956 INFO:tasks.workunit.client.1.smithi053.stdout:8/798: dread d3/d28/fad [0,4194304] 0 2024-09-21T04:35:15.956 INFO:tasks.workunit.client.0.smithi007.stdout:4/204: write d2/d17/d2e/f33 [975578,48748] 0 2024-09-21T04:35:15.958 INFO:tasks.workunit.client.1.smithi053.stdout:0/705: stat d5/f8d 0 2024-09-21T04:35:15.960 INFO:tasks.workunit.client.0.smithi007.stdout:5/435: creat d2/f8d x:0 0 0 2024-09-21T04:35:15.968 INFO:tasks.workunit.client.1.smithi053.stdout:3/753: creat d16/d17/d21/d8b/d7d/dc6/f11a x:0 0 0 2024-09-21T04:35:15.969 INFO:tasks.workunit.client.0.smithi007.stdout:5/436: fsync d2/dd/d23/d49/d6f/f7f 0 2024-09-21T04:35:15.974 INFO:tasks.workunit.client.0.smithi007.stdout:5/437: fsync d2/dd/d1d/f1f 0 2024-09-21T04:35:15.976 INFO:tasks.workunit.client.1.smithi053.stdout:6/724: dread d2/d1b/d3d/d49/de/d32/d80/f21 [0,4194304] 0 2024-09-21T04:35:15.976 INFO:tasks.workunit.client.0.smithi007.stdout:5/438: fsync d2/d1e/f37 0 2024-09-21T04:35:15.977 INFO:tasks.workunit.client.1.smithi053.stdout:5/770: creat d0/dd/d5c/d92/de6/ff9 x:0 0 0 2024-09-21T04:35:15.979 INFO:tasks.workunit.client.0.smithi007.stdout:2/158: dwrite d0/f19 [0,4194304] 0 2024-09-21T04:35:15.980 INFO:tasks.workunit.client.0.smithi007.stdout:2/159: chown d0/f18 50 1 2024-09-21T04:35:15.983 INFO:tasks.workunit.client.1.smithi053.stdout:2/779: rmdir d1/d18/d1f/d27/d58/d5c/d91/dd0/d117 0 2024-09-21T04:35:15.989 INFO:tasks.workunit.client.1.smithi053.stdout:4/832: link d1/dd/de/df5/d3f/f9a d1/d27/f10f 0 2024-09-21T04:35:15.989 INFO:tasks.workunit.client.0.smithi007.stdout:2/160: dwrite d0/f1c [0,4194304] 0 2024-09-21T04:35:15.990 INFO:tasks.workunit.client.1.smithi053.stdout:4/833: truncate d1/d27/d90/fb1 788774 0 2024-09-21T04:35:15.992 INFO:tasks.workunit.client.1.smithi053.stdout:4/834: write d1/d27/d2e/f61 [824747,99331] 0 2024-09-21T04:35:15.992 INFO:tasks.workunit.client.0.smithi007.stdout:2/161: creat d0/d2/f21 x:0 0 0 2024-09-21T04:35:15.993 INFO:tasks.workunit.client.0.smithi007.stdout:2/162: chown d0/d2/l17 22099 1 2024-09-21T04:35:15.994 INFO:tasks.workunit.client.1.smithi053.stdout:0/706: symlink d5/d10/lec 0 2024-09-21T04:35:16.000 INFO:tasks.workunit.client.1.smithi053.stdout:0/707: dread d5/de/fb6 [0,4194304] 0 2024-09-21T04:35:16.001 INFO:tasks.workunit.client.0.smithi007.stdout:2/163: dwrite d0/f19 [0,4194304] 0 2024-09-21T04:35:16.004 INFO:tasks.workunit.client.0.smithi007.stdout:2/164: symlink d0/d11/l22 0 2024-09-21T04:35:16.006 INFO:tasks.workunit.client.1.smithi053.stdout:8/799: link d3/d5/d38/d64/d9d/cab d3/df/d49/c104 0 2024-09-21T04:35:16.009 INFO:tasks.workunit.client.1.smithi053.stdout:6/725: dread d2/d1b/d3d/d49/de/d96/f7b [0,4194304] 0 2024-09-21T04:35:16.013 INFO:tasks.workunit.client.1.smithi053.stdout:0/708: symlink d5/d10/d4c/db1/dbc/dca/led 0 2024-09-21T04:35:16.014 INFO:tasks.workunit.client.1.smithi053.stdout:8/800: mknod d3/df/d49/d100/c105 0 2024-09-21T04:35:16.017 INFO:tasks.workunit.client.1.smithi053.stdout:0/709: creat d5/d10/d4c/db1/dbc/d8c/fee x:0 0 0 2024-09-21T04:35:16.018 INFO:tasks.workunit.client.1.smithi053.stdout:2/780: rename d1/d18/d1f/d27/d58/c63 to d1/d18/d77/c118 0 2024-09-21T04:35:16.018 INFO:tasks.workunit.client.1.smithi053.stdout:8/801: dread d3/d39/fb0 [4194304,4194304] 0 2024-09-21T04:35:16.019 INFO:tasks.workunit.client.1.smithi053.stdout:8/802: stat d3/d5/d4c/d8e/cfc 0 2024-09-21T04:35:16.020 INFO:tasks.workunit.client.1.smithi053.stdout:0/710: write d5/f92 [877779,7745] 0 2024-09-21T04:35:16.020 INFO:tasks.workunit.client.1.smithi053.stdout:2/781: write d1/d18/d77/d9b/faa [2414405,98682] 0 2024-09-21T04:35:16.022 INFO:tasks.workunit.client.1.smithi053.stdout:0/711: chown d5/d10/d4c/db1/dbc 28806 1 2024-09-21T04:35:16.024 INFO:tasks.workunit.client.1.smithi053.stdout:2/782: unlink d1/d18/d77/d9b/faa 0 2024-09-21T04:35:16.025 INFO:tasks.workunit.client.1.smithi053.stdout:8/803: mknod d3/d5/d4c/d53/d66/c106 0 2024-09-21T04:35:16.026 INFO:tasks.workunit.client.1.smithi053.stdout:2/783: read d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/fff [3379997,108762] 0 2024-09-21T04:35:16.026 INFO:tasks.workunit.client.1.smithi053.stdout:0/712: symlink d5/d10/d4c/db1/dbc/dca/db5/lef 0 2024-09-21T04:35:16.027 INFO:tasks.workunit.client.1.smithi053.stdout:8/804: write d3/df/dba/fca [3177943,52929] 0 2024-09-21T04:35:16.028 INFO:tasks.workunit.client.1.smithi053.stdout:2/784: fsync d1/d18/d1f/d27/d41/d4b/d4d/f88 0 2024-09-21T04:35:16.033 INFO:tasks.workunit.client.1.smithi053.stdout:8/805: link d3/d5/d4b/l51 d3/d5/d4b/d5c/d86/l107 0 2024-09-21T04:35:16.034 INFO:tasks.workunit.client.1.smithi053.stdout:8/806: read - d3/d5/d4c/d8e/ff5 zero size 2024-09-21T04:35:16.035 INFO:tasks.workunit.client.1.smithi053.stdout:0/713: mknod d5/de/d42/cf0 0 2024-09-21T04:35:16.035 INFO:tasks.workunit.client.1.smithi053.stdout:8/807: dread - d3/d5/d4b/fa7 zero size 2024-09-21T04:35:16.036 INFO:tasks.workunit.client.1.smithi053.stdout:0/714: fdatasync d5/dd/d50/f6e 0 2024-09-21T04:35:16.040 INFO:tasks.workunit.client.1.smithi053.stdout:0/715: link d5/f15 d5/de/d1c/ff1 0 2024-09-21T04:35:16.045 INFO:tasks.workunit.client.1.smithi053.stdout:0/716: dread d5/de/d9e/faf [0,4194304] 0 2024-09-21T04:35:16.047 INFO:tasks.workunit.client.1.smithi053.stdout:0/717: truncate d5/d10/d4c/db1/dbc/dca/db5/d99/fce 522983 0 2024-09-21T04:35:16.048 INFO:tasks.workunit.client.1.smithi053.stdout:4/835: symlink d1/d27/d53/l110 0 2024-09-21T04:35:16.050 INFO:tasks.workunit.client.1.smithi053.stdout:8/808: dread d3/df/f81 [0,4194304] 0 2024-09-21T04:35:16.059 INFO:tasks.workunit.client.1.smithi053.stdout:9/714: truncate d0/d76/fc9 1068584 0 2024-09-21T04:35:16.061 INFO:tasks.workunit.client.0.smithi007.stdout:4/205: getdents d2/d17/d2e 0 2024-09-21T04:35:16.061 INFO:tasks.workunit.client.1.smithi053.stdout:1/773: link d7/da8/d74/d91/cd0 d7/d47/d79/cfc 0 2024-09-21T04:35:16.063 INFO:tasks.workunit.client.1.smithi053.stdout:8/809: dread d3/df/d14/d16/f29 [0,4194304] 0 2024-09-21T04:35:16.064 INFO:tasks.workunit.client.0.smithi007.stdout:4/206: creat d2/f44 x:0 0 0 2024-09-21T04:35:16.064 INFO:tasks.workunit.client.1.smithi053.stdout:1/774: rmdir d7/d63/d96/dc7 39 2024-09-21T04:35:16.065 INFO:tasks.workunit.client.1.smithi053.stdout:8/810: chown d3/d5/d38/d64/d9d/da9/fe3 503607 1 2024-09-21T04:35:16.067 INFO:tasks.workunit.client.1.smithi053.stdout:7/652: dwrite d0/d6f/dbf/d50/d8b/f1c [0,4194304] 0 2024-09-21T04:35:16.070 INFO:tasks.workunit.client.1.smithi053.stdout:8/811: unlink d3/d5/cf4 0 2024-09-21T04:35:16.075 INFO:tasks.workunit.client.1.smithi053.stdout:8/812: dread d3/d39/fb0 [4194304,4194304] 0 2024-09-21T04:35:16.079 INFO:tasks.workunit.client.1.smithi053.stdout:8/813: getdents d3/d5/d4b/d5c/d86/dcd 0 2024-09-21T04:35:16.079 INFO:tasks.workunit.client.1.smithi053.stdout:8/814: readlink d3/d28/leb 0 2024-09-21T04:35:16.086 INFO:tasks.workunit.client.0.smithi007.stdout:5/439: write d2/d27/d3d/f85 [943001,110136] 0 2024-09-21T04:35:16.089 INFO:tasks.workunit.client.0.smithi007.stdout:5/440: symlink d2/dd/d23/d49/l8e 0 2024-09-21T04:35:16.090 INFO:tasks.workunit.client.1.smithi053.stdout:3/754: write d16/d17/d21/d26/d2b/faa [1112911,128187] 0 2024-09-21T04:35:16.093 INFO:tasks.workunit.client.1.smithi053.stdout:3/755: creat d16/d17/d21/d26/d3e/f11b x:0 0 0 2024-09-21T04:35:16.094 INFO:tasks.workunit.client.1.smithi053.stdout:3/756: chown d16/d17/d21/d26/d71/df1 257490 1 2024-09-21T04:35:16.097 INFO:tasks.workunit.client.1.smithi053.stdout:0/718: getdents d5/d10/d4c/db1/dbc/dca 0 2024-09-21T04:35:16.100 INFO:tasks.workunit.client.1.smithi053.stdout:3/757: link fa d16/d1b/dad/f11c 0 2024-09-21T04:35:16.100 INFO:tasks.workunit.client.0.smithi007.stdout:8/131: sync 2024-09-21T04:35:16.104 INFO:tasks.workunit.client.1.smithi053.stdout:0/719: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f91 [0,4194304] 0 2024-09-21T04:35:16.107 INFO:tasks.workunit.client.0.smithi007.stdout:2/165: write d0/f7 [2822863,120660] 0 2024-09-21T04:35:16.107 INFO:tasks.workunit.client.1.smithi053.stdout:3/758: link d16/d17/d21/d42/d43/d86/fc7 d16/d17/d21/d8b/d7d/d57/df5/f11d 0 2024-09-21T04:35:16.109 INFO:tasks.workunit.client.0.smithi007.stdout:2/166: fdatasync d0/d2/fc 0 2024-09-21T04:35:16.109 INFO:tasks.workunit.client.0.smithi007.stdout:8/132: read d0/f15 [953426,84930] 0 2024-09-21T04:35:16.110 INFO:tasks.workunit.client.0.smithi007.stdout:5/441: dwrite d2/dd/d23/d49/f80 [0,4194304] 0 2024-09-21T04:35:16.112 INFO:tasks.workunit.client.0.smithi007.stdout:2/167: write d0/fd [3915434,92293] 0 2024-09-21T04:35:16.112 INFO:tasks.workunit.client.0.smithi007.stdout:8/133: chown d0/f20 5676 1 2024-09-21T04:35:16.114 INFO:tasks.workunit.client.1.smithi053.stdout:3/759: chown d16/d17/d21/d42/d43/c93 1599422 1 2024-09-21T04:35:16.115 INFO:tasks.workunit.client.1.smithi053.stdout:4/836: truncate d1/dd/de/df5/d20/d73/d10b/d41/fa3 76280 0 2024-09-21T04:35:16.118 INFO:tasks.workunit.client.1.smithi053.stdout:0/720: rename d5/d10/d4c/db1/dbc/dca/db5/c7f to d5/d10/cf2 0 2024-09-21T04:35:16.122 INFO:tasks.workunit.client.0.smithi007.stdout:2/168: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:16.123 INFO:tasks.workunit.client.0.smithi007.stdout:5/442: mknod d2/dd/d1d/d20/c8f 0 2024-09-21T04:35:16.125 INFO:tasks.workunit.client.0.smithi007.stdout:4/207: write d2/d7/f24 [747333,46297] 0 2024-09-21T04:35:16.128 INFO:tasks.workunit.client.1.smithi053.stdout:1/775: write d7/d9/d43/d54/f80 [358593,120167] 0 2024-09-21T04:35:16.129 INFO:tasks.workunit.client.1.smithi053.stdout:9/715: dwrite d0/d2/d9/d1f/f4f [0,4194304] 0 2024-09-21T04:35:16.130 INFO:tasks.workunit.client.1.smithi053.stdout:3/760: rename d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/d97 to d16/d17/d21/d42/da0/dfb/d11e 0 2024-09-21T04:35:16.131 INFO:tasks.workunit.client.1.smithi053.stdout:1/776: write d7/d9/da/f50 [1752387,29467] 0 2024-09-21T04:35:16.131 INFO:tasks.workunit.client.0.smithi007.stdout:8/134: dwrite d0/d3/d8/fa [0,4194304] 0 2024-09-21T04:35:16.132 INFO:tasks.workunit.client.1.smithi053.stdout:7/653: write d0/f13 [94470,53155] 0 2024-09-21T04:35:16.134 INFO:tasks.workunit.client.0.smithi007.stdout:5/443: creat d2/d27/d42/d53/f90 x:0 0 0 2024-09-21T04:35:16.134 INFO:tasks.workunit.client.0.smithi007.stdout:4/208: mkdir d2/d17/d2e/d45 0 2024-09-21T04:35:16.137 INFO:tasks.workunit.client.1.smithi053.stdout:7/654: dread d0/f20 [0,4194304] 0 2024-09-21T04:35:16.138 INFO:tasks.workunit.client.0.smithi007.stdout:5/444: rmdir d2/d27/d42 39 2024-09-21T04:35:16.139 INFO:tasks.workunit.client.0.smithi007.stdout:5/445: dread - d2/d27/f87 zero size 2024-09-21T04:35:16.142 INFO:tasks.workunit.client.0.smithi007.stdout:8/135: symlink d0/d3/l22 0 2024-09-21T04:35:16.146 INFO:tasks.workunit.client.1.smithi053.stdout:9/716: mknod d0/d2/d9/d34/d58/d92/cf5 0 2024-09-21T04:35:16.149 INFO:tasks.workunit.client.0.smithi007.stdout:8/136: dwrite d0/d3/d8/fa [0,4194304] 0 2024-09-21T04:35:16.153 INFO:tasks.workunit.client.0.smithi007.stdout:5/446: rename d2/d27/d42/d53/f73 to d2/dd/d23/d5d/d61/f91 0 2024-09-21T04:35:16.154 INFO:tasks.workunit.client.0.smithi007.stdout:8/137: rename d0/d3/d8 to d0/d3/d8/de/d23 22 2024-09-21T04:35:16.155 INFO:tasks.workunit.client.0.smithi007.stdout:8/138: write d0/f20 [254586,43924] 0 2024-09-21T04:35:16.157 INFO:tasks.workunit.client.1.smithi053.stdout:3/761: rename d16/d17/d21/d42/c7f to d16/d1b/dad/c11f 0 2024-09-21T04:35:16.160 INFO:tasks.workunit.client.1.smithi053.stdout:0/721: write d5/f8d [1911420,130912] 0 2024-09-21T04:35:16.161 INFO:tasks.workunit.client.1.smithi053.stdout:9/717: symlink d0/d2/ddf/lf6 0 2024-09-21T04:35:16.165 INFO:tasks.workunit.client.0.smithi007.stdout:5/447: unlink d2/c31 0 2024-09-21T04:35:16.166 INFO:tasks.workunit.client.1.smithi053.stdout:1/777: symlink d7/d63/d96/de2/lfd 0 2024-09-21T04:35:16.167 INFO:tasks.workunit.client.0.smithi007.stdout:8/139: symlink d0/l24 0 2024-09-21T04:35:16.171 INFO:tasks.workunit.client.0.smithi007.stdout:8/140: dread d0/f20 [0,4194304] 0 2024-09-21T04:35:16.172 INFO:tasks.workunit.client.0.smithi007.stdout:8/141: fdatasync d0/d3/d16/f1b 0 2024-09-21T04:35:16.173 INFO:tasks.workunit.client.0.smithi007.stdout:8/142: rename d0 to d0/d3/d16/d25 22 2024-09-21T04:35:16.176 INFO:tasks.workunit.client.1.smithi053.stdout:9/718: mkdir d0/d1c/d52/df7 0 2024-09-21T04:35:16.177 INFO:tasks.workunit.client.1.smithi053.stdout:4/837: link d1/dd/de/df5/d20/d73/d92/fb7 d1/dd/de/df5/f111 0 2024-09-21T04:35:16.177 INFO:tasks.workunit.client.1.smithi053.stdout:9/719: write d0/d1a/d89/da1/fe1 [965035,116216] 0 2024-09-21T04:35:16.180 INFO:tasks.workunit.client.0.smithi007.stdout:8/143: unlink d0/f15 0 2024-09-21T04:35:16.181 INFO:tasks.workunit.client.1.smithi053.stdout:7/655: write d0/d6f/dbf/d50/fa4 [665479,48064] 0 2024-09-21T04:35:16.182 INFO:tasks.workunit.client.0.smithi007.stdout:5/448: getdents d2/d27 0 2024-09-21T04:35:16.182 INFO:tasks.workunit.client.1.smithi053.stdout:7/656: write d0/d6f/dbf/d5c/f8a [317696,64651] 0 2024-09-21T04:35:16.183 INFO:tasks.workunit.client.0.smithi007.stdout:5/449: chown d2/d1e/l8a 503725 1 2024-09-21T04:35:16.184 INFO:tasks.workunit.client.0.smithi007.stdout:8/144: mkdir d0/d3/d8/de/d26 0 2024-09-21T04:35:16.185 INFO:tasks.workunit.client.1.smithi053.stdout:4/838: readlink d1/dd/d95/lca 0 2024-09-21T04:35:16.186 INFO:tasks.workunit.client.1.smithi053.stdout:1/778: symlink d7/d9/d43/d54/d7f/d36/d81/lfe 0 2024-09-21T04:35:16.187 INFO:tasks.workunit.client.0.smithi007.stdout:5/450: rename d2/dd/l10 to d2/dd/d23/d5d/d61/l92 0 2024-09-21T04:35:16.188 INFO:tasks.workunit.client.1.smithi053.stdout:7/657: mknod d0/d6f/dbf/d5d/cd5 0 2024-09-21T04:35:16.189 INFO:tasks.workunit.client.1.smithi053.stdout:4/839: mkdir d1/dd/de/df5/d20/d5b/d87/d112 0 2024-09-21T04:35:16.190 INFO:tasks.workunit.client.1.smithi053.stdout:9/720: truncate d0/d1a/d4c/fab 303698 0 2024-09-21T04:35:16.191 INFO:tasks.workunit.client.0.smithi007.stdout:5/451: truncate d2/f66 174719 0 2024-09-21T04:35:16.191 INFO:tasks.workunit.client.1.smithi053.stdout:7/658: fsync d0/d2b/f41 0 2024-09-21T04:35:16.192 INFO:tasks.workunit.client.1.smithi053.stdout:4/840: read - d1/dd/de/f83 zero size 2024-09-21T04:35:16.192 INFO:tasks.workunit.client.1.smithi053.stdout:7/659: fdatasync d0/d24/f65 0 2024-09-21T04:35:16.201 INFO:tasks.workunit.client.0.smithi007.stdout:5/452: write d2/d1e/f58 [4728346,31316] 0 2024-09-21T04:35:16.215 INFO:tasks.workunit.client.0.smithi007.stdout:5/453: dread d2/dd/f14 [0,4194304] 0 2024-09-21T04:35:16.216 INFO:tasks.workunit.client.0.smithi007.stdout:5/454: read - d2/dd/d23/d2a/f75 zero size 2024-09-21T04:35:16.218 INFO:tasks.workunit.client.0.smithi007.stdout:5/455: mknod d2/dd/d1d/d20/c93 0 2024-09-21T04:35:16.220 INFO:tasks.workunit.client.0.smithi007.stdout:5/456: mkdir d2/dd/d23/d2a/d94 0 2024-09-21T04:35:16.225 INFO:tasks.workunit.client.0.smithi007.stdout:5/457: truncate d2/d1e/f58 2371107 0 2024-09-21T04:35:16.229 INFO:tasks.workunit.client.0.smithi007.stdout:5/458: dread d2/f5a [0,4194304] 0 2024-09-21T04:35:16.229 INFO:tasks.workunit.client.0.smithi007.stdout:5/459: chown d2/dd 14639938 1 2024-09-21T04:35:16.230 INFO:tasks.workunit.client.0.smithi007.stdout:5/460: read d2/dd/d1d/d20/f6d [237406,23975] 0 2024-09-21T04:35:16.236 INFO:tasks.workunit.client.0.smithi007.stdout:5/461: dread d2/dd/d23/d49/f4a [0,4194304] 0 2024-09-21T04:35:16.237 INFO:tasks.workunit.client.1.smithi053.stdout:5/771: sync 2024-09-21T04:35:16.237 INFO:tasks.workunit.client.1.smithi053.stdout:9/721: mkdir d0/d1c/d52/db5/df8 0 2024-09-21T04:35:16.238 INFO:tasks.workunit.client.1.smithi053.stdout:0/722: truncate d5/de/d9e/faf 457072 0 2024-09-21T04:35:16.239 INFO:tasks.workunit.client.0.smithi007.stdout:5/462: mknod d2/d27/d42/d53/c95 0 2024-09-21T04:35:16.240 INFO:tasks.workunit.client.1.smithi053.stdout:7/660: creat d0/d6f/dbf/d50/d8b/d70/fd6 x:0 0 0 2024-09-21T04:35:16.240 INFO:tasks.workunit.client.1.smithi053.stdout:5/772: dread d0/dd/d48/d8c/dad/fd8 [0,4194304] 0 2024-09-21T04:35:16.241 INFO:tasks.workunit.client.0.smithi007.stdout:5/463: truncate d2/d27/d3d/d56/f81 5141025 0 2024-09-21T04:35:16.241 INFO:tasks.workunit.client.1.smithi053.stdout:4/841: truncate d1/dd/de/df5/f84 965215 0 2024-09-21T04:35:16.244 INFO:tasks.workunit.client.1.smithi053.stdout:0/723: symlink d5/d10/d4c/db1/dbc/dca/lf3 0 2024-09-21T04:35:16.245 INFO:tasks.workunit.client.1.smithi053.stdout:5/773: mkdir d0/dd/d5c/d92/dfa 0 2024-09-21T04:35:16.248 INFO:tasks.workunit.client.1.smithi053.stdout:9/722: write d0/df/f48 [5022957,12103] 0 2024-09-21T04:35:16.250 INFO:tasks.workunit.client.1.smithi053.stdout:0/724: rmdir d5/de/d9e/dd8 39 2024-09-21T04:35:16.250 INFO:tasks.workunit.client.0.smithi007.stdout:5/464: getdents d2/dd/d23 0 2024-09-21T04:35:16.251 INFO:tasks.workunit.client.0.smithi007.stdout:5/465: fsync d2/f50 0 2024-09-21T04:35:16.252 INFO:tasks.workunit.client.0.smithi007.stdout:5/466: fdatasync d2/dd/d23/d2a/f7a 0 2024-09-21T04:35:16.254 INFO:tasks.workunit.client.1.smithi053.stdout:4/842: write d1/dd/de/df5/d20/d4e/d51/da4/dc3/d100/f108 [484394,90636] 0 2024-09-21T04:35:16.256 INFO:tasks.workunit.client.1.smithi053.stdout:4/843: write d1/d27/d2e/d46/fbd [3153114,130837] 0 2024-09-21T04:35:16.258 INFO:tasks.workunit.client.1.smithi053.stdout:5/774: write d0/dd/f37 [1706271,120483] 0 2024-09-21T04:35:16.263 INFO:tasks.workunit.client.1.smithi053.stdout:0/725: dread d5/f15 [4194304,4194304] 0 2024-09-21T04:35:16.265 INFO:tasks.workunit.client.1.smithi053.stdout:9/723: write d0/d1a/d4c/fab [1134938,43387] 0 2024-09-21T04:35:16.267 INFO:tasks.workunit.client.0.smithi007.stdout:5/467: truncate d2/dd/d23/d49/d44/d54/f59 3027443 0 2024-09-21T04:35:16.268 INFO:tasks.workunit.client.1.smithi053.stdout:9/724: creat d0/d1c/d21/d46/ff9 x:0 0 0 2024-09-21T04:35:16.268 INFO:tasks.workunit.client.1.smithi053.stdout:5/775: write d0/dd/d5c/d92/ddb/d51/d52/fc1 [692973,97014] 0 2024-09-21T04:35:16.269 INFO:tasks.workunit.client.1.smithi053.stdout:0/726: rename d5/d10/d4c/db1/dbc/lb0 to d5/d10/d4c/db1/dbc/dca/d21/d96/lf4 0 2024-09-21T04:35:16.270 INFO:tasks.workunit.client.1.smithi053.stdout:4/844: write d1/dd/de/df5/d20/d73/d10b/d41/fa3 [166578,109208] 0 2024-09-21T04:35:16.270 INFO:tasks.workunit.client.1.smithi053.stdout:5/776: dread d0/d8/f10 [0,4194304] 0 2024-09-21T04:35:16.271 INFO:tasks.workunit.client.0.smithi007.stdout:5/468: creat d2/d27/d3d/d56/f96 x:0 0 0 2024-09-21T04:35:16.271 INFO:tasks.workunit.client.1.smithi053.stdout:5/777: stat d0/dd/d48/fa9 0 2024-09-21T04:35:16.272 INFO:tasks.workunit.client.1.smithi053.stdout:9/725: creat d0/d1c/d52/daa/ffa x:0 0 0 2024-09-21T04:35:16.273 INFO:tasks.workunit.client.1.smithi053.stdout:4/845: creat d1/dd/de/f113 x:0 0 0 2024-09-21T04:35:16.273 INFO:tasks.workunit.client.0.smithi007.stdout:5/469: creat d2/dd/d28/f97 x:0 0 0 2024-09-21T04:35:16.275 INFO:tasks.workunit.client.1.smithi053.stdout:5/778: truncate d0/d8/f10 660059 0 2024-09-21T04:35:16.277 INFO:tasks.workunit.client.1.smithi053.stdout:4/846: creat d1/dd/de/df5/d20/d4e/f114 x:0 0 0 2024-09-21T04:35:16.279 INFO:tasks.workunit.client.1.smithi053.stdout:0/727: dwrite d5/de/d1c/f7e [0,4194304] 0 2024-09-21T04:35:16.281 INFO:tasks.workunit.client.1.smithi053.stdout:5/779: symlink d0/d8/d2a/df5/lfb 0 2024-09-21T04:35:16.282 INFO:tasks.workunit.client.1.smithi053.stdout:9/726: dwrite d0/d2/d9/d1f/f4f [4194304,4194304] 0 2024-09-21T04:35:16.282 INFO:tasks.workunit.client.1.smithi053.stdout:4/847: mkdir d1/dd/de/df5/d20/d5b/d87/dd5/d115 0 2024-09-21T04:35:16.285 INFO:tasks.workunit.client.1.smithi053.stdout:9/727: write d0/d1c/d21/d46/ff9 [203418,64565] 0 2024-09-21T04:35:16.285 INFO:tasks.workunit.client.1.smithi053.stdout:4/848: dread - d1/dd/de/df5/d20/d73/d92/f9e zero size 2024-09-21T04:35:16.286 INFO:tasks.workunit.client.1.smithi053.stdout:5/780: symlink d0/dd/d48/d8c/lfc 0 2024-09-21T04:35:16.287 INFO:tasks.workunit.client.1.smithi053.stdout:9/728: chown d0/d1c/d52/cd4 72 1 2024-09-21T04:35:16.288 INFO:tasks.workunit.client.1.smithi053.stdout:0/728: mkdir d5/d10/d4c/db1/dbc/dca/db5/d99/df5 0 2024-09-21T04:35:16.290 INFO:tasks.workunit.client.0.smithi007.stdout:5/470: write d2/dd/d28/f64 [980421,117734] 0 2024-09-21T04:35:16.296 INFO:tasks.workunit.client.1.smithi053.stdout:0/729: dread d5/de/d42/d78/f90 [0,4194304] 0 2024-09-21T04:35:16.296 INFO:tasks.workunit.client.1.smithi053.stdout:9/729: dwrite d0/d2/d4e/fc6 [0,4194304] 0 2024-09-21T04:35:16.298 INFO:tasks.workunit.client.1.smithi053.stdout:0/730: write d5/de/d42/f48 [272592,87371] 0 2024-09-21T04:35:16.298 INFO:tasks.workunit.client.1.smithi053.stdout:0/731: chown d5/de/d42/d78/c8a 4917028 1 2024-09-21T04:35:16.308 INFO:tasks.workunit.client.1.smithi053.stdout:4/849: creat d1/dd/de/df5/d20/d73/d92/dd7/f116 x:0 0 0 2024-09-21T04:35:16.311 INFO:tasks.workunit.client.0.smithi007.stdout:1/254: sync 2024-09-21T04:35:16.311 INFO:tasks.workunit.client.1.smithi053.stdout:9/730: mkdir d0/d1c/d52/d6f/dfb 0 2024-09-21T04:35:16.312 INFO:tasks.workunit.client.1.smithi053.stdout:9/731: readlink d0/d76/lad 0 2024-09-21T04:35:16.312 INFO:tasks.workunit.client.0.smithi007.stdout:7/243: sync 2024-09-21T04:35:16.312 INFO:tasks.workunit.client.0.smithi007.stdout:6/102: sync 2024-09-21T04:35:16.312 INFO:tasks.workunit.client.1.smithi053.stdout:9/732: dread - d0/d1c/d52/db5/fda zero size 2024-09-21T04:35:16.319 INFO:tasks.workunit.client.0.smithi007.stdout:1/255: dwrite d5/f28 [0,4194304] 0 2024-09-21T04:35:16.320 INFO:tasks.workunit.client.0.smithi007.stdout:7/244: dread d3/d6/d13/f29 [0,4194304] 0 2024-09-21T04:35:16.321 INFO:tasks.workunit.client.0.smithi007.stdout:7/245: write d3/d6/d13/f29 [1452136,5998] 0 2024-09-21T04:35:16.321 INFO:tasks.workunit.client.1.smithi053.stdout:0/732: fsync d5/de/d1c/f7e 0 2024-09-21T04:35:16.324 INFO:tasks.workunit.client.1.smithi053.stdout:0/733: write d5/de/fd7 [415705,97692] 0 2024-09-21T04:35:16.326 INFO:tasks.workunit.client.1.smithi053.stdout:5/781: dread d0/dd/d5c/d92/ddb/d35/d61/fa5 [0,4194304] 0 2024-09-21T04:35:16.331 INFO:tasks.workunit.client.1.smithi053.stdout:0/734: dwrite d5/d10/d4c/db1/dbc/dca/db5/fd6 [0,4194304] 0 2024-09-21T04:35:16.338 INFO:tasks.workunit.client.1.smithi053.stdout:0/735: dwrite d5/de/f67 [4194304,4194304] 0 2024-09-21T04:35:16.383 INFO:tasks.workunit.client.1.smithi053.stdout:6/726: sync 2024-09-21T04:35:16.389 INFO:tasks.workunit.client.1.smithi053.stdout:6/727: dwrite d2/d1b/d1e/d2a/d4b/d77/d88/fca [0,4194304] 0 2024-09-21T04:35:16.395 INFO:tasks.workunit.client.1.smithi053.stdout:6/728: dread d2/fa [0,4194304] 0 2024-09-21T04:35:16.395 INFO:tasks.workunit.client.1.smithi053.stdout:6/729: readlink d2/d1d/dbb/lc6 0 2024-09-21T04:35:16.397 INFO:tasks.workunit.client.1.smithi053.stdout:6/730: write d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f8a [221958,10751] 0 2024-09-21T04:35:16.431 INFO:tasks.workunit.client.0.smithi007.stdout:5/471: rmdir d2/d27/d42/d83 0 2024-09-21T04:35:16.434 INFO:tasks.workunit.client.0.smithi007.stdout:6/103: creat d3/de/d12/d19/f23 x:0 0 0 2024-09-21T04:35:16.438 INFO:tasks.workunit.client.0.smithi007.stdout:7/246: rename d3/d6/d13/f39 to d3/d6/d10/d4f/f5c 0 2024-09-21T04:35:16.440 INFO:tasks.workunit.client.0.smithi007.stdout:5/472: chown d2/dd/d23/d2a/f63 20056396 1 2024-09-21T04:35:16.442 INFO:tasks.workunit.client.1.smithi053.stdout:0/736: symlink d5/d10/d4c/db1/dbc/dca/d21/d96/lf6 0 2024-09-21T04:35:16.444 INFO:tasks.workunit.client.0.smithi007.stdout:6/104: creat d3/de/d12/d19/f24 x:0 0 0 2024-09-21T04:35:16.446 INFO:tasks.workunit.client.0.smithi007.stdout:1/256: mknod d5/d6/d21/c4d 0 2024-09-21T04:35:16.447 INFO:tasks.workunit.client.1.smithi053.stdout:6/731: fsync d2/d1b/f7d 0 2024-09-21T04:35:16.447 INFO:tasks.workunit.client.1.smithi053.stdout:5/782: write d0/dd/f3a [1781497,29450] 0 2024-09-21T04:35:16.450 INFO:tasks.workunit.client.1.smithi053.stdout:9/733: creat d0/d1c/d52/df7/ffc x:0 0 0 2024-09-21T04:35:16.451 INFO:tasks.workunit.client.0.smithi007.stdout:7/247: creat d3/d6/d13/f5d x:0 0 0 2024-09-21T04:35:16.452 INFO:tasks.workunit.client.0.smithi007.stdout:6/105: mknod d3/de/d12/c25 0 2024-09-21T04:35:16.453 INFO:tasks.workunit.client.1.smithi053.stdout:0/737: read d5/de/f2a [6620107,75334] 0 2024-09-21T04:35:16.454 INFO:tasks.workunit.client.1.smithi053.stdout:5/783: rmdir d0/dd/d5c/d80 39 2024-09-21T04:35:16.455 INFO:tasks.workunit.client.0.smithi007.stdout:1/257: write d5/d6/d21/d3c/f43 [374294,18016] 0 2024-09-21T04:35:16.456 INFO:tasks.workunit.client.0.smithi007.stdout:7/248: write d3/d6/f2d [4151118,108722] 0 2024-09-21T04:35:16.456 INFO:tasks.workunit.client.1.smithi053.stdout:6/732: mkdir d2/d1b/d1e/d2a/d4b/deb 0 2024-09-21T04:35:16.457 INFO:tasks.workunit.client.0.smithi007.stdout:1/258: read - d5/d6/d21/f47 zero size 2024-09-21T04:35:16.458 INFO:tasks.workunit.client.1.smithi053.stdout:4/850: getdents d1/dd 0 2024-09-21T04:35:16.461 INFO:tasks.workunit.client.1.smithi053.stdout:5/784: truncate d0/d8/f2b 2261933 0 2024-09-21T04:35:16.461 INFO:tasks.workunit.client.0.smithi007.stdout:6/106: truncate d3/de/d12/d19/f1b 193737 0 2024-09-21T04:35:16.462 INFO:tasks.workunit.client.1.smithi053.stdout:5/785: chown d0/dd/d5c/d92/ddb/d35/d61/fa5 2942377 1 2024-09-21T04:35:16.465 INFO:tasks.workunit.client.1.smithi053.stdout:4/851: dwrite d1/dd/de/df5/d3f/f77 [0,4194304] 0 2024-09-21T04:35:16.465 INFO:tasks.workunit.client.1.smithi053.stdout:4/852: chown d1/dd/de/df5/d20/d5b/d87/dd5/cd8 28 1 2024-09-21T04:35:16.466 INFO:tasks.workunit.client.1.smithi053.stdout:4/853: readlink d1/l5 0 2024-09-21T04:35:16.467 INFO:tasks.workunit.client.1.smithi053.stdout:4/854: dread - d1/dd/de/df5/d20/dee/ff6 zero size 2024-09-21T04:35:16.469 INFO:tasks.workunit.client.1.smithi053.stdout:0/738: write d5/dd/d45/f6c [334245,90171] 0 2024-09-21T04:35:16.475 INFO:tasks.workunit.client.0.smithi007.stdout:7/249: mkdir d3/d6/d2e/d5e 0 2024-09-21T04:35:16.476 INFO:tasks.workunit.client.1.smithi053.stdout:5/786: mkdir d0/dd/d48/d8c/dfd 0 2024-09-21T04:35:16.477 INFO:tasks.workunit.client.0.smithi007.stdout:1/259: truncate d5/d6/d21/d23/f24 3917762 0 2024-09-21T04:35:16.478 INFO:tasks.workunit.client.1.smithi053.stdout:5/787: dread - d0/dd/d5c/d92/ddb/fee zero size 2024-09-21T04:35:16.478 INFO:tasks.workunit.client.1.smithi053.stdout:6/733: dwrite d2/d1b/d3d/d49/f11 [4194304,4194304] 0 2024-09-21T04:35:16.479 INFO:tasks.workunit.client.0.smithi007.stdout:1/260: write d5/d6/d21/f2c [1353750,33403] 0 2024-09-21T04:35:16.482 INFO:tasks.workunit.client.0.smithi007.stdout:7/250: mkdir d3/d6/d14/d20/d33/d5f 0 2024-09-21T04:35:16.483 INFO:tasks.workunit.client.0.smithi007.stdout:7/251: write d3/d6/f7 [5167765,68137] 0 2024-09-21T04:35:16.485 INFO:tasks.workunit.client.1.smithi053.stdout:9/734: getdents d0/d1c/d52/d53 0 2024-09-21T04:35:16.486 INFO:tasks.workunit.client.1.smithi053.stdout:2/785: sync 2024-09-21T04:35:16.489 INFO:tasks.workunit.client.0.smithi007.stdout:7/252: read d3/d6/d14/d20/f3f [4069593,100496] 0 2024-09-21T04:35:16.493 INFO:tasks.workunit.client.1.smithi053.stdout:6/734: symlink d2/d1d/dbb/lec 0 2024-09-21T04:35:16.493 INFO:tasks.workunit.client.1.smithi053.stdout:4/855: dwrite d1/d27/d53/f78 [8388608,4194304] 0 2024-09-21T04:35:16.502 INFO:tasks.workunit.client.1.smithi053.stdout:9/735: creat d0/d2/d4e/d5c/ffd x:0 0 0 2024-09-21T04:35:16.503 INFO:tasks.workunit.client.1.smithi053.stdout:9/736: write d0/d1a/d89/fbb [586071,24247] 0 2024-09-21T04:35:16.503 INFO:tasks.workunit.client.0.smithi007.stdout:9/180: sync 2024-09-21T04:35:16.505 INFO:tasks.workunit.client.0.smithi007.stdout:9/181: write d13/d18/f1f [2743799,25618] 0 2024-09-21T04:35:16.507 INFO:tasks.workunit.client.1.smithi053.stdout:2/786: creat d1/d18/d1f/d27/d58/d5c/d91/dd0/f119 x:0 0 0 2024-09-21T04:35:16.509 INFO:tasks.workunit.client.0.smithi007.stdout:7/253: dread d3/d6/f2f [0,4194304] 0 2024-09-21T04:35:16.511 INFO:tasks.workunit.client.0.smithi007.stdout:1/261: link d5/f1f d5/d45/f4e 0 2024-09-21T04:35:16.517 INFO:tasks.workunit.client.1.smithi053.stdout:6/735: dread d2/d1b/d3d/d49/de/d32/d80/f26 [0,4194304] 0 2024-09-21T04:35:16.519 INFO:tasks.workunit.client.0.smithi007.stdout:7/254: dread d3/d6/d14/f18 [0,4194304] 0 2024-09-21T04:35:16.520 INFO:tasks.workunit.client.1.smithi053.stdout:4/856: dread d1/d27/d2e/d46/fbd [0,4194304] 0 2024-09-21T04:35:16.520 INFO:tasks.workunit.client.0.smithi007.stdout:7/255: readlink d3/d6/d13/l42 0 2024-09-21T04:35:16.522 INFO:tasks.workunit.client.1.smithi053.stdout:0/739: link d5/d10/d4c/db1/dbc/dca/d21/d96/lf4 d5/d10/d4c/db1/dbc/dca/d21/d96/lf7 0 2024-09-21T04:35:16.523 INFO:tasks.workunit.client.0.smithi007.stdout:1/262: symlink d5/d6/d21/d2b/d4b/l4f 0 2024-09-21T04:35:16.524 INFO:tasks.workunit.client.1.smithi053.stdout:6/736: dread d2/d1b/d1e/d2a/d4b/f68 [0,4194304] 0 2024-09-21T04:35:16.525 INFO:tasks.workunit.client.1.smithi053.stdout:9/737: symlink d0/d1a/lfe 0 2024-09-21T04:35:16.526 INFO:tasks.workunit.client.1.smithi053.stdout:5/788: dwrite d0/d8/f2b [0,4194304] 0 2024-09-21T04:35:16.527 INFO:tasks.workunit.client.1.smithi053.stdout:5/789: readlink d0/l12 0 2024-09-21T04:35:16.527 INFO:tasks.workunit.client.0.smithi007.stdout:7/256: unlink d3/d6/cd 0 2024-09-21T04:35:16.528 INFO:tasks.workunit.client.1.smithi053.stdout:5/790: stat d0/dd/d5c/d92/ddb/d51/cc4 0 2024-09-21T04:35:16.528 INFO:tasks.workunit.client.1.smithi053.stdout:4/857: creat d1/dd/de/df5/d20/d73/d10b/f117 x:0 0 0 2024-09-21T04:35:16.529 INFO:tasks.workunit.client.1.smithi053.stdout:7/661: sync 2024-09-21T04:35:16.529 INFO:tasks.workunit.client.1.smithi053.stdout:8/815: sync 2024-09-21T04:35:16.529 INFO:tasks.workunit.client.1.smithi053.stdout:1/779: sync 2024-09-21T04:35:16.529 INFO:tasks.workunit.client.1.smithi053.stdout:3/762: sync 2024-09-21T04:35:16.529 INFO:tasks.workunit.client.1.smithi053.stdout:4/858: stat d1/d27 0 2024-09-21T04:35:16.529 INFO:tasks.workunit.client.0.smithi007.stdout:7/257: write d3/d6/d13/f5d [945121,103460] 0 2024-09-21T04:35:16.530 INFO:tasks.workunit.client.0.smithi007.stdout:1/263: mknod d5/d6/d21/d23/d42/c50 0 2024-09-21T04:35:16.533 INFO:tasks.workunit.client.1.smithi053.stdout:2/787: symlink d1/d18/d1f/d22/d2d/dcd/l11a 0 2024-09-21T04:35:16.535 INFO:tasks.workunit.client.0.smithi007.stdout:9/182: dwrite fa [0,4194304] 0 2024-09-21T04:35:16.535 INFO:tasks.workunit.client.1.smithi053.stdout:9/738: dwrite d0/d1c/d21/d8e/fba [0,4194304] 0 2024-09-21T04:35:16.539 INFO:tasks.workunit.client.0.smithi007.stdout:1/264: rename d5/d6/d21/f3f to d5/d6/d21/d2b/f51 0 2024-09-21T04:35:16.541 INFO:tasks.workunit.client.0.smithi007.stdout:9/183: rmdir d13/d2a 39 2024-09-21T04:35:16.543 INFO:tasks.workunit.client.0.smithi007.stdout:7/258: dwrite d3/d6/d13/d4b/f57 [0,4194304] 0 2024-09-21T04:35:16.545 INFO:tasks.workunit.client.1.smithi053.stdout:0/740: creat d5/de/d9e/dd8/ff8 x:0 0 0 2024-09-21T04:35:16.550 INFO:tasks.workunit.client.1.smithi053.stdout:9/739: dwrite d0/df/f48 [0,4194304] 0 2024-09-21T04:35:16.554 INFO:tasks.workunit.client.0.smithi007.stdout:1/265: rmdir d5/d6/d21/d3c 39 2024-09-21T04:35:16.556 INFO:tasks.workunit.client.1.smithi053.stdout:8/816: write d3/f9c [258211,9857] 0 2024-09-21T04:35:16.556 INFO:tasks.workunit.client.1.smithi053.stdout:1/780: creat d7/d9/da/d40/d49/fff x:0 0 0 2024-09-21T04:35:16.556 INFO:tasks.workunit.client.1.smithi053.stdout:7/662: dwrite d0/d2b/d36/d64/fd1 [0,4194304] 0 2024-09-21T04:35:16.557 INFO:tasks.workunit.client.1.smithi053.stdout:3/763: dread - d16/d17/d21/d8b/d7d/f88 zero size 2024-09-21T04:35:16.557 INFO:tasks.workunit.client.1.smithi053.stdout:8/817: stat d3/d5/d4c/d53/d66/c106 0 2024-09-21T04:35:16.560 INFO:tasks.workunit.client.0.smithi007.stdout:7/259: mknod d3/d6/d14/c60 0 2024-09-21T04:35:16.561 INFO:tasks.workunit.client.1.smithi053.stdout:4/859: creat d1/dd/de/df5/d20/d4e/d51/da4/f118 x:0 0 0 2024-09-21T04:35:16.562 INFO:tasks.workunit.client.1.smithi053.stdout:2/788: fdatasync d1/d18/d1f/f4a 0 2024-09-21T04:35:16.564 INFO:tasks.workunit.client.1.smithi053.stdout:6/737: dread d2/d1b/d1e/fb0 [0,4194304] 0 2024-09-21T04:35:16.566 INFO:tasks.workunit.client.0.smithi007.stdout:7/260: mknod d3/d6/d14/d20/d58/c61 0 2024-09-21T04:35:16.567 INFO:tasks.workunit.client.1.smithi053.stdout:0/741: chown d5/d10/d4c/db1/dbc/dca/f36 2085 1 2024-09-21T04:35:16.571 INFO:tasks.workunit.client.1.smithi053.stdout:3/764: mknod d16/d1b/dad/c120 0 2024-09-21T04:35:16.571 INFO:tasks.workunit.client.1.smithi053.stdout:2/789: dwrite d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/fe1 [0,4194304] 0 2024-09-21T04:35:16.572 INFO:tasks.workunit.client.0.smithi007.stdout:1/266: getdents d5/d6/d21 0 2024-09-21T04:35:16.573 INFO:tasks.workunit.client.0.smithi007.stdout:9/184: write d13/f15 [3171765,113121] 0 2024-09-21T04:35:16.573 INFO:tasks.workunit.client.1.smithi053.stdout:7/663: mkdir d0/d6f/dbf/da2/dd7 0 2024-09-21T04:35:16.575 INFO:tasks.workunit.client.1.smithi053.stdout:8/818: creat d3/d5/d4b/d5c/d86/f108 x:0 0 0 2024-09-21T04:35:16.575 INFO:tasks.workunit.client.1.smithi053.stdout:4/860: creat d1/dd/de/df5/d20/d4e/d51/f119 x:0 0 0 2024-09-21T04:35:16.576 INFO:tasks.workunit.client.1.smithi053.stdout:2/790: dread d1/d18/d1f/d27/d58/d5c/fc0 [0,4194304] 0 2024-09-21T04:35:16.578 INFO:tasks.workunit.client.0.smithi007.stdout:9/185: stat d13/c29 0 2024-09-21T04:35:16.581 INFO:tasks.workunit.client.1.smithi053.stdout:6/738: truncate d2/d1b/d1e/f4f 1960934 0 2024-09-21T04:35:16.581 INFO:tasks.workunit.client.1.smithi053.stdout:9/740: dwrite d0/d1c/d25/f99 [0,4194304] 0 2024-09-21T04:35:16.582 INFO:tasks.workunit.client.0.smithi007.stdout:9/186: write f10 [3216695,59398] 0 2024-09-21T04:35:16.582 INFO:tasks.workunit.client.1.smithi053.stdout:9/741: chown d0/d1c/d21/c51 117 1 2024-09-21T04:35:16.583 INFO:tasks.workunit.client.1.smithi053.stdout:7/664: creat d0/d6f/dbf/d50/d84/db8/fd8 x:0 0 0 2024-09-21T04:35:16.586 INFO:tasks.workunit.client.1.smithi053.stdout:5/791: getdents d0/dd/d5c/d92 0 2024-09-21T04:35:16.586 INFO:tasks.workunit.client.1.smithi053.stdout:7/665: chown d0/d6f/dbf/d50/d8b/d70/fd6 3404980 1 2024-09-21T04:35:16.587 INFO:tasks.workunit.client.1.smithi053.stdout:1/781: link d7/d9/d43/d54/d7f/d59/dc3/fca d7/d9/da/d40/d49/d77/f100 0 2024-09-21T04:35:16.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/267: dread d5/f39 [0,4194304] 0 2024-09-21T04:35:16.591 INFO:tasks.workunit.client.1.smithi053.stdout:8/819: getdents d3/d5/d38/d64/d9d/dff 0 2024-09-21T04:35:16.592 INFO:tasks.workunit.client.0.smithi007.stdout:7/261: dread d3/d6/d14/f45 [4194304,4194304] 0 2024-09-21T04:35:16.592 INFO:tasks.workunit.client.1.smithi053.stdout:4/861: dread d1/dd/de/df5/d20/f75 [0,4194304] 0 2024-09-21T04:35:16.593 INFO:tasks.workunit.client.0.smithi007.stdout:9/187: dwrite d13/d18/f26 [0,4194304] 0 2024-09-21T04:35:16.594 INFO:tasks.workunit.client.1.smithi053.stdout:0/742: creat d5/ff9 x:0 0 0 2024-09-21T04:35:16.595 INFO:tasks.workunit.client.1.smithi053.stdout:1/782: chown d7/d9/d43/d94/ld2 33987790 1 2024-09-21T04:35:16.595 INFO:tasks.workunit.client.0.smithi007.stdout:1/268: symlink d5/d6/d21/d2b/d4b/l52 0 2024-09-21T04:35:16.596 INFO:tasks.workunit.client.1.smithi053.stdout:0/743: fdatasync d5/d10/d4c/ddf/fe6 0 2024-09-21T04:35:16.596 INFO:tasks.workunit.client.0.smithi007.stdout:1/269: dread - d5/d6/d21/f47 zero size 2024-09-21T04:35:16.597 INFO:tasks.workunit.client.0.smithi007.stdout:1/270: chown d5/d6/d21/f30 0 1 2024-09-21T04:35:16.598 INFO:tasks.workunit.client.1.smithi053.stdout:8/820: dwrite d3/f3a [0,4194304] 0 2024-09-21T04:35:16.600 INFO:tasks.workunit.client.1.smithi053.stdout:0/744: write d5/d10/d4c/db1/dbc/fcf [885507,118698] 0 2024-09-21T04:35:16.600 INFO:tasks.workunit.client.1.smithi053.stdout:3/765: rmdir d16/d17/d21/d26/d71/df1 0 2024-09-21T04:35:16.602 INFO:tasks.workunit.client.0.smithi007.stdout:9/188: rename d13/d18/f1f to d13/d2a/f3d 0 2024-09-21T04:35:16.606 INFO:tasks.workunit.client.1.smithi053.stdout:0/745: write d5/ff9 [64802,112536] 0 2024-09-21T04:35:16.609 INFO:tasks.workunit.client.1.smithi053.stdout:3/766: dwrite d16/d17/d21/d8b/d7d/ff3 [0,4194304] 0 2024-09-21T04:35:16.612 INFO:tasks.workunit.client.1.smithi053.stdout:5/792: symlink d0/dd/d5c/d80/lfe 0 2024-09-21T04:35:16.612 INFO:tasks.workunit.client.1.smithi053.stdout:5/793: chown d0/dd/d5c/l5e 33 1 2024-09-21T04:35:16.616 INFO:tasks.workunit.client.0.smithi007.stdout:9/189: symlink d13/l3e 0 2024-09-21T04:35:16.617 INFO:tasks.workunit.client.0.smithi007.stdout:9/190: write f6 [4831927,130947] 0 2024-09-21T04:35:16.621 INFO:tasks.workunit.client.1.smithi053.stdout:7/666: truncate d0/f49 239587 0 2024-09-21T04:35:16.622 INFO:tasks.workunit.client.1.smithi053.stdout:6/739: rmdir d2/d1b/d3d/d49/de/d32/d80/d2e/de7 0 2024-09-21T04:35:16.623 INFO:tasks.workunit.client.0.smithi007.stdout:1/271: getdents d5/d6 0 2024-09-21T04:35:16.623 INFO:tasks.workunit.client.1.smithi053.stdout:8/821: mknod d3/df/d14/d16/dde/c109 0 2024-09-21T04:35:16.625 INFO:tasks.workunit.client.0.smithi007.stdout:9/191: creat d13/d18/f3f x:0 0 0 2024-09-21T04:35:16.626 INFO:tasks.workunit.client.0.smithi007.stdout:9/192: dread - d13/d18/f27 zero size 2024-09-21T04:35:16.632 INFO:tasks.workunit.client.0.smithi007.stdout:9/193: rename f7 to d13/d2a/f40 0 2024-09-21T04:35:16.636 INFO:tasks.workunit.client.0.smithi007.stdout:9/194: symlink d13/d18/l41 0 2024-09-21T04:35:16.640 INFO:tasks.workunit.client.0.smithi007.stdout:9/195: mkdir d13/d2a/d42 0 2024-09-21T04:35:16.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/272: dread d5/d6/d21/d23/f24 [0,4194304] 0 2024-09-21T04:35:16.644 INFO:tasks.workunit.client.0.smithi007.stdout:1/273: mknod d5/c53 0 2024-09-21T04:35:16.648 INFO:tasks.workunit.client.0.smithi007.stdout:1/274: link d5/d6/d21/d23/c32 d5/d6/d21/c54 0 2024-09-21T04:35:16.651 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:16 smithi007 ceph-mon[27954]: pgmap v29: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 66 MiB/s rd, 285 MiB/s wr, 530 op/s 2024-09-21T04:35:16.652 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:16 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:16.652 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:16 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:16.652 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:16 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:35:16.652 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:16 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:35:16.654 INFO:tasks.workunit.client.0.smithi007.stdout:9/196: dwrite d13/d18/f3c [0,4194304] 0 2024-09-21T04:35:16.656 INFO:tasks.workunit.client.0.smithi007.stdout:1/275: dwrite d5/f26 [4194304,4194304] 0 2024-09-21T04:35:16.658 INFO:tasks.workunit.client.0.smithi007.stdout:1/276: chown d5/d45/f46 1 1 2024-09-21T04:35:16.660 INFO:tasks.workunit.client.0.smithi007.stdout:9/197: dread d13/d2a/f32 [0,4194304] 0 2024-09-21T04:35:16.661 INFO:tasks.workunit.client.0.smithi007.stdout:1/277: mkdir d5/d6/d21/d23/d55 0 2024-09-21T04:35:16.662 INFO:tasks.workunit.client.0.smithi007.stdout:9/198: chown d13/d18/c3a 130 1 2024-09-21T04:35:16.665 INFO:tasks.workunit.client.0.smithi007.stdout:1/278: rename d5/d6/d21/d23/d42/f4c to d5/d6/d21/d23/d42/f56 0 2024-09-21T04:35:16.668 INFO:tasks.workunit.client.0.smithi007.stdout:9/199: dwrite d13/d2a/f2d [0,4194304] 0 2024-09-21T04:35:16.673 INFO:tasks.workunit.client.0.smithi007.stdout:9/200: link d13/d18/f3f d13/d18/f43 0 2024-09-21T04:35:16.676 INFO:tasks.workunit.client.0.smithi007.stdout:9/201: dread f10 [4194304,4194304] 0 2024-09-21T04:35:16.677 INFO:tasks.workunit.client.0.smithi007.stdout:1/279: dwrite d5/d6/d21/d23/d42/f56 [0,4194304] 0 2024-09-21T04:35:16.679 INFO:tasks.workunit.client.0.smithi007.stdout:1/280: write d5/d6/d21/f47 [764380,96559] 0 2024-09-21T04:35:16.682 INFO:tasks.workunit.client.0.smithi007.stdout:1/281: unlink d5/d6/d21/d2b/c3b 0 2024-09-21T04:35:16.683 INFO:tasks.workunit.client.0.smithi007.stdout:1/282: rename d5/d6 to d5/d6/d21/d3c/d57 22 2024-09-21T04:35:16.687 INFO:tasks.workunit.client.0.smithi007.stdout:9/202: truncate d13/f20 3920001 0 2024-09-21T04:35:16.688 INFO:tasks.workunit.client.0.smithi007.stdout:1/283: read d5/d6/f8 [2319971,82041] 0 2024-09-21T04:35:16.688 INFO:tasks.workunit.client.1.smithi053.stdout:1/783: symlink d7/d9/d43/d54/d7f/d59/l101 0 2024-09-21T04:35:16.688 INFO:tasks.workunit.client.0.smithi007.stdout:9/203: chown d13/d18/f1a 0 1 2024-09-21T04:35:16.689 INFO:tasks.workunit.client.1.smithi053.stdout:1/784: fdatasync d7/d47/ff2 0 2024-09-21T04:35:16.690 INFO:tasks.workunit.client.1.smithi053.stdout:8/822: write d3/df/d49/fe6 [99254,9399] 0 2024-09-21T04:35:16.690 INFO:tasks.workunit.client.1.smithi053.stdout:4/862: dread d1/dd/de/df5/fb5 [0,4194304] 0 2024-09-21T04:35:16.691 INFO:tasks.workunit.client.1.smithi053.stdout:0/746: write d5/d10/f2b [616585,23045] 0 2024-09-21T04:35:16.691 INFO:tasks.workunit.client.1.smithi053.stdout:7/667: write d0/d6f/dbf/d50/d8b/f81 [400044,87983] 0 2024-09-21T04:35:16.691 INFO:tasks.workunit.client.1.smithi053.stdout:5/794: write d0/dd/d48/d8c/dad/db9/fba [1394568,83108] 0 2024-09-21T04:35:16.692 INFO:tasks.workunit.client.0.smithi007.stdout:9/204: rmdir d13/d2a/d42 0 2024-09-21T04:35:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:16 smithi053 ceph-mon[35554]: pgmap v29: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 66 MiB/s rd, 285 MiB/s wr, 530 op/s 2024-09-21T04:35:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:16 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:16 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:16 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:35:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:16 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:35:16.694 INFO:tasks.workunit.client.1.smithi053.stdout:4/863: truncate d1/dd/fb2 4385709 0 2024-09-21T04:35:16.697 INFO:tasks.workunit.client.1.smithi053.stdout:6/740: getdents d2/d1b/d1e/d27 0 2024-09-21T04:35:16.697 INFO:tasks.workunit.client.1.smithi053.stdout:7/668: truncate d0/d24/f5a 361341 0 2024-09-21T04:35:16.698 INFO:tasks.workunit.client.1.smithi053.stdout:5/795: creat d0/d8/d2a/df5/fff x:0 0 0 2024-09-21T04:35:16.700 INFO:tasks.workunit.client.1.smithi053.stdout:8/823: truncate d3/df/d14/d16/f29 3364340 0 2024-09-21T04:35:16.701 INFO:tasks.workunit.client.1.smithi053.stdout:6/741: chown d2/d1b/d1e/d27/d90/la2 103 1 2024-09-21T04:35:16.701 INFO:tasks.workunit.client.1.smithi053.stdout:5/796: read - d0/dd/d67/fe9 zero size 2024-09-21T04:35:16.701 INFO:tasks.workunit.client.1.smithi053.stdout:1/785: dwrite d7/d9/da/f2f [0,4194304] 0 2024-09-21T04:35:16.703 INFO:tasks.workunit.client.1.smithi053.stdout:8/824: fsync d3/d5/db1/fe4 0 2024-09-21T04:35:16.703 INFO:tasks.workunit.client.1.smithi053.stdout:4/864: creat d1/dd/de/df5/d20/d4e/d51/da4/f11a x:0 0 0 2024-09-21T04:35:16.703 INFO:tasks.workunit.client.1.smithi053.stdout:6/742: write d2/d1b/f53 [3371876,12992] 0 2024-09-21T04:35:16.703 INFO:tasks.workunit.client.0.smithi007.stdout:9/205: dwrite d13/f23 [0,4194304] 0 2024-09-21T04:35:16.703 INFO:tasks.workunit.client.1.smithi053.stdout:0/747: creat d5/de/ffa x:0 0 0 2024-09-21T04:35:16.704 INFO:tasks.workunit.client.1.smithi053.stdout:6/743: chown d2/d1d/f50 258213 1 2024-09-21T04:35:16.706 INFO:tasks.workunit.client.1.smithi053.stdout:4/865: write d1/d27/d53/d88/fb9 [816716,116107] 0 2024-09-21T04:35:16.708 INFO:tasks.workunit.client.0.smithi007.stdout:9/206: rename d13/d18/f26 to d13/d18/f44 0 2024-09-21T04:35:16.709 INFO:tasks.workunit.client.0.smithi007.stdout:9/207: write f6 [2368979,55710] 0 2024-09-21T04:35:16.715 INFO:tasks.workunit.client.1.smithi053.stdout:0/748: mknod d5/d10/d4c/db1/dbc/dca/db5/dd5/cfb 0 2024-09-21T04:35:16.720 INFO:tasks.workunit.client.1.smithi053.stdout:4/866: creat d1/dd/de/df5/d3f/d74/de1/f11b x:0 0 0 2024-09-21T04:35:16.720 INFO:tasks.workunit.client.1.smithi053.stdout:5/797: symlink d0/dd/d5c/d92/ddb/d35/d61/def/l100 0 2024-09-21T04:35:16.720 INFO:tasks.workunit.client.1.smithi053.stdout:8/825: dread d3/d5/d21/f78 [0,4194304] 0 2024-09-21T04:35:16.720 INFO:tasks.workunit.client.0.smithi007.stdout:9/208: truncate d13/d18/f44 1919999 0 2024-09-21T04:35:16.722 INFO:tasks.workunit.client.1.smithi053.stdout:0/749: chown d5/d10/d4c/db1/dbc/dca/d21/d96/lf7 15057660 1 2024-09-21T04:35:16.722 INFO:tasks.workunit.client.1.smithi053.stdout:4/867: chown d1/dd/de/df5/c15 1514 1 2024-09-21T04:35:16.723 INFO:tasks.workunit.client.1.smithi053.stdout:6/744: creat d2/fed x:0 0 0 2024-09-21T04:35:16.723 INFO:tasks.workunit.client.1.smithi053.stdout:9/742: sync 2024-09-21T04:35:16.723 INFO:tasks.workunit.client.1.smithi053.stdout:2/791: sync 2024-09-21T04:35:16.725 INFO:tasks.workunit.client.1.smithi053.stdout:5/798: unlink d0/dd/d5c/d92/ddb/d35/d61/f91 0 2024-09-21T04:35:16.728 INFO:tasks.workunit.client.1.smithi053.stdout:0/750: mkdir d5/de/dfc 0 2024-09-21T04:35:16.730 INFO:tasks.workunit.client.1.smithi053.stdout:2/792: dread d1/d18/d1f/f8a [0,4194304] 0 2024-09-21T04:35:16.731 INFO:tasks.workunit.client.1.smithi053.stdout:4/868: creat d1/dd/de/df5/d20/d73/d10b/d41/f11c x:0 0 0 2024-09-21T04:35:16.732 INFO:tasks.workunit.client.1.smithi053.stdout:6/745: dread d2/d1b/d3d/d49/d5c/f5d [0,4194304] 0 2024-09-21T04:35:16.732 INFO:tasks.workunit.client.1.smithi053.stdout:7/669: truncate d0/d6f/dbf/d5c/f8a 86091 0 2024-09-21T04:35:16.732 INFO:tasks.workunit.client.1.smithi053.stdout:8/826: dwrite d3/d5/d38/d64/d9d/da9/ff1 [0,4194304] 0 2024-09-21T04:35:16.732 INFO:tasks.workunit.client.0.smithi007.stdout:9/209: unlink d13/f20 0 2024-09-21T04:35:16.738 INFO:tasks.workunit.client.1.smithi053.stdout:7/670: truncate d0/d6f/dbf/d5c/f85 475628 0 2024-09-21T04:35:16.738 INFO:tasks.workunit.client.0.smithi007.stdout:9/210: creat d13/d18/f45 x:0 0 0 2024-09-21T04:35:16.738 INFO:tasks.workunit.client.0.smithi007.stdout:9/211: chown d13/d2a/c2b 3225 1 2024-09-21T04:35:16.739 INFO:tasks.workunit.client.1.smithi053.stdout:7/671: write d0/d2b/d36/d64/fb5 [818839,104655] 0 2024-09-21T04:35:16.739 INFO:tasks.workunit.client.1.smithi053.stdout:9/743: write d0/d2/d9/f7b [2577215,121866] 0 2024-09-21T04:35:16.742 INFO:tasks.workunit.client.1.smithi053.stdout:2/793: rmdir d1/d18/d1f/d22/d2d/d106 39 2024-09-21T04:35:16.742 INFO:tasks.workunit.client.1.smithi053.stdout:0/751: dwrite d5/d10/d4c/db1/dbc/dca/d21/d95/f9b [0,4194304] 0 2024-09-21T04:35:16.742 INFO:tasks.workunit.client.1.smithi053.stdout:7/672: chown d0/d6f/dbf/d5d/l7c 275 1 2024-09-21T04:35:16.744 INFO:tasks.workunit.client.1.smithi053.stdout:7/673: readlink d0/d6f/dbf/d50/l51 0 2024-09-21T04:35:16.744 INFO:tasks.workunit.client.0.smithi007.stdout:9/212: write d13/f1d [145759,75521] 0 2024-09-21T04:35:16.745 INFO:tasks.workunit.client.1.smithi053.stdout:2/794: fsync d1/d18/d1f/d27/d58/d5c/fd4 0 2024-09-21T04:35:16.747 INFO:tasks.workunit.client.1.smithi053.stdout:8/827: mkdir d3/df/d14/d16/d10a 0 2024-09-21T04:35:16.747 INFO:tasks.workunit.client.1.smithi053.stdout:7/674: write d0/d6f/dbf/d50/d8b/f1c [1015919,96516] 0 2024-09-21T04:35:16.748 INFO:tasks.workunit.client.1.smithi053.stdout:0/752: truncate d5/d10/d4c/db1/dbc/dca/d21/fc9 970435 0 2024-09-21T04:35:16.749 INFO:tasks.workunit.client.0.smithi007.stdout:9/213: mknod d13/d2a/c46 0 2024-09-21T04:35:16.751 INFO:tasks.workunit.client.1.smithi053.stdout:6/746: symlink d2/d1b/d1e/d2a/d4b/d77/lee 0 2024-09-21T04:35:16.753 INFO:tasks.workunit.client.1.smithi053.stdout:5/799: mknod d0/dd/d48/d8c/dfd/c101 0 2024-09-21T04:35:16.755 INFO:tasks.workunit.client.1.smithi053.stdout:5/800: write d0/fa4 [1479622,89340] 0 2024-09-21T04:35:16.756 INFO:tasks.workunit.client.1.smithi053.stdout:0/753: dwrite d5/de/d42/fa4 [0,4194304] 0 2024-09-21T04:35:16.756 INFO:tasks.workunit.client.1.smithi053.stdout:4/869: read d1/d27/f89 [8282782,53723] 0 2024-09-21T04:35:16.757 INFO:tasks.workunit.client.0.smithi007.stdout:9/214: dwrite d13/d2a/f31 [0,4194304] 0 2024-09-21T04:35:16.762 INFO:tasks.workunit.client.0.smithi007.stdout:9/215: rename d13/d18/f43 to d13/d2a/f47 0 2024-09-21T04:35:16.766 INFO:tasks.workunit.client.1.smithi053.stdout:6/747: dwrite d2/d1b/d1e/d2a/d4b/d77/d88/da4/fdf [0,4194304] 0 2024-09-21T04:35:16.768 INFO:tasks.workunit.client.1.smithi053.stdout:6/748: chown d2/d1b/d3d/d49/de/l69 4134846 1 2024-09-21T04:35:16.769 INFO:tasks.workunit.client.0.smithi007.stdout:9/216: mknod d13/c48 0 2024-09-21T04:35:16.771 INFO:tasks.workunit.client.1.smithi053.stdout:5/801: creat d0/d8/db/d68/f102 x:0 0 0 2024-09-21T04:35:16.773 INFO:tasks.workunit.client.1.smithi053.stdout:9/744: link d0/d2/d2b/le4 d0/d6c/lff 0 2024-09-21T04:35:16.775 INFO:tasks.workunit.client.0.smithi007.stdout:9/217: dread d13/d2a/f32 [0,4194304] 0 2024-09-21T04:35:16.778 INFO:tasks.workunit.client.1.smithi053.stdout:2/795: dread d1/d18/d1f/d22/d54/fb0 [0,4194304] 0 2024-09-21T04:35:16.780 INFO:tasks.workunit.client.1.smithi053.stdout:5/802: symlink d0/dd/d5c/d92/de8/l103 0 2024-09-21T04:35:16.781 INFO:tasks.workunit.client.0.smithi007.stdout:9/218: dwrite d13/f15 [0,4194304] 0 2024-09-21T04:35:16.781 INFO:tasks.workunit.client.1.smithi053.stdout:9/745: creat d0/d6c/f100 x:0 0 0 2024-09-21T04:35:16.782 INFO:tasks.workunit.client.1.smithi053.stdout:6/749: dread d2/d1b/d1e/d2a/d4b/f81 [0,4194304] 0 2024-09-21T04:35:16.783 INFO:tasks.workunit.client.1.smithi053.stdout:0/754: creat d5/d10/d4c/db1/dbc/dca/db5/d99/df5/ffd x:0 0 0 2024-09-21T04:35:16.784 INFO:tasks.workunit.client.1.smithi053.stdout:8/828: getdents d3/df/d14/d16 0 2024-09-21T04:35:16.785 INFO:tasks.workunit.client.1.smithi053.stdout:0/755: truncate d5/dd/d50/f9d 425229 0 2024-09-21T04:35:16.785 INFO:tasks.workunit.client.1.smithi053.stdout:8/829: write d3/d5/d38/f4e [841545,95536] 0 2024-09-21T04:35:16.787 INFO:tasks.workunit.client.1.smithi053.stdout:7/675: getdents d0 0 2024-09-21T04:35:16.787 INFO:tasks.workunit.client.1.smithi053.stdout:4/870: rmdir d1/dd/de/df5/d20/d5b/d87/d112 0 2024-09-21T04:35:16.788 INFO:tasks.workunit.client.1.smithi053.stdout:8/830: fsync d3/df/d14/fd8 0 2024-09-21T04:35:16.789 INFO:tasks.workunit.client.0.smithi007.stdout:9/219: dwrite d13/d18/f34 [0,4194304] 0 2024-09-21T04:35:16.790 INFO:tasks.workunit.client.1.smithi053.stdout:7/676: chown d0/d6f/dbf/d5d/f8c 0 1 2024-09-21T04:35:16.793 INFO:tasks.workunit.client.1.smithi053.stdout:0/756: dread d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/fd9 [0,4194304] 0 2024-09-21T04:35:16.795 INFO:tasks.workunit.client.0.smithi007.stdout:9/220: rename cf to d13/d2a/c49 0 2024-09-21T04:35:16.802 INFO:tasks.workunit.client.1.smithi053.stdout:7/677: rename d0/d6f/fd2 to d0/d24/fd9 0 2024-09-21T04:35:16.802 INFO:tasks.workunit.client.1.smithi053.stdout:2/796: dread d1/d18/d1f/d9d/fa8 [0,4194304] 0 2024-09-21T04:35:16.803 INFO:tasks.workunit.client.1.smithi053.stdout:5/803: dread d0/dd/d5c/d92/ddb/d35/f64 [0,4194304] 0 2024-09-21T04:35:16.803 INFO:tasks.workunit.client.1.smithi053.stdout:6/750: creat d2/d1b/d3d/d49/de/fef x:0 0 0 2024-09-21T04:35:16.804 INFO:tasks.workunit.client.1.smithi053.stdout:4/871: dread d1/dd/de/df5/d20/d4e/d51/da4/df1/f85 [0,4194304] 0 2024-09-21T04:35:16.804 INFO:tasks.workunit.client.1.smithi053.stdout:9/746: dread d0/d1a/d4c/f86 [0,4194304] 0 2024-09-21T04:35:16.805 INFO:tasks.workunit.client.0.smithi007.stdout:9/221: getdents d13 0 2024-09-21T04:35:16.805 INFO:tasks.workunit.client.1.smithi053.stdout:6/751: fdatasync d2/d1b/d3d/d49/de/d32/d80/f57 0 2024-09-21T04:35:16.807 INFO:tasks.workunit.client.0.smithi007.stdout:9/222: symlink d13/d2a/l4a 0 2024-09-21T04:35:16.810 INFO:tasks.workunit.client.1.smithi053.stdout:8/831: dwrite d3/df/d10/f90 [0,4194304] 0 2024-09-21T04:35:16.812 INFO:tasks.workunit.client.1.smithi053.stdout:8/832: write d3/df/dba/fca [912734,21938] 0 2024-09-21T04:35:16.822 INFO:tasks.workunit.client.1.smithi053.stdout:3/767: sync 2024-09-21T04:35:16.825 INFO:tasks.workunit.client.0.smithi007.stdout:0/180: sync 2024-09-21T04:35:16.827 INFO:tasks.workunit.client.0.smithi007.stdout:9/223: write d13/d18/f3f [884375,48963] 0 2024-09-21T04:35:16.828 INFO:tasks.workunit.client.0.smithi007.stdout:9/224: chown d13/d18/l2e 14 1 2024-09-21T04:35:16.829 INFO:tasks.workunit.client.0.smithi007.stdout:9/225: read d13/d2a/f31 [459561,73793] 0 2024-09-21T04:35:16.829 INFO:tasks.workunit.client.1.smithi053.stdout:0/757: symlink d5/de/dfc/lfe 0 2024-09-21T04:35:16.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/181: dread d6/d16/f2a [0,4194304] 0 2024-09-21T04:35:16.831 INFO:tasks.workunit.client.0.smithi007.stdout:9/226: truncate d13/d18/f3f 1204559 0 2024-09-21T04:35:16.832 INFO:tasks.workunit.client.1.smithi053.stdout:2/797: mknod d1/c11b 0 2024-09-21T04:35:16.834 INFO:tasks.workunit.client.1.smithi053.stdout:7/678: mknod d0/d6f/dbf/d5c/cda 0 2024-09-21T04:35:16.834 INFO:tasks.workunit.client.0.smithi007.stdout:0/182: symlink d6/d16/d18/l45 0 2024-09-21T04:35:16.835 INFO:tasks.workunit.client.1.smithi053.stdout:9/747: rmdir d0/d1c 39 2024-09-21T04:35:16.836 INFO:tasks.workunit.client.0.smithi007.stdout:9/227: write d13/d2a/f3d [2780004,106399] 0 2024-09-21T04:35:16.839 INFO:tasks.workunit.client.1.smithi053.stdout:2/798: dwrite d1/d18/d1f/da2/dac/fc5 [0,4194304] 0 2024-09-21T04:35:16.839 INFO:tasks.workunit.client.1.smithi053.stdout:4/872: rename d1/dd/de/df5/d20/f6f to d1/d27/d90/f11d 0 2024-09-21T04:35:16.840 INFO:tasks.workunit.client.0.smithi007.stdout:0/183: symlink d6/d16/d18/d2e/d3f/l46 0 2024-09-21T04:35:16.841 INFO:tasks.workunit.client.1.smithi053.stdout:6/752: symlink d2/d1b/d3d/lf0 0 2024-09-21T04:35:16.842 INFO:tasks.workunit.client.1.smithi053.stdout:7/679: dwrite d0/d6f/dbf/d50/d8b/f42 [0,4194304] 0 2024-09-21T04:35:16.847 INFO:tasks.workunit.client.1.smithi053.stdout:6/753: read d2/d1b/d1e/d2a/d4b/f58 [329765,103456] 0 2024-09-21T04:35:16.847 INFO:tasks.workunit.client.0.smithi007.stdout:0/184: dwrite d6/d16/f28 [0,4194304] 0 2024-09-21T04:35:16.852 INFO:tasks.workunit.client.0.smithi007.stdout:0/185: rmdir d6/d16/d18 39 2024-09-21T04:35:16.860 INFO:tasks.workunit.client.1.smithi053.stdout:2/799: creat d1/d18/d1f/d27/f11c x:0 0 0 2024-09-21T04:35:16.861 INFO:tasks.workunit.client.1.smithi053.stdout:2/800: stat d1/d18/c47 0 2024-09-21T04:35:16.861 INFO:tasks.workunit.client.1.smithi053.stdout:2/801: read - d1/d18/d1f/d27/f11c zero size 2024-09-21T04:35:16.862 INFO:tasks.workunit.client.1.smithi053.stdout:7/680: rename d0/d6e to d0/d2b/d36/d64/ddb 0 2024-09-21T04:35:16.864 INFO:tasks.workunit.client.1.smithi053.stdout:6/754: creat d2/d1b/d1e/d27/d41/dc9/ff1 x:0 0 0 2024-09-21T04:35:16.866 INFO:tasks.workunit.client.1.smithi053.stdout:6/755: chown d2/d1b/d3d/d49/de/d96/ca7 418840 1 2024-09-21T04:35:16.867 INFO:tasks.workunit.client.1.smithi053.stdout:2/802: dwrite d1/d18/f49 [4194304,4194304] 0 2024-09-21T04:35:16.868 INFO:tasks.workunit.client.1.smithi053.stdout:4/873: mknod d1/c11e 0 2024-09-21T04:35:16.869 INFO:tasks.workunit.client.1.smithi053.stdout:2/803: chown d1/d18/d1f/d27/f55 6 1 2024-09-21T04:35:16.870 INFO:tasks.workunit.client.0.smithi007.stdout:9/228: truncate d13/d18/f3c 2703242 0 2024-09-21T04:35:16.872 INFO:tasks.workunit.client.1.smithi053.stdout:3/768: write d16/d17/d21/f39 [1252415,107891] 0 2024-09-21T04:35:16.872 INFO:tasks.workunit.client.1.smithi053.stdout:6/756: rename d2/d1b/d3d/d49/dad to d2/d1d/de0/df2 0 2024-09-21T04:35:16.873 INFO:tasks.workunit.client.0.smithi007.stdout:0/186: truncate f1 2252519 0 2024-09-21T04:35:16.875 INFO:tasks.workunit.client.0.smithi007.stdout:9/229: creat d13/d18/f4b x:0 0 0 2024-09-21T04:35:16.875 INFO:tasks.workunit.client.1.smithi053.stdout:4/874: dread d1/dd/de/df5/f111 [0,4194304] 0 2024-09-21T04:35:16.876 INFO:tasks.workunit.client.0.smithi007.stdout:9/230: stat d13/c29 0 2024-09-21T04:35:16.876 INFO:tasks.workunit.client.1.smithi053.stdout:2/804: rmdir d1/d18/d1f/d27/d58/d5c/d91/dc1 39 2024-09-21T04:35:16.877 INFO:tasks.workunit.client.1.smithi053.stdout:2/805: chown d1/d18/f5b 9946 1 2024-09-21T04:35:16.879 INFO:tasks.workunit.client.1.smithi053.stdout:0/758: getdents d5/d10/d4c/db1/dbc/dca/d21 0 2024-09-21T04:35:16.880 INFO:tasks.workunit.client.1.smithi053.stdout:8/833: write d3/d5/d38/f3d [1341881,105572] 0 2024-09-21T04:35:16.880 INFO:tasks.workunit.client.0.smithi007.stdout:9/231: fsync d13/d18/f35 0 2024-09-21T04:35:16.881 INFO:tasks.workunit.client.1.smithi053.stdout:9/748: write d0/d2/d4e/d5c/f78 [817057,122548] 0 2024-09-21T04:35:16.881 INFO:tasks.workunit.client.1.smithi053.stdout:0/759: readlink d5/d10/d4c/db1/dbc/dca/db5/dd5/le8 0 2024-09-21T04:35:16.883 INFO:tasks.workunit.client.1.smithi053.stdout:6/757: mkdir d2/d1b/d1e/d27/df3 0 2024-09-21T04:35:16.884 INFO:tasks.workunit.client.1.smithi053.stdout:4/875: mkdir d1/dd/de/d11f 0 2024-09-21T04:35:16.885 INFO:tasks.workunit.client.1.smithi053.stdout:8/834: fsync d3/d5/d38/f3d 0 2024-09-21T04:35:16.886 INFO:tasks.workunit.client.0.smithi007.stdout:9/232: rename c12 to d13/c4c 0 2024-09-21T04:35:16.887 INFO:tasks.workunit.client.1.smithi053.stdout:7/681: chown d0/d24/d46/f4a 376606 1 2024-09-21T04:35:16.887 INFO:tasks.workunit.client.1.smithi053.stdout:0/760: rmdir d5/de/d42/d78 39 2024-09-21T04:35:16.890 INFO:tasks.workunit.client.1.smithi053.stdout:9/749: rename d0/f4b to d0/d1a/d89/da1/f101 0 2024-09-21T04:35:16.891 INFO:tasks.workunit.client.0.smithi007.stdout:0/187: dread d6/fd [0,4194304] 0 2024-09-21T04:35:16.892 INFO:tasks.workunit.client.1.smithi053.stdout:4/876: stat d1/c3d 0 2024-09-21T04:35:16.896 INFO:tasks.workunit.client.0.smithi007.stdout:0/188: write d6/d16/d18/d27/f35 [724626,124370] 0 2024-09-21T04:35:16.897 INFO:tasks.workunit.client.1.smithi053.stdout:8/835: fdatasync d3/d5/d38/f47 0 2024-09-21T04:35:16.899 INFO:tasks.workunit.client.1.smithi053.stdout:3/769: creat d16/d1b/dc3/f121 x:0 0 0 2024-09-21T04:35:16.900 INFO:tasks.workunit.client.1.smithi053.stdout:6/758: dwrite d2/d1d/f7e [0,4194304] 0 2024-09-21T04:35:16.903 INFO:tasks.workunit.client.0.smithi007.stdout:0/189: mkdir d6/d16/d47 0 2024-09-21T04:35:16.904 INFO:tasks.workunit.client.1.smithi053.stdout:7/682: dread d0/f13 [0,4194304] 0 2024-09-21T04:35:16.907 INFO:tasks.workunit.client.1.smithi053.stdout:1/786: sync 2024-09-21T04:35:16.907 INFO:tasks.workunit.client.1.smithi053.stdout:2/806: dwrite d1/f5 [4194304,4194304] 0 2024-09-21T04:35:16.913 INFO:tasks.workunit.client.1.smithi053.stdout:8/836: creat d3/d5/d4c/d53/d66/f10b x:0 0 0 2024-09-21T04:35:16.915 INFO:tasks.workunit.client.0.smithi007.stdout:9/233: dwrite d13/d18/f4b [0,4194304] 0 2024-09-21T04:35:16.915 INFO:tasks.workunit.client.1.smithi053.stdout:3/770: dwrite d16/d17/d21/d60/fd6 [0,4194304] 0 2024-09-21T04:35:16.915 INFO:tasks.workunit.client.1.smithi053.stdout:1/787: chown d7/d9/d43/d54/d7f/d59/l101 3552001 1 2024-09-21T04:35:16.917 INFO:tasks.workunit.client.1.smithi053.stdout:0/761: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/fd9 [0,4194304] 0 2024-09-21T04:35:16.919 INFO:tasks.workunit.client.0.smithi007.stdout:0/190: dwrite d6/d16/d18/d39/f3b [0,4194304] 0 2024-09-21T04:35:16.921 INFO:tasks.workunit.client.1.smithi053.stdout:7/683: dwrite d0/f13 [0,4194304] 0 2024-09-21T04:35:16.921 INFO:tasks.workunit.client.1.smithi053.stdout:8/837: chown d3/d5/d4b/d5c/c71 0 1 2024-09-21T04:35:16.925 INFO:tasks.workunit.client.1.smithi053.stdout:6/759: mknod d2/d1b/d3d/d49/d5c/d8d/cf4 0 2024-09-21T04:35:16.926 INFO:tasks.workunit.client.1.smithi053.stdout:4/877: symlink d1/dd/de/df5/l120 0 2024-09-21T04:35:16.931 INFO:tasks.workunit.client.1.smithi053.stdout:5/804: sync 2024-09-21T04:35:16.933 INFO:tasks.workunit.client.1.smithi053.stdout:9/750: rename d0/d1c/c9d to d0/d2/ddf/c102 0 2024-09-21T04:35:16.933 INFO:tasks.workunit.client.1.smithi053.stdout:1/788: dwrite d7/d9/da/f2f [0,4194304] 0 2024-09-21T04:35:16.935 INFO:tasks.workunit.client.1.smithi053.stdout:2/807: mknod d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/c11d 0 2024-09-21T04:35:16.937 INFO:tasks.workunit.client.0.smithi007.stdout:9/234: dwrite d13/d2a/f40 [0,4194304] 0 2024-09-21T04:35:16.938 INFO:tasks.workunit.client.0.smithi007.stdout:9/235: chown d13/d18/l41 18340 1 2024-09-21T04:35:16.939 INFO:tasks.workunit.client.0.smithi007.stdout:9/236: rename d13/d18 to d13/d18/d4d 22 2024-09-21T04:35:16.942 INFO:tasks.workunit.client.1.smithi053.stdout:0/762: readlink d5/d10/d4c/db1/dbc/dca/d53/l8b 0 2024-09-21T04:35:16.942 INFO:tasks.workunit.client.0.smithi007.stdout:0/191: link d6/d16/d18/d2e/c31 d6/d16/d18/d27/c48 0 2024-09-21T04:35:16.947 INFO:tasks.workunit.client.0.smithi007.stdout:9/237: write d13/f15 [2691188,43497] 0 2024-09-21T04:35:16.950 INFO:tasks.workunit.client.1.smithi053.stdout:4/878: rename d1/dd/de/df5/d20/d4e/d51/da4/df1/d8e to d1/dd/de/df5/d20/d5b/d87/dd5/d115/d121 0 2024-09-21T04:35:16.952 INFO:tasks.workunit.client.1.smithi053.stdout:6/760: truncate d2/d1d/de0/f94 576667 0 2024-09-21T04:35:16.954 INFO:tasks.workunit.client.0.smithi007.stdout:0/192: truncate d6/d16/f25 268731 0 2024-09-21T04:35:16.955 INFO:tasks.workunit.client.1.smithi053.stdout:9/751: symlink d0/d2/d9/d34/d58/l103 0 2024-09-21T04:35:16.956 INFO:tasks.workunit.client.1.smithi053.stdout:2/808: unlink d1/d18/d1f/f4a 0 2024-09-21T04:35:16.956 INFO:tasks.workunit.client.1.smithi053.stdout:8/838: dread d3/df/d14/d16/f98 [0,4194304] 0 2024-09-21T04:35:16.959 INFO:tasks.workunit.client.1.smithi053.stdout:7/684: symlink d0/d6f/dbf/d50/d84/db8/ldc 0 2024-09-21T04:35:16.960 INFO:tasks.workunit.client.1.smithi053.stdout:3/771: dread d16/d1b/f2f [0,4194304] 0 2024-09-21T04:35:16.961 INFO:tasks.workunit.client.0.smithi007.stdout:0/193: dwrite d6/d16/d18/d39/f3b [4194304,4194304] 0 2024-09-21T04:35:16.962 INFO:tasks.workunit.client.1.smithi053.stdout:0/763: fdatasync d5/f15 0 2024-09-21T04:35:16.964 INFO:tasks.workunit.client.0.smithi007.stdout:9/238: rename d13/d18/f44 to d13/f4e 0 2024-09-21T04:35:16.964 INFO:tasks.workunit.client.1.smithi053.stdout:1/789: dwrite d7/d9/da/f86 [0,4194304] 0 2024-09-21T04:35:16.964 INFO:tasks.workunit.client.1.smithi053.stdout:3/772: chown d16/d17/f10d 146417 1 2024-09-21T04:35:16.966 INFO:tasks.workunit.client.0.smithi007.stdout:0/194: unlink d6/c1c 0 2024-09-21T04:35:16.973 INFO:tasks.workunit.client.1.smithi053.stdout:8/839: creat d3/d5/db1/f10c x:0 0 0 2024-09-21T04:35:16.974 INFO:tasks.workunit.client.1.smithi053.stdout:2/809: dwrite d1/d18/d1f/d27/d41/d4b/d4d/f88 [0,4194304] 0 2024-09-21T04:35:16.974 INFO:tasks.workunit.client.1.smithi053.stdout:7/685: mkdir d0/d24/d46/ddd 0 2024-09-21T04:35:16.974 INFO:tasks.workunit.client.1.smithi053.stdout:8/840: write d3/d5/d4b/d5c/d86/f108 [765539,31741] 0 2024-09-21T04:35:16.979 INFO:tasks.workunit.client.1.smithi053.stdout:5/805: creat d0/dd/f104 x:0 0 0 2024-09-21T04:35:16.979 INFO:tasks.workunit.client.1.smithi053.stdout:2/810: dread d1/d18/d1f/d27/fa0 [0,4194304] 0 2024-09-21T04:35:16.980 INFO:tasks.workunit.client.0.smithi007.stdout:9/239: creat d13/f4f x:0 0 0 2024-09-21T04:35:16.983 INFO:tasks.workunit.client.1.smithi053.stdout:6/761: mkdir d2/d1b/d1e/d2a/d4b/deb/df5 0 2024-09-21T04:35:16.985 INFO:tasks.workunit.client.1.smithi053.stdout:0/764: mknod d5/de/d9e/dd2/cff 0 2024-09-21T04:35:16.985 INFO:tasks.workunit.client.0.smithi007.stdout:9/240: chown d13/l17 842 1 2024-09-21T04:35:16.986 INFO:tasks.workunit.client.0.smithi007.stdout:0/195: getdents d6/d16/d18/d39 0 2024-09-21T04:35:16.987 INFO:tasks.workunit.client.1.smithi053.stdout:3/773: creat d16/d17/d21/d8b/d7d/d57/f122 x:0 0 0 2024-09-21T04:35:16.988 INFO:tasks.workunit.client.1.smithi053.stdout:3/774: write d16/d17/d21/d42/d43/d86/fb3 [5040001,14095] 0 2024-09-21T04:35:16.989 INFO:tasks.workunit.client.1.smithi053.stdout:9/752: rmdir d0/d76/d79 39 2024-09-21T04:35:16.993 INFO:tasks.workunit.client.0.smithi007.stdout:9/241: unlink d13/d18/f25 0 2024-09-21T04:35:16.994 INFO:tasks.workunit.client.0.smithi007.stdout:0/196: getdents d6/d16/d47 0 2024-09-21T04:35:16.995 INFO:tasks.workunit.client.0.smithi007.stdout:0/197: read d6/d16/d18/d39/f3b [493954,76893] 0 2024-09-21T04:35:16.995 INFO:tasks.workunit.client.1.smithi053.stdout:7/686: rename d0/d6f/dbf/d50/c57 to d0/d6f/dbf/d5c/cde 0 2024-09-21T04:35:16.996 INFO:tasks.workunit.client.0.smithi007.stdout:0/198: chown d6/d16/d18/d27/f33 35661 1 2024-09-21T04:35:16.997 INFO:tasks.workunit.client.0.smithi007.stdout:9/242: fdatasync d13/d2a/f2d 0 2024-09-21T04:35:16.998 INFO:tasks.workunit.client.1.smithi053.stdout:5/806: creat d0/dd/d5c/d92/de8/f105 x:0 0 0 2024-09-21T04:35:16.999 INFO:tasks.workunit.client.1.smithi053.stdout:4/879: dwrite d1/d27/fce [0,4194304] 0 2024-09-21T04:35:17.000 INFO:tasks.workunit.client.1.smithi053.stdout:0/765: fsync d5/de/d1c/f29 0 2024-09-21T04:35:17.001 INFO:tasks.workunit.client.1.smithi053.stdout:1/790: mkdir d7/d9/da/d40/d102 0 2024-09-21T04:35:17.002 INFO:tasks.workunit.client.1.smithi053.stdout:8/841: dread d3/df/d14/d16/f23 [0,4194304] 0 2024-09-21T04:35:17.004 INFO:tasks.workunit.client.1.smithi053.stdout:5/807: dread d0/dd/fcd [0,4194304] 0 2024-09-21T04:35:17.005 INFO:tasks.workunit.client.0.smithi007.stdout:0/199: mknod d6/d16/d47/c49 0 2024-09-21T04:35:17.006 INFO:tasks.workunit.client.1.smithi053.stdout:9/753: stat d0/d2/d2b/c3b 0 2024-09-21T04:35:17.010 INFO:tasks.workunit.client.0.smithi007.stdout:0/200: read d6/d16/f25 [1739,113740] 0 2024-09-21T04:35:17.012 INFO:tasks.workunit.client.1.smithi053.stdout:1/791: dread d7/d9/d43/d94/fd1 [0,4194304] 0 2024-09-21T04:35:17.013 INFO:tasks.workunit.client.0.smithi007.stdout:9/243: dread d13/f15 [0,4194304] 0 2024-09-21T04:35:17.014 INFO:tasks.workunit.client.1.smithi053.stdout:0/766: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f91 [0,4194304] 0 2024-09-21T04:35:17.014 INFO:tasks.workunit.client.1.smithi053.stdout:5/808: dwrite d0/d8/d2a/f57 [0,4194304] 0 2024-09-21T04:35:17.016 INFO:tasks.workunit.client.1.smithi053.stdout:4/880: dread - d1/dd/de/df5/d20/d4e/d51/fab zero size 2024-09-21T04:35:17.017 INFO:tasks.workunit.client.0.smithi007.stdout:0/201: dwrite d6/d16/d18/d27/f35 [0,4194304] 0 2024-09-21T04:35:17.018 INFO:tasks.workunit.client.1.smithi053.stdout:4/881: chown d1/dd/de/df5/f105 65 1 2024-09-21T04:35:17.020 INFO:tasks.workunit.client.1.smithi053.stdout:3/775: link d16/d17/faf d16/d17/d3c/f123 0 2024-09-21T04:35:17.021 INFO:tasks.workunit.client.0.smithi007.stdout:9/244: mknod d13/c50 0 2024-09-21T04:35:17.022 INFO:tasks.workunit.client.1.smithi053.stdout:7/687: write d0/d6f/dbf/d50/d8b/f31 [527959,39466] 0 2024-09-21T04:35:17.024 INFO:tasks.workunit.client.0.smithi007.stdout:0/202: creat d6/d16/d18/d2e/f4a x:0 0 0 2024-09-21T04:35:17.024 INFO:tasks.workunit.client.1.smithi053.stdout:3/776: fdatasync d16/d17/d21/d42/d43/d86/f116 0 2024-09-21T04:35:17.024 INFO:tasks.workunit.client.1.smithi053.stdout:7/688: dread - d0/d6f/dbf/d5c/dac/fce zero size 2024-09-21T04:35:17.025 INFO:tasks.workunit.client.0.smithi007.stdout:0/203: readlink d6/d16/d18/l1f 0 2024-09-21T04:35:17.025 INFO:tasks.workunit.client.1.smithi053.stdout:8/842: creat d3/d5/d4b/d5c/f10d x:0 0 0 2024-09-21T04:35:17.027 INFO:tasks.workunit.client.0.smithi007.stdout:9/245: dread d13/d2a/f40 [0,4194304] 0 2024-09-21T04:35:17.029 INFO:tasks.workunit.client.0.smithi007.stdout:0/204: creat d6/d16/d47/f4b x:0 0 0 2024-09-21T04:35:17.030 INFO:tasks.workunit.client.1.smithi053.stdout:2/811: rename d1/d18/d1f/d22/l8e to d1/d18/d1f/d9d/l11e 0 2024-09-21T04:35:17.033 INFO:tasks.workunit.client.1.smithi053.stdout:1/792: rmdir d7/d9/da/d3c 39 2024-09-21T04:35:17.036 INFO:tasks.workunit.client.0.smithi007.stdout:9/246: dwrite d13/f23 [0,4194304] 0 2024-09-21T04:35:17.040 INFO:tasks.workunit.client.1.smithi053.stdout:4/882: rmdir d1/dd/de 39 2024-09-21T04:35:17.041 INFO:tasks.workunit.client.0.smithi007.stdout:9/247: mknod d13/d18/c51 0 2024-09-21T04:35:17.042 INFO:tasks.workunit.client.1.smithi053.stdout:3/777: symlink d16/d17/d21/d42/d43/l124 0 2024-09-21T04:35:17.042 INFO:tasks.workunit.client.0.smithi007.stdout:0/205: link d6/d16/c2f d6/d16/d18/d2e/c4c 0 2024-09-21T04:35:17.043 INFO:tasks.workunit.client.1.smithi053.stdout:0/767: write d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f91 [4515360,90978] 0 2024-09-21T04:35:17.044 INFO:tasks.workunit.client.1.smithi053.stdout:0/768: write d5/d10/d4c/db1/fe9 [719945,34487] 0 2024-09-21T04:35:17.044 INFO:tasks.workunit.client.1.smithi053.stdout:7/689: symlink d0/d6f/dbf/d50/d84/ldf 0 2024-09-21T04:35:17.049 INFO:tasks.workunit.client.0.smithi007.stdout:9/248: dread d13/f28 [0,4194304] 0 2024-09-21T04:35:17.049 INFO:tasks.workunit.client.0.smithi007.stdout:9/249: readlink d13/l37 0 2024-09-21T04:35:17.050 INFO:tasks.workunit.client.1.smithi053.stdout:6/762: rename d2/d1b/d1e/d2a/l40 to d2/d1b/d1e/dc1/lf6 0 2024-09-21T04:35:17.052 INFO:tasks.workunit.client.1.smithi053.stdout:6/763: chown d2/d1b/f3a 39 1 2024-09-21T04:35:17.052 INFO:tasks.workunit.client.1.smithi053.stdout:9/754: dwrite d0/d1c/d52/d53/f80 [0,4194304] 0 2024-09-21T04:35:17.054 INFO:tasks.workunit.client.1.smithi053.stdout:1/793: symlink d7/d9/da/d40/d49/d77/l103 0 2024-09-21T04:35:17.054 INFO:tasks.workunit.client.0.smithi007.stdout:9/250: creat d13/d2a/f52 x:0 0 0 2024-09-21T04:35:17.055 INFO:tasks.workunit.client.1.smithi053.stdout:5/809: write d0/d8/db/f76 [114655,105977] 0 2024-09-21T04:35:17.056 INFO:tasks.workunit.client.0.smithi007.stdout:0/206: creat d6/d16/f4d x:0 0 0 2024-09-21T04:35:17.057 INFO:tasks.workunit.client.0.smithi007.stdout:0/207: fsync d6/d16/d47/f4b 0 2024-09-21T04:35:17.058 INFO:tasks.workunit.client.0.smithi007.stdout:0/208: fsync d6/d16/f28 0 2024-09-21T04:35:17.058 INFO:tasks.workunit.client.1.smithi053.stdout:3/778: truncate d16/d1b/f59 5849176 0 2024-09-21T04:35:17.059 INFO:tasks.workunit.client.1.smithi053.stdout:5/810: read d0/dd/d5c/f7a [556492,104270] 0 2024-09-21T04:35:17.059 INFO:tasks.workunit.client.1.smithi053.stdout:7/690: creat d0/d6f/dbf/d50/d8b/d70/fe0 x:0 0 0 2024-09-21T04:35:17.061 INFO:tasks.workunit.client.1.smithi053.stdout:6/764: dwrite d2/d1b/fb5 [0,4194304] 0 2024-09-21T04:35:17.061 INFO:tasks.workunit.client.1.smithi053.stdout:0/769: unlink d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f86 0 2024-09-21T04:35:17.062 INFO:tasks.workunit.client.1.smithi053.stdout:8/843: getdents d3/d5/d4c/dd0 0 2024-09-21T04:35:17.065 INFO:tasks.workunit.client.1.smithi053.stdout:8/844: dread d3/f9c [0,4194304] 0 2024-09-21T04:35:17.065 INFO:tasks.workunit.client.0.smithi007.stdout:9/251: rename d13/c39 to d13/c53 0 2024-09-21T04:35:17.067 INFO:tasks.workunit.client.1.smithi053.stdout:8/845: dread d3/d5/d4b/d5c/d86/fc5 [0,4194304] 0 2024-09-21T04:35:17.072 INFO:tasks.workunit.client.0.smithi007.stdout:9/252: rename d13/f15 to d13/d18/f54 0 2024-09-21T04:35:17.072 INFO:tasks.workunit.client.1.smithi053.stdout:4/883: dwrite d1/dd/de/df5/d3f/f50 [0,4194304] 0 2024-09-21T04:35:17.076 INFO:tasks.workunit.client.1.smithi053.stdout:5/811: unlink d0/d66/cab 0 2024-09-21T04:35:17.078 INFO:tasks.workunit.client.0.smithi007.stdout:0/209: dread d6/d16/d18/d27/f33 [0,4194304] 0 2024-09-21T04:35:17.082 INFO:tasks.workunit.client.1.smithi053.stdout:3/779: dwrite d16/d17/d21/d26/d49/fe0 [0,4194304] 0 2024-09-21T04:35:17.082 INFO:tasks.workunit.client.0.smithi007.stdout:0/210: truncate d6/d16/d47/f4b 754153 0 2024-09-21T04:35:17.094 INFO:tasks.workunit.client.0.smithi007.stdout:0/211: link d6/d16/d18/d27/c2c d6/d3d/c4e 0 2024-09-21T04:35:17.098 INFO:tasks.workunit.client.0.smithi007.stdout:9/253: dwrite d13/d2a/f31 [0,4194304] 0 2024-09-21T04:35:17.098 INFO:tasks.workunit.client.1.smithi053.stdout:4/884: creat d1/dd/de/df5/d20/d5b/d87/dd5/d115/f122 x:0 0 0 2024-09-21T04:35:17.099 INFO:tasks.workunit.client.1.smithi053.stdout:4/885: readlink d1/dd/de/l99 0 2024-09-21T04:35:17.104 INFO:tasks.workunit.client.1.smithi053.stdout:3/780: mkdir d16/d17/d21/d42/d43/d125 0 2024-09-21T04:35:17.105 INFO:tasks.workunit.client.0.smithi007.stdout:9/254: dwrite d13/d18/f36 [0,4194304] 0 2024-09-21T04:35:17.108 INFO:tasks.workunit.client.1.smithi053.stdout:2/812: getdents d1/d18/d1f/d22 0 2024-09-21T04:35:17.113 INFO:tasks.workunit.client.0.smithi007.stdout:9/255: fsync d13/f21 0 2024-09-21T04:35:17.114 INFO:tasks.workunit.client.0.smithi007.stdout:9/256: chown d13/d2a/f31 795268 1 2024-09-21T04:35:17.115 INFO:tasks.workunit.client.0.smithi007.stdout:0/212: truncate d6/f13 2218685 0 2024-09-21T04:35:17.116 INFO:tasks.workunit.client.1.smithi053.stdout:8/846: mkdir d3/d5/d4c/dd0/d10e 0 2024-09-21T04:35:17.116 INFO:tasks.workunit.client.1.smithi053.stdout:1/794: link d7/d47/dae/lda d7/d9/da/d40/d49/d77/l104 0 2024-09-21T04:35:17.153 INFO:tasks.workunit.client.1.smithi053.stdout:7/691: mknod d0/dcd/ce1 0 2024-09-21T04:35:17.154 INFO:tasks.workunit.client.0.smithi007.stdout:3/150: sync 2024-09-21T04:35:17.154 INFO:tasks.workunit.client.1.smithi053.stdout:7/692: chown d0/d6f/c9d 15132 1 2024-09-21T04:35:17.154 INFO:tasks.workunit.client.0.smithi007.stdout:3/151: read dc/f10 [5505005,94709] 0 2024-09-21T04:35:17.161 INFO:tasks.workunit.client.0.smithi007.stdout:3/152: creat dc/d18/f28 x:0 0 0 2024-09-21T04:35:17.161 INFO:tasks.workunit.client.0.smithi007.stdout:3/153: stat dc/d22/f25 0 2024-09-21T04:35:17.166 INFO:tasks.workunit.client.1.smithi053.stdout:3/781: fdatasync d16/d17/d21/d26/d3e/fed 0 2024-09-21T04:35:17.167 INFO:tasks.workunit.client.0.smithi007.stdout:0/213: dwrite d6/d16/d18/d39/f3b [4194304,4194304] 0 2024-09-21T04:35:17.170 INFO:tasks.workunit.client.1.smithi053.stdout:5/812: truncate d0/d8/db/f7f 1844465 0 2024-09-21T04:35:17.172 INFO:tasks.workunit.client.1.smithi053.stdout:2/813: mknod d1/d18/d1f/d22/d2d/dcd/dee/c11f 0 2024-09-21T04:35:17.173 INFO:tasks.workunit.client.1.smithi053.stdout:9/755: getdents d0/d1c/d52/df7 0 2024-09-21T04:35:17.179 INFO:tasks.workunit.client.1.smithi053.stdout:6/765: rmdir d2/d1b/d1e/d27/df3 0 2024-09-21T04:35:17.180 INFO:tasks.workunit.client.1.smithi053.stdout:0/770: getdents d5/d10/d4c/db1/dbc/dca/db5/d99 0 2024-09-21T04:35:17.182 INFO:tasks.workunit.client.1.smithi053.stdout:5/813: write d0/dd/d5c/d92/de8/fec [5077434,128674] 0 2024-09-21T04:35:17.182 INFO:tasks.workunit.client.1.smithi053.stdout:0/771: read d5/f97 [2506162,32485] 0 2024-09-21T04:35:17.182 INFO:tasks.workunit.client.1.smithi053.stdout:2/814: symlink d1/d18/d77/d9b/l120 0 2024-09-21T04:35:17.183 INFO:tasks.workunit.client.1.smithi053.stdout:2/815: fsync d1/f8c 0 2024-09-21T04:35:17.185 INFO:tasks.workunit.client.1.smithi053.stdout:4/886: rename d1/dd/de/df5/d20/d4e/d51/da4/dc3/d100 to d1/d123 0 2024-09-21T04:35:17.187 INFO:tasks.workunit.client.1.smithi053.stdout:6/766: creat d2/d1b/d3d/d49/de/d32/d80/ff7 x:0 0 0 2024-09-21T04:35:17.189 INFO:tasks.workunit.client.1.smithi053.stdout:3/782: symlink d16/d17/l126 0 2024-09-21T04:35:17.189 INFO:tasks.workunit.client.1.smithi053.stdout:8/847: write d3/df/d10/f48 [3647744,101584] 0 2024-09-21T04:35:17.191 INFO:tasks.workunit.client.1.smithi053.stdout:1/795: write d7/d9/d43/d54/d7f/d59/f9d [1085398,34548] 0 2024-09-21T04:35:17.191 INFO:tasks.workunit.client.1.smithi053.stdout:9/756: symlink d0/d2/d9/d34/d58/d92/de6/l104 0 2024-09-21T04:35:17.193 INFO:tasks.workunit.client.1.smithi053.stdout:5/814: readlink d0/dd/d48/d8c/la1 0 2024-09-21T04:35:17.194 INFO:tasks.workunit.client.1.smithi053.stdout:2/816: rmdir d1/d18/d77/d83 39 2024-09-21T04:35:17.195 INFO:tasks.workunit.client.1.smithi053.stdout:2/817: read d1/d18/d1f/d27/d58/d5c/fd4 [2866938,37228] 0 2024-09-21T04:35:17.196 INFO:tasks.workunit.client.1.smithi053.stdout:2/818: chown d1/fb 59021056 1 2024-09-21T04:35:17.197 INFO:tasks.workunit.client.1.smithi053.stdout:8/848: mkdir d3/d5/d38/d64/d9d/d10f 0 2024-09-21T04:35:17.199 INFO:tasks.workunit.client.1.smithi053.stdout:3/783: rename d16/d17/d3c to d16/d17/d21/d42/d43/d125/d127 0 2024-09-21T04:35:17.205 INFO:tasks.workunit.client.1.smithi053.stdout:0/772: link d5/d10/d4c/db1/dbc/dca/d21/f33 d5/de/d1c/da3/f100 0 2024-09-21T04:35:17.205 INFO:tasks.workunit.client.1.smithi053.stdout:4/887: creat d1/dd/de/df5/d20/d73/d10b/dec/f124 x:0 0 0 2024-09-21T04:35:17.206 INFO:tasks.workunit.client.1.smithi053.stdout:8/849: truncate d3/f9c 847068 0 2024-09-21T04:35:17.206 INFO:tasks.workunit.client.1.smithi053.stdout:7/693: link d0/d2b/l3b d0/d2b/d36/d64/le2 0 2024-09-21T04:35:17.208 INFO:tasks.workunit.client.1.smithi053.stdout:0/773: chown d5/de/d42/c84 16 1 2024-09-21T04:35:17.208 INFO:tasks.workunit.client.1.smithi053.stdout:6/767: link d2/d1b/d3d/d49/de/d32/lb7 d2/d1b/d1e/dc1/lf8 0 2024-09-21T04:35:17.210 INFO:tasks.workunit.client.1.smithi053.stdout:2/819: dwrite d1/d18/d1f/d22/d2d/dcd/dcf/f103 [0,4194304] 0 2024-09-21T04:35:17.211 INFO:tasks.workunit.client.1.smithi053.stdout:5/815: rename d0/dd/d5c/d92/lc2 to d0/d8/l106 0 2024-09-21T04:35:17.212 INFO:tasks.workunit.client.1.smithi053.stdout:9/757: getdents d0/d2/d4e 0 2024-09-21T04:35:17.212 INFO:tasks.workunit.client.1.smithi053.stdout:3/784: truncate d16/f1d 185307 0 2024-09-21T04:35:17.213 INFO:tasks.workunit.client.1.smithi053.stdout:6/768: creat d2/d1b/ff9 x:0 0 0 2024-09-21T04:35:17.214 INFO:tasks.workunit.client.1.smithi053.stdout:7/694: rename d0/d24/d46/f7d to d0/d24/d46/fe3 0 2024-09-21T04:35:17.215 INFO:tasks.workunit.client.1.smithi053.stdout:8/850: symlink d3/d5/d4c/dd0/d10e/l110 0 2024-09-21T04:35:17.216 INFO:tasks.workunit.client.1.smithi053.stdout:0/774: truncate d5/d10/d4c/db1/dbc/dca/d53/fbe 1276249 0 2024-09-21T04:35:17.217 INFO:tasks.workunit.client.1.smithi053.stdout:2/820: rmdir d1/d18/d1f/d22/d2d/deb 39 2024-09-21T04:35:17.218 INFO:tasks.workunit.client.1.smithi053.stdout:0/775: stat d5/d10/d4c/db1/dbc/dca/db5/la1 0 2024-09-21T04:35:17.222 INFO:tasks.workunit.client.1.smithi053.stdout:6/769: dwrite d2/d1b/f6f [0,4194304] 0 2024-09-21T04:35:17.226 INFO:tasks.workunit.client.1.smithi053.stdout:2/821: dwrite d1/d18/d1f/d27/d58/d5c/f113 [0,4194304] 0 2024-09-21T04:35:17.226 INFO:tasks.workunit.client.1.smithi053.stdout:3/785: rename d16/d17/d21/d42/d43/d125/d127/f114 to d16/d17/d21/d42/d43/d125/f128 0 2024-09-21T04:35:17.228 INFO:tasks.workunit.client.1.smithi053.stdout:8/851: truncate d3/d5/d21/f78 4929920 0 2024-09-21T04:35:17.229 INFO:tasks.workunit.client.1.smithi053.stdout:2/822: truncate d1/d18/d1f/da2/dac/f105 530325 0 2024-09-21T04:35:17.233 INFO:tasks.workunit.client.1.smithi053.stdout:7/695: unlink d0/d2b/l3b 0 2024-09-21T04:35:17.233 INFO:tasks.workunit.client.1.smithi053.stdout:3/786: symlink d16/d17/d21/d8b/d7d/dc6/l129 0 2024-09-21T04:35:17.234 INFO:tasks.workunit.client.1.smithi053.stdout:0/776: unlink d5/c85 0 2024-09-21T04:35:17.235 INFO:tasks.workunit.client.1.smithi053.stdout:3/787: write d16/d17/d21/d26/d2b/fa2 [163065,24845] 0 2024-09-21T04:35:17.236 INFO:tasks.workunit.client.1.smithi053.stdout:3/788: dread - d16/d17/d21/d42/ff0 zero size 2024-09-21T04:35:17.237 INFO:tasks.workunit.client.1.smithi053.stdout:3/789: dread - d16/d17/d21/d42/d43/d86/fce zero size 2024-09-21T04:35:17.238 INFO:tasks.workunit.client.0.smithi007.stdout:1/284: sync 2024-09-21T04:35:17.238 INFO:tasks.workunit.client.0.smithi007.stdout:4/209: sync 2024-09-21T04:35:17.238 INFO:tasks.workunit.client.0.smithi007.stdout:5/473: sync 2024-09-21T04:35:17.239 INFO:tasks.workunit.client.0.smithi007.stdout:8/145: sync 2024-09-21T04:35:17.239 INFO:tasks.workunit.client.0.smithi007.stdout:6/107: sync 2024-09-21T04:35:17.239 INFO:tasks.workunit.client.0.smithi007.stdout:7/262: sync 2024-09-21T04:35:17.239 INFO:tasks.workunit.client.0.smithi007.stdout:2/169: sync 2024-09-21T04:35:17.239 INFO:tasks.workunit.client.1.smithi053.stdout:4/888: link d1/dd/de/df5/d20/f34 d1/dd/de/df5/d20/d5b/d87/dd5/d115/d121/f125 0 2024-09-21T04:35:17.243 INFO:tasks.workunit.client.0.smithi007.stdout:7/263: fdatasync d3/d6/d13/f47 0 2024-09-21T04:35:17.243 INFO:tasks.workunit.client.0.smithi007.stdout:1/285: write d5/f28 [5162929,53611] 0 2024-09-21T04:35:17.243 INFO:tasks.workunit.client.0.smithi007.stdout:0/214: chown d6/d3d/c4e 375 1 2024-09-21T04:35:17.244 INFO:tasks.workunit.client.1.smithi053.stdout:5/816: rename d0/dd/d5c/d92/ddb/d35/d61/f9a to d0/d66/f107 0 2024-09-21T04:35:17.245 INFO:tasks.workunit.client.1.smithi053.stdout:4/889: dwrite d1/d27/fce [0,4194304] 0 2024-09-21T04:35:17.246 INFO:tasks.workunit.client.1.smithi053.stdout:6/770: write d2/d1b/d1e/d27/d41/dc9/ff1 [452964,88120] 0 2024-09-21T04:35:17.247 INFO:tasks.workunit.client.1.smithi053.stdout:8/852: creat d3/d5/d38/d64/d73/d91/f111 x:0 0 0 2024-09-21T04:35:17.249 INFO:tasks.workunit.client.1.smithi053.stdout:2/823: dread - d1/d18/d1f/d27/d41/d4b/d4d/fa3 zero size 2024-09-21T04:35:17.254 INFO:tasks.workunit.client.1.smithi053.stdout:0/777: mkdir d5/d10/d4c/db1/dbc/dca/d21/d101 0 2024-09-21T04:35:17.254 INFO:tasks.workunit.client.1.smithi053.stdout:5/817: dread d0/dd/d48/d8c/fc6 [0,4194304] 0 2024-09-21T04:35:17.256 INFO:tasks.workunit.client.1.smithi053.stdout:9/758: getdents d0/d1c/d25 0 2024-09-21T04:35:17.257 INFO:tasks.workunit.client.1.smithi053.stdout:9/759: chown d0/d2/d9/de7/fce 4 1 2024-09-21T04:35:17.258 INFO:tasks.workunit.client.1.smithi053.stdout:9/760: write d0/df/faf [1079310,82218] 0 2024-09-21T04:35:17.263 INFO:tasks.workunit.client.1.smithi053.stdout:7/696: write d0/d6f/dbf/d50/d8b/ff [1247902,91553] 0 2024-09-21T04:35:17.264 INFO:tasks.workunit.client.1.smithi053.stdout:6/771: dread d2/d1b/d1e/d2a/d4b/d77/d88/da4/fcc [0,4194304] 0 2024-09-21T04:35:17.264 INFO:tasks.workunit.client.1.smithi053.stdout:8/853: symlink d3/d5/d4c/dd0/d10e/l112 0 2024-09-21T04:35:17.267 INFO:tasks.workunit.client.1.smithi053.stdout:3/790: write d16/d17/d21/d42/d43/d125/d127/f79 [2325627,89923] 0 2024-09-21T04:35:17.269 INFO:tasks.workunit.client.1.smithi053.stdout:5/818: creat d0/d8/d2a/f108 x:0 0 0 2024-09-21T04:35:17.269 INFO:tasks.workunit.client.1.smithi053.stdout:0/778: chown d5/de/d42/d78/c8a 6187594 1 2024-09-21T04:35:17.271 INFO:tasks.workunit.client.1.smithi053.stdout:9/761: mkdir d0/d2/d2b/d105 0 2024-09-21T04:35:17.271 INFO:tasks.workunit.client.1.smithi053.stdout:4/890: dwrite d1/dd/d62/ff2 [0,4194304] 0 2024-09-21T04:35:17.272 INFO:tasks.workunit.client.1.smithi053.stdout:4/891: readlink d1/d27/d2e/l39 0 2024-09-21T04:35:17.272 INFO:tasks.workunit.client.1.smithi053.stdout:9/762: dread - d0/df/fee zero size 2024-09-21T04:35:17.277 INFO:tasks.workunit.client.1.smithi053.stdout:8/854: truncate d3/d39/f2f 1213296 0 2024-09-21T04:35:17.282 INFO:tasks.workunit.client.0.smithi007.stdout:6/108: mknod d3/de/d12/c26 0 2024-09-21T04:35:17.282 INFO:tasks.workunit.client.0.smithi007.stdout:5/474: creat d2/dd/d28/f98 x:0 0 0 2024-09-21T04:35:17.282 INFO:tasks.workunit.client.1.smithi053.stdout:5/819: unlink d0/dd/f23 0 2024-09-21T04:35:17.283 INFO:tasks.workunit.client.0.smithi007.stdout:3/154: dwrite dc/f10 [0,4194304] 0 2024-09-21T04:35:17.285 INFO:tasks.workunit.client.1.smithi053.stdout:3/791: mkdir d16/d17/d21/d8b/d7d/d57/dc1/d53/d12a 0 2024-09-21T04:35:17.286 INFO:tasks.workunit.client.1.smithi053.stdout:3/792: fdatasync d16/d1b/dad/fb1 0 2024-09-21T04:35:17.287 INFO:tasks.workunit.client.1.smithi053.stdout:0/779: truncate d5/f4a 928974 0 2024-09-21T04:35:17.290 INFO:tasks.workunit.client.1.smithi053.stdout:9/763: mknod d0/d1c/d21/c106 0 2024-09-21T04:35:17.290 INFO:tasks.workunit.client.0.smithi007.stdout:6/109: write d3/de/d12/f1f [236643,79944] 0 2024-09-21T04:35:17.291 INFO:tasks.workunit.client.0.smithi007.stdout:3/155: write dc/f1c [3786964,118405] 0 2024-09-21T04:35:17.292 INFO:tasks.workunit.client.0.smithi007.stdout:0/215: rename d6/l32 to d6/d3d/l4f 0 2024-09-21T04:35:17.293 INFO:tasks.workunit.client.0.smithi007.stdout:4/210: dwrite d2/fd [0,4194304] 0 2024-09-21T04:35:17.293 INFO:tasks.workunit.client.1.smithi053.stdout:6/772: symlink d2/d1b/d1e/d2a/lfa 0 2024-09-21T04:35:17.293 INFO:tasks.workunit.client.0.smithi007.stdout:6/110: chown d3/de/f1a 0 1 2024-09-21T04:35:17.294 INFO:tasks.workunit.client.0.smithi007.stdout:2/170: creat d0/f23 x:0 0 0 2024-09-21T04:35:17.296 INFO:tasks.workunit.client.1.smithi053.stdout:7/697: stat d0/d2b/d36/d64/d9e/lab 0 2024-09-21T04:35:17.297 INFO:tasks.workunit.client.0.smithi007.stdout:6/111: chown d3/de/d12 7629236 1 2024-09-21T04:35:17.299 INFO:tasks.workunit.client.0.smithi007.stdout:3/156: mkdir dc/d22/d29 0 2024-09-21T04:35:17.301 INFO:tasks.workunit.client.1.smithi053.stdout:5/820: rename d0/d8/d2a/c46 to d0/dd/d48/d8c/c109 0 2024-09-21T04:35:17.302 INFO:tasks.workunit.client.0.smithi007.stdout:4/211: symlink d2/d7/l46 0 2024-09-21T04:35:17.304 INFO:tasks.workunit.client.1.smithi053.stdout:1/796: sync 2024-09-21T04:35:17.306 INFO:tasks.workunit.client.1.smithi053.stdout:1/797: chown d7/d9/d43/d54/d7f/d59/dc3 1937212 1 2024-09-21T04:35:17.306 INFO:tasks.workunit.client.0.smithi007.stdout:3/157: fsync dc/f23 0 2024-09-21T04:35:17.306 INFO:tasks.workunit.client.1.smithi053.stdout:1/798: chown d7/d9/d43/d94/d69 0 1 2024-09-21T04:35:17.307 INFO:tasks.workunit.client.0.smithi007.stdout:7/264: dwrite d3/d6/d13/f47 [0,4194304] 0 2024-09-21T04:35:17.307 INFO:tasks.workunit.client.1.smithi053.stdout:9/764: mkdir d0/d1c/d21/d8e/d107 0 2024-09-21T04:35:17.308 INFO:tasks.workunit.client.0.smithi007.stdout:4/212: chown d2/f23 12940792 1 2024-09-21T04:35:17.308 INFO:tasks.workunit.client.1.smithi053.stdout:1/799: readlink d7/d9/da/d11/de5/lee 0 2024-09-21T04:35:17.310 INFO:tasks.workunit.client.1.smithi053.stdout:6/773: rmdir d2/d1d/de0 39 2024-09-21T04:35:17.310 INFO:tasks.workunit.client.1.smithi053.stdout:6/774: read - d2/d1b/d3d/fe1 zero size 2024-09-21T04:35:17.312 INFO:tasks.workunit.client.0.smithi007.stdout:8/146: dread d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:17.312 INFO:tasks.workunit.client.1.smithi053.stdout:2/824: getdents d1/d18/d1f/d22/d43/da6 0 2024-09-21T04:35:17.315 INFO:tasks.workunit.client.0.smithi007.stdout:0/216: symlink d6/d16/d18/d39/l50 0 2024-09-21T04:35:17.316 INFO:tasks.workunit.client.0.smithi007.stdout:2/171: dread d0/f1c [0,4194304] 0 2024-09-21T04:35:17.319 INFO:tasks.workunit.client.1.smithi053.stdout:5/821: mkdir d0/d8/db/d68/d10a 0 2024-09-21T04:35:17.319 INFO:tasks.workunit.client.0.smithi007.stdout:1/286: dread d5/d6/d21/d2b/f51 [0,4194304] 0 2024-09-21T04:35:17.320 INFO:tasks.workunit.client.0.smithi007.stdout:5/475: dread d2/dd/d23/d49/d6f/f7f [0,4194304] 0 2024-09-21T04:35:17.321 INFO:tasks.workunit.client.1.smithi053.stdout:3/793: mknod d16/d17/d21/d8b/c12b 0 2024-09-21T04:35:17.322 INFO:tasks.workunit.client.0.smithi007.stdout:3/158: dwrite dc/f13 [0,4194304] 0 2024-09-21T04:35:17.324 INFO:tasks.workunit.client.0.smithi007.stdout:3/159: write dc/f21 [1602390,70556] 0 2024-09-21T04:35:17.325 INFO:tasks.workunit.client.0.smithi007.stdout:5/476: truncate d2/dd/d23/d5d/d61/f74 606121 0 2024-09-21T04:35:17.325 INFO:tasks.workunit.client.0.smithi007.stdout:3/160: stat dc/d18/f28 0 2024-09-21T04:35:17.326 INFO:tasks.workunit.client.0.smithi007.stdout:7/265: fsync d3/d6/d14/d20/f35 0 2024-09-21T04:35:17.327 INFO:tasks.workunit.client.0.smithi007.stdout:7/266: chown d3/d6/d14/d20/d58 46 1 2024-09-21T04:35:17.328 INFO:tasks.workunit.client.0.smithi007.stdout:4/213: creat d2/d17/d18/f47 x:0 0 0 2024-09-21T04:35:17.330 INFO:tasks.workunit.client.0.smithi007.stdout:4/214: write d2/d7/f10 [1547461,87958] 0 2024-09-21T04:35:17.331 INFO:tasks.workunit.client.1.smithi053.stdout:6/775: readlink d2/d1b/d1e/d27/d90/la2 0 2024-09-21T04:35:17.333 INFO:tasks.workunit.client.1.smithi053.stdout:8/855: creat d3/df/f113 x:0 0 0 2024-09-21T04:35:17.335 INFO:tasks.workunit.client.1.smithi053.stdout:2/825: mkdir d1/d18/d1f/d22/d2d/dcd/dee/d121 0 2024-09-21T04:35:17.337 INFO:tasks.workunit.client.1.smithi053.stdout:7/698: rename d0/d6f/dbf/d5c/cde to d0/d2b/d36/d64/d9e/ce4 0 2024-09-21T04:35:17.339 INFO:tasks.workunit.client.1.smithi053.stdout:5/822: symlink d0/dd/d48/d8c/dfd/l10b 0 2024-09-21T04:35:17.340 INFO:tasks.workunit.client.0.smithi007.stdout:5/477: fsync d2/dd/d23/d49/d44/d54/f88 0 2024-09-21T04:35:17.342 INFO:tasks.workunit.client.1.smithi053.stdout:0/780: creat d5/d10/d4c/db1/dbc/dca/d21/d101/f102 x:0 0 0 2024-09-21T04:35:17.342 INFO:tasks.workunit.client.0.smithi007.stdout:5/478: truncate d2/dd/f14 5575635 0 2024-09-21T04:35:17.342 INFO:tasks.workunit.client.1.smithi053.stdout:0/781: chown d5/f97 28 1 2024-09-21T04:35:17.347 INFO:tasks.workunit.client.0.smithi007.stdout:8/147: getdents d0/d3/d8/de/d26 0 2024-09-21T04:35:17.347 INFO:tasks.workunit.client.1.smithi053.stdout:4/892: getdents d1/dd/de/df5/d20/d5b/d87/dd5/d115 0 2024-09-21T04:35:17.347 INFO:tasks.workunit.client.1.smithi053.stdout:9/765: symlink d0/d1a/d89/da1/dea/l108 0 2024-09-21T04:35:17.347 INFO:tasks.workunit.client.1.smithi053.stdout:9/766: write d0/d1a/f81 [328289,79311] 0 2024-09-21T04:35:17.348 INFO:tasks.workunit.client.0.smithi007.stdout:8/148: chown d0/c1a 0 1 2024-09-21T04:35:17.348 INFO:tasks.workunit.client.0.smithi007.stdout:4/215: creat d2/d7/f48 x:0 0 0 2024-09-21T04:35:17.350 INFO:tasks.workunit.client.1.smithi053.stdout:4/893: dwrite d1/dd/de/df5/d3f/f50 [0,4194304] 0 2024-09-21T04:35:17.350 INFO:tasks.workunit.client.1.smithi053.stdout:2/826: creat d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/f122 x:0 0 0 2024-09-21T04:35:17.351 INFO:tasks.workunit.client.1.smithi053.stdout:4/894: read - d1/d27/d2e/fc1 zero size 2024-09-21T04:35:17.351 INFO:tasks.workunit.client.1.smithi053.stdout:2/827: chown d1/d18/d1f/d27/d58/d5c/d91 64939 1 2024-09-21T04:35:17.351 INFO:tasks.workunit.client.0.smithi007.stdout:2/172: symlink d0/l24 0 2024-09-21T04:35:17.351 INFO:tasks.workunit.client.0.smithi007.stdout:4/216: chown d2/f25 1301117 1 2024-09-21T04:35:17.352 INFO:tasks.workunit.client.1.smithi053.stdout:4/895: chown d1/d27/d2e/l39 1888 1 2024-09-21T04:35:17.352 INFO:tasks.workunit.client.1.smithi053.stdout:8/856: rename d3/df/d14/d16/lb3 to d3/d5/d4c/d53/d66/l114 0 2024-09-21T04:35:17.353 INFO:tasks.workunit.client.0.smithi007.stdout:7/267: dwrite d3/d6/f2f [0,4194304] 0 2024-09-21T04:35:17.356 INFO:tasks.workunit.client.0.smithi007.stdout:0/217: rename d6/d16/d18/d2e/c31 to d6/d3d/c51 0 2024-09-21T04:35:17.359 INFO:tasks.workunit.client.0.smithi007.stdout:3/161: rename dc/d18 to dc/d18/d2a 22 2024-09-21T04:35:17.360 INFO:tasks.workunit.client.0.smithi007.stdout:5/479: dread d2/f55 [0,4194304] 0 2024-09-21T04:35:17.362 INFO:tasks.workunit.client.0.smithi007.stdout:6/112: getdents d3/de/d12 0 2024-09-21T04:35:17.363 INFO:tasks.workunit.client.1.smithi053.stdout:6/776: mkdir d2/d1b/d1e/dfb 0 2024-09-21T04:35:17.363 INFO:tasks.workunit.client.0.smithi007.stdout:0/218: dread d6/d16/d47/f4b [0,4194304] 0 2024-09-21T04:35:17.364 INFO:tasks.workunit.client.0.smithi007.stdout:2/173: creat d0/d2/f25 x:0 0 0 2024-09-21T04:35:17.364 INFO:tasks.workunit.client.1.smithi053.stdout:9/767: readlink d0/d6c/lff 0 2024-09-21T04:35:17.364 INFO:tasks.workunit.client.1.smithi053.stdout:6/777: stat d2/d1b/d1e/d2a/d4b/d77/d88/da4/fdf 0 2024-09-21T04:35:17.365 INFO:tasks.workunit.client.0.smithi007.stdout:5/480: mknod d2/d27/d42/c99 0 2024-09-21T04:35:17.366 INFO:tasks.workunit.client.1.smithi053.stdout:2/828: rmdir d1/d18/d1f/d27/d58/d5c/d91 39 2024-09-21T04:35:17.366 INFO:tasks.workunit.client.1.smithi053.stdout:2/829: readlink d1/d18/d77/l99 0 2024-09-21T04:35:17.367 INFO:tasks.workunit.client.1.smithi053.stdout:4/896: dread - d1/dd/de/df5/d20/d5b/f80 zero size 2024-09-21T04:35:17.368 INFO:tasks.workunit.client.1.smithi053.stdout:2/830: truncate d1/d18/d1f/d27/d41/f114 826872 0 2024-09-21T04:35:17.369 INFO:tasks.workunit.client.0.smithi007.stdout:8/149: dwrite d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:17.369 INFO:tasks.workunit.client.0.smithi007.stdout:2/174: dread - d0/d2/f16 zero size 2024-09-21T04:35:17.372 INFO:tasks.workunit.client.0.smithi007.stdout:6/113: dread d3/de/d12/d19/f1c [0,4194304] 0 2024-09-21T04:35:17.373 INFO:tasks.workunit.client.1.smithi053.stdout:0/782: mkdir d5/d10/d4c/ddf/d103 0 2024-09-21T04:35:17.374 INFO:tasks.workunit.client.0.smithi007.stdout:7/268: dwrite d3/d6/d13/f3c [0,4194304] 0 2024-09-21T04:35:17.375 INFO:tasks.workunit.client.1.smithi053.stdout:6/778: fdatasync d2/d1d/f54 0 2024-09-21T04:35:17.377 INFO:tasks.workunit.client.0.smithi007.stdout:3/162: dwrite dc/f23 [0,4194304] 0 2024-09-21T04:35:17.378 INFO:tasks.workunit.client.1.smithi053.stdout:9/768: creat d0/d1c/d52/db5/f109 x:0 0 0 2024-09-21T04:35:17.381 INFO:tasks.workunit.client.1.smithi053.stdout:8/857: mkdir d3/df/d49/d103/d115 0 2024-09-21T04:35:17.382 INFO:tasks.workunit.client.1.smithi053.stdout:8/858: chown d3/df/d14/d16 186 1 2024-09-21T04:35:17.388 INFO:tasks.workunit.client.1.smithi053.stdout:2/831: creat d1/f123 x:0 0 0 2024-09-21T04:35:17.390 INFO:tasks.workunit.client.1.smithi053.stdout:7/699: mknod d0/d2b/d36/d64/da7/ce5 0 2024-09-21T04:35:17.391 INFO:tasks.workunit.client.0.smithi007.stdout:5/481: dwrite d2/dd/f16 [8388608,4194304] 0 2024-09-21T04:35:17.392 INFO:tasks.workunit.client.0.smithi007.stdout:6/114: mknod d3/de/c27 0 2024-09-21T04:35:17.392 INFO:tasks.workunit.client.1.smithi053.stdout:7/700: write d0/d6f/dbf/d50/d8b/f81 [910050,130476] 0 2024-09-21T04:35:17.392 INFO:tasks.workunit.client.1.smithi053.stdout:3/794: getdents d16/d17/d21/d60 0 2024-09-21T04:35:17.393 INFO:tasks.workunit.client.1.smithi053.stdout:3/795: chown d16/d17/d21/d42/d43/d125/d127 432883030 1 2024-09-21T04:35:17.394 INFO:tasks.workunit.client.0.smithi007.stdout:7/269: rmdir d3/d6 39 2024-09-21T04:35:17.396 INFO:tasks.workunit.client.0.smithi007.stdout:6/115: truncate d3/f15 115982 0 2024-09-21T04:35:17.398 INFO:tasks.workunit.client.0.smithi007.stdout:1/287: dwrite d5/d45/f4e [0,4194304] 0 2024-09-21T04:35:17.399 INFO:tasks.workunit.client.1.smithi053.stdout:3/796: dwrite d16/d1b/dc3/f6c [0,4194304] 0 2024-09-21T04:35:17.400 INFO:tasks.workunit.client.1.smithi053.stdout:3/797: dread - d16/d17/d21/d42/da0/ffc zero size 2024-09-21T04:35:17.400 INFO:tasks.workunit.client.1.smithi053.stdout:0/783: unlink d5/d10/f2b 0 2024-09-21T04:35:17.402 INFO:tasks.workunit.client.0.smithi007.stdout:4/217: truncate d2/f43 945478 0 2024-09-21T04:35:17.406 INFO:tasks.workunit.client.0.smithi007.stdout:0/219: dread d6/f12 [0,4194304] 0 2024-09-21T04:35:17.406 INFO:tasks.workunit.client.0.smithi007.stdout:8/150: mknod d0/d3/c27 0 2024-09-21T04:35:17.408 INFO:tasks.workunit.client.0.smithi007.stdout:8/151: readlink d0/d3/d8/lc 0 2024-09-21T04:35:17.408 INFO:tasks.workunit.client.1.smithi053.stdout:6/779: symlink d2/d1b/d1e/d27/d41/lfc 0 2024-09-21T04:35:17.411 INFO:tasks.workunit.client.1.smithi053.stdout:9/769: rename d0/d1c/l2d to d0/df/l10a 0 2024-09-21T04:35:17.414 INFO:tasks.workunit.client.1.smithi053.stdout:6/780: dwrite d2/d1b/f6f [0,4194304] 0 2024-09-21T04:35:17.415 INFO:tasks.workunit.client.1.smithi053.stdout:8/859: creat d3/d5/d4c/dc3/f116 x:0 0 0 2024-09-21T04:35:17.416 INFO:tasks.workunit.client.0.smithi007.stdout:5/482: truncate d2/f5a 479508 0 2024-09-21T04:35:17.417 INFO:tasks.workunit.client.1.smithi053.stdout:4/897: write d1/dd/de/df5/fb5 [3753615,96081] 0 2024-09-21T04:35:17.422 INFO:tasks.workunit.client.0.smithi007.stdout:1/288: write d5/d6/d21/d23/f2a [1574805,15728] 0 2024-09-21T04:35:17.424 INFO:tasks.workunit.client.0.smithi007.stdout:3/163: mknod dc/d18/d26/c2b 0 2024-09-21T04:35:17.428 INFO:tasks.workunit.client.0.smithi007.stdout:5/483: dwrite d2/dd/d23/d5d/f89 [0,4194304] 0 2024-09-21T04:35:17.433 INFO:tasks.workunit.client.0.smithi007.stdout:8/152: rmdir d0 39 2024-09-21T04:35:17.434 INFO:tasks.workunit.client.0.smithi007.stdout:5/484: read - d2/dd/d23/d5d/d61/f91 zero size 2024-09-21T04:35:17.436 INFO:tasks.workunit.client.0.smithi007.stdout:1/289: dwrite d5/d6/d21/f47 [0,4194304] 0 2024-09-21T04:35:17.436 INFO:tasks.workunit.client.1.smithi053.stdout:0/784: write d5/d10/d4c/db1/dbc/dca/db5/d99/fce [871754,54953] 0 2024-09-21T04:35:17.437 INFO:tasks.workunit.client.1.smithi053.stdout:3/798: dwrite d16/d1b/f2f [0,4194304] 0 2024-09-21T04:35:17.439 INFO:tasks.workunit.client.1.smithi053.stdout:0/785: stat d5/d10/d4c/db1/dbc/dca/db5/dd5/le8 0 2024-09-21T04:35:17.439 INFO:tasks.workunit.client.0.smithi007.stdout:7/270: mkdir d3/d6/d10/d62 0 2024-09-21T04:35:17.441 INFO:tasks.workunit.client.1.smithi053.stdout:9/770: chown d0/df/l10a 7 1 2024-09-21T04:35:17.444 INFO:tasks.workunit.client.1.smithi053.stdout:8/860: mkdir d3/df/d14/d16/d117 0 2024-09-21T04:35:17.445 INFO:tasks.workunit.client.1.smithi053.stdout:4/898: creat d1/dd/de/df5/d3f/d74/de1/f126 x:0 0 0 2024-09-21T04:35:17.446 INFO:tasks.workunit.client.0.smithi007.stdout:3/164: mknod dc/d18/d26/c2c 0 2024-09-21T04:35:17.446 INFO:tasks.workunit.client.1.smithi053.stdout:1/800: sync 2024-09-21T04:35:17.447 INFO:tasks.workunit.client.1.smithi053.stdout:4/899: write d1/dd/de/df5/d3f/d74/de1/f11b [132854,60977] 0 2024-09-21T04:35:17.447 INFO:tasks.workunit.client.0.smithi007.stdout:3/165: truncate dc/d18/f28 591609 0 2024-09-21T04:35:17.448 INFO:tasks.workunit.client.1.smithi053.stdout:1/801: write d7/d9/d43/d94/fa7 [472496,16756] 0 2024-09-21T04:35:17.449 INFO:tasks.workunit.client.1.smithi053.stdout:4/900: stat d1/dd/de/df5/d20/d73/d10b/c47 0 2024-09-21T04:35:17.449 INFO:tasks.workunit.client.1.smithi053.stdout:7/701: creat d0/d2b/d36/d64/dc0/fe6 x:0 0 0 2024-09-21T04:35:17.450 INFO:tasks.workunit.client.1.smithi053.stdout:4/901: stat d1/dd/de/df5/d20/d5b/fbe 0 2024-09-21T04:35:17.451 INFO:tasks.workunit.client.1.smithi053.stdout:4/902: dread - d1/dd/de/df5/d20/d4e/d51/fab zero size 2024-09-21T04:35:17.451 INFO:tasks.workunit.client.0.smithi007.stdout:0/220: mkdir d6/d16/d52 0 2024-09-21T04:35:17.452 INFO:tasks.workunit.client.1.smithi053.stdout:3/799: fsync d16/d17/d21/d26/d3e/f99 0 2024-09-21T04:35:17.453 INFO:tasks.workunit.client.1.smithi053.stdout:3/800: write d16/d17/d21/d26/f102 [331183,72420] 0 2024-09-21T04:35:17.453 INFO:tasks.workunit.client.1.smithi053.stdout:6/781: mknod d2/d1d/cfd 0 2024-09-21T04:35:17.455 INFO:tasks.workunit.client.1.smithi053.stdout:9/771: write d0/d2/d4e/fca [5046959,42493] 0 2024-09-21T04:35:17.457 INFO:tasks.workunit.client.0.smithi007.stdout:2/175: getdents d0/d11 0 2024-09-21T04:35:17.459 INFO:tasks.workunit.client.0.smithi007.stdout:6/116: creat d3/f28 x:0 0 0 2024-09-21T04:35:17.459 INFO:tasks.workunit.client.1.smithi053.stdout:2/832: link d1/fb d1/d18/d1f/d22/d2d/dcd/dcf/f124 0 2024-09-21T04:35:17.460 INFO:tasks.workunit.client.1.smithi053.stdout:4/903: symlink d1/ddb/l127 0 2024-09-21T04:35:17.461 INFO:tasks.workunit.client.0.smithi007.stdout:4/218: write d2/f23 [443593,30040] 0 2024-09-21T04:35:17.461 INFO:tasks.workunit.client.0.smithi007.stdout:8/153: dwrite d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:17.462 INFO:tasks.workunit.client.1.smithi053.stdout:8/861: dwrite d3/d39/f27 [4194304,4194304] 0 2024-09-21T04:35:17.463 INFO:tasks.workunit.client.0.smithi007.stdout:8/154: stat d0 0 2024-09-21T04:35:17.468 INFO:tasks.workunit.client.1.smithi053.stdout:0/786: creat d5/d10/d4c/ddf/d103/f104 x:0 0 0 2024-09-21T04:35:17.468 INFO:tasks.workunit.client.1.smithi053.stdout:0/787: stat d5/fab 0 2024-09-21T04:35:17.470 INFO:tasks.workunit.client.1.smithi053.stdout:1/802: fdatasync d7/d63/d96/dc7/fed 0 2024-09-21T04:35:17.471 INFO:tasks.workunit.client.0.smithi007.stdout:1/290: mkdir d5/d6/d21/d23/d55/d58 0 2024-09-21T04:35:17.473 INFO:tasks.workunit.client.1.smithi053.stdout:2/833: creat d1/d18/d77/d9b/f125 x:0 0 0 2024-09-21T04:35:17.474 INFO:tasks.workunit.client.1.smithi053.stdout:2/834: chown d1/d18/d1f/d22/d2d/dcd/l10d 69642198 1 2024-09-21T04:35:17.477 INFO:tasks.workunit.client.0.smithi007.stdout:2/176: dwrite d0/d2/f21 [0,4194304] 0 2024-09-21T04:35:17.477 INFO:tasks.workunit.client.0.smithi007.stdout:2/177: truncate d0/f19 5037784 0 2024-09-21T04:35:17.478 INFO:tasks.workunit.client.0.smithi007.stdout:0/221: rename d6/d16/d18/d2e/c4c to d6/d16/d18/d2e/c53 0 2024-09-21T04:35:17.479 INFO:tasks.workunit.client.1.smithi053.stdout:8/862: symlink d3/d5/d38/d64/d9d/l118 0 2024-09-21T04:35:17.481 INFO:tasks.workunit.client.1.smithi053.stdout:3/801: rename d16/d17/d21/d26/d2b/cc5 to d16/d1b/c12c 0 2024-09-21T04:35:17.482 INFO:tasks.workunit.client.0.smithi007.stdout:8/155: read d0/d3/d8/de/f18 [2750073,86648] 0 2024-09-21T04:35:17.484 INFO:tasks.workunit.client.1.smithi053.stdout:9/772: mkdir d0/d1a/d89/de3/d10b 0 2024-09-21T04:35:17.485 INFO:tasks.workunit.client.0.smithi007.stdout:4/219: dread d2/d17/d18/f1e [0,4194304] 0 2024-09-21T04:35:17.486 INFO:tasks.workunit.client.1.smithi053.stdout:9/773: chown d0/d1c/d21/d8e/d107 105559 1 2024-09-21T04:35:17.487 INFO:tasks.workunit.client.1.smithi053.stdout:0/788: rmdir d5/d10/d4c/db1/dbc/dca/d21/d101 39 2024-09-21T04:35:17.488 INFO:tasks.workunit.client.1.smithi053.stdout:9/774: fdatasync d0/d2/d9/d34/f59 0 2024-09-21T04:35:17.488 INFO:tasks.workunit.client.1.smithi053.stdout:8/863: dwrite d3/d28/f68 [4194304,4194304] 0 2024-09-21T04:35:17.490 INFO:tasks.workunit.client.1.smithi053.stdout:9/775: fsync d0/d1c/d52/db5/f109 0 2024-09-21T04:35:17.492 INFO:tasks.workunit.client.1.smithi053.stdout:1/803: symlink d7/d9/da/d40/d49/l105 0 2024-09-21T04:35:17.495 INFO:tasks.workunit.client.1.smithi053.stdout:4/904: dread d1/dd/de/df5/d20/d4e/d51/da4/df1/f40 [0,4194304] 0 2024-09-21T04:35:17.497 INFO:tasks.workunit.client.0.smithi007.stdout:4/220: mknod d2/d17/d29/c49 0 2024-09-21T04:35:17.498 INFO:tasks.workunit.client.1.smithi053.stdout:2/835: write d1/d18/d1f/d27/d58/d5c/d91/dd0/f10f [3065862,126547] 0 2024-09-21T04:35:17.498 INFO:tasks.workunit.client.1.smithi053.stdout:0/789: dwrite d5/de/d9e/dd8/ff8 [0,4194304] 0 2024-09-21T04:35:17.501 INFO:tasks.workunit.client.0.smithi007.stdout:3/166: getdents dc/d22 0 2024-09-21T04:35:17.501 INFO:tasks.workunit.client.1.smithi053.stdout:0/790: write d5/f8d [2056149,30110] 0 2024-09-21T04:35:17.504 INFO:tasks.workunit.client.1.smithi053.stdout:8/864: fdatasync d3/d5/f19 0 2024-09-21T04:35:17.507 INFO:tasks.workunit.client.0.smithi007.stdout:4/221: rmdir d2/d17/d18 39 2024-09-21T04:35:17.508 INFO:tasks.workunit.client.0.smithi007.stdout:4/222: readlink d2/d17/l35 0 2024-09-21T04:35:17.508 INFO:tasks.workunit.client.1.smithi053.stdout:9/776: symlink d0/d1a/d4c/l10c 0 2024-09-21T04:35:17.509 INFO:tasks.workunit.client.0.smithi007.stdout:6/117: dread d3/f6 [0,4194304] 0 2024-09-21T04:35:17.510 INFO:tasks.workunit.client.0.smithi007.stdout:3/167: truncate fb 3990063 0 2024-09-21T04:35:17.510 INFO:tasks.workunit.client.1.smithi053.stdout:6/782: getdents d2/d1d/dbb 0 2024-09-21T04:35:17.512 INFO:tasks.workunit.client.0.smithi007.stdout:4/223: fdatasync d2/d17/d18/f41 0 2024-09-21T04:35:17.514 INFO:tasks.workunit.client.0.smithi007.stdout:4/224: chown d2/d7/f10 14 1 2024-09-21T04:35:17.514 INFO:tasks.workunit.client.0.smithi007.stdout:6/118: creat d3/de/d12/f29 x:0 0 0 2024-09-21T04:35:17.518 INFO:tasks.workunit.client.0.smithi007.stdout:1/291: dwrite d5/d45/f46 [0,4194304] 0 2024-09-21T04:35:17.522 INFO:tasks.workunit.client.1.smithi053.stdout:3/802: mknod d16/d17/d21/d8b/d7d/d57/dc1/d53/d12a/c12d 0 2024-09-21T04:35:17.522 INFO:tasks.workunit.client.0.smithi007.stdout:4/225: dread d2/f23 [0,4194304] 0 2024-09-21T04:35:17.523 INFO:tasks.workunit.client.0.smithi007.stdout:2/178: truncate d0/d2/f6 1150773 0 2024-09-21T04:35:17.524 INFO:tasks.workunit.client.0.smithi007.stdout:3/168: mkdir dc/d22/d29/d2d 0 2024-09-21T04:35:17.526 INFO:tasks.workunit.client.0.smithi007.stdout:5/485: dwrite d2/f5a [0,4194304] 0 2024-09-21T04:35:17.526 INFO:tasks.workunit.client.1.smithi053.stdout:2/836: creat d1/d18/d1f/da2/dac/f126 x:0 0 0 2024-09-21T04:35:17.527 INFO:tasks.workunit.client.0.smithi007.stdout:5/486: fsync d2/d27/d3d/f85 0 2024-09-21T04:35:17.528 INFO:tasks.workunit.client.0.smithi007.stdout:5/487: write d2/f70 [40321,72405] 0 2024-09-21T04:35:17.529 INFO:tasks.workunit.client.0.smithi007.stdout:6/119: mknod d3/de/d12/d19/c2a 0 2024-09-21T04:35:17.531 INFO:tasks.workunit.client.0.smithi007.stdout:5/488: write d2/dd/d28/f98 [362470,38483] 0 2024-09-21T04:35:17.534 INFO:tasks.workunit.client.0.smithi007.stdout:1/292: creat d5/d6/d21/d23/d42/f59 x:0 0 0 2024-09-21T04:35:17.538 INFO:tasks.workunit.client.1.smithi053.stdout:9/777: readlink d0/d1c/l20 0 2024-09-21T04:35:17.539 INFO:tasks.workunit.client.1.smithi053.stdout:1/804: dwrite d7/d9/d43/d54/d7f/d36/f4e [0,4194304] 0 2024-09-21T04:35:17.539 INFO:tasks.workunit.client.0.smithi007.stdout:5/489: dwrite d2/d27/d3d/d56/f96 [0,4194304] 0 2024-09-21T04:35:17.539 INFO:tasks.workunit.client.1.smithi053.stdout:9/778: dread - d0/d1c/dcb/fd9 zero size 2024-09-21T04:35:17.541 INFO:tasks.workunit.client.0.smithi007.stdout:4/226: unlink d2/d17/d18/f1f 0 2024-09-21T04:35:17.541 INFO:tasks.workunit.client.1.smithi053.stdout:1/805: chown d7/d9/da/d40/d58/c71 23805 1 2024-09-21T04:35:17.542 INFO:tasks.workunit.client.1.smithi053.stdout:6/783: symlink d2/d1b/lfe 0 2024-09-21T04:35:17.543 INFO:tasks.workunit.client.1.smithi053.stdout:9/779: read d0/df/f48 [281045,19017] 0 2024-09-21T04:35:17.547 INFO:tasks.workunit.client.0.smithi007.stdout:4/227: dread d2/fd [0,4194304] 0 2024-09-21T04:35:17.548 INFO:tasks.workunit.client.0.smithi007.stdout:2/179: rename d0/d2/l17 to d0/d2/l26 0 2024-09-21T04:35:17.549 INFO:tasks.workunit.client.0.smithi007.stdout:2/180: stat d0/l24 0 2024-09-21T04:35:17.550 INFO:tasks.workunit.client.0.smithi007.stdout:2/181: truncate d0/d2/fc 4959297 0 2024-09-21T04:35:17.551 INFO:tasks.workunit.client.0.smithi007.stdout:6/120: rename d3/de/d12/f14 to d3/de/d12/f2b 0 2024-09-21T04:35:17.552 INFO:tasks.workunit.client.0.smithi007.stdout:2/182: truncate d0/d2/f25 1012958 0 2024-09-21T04:35:17.552 INFO:tasks.workunit.client.0.smithi007.stdout:2/183: fdatasync d0/d2/f21 0 2024-09-21T04:35:17.556 INFO:tasks.workunit.client.0.smithi007.stdout:1/293: write d5/d6/d21/f30 [223638,76288] 0 2024-09-21T04:35:17.557 INFO:tasks.workunit.client.0.smithi007.stdout:1/294: write d5/d45/f46 [2794444,13439] 0 2024-09-21T04:35:17.558 INFO:tasks.workunit.client.1.smithi053.stdout:5/823: sync 2024-09-21T04:35:17.559 INFO:tasks.workunit.client.1.smithi053.stdout:1/806: rename d7/d9/d43/d94 to d7/d63/d96/dac/d106 0 2024-09-21T04:35:17.559 INFO:tasks.workunit.client.1.smithi053.stdout:5/824: write d0/d8/f1f [1804586,116862] 0 2024-09-21T04:35:17.560 INFO:tasks.workunit.client.1.smithi053.stdout:5/825: chown d0/dd/d67/ld1 0 1 2024-09-21T04:35:17.561 INFO:tasks.workunit.client.0.smithi007.stdout:5/490: mkdir d2/dd/d23/d5d/d9a 0 2024-09-21T04:35:17.561 INFO:tasks.workunit.client.1.smithi053.stdout:5/826: fsync d0/d8/db/f78 0 2024-09-21T04:35:17.562 INFO:tasks.workunit.client.1.smithi053.stdout:6/784: creat d2/d1b/d1e/dc1/fff x:0 0 0 2024-09-21T04:35:17.563 INFO:tasks.workunit.client.1.smithi053.stdout:9/780: mkdir d0/d1c/d52/df7/d10d 0 2024-09-21T04:35:17.566 INFO:tasks.workunit.client.1.smithi053.stdout:2/837: creat d1/d18/d77/d83/f127 x:0 0 0 2024-09-21T04:35:17.567 INFO:tasks.workunit.client.1.smithi053.stdout:8/865: write d3/d28/f2e [384966,67913] 0 2024-09-21T04:35:17.567 INFO:tasks.workunit.client.0.smithi007.stdout:3/169: symlink dc/d22/d29/d2d/l2e 0 2024-09-21T04:35:17.570 INFO:tasks.workunit.client.0.smithi007.stdout:6/121: rmdir d3 39 2024-09-21T04:35:17.574 INFO:tasks.workunit.client.0.smithi007.stdout:4/228: write d2/f38 [3822495,121417] 0 2024-09-21T04:35:17.576 INFO:tasks.workunit.client.1.smithi053.stdout:6/785: creat d2/d1b/d1e/d2a/d4b/d77/f100 x:0 0 0 2024-09-21T04:35:17.578 INFO:tasks.workunit.client.1.smithi053.stdout:9/781: creat d0/d2/d9/d34/f10e x:0 0 0 2024-09-21T04:35:17.579 INFO:tasks.workunit.client.0.smithi007.stdout:4/229: dread d2/d17/f26 [0,4194304] 0 2024-09-21T04:35:17.579 INFO:tasks.workunit.client.1.smithi053.stdout:8/866: rmdir d3/d5/d4c/d8e 39 2024-09-21T04:35:17.581 INFO:tasks.workunit.client.0.smithi007.stdout:1/295: mknod d5/d6/d21/d23/d55/c5a 0 2024-09-21T04:35:17.581 INFO:tasks.workunit.client.0.smithi007.stdout:1/296: read d5/d45/f4e [3066491,22852] 0 2024-09-21T04:35:17.582 INFO:tasks.workunit.client.1.smithi053.stdout:2/838: rename d1/d18/d1f/d22/cae to d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/c128 0 2024-09-21T04:35:17.584 INFO:tasks.workunit.client.0.smithi007.stdout:3/170: rmdir dc/d22 39 2024-09-21T04:35:17.585 INFO:tasks.workunit.client.1.smithi053.stdout:5/827: dwrite d0/d8/f1d [0,4194304] 0 2024-09-21T04:35:17.586 INFO:tasks.workunit.client.1.smithi053.stdout:5/828: dread - d0/d8/db/d68/ff0 zero size 2024-09-21T04:35:17.587 INFO:tasks.workunit.client.1.smithi053.stdout:8/867: rmdir d3/d5/d38 39 2024-09-21T04:35:17.590 INFO:tasks.workunit.client.1.smithi053.stdout:5/829: write d0/dd/d48/d8c/dad/db9/fcf [2390336,28488] 0 2024-09-21T04:35:17.590 INFO:tasks.workunit.client.1.smithi053.stdout:2/839: creat d1/d18/d1f/d27/d58/db7/f129 x:0 0 0 2024-09-21T04:35:17.590 INFO:tasks.workunit.client.1.smithi053.stdout:8/868: chown d3/d5/d4c/dc3/ddb/de7/fef 1736178847 1 2024-09-21T04:35:17.597 INFO:tasks.workunit.client.1.smithi053.stdout:9/782: dwrite d0/d1c/d25/f54 [0,4194304] 0 2024-09-21T04:35:17.597 INFO:tasks.workunit.client.0.smithi007.stdout:6/122: readlink d3/ld 0 2024-09-21T04:35:17.600 INFO:tasks.workunit.client.1.smithi053.stdout:5/830: creat d0/dd/d67/f10c x:0 0 0 2024-09-21T04:35:17.601 INFO:tasks.workunit.client.0.smithi007.stdout:2/184: creat d0/f27 x:0 0 0 2024-09-21T04:35:17.602 INFO:tasks.workunit.client.1.smithi053.stdout:1/807: dwrite d7/d63/d96/dac/d106/f61 [0,4194304] 0 2024-09-21T04:35:17.602 INFO:tasks.workunit.client.0.smithi007.stdout:2/185: stat d0/d2 0 2024-09-21T04:35:17.603 INFO:tasks.workunit.client.0.smithi007.stdout:4/230: unlink d2/d7/ff 0 2024-09-21T04:35:17.603 INFO:tasks.workunit.client.0.smithi007.stdout:3/171: dwrite dc/d22/f24 [4194304,4194304] 0 2024-09-21T04:35:17.604 INFO:tasks.workunit.client.1.smithi053.stdout:6/786: dread d2/d1d/de0/fa6 [0,4194304] 0 2024-09-21T04:35:17.605 INFO:tasks.workunit.client.1.smithi053.stdout:1/808: write d7/d9/d43/d54/d7f/fe3 [763170,74951] 0 2024-09-21T04:35:17.608 INFO:tasks.workunit.client.1.smithi053.stdout:9/783: creat d0/d1a/d89/da1/dea/f10f x:0 0 0 2024-09-21T04:35:17.610 INFO:tasks.workunit.client.1.smithi053.stdout:1/809: chown d7/d47/fd6 982407798 1 2024-09-21T04:35:17.610 INFO:tasks.workunit.client.1.smithi053.stdout:9/784: chown d0/d1c/d52/df7/d10d 594872 1 2024-09-21T04:35:17.613 INFO:tasks.workunit.client.0.smithi007.stdout:3/172: mknod dc/d18/d26/c2f 0 2024-09-21T04:35:17.613 INFO:tasks.workunit.client.0.smithi007.stdout:1/297: creat d5/f5b x:0 0 0 2024-09-21T04:35:17.615 INFO:tasks.workunit.client.1.smithi053.stdout:6/787: rmdir d2/d1b/d1e/d2a/d4b/d77/d88 39 2024-09-21T04:35:17.616 INFO:tasks.workunit.client.1.smithi053.stdout:9/785: write d0/d1c/d21/d8e/fef [267124,116358] 0 2024-09-21T04:35:17.616 INFO:tasks.workunit.client.1.smithi053.stdout:5/831: dwrite d0/dd/d5c/d92/ddb/d51/d52/ff6 [0,4194304] 0 2024-09-21T04:35:17.616 INFO:tasks.workunit.client.0.smithi007.stdout:4/231: mkdir d2/d17/d4a 0 2024-09-21T04:35:17.619 INFO:tasks.workunit.client.1.smithi053.stdout:6/788: chown d2/d1b/d1e/d2a 232474 1 2024-09-21T04:35:17.619 INFO:tasks.workunit.client.0.smithi007.stdout:3/173: creat dc/d22/d29/d2d/f30 x:0 0 0 2024-09-21T04:35:17.621 INFO:tasks.workunit.client.1.smithi053.stdout:2/840: dwrite d1/d18/d1f/d22/d54/ff6 [0,4194304] 0 2024-09-21T04:35:17.623 INFO:tasks.workunit.client.1.smithi053.stdout:2/841: stat d1/d18/d1f/c70 0 2024-09-21T04:35:17.623 INFO:tasks.workunit.client.0.smithi007.stdout:1/298: dwrite d5/f48 [0,4194304] 0 2024-09-21T04:35:17.625 INFO:tasks.workunit.client.0.smithi007.stdout:4/232: rename d2/d17/d18/f1e to d2/d17/d2e/d45/f4b 0 2024-09-21T04:35:17.627 INFO:tasks.workunit.client.0.smithi007.stdout:4/233: write d2/d3c/f3f [5232926,117498] 0 2024-09-21T04:35:17.631 INFO:tasks.workunit.client.1.smithi053.stdout:9/786: symlink d0/d1a/d89/de3/l110 0 2024-09-21T04:35:17.633 INFO:tasks.workunit.client.0.smithi007.stdout:1/299: dwrite d5/d6/d21/d23/f2a [0,4194304] 0 2024-09-21T04:35:17.634 INFO:tasks.workunit.client.0.smithi007.stdout:1/300: readlink d5/d6/d21/d2b/d4b/l52 0 2024-09-21T04:35:17.637 INFO:tasks.workunit.client.1.smithi053.stdout:5/832: dread d0/d8/db/f78 [0,4194304] 0 2024-09-21T04:35:17.638 INFO:tasks.workunit.client.1.smithi053.stdout:5/833: chown d0/dd/d5c/d89/fb4 86513783 1 2024-09-21T04:35:17.639 INFO:tasks.workunit.client.0.smithi007.stdout:4/234: creat d2/f4c x:0 0 0 2024-09-21T04:35:17.641 INFO:tasks.workunit.client.1.smithi053.stdout:6/789: symlink d2/l101 0 2024-09-21T04:35:17.641 INFO:tasks.workunit.client.1.smithi053.stdout:8/869: truncate d3/d5/d38/f4e 4942038 0 2024-09-21T04:35:17.642 INFO:tasks.workunit.client.0.smithi007.stdout:1/301: dread d5/d6/d21/d23/f24 [0,4194304] 0 2024-09-21T04:35:17.643 INFO:tasks.workunit.client.1.smithi053.stdout:9/787: mkdir d0/d1c/dcb/d111 0 2024-09-21T04:35:17.644 INFO:tasks.workunit.client.1.smithi053.stdout:1/810: getdents d7/da8/d74/d91 0 2024-09-21T04:35:17.645 INFO:tasks.workunit.client.0.smithi007.stdout:6/123: dwrite d3/f18 [0,4194304] 0 2024-09-21T04:35:17.647 INFO:tasks.workunit.client.0.smithi007.stdout:6/124: fsync d3/de/d12/f21 0 2024-09-21T04:35:17.650 INFO:tasks.workunit.client.1.smithi053.stdout:8/870: truncate d3/f93 412627 0 2024-09-21T04:35:17.651 INFO:tasks.workunit.client.0.smithi007.stdout:4/235: dwrite d2/f38 [0,4194304] 0 2024-09-21T04:35:17.654 INFO:tasks.workunit.client.1.smithi053.stdout:6/790: truncate d2/d1b/d1e/d2a/d4b/f58 1010598 0 2024-09-21T04:35:17.654 INFO:tasks.workunit.client.0.smithi007.stdout:1/302: dread d5/fe [0,4194304] 0 2024-09-21T04:35:17.656 INFO:tasks.workunit.client.0.smithi007.stdout:6/125: rename d3/de/c27 to d3/de/d12/d19/c2c 0 2024-09-21T04:35:17.657 INFO:tasks.workunit.client.0.smithi007.stdout:1/303: read d5/d6/d21/f2c [103171,585] 0 2024-09-21T04:35:17.659 INFO:tasks.workunit.client.0.smithi007.stdout:6/126: truncate d3/de/d12/d19/f1c 1078654 0 2024-09-21T04:35:17.659 INFO:tasks.workunit.client.0.smithi007.stdout:6/127: chown d3 4943 1 2024-09-21T04:35:17.659 INFO:tasks.workunit.client.1.smithi053.stdout:2/842: write d1/d18/d1f/d27/d58/d5c/d91/fc9 [33819,97734] 0 2024-09-21T04:35:17.660 INFO:tasks.workunit.client.0.smithi007.stdout:6/128: dread - d3/de/d12/f21 zero size 2024-09-21T04:35:17.661 INFO:tasks.workunit.client.1.smithi053.stdout:9/788: mknod d0/d1c/c112 0 2024-09-21T04:35:17.661 INFO:tasks.workunit.client.0.smithi007.stdout:6/129: dread - d3/f28 zero size 2024-09-21T04:35:17.663 INFO:tasks.workunit.client.0.smithi007.stdout:4/236: symlink d2/d17/l4d 0 2024-09-21T04:35:17.663 INFO:tasks.workunit.client.1.smithi053.stdout:7/702: sync 2024-09-21T04:35:17.664 INFO:tasks.workunit.client.0.smithi007.stdout:4/237: chown d2/d17/f2a 3291361 1 2024-09-21T04:35:17.664 INFO:tasks.workunit.client.1.smithi053.stdout:7/703: rename d0/d6f/dbf to d0/d6f/dbf/d50/d8b/de7 22 2024-09-21T04:35:17.665 INFO:tasks.workunit.client.1.smithi053.stdout:8/871: unlink d3/d5/d4c/d53/d66/c106 0 2024-09-21T04:35:17.666 INFO:tasks.workunit.client.1.smithi053.stdout:1/811: dread d7/d9/da/f50 [0,4194304] 0 2024-09-21T04:35:17.668 INFO:tasks.workunit.client.1.smithi053.stdout:5/834: write d0/dd/fcd [1445195,63576] 0 2024-09-21T04:35:17.670 INFO:tasks.workunit.client.0.smithi007.stdout:6/130: mknod d3/de/d12/c2d 0 2024-09-21T04:35:17.671 INFO:tasks.workunit.client.0.smithi007.stdout:6/131: truncate d3/f18 4970012 0 2024-09-21T04:35:17.671 INFO:tasks.workunit.client.1.smithi053.stdout:2/843: truncate d1/d18/d1f/d27/f55 4353290 0 2024-09-21T04:35:17.672 INFO:tasks.workunit.client.0.smithi007.stdout:6/132: dread d3/f1e [0,4194304] 0 2024-09-21T04:35:17.674 INFO:tasks.workunit.client.0.smithi007.stdout:4/238: rename d2/d17/l32 to d2/d3c/l4e 0 2024-09-21T04:35:17.680 INFO:tasks.workunit.client.1.smithi053.stdout:1/812: mkdir d7/d63/d96/dc7/d107 0 2024-09-21T04:35:17.683 INFO:tasks.workunit.client.1.smithi053.stdout:6/791: truncate d2/d1b/d1e/f4f 469127 0 2024-09-21T04:35:17.684 INFO:tasks.workunit.client.0.smithi007.stdout:6/133: rename d3/de/l22 to d3/de/d12/l2e 0 2024-09-21T04:35:17.686 INFO:tasks.workunit.client.0.smithi007.stdout:1/304: link d5/d6/f8 d5/d6/d21/d2b/f5c 0 2024-09-21T04:35:17.686 INFO:tasks.workunit.client.0.smithi007.stdout:1/305: fdatasync d5/d45/f46 0 2024-09-21T04:35:17.687 INFO:tasks.workunit.client.0.smithi007.stdout:1/306: chown d5/d6/d21/f2c 70351719 1 2024-09-21T04:35:17.689 INFO:tasks.workunit.client.1.smithi053.stdout:2/844: rmdir d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9 39 2024-09-21T04:35:17.692 INFO:tasks.workunit.client.0.smithi007.stdout:1/307: rmdir d5/d45 39 2024-09-21T04:35:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:17 smithi053 ceph-mon[35554]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:35:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:17 smithi053 ceph-mon[35554]: Upgrade: Need to upgrade myself (mgr.smithi053.lnadao) 2024-09-21T04:35:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:17 smithi053 ceph-mon[35554]: Upgrade: Need to upgrade myself (mgr.smithi053.lnadao) 2024-09-21T04:35:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:17 smithi007 ceph-mon[27954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:35:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:17 smithi007 ceph-mon[27954]: Upgrade: Need to upgrade myself (mgr.smithi053.lnadao) 2024-09-21T04:35:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:17 smithi007 ceph-mon[27954]: Upgrade: Need to upgrade myself (mgr.smithi053.lnadao) 2024-09-21T04:35:17.694 INFO:tasks.workunit.client.1.smithi053.stdout:7/704: rename d0/d6f/dbf/d50/c7a to d0/d6f/dbf/d5d/d9f/ce8 0 2024-09-21T04:35:17.698 INFO:tasks.workunit.client.0.smithi007.stdout:1/308: dread d5/d6/d21/f30 [0,4194304] 0 2024-09-21T04:35:17.699 INFO:tasks.workunit.client.1.smithi053.stdout:8/872: mkdir d3/d5/d38/d64/d73/d119 0 2024-09-21T04:35:17.700 INFO:tasks.workunit.client.0.smithi007.stdout:6/134: creat d3/f2f x:0 0 0 2024-09-21T04:35:17.700 INFO:tasks.workunit.client.1.smithi053.stdout:9/789: write d0/d1c/d52/daa/fbc [1392399,105792] 0 2024-09-21T04:35:17.701 INFO:tasks.workunit.client.1.smithi053.stdout:1/813: mknod d7/d47/d79/d99/db0/c108 0 2024-09-21T04:35:17.702 INFO:tasks.workunit.client.1.smithi053.stdout:1/814: read d7/da8/d74/d91/f9a [1654114,44865] 0 2024-09-21T04:35:17.707 INFO:tasks.workunit.client.1.smithi053.stdout:7/705: rename d0/d6f/dbf/d5d/f8c to d0/d2b/d36/d64/d9e/fe9 0 2024-09-21T04:35:17.710 INFO:tasks.workunit.client.1.smithi053.stdout:6/792: unlink d2/d1b/d1e/d2a/d4b/d77/d88/da4/fdf 0 2024-09-21T04:35:17.711 INFO:tasks.workunit.client.1.smithi053.stdout:2/845: mknod d1/d18/d1f/d22/d43/c12a 0 2024-09-21T04:35:17.712 INFO:tasks.workunit.client.0.smithi007.stdout:1/309: write d5/d6/d21/f2c [4670891,32638] 0 2024-09-21T04:35:17.715 INFO:tasks.workunit.client.1.smithi053.stdout:1/815: rename d7/d9/da/d4c/df4/df6 to d7/d9/da/d40/d58/d109 0 2024-09-21T04:35:17.716 INFO:tasks.workunit.client.1.smithi053.stdout:9/790: creat d0/d1c/d52/d6f/dfb/f113 x:0 0 0 2024-09-21T04:35:17.717 INFO:tasks.workunit.client.0.smithi007.stdout:6/135: dwrite d3/de/d12/d19/f20 [0,4194304] 0 2024-09-21T04:35:17.718 INFO:tasks.workunit.client.1.smithi053.stdout:6/793: stat d2/d1b/d1e/d2a/f7c 0 2024-09-21T04:35:17.718 INFO:tasks.workunit.client.0.smithi007.stdout:6/136: chown d3/f15 3 1 2024-09-21T04:35:17.719 INFO:tasks.workunit.client.1.smithi053.stdout:8/873: creat d3/d5/d4c/d53/f11a x:0 0 0 2024-09-21T04:35:17.719 INFO:tasks.workunit.client.1.smithi053.stdout:6/794: chown d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/caf 54 1 2024-09-21T04:35:17.720 INFO:tasks.workunit.client.0.smithi007.stdout:1/310: creat d5/f5d x:0 0 0 2024-09-21T04:35:17.722 INFO:tasks.workunit.client.0.smithi007.stdout:6/137: chown d3/fc 56169 1 2024-09-21T04:35:17.723 INFO:tasks.workunit.client.0.smithi007.stdout:6/138: chown d3/ld 0 1 2024-09-21T04:35:17.725 INFO:tasks.workunit.client.1.smithi053.stdout:7/706: creat d0/d6f/dbf/da2/dd7/fea x:0 0 0 2024-09-21T04:35:17.727 INFO:tasks.workunit.client.0.smithi007.stdout:1/311: rename d5/d6/d21/d23 to d5/d45/d5e 0 2024-09-21T04:35:17.729 INFO:tasks.workunit.client.0.smithi007.stdout:6/139: chown d3/de/d12/d19/c2c 416238 1 2024-09-21T04:35:17.730 INFO:tasks.workunit.client.0.smithi007.stdout:6/140: stat d3/de/d12/d19/f24 0 2024-09-21T04:35:17.732 INFO:tasks.workunit.client.0.smithi007.stdout:6/141: readlink d3/de/d12/l2e 0 2024-09-21T04:35:17.736 INFO:tasks.workunit.client.1.smithi053.stdout:7/707: creat d0/d6f/dbf/d95/feb x:0 0 0 2024-09-21T04:35:17.739 INFO:tasks.workunit.client.1.smithi053.stdout:8/874: write d3/d5/d38/d64/f82 [986477,14159] 0 2024-09-21T04:35:17.739 INFO:tasks.workunit.client.1.smithi053.stdout:6/795: write d2/d1b/d1e/d27/d41/f97 [1617950,129003] 0 2024-09-21T04:35:17.739 INFO:tasks.workunit.client.0.smithi007.stdout:6/142: dwrite d3/f28 [0,4194304] 0 2024-09-21T04:35:17.740 INFO:tasks.workunit.client.1.smithi053.stdout:7/708: symlink d0/d2b/d36/lec 0 2024-09-21T04:35:17.741 INFO:tasks.workunit.client.1.smithi053.stdout:2/846: truncate d1/d18/d1f/d27/d58/f74 3972967 0 2024-09-21T04:35:17.741 INFO:tasks.workunit.client.1.smithi053.stdout:8/875: write d3/d5/f8f [1411005,102364] 0 2024-09-21T04:35:17.742 INFO:tasks.workunit.client.1.smithi053.stdout:1/816: write d7/d9/da/d3c/f65 [1099225,37535] 0 2024-09-21T04:35:17.742 INFO:tasks.workunit.client.0.smithi007.stdout:1/312: write d5/d6/f20 [1882296,9595] 0 2024-09-21T04:35:17.743 INFO:tasks.workunit.client.0.smithi007.stdout:1/313: fsync d5/f5d 0 2024-09-21T04:35:17.746 INFO:tasks.workunit.client.1.smithi053.stdout:2/847: write d1/d18/d1f/da2/dac/f105 [1360100,58228] 0 2024-09-21T04:35:17.748 INFO:tasks.workunit.client.1.smithi053.stdout:6/796: creat d2/d1b/d3d/d49/de/d96/f102 x:0 0 0 2024-09-21T04:35:17.750 INFO:tasks.workunit.client.0.smithi007.stdout:1/314: mknod d5/d45/d5e/d42/c5f 0 2024-09-21T04:35:17.753 INFO:tasks.workunit.client.1.smithi053.stdout:8/876: dwrite d3/df/d49/f55 [0,4194304] 0 2024-09-21T04:35:17.754 INFO:tasks.workunit.client.0.smithi007.stdout:1/315: dread d5/d6/fa [0,4194304] 0 2024-09-21T04:35:17.755 INFO:tasks.workunit.client.1.smithi053.stdout:2/848: rmdir d1/d18/d1f/d27/d58 39 2024-09-21T04:35:17.756 INFO:tasks.workunit.client.1.smithi053.stdout:2/849: chown d1/d18/d1f/d22/d43/f104 518664 1 2024-09-21T04:35:17.758 INFO:tasks.workunit.client.1.smithi053.stdout:4/905: sync 2024-09-21T04:35:17.761 INFO:tasks.workunit.client.0.smithi007.stdout:6/143: dwrite d3/fc [0,4194304] 0 2024-09-21T04:35:17.762 INFO:tasks.workunit.client.0.smithi007.stdout:6/144: rename d3 to d3/de/d30 22 2024-09-21T04:35:17.763 INFO:tasks.workunit.client.1.smithi053.stdout:8/877: mkdir d3/d5/d4b/d5c/d86/dcd/d11b 0 2024-09-21T04:35:17.764 INFO:tasks.workunit.client.1.smithi053.stdout:1/817: dread d7/d63/d96/dac/d106/fa7 [0,4194304] 0 2024-09-21T04:35:17.765 INFO:tasks.workunit.client.1.smithi053.stdout:1/818: fdatasync d7/d9/da/fd9 0 2024-09-21T04:35:17.766 INFO:tasks.workunit.client.0.smithi007.stdout:6/145: creat d3/de/d12/d19/f31 x:0 0 0 2024-09-21T04:35:17.768 INFO:tasks.workunit.client.1.smithi053.stdout:7/709: link d0/d6f/dbf/d5d/d9f/cb4 d0/d6f/ced 0 2024-09-21T04:35:17.769 INFO:tasks.workunit.client.0.smithi007.stdout:6/146: write d3/de/d12/d19/f20 [2781789,74309] 0 2024-09-21T04:35:17.770 INFO:tasks.workunit.client.1.smithi053.stdout:2/850: rename d1/d18/d1f/c26 to d1/d18/d1f/d9d/c12b 0 2024-09-21T04:35:17.772 INFO:tasks.workunit.client.0.smithi007.stdout:1/316: dwrite d5/fe [0,4194304] 0 2024-09-21T04:35:17.773 INFO:tasks.workunit.client.1.smithi053.stdout:6/797: dwrite d2/d1b/d3d/d49/d5c/d8d/fbc [0,4194304] 0 2024-09-21T04:35:17.774 INFO:tasks.workunit.client.1.smithi053.stdout:6/798: fdatasync d2/d1b/d3d/d49/f42 0 2024-09-21T04:35:17.775 INFO:tasks.workunit.client.1.smithi053.stdout:1/819: truncate d7/d47/ff2 1932732 0 2024-09-21T04:35:17.781 INFO:tasks.workunit.client.0.smithi007.stdout:1/317: creat d5/d45/d5e/f60 x:0 0 0 2024-09-21T04:35:17.787 INFO:tasks.workunit.client.1.smithi053.stdout:8/878: creat d3/d5/d1a/f11c x:0 0 0 2024-09-21T04:35:17.788 INFO:tasks.workunit.client.0.smithi007.stdout:1/318: dwrite d5/fe [0,4194304] 0 2024-09-21T04:35:17.788 INFO:tasks.workunit.client.1.smithi053.stdout:8/879: write d3/d5/db1/fe4 [114267,50283] 0 2024-09-21T04:35:17.791 INFO:tasks.workunit.client.0.smithi007.stdout:6/147: getdents d3/de/d12/d19 0 2024-09-21T04:35:17.792 INFO:tasks.workunit.client.0.smithi007.stdout:1/319: mkdir d5/d6/d21/d2b/d61 0 2024-09-21T04:35:17.794 INFO:tasks.workunit.client.1.smithi053.stdout:4/906: rename d1/f2d to d1/dd/f128 0 2024-09-21T04:35:17.796 INFO:tasks.workunit.client.0.smithi007.stdout:1/320: creat d5/d45/d5e/d42/f62 x:0 0 0 2024-09-21T04:35:17.798 INFO:tasks.workunit.client.0.smithi007.stdout:6/148: fsync d3/de/d12/d19/f1b 0 2024-09-21T04:35:17.799 INFO:tasks.workunit.client.1.smithi053.stdout:8/880: creat d3/d5/d4c/dd0/f11d x:0 0 0 2024-09-21T04:35:17.800 INFO:tasks.workunit.client.0.smithi007.stdout:6/149: write d3/de/d12/d19/f24 [787385,89693] 0 2024-09-21T04:35:17.800 INFO:tasks.workunit.client.0.smithi007.stdout:6/150: read - d3/de/d12/f2b zero size 2024-09-21T04:35:17.800 INFO:tasks.workunit.client.1.smithi053.stdout:6/799: dwrite d2/d1b/d3d/d49/de/f43 [0,4194304] 0 2024-09-21T04:35:17.801 INFO:tasks.workunit.client.0.smithi007.stdout:1/321: creat d5/d6/d21/d2b/f63 x:0 0 0 2024-09-21T04:35:17.802 INFO:tasks.workunit.client.0.smithi007.stdout:6/151: dread d3/de/d12/d19/f1b [0,4194304] 0 2024-09-21T04:35:17.804 INFO:tasks.workunit.client.0.smithi007.stdout:1/322: mkdir d5/d6/d21/d64 0 2024-09-21T04:35:17.810 INFO:tasks.workunit.client.1.smithi053.stdout:7/710: rename d0/d6f/dbf/d50/d8b/l12 to d0/d2b/d36/d64/dc0/lee 0 2024-09-21T04:35:17.810 INFO:tasks.workunit.client.1.smithi053.stdout:4/907: mknod d1/dd/de/df5/db6/c129 0 2024-09-21T04:35:17.810 INFO:tasks.workunit.client.0.smithi007.stdout:1/323: rename d5/d6/d21/f30 to d5/d45/d5e/d55/f65 0 2024-09-21T04:35:17.810 INFO:tasks.workunit.client.0.smithi007.stdout:1/324: fdatasync d5/d6/d21/f47 0 2024-09-21T04:35:17.811 INFO:tasks.workunit.client.0.smithi007.stdout:1/325: chown d5/d6/d21/d3c/f43 3810684 1 2024-09-21T04:35:17.816 INFO:tasks.workunit.client.1.smithi053.stdout:7/711: dwrite d0/d6f/dbf/d5c/f85 [0,4194304] 0 2024-09-21T04:35:17.824 INFO:tasks.workunit.client.1.smithi053.stdout:6/800: dread d2/d1b/d3d/d49/de/d32/d80/f4e [4194304,4194304] 0 2024-09-21T04:35:17.824 INFO:tasks.workunit.client.1.smithi053.stdout:1/820: rmdir d7/d9/da/d40/d102 0 2024-09-21T04:35:17.825 INFO:tasks.workunit.client.0.smithi007.stdout:6/152: mknod d3/c32 0 2024-09-21T04:35:17.827 INFO:tasks.workunit.client.0.smithi007.stdout:6/153: write d3/f6 [718883,48891] 0 2024-09-21T04:35:17.831 INFO:tasks.workunit.client.0.smithi007.stdout:1/326: fsync d5/d6/d21/d2b/f2d 0 2024-09-21T04:35:17.831 INFO:tasks.workunit.client.1.smithi053.stdout:0/791: sync 2024-09-21T04:35:17.831 INFO:tasks.workunit.client.1.smithi053.stdout:3/803: sync 2024-09-21T04:35:17.833 INFO:tasks.workunit.client.0.smithi007.stdout:1/327: chown d5/d6/d21/d2b/c3d 7 1 2024-09-21T04:35:17.834 INFO:tasks.workunit.client.1.smithi053.stdout:2/851: dwrite d1/d18/d1f/d27/d58/fb3 [0,4194304] 0 2024-09-21T04:35:17.834 INFO:tasks.workunit.client.0.smithi007.stdout:6/154: dwrite d3/f2f [0,4194304] 0 2024-09-21T04:35:17.845 INFO:tasks.workunit.client.0.smithi007.stdout:6/155: rmdir d3/de 39 2024-09-21T04:35:17.846 INFO:tasks.workunit.client.1.smithi053.stdout:1/821: dread d7/f56 [0,4194304] 0 2024-09-21T04:35:17.849 INFO:tasks.workunit.client.0.smithi007.stdout:1/328: dwrite d5/d6/f4a [0,4194304] 0 2024-09-21T04:35:17.857 INFO:tasks.workunit.client.0.smithi007.stdout:6/156: dwrite d3/de/d12/d19/f1c [0,4194304] 0 2024-09-21T04:35:17.864 INFO:tasks.workunit.client.1.smithi053.stdout:5/835: sync 2024-09-21T04:35:17.864 INFO:tasks.workunit.client.1.smithi053.stdout:9/791: sync 2024-09-21T04:35:17.864 INFO:tasks.workunit.client.1.smithi053.stdout:9/792: chown d0/df/faf 12642 1 2024-09-21T04:35:17.866 INFO:tasks.workunit.client.1.smithi053.stdout:9/793: write d0/d1a/f81 [715648,48253] 0 2024-09-21T04:35:17.867 INFO:tasks.workunit.client.0.smithi007.stdout:6/157: creat d3/f33 x:0 0 0 2024-09-21T04:35:17.871 INFO:tasks.workunit.client.0.smithi007.stdout:6/158: symlink d3/de/d12/l34 0 2024-09-21T04:35:17.872 INFO:tasks.workunit.client.0.smithi007.stdout:1/329: dwrite d5/f17 [0,4194304] 0 2024-09-21T04:35:17.875 INFO:tasks.workunit.client.0.smithi007.stdout:6/159: creat d3/de/d12/d19/f35 x:0 0 0 2024-09-21T04:35:17.880 INFO:tasks.workunit.client.0.smithi007.stdout:1/330: dwrite d5/d45/d5e/f2a [0,4194304] 0 2024-09-21T04:35:17.882 INFO:tasks.workunit.client.0.smithi007.stdout:6/160: creat d3/de/d12/d19/f36 x:0 0 0 2024-09-21T04:35:17.885 INFO:tasks.workunit.client.0.smithi007.stdout:1/331: creat d5/d6/d21/d2b/d4b/f66 x:0 0 0 2024-09-21T04:35:17.887 INFO:tasks.workunit.client.0.smithi007.stdout:6/161: write d3/de/d12/f2b [144645,81426] 0 2024-09-21T04:35:17.892 INFO:tasks.workunit.client.0.smithi007.stdout:1/332: link d5/d45/d5e/d42/f59 d5/d45/d5e/d55/d58/f67 0 2024-09-21T04:35:17.894 INFO:tasks.workunit.client.0.smithi007.stdout:1/333: write d5/fc [782733,39312] 0 2024-09-21T04:35:17.894 INFO:tasks.workunit.client.1.smithi053.stdout:4/908: symlink d1/dd/de/df5/d20/d5b/d87/dd5/d115/d121/db4/l12a 0 2024-09-21T04:35:17.895 INFO:tasks.workunit.client.0.smithi007.stdout:1/334: stat d5/d6/f8 0 2024-09-21T04:35:17.895 INFO:tasks.workunit.client.0.smithi007.stdout:6/162: dwrite d3/de/f10 [0,4194304] 0 2024-09-21T04:35:17.896 INFO:tasks.workunit.client.1.smithi053.stdout:4/909: read - d1/dd/de/df5/d20/d73/d92/dd7/f116 zero size 2024-09-21T04:35:17.897 INFO:tasks.workunit.client.0.smithi007.stdout:6/163: write d3/de/d12/d19/f20 [2782581,4016] 0 2024-09-21T04:35:17.897 INFO:tasks.workunit.client.1.smithi053.stdout:6/801: chown d2/d1d/c24 459 1 2024-09-21T04:35:17.898 INFO:tasks.workunit.client.1.smithi053.stdout:4/910: chown d1/dd/de/df5/d20/dee/ff6 0 1 2024-09-21T04:35:17.901 INFO:tasks.workunit.client.1.smithi053.stdout:3/804: mkdir d16/d17/d21/d42/d43/d125/d12e 0 2024-09-21T04:35:17.903 INFO:tasks.workunit.client.1.smithi053.stdout:3/805: readlink d16/d17/d21/d8b/d7d/d57/d90/l10c 0 2024-09-21T04:35:17.903 INFO:tasks.workunit.client.0.smithi007.stdout:6/164: mknod d3/de/d12/d19/c37 0 2024-09-21T04:35:17.904 INFO:tasks.workunit.client.1.smithi053.stdout:7/712: dwrite d0/d2b/d36/d64/fb5 [0,4194304] 0 2024-09-21T04:35:17.904 INFO:tasks.workunit.client.1.smithi053.stdout:1/822: fdatasync d7/d9/da/d11/f33 0 2024-09-21T04:35:17.905 INFO:tasks.workunit.client.1.smithi053.stdout:2/852: write d1/d18/d1f/d22/d2d/dcd/dcf/f103 [422106,114183] 0 2024-09-21T04:35:17.907 INFO:tasks.workunit.client.1.smithi053.stdout:1/823: read d7/d9/d43/d54/d7f/d36/f4e [2054751,20222] 0 2024-09-21T04:35:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:1/335: link d5/c53 d5/d6/c68 0 2024-09-21T04:35:17.911 INFO:tasks.workunit.client.1.smithi053.stdout:9/794: unlink d0/d1c/d52/d6d/cd2 0 2024-09-21T04:35:17.916 INFO:tasks.workunit.client.0.smithi007.stdout:6/165: rename d3/f18 to d3/f38 0 2024-09-21T04:35:17.920 INFO:tasks.workunit.client.0.smithi007.stdout:6/166: creat d3/de/d12/d19/f39 x:0 0 0 2024-09-21T04:35:17.921 INFO:tasks.workunit.client.0.smithi007.stdout:6/167: fsync d3/f2f 0 2024-09-21T04:35:17.922 INFO:tasks.workunit.client.1.smithi053.stdout:4/911: rename d1/d27/d53/cd1 to d1/dd/de/c12b 0 2024-09-21T04:35:17.927 INFO:tasks.workunit.client.0.smithi007.stdout:6/168: rmdir d3/de/d12 39 2024-09-21T04:35:17.928 INFO:tasks.workunit.client.0.smithi007.stdout:6/169: write d3/f6 [91368,73737] 0 2024-09-21T04:35:17.929 INFO:tasks.workunit.client.0.smithi007.stdout:1/336: getdents d5/d45/d5e/d42 0 2024-09-21T04:35:17.933 INFO:tasks.workunit.client.1.smithi053.stdout:2/853: creat d1/d18/d1f/d22/d2d/dcd/f12c x:0 0 0 2024-09-21T04:35:17.934 INFO:tasks.workunit.client.0.smithi007.stdout:1/337: symlink d5/d6/d21/d2b/d4b/l69 0 2024-09-21T04:35:17.934 INFO:tasks.workunit.client.1.smithi053.stdout:1/824: truncate d7/d47/fa4 1277476 0 2024-09-21T04:35:17.935 INFO:tasks.workunit.client.1.smithi053.stdout:1/825: chown d7/d63/f7d 13795397 1 2024-09-21T04:35:17.935 INFO:tasks.workunit.client.0.smithi007.stdout:6/170: mknod d3/de/c3a 0 2024-09-21T04:35:17.936 INFO:tasks.workunit.client.1.smithi053.stdout:9/795: stat d0/d76/d79/lb4 0 2024-09-21T04:35:17.938 INFO:tasks.workunit.client.1.smithi053.stdout:9/796: chown d0/d2/d4e/d5c/cde 22509577 1 2024-09-21T04:35:17.938 INFO:tasks.workunit.client.1.smithi053.stdout:8/881: write d3/d39/f87 [688302,53022] 0 2024-09-21T04:35:17.942 INFO:tasks.workunit.client.1.smithi053.stdout:4/912: creat d1/d27/df8/f12c x:0 0 0 2024-09-21T04:35:17.943 INFO:tasks.workunit.client.1.smithi053.stdout:5/836: write d0/dd/d5c/d92/ddb/d51/fe2 [272782,55973] 0 2024-09-21T04:35:17.944 INFO:tasks.workunit.client.0.smithi007.stdout:1/338: link d5/d45/d5e/d42/c49 d5/d45/d5e/d42/c6a 0 2024-09-21T04:35:17.944 INFO:tasks.workunit.client.1.smithi053.stdout:0/792: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/fbd [0,4194304] 0 2024-09-21T04:35:17.944 INFO:tasks.workunit.client.0.smithi007.stdout:6/171: dwrite d3/de/d12/f2b [0,4194304] 0 2024-09-21T04:35:17.945 INFO:tasks.workunit.client.0.smithi007.stdout:1/339: fsync d5/fc 0 2024-09-21T04:35:17.949 INFO:tasks.workunit.client.1.smithi053.stdout:1/826: rmdir d7/d47/d79/d99 39 2024-09-21T04:35:17.953 INFO:tasks.workunit.client.0.smithi007.stdout:1/340: rename d5/d45/f4e to d5/d2f/f6b 0 2024-09-21T04:35:17.954 INFO:tasks.workunit.client.1.smithi053.stdout:9/797: mkdir d0/d2/d4e/d114 0 2024-09-21T04:35:17.955 INFO:tasks.workunit.client.0.smithi007.stdout:6/172: mkdir d3/d3b 0 2024-09-21T04:35:17.955 INFO:tasks.workunit.client.0.smithi007.stdout:6/173: stat d3/de/d12/f21 0 2024-09-21T04:35:17.956 INFO:tasks.workunit.client.1.smithi053.stdout:6/802: getdents d2/d1b/d3d/d49/d5c 0 2024-09-21T04:35:17.956 INFO:tasks.workunit.client.1.smithi053.stdout:9/798: write d0/d1a/d4c/d8f/f97 [2802700,114103] 0 2024-09-21T04:35:17.957 INFO:tasks.workunit.client.0.smithi007.stdout:1/341: symlink d5/d45/d5e/d55/l6c 0 2024-09-21T04:35:17.957 INFO:tasks.workunit.client.0.smithi007.stdout:6/174: write d3/de/d12/d19/f24 [583901,9496] 0 2024-09-21T04:35:17.958 INFO:tasks.workunit.client.0.smithi007.stdout:1/342: readlink d5/d6/d21/d2b/d4b/l4f 0 2024-09-21T04:35:17.959 INFO:tasks.workunit.client.1.smithi053.stdout:8/882: creat d3/d5/d4c/d53/d66/f11e x:0 0 0 2024-09-21T04:35:17.960 INFO:tasks.workunit.client.1.smithi053.stdout:6/803: write d2/d1b/d1e/d27/d41/dc9/ff1 [610435,40655] 0 2024-09-21T04:35:17.961 INFO:tasks.workunit.client.1.smithi053.stdout:4/913: creat d1/d27/d53/d88/f12d x:0 0 0 2024-09-21T04:35:17.963 INFO:tasks.workunit.client.1.smithi053.stdout:3/806: dwrite d16/d1b/f1f [0,4194304] 0 2024-09-21T04:35:17.963 INFO:tasks.workunit.client.0.smithi007.stdout:1/343: mknod d5/d6/d21/d2b/c6d 0 2024-09-21T04:35:17.964 INFO:tasks.workunit.client.0.smithi007.stdout:1/344: chown d5/d6/d21 180 1 2024-09-21T04:35:17.964 INFO:tasks.workunit.client.1.smithi053.stdout:3/807: chown d16/d17/d21/d8b/d7d/ff3 1 1 2024-09-21T04:35:17.965 INFO:tasks.workunit.client.1.smithi053.stdout:6/804: read d2/d1b/d3d/d49/d5c/f74 [490238,74426] 0 2024-09-21T04:35:17.966 INFO:tasks.workunit.client.1.smithi053.stdout:5/837: rename d0/d8/d2a/f57 to d0/dd/d5c/d92/f10d 0 2024-09-21T04:35:17.966 INFO:tasks.workunit.client.1.smithi053.stdout:6/805: readlink d2/d1b/d1e/d2a/d4b/d77/l93 0 2024-09-21T04:35:17.967 INFO:tasks.workunit.client.1.smithi053.stdout:6/806: read - d2/d1b/d1e/dc1/fff zero size 2024-09-21T04:35:17.967 INFO:tasks.workunit.client.1.smithi053.stdout:2/854: symlink d1/df2/l12d 0 2024-09-21T04:35:17.969 INFO:tasks.workunit.client.1.smithi053.stdout:6/807: read d2/d1b/d1e/f9e [2039069,85210] 0 2024-09-21T04:35:17.970 INFO:tasks.workunit.client.1.smithi053.stdout:6/808: stat d2/d1b/d1e/d27/d41 0 2024-09-21T04:35:17.971 INFO:tasks.workunit.client.1.smithi053.stdout:6/809: read d2/d1b/d1e/d27/d41/f97 [1660691,53289] 0 2024-09-21T04:35:17.971 INFO:tasks.workunit.client.0.smithi007.stdout:6/175: link d3/de/d12/c26 d3/d3b/c3c 0 2024-09-21T04:35:17.975 INFO:tasks.workunit.client.1.smithi053.stdout:8/883: symlink d3/df/d49/l11f 0 2024-09-21T04:35:17.976 INFO:tasks.workunit.client.0.smithi007.stdout:6/176: stat d3/c7 0 2024-09-21T04:35:17.977 INFO:tasks.workunit.client.1.smithi053.stdout:7/713: getdents d0/dcd 0 2024-09-21T04:35:17.978 INFO:tasks.workunit.client.1.smithi053.stdout:6/810: dread d2/d1b/d3d/d49/d5c/f74 [0,4194304] 0 2024-09-21T04:35:17.979 INFO:tasks.workunit.client.1.smithi053.stdout:7/714: write d0/d6f/dbf/d50/f54 [1021070,123708] 0 2024-09-21T04:35:17.983 INFO:tasks.workunit.client.1.smithi053.stdout:4/914: symlink d1/dd/de/df5/d20/d4e/d51/da4/dc3/l12e 0 2024-09-21T04:35:17.984 INFO:tasks.workunit.client.1.smithi053.stdout:3/808: symlink d16/d17/d21/d42/d43/d125/d127/l12f 0 2024-09-21T04:35:17.985 INFO:tasks.workunit.client.1.smithi053.stdout:5/838: symlink d0/d66/l10e 0 2024-09-21T04:35:17.986 INFO:tasks.workunit.client.1.smithi053.stdout:9/799: rename d0/d1c/d21/l5b to d0/d1a/d4c/d8f/l115 0 2024-09-21T04:35:17.987 INFO:tasks.workunit.client.1.smithi053.stdout:2/855: truncate d1/f6d 954944 0 2024-09-21T04:35:17.990 INFO:tasks.workunit.client.1.smithi053.stdout:7/715: dread d0/d6f/dbf/d50/d8b/f42 [0,4194304] 0 2024-09-21T04:35:17.990 INFO:tasks.workunit.client.1.smithi053.stdout:0/793: mknod d5/dd/d45/c105 0 2024-09-21T04:35:17.990 INFO:tasks.workunit.client.0.smithi007.stdout:1/345: write d5/d45/d5e/d42/f59 [667329,85580] 0 2024-09-21T04:35:17.991 INFO:tasks.workunit.client.1.smithi053.stdout:0/794: chown d5/d10/d4c/db1/dbc/dca/db5 2410264 1 2024-09-21T04:35:17.992 INFO:tasks.workunit.client.1.smithi053.stdout:6/811: fsync d2/d1b/d3d/d49/de/d32/d80/f21 0 2024-09-21T04:35:17.994 INFO:tasks.workunit.client.1.smithi053.stdout:3/809: rmdir d16/d17/d21/d42/d43 39 2024-09-21T04:35:17.996 INFO:tasks.workunit.client.1.smithi053.stdout:5/839: creat d0/dd/d48/d8c/f10f x:0 0 0 2024-09-21T04:35:17.996 INFO:tasks.workunit.client.0.smithi007.stdout:6/177: truncate d3/fc 1218 0 2024-09-21T04:35:17.997 INFO:tasks.workunit.client.0.smithi007.stdout:1/346: creat d5/d6/d21/d2b/d61/f6e x:0 0 0 2024-09-21T04:35:17.997 INFO:tasks.workunit.client.0.smithi007.stdout:6/178: write d3/de/d12/f1f [212880,74975] 0 2024-09-21T04:35:17.998 INFO:tasks.workunit.client.1.smithi053.stdout:6/812: dwrite d2/d1b/d3d/d49/de/d32/d80/ff7 [0,4194304] 0 2024-09-21T04:35:17.999 INFO:tasks.workunit.client.1.smithi053.stdout:8/884: rename d3/df/d14/d16/f23 to d3/df/dba/f120 0 2024-09-21T04:35:18.000 INFO:tasks.workunit.client.1.smithi053.stdout:9/800: symlink d0/d1a/d89/da1/dea/l116 0 2024-09-21T04:35:18.001 INFO:tasks.workunit.client.1.smithi053.stdout:8/885: dread - d3/d5/db1/f101 zero size 2024-09-21T04:35:18.004 INFO:tasks.workunit.client.0.smithi007.stdout:1/347: creat d5/d6/d21/d2b/f6f x:0 0 0 2024-09-21T04:35:18.006 INFO:tasks.workunit.client.1.smithi053.stdout:2/856: dread d1/d18/d77/d9b/fed [0,4194304] 0 2024-09-21T04:35:18.008 INFO:tasks.workunit.client.0.smithi007.stdout:1/348: creat d5/d6/d21/d2b/d4b/f70 x:0 0 0 2024-09-21T04:35:18.010 INFO:tasks.workunit.client.0.smithi007.stdout:1/349: chown d5/d45/d5e/d42/c5f 5117014 1 2024-09-21T04:35:18.012 INFO:tasks.workunit.client.0.smithi007.stdout:6/179: rename d3/d3b/c3c to d3/de/d12/c3d 0 2024-09-21T04:35:18.013 INFO:tasks.workunit.client.0.smithi007.stdout:6/180: write d3/de/d12/d19/f20 [1893815,4033] 0 2024-09-21T04:35:18.013 INFO:tasks.workunit.client.1.smithi053.stdout:0/795: creat d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f106 x:0 0 0 2024-09-21T04:35:18.014 INFO:tasks.workunit.client.0.smithi007.stdout:1/350: rmdir d5/d6/d21 39 2024-09-21T04:35:18.020 INFO:tasks.workunit.client.0.smithi007.stdout:6/181: dwrite d3/de/d12/d19/f36 [0,4194304] 0 2024-09-21T04:35:18.021 INFO:tasks.workunit.client.1.smithi053.stdout:4/915: rename d1/dd/de/df5/d20/d5b/d87/f93 to d1/d27/d53/d88/f12f 0 2024-09-21T04:35:18.024 INFO:tasks.workunit.client.1.smithi053.stdout:4/916: dread d1/d27/d90/fb1 [0,4194304] 0 2024-09-21T04:35:18.029 INFO:tasks.workunit.client.1.smithi053.stdout:9/801: creat d0/d1c/d52/d6d/f117 x:0 0 0 2024-09-21T04:35:18.030 INFO:tasks.workunit.client.0.smithi007.stdout:6/182: mkdir d3/d3e 0 2024-09-21T04:35:18.031 INFO:tasks.workunit.client.0.smithi007.stdout:1/351: mkdir d5/d6/d21/d64/d71 0 2024-09-21T04:35:18.032 INFO:tasks.workunit.client.1.smithi053.stdout:7/716: creat d0/d6f/dbf/d50/dc7/fef x:0 0 0 2024-09-21T04:35:18.032 INFO:tasks.workunit.client.1.smithi053.stdout:9/802: chown d0/d1a/d89/ff4 47913 1 2024-09-21T04:35:18.033 INFO:tasks.workunit.client.0.smithi007.stdout:1/352: chown d5/d45/d5e/d55/c5a 68797083 1 2024-09-21T04:35:18.034 INFO:tasks.workunit.client.1.smithi053.stdout:4/917: dwrite d1/dd/de/df5/d20/d73/d92/f9e [0,4194304] 0 2024-09-21T04:35:18.035 INFO:tasks.workunit.client.1.smithi053.stdout:0/796: read d5/de/d1c/f3d [1269293,101754] 0 2024-09-21T04:35:18.036 INFO:tasks.workunit.client.0.smithi007.stdout:6/183: creat d3/de/f3f x:0 0 0 2024-09-21T04:35:18.036 INFO:tasks.workunit.client.1.smithi053.stdout:3/810: dwrite d16/d17/d21/d26/d3e/fdb [0,4194304] 0 2024-09-21T04:35:18.041 INFO:tasks.workunit.client.0.smithi007.stdout:6/184: creat d3/d3e/f40 x:0 0 0 2024-09-21T04:35:18.042 INFO:tasks.workunit.client.0.smithi007.stdout:6/185: chown d3/de/d12/d19/f20 66010733 1 2024-09-21T04:35:18.043 INFO:tasks.workunit.client.1.smithi053.stdout:6/813: getdents d2/d1b/d1e/d2a/d4b/deb/df5 0 2024-09-21T04:35:18.043 INFO:tasks.workunit.client.1.smithi053.stdout:8/886: mknod d3/d5/d4c/d53/c121 0 2024-09-21T04:35:18.044 INFO:tasks.workunit.client.0.smithi007.stdout:6/186: rmdir d3 39 2024-09-21T04:35:18.045 INFO:tasks.workunit.client.1.smithi053.stdout:2/857: mkdir d1/d18/d1f/d22/d12e 0 2024-09-21T04:35:18.047 INFO:tasks.workunit.client.1.smithi053.stdout:9/803: dread - d0/d2/d9/d34/f82 zero size 2024-09-21T04:35:18.048 INFO:tasks.workunit.client.1.smithi053.stdout:0/797: creat d5/d47/f107 x:0 0 0 2024-09-21T04:35:18.048 INFO:tasks.workunit.client.1.smithi053.stdout:4/918: fdatasync d1/dd/de/df5/d20/d5b/d87/fad 0 2024-09-21T04:35:18.049 INFO:tasks.workunit.client.1.smithi053.stdout:2/858: fsync d1/d18/d1f/d22/d43/da6/f100 0 2024-09-21T04:35:18.050 INFO:tasks.workunit.client.1.smithi053.stdout:3/811: dwrite d16/d17/d21/d26/d3e/f99 [0,4194304] 0 2024-09-21T04:35:18.050 INFO:tasks.workunit.client.1.smithi053.stdout:2/859: dread - d1/d18/d1f/fb8 zero size 2024-09-21T04:35:18.052 INFO:tasks.workunit.client.1.smithi053.stdout:8/887: mknod d3/df/d49/d100/c122 0 2024-09-21T04:35:18.057 INFO:tasks.workunit.client.1.smithi053.stdout:0/798: rmdir d5/de/dfc 39 2024-09-21T04:35:18.058 INFO:tasks.workunit.client.1.smithi053.stdout:1/827: sync 2024-09-21T04:35:18.059 INFO:tasks.workunit.client.1.smithi053.stdout:7/717: creat d0/ff0 x:0 0 0 2024-09-21T04:35:18.060 INFO:tasks.workunit.client.0.smithi007.stdout:1/353: truncate d5/f28 4415290 0 2024-09-21T04:35:18.061 INFO:tasks.workunit.client.0.smithi007.stdout:1/354: chown d5/f27 193295195 1 2024-09-21T04:35:18.061 INFO:tasks.workunit.client.0.smithi007.stdout:1/355: dread - d5/d45/d5e/d42/f62 zero size 2024-09-21T04:35:18.063 INFO:tasks.workunit.client.0.smithi007.stdout:1/356: truncate d5/d6/d21/d2b/f63 319980 0 2024-09-21T04:35:18.063 INFO:tasks.workunit.client.0.smithi007.stdout:1/357: fdatasync d5/f25 0 2024-09-21T04:35:18.106 INFO:tasks.workunit.client.1.smithi053.stdout:3/812: rename d16/d17/d21/d60/fd6 to d16/d17/d21/d26/d2b/dbb/f130 0 2024-09-21T04:35:18.106 INFO:tasks.workunit.client.1.smithi053.stdout:5/840: truncate d0/dd/d5c/d92/ddb/d51/d52/fa7 1287026 0 2024-09-21T04:35:18.111 INFO:tasks.workunit.client.0.smithi007.stdout:1/358: rename d5/d6/d21/c54 to d5/d6/d21/d64/d71/c72 0 2024-09-21T04:35:18.114 INFO:tasks.workunit.client.0.smithi007.stdout:1/359: fsync d5/d6/fa 0 2024-09-21T04:35:18.116 INFO:tasks.workunit.client.1.smithi053.stdout:6/814: write d2/d1b/d1e/d2a/d4b/d77/d88/fab [257063,125417] 0 2024-09-21T04:35:18.119 INFO:tasks.workunit.client.0.smithi007.stdout:6/187: truncate d3/de/d12/d19/f24 144673 0 2024-09-21T04:35:18.120 INFO:tasks.workunit.client.0.smithi007.stdout:6/188: write d3/de/d12/d19/f39 [112040,91332] 0 2024-09-21T04:35:18.121 INFO:tasks.workunit.client.0.smithi007.stdout:1/360: rename d5/f27 to d5/d45/d5e/d55/d58/f73 0 2024-09-21T04:35:18.129 INFO:tasks.workunit.client.0.smithi007.stdout:6/189: mknod d3/d3b/c41 0 2024-09-21T04:35:18.130 INFO:tasks.workunit.client.1.smithi053.stdout:9/804: link d0/d1c/d21/c106 d0/d76/d79/c118 0 2024-09-21T04:35:18.131 INFO:tasks.workunit.client.0.smithi007.stdout:1/361: creat d5/d6/d21/d2b/d4b/f74 x:0 0 0 2024-09-21T04:35:18.132 INFO:tasks.workunit.client.0.smithi007.stdout:1/362: truncate d5/f5d 699599 0 2024-09-21T04:35:18.132 INFO:tasks.workunit.client.1.smithi053.stdout:1/828: symlink d7/d9/d43/d54/l10a 0 2024-09-21T04:35:18.132 INFO:tasks.workunit.client.0.smithi007.stdout:1/363: chown d5/d45/d5e/d42/c5f 1855643 1 2024-09-21T04:35:18.134 INFO:tasks.workunit.client.0.smithi007.stdout:1/364: truncate d5/d45/f46 4639640 0 2024-09-21T04:35:18.136 INFO:tasks.workunit.client.1.smithi053.stdout:4/919: dwrite d1/dd/de/df5/d20/d73/f79 [0,4194304] 0 2024-09-21T04:35:18.136 INFO:tasks.workunit.client.1.smithi053.stdout:7/718: symlink d0/d6f/dbf/d5c/dac/lf1 0 2024-09-21T04:35:18.138 INFO:tasks.workunit.client.1.smithi053.stdout:3/813: dread d16/d1b/f59 [0,4194304] 0 2024-09-21T04:35:18.139 INFO:tasks.workunit.client.0.smithi007.stdout:1/365: creat d5/d6/d21/d2b/d4b/f75 x:0 0 0 2024-09-21T04:35:18.140 INFO:tasks.workunit.client.1.smithi053.stdout:3/814: chown d16/d17 234244701 1 2024-09-21T04:35:18.140 INFO:tasks.workunit.client.1.smithi053.stdout:2/860: dwrite d1/d18/d1f/d27/d41/d4b/f79 [0,4194304] 0 2024-09-21T04:35:18.143 INFO:tasks.workunit.client.0.smithi007.stdout:6/190: creat d3/de/f42 x:0 0 0 2024-09-21T04:35:18.143 INFO:tasks.workunit.client.0.smithi007.stdout:1/366: dread d5/d45/d5e/f24 [0,4194304] 0 2024-09-21T04:35:18.165 INFO:tasks.workunit.client.0.smithi007.stdout:6/191: chown d3/f33 31 1 2024-09-21T04:35:18.165 INFO:tasks.workunit.client.0.smithi007.stdout:6/192: readlink d3/l4 0 2024-09-21T04:35:18.165 INFO:tasks.workunit.client.0.smithi007.stdout:6/193: dread - d3/d3e/f40 zero size 2024-09-21T04:35:18.165 INFO:tasks.workunit.client.0.smithi007.stdout:6/194: rename d3/de/d12/d19/f31 to d3/d3e/f43 0 2024-09-21T04:35:18.165 INFO:tasks.workunit.client.0.smithi007.stdout:6/195: fsync d3/fc 0 2024-09-21T04:35:18.166 INFO:tasks.workunit.client.1.smithi053.stdout:5/841: symlink d0/d8/d2a/dc8/l110 0 2024-09-21T04:35:18.166 INFO:tasks.workunit.client.1.smithi053.stdout:9/805: rmdir d0/d2/d9/d34/d58 39 2024-09-21T04:35:18.166 INFO:tasks.workunit.client.1.smithi053.stdout:9/806: stat d0/d2/d2b/cf0 0 2024-09-21T04:35:18.166 INFO:tasks.workunit.client.1.smithi053.stdout:0/799: link d5/d10/d4c/db1/dbc/dca/lf3 d5/d10/d4c/db1/dbc/d8c/l108 0 2024-09-21T04:35:18.166 INFO:tasks.workunit.client.1.smithi053.stdout:1/829: dwrite d7/d47/d79/d99/fe4 [0,4194304] 0 2024-09-21T04:35:18.166 INFO:tasks.workunit.client.1.smithi053.stdout:1/830: chown d7/d9/d43/d54/d7f/d36/l44 6717213 1 2024-09-21T04:35:18.166 INFO:tasks.workunit.client.1.smithi053.stdout:4/920: readlink d1/dd/de/df5/ldd 0 2024-09-21T04:35:18.167 INFO:tasks.workunit.client.0.smithi007.stdout:6/196: dread - d3/de/d12/f21 zero size 2024-09-21T04:35:18.167 INFO:tasks.workunit.client.0.smithi007.stdout:1/367: write d5/d6/f31 [781097,27612] 0 2024-09-21T04:35:18.168 INFO:tasks.workunit.client.1.smithi053.stdout:9/807: dread d0/d2/fa [0,4194304] 0 2024-09-21T04:35:18.171 INFO:tasks.workunit.client.1.smithi053.stdout:2/861: rmdir d1/df2 39 2024-09-21T04:35:18.172 INFO:tasks.workunit.client.1.smithi053.stdout:2/862: write d1/d18/d1f/da2/dac/fc5 [4698131,16417] 0 2024-09-21T04:35:18.173 INFO:tasks.workunit.client.0.smithi007.stdout:1/368: creat d5/d45/d5e/f76 x:0 0 0 2024-09-21T04:35:18.174 INFO:tasks.workunit.client.1.smithi053.stdout:3/815: symlink d16/d17/d21/d8b/d7d/d57/df5/l131 0 2024-09-21T04:35:18.175 INFO:tasks.workunit.client.0.smithi007.stdout:6/197: dwrite d3/de/f42 [0,4194304] 0 2024-09-21T04:35:18.178 INFO:tasks.workunit.client.1.smithi053.stdout:8/888: rename d3/d5/c92 to d3/d5/d38/c123 0 2024-09-21T04:35:18.180 INFO:tasks.workunit.client.1.smithi053.stdout:5/842: mkdir d0/dd/d48/d8c/dad/db9/deb/d111 0 2024-09-21T04:35:18.186 INFO:tasks.workunit.client.1.smithi053.stdout:7/719: unlink d0/d6f/dbf/d5c/l71 0 2024-09-21T04:35:18.187 INFO:tasks.workunit.client.1.smithi053.stdout:1/831: fsync d7/d9/d43/d54/d7f/d36/fd3 0 2024-09-21T04:35:18.188 INFO:tasks.workunit.client.1.smithi053.stdout:4/921: creat d1/dd/de/df5/d20/d5b/d87/dd5/f130 x:0 0 0 2024-09-21T04:35:18.189 INFO:tasks.workunit.client.1.smithi053.stdout:6/815: link d2/d1d/f54 d2/d1b/d3d/d49/de/d32/f103 0 2024-09-21T04:35:18.192 INFO:tasks.workunit.client.0.smithi007.stdout:1/369: dread d5/f15 [0,4194304] 0 2024-09-21T04:35:18.192 INFO:tasks.workunit.client.1.smithi053.stdout:6/816: fsync d2/d1b/d3d/d49/de/d32/d80/f57 0 2024-09-21T04:35:18.193 INFO:tasks.workunit.client.1.smithi053.stdout:3/816: unlink d16/d1b/f75 0 2024-09-21T04:35:18.194 INFO:tasks.workunit.client.1.smithi053.stdout:5/843: rename d0/dd/d5c/d92/ddb/d51/d52/fbb to d0/d8/db6/dbc/df4/f112 0 2024-09-21T04:35:18.195 INFO:tasks.workunit.client.0.smithi007.stdout:1/370: mknod d5/d6/d21/d64/d71/c77 0 2024-09-21T04:35:18.196 INFO:tasks.workunit.client.0.smithi007.stdout:1/371: write d5/f5b [576906,31602] 0 2024-09-21T04:35:18.197 INFO:tasks.workunit.client.0.smithi007.stdout:1/372: chown d5/d6/d21/d2b/d4b/l69 2 1 2024-09-21T04:35:18.198 INFO:tasks.workunit.client.1.smithi053.stdout:2/863: dwrite d1/d18/d1f/d27/d58/d5c/d91/dc1/f10e [0,4194304] 0 2024-09-21T04:35:18.199 INFO:tasks.workunit.client.1.smithi053.stdout:7/720: fsync d0/d6f/dbf/f6b 0 2024-09-21T04:35:18.201 INFO:tasks.workunit.client.1.smithi053.stdout:3/817: dwrite d16/d17/d21/d8b/d7d/dc6/f11a [0,4194304] 0 2024-09-21T04:35:18.202 INFO:tasks.workunit.client.1.smithi053.stdout:9/808: link d0/d1c/d52/daa/ffa d0/d1c/d25/f119 0 2024-09-21T04:35:18.203 INFO:tasks.workunit.client.1.smithi053.stdout:9/809: chown d0/d1c/d52/f93 20134616 1 2024-09-21T04:35:18.203 INFO:tasks.workunit.client.1.smithi053.stdout:9/810: chown d0/d1c/d52/d6d 7 1 2024-09-21T04:35:18.205 INFO:tasks.workunit.client.1.smithi053.stdout:6/817: mkdir d2/d1b/d3d/d104 0 2024-09-21T04:35:18.213 INFO:tasks.workunit.client.1.smithi053.stdout:0/800: truncate d5/dd/f58 2681343 0 2024-09-21T04:35:18.216 INFO:tasks.workunit.client.1.smithi053.stdout:1/832: symlink d7/l10b 0 2024-09-21T04:35:18.217 INFO:tasks.workunit.client.1.smithi053.stdout:1/833: chown d7/d9/d43/f6e 1897 1 2024-09-21T04:35:18.218 INFO:tasks.workunit.client.1.smithi053.stdout:1/834: chown d7/d9/da/d3c/f46 1321228038 1 2024-09-21T04:35:18.219 INFO:tasks.workunit.client.1.smithi053.stdout:4/922: creat d1/dd/de/df5/d20/d73/d10b/d41/de0/f131 x:0 0 0 2024-09-21T04:35:18.220 INFO:tasks.workunit.client.1.smithi053.stdout:3/818: rmdir d16/d17/d21/d8b/d7d/d9d 39 2024-09-21T04:35:18.221 INFO:tasks.workunit.client.1.smithi053.stdout:3/819: fdatasync d16/d1b/dc3/fb2 0 2024-09-21T04:35:18.221 INFO:tasks.workunit.client.1.smithi053.stdout:4/923: truncate d1/dd/de/df5/d3f/d74/de1/fed 310677 0 2024-09-21T04:35:18.222 INFO:tasks.workunit.client.0.smithi007.stdout:6/198: dwrite d3/de/d12/d19/f1b [0,4194304] 0 2024-09-21T04:35:18.223 INFO:tasks.workunit.client.1.smithi053.stdout:3/820: write d16/d17/d21/d26/d2b/f113 [774839,130396] 0 2024-09-21T04:35:18.227 INFO:tasks.workunit.client.0.smithi007.stdout:6/199: rmdir d3/d3e 39 2024-09-21T04:35:18.228 INFO:tasks.workunit.client.1.smithi053.stdout:6/818: rmdir d2/d1b/d3d/d49/de/d32 39 2024-09-21T04:35:18.228 INFO:tasks.workunit.client.1.smithi053.stdout:6/819: dread - d2/d1b/d3d/fe1 zero size 2024-09-21T04:35:18.229 INFO:tasks.workunit.client.1.smithi053.stdout:6/820: stat d2/d1b/d3d/d49/de/d96/c67 0 2024-09-21T04:35:18.230 INFO:tasks.workunit.client.0.smithi007.stdout:1/373: write d5/f28 [187613,47979] 0 2024-09-21T04:35:18.231 INFO:tasks.workunit.client.1.smithi053.stdout:0/801: rmdir d5/d10 39 2024-09-21T04:35:18.233 INFO:tasks.workunit.client.1.smithi053.stdout:2/864: mkdir d1/d18/d1f/d22/d12f 0 2024-09-21T04:35:18.235 INFO:tasks.workunit.client.1.smithi053.stdout:2/865: chown d1/d18/d1f/d22/d43/f4f 0 1 2024-09-21T04:35:18.235 INFO:tasks.workunit.client.0.smithi007.stdout:6/200: dwrite d3/de/d12/d19/f39 [0,4194304] 0 2024-09-21T04:35:18.236 INFO:tasks.workunit.client.0.smithi007.stdout:1/374: creat d5/d45/d5e/d42/f78 x:0 0 0 2024-09-21T04:35:18.237 INFO:tasks.workunit.client.1.smithi053.stdout:1/835: chown d7/d47/la5 12 1 2024-09-21T04:35:18.242 INFO:tasks.workunit.client.0.smithi007.stdout:6/201: mknod d3/d3b/c44 0 2024-09-21T04:35:18.243 INFO:tasks.workunit.client.0.smithi007.stdout:6/202: chown d3/de/d12/c2d 3077812 1 2024-09-21T04:35:18.244 INFO:tasks.workunit.client.0.smithi007.stdout:1/375: creat d5/d6/d21/d2b/d4b/f79 x:0 0 0 2024-09-21T04:35:18.246 INFO:tasks.workunit.client.0.smithi007.stdout:1/376: write d5/d45/d5e/d42/f78 [1013655,21901] 0 2024-09-21T04:35:18.247 INFO:tasks.workunit.client.1.smithi053.stdout:9/811: creat d0/d1c/d52/df7/d10d/f11a x:0 0 0 2024-09-21T04:35:18.247 INFO:tasks.workunit.client.1.smithi053.stdout:9/812: stat d0/d2/f13 0 2024-09-21T04:35:18.248 INFO:tasks.workunit.client.1.smithi053.stdout:8/889: getdents d3/d5/db1 0 2024-09-21T04:35:18.253 INFO:tasks.workunit.client.1.smithi053.stdout:0/802: readlink d5/d10/lcc 0 2024-09-21T04:35:18.255 INFO:tasks.workunit.client.0.smithi007.stdout:1/377: dwrite d5/d6/d21/d2b/d4b/f79 [0,4194304] 0 2024-09-21T04:35:18.257 INFO:tasks.workunit.client.0.smithi007.stdout:1/378: chown d5/d45/d5e/d55/d58/f67 7535 1 2024-09-21T04:35:18.257 INFO:tasks.workunit.client.1.smithi053.stdout:5/844: creat d0/dd/d48/d8c/dad/f113 x:0 0 0 2024-09-21T04:35:18.259 INFO:tasks.workunit.client.1.smithi053.stdout:3/821: write d16/d17/d21/d42/da0/dfb/d11e/f105 [601280,67019] 0 2024-09-21T04:35:18.262 INFO:tasks.workunit.client.0.smithi007.stdout:1/379: creat d5/d6/d21/d64/d71/f7a x:0 0 0 2024-09-21T04:35:18.264 INFO:tasks.workunit.client.1.smithi053.stdout:6/821: write d2/d1b/d3d/f6c [4513784,29271] 0 2024-09-21T04:35:18.266 INFO:tasks.workunit.client.0.smithi007.stdout:1/380: creat d5/d45/d5e/d42/f7b x:0 0 0 2024-09-21T04:35:18.266 INFO:tasks.workunit.client.0.smithi007.stdout:1/381: readlink d5/l2e 0 2024-09-21T04:35:18.267 INFO:tasks.workunit.client.0.smithi007.stdout:6/203: getdents d3/d3b 0 2024-09-21T04:35:18.269 INFO:tasks.workunit.client.1.smithi053.stdout:1/836: rename d7/d63/d6b/f8a to d7/d63/d96/de2/f10c 0 2024-09-21T04:35:18.269 INFO:tasks.workunit.client.1.smithi053.stdout:1/837: chown d7/da8/d74/dce 917372 1 2024-09-21T04:35:18.271 INFO:tasks.workunit.client.1.smithi053.stdout:4/924: mkdir d1/d132 0 2024-09-21T04:35:18.272 INFO:tasks.workunit.client.0.smithi007.stdout:1/382: mknod d5/d2f/c7c 0 2024-09-21T04:35:18.273 INFO:tasks.workunit.client.1.smithi053.stdout:7/721: truncate d0/fd 3499039 0 2024-09-21T04:35:18.273 INFO:tasks.workunit.client.0.smithi007.stdout:1/383: write d5/fc [209559,49762] 0 2024-09-21T04:35:18.274 INFO:tasks.workunit.client.0.smithi007.stdout:1/384: read - d5/d6/d21/d2b/d4b/f70 zero size 2024-09-21T04:35:18.275 INFO:tasks.workunit.client.0.smithi007.stdout:6/204: rename d3/de/d12/d19/f23 to d3/d3b/f45 0 2024-09-21T04:35:18.278 INFO:tasks.workunit.client.0.smithi007.stdout:1/385: creat d5/d45/f7d x:0 0 0 2024-09-21T04:35:18.281 INFO:tasks.workunit.client.0.smithi007.stdout:1/386: mknod d5/d45/d5e/d55/d58/c7e 0 2024-09-21T04:35:18.283 INFO:tasks.workunit.client.1.smithi053.stdout:7/722: dread d0/d6f/dbf/d50/d8b/f1b [0,4194304] 0 2024-09-21T04:35:18.283 INFO:tasks.workunit.client.0.smithi007.stdout:1/387: read d5/d45/d5e/d42/f56 [2762693,14149] 0 2024-09-21T04:35:18.284 INFO:tasks.workunit.client.1.smithi053.stdout:3/822: creat d16/d17/d21/d8b/d7d/d57/dc1/f132 x:0 0 0 2024-09-21T04:35:18.288 INFO:tasks.workunit.client.0.smithi007.stdout:1/388: dread d5/d45/d5e/f2a [0,4194304] 0 2024-09-21T04:35:18.288 INFO:tasks.workunit.client.1.smithi053.stdout:6/822: creat d2/d1b/d3d/de9/f105 x:0 0 0 2024-09-21T04:35:18.291 INFO:tasks.workunit.client.1.smithi053.stdout:2/866: creat d1/d18/d1f/d27/d58/db7/d112/d115/f130 x:0 0 0 2024-09-21T04:35:18.292 INFO:tasks.workunit.client.1.smithi053.stdout:1/838: creat d7/d9/da/d40/d49/d77/f10d x:0 0 0 2024-09-21T04:35:18.293 INFO:tasks.workunit.client.1.smithi053.stdout:1/839: read - d7/d24/fe1 zero size 2024-09-21T04:35:18.293 INFO:tasks.workunit.client.0.smithi007.stdout:1/389: symlink d5/d45/d5e/l7f 0 2024-09-21T04:35:18.296 INFO:tasks.workunit.client.1.smithi053.stdout:4/925: creat d1/dd/de/df5/d20/d73/d92/dd7/f133 x:0 0 0 2024-09-21T04:35:18.296 INFO:tasks.workunit.client.0.smithi007.stdout:1/390: fdatasync d5/d6/d21/d3c/f43 0 2024-09-21T04:35:18.297 INFO:tasks.workunit.client.0.smithi007.stdout:6/205: dwrite d3/fc [0,4194304] 0 2024-09-21T04:35:18.298 INFO:tasks.workunit.client.1.smithi053.stdout:9/813: symlink d0/d2/d9/d34/db2/l11b 0 2024-09-21T04:35:18.298 INFO:tasks.workunit.client.0.smithi007.stdout:1/391: chown d5/d45/d5e/d42/c5f 3947 1 2024-09-21T04:35:18.298 INFO:tasks.workunit.client.1.smithi053.stdout:9/814: dread - d0/d1c/dcb/fd9 zero size 2024-09-21T04:35:18.299 INFO:tasks.workunit.client.1.smithi053.stdout:9/815: stat d0/d1c/d52/daa 0 2024-09-21T04:35:18.300 INFO:tasks.workunit.client.1.smithi053.stdout:8/890: symlink d3/d5/d4b/d5c/d86/dcd/d11b/l124 0 2024-09-21T04:35:18.301 INFO:tasks.workunit.client.1.smithi053.stdout:0/803: symlink d5/d10/d4c/db1/dbc/dca/d21/l109 0 2024-09-21T04:35:18.304 INFO:tasks.workunit.client.1.smithi053.stdout:5/845: symlink d0/dd/d48/d8c/dad/db9/deb/df7/l114 0 2024-09-21T04:35:18.304 INFO:tasks.workunit.client.0.smithi007.stdout:1/392: unlink d5/d6/d21/d2b/d4b/l4f 0 2024-09-21T04:35:18.305 INFO:tasks.workunit.client.0.smithi007.stdout:1/393: write d5/d6/f31 [248619,69468] 0 2024-09-21T04:35:18.306 INFO:tasks.workunit.client.1.smithi053.stdout:5/846: dread - d0/dd/d48/d8c/dad/f113 zero size 2024-09-21T04:35:18.306 INFO:tasks.workunit.client.1.smithi053.stdout:7/723: mkdir d0/d6f/dbf/d50/d8b/d70/df2 0 2024-09-21T04:35:18.307 INFO:tasks.workunit.client.0.smithi007.stdout:1/394: chown d5/d6/d21/d2b/d4b/f79 64997517 1 2024-09-21T04:35:18.307 INFO:tasks.workunit.client.0.smithi007.stdout:6/206: mkdir d3/d3e/d46 0 2024-09-21T04:35:18.307 INFO:tasks.workunit.client.1.smithi053.stdout:9/816: dwrite d0/d2/d9/d1f/f4f [4194304,4194304] 0 2024-09-21T04:35:18.314 INFO:tasks.workunit.client.1.smithi053.stdout:0/804: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/f98 [4194304,4194304] 0 2024-09-21T04:35:18.314 INFO:tasks.workunit.client.1.smithi053.stdout:0/805: write d5/d10/d4c/db1/dbc/dca/db5/d99/fce [1137322,42494] 0 2024-09-21T04:35:18.315 INFO:tasks.workunit.client.0.smithi007.stdout:1/395: rename d5/d45/d5e/f2a to d5/d45/d5e/d55/f80 0 2024-09-21T04:35:18.315 INFO:tasks.workunit.client.0.smithi007.stdout:1/396: chown d5/d45/d5e/l7f 16199 1 2024-09-21T04:35:18.316 INFO:tasks.workunit.client.0.smithi007.stdout:9/257: sync 2024-09-21T04:35:18.316 INFO:tasks.workunit.client.0.smithi007.stdout:7/271: sync 2024-09-21T04:35:18.320 INFO:tasks.workunit.client.1.smithi053.stdout:8/891: truncate d3/d5/d38/d64/d9d/da9/fb4 364368 0 2024-09-21T04:35:18.320 INFO:tasks.workunit.client.0.smithi007.stdout:6/207: link d3/de/d12/d19/c2c d3/d3e/d46/c47 0 2024-09-21T04:35:18.324 INFO:tasks.workunit.client.1.smithi053.stdout:5/847: creat d0/d8/db6/dbc/f115 x:0 0 0 2024-09-21T04:35:18.324 INFO:tasks.workunit.client.0.smithi007.stdout:1/397: dwrite d5/d6/d21/d2b/d61/f6e [0,4194304] 0 2024-09-21T04:35:18.326 INFO:tasks.workunit.client.0.smithi007.stdout:9/258: creat d13/d18/f55 x:0 0 0 2024-09-21T04:35:18.327 INFO:tasks.workunit.client.0.smithi007.stdout:1/398: rename d5/d6/d21/d2b/d4b to d5/d6/d21/d2b/d4b/d81 22 2024-09-21T04:35:18.333 INFO:tasks.workunit.client.0.smithi007.stdout:6/208: creat d3/d3e/f48 x:0 0 0 2024-09-21T04:35:18.334 INFO:tasks.workunit.client.1.smithi053.stdout:6/823: symlink d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/l106 0 2024-09-21T04:35:18.335 INFO:tasks.workunit.client.0.smithi007.stdout:1/399: dwrite d5/f5d [0,4194304] 0 2024-09-21T04:35:18.335 INFO:tasks.workunit.client.0.smithi007.stdout:6/209: chown d3/d3e/f48 979499 1 2024-09-21T04:35:18.336 INFO:tasks.workunit.client.0.smithi007.stdout:1/400: stat d5/d6/d21/d3c/f43 0 2024-09-21T04:35:18.337 INFO:tasks.workunit.client.1.smithi053.stdout:4/926: write d1/dd/de/df5/d20/d4e/faa [4380466,9640] 0 2024-09-21T04:35:18.338 INFO:tasks.workunit.client.0.smithi007.stdout:9/259: creat d13/d2a/f56 x:0 0 0 2024-09-21T04:35:18.339 INFO:tasks.workunit.client.1.smithi053.stdout:2/867: symlink d1/d18/d1f/d22/d2d/dcd/dee/d121/l131 0 2024-09-21T04:35:18.340 INFO:tasks.workunit.client.0.smithi007.stdout:9/260: write d13/f23 [756695,21491] 0 2024-09-21T04:35:18.341 INFO:tasks.workunit.client.1.smithi053.stdout:1/840: mkdir d7/d47/d79/d10e 0 2024-09-21T04:35:18.345 INFO:tasks.workunit.client.0.smithi007.stdout:7/272: mkdir d3/d6/d2e/d63 0 2024-09-21T04:35:18.346 INFO:tasks.workunit.client.1.smithi053.stdout:3/823: write d16/f30 [677189,37731] 0 2024-09-21T04:35:18.347 INFO:tasks.workunit.client.0.smithi007.stdout:7/273: write d3/d6/d13/d4b/f57 [3121821,60373] 0 2024-09-21T04:35:18.351 INFO:tasks.workunit.client.1.smithi053.stdout:9/817: dwrite d0/d2/d9/f19 [0,4194304] 0 2024-09-21T04:35:18.353 INFO:tasks.workunit.client.0.smithi007.stdout:9/261: write d13/f1d [777791,92163] 0 2024-09-21T04:35:18.357 INFO:tasks.workunit.client.1.smithi053.stdout:4/927: creat d1/d27/d90/f134 x:0 0 0 2024-09-21T04:35:18.358 INFO:tasks.workunit.client.1.smithi053.stdout:4/928: stat d1/dd/de/df5/d20 0 2024-09-21T04:35:18.358 INFO:tasks.workunit.client.1.smithi053.stdout:2/868: truncate d1/f7b 494466 0 2024-09-21T04:35:18.361 INFO:tasks.workunit.client.0.smithi007.stdout:6/210: creat d3/de/f49 x:0 0 0 2024-09-21T04:35:18.361 INFO:tasks.workunit.client.0.smithi007.stdout:6/211: read d3/f6 [3461925,40698] 0 2024-09-21T04:35:18.362 INFO:tasks.workunit.client.0.smithi007.stdout:6/212: dread - d3/d3e/f40 zero size 2024-09-21T04:35:18.363 INFO:tasks.workunit.client.0.smithi007.stdout:6/213: chown d3/c32 30002 1 2024-09-21T04:35:18.364 INFO:tasks.workunit.client.1.smithi053.stdout:8/892: mkdir d3/d5/d38/d64/d73/d119/d125 0 2024-09-21T04:35:18.364 INFO:tasks.workunit.client.0.smithi007.stdout:7/274: link d3/d6/d13/f5d d3/d6/d10/d43/f64 0 2024-09-21T04:35:18.366 INFO:tasks.workunit.client.1.smithi053.stdout:3/824: rmdir d16/d17/d21/d8b/d7d/d57/dc1/d53/d82 39 2024-09-21T04:35:18.366 INFO:tasks.workunit.client.0.smithi007.stdout:1/401: link d5/d45/d5e/d55/l6c d5/d45/d5e/d55/l82 0 2024-09-21T04:35:18.367 INFO:tasks.workunit.client.0.smithi007.stdout:1/402: fdatasync d5/fc 0 2024-09-21T04:35:18.367 INFO:tasks.workunit.client.0.smithi007.stdout:1/403: fdatasync d5/f5d 0 2024-09-21T04:35:18.369 INFO:tasks.workunit.client.1.smithi053.stdout:7/724: symlink d0/d2b/d36/d64/lf3 0 2024-09-21T04:35:18.369 INFO:tasks.workunit.client.1.smithi053.stdout:6/824: dread d2/d1b/d3d/d49/de/d32/f103 [0,4194304] 0 2024-09-21T04:35:18.370 INFO:tasks.workunit.client.1.smithi053.stdout:5/848: fdatasync d0/d8/db/d68/f8b 0 2024-09-21T04:35:18.372 INFO:tasks.workunit.client.0.smithi007.stdout:9/262: write d13/f1e [4871991,122409] 0 2024-09-21T04:35:18.372 INFO:tasks.workunit.client.1.smithi053.stdout:9/818: symlink d0/d1c/d21/d46/l11c 0 2024-09-21T04:35:18.373 INFO:tasks.workunit.client.0.smithi007.stdout:6/214: mknod d3/de/d12/d19/c4a 0 2024-09-21T04:35:18.374 INFO:tasks.workunit.client.1.smithi053.stdout:9/819: fsync d0/d2/d9/d1f/f4f 0 2024-09-21T04:35:18.375 INFO:tasks.workunit.client.1.smithi053.stdout:4/929: creat d1/dd/de/df5/d20/d4e/d51/da4/dc3/f135 x:0 0 0 2024-09-21T04:35:18.376 INFO:tasks.workunit.client.1.smithi053.stdout:0/806: dwrite d5/d10/d4c/db1/dbc/dca/d21/fa0 [0,4194304] 0 2024-09-21T04:35:18.382 INFO:tasks.workunit.client.1.smithi053.stdout:8/893: rename d3/d39/f3f to d3/d5/d4c/dc3/f126 0 2024-09-21T04:35:18.383 INFO:tasks.workunit.client.0.smithi007.stdout:1/404: rename d5/d2f/f6b to d5/d6/d21/d2b/d4b/f83 0 2024-09-21T04:35:18.385 INFO:tasks.workunit.client.0.smithi007.stdout:1/405: chown d5/d6/d21/d2b/f6f 7 1 2024-09-21T04:35:18.385 INFO:tasks.workunit.client.1.smithi053.stdout:3/825: truncate d16/d17/d21/f64 795882 0 2024-09-21T04:35:18.385 INFO:tasks.workunit.client.0.smithi007.stdout:1/406: chown d5/d6/d21/d2b/f2d 1747 1 2024-09-21T04:35:18.388 INFO:tasks.workunit.client.0.smithi007.stdout:9/263: symlink d13/d2a/l57 0 2024-09-21T04:35:18.390 INFO:tasks.workunit.client.1.smithi053.stdout:5/849: dread - d0/dd/d5c/d80/fd7 zero size 2024-09-21T04:35:18.392 INFO:tasks.workunit.client.0.smithi007.stdout:1/407: symlink d5/d6/d21/d2b/d61/l84 0 2024-09-21T04:35:18.394 INFO:tasks.workunit.client.0.smithi007.stdout:7/275: dwrite d3/d6/d14/d20/d33/f25 [4194304,4194304] 0 2024-09-21T04:35:18.396 INFO:tasks.workunit.client.1.smithi053.stdout:9/820: dread d0/d6c/fa7 [0,4194304] 0 2024-09-21T04:35:18.396 INFO:tasks.workunit.client.0.smithi007.stdout:7/276: dread - d3/d6/d10/d4f/f5c zero size 2024-09-21T04:35:18.396 INFO:tasks.workunit.client.1.smithi053.stdout:0/807: dread d5/de/d42/fa4 [0,4194304] 0 2024-09-21T04:35:18.397 INFO:tasks.workunit.client.1.smithi053.stdout:2/869: mkdir d1/d18/d1f/d27/d58/d5c/d132 0 2024-09-21T04:35:18.401 INFO:tasks.workunit.client.0.smithi007.stdout:1/408: fsync d5/d45/d5e/d42/f56 0 2024-09-21T04:35:18.402 INFO:tasks.workunit.client.0.smithi007.stdout:1/409: fdatasync d5/d6/d21/d3c/f43 0 2024-09-21T04:35:18.404 INFO:tasks.workunit.client.0.smithi007.stdout:1/410: chown d5/d6/d21/d2b/d4b/f70 43233086 1 2024-09-21T04:35:18.404 INFO:tasks.workunit.client.0.smithi007.stdout:9/264: dread d13/d18/f4b [0,4194304] 0 2024-09-21T04:35:18.405 INFO:tasks.workunit.client.0.smithi007.stdout:1/411: write d5/d45/d5e/f76 [501019,71648] 0 2024-09-21T04:35:18.406 INFO:tasks.workunit.client.0.smithi007.stdout:9/265: write fa [112658,112211] 0 2024-09-21T04:35:18.407 INFO:tasks.workunit.client.0.smithi007.stdout:9/266: dread - d13/d2a/f52 zero size 2024-09-21T04:35:18.409 INFO:tasks.workunit.client.1.smithi053.stdout:7/725: symlink d0/dcd/lf4 0 2024-09-21T04:35:18.409 INFO:tasks.workunit.client.0.smithi007.stdout:6/215: link d3/c32 d3/c4b 0 2024-09-21T04:35:18.411 INFO:tasks.workunit.client.1.smithi053.stdout:6/825: getdents d2/d1b/d1e/d2a/d4b/deb/df5 0 2024-09-21T04:35:18.413 INFO:tasks.workunit.client.0.smithi007.stdout:3/174: sync 2024-09-21T04:35:18.413 INFO:tasks.workunit.client.0.smithi007.stdout:8/156: sync 2024-09-21T04:35:18.413 INFO:tasks.workunit.client.0.smithi007.stdout:0/222: sync 2024-09-21T04:35:18.413 INFO:tasks.workunit.client.0.smithi007.stdout:4/239: sync 2024-09-21T04:35:18.413 INFO:tasks.workunit.client.0.smithi007.stdout:2/186: sync 2024-09-21T04:35:18.413 INFO:tasks.workunit.client.0.smithi007.stdout:5/491: sync 2024-09-21T04:35:18.414 INFO:tasks.workunit.client.1.smithi053.stdout:5/850: mknod d0/dd/d5c/d92/de6/c116 0 2024-09-21T04:35:18.414 INFO:tasks.workunit.client.0.smithi007.stdout:3/175: readlink dc/l20 0 2024-09-21T04:35:18.415 INFO:tasks.workunit.client.0.smithi007.stdout:1/412: symlink d5/d6/d21/d3c/l85 0 2024-09-21T04:35:18.416 INFO:tasks.workunit.client.1.smithi053.stdout:3/826: write d16/d17/faf [2275874,122060] 0 2024-09-21T04:35:18.416 INFO:tasks.workunit.client.0.smithi007.stdout:8/157: write d0/d3/d16/f1b [359167,29102] 0 2024-09-21T04:35:18.417 INFO:tasks.workunit.client.0.smithi007.stdout:9/267: mknod d13/d18/c58 0 2024-09-21T04:35:18.418 INFO:tasks.workunit.client.0.smithi007.stdout:1/413: write d5/d6/d21/d2b/d4b/f74 [452007,118293] 0 2024-09-21T04:35:18.420 INFO:tasks.workunit.client.0.smithi007.stdout:7/277: write d3/d6/d2e/f38 [59345,101849] 0 2024-09-21T04:35:18.420 INFO:tasks.workunit.client.1.smithi053.stdout:0/808: symlink d5/d10/d4c/db1/dbc/l10a 0 2024-09-21T04:35:18.421 INFO:tasks.workunit.client.1.smithi053.stdout:2/870: fsync d1/d18/d1f/d22/d2d/ffa 0 2024-09-21T04:35:18.422 INFO:tasks.workunit.client.1.smithi053.stdout:0/809: chown d5/d10/d4c/db1/dbc/dca/db5/fd6 21 1 2024-09-21T04:35:18.423 INFO:tasks.workunit.client.0.smithi007.stdout:2/187: creat d0/d11/f28 x:0 0 0 2024-09-21T04:35:18.423 INFO:tasks.workunit.client.0.smithi007.stdout:2/188: fdatasync d0/f23 0 2024-09-21T04:35:18.430 INFO:tasks.workunit.client.1.smithi053.stdout:5/851: creat d0/d8/d2a/f117 x:0 0 0 2024-09-21T04:35:18.431 INFO:tasks.workunit.client.1.smithi053.stdout:7/726: write d0/d2b/d36/f5e [35691,100159] 0 2024-09-21T04:35:18.432 INFO:tasks.workunit.client.0.smithi007.stdout:6/216: truncate d3/f1e 435906 0 2024-09-21T04:35:18.434 INFO:tasks.workunit.client.0.smithi007.stdout:9/268: mknod d13/d2a/c59 0 2024-09-21T04:35:18.434 INFO:tasks.workunit.client.0.smithi007.stdout:7/278: dread d3/d6/d14/f45 [0,4194304] 0 2024-09-21T04:35:18.435 INFO:tasks.workunit.client.0.smithi007.stdout:1/414: mkdir d5/d6/d21/d2b/d86 0 2024-09-21T04:35:18.435 INFO:tasks.workunit.client.0.smithi007.stdout:9/269: fsync d13/d2a/f2d 0 2024-09-21T04:35:18.436 INFO:tasks.workunit.client.0.smithi007.stdout:5/492: mkdir d2/d27/d3d/d9b 0 2024-09-21T04:35:18.438 INFO:tasks.workunit.client.0.smithi007.stdout:6/217: dread d3/f28 [0,4194304] 0 2024-09-21T04:35:18.438 INFO:tasks.workunit.client.1.smithi053.stdout:2/871: rename d1/fdd to d1/d18/f133 0 2024-09-21T04:35:18.439 INFO:tasks.workunit.client.0.smithi007.stdout:5/493: write d2/d1e/f24 [13115395,105686] 0 2024-09-21T04:35:18.439 INFO:tasks.workunit.client.1.smithi053.stdout:2/872: write d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/f107 [2148902,30733] 0 2024-09-21T04:35:18.442 INFO:tasks.workunit.client.1.smithi053.stdout:8/894: link d3/d5/d38/fe0 d3/d5/d4c/d8e/f127 0 2024-09-21T04:35:18.443 INFO:tasks.workunit.client.0.smithi007.stdout:1/415: dread d5/f1f [0,4194304] 0 2024-09-21T04:35:18.443 INFO:tasks.workunit.client.0.smithi007.stdout:4/240: mkdir d2/d17/d4a/d4f 0 2024-09-21T04:35:18.445 INFO:tasks.workunit.client.1.smithi053.stdout:6/826: mknod d2/d1b/d3d/d49/de/c107 0 2024-09-21T04:35:18.449 INFO:tasks.workunit.client.0.smithi007.stdout:0/223: truncate d6/d16/f2a 1597233 0 2024-09-21T04:35:18.450 INFO:tasks.workunit.client.1.smithi053.stdout:7/727: unlink d0/d6f/dbf/d50/l66 0 2024-09-21T04:35:18.451 INFO:tasks.workunit.client.0.smithi007.stdout:8/158: symlink d0/l28 0 2024-09-21T04:35:18.454 INFO:tasks.workunit.client.1.smithi053.stdout:4/930: getdents d1/dd/de/df5/d3f/d74/de1 0 2024-09-21T04:35:18.455 INFO:tasks.workunit.client.0.smithi007.stdout:7/279: chown d3/d6/d13/c23 249835 1 2024-09-21T04:35:18.456 INFO:tasks.workunit.client.0.smithi007.stdout:3/176: truncate dc/f15 248309 0 2024-09-21T04:35:18.460 INFO:tasks.workunit.client.1.smithi053.stdout:2/873: dread - d1/d18/d1f/d22/d43/da6/fe7 zero size 2024-09-21T04:35:18.461 INFO:tasks.workunit.client.0.smithi007.stdout:5/494: symlink d2/dd/d23/d49/d44/l9c 0 2024-09-21T04:35:18.462 INFO:tasks.workunit.client.1.smithi053.stdout:2/874: write d1/d18/d1f/d22/d2d/dcd/dcf/f103 [3643406,36754] 0 2024-09-21T04:35:18.463 INFO:tasks.workunit.client.0.smithi007.stdout:5/495: write d2/dd/d23/d49/f80 [4488133,63581] 0 2024-09-21T04:35:18.467 INFO:tasks.workunit.client.0.smithi007.stdout:5/496: write d2/d27/d42/d53/f90 [102358,25082] 0 2024-09-21T04:35:18.468 INFO:tasks.workunit.client.1.smithi053.stdout:2/875: dwrite d1/d18/d1f/d27/d58/d5c/d91/dd0/fa4 [0,4194304] 0 2024-09-21T04:35:18.471 INFO:tasks.workunit.client.1.smithi053.stdout:5/852: mknod d0/dd/d5c/d92/ddb/c118 0 2024-09-21T04:35:18.472 INFO:tasks.workunit.client.0.smithi007.stdout:3/177: dread dc/f14 [0,4194304] 0 2024-09-21T04:35:18.473 INFO:tasks.workunit.client.0.smithi007.stdout:5/497: dread d2/dd/d23/d2a/f2f [0,4194304] 0 2024-09-21T04:35:18.473 INFO:tasks.workunit.client.0.smithi007.stdout:3/178: chown dc/d22/d29/d2d/f30 153 1 2024-09-21T04:35:18.474 INFO:tasks.workunit.client.0.smithi007.stdout:0/224: rmdir d6/d16/d18/d27 39 2024-09-21T04:35:18.480 INFO:tasks.workunit.client.0.smithi007.stdout:5/498: write d2/dd/d23/d49/d44/d54/f59 [2912313,117792] 0 2024-09-21T04:35:18.481 INFO:tasks.workunit.client.1.smithi053.stdout:8/895: creat d3/df/d14/d16/d117/f128 x:0 0 0 2024-09-21T04:35:18.483 INFO:tasks.workunit.client.0.smithi007.stdout:8/159: fsync d0/d3/d8/de/f18 0 2024-09-21T04:35:18.484 INFO:tasks.workunit.client.0.smithi007.stdout:4/241: write d2/d17/d2e/f33 [1678633,88953] 0 2024-09-21T04:35:18.486 INFO:tasks.workunit.client.0.smithi007.stdout:7/280: creat d3/d6/d10/d4f/f65 x:0 0 0 2024-09-21T04:35:18.488 INFO:tasks.workunit.client.0.smithi007.stdout:9/270: mkdir d13/d5a 0 2024-09-21T04:35:18.488 INFO:tasks.workunit.client.0.smithi007.stdout:4/242: write d2/f38 [5407988,61057] 0 2024-09-21T04:35:18.489 INFO:tasks.workunit.client.1.smithi053.stdout:2/876: mkdir d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/d134 0 2024-09-21T04:35:18.489 INFO:tasks.workunit.client.0.smithi007.stdout:5/499: truncate d2/dd/d23/d49/d44/f84 930902 0 2024-09-21T04:35:18.492 INFO:tasks.workunit.client.0.smithi007.stdout:3/179: dwrite dc/f13 [4194304,4194304] 0 2024-09-21T04:35:18.492 INFO:tasks.workunit.client.0.smithi007.stdout:1/416: dwrite d5/d45/d5e/f24 [0,4194304] 0 2024-09-21T04:35:18.493 INFO:tasks.workunit.client.0.smithi007.stdout:3/180: chown dc/d18 7 1 2024-09-21T04:35:18.493 INFO:tasks.workunit.client.1.smithi053.stdout:5/853: rename d0/dd/d5c/d92/de8 to d0/d8/db6/dbc/df4/d119 0 2024-09-21T04:35:18.495 INFO:tasks.workunit.client.0.smithi007.stdout:6/218: creat d3/d3e/d46/f4c x:0 0 0 2024-09-21T04:35:18.500 INFO:tasks.workunit.client.1.smithi053.stdout:1/841: sync 2024-09-21T04:35:18.501 INFO:tasks.workunit.client.1.smithi053.stdout:2/877: unlink d1/d18/d1f/d22/d2d/dcd/dcf/ffb 0 2024-09-21T04:35:18.502 INFO:tasks.workunit.client.1.smithi053.stdout:5/854: dread - d0/dd/d5c/d89/dc5/fce zero size 2024-09-21T04:35:18.503 INFO:tasks.workunit.client.0.smithi007.stdout:2/189: rename d0/d2/f21 to d0/f29 0 2024-09-21T04:35:18.504 INFO:tasks.workunit.client.1.smithi053.stdout:5/855: write d0/d8/d2a/f108 [371001,21668] 0 2024-09-21T04:35:18.504 INFO:tasks.workunit.client.1.smithi053.stdout:7/728: mkdir d0/dca/df5 0 2024-09-21T04:35:18.506 INFO:tasks.workunit.client.1.smithi053.stdout:3/827: getdents d16/d17/d21/d42/d43/d125/d127 0 2024-09-21T04:35:18.508 INFO:tasks.workunit.client.1.smithi053.stdout:3/828: write d16/d17/faf [862679,43404] 0 2024-09-21T04:35:18.509 INFO:tasks.workunit.client.0.smithi007.stdout:0/225: creat d6/d16/d18/d39/f54 x:0 0 0 2024-09-21T04:35:18.509 INFO:tasks.workunit.client.1.smithi053.stdout:6/827: link d2/d1b/d3d/d49/de/d32/d80/dcd/ldb d2/d1b/d1e/d27/d41/dc9/l108 0 2024-09-21T04:35:18.511 INFO:tasks.workunit.client.1.smithi053.stdout:1/842: creat d7/d9/d43/f10f x:0 0 0 2024-09-21T04:35:18.512 INFO:tasks.workunit.client.1.smithi053.stdout:6/828: read d2/d1b/d1e/d2a/d4b/d77/d88/fca [1674865,38458] 0 2024-09-21T04:35:18.512 INFO:tasks.workunit.client.1.smithi053.stdout:1/843: fsync d7/d63/d6b/f8d 0 2024-09-21T04:35:18.513 INFO:tasks.workunit.client.1.smithi053.stdout:5/856: creat d0/d8/d2a/dc8/f11a x:0 0 0 2024-09-21T04:35:18.514 INFO:tasks.workunit.client.1.smithi053.stdout:1/844: chown d7/d9/d43/d54/d7f/d36/fd3 11 1 2024-09-21T04:35:18.515 INFO:tasks.workunit.client.1.smithi053.stdout:6/829: truncate d2/d1b/d1e/d2a/d4b/fd9 9174714 0 2024-09-21T04:35:18.517 INFO:tasks.workunit.client.1.smithi053.stdout:1/845: truncate d7/d9/da/d4c/ffa 203563 0 2024-09-21T04:35:18.518 INFO:tasks.workunit.client.0.smithi007.stdout:8/160: symlink d0/d3/d16/l29 0 2024-09-21T04:35:18.519 INFO:tasks.workunit.client.1.smithi053.stdout:8/896: truncate d3/d5/d38/d64/d9d/fdf 1290959 0 2024-09-21T04:35:18.522 INFO:tasks.workunit.client.0.smithi007.stdout:4/243: creat d2/d17/d2e/f50 x:0 0 0 2024-09-21T04:35:18.523 INFO:tasks.workunit.client.0.smithi007.stdout:4/244: truncate d2/d17/d2e/d45/f4b 4981225 0 2024-09-21T04:35:18.524 INFO:tasks.workunit.client.0.smithi007.stdout:5/500: symlink d2/dd/d23/d49/d6f/l9d 0 2024-09-21T04:35:18.524 INFO:tasks.workunit.client.0.smithi007.stdout:4/245: truncate d2/d7/f48 239417 0 2024-09-21T04:35:18.525 INFO:tasks.workunit.client.0.smithi007.stdout:4/246: write d2/d17/d18/f2d [1793581,11910] 0 2024-09-21T04:35:18.526 INFO:tasks.workunit.client.1.smithi053.stdout:5/857: symlink d0/d66/l11b 0 2024-09-21T04:35:18.527 INFO:tasks.workunit.client.0.smithi007.stdout:1/417: rmdir d5/d45/d5e 39 2024-09-21T04:35:18.527 INFO:tasks.workunit.client.0.smithi007.stdout:4/247: write d2/f1d [1305442,42330] 0 2024-09-21T04:35:18.529 INFO:tasks.workunit.client.0.smithi007.stdout:4/248: chown d2/d17/d29 34940 1 2024-09-21T04:35:18.530 INFO:tasks.workunit.client.0.smithi007.stdout:4/249: readlink d2/d17/d2e/l40 0 2024-09-21T04:35:18.533 INFO:tasks.workunit.client.1.smithi053.stdout:1/846: truncate d7/d47/f4d 159873 0 2024-09-21T04:35:18.534 INFO:tasks.workunit.client.0.smithi007.stdout:5/501: dwrite d2/dd/d23/d2a/f75 [0,4194304] 0 2024-09-21T04:35:18.536 INFO:tasks.workunit.client.1.smithi053.stdout:3/829: rename d16/d17/d21/d8b/d7d/dc6/dec/cf9 to d16/d17/d21/d42/d43/d125/c133 0 2024-09-21T04:35:18.536 INFO:tasks.workunit.client.0.smithi007.stdout:2/190: mknod d0/d2/c2a 0 2024-09-21T04:35:18.537 INFO:tasks.workunit.client.0.smithi007.stdout:2/191: chown d0/d2/c15 245 1 2024-09-21T04:35:18.539 INFO:tasks.workunit.client.1.smithi053.stdout:2/878: link d1/d18/d1f/d22/d54/ca5 d1/d18/c135 0 2024-09-21T04:35:18.541 INFO:tasks.workunit.client.1.smithi053.stdout:6/830: write d2/d1b/d3d/d49/de/f73 [92088,85855] 0 2024-09-21T04:35:18.541 INFO:tasks.workunit.client.1.smithi053.stdout:7/729: link d0/d2b/d36/d64/fb5 d0/d6f/dbf/d50/d8b/d70/dc1/ff6 0 2024-09-21T04:35:18.542 INFO:tasks.workunit.client.1.smithi053.stdout:2/879: write d1/d18/d1f/fba [4759338,92281] 0 2024-09-21T04:35:18.542 INFO:tasks.workunit.client.0.smithi007.stdout:9/271: unlink d13/f4e 0 2024-09-21T04:35:18.543 INFO:tasks.workunit.client.0.smithi007.stdout:9/272: chown d13/d2a/f47 606 1 2024-09-21T04:35:18.544 INFO:tasks.workunit.client.0.smithi007.stdout:7/281: mkdir d3/d6/d10/d62/d66 0 2024-09-21T04:35:18.545 INFO:tasks.workunit.client.1.smithi053.stdout:6/831: fdatasync d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f8a 0 2024-09-21T04:35:18.545 INFO:tasks.workunit.client.0.smithi007.stdout:5/502: dread d2/d27/d3d/f85 [0,4194304] 0 2024-09-21T04:35:18.545 INFO:tasks.workunit.client.1.smithi053.stdout:3/830: rmdir d16/d17/d21/d42/d43/d86 39 2024-09-21T04:35:18.546 INFO:tasks.workunit.client.1.smithi053.stdout:7/730: read d0/d2b/f43 [444143,55158] 0 2024-09-21T04:35:18.546 INFO:tasks.workunit.client.0.smithi007.stdout:5/503: chown d2/dd/d1d/f39 7840 1 2024-09-21T04:35:18.546 INFO:tasks.workunit.client.1.smithi053.stdout:6/832: chown d2/d1b/d3d/d49/f11 205849783 1 2024-09-21T04:35:18.548 INFO:tasks.workunit.client.0.smithi007.stdout:5/504: truncate d2/dd/d23/d49/f4a 1342302 0 2024-09-21T04:35:18.549 INFO:tasks.workunit.client.1.smithi053.stdout:2/880: symlink d1/d18/d1f/da2/l136 0 2024-09-21T04:35:18.550 INFO:tasks.workunit.client.1.smithi053.stdout:2/881: chown d1/d18/d1f/da2/dac/f105 4 1 2024-09-21T04:35:18.550 INFO:tasks.workunit.client.0.smithi007.stdout:4/250: unlink d2/f38 0 2024-09-21T04:35:18.554 INFO:tasks.workunit.client.0.smithi007.stdout:2/192: rename d0/f3 to d0/d2/f2b 0 2024-09-21T04:35:18.555 INFO:tasks.workunit.client.0.smithi007.stdout:0/226: symlink d6/d16/l55 0 2024-09-21T04:35:18.557 INFO:tasks.workunit.client.1.smithi053.stdout:7/731: creat d0/dca/ff7 x:0 0 0 2024-09-21T04:35:18.558 INFO:tasks.workunit.client.0.smithi007.stdout:8/161: fdatasync d0/d3/d16/f21 0 2024-09-21T04:35:18.560 INFO:tasks.workunit.client.0.smithi007.stdout:7/282: unlink d3/d6/d10/l1c 0 2024-09-21T04:35:18.562 INFO:tasks.workunit.client.0.smithi007.stdout:1/418: symlink d5/d45/d5e/l87 0 2024-09-21T04:35:18.562 INFO:tasks.workunit.client.1.smithi053.stdout:7/732: dread d0/d6f/dbf/d50/d84/f9c [0,4194304] 0 2024-09-21T04:35:18.563 INFO:tasks.workunit.client.0.smithi007.stdout:1/419: write d5/d6/d21/f2c [1552216,26799] 0 2024-09-21T04:35:18.568 INFO:tasks.workunit.client.1.smithi053.stdout:8/897: rename d3/d5/l15 to d3/df/d14/l129 0 2024-09-21T04:35:18.569 INFO:tasks.workunit.client.1.smithi053.stdout:8/898: write d3/df/f113 [21508,66803] 0 2024-09-21T04:35:18.570 INFO:tasks.workunit.client.1.smithi053.stdout:8/899: write d3/f3a [683942,129518] 0 2024-09-21T04:35:18.571 INFO:tasks.workunit.client.0.smithi007.stdout:6/219: link d3/de/d12/d19/f35 d3/de/f4d 0 2024-09-21T04:35:18.573 INFO:tasks.workunit.client.1.smithi053.stdout:9/821: sync 2024-09-21T04:35:18.575 INFO:tasks.workunit.client.1.smithi053.stdout:1/847: getdents d7 0 2024-09-21T04:35:18.577 INFO:tasks.workunit.client.1.smithi053.stdout:7/733: creat d0/d6f/dbf/d5c/ff8 x:0 0 0 2024-09-21T04:35:18.578 INFO:tasks.workunit.client.0.smithi007.stdout:4/251: write d2/d7/f8 [697701,38589] 0 2024-09-21T04:35:18.578 INFO:tasks.workunit.client.0.smithi007.stdout:2/193: symlink d0/d2/l2c 0 2024-09-21T04:35:18.579 INFO:tasks.workunit.client.1.smithi053.stdout:2/882: symlink d1/d18/d1f/d22/d2d/deb/l137 0 2024-09-21T04:35:18.580 INFO:tasks.workunit.client.0.smithi007.stdout:2/194: read d0/d11/f20 [146962,11849] 0 2024-09-21T04:35:18.582 INFO:tasks.workunit.client.1.smithi053.stdout:3/831: write d16/d17/d21/d8b/d7d/d9d/f115 [109238,13925] 0 2024-09-21T04:35:18.585 INFO:tasks.workunit.client.1.smithi053.stdout:8/900: unlink d3/d5/d4b/f7e 0 2024-09-21T04:35:18.585 INFO:tasks.workunit.client.0.smithi007.stdout:3/181: getdents dc/d22/d29/d2d 0 2024-09-21T04:35:18.585 INFO:tasks.workunit.client.0.smithi007.stdout:3/182: chown dc/d18 278558 1 2024-09-21T04:35:18.586 INFO:tasks.workunit.client.0.smithi007.stdout:5/505: mkdir d2/d27/d3d/d9e 0 2024-09-21T04:35:18.586 INFO:tasks.workunit.client.0.smithi007.stdout:5/506: fsync d2/f8d 0 2024-09-21T04:35:18.586 INFO:tasks.workunit.client.1.smithi053.stdout:1/848: mknod d7/d47/c110 0 2024-09-21T04:35:18.587 INFO:tasks.workunit.client.0.smithi007.stdout:1/420: symlink d5/d6/d21/d2b/d61/l88 0 2024-09-21T04:35:18.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/421: dread d5/d6/d21/d2b/f2d [0,4194304] 0 2024-09-21T04:35:18.592 INFO:tasks.workunit.client.1.smithi053.stdout:2/883: mkdir d1/d18/d1f/d27/d71/d138 0 2024-09-21T04:35:18.594 INFO:tasks.workunit.client.1.smithi053.stdout:5/858: rename d0/dd/d5c/d92/ddb/d51/d52/fa7 to d0/dd/d6d/f11c 0 2024-09-21T04:35:18.594 INFO:tasks.workunit.client.0.smithi007.stdout:6/220: chown d3/de/d12/c3d 5074 1 2024-09-21T04:35:18.595 INFO:tasks.workunit.client.1.smithi053.stdout:5/859: dread - d0/d8/db6/dbc/df4/d119/f105 zero size 2024-09-21T04:35:18.595 INFO:tasks.workunit.client.1.smithi053.stdout:5/860: stat d0/dd/d67/c79 0 2024-09-21T04:35:18.596 INFO:tasks.workunit.client.1.smithi053.stdout:3/832: read d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fbf [106948,53934] 0 2024-09-21T04:35:18.599 INFO:tasks.workunit.client.1.smithi053.stdout:8/901: dread - d3/d5/d4b/d5c/fa6 zero size 2024-09-21T04:35:18.601 INFO:tasks.workunit.client.0.smithi007.stdout:6/221: dwrite d3/de/f49 [0,4194304] 0 2024-09-21T04:35:18.601 INFO:tasks.workunit.client.1.smithi053.stdout:3/833: dwrite d16/d17/d21/d8b/f66 [0,4194304] 0 2024-09-21T04:35:18.602 INFO:tasks.workunit.client.1.smithi053.stdout:1/849: symlink d7/d9/da/d40/d49/l111 0 2024-09-21T04:35:18.607 INFO:tasks.workunit.client.0.smithi007.stdout:4/252: write d2/f12 [5097099,117899] 0 2024-09-21T04:35:18.608 INFO:tasks.workunit.client.0.smithi007.stdout:4/253: fsync d2/f6 0 2024-09-21T04:35:18.608 INFO:tasks.workunit.client.0.smithi007.stdout:2/195: symlink d0/d11/l2d 0 2024-09-21T04:35:18.609 INFO:tasks.workunit.client.0.smithi007.stdout:4/254: stat d2/d17/d4a/d4f 0 2024-09-21T04:35:18.609 INFO:tasks.workunit.client.1.smithi053.stdout:7/734: dwrite d0/d2b/d36/f87 [0,4194304] 0 2024-09-21T04:35:18.612 INFO:tasks.workunit.client.1.smithi053.stdout:6/833: rename d2/d1b/d3d/d49/de/d32/d80/f57 to d2/d1b/d3d/d49/de/d32/d80/dcd/f109 0 2024-09-21T04:35:18.612 INFO:tasks.workunit.client.1.smithi053.stdout:2/884: read d1/d18/d1f/d27/d41/f64 [3057317,102836] 0 2024-09-21T04:35:18.614 INFO:tasks.workunit.client.1.smithi053.stdout:0/810: sync 2024-09-21T04:35:18.615 INFO:tasks.workunit.client.1.smithi053.stdout:4/931: sync 2024-09-21T04:35:18.617 INFO:tasks.workunit.client.1.smithi053.stdout:0/811: write d5/de/d42/f48 [343548,3329] 0 2024-09-21T04:35:18.619 INFO:tasks.workunit.client.0.smithi007.stdout:4/255: dwrite d2/d7/fa [4194304,4194304] 0 2024-09-21T04:35:18.621 INFO:tasks.workunit.client.0.smithi007.stdout:9/273: rename d13/d2a/f52 to d13/f5b 0 2024-09-21T04:35:18.628 INFO:tasks.workunit.client.1.smithi053.stdout:6/834: fsync d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/f6a 0 2024-09-21T04:35:18.632 INFO:tasks.workunit.client.1.smithi053.stdout:4/932: symlink d1/dd/de/df5/d20/d4e/d51/da4/l136 0 2024-09-21T04:35:18.632 INFO:tasks.workunit.client.0.smithi007.stdout:2/196: dread d0/d2/f2b [0,4194304] 0 2024-09-21T04:35:18.634 INFO:tasks.workunit.client.1.smithi053.stdout:0/812: truncate d5/d10/d4c/db1/dbc/dca/d21/fe7 997026 0 2024-09-21T04:35:18.634 INFO:tasks.workunit.client.0.smithi007.stdout:7/283: rename d3/d6/d14/f45 to d3/d6/d14/d20/d58/f67 0 2024-09-21T04:35:18.637 INFO:tasks.workunit.client.0.smithi007.stdout:9/274: fsync d13/d18/f1a 0 2024-09-21T04:35:18.639 INFO:tasks.workunit.client.1.smithi053.stdout:3/834: dread d16/d17/f2a [0,4194304] 0 2024-09-21T04:35:18.641 INFO:tasks.workunit.client.1.smithi053.stdout:9/822: rename d0/d2/d2b/cc1 to d0/d1a/d89/da1/c11d 0 2024-09-21T04:35:18.642 INFO:tasks.workunit.client.1.smithi053.stdout:9/823: chown d0/d2/d4e/d5c/l67 6147 1 2024-09-21T04:35:18.643 INFO:tasks.workunit.client.0.smithi007.stdout:0/227: getdents d6 0 2024-09-21T04:35:18.644 INFO:tasks.workunit.client.0.smithi007.stdout:5/507: dread d2/dd/d1d/d20/f6d [0,4194304] 0 2024-09-21T04:35:18.645 INFO:tasks.workunit.client.0.smithi007.stdout:0/228: read d6/f13 [1492408,123767] 0 2024-09-21T04:35:18.646 INFO:tasks.workunit.client.0.smithi007.stdout:6/222: mkdir d3/de/d4e 0 2024-09-21T04:35:18.646 INFO:tasks.workunit.client.1.smithi053.stdout:5/861: write d0/dd/d5c/d92/ddb/d51/fa0 [309548,118418] 0 2024-09-21T04:35:18.647 INFO:tasks.workunit.client.0.smithi007.stdout:6/223: write d3/d3e/f48 [371181,71470] 0 2024-09-21T04:35:18.647 INFO:tasks.workunit.client.0.smithi007.stdout:8/162: link d0/c4 d0/d3/d8/de/c2a 0 2024-09-21T04:35:18.648 INFO:tasks.workunit.client.0.smithi007.stdout:6/224: stat d3/de/lf 0 2024-09-21T04:35:18.649 INFO:tasks.workunit.client.0.smithi007.stdout:6/225: write d3/de/f3f [506484,68407] 0 2024-09-21T04:35:18.652 INFO:tasks.workunit.client.1.smithi053.stdout:0/813: mkdir d5/d10/d4c/db1/dbc/dca/db5/d10b 0 2024-09-21T04:35:18.655 INFO:tasks.workunit.client.1.smithi053.stdout:0/814: dread d5/d10/d4c/db1/dbc/dca/d21/fa0 [0,4194304] 0 2024-09-21T04:35:18.656 INFO:tasks.workunit.client.1.smithi053.stdout:8/902: rename d3/d5/d38/d64/d9d/d10f to d3/d5/d38/d64/d73/d119/d12a 0 2024-09-21T04:35:18.657 INFO:tasks.workunit.client.1.smithi053.stdout:8/903: stat d3/d5/d4b/d5c/c71 0 2024-09-21T04:35:18.659 INFO:tasks.workunit.client.0.smithi007.stdout:3/183: rename dc/d22/f25 to dc/d18/f31 0 2024-09-21T04:35:18.660 INFO:tasks.workunit.client.1.smithi053.stdout:5/862: chown d0/dd/d6d/f11c 132823926 1 2024-09-21T04:35:18.660 INFO:tasks.workunit.client.1.smithi053.stdout:5/863: stat d0/d8/db/d68/f102 0 2024-09-21T04:35:18.663 INFO:tasks.workunit.client.0.smithi007.stdout:5/508: chown d2/dd/d23/d5d/d61/l92 543729651 1 2024-09-21T04:35:18.665 INFO:tasks.workunit.client.0.smithi007.stdout:0/229: truncate d6/f13 2696062 0 2024-09-21T04:35:18.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/163: fsync d0/d3/d16/f21 0 2024-09-21T04:35:18.668 INFO:tasks.workunit.client.1.smithi053.stdout:7/735: dwrite d0/d6f/f72 [0,4194304] 0 2024-09-21T04:35:18.668 INFO:tasks.workunit.client.1.smithi053.stdout:3/835: write f9 [9145099,97527] 0 2024-09-21T04:35:18.669 INFO:tasks.workunit.client.0.smithi007.stdout:1/422: getdents d5/d45/d5e/d55/d58 0 2024-09-21T04:35:18.674 INFO:tasks.workunit.client.1.smithi053.stdout:0/815: symlink d5/d10/d4c/db1/dbc/d8c/l10c 0 2024-09-21T04:35:18.676 INFO:tasks.workunit.client.1.smithi053.stdout:6/835: write d2/d1d/de0/f94 [616587,42141] 0 2024-09-21T04:35:18.676 INFO:tasks.workunit.client.0.smithi007.stdout:4/256: creat d2/d17/f51 x:0 0 0 2024-09-21T04:35:18.677 INFO:tasks.workunit.client.1.smithi053.stdout:8/904: symlink d3/df/d10/l12b 0 2024-09-21T04:35:18.677 INFO:tasks.workunit.client.1.smithi053.stdout:8/905: readlink d3/d5/d21/l7b 0 2024-09-21T04:35:18.679 INFO:tasks.workunit.client.0.smithi007.stdout:4/257: readlink d2/d17/d2e/l40 0 2024-09-21T04:35:18.679 INFO:tasks.workunit.client.1.smithi053.stdout:5/864: write d0/f2f [1195741,21301] 0 2024-09-21T04:35:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:1/423: dwrite d5/f5b [0,4194304] 0 2024-09-21T04:35:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:7/284: rename d3/d6/d14/l19 to d3/d6/d14/d20/l68 0 2024-09-21T04:35:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:2/197: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:18.684 INFO:tasks.workunit.client.0.smithi007.stdout:3/184: mkdir dc/d22/d29/d32 0 2024-09-21T04:35:18.684 INFO:tasks.workunit.client.0.smithi007.stdout:1/424: chown d5/d6/d21/d3c 90454 1 2024-09-21T04:35:18.685 INFO:tasks.workunit.client.1.smithi053.stdout:7/736: readlink d0/d24/l2c 0 2024-09-21T04:35:18.686 INFO:tasks.workunit.client.0.smithi007.stdout:1/425: fdatasync d5/d45/d5e/d42/f78 0 2024-09-21T04:35:18.686 INFO:tasks.workunit.client.1.smithi053.stdout:3/836: fdatasync d16/d17/d21/d42/d43/d86/fce 0 2024-09-21T04:35:18.688 INFO:tasks.workunit.client.0.smithi007.stdout:2/198: write d0/fd [2584669,45015] 0 2024-09-21T04:35:18.688 INFO:tasks.workunit.client.0.smithi007.stdout:0/230: creat d6/d16/d18/d2e/f56 x:0 0 0 2024-09-21T04:35:18.688 INFO:tasks.workunit.client.1.smithi053.stdout:0/816: rmdir d5/d10/d4c/db1/dbc/dca/d21 39 2024-09-21T04:35:18.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:18 smithi053 ceph-mon[35554]: pgmap v30: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 58 MiB/s rd, 210 MiB/s wr, 382 op/s 2024-09-21T04:35:18.692 INFO:tasks.workunit.client.1.smithi053.stdout:8/906: mknod d3/df/dba/c12c 0 2024-09-21T04:35:18.693 INFO:tasks.workunit.client.0.smithi007.stdout:8/164: dread d0/d3/d16/f21 [4194304,4194304] 0 2024-09-21T04:35:18.694 INFO:tasks.workunit.client.0.smithi007.stdout:0/231: write d6/d16/f28 [3614771,66577] 0 2024-09-21T04:35:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:18 smithi007 ceph-mon[27954]: pgmap v30: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 58 MiB/s rd, 210 MiB/s wr, 382 op/s 2024-09-21T04:35:18.694 INFO:tasks.workunit.client.1.smithi053.stdout:5/865: creat d0/dd/d48/d8c/dad/db9/deb/f11d x:0 0 0 2024-09-21T04:35:18.695 INFO:tasks.workunit.client.0.smithi007.stdout:1/426: chown d5/d6/d21/d2b/d4b/f70 6 1 2024-09-21T04:35:18.697 INFO:tasks.workunit.client.1.smithi053.stdout:3/837: unlink d16/d17/l126 0 2024-09-21T04:35:18.699 INFO:tasks.workunit.client.1.smithi053.stdout:3/838: write d16/d17/d21/d42/d43/d86/fb3 [1435329,78091] 0 2024-09-21T04:35:18.699 INFO:tasks.workunit.client.1.smithi053.stdout:8/907: dwrite d3/d5/db1/f10c [0,4194304] 0 2024-09-21T04:35:18.700 INFO:tasks.workunit.client.0.smithi007.stdout:9/275: rename f6 to d13/d18/f5c 0 2024-09-21T04:35:18.701 INFO:tasks.workunit.client.1.smithi053.stdout:1/850: rename d7/d9/da/d11/l16 to d7/d9/da/d40/l112 0 2024-09-21T04:35:18.702 INFO:tasks.workunit.client.0.smithi007.stdout:0/232: fdatasync d6/d16/f28 0 2024-09-21T04:35:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:8/165: dwrite d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:9/276: read d13/d18/f34 [2285234,119017] 0 2024-09-21T04:35:18.707 INFO:tasks.workunit.client.1.smithi053.stdout:6/836: mknod d2/c10a 0 2024-09-21T04:35:18.710 INFO:tasks.workunit.client.1.smithi053.stdout:3/839: dwrite d16/d17/d21/d26/d2b/faa [0,4194304] 0 2024-09-21T04:35:18.710 INFO:tasks.workunit.client.1.smithi053.stdout:7/737: symlink d0/d6f/dbf/lf9 0 2024-09-21T04:35:18.712 INFO:tasks.workunit.client.0.smithi007.stdout:5/509: truncate d2/f19 814458 0 2024-09-21T04:35:18.714 INFO:tasks.workunit.client.0.smithi007.stdout:6/226: creat d3/de/f4f x:0 0 0 2024-09-21T04:35:18.715 INFO:tasks.workunit.client.0.smithi007.stdout:6/227: dread - d3/de/f4f zero size 2024-09-21T04:35:18.717 INFO:tasks.workunit.client.0.smithi007.stdout:6/228: write d3/de/d12/f29 [35684,101657] 0 2024-09-21T04:35:18.718 INFO:tasks.workunit.client.1.smithi053.stdout:8/908: dread d3/d5/d21/fe2 [0,4194304] 0 2024-09-21T04:35:18.719 INFO:tasks.workunit.client.1.smithi053.stdout:8/909: chown d3/d28/f7c 5 1 2024-09-21T04:35:18.720 INFO:tasks.workunit.client.1.smithi053.stdout:9/824: rename d0/d2/d2b/le4 to d0/d1a/d89/l11e 0 2024-09-21T04:35:18.720 INFO:tasks.workunit.client.1.smithi053.stdout:2/885: sync 2024-09-21T04:35:18.721 INFO:tasks.workunit.client.1.smithi053.stdout:9/825: read - d0/d1c/d52/d6d/fec zero size 2024-09-21T04:35:18.723 INFO:tasks.workunit.client.0.smithi007.stdout:1/427: mknod d5/d6/d21/d2b/d61/c89 0 2024-09-21T04:35:18.726 INFO:tasks.workunit.client.1.smithi053.stdout:7/738: creat d0/d6f/ffa x:0 0 0 2024-09-21T04:35:18.727 INFO:tasks.workunit.client.0.smithi007.stdout:6/229: dwrite d3/de/d12/d19/f39 [0,4194304] 0 2024-09-21T04:35:18.727 INFO:tasks.workunit.client.0.smithi007.stdout:1/428: chown d5/d6/d21/d64/d71/c77 6 1 2024-09-21T04:35:18.729 INFO:tasks.workunit.client.0.smithi007.stdout:5/510: dread d2/dd/d28/f64 [0,4194304] 0 2024-09-21T04:35:18.729 INFO:tasks.workunit.client.0.smithi007.stdout:6/230: stat d3/de/c16 0 2024-09-21T04:35:18.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/429: readlink d5/d45/d5e/d55/l6c 0 2024-09-21T04:35:18.730 INFO:tasks.workunit.client.1.smithi053.stdout:6/837: dread d2/d1b/f5a [0,4194304] 0 2024-09-21T04:35:18.731 INFO:tasks.workunit.client.0.smithi007.stdout:6/231: chown d3/de/lf 105 1 2024-09-21T04:35:18.731 INFO:tasks.workunit.client.0.smithi007.stdout:5/511: write d2/dd/d28/f64 [1491262,30381] 0 2024-09-21T04:35:18.733 INFO:tasks.workunit.client.0.smithi007.stdout:5/512: write d2/d27/d42/f62 [748387,34203] 0 2024-09-21T04:35:18.734 INFO:tasks.workunit.client.0.smithi007.stdout:5/513: stat d2/d27/d3d/d56 0 2024-09-21T04:35:18.735 INFO:tasks.workunit.client.0.smithi007.stdout:3/185: rename dc/d18/d26/c2c to dc/d18/c33 0 2024-09-21T04:35:18.736 INFO:tasks.workunit.client.0.smithi007.stdout:5/514: readlink d2/dd/d23/d5d/d61/l92 0 2024-09-21T04:35:18.737 INFO:tasks.workunit.client.0.smithi007.stdout:3/186: truncate dc/d18/f28 1163391 0 2024-09-21T04:35:18.738 INFO:tasks.workunit.client.0.smithi007.stdout:3/187: chown dc/c16 39567 1 2024-09-21T04:35:18.739 INFO:tasks.workunit.client.1.smithi053.stdout:3/840: dread d16/d17/d21/d42/d43/d125/d127/f123 [0,4194304] 0 2024-09-21T04:35:18.743 INFO:tasks.workunit.client.0.smithi007.stdout:7/285: mkdir d3/d6/d10/d62/d66/d69 0 2024-09-21T04:35:18.744 INFO:tasks.workunit.client.0.smithi007.stdout:8/166: rmdir d0/d3/d8 39 2024-09-21T04:35:18.744 INFO:tasks.workunit.client.1.smithi053.stdout:8/910: dread d3/d28/f8b [0,4194304] 0 2024-09-21T04:35:18.744 INFO:tasks.workunit.client.1.smithi053.stdout:8/911: fsync d3/d5/d38/d64/f67 0 2024-09-21T04:35:18.750 INFO:tasks.workunit.client.1.smithi053.stdout:1/851: dwrite d7/d9/d43/d54/d7f/fb6 [0,4194304] 0 2024-09-21T04:35:18.751 INFO:tasks.workunit.client.1.smithi053.stdout:1/852: chown d7/d9/da/d11/l78 1904 1 2024-09-21T04:35:18.751 INFO:tasks.workunit.client.0.smithi007.stdout:4/258: creat d2/d17/f52 x:0 0 0 2024-09-21T04:35:18.752 INFO:tasks.workunit.client.0.smithi007.stdout:4/259: stat d2/d17/f52 0 2024-09-21T04:35:18.752 INFO:tasks.workunit.client.1.smithi053.stdout:8/912: creat d3/d5/d4c/dc3/ddb/f12d x:0 0 0 2024-09-21T04:35:18.753 INFO:tasks.workunit.client.0.smithi007.stdout:1/430: mkdir d5/d6/d21/d2b/d4b/d8a 0 2024-09-21T04:35:18.754 INFO:tasks.workunit.client.1.smithi053.stdout:2/886: write d1/d18/f31 [92996,52992] 0 2024-09-21T04:35:18.755 INFO:tasks.workunit.client.1.smithi053.stdout:5/866: rename d0/dd/d5c/c5d to d0/dd/d48/c11e 0 2024-09-21T04:35:18.756 INFO:tasks.workunit.client.0.smithi007.stdout:6/232: rmdir d3/de/d12 39 2024-09-21T04:35:18.756 INFO:tasks.workunit.client.1.smithi053.stdout:5/867: fsync d0/d8/f1d 0 2024-09-21T04:35:18.757 INFO:tasks.workunit.client.0.smithi007.stdout:6/233: fdatasync d3/de/f1a 0 2024-09-21T04:35:18.760 INFO:tasks.workunit.client.1.smithi053.stdout:8/913: mknod d3/d5/d4c/dc3/ddb/de7/c12e 0 2024-09-21T04:35:18.761 INFO:tasks.workunit.client.1.smithi053.stdout:8/914: fdatasync d3/df/d49/fe6 0 2024-09-21T04:35:18.761 INFO:tasks.workunit.client.0.smithi007.stdout:5/515: creat d2/dd/d23/d2a/f9f x:0 0 0 2024-09-21T04:35:18.764 INFO:tasks.workunit.client.0.smithi007.stdout:3/188: mkdir dc/d22/d34 0 2024-09-21T04:35:18.767 INFO:tasks.workunit.client.1.smithi053.stdout:0/817: rename d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f106 to d5/d10/d4c/db1/dbc/dca/d21/d96/f10d 0 2024-09-21T04:35:18.770 INFO:tasks.workunit.client.0.smithi007.stdout:1/431: dread d5/d6/d21/d2b/f5c [0,4194304] 0 2024-09-21T04:35:18.771 INFO:tasks.workunit.client.0.smithi007.stdout:1/432: chown d5/d6/d21/d2b/d4b/l52 586308343 1 2024-09-21T04:35:18.771 INFO:tasks.workunit.client.0.smithi007.stdout:0/233: mkdir d6/d16/d57 0 2024-09-21T04:35:18.772 INFO:tasks.workunit.client.0.smithi007.stdout:9/277: symlink d13/l5d 0 2024-09-21T04:35:18.775 INFO:tasks.workunit.client.1.smithi053.stdout:1/853: dwrite d7/d63/d96/dac/fc0 [0,4194304] 0 2024-09-21T04:35:18.778 INFO:tasks.workunit.client.1.smithi053.stdout:0/818: dread d5/de/d1c/f3d [0,4194304] 0 2024-09-21T04:35:18.782 INFO:tasks.workunit.client.1.smithi053.stdout:9/826: rename d0/d1c/d52/fb6 to d0/d1c/d52/d6f/dfb/f11f 0 2024-09-21T04:35:18.784 INFO:tasks.workunit.client.1.smithi053.stdout:9/827: dread - d0/d1c/d21/d46/fe5 zero size 2024-09-21T04:35:18.784 INFO:tasks.workunit.client.1.smithi053.stdout:5/868: dwrite d0/dd/d5c/d89/fb4 [0,4194304] 0 2024-09-21T04:35:18.784 INFO:tasks.workunit.client.0.smithi007.stdout:2/199: creat d0/f2e x:0 0 0 2024-09-21T04:35:18.790 INFO:tasks.workunit.client.1.smithi053.stdout:5/869: dwrite d0/d8/f1f [0,4194304] 0 2024-09-21T04:35:18.793 INFO:tasks.workunit.client.1.smithi053.stdout:1/854: mknod d7/da8/d74/d91/c113 0 2024-09-21T04:35:18.793 INFO:tasks.workunit.client.1.smithi053.stdout:0/819: creat d5/d10/d4c/db1/dbc/dca/d53/f10e x:0 0 0 2024-09-21T04:35:18.795 INFO:tasks.workunit.client.0.smithi007.stdout:3/189: unlink dc/c11 0 2024-09-21T04:35:18.797 INFO:tasks.workunit.client.0.smithi007.stdout:1/433: creat d5/d6/d21/f8b x:0 0 0 2024-09-21T04:35:18.798 INFO:tasks.workunit.client.1.smithi053.stdout:7/739: rename d0/d24/l27 to d0/d6f/dbf/d95/lfb 0 2024-09-21T04:35:18.800 INFO:tasks.workunit.client.0.smithi007.stdout:0/234: write d6/d16/d18/d27/f3a [77974,29818] 0 2024-09-21T04:35:18.803 INFO:tasks.workunit.client.0.smithi007.stdout:1/434: dread d5/d6/d21/d2b/d61/f6e [0,4194304] 0 2024-09-21T04:35:18.803 INFO:tasks.workunit.client.1.smithi053.stdout:9/828: mkdir d0/d2/d9/d34/db2/d120 0 2024-09-21T04:35:18.805 INFO:tasks.workunit.client.0.smithi007.stdout:7/286: mkdir d3/d6/d2e/d63/d6a 0 2024-09-21T04:35:18.806 INFO:tasks.workunit.client.0.smithi007.stdout:1/435: write d5/d6/d21/d2b/d4b/f70 [147443,10596] 0 2024-09-21T04:35:18.807 INFO:tasks.workunit.client.0.smithi007.stdout:8/167: unlink d0/d3/d8/de/lf 0 2024-09-21T04:35:18.807 INFO:tasks.workunit.client.1.smithi053.stdout:8/915: write d3/df/dba/f120 [1939121,26024] 0 2024-09-21T04:35:18.810 INFO:tasks.workunit.client.0.smithi007.stdout:0/235: dwrite d6/d16/f22 [0,4194304] 0 2024-09-21T04:35:18.811 INFO:tasks.workunit.client.0.smithi007.stdout:8/168: chown d0/d3/c11 742 1 2024-09-21T04:35:18.811 INFO:tasks.workunit.client.1.smithi053.stdout:4/933: sync 2024-09-21T04:35:18.815 INFO:tasks.workunit.client.0.smithi007.stdout:6/234: creat d3/de/d12/f50 x:0 0 0 2024-09-21T04:35:18.815 INFO:tasks.workunit.client.0.smithi007.stdout:2/200: dread d0/d2/f6 [0,4194304] 0 2024-09-21T04:35:18.815 INFO:tasks.workunit.client.1.smithi053.stdout:1/855: creat d7/d9/d43/f114 x:0 0 0 2024-09-21T04:35:18.816 INFO:tasks.workunit.client.1.smithi053.stdout:0/820: mknod d5/d10/d4c/db1/dbc/dca/d21/d96/c10f 0 2024-09-21T04:35:18.816 INFO:tasks.workunit.client.1.smithi053.stdout:2/887: rename d1/d18/d1f/d27/d41/d4b/c84 to d1/d18/d1f/da2/dac/c139 0 2024-09-21T04:35:18.816 INFO:tasks.workunit.client.0.smithi007.stdout:3/190: mknod dc/d22/d29/c35 0 2024-09-21T04:35:18.817 INFO:tasks.workunit.client.1.smithi053.stdout:9/829: symlink d0/d1a/d4c/l121 0 2024-09-21T04:35:18.817 INFO:tasks.workunit.client.1.smithi053.stdout:1/856: stat d7/d9/c27 0 2024-09-21T04:35:18.818 INFO:tasks.workunit.client.1.smithi053.stdout:2/888: chown d1/d18/d1f/d9d 84 1 2024-09-21T04:35:18.820 INFO:tasks.workunit.client.1.smithi053.stdout:9/830: write d0/d1a/d89/da1/dea/f10f [871413,95813] 0 2024-09-21T04:35:18.821 INFO:tasks.workunit.client.1.smithi053.stdout:0/821: truncate d5/d10/d4c/db1/dbc/fcf 1412314 0 2024-09-21T04:35:18.821 INFO:tasks.workunit.client.0.smithi007.stdout:7/287: symlink d3/d6/d13/l6b 0 2024-09-21T04:35:18.822 INFO:tasks.workunit.client.1.smithi053.stdout:9/831: write d0/d1c/d21/d8e/fba [307525,102977] 0 2024-09-21T04:35:18.823 INFO:tasks.workunit.client.0.smithi007.stdout:1/436: mknod d5/d45/d5e/d42/c8c 0 2024-09-21T04:35:18.825 INFO:tasks.workunit.client.0.smithi007.stdout:0/236: mknod d6/d16/d18/d39/c58 0 2024-09-21T04:35:18.830 INFO:tasks.workunit.client.0.smithi007.stdout:7/288: dwrite d3/d6/d14/d20/d33/f41 [0,4194304] 0 2024-09-21T04:35:18.831 INFO:tasks.workunit.client.1.smithi053.stdout:9/832: dread d0/d1c/d25/f99 [0,4194304] 0 2024-09-21T04:35:18.834 INFO:tasks.workunit.client.0.smithi007.stdout:8/169: dwrite d0/d3/d8/f9 [0,4194304] 0 2024-09-21T04:35:18.835 INFO:tasks.workunit.client.0.smithi007.stdout:8/170: readlink d0/d3/d16/l1e 0 2024-09-21T04:35:18.840 INFO:tasks.workunit.client.0.smithi007.stdout:9/278: creat d13/f5e x:0 0 0 2024-09-21T04:35:18.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/260: link d2/d7/f8 d2/d3c/f53 0 2024-09-21T04:35:18.846 INFO:tasks.workunit.client.0.smithi007.stdout:5/516: getdents d2/d27/d3d 0 2024-09-21T04:35:18.850 INFO:tasks.workunit.client.0.smithi007.stdout:2/201: symlink d0/l2f 0 2024-09-21T04:35:18.851 INFO:tasks.workunit.client.0.smithi007.stdout:2/202: chown d0/d11 13682105 1 2024-09-21T04:35:18.852 INFO:tasks.workunit.client.0.smithi007.stdout:4/261: dwrite d2/d17/f2a [0,4194304] 0 2024-09-21T04:35:18.858 INFO:tasks.workunit.client.0.smithi007.stdout:3/191: fdatasync fb 0 2024-09-21T04:35:18.859 INFO:tasks.workunit.client.1.smithi053.stdout:6/838: sync 2024-09-21T04:35:18.860 INFO:tasks.workunit.client.0.smithi007.stdout:1/437: mkdir d5/d6/d21/d2b/d4b/d8a/d8d 0 2024-09-21T04:35:18.862 INFO:tasks.workunit.client.0.smithi007.stdout:0/237: write d6/d16/f25 [490290,24359] 0 2024-09-21T04:35:18.865 INFO:tasks.workunit.client.0.smithi007.stdout:5/517: write d2/dd/d23/d5d/d61/f65 [3106140,29610] 0 2024-09-21T04:35:18.867 INFO:tasks.workunit.client.0.smithi007.stdout:7/289: symlink d3/l6c 0 2024-09-21T04:35:18.870 INFO:tasks.workunit.client.0.smithi007.stdout:4/262: creat d2/d17/d18/f54 x:0 0 0 2024-09-21T04:35:18.871 INFO:tasks.workunit.client.0.smithi007.stdout:4/263: fdatasync d2/d17/f26 0 2024-09-21T04:35:18.875 INFO:tasks.workunit.client.0.smithi007.stdout:3/192: symlink dc/d22/d29/d2d/l36 0 2024-09-21T04:35:18.877 INFO:tasks.workunit.client.0.smithi007.stdout:9/279: symlink d13/d5a/l5f 0 2024-09-21T04:35:18.878 INFO:tasks.workunit.client.0.smithi007.stdout:9/280: write d13/d18/f35 [1371543,3234] 0 2024-09-21T04:35:18.878 INFO:tasks.workunit.client.0.smithi007.stdout:1/438: fsync d5/d6/d21/d2b/f51 0 2024-09-21T04:35:18.879 INFO:tasks.workunit.client.0.smithi007.stdout:5/518: fdatasync d2/dd/d28/f8c 0 2024-09-21T04:35:18.880 INFO:tasks.workunit.client.1.smithi053.stdout:7/740: mkdir d0/d2b/d36/d64/da7/dfc 0 2024-09-21T04:35:18.881 INFO:tasks.workunit.client.1.smithi053.stdout:7/741: write d0/d2b/d36/f80 [541612,125506] 0 2024-09-21T04:35:18.881 INFO:tasks.workunit.client.1.smithi053.stdout:5/870: creat d0/dd/d48/f11f x:0 0 0 2024-09-21T04:35:18.883 INFO:tasks.workunit.client.0.smithi007.stdout:6/235: getdents d3/de 0 2024-09-21T04:35:18.884 INFO:tasks.workunit.client.0.smithi007.stdout:6/236: fsync d3/de/f1a 0 2024-09-21T04:35:18.884 INFO:tasks.workunit.client.1.smithi053.stdout:1/857: symlink d7/d47/dae/l115 0 2024-09-21T04:35:18.884 INFO:tasks.workunit.client.1.smithi053.stdout:5/871: truncate d0/dd/d5c/d92/ddb/d51/fe2 1310249 0 2024-09-21T04:35:18.886 INFO:tasks.workunit.client.0.smithi007.stdout:4/264: rename d2/d17/d4a to d2/d7/d55 0 2024-09-21T04:35:18.888 INFO:tasks.workunit.client.1.smithi053.stdout:4/934: creat d1/dd/de/df5/d3f/d74/de1/f137 x:0 0 0 2024-09-21T04:35:18.889 INFO:tasks.workunit.client.0.smithi007.stdout:8/171: creat d0/d3/f2b x:0 0 0 2024-09-21T04:35:18.890 INFO:tasks.workunit.client.1.smithi053.stdout:1/858: dwrite d7/d9/d43/d54/d7f/fe3 [0,4194304] 0 2024-09-21T04:35:18.891 INFO:tasks.workunit.client.1.smithi053.stdout:2/889: mkdir d1/d18/d77/d9b/d13a 0 2024-09-21T04:35:18.892 INFO:tasks.workunit.client.1.smithi053.stdout:2/890: readlink d1/d18/lf7 0 2024-09-21T04:35:18.896 INFO:tasks.workunit.client.0.smithi007.stdout:0/238: creat d6/d16/d57/f59 x:0 0 0 2024-09-21T04:35:18.897 INFO:tasks.workunit.client.0.smithi007.stdout:8/172: dwrite d0/d3/d8/f10 [0,4194304] 0 2024-09-21T04:35:18.900 INFO:tasks.workunit.client.0.smithi007.stdout:1/439: symlink d5/d6/d21/d64/d71/l8e 0 2024-09-21T04:35:18.900 INFO:tasks.workunit.client.0.smithi007.stdout:1/440: fdatasync d5/f39 0 2024-09-21T04:35:18.901 INFO:tasks.workunit.client.0.smithi007.stdout:1/441: chown d5/d45/d5e/f76 0 1 2024-09-21T04:35:18.902 INFO:tasks.workunit.client.0.smithi007.stdout:1/442: write d5/fc [4984775,33645] 0 2024-09-21T04:35:18.903 INFO:tasks.workunit.client.0.smithi007.stdout:2/203: link d0/d11/f28 d0/f30 0 2024-09-21T04:35:18.905 INFO:tasks.workunit.client.1.smithi053.stdout:6/839: fsync d2/d1d/f54 0 2024-09-21T04:35:18.906 INFO:tasks.workunit.client.0.smithi007.stdout:4/265: symlink d2/d7/d55/l56 0 2024-09-21T04:35:18.907 INFO:tasks.workunit.client.1.smithi053.stdout:4/935: dread d1/d27/d2e/d46/fbd [0,4194304] 0 2024-09-21T04:35:18.910 INFO:tasks.workunit.client.1.smithi053.stdout:8/916: truncate d3/d5/d38/f70 2519427 0 2024-09-21T04:35:18.911 INFO:tasks.workunit.client.0.smithi007.stdout:3/193: mkdir dc/d22/d29/d32/d37 0 2024-09-21T04:35:18.912 INFO:tasks.workunit.client.1.smithi053.stdout:5/872: write d0/dd/d5c/d92/ddb/d35/d61/fa5 [3433299,94329] 0 2024-09-21T04:35:18.914 INFO:tasks.workunit.client.1.smithi053.stdout:5/873: write d0/d8/db6/dbc/df4/d119/f105 [942074,51660] 0 2024-09-21T04:35:18.915 INFO:tasks.workunit.client.1.smithi053.stdout:0/822: symlink d5/dd/l110 0 2024-09-21T04:35:18.917 INFO:tasks.workunit.client.1.smithi053.stdout:8/917: dwrite d3/df/d10/fcf [0,4194304] 0 2024-09-21T04:35:18.919 INFO:tasks.workunit.client.0.smithi007.stdout:2/204: mkdir d0/d11/d31 0 2024-09-21T04:35:18.919 INFO:tasks.workunit.client.0.smithi007.stdout:0/239: dread d6/d16/f28 [0,4194304] 0 2024-09-21T04:35:18.919 INFO:tasks.workunit.client.1.smithi053.stdout:8/918: truncate d3/d5/d4c/dc3/ddb/de7/fef 178335 0 2024-09-21T04:35:18.921 INFO:tasks.workunit.client.0.smithi007.stdout:6/237: fsync d3/de/d12/d19/f20 0 2024-09-21T04:35:18.923 INFO:tasks.workunit.client.1.smithi053.stdout:9/833: symlink d0/d1a/l122 0 2024-09-21T04:35:18.924 INFO:tasks.workunit.client.1.smithi053.stdout:9/834: fsync d0/d1a/d89/fbb 0 2024-09-21T04:35:18.926 INFO:tasks.workunit.client.0.smithi007.stdout:1/443: dwrite d5/f1f [0,4194304] 0 2024-09-21T04:35:18.926 INFO:tasks.workunit.client.0.smithi007.stdout:4/266: creat d2/d17/d18/f57 x:0 0 0 2024-09-21T04:35:18.926 INFO:tasks.workunit.client.0.smithi007.stdout:1/444: dread - d5/d6/d21/d2b/d4b/f75 zero size 2024-09-21T04:35:18.930 INFO:tasks.workunit.client.0.smithi007.stdout:8/173: link d0/d3/d16/f1b d0/d3/d16/f2c 0 2024-09-21T04:35:18.931 INFO:tasks.workunit.client.1.smithi053.stdout:6/840: creat d2/d1b/d1e/d27/d41/f10b x:0 0 0 2024-09-21T04:35:18.932 INFO:tasks.workunit.client.1.smithi053.stdout:4/936: mknod d1/dd/de/c138 0 2024-09-21T04:35:18.933 INFO:tasks.workunit.client.1.smithi053.stdout:6/841: truncate d2/d1b/d3d/f5e 2496601 0 2024-09-21T04:35:18.935 INFO:tasks.workunit.client.1.smithi053.stdout:7/742: truncate d0/f49 228069 0 2024-09-21T04:35:18.937 INFO:tasks.workunit.client.1.smithi053.stdout:8/919: dread d3/d4f/fe9 [0,4194304] 0 2024-09-21T04:35:18.938 INFO:tasks.workunit.client.0.smithi007.stdout:0/240: dwrite d6/d16/d18/d27/f33 [0,4194304] 0 2024-09-21T04:35:18.941 INFO:tasks.workunit.client.0.smithi007.stdout:2/205: dread d0/f1c [0,4194304] 0 2024-09-21T04:35:18.941 INFO:tasks.workunit.client.0.smithi007.stdout:4/267: creat d2/d17/d29/f58 x:0 0 0 2024-09-21T04:35:18.942 INFO:tasks.workunit.client.1.smithi053.stdout:0/823: dread d5/fab [0,4194304] 0 2024-09-21T04:35:18.942 INFO:tasks.workunit.client.1.smithi053.stdout:6/842: dwrite d2/d1d/de0/f94 [0,4194304] 0 2024-09-21T04:35:18.943 INFO:tasks.workunit.client.0.smithi007.stdout:2/206: write d0/f23 [94014,37405] 0 2024-09-21T04:35:18.943 INFO:tasks.workunit.client.1.smithi053.stdout:3/841: sync 2024-09-21T04:35:18.944 INFO:tasks.workunit.client.0.smithi007.stdout:1/445: creat d5/d45/d5e/f8f x:0 0 0 2024-09-21T04:35:18.950 INFO:tasks.workunit.client.0.smithi007.stdout:3/194: dread fb [0,4194304] 0 2024-09-21T04:35:18.951 INFO:tasks.workunit.client.0.smithi007.stdout:0/241: dwrite d6/d16/d18/d2e/f4a [0,4194304] 0 2024-09-21T04:35:18.952 INFO:tasks.workunit.client.0.smithi007.stdout:0/242: chown d6/d16 425 1 2024-09-21T04:35:18.956 INFO:tasks.workunit.client.0.smithi007.stdout:4/268: creat d2/d7/f59 x:0 0 0 2024-09-21T04:35:18.956 INFO:tasks.workunit.client.0.smithi007.stdout:3/195: chown dc/f14 3681 1 2024-09-21T04:35:18.958 INFO:tasks.workunit.client.1.smithi053.stdout:9/835: fsync d0/d2/d9/fcc 0 2024-09-21T04:35:18.960 INFO:tasks.workunit.client.1.smithi053.stdout:4/937: mkdir d1/dd/de/d139 0 2024-09-21T04:35:18.961 INFO:tasks.workunit.client.0.smithi007.stdout:3/196: write dc/d22/f24 [9428047,101640] 0 2024-09-21T04:35:18.962 INFO:tasks.workunit.client.0.smithi007.stdout:1/446: dwrite d5/d45/d5e/d55/d58/f67 [0,4194304] 0 2024-09-21T04:35:18.963 INFO:tasks.workunit.client.1.smithi053.stdout:2/891: truncate d1/d18/d1f/d27/d41/d4b/f79 971056 0 2024-09-21T04:35:18.966 INFO:tasks.workunit.client.0.smithi007.stdout:0/243: dwrite d6/d16/d18/d2e/d3f/f43 [0,4194304] 0 2024-09-21T04:35:18.967 INFO:tasks.workunit.client.0.smithi007.stdout:6/238: write d3/de/d12/d19/f24 [512493,62759] 0 2024-09-21T04:35:18.969 INFO:tasks.workunit.client.1.smithi053.stdout:8/920: read d3/d5/d4c/f6d [1475990,50734] 0 2024-09-21T04:35:18.970 INFO:tasks.workunit.client.1.smithi053.stdout:8/921: write d3/d5/d38/d64/d9d/da9/fe3 [3892567,120041] 0 2024-09-21T04:35:18.971 INFO:tasks.workunit.client.1.smithi053.stdout:8/922: chown d3/d39/f27 655 1 2024-09-21T04:35:18.972 INFO:tasks.workunit.client.0.smithi007.stdout:4/269: creat d2/f5a x:0 0 0 2024-09-21T04:35:18.972 INFO:tasks.workunit.client.1.smithi053.stdout:8/923: dread - d3/d5/d4c/d8e/ff5 zero size 2024-09-21T04:35:18.975 INFO:tasks.workunit.client.0.smithi007.stdout:0/244: write d6/d16/f4d [804549,103098] 0 2024-09-21T04:35:18.976 INFO:tasks.workunit.client.0.smithi007.stdout:2/207: mkdir d0/d32 0 2024-09-21T04:35:18.977 INFO:tasks.workunit.client.1.smithi053.stdout:6/843: truncate d2/d1b/d1e/fb1 1063273 0 2024-09-21T04:35:18.992 INFO:tasks.workunit.client.1.smithi053.stdout:1/859: sync 2024-09-21T04:35:19.094 INFO:tasks.workunit.client.0.smithi007.stdout:4/270: write d2/d17/f52 [411624,47757] 0 2024-09-21T04:35:19.096 INFO:tasks.workunit.client.0.smithi007.stdout:3/197: dwrite fa [8388608,4194304] 0 2024-09-21T04:35:19.098 INFO:tasks.workunit.client.0.smithi007.stdout:8/174: dwrite d0/d3/d8/de/f17 [0,4194304] 0 2024-09-21T04:35:19.099 INFO:tasks.workunit.client.0.smithi007.stdout:8/175: readlink d0/l24 0 2024-09-21T04:35:19.111 INFO:tasks.workunit.client.0.smithi007.stdout:1/447: truncate d5/d45/d5e/d42/f56 1337370 0 2024-09-21T04:35:19.113 INFO:tasks.workunit.client.0.smithi007.stdout:6/239: dwrite d3/d3e/f43 [0,4194304] 0 2024-09-21T04:35:19.113 INFO:tasks.workunit.client.1.smithi053.stdout:4/938: mkdir d1/dd/de/df5/d20/d5b/d87/dd5/d115/d121/d13a 0 2024-09-21T04:35:19.114 INFO:tasks.workunit.client.1.smithi053.stdout:0/824: write d5/f15 [2608909,56421] 0 2024-09-21T04:35:19.116 INFO:tasks.workunit.client.1.smithi053.stdout:7/743: creat d0/d6f/dbf/d50/d8b/d70/dc1/ffd x:0 0 0 2024-09-21T04:35:19.117 INFO:tasks.workunit.client.1.smithi053.stdout:5/874: write d0/dd/d5c/d92/ddb/d35/f7b [1592235,46394] 0 2024-09-21T04:35:19.118 INFO:tasks.workunit.client.1.smithi053.stdout:2/892: mknod d1/d18/d1f/d22/d54/c13b 0 2024-09-21T04:35:19.120 INFO:tasks.workunit.client.0.smithi007.stdout:3/198: mkdir dc/d22/d29/d32/d38 0 2024-09-21T04:35:19.120 INFO:tasks.workunit.client.1.smithi053.stdout:7/744: write d0/d6f/dbf/d50/d8b/ff [32138,62645] 0 2024-09-21T04:35:19.121 INFO:tasks.workunit.client.0.smithi007.stdout:2/208: dread d0/fd [0,4194304] 0 2024-09-21T04:35:19.122 INFO:tasks.workunit.client.0.smithi007.stdout:4/271: symlink d2/d17/d2e/d45/l5b 0 2024-09-21T04:35:19.122 INFO:tasks.workunit.client.1.smithi053.stdout:4/939: dwrite d1/d27/f9d [4194304,4194304] 0 2024-09-21T04:35:19.123 INFO:tasks.workunit.client.1.smithi053.stdout:7/745: write d0/d6f/dbf/d95/fc9 [297727,114539] 0 2024-09-21T04:35:19.125 INFO:tasks.workunit.client.1.smithi053.stdout:7/746: truncate d0/d6f/ffa 592461 0 2024-09-21T04:35:19.129 INFO:tasks.workunit.client.1.smithi053.stdout:8/924: rename d3/d39/f27 to d3/df/d49/d103/f12f 0 2024-09-21T04:35:19.129 INFO:tasks.workunit.client.1.smithi053.stdout:9/836: write d0/f15 [267448,31195] 0 2024-09-21T04:35:19.130 INFO:tasks.workunit.client.1.smithi053.stdout:3/842: symlink d16/d1b/dc3/l134 0 2024-09-21T04:35:19.131 INFO:tasks.workunit.client.0.smithi007.stdout:6/240: creat d3/de/d12/d19/f51 x:0 0 0 2024-09-21T04:35:19.136 INFO:tasks.workunit.client.0.smithi007.stdout:6/241: dread d3/de/f49 [0,4194304] 0 2024-09-21T04:35:19.141 INFO:tasks.workunit.client.0.smithi007.stdout:3/199: dread fa [0,4194304] 0 2024-09-21T04:35:19.141 INFO:tasks.workunit.client.0.smithi007.stdout:4/272: unlink d2/d17/d2e/c3d 0 2024-09-21T04:35:19.142 INFO:tasks.workunit.client.0.smithi007.stdout:3/200: write dc/f1c [2918252,17334] 0 2024-09-21T04:35:19.144 INFO:tasks.workunit.client.0.smithi007.stdout:3/201: dread - dc/d22/d29/d2d/f30 zero size 2024-09-21T04:35:19.148 INFO:tasks.workunit.client.0.smithi007.stdout:8/176: dwrite d0/f20 [4194304,4194304] 0 2024-09-21T04:35:19.149 INFO:tasks.workunit.client.1.smithi053.stdout:7/747: creat d0/d6f/dbf/d50/d84/ffe x:0 0 0 2024-09-21T04:35:19.152 INFO:tasks.workunit.client.0.smithi007.stdout:8/177: write d0/d3/d8/de/f17 [4217041,46663] 0 2024-09-21T04:35:19.152 INFO:tasks.workunit.client.1.smithi053.stdout:5/875: write d0/d8/f31 [8305684,32603] 0 2024-09-21T04:35:19.154 INFO:tasks.workunit.client.0.smithi007.stdout:3/202: dwrite dc/d18/f28 [0,4194304] 0 2024-09-21T04:35:19.154 INFO:tasks.workunit.client.0.smithi007.stdout:3/203: chown dc 60707 1 2024-09-21T04:35:19.157 INFO:tasks.workunit.client.0.smithi007.stdout:6/242: symlink d3/de/d12/l52 0 2024-09-21T04:35:19.160 INFO:tasks.workunit.client.1.smithi053.stdout:6/844: rename d2/d1b/d3d/d49/de/f8b to d2/d1b/d1e/d27/d41/f10c 0 2024-09-21T04:35:19.165 INFO:tasks.workunit.client.1.smithi053.stdout:9/837: mknod d0/d1a/d89/da1/dea/c123 0 2024-09-21T04:35:19.166 INFO:tasks.workunit.client.0.smithi007.stdout:1/448: creat d5/f90 x:0 0 0 2024-09-21T04:35:19.167 INFO:tasks.workunit.client.0.smithi007.stdout:1/449: stat d5/d6/d21 0 2024-09-21T04:35:19.170 INFO:tasks.workunit.client.1.smithi053.stdout:4/940: truncate d1/d27/d2e/d46/f8d 1286254 0 2024-09-21T04:35:19.173 INFO:tasks.workunit.client.0.smithi007.stdout:0/245: link d6/d16/f2a d6/f5a 0 2024-09-21T04:35:19.174 INFO:tasks.workunit.client.0.smithi007.stdout:1/450: dwrite d5/d45/d5e/f24 [0,4194304] 0 2024-09-21T04:35:19.178 INFO:tasks.workunit.client.1.smithi053.stdout:3/843: rename d16/f3d to d16/d17/d21/d60/f135 0 2024-09-21T04:35:19.179 INFO:tasks.workunit.client.1.smithi053.stdout:6/845: mkdir d2/d1b/d3d/d49/de/d32/d80/dcd/d10d 0 2024-09-21T04:35:19.181 INFO:tasks.workunit.client.0.smithi007.stdout:3/204: creat dc/d18/f39 x:0 0 0 2024-09-21T04:35:19.183 INFO:tasks.workunit.client.1.smithi053.stdout:8/925: symlink d3/d5/d21/l130 0 2024-09-21T04:35:19.185 INFO:tasks.workunit.client.1.smithi053.stdout:1/860: creat d7/d9/d43/d54/d7f/f116 x:0 0 0 2024-09-21T04:35:19.185 INFO:tasks.workunit.client.0.smithi007.stdout:2/209: link d0/l2f d0/d2/l33 0 2024-09-21T04:35:19.186 INFO:tasks.workunit.client.1.smithi053.stdout:8/926: read d3/d5/db1/fd1 [1709783,75299] 0 2024-09-21T04:35:19.187 INFO:tasks.workunit.client.1.smithi053.stdout:9/838: creat d0/d2/d2b/f124 x:0 0 0 2024-09-21T04:35:19.188 INFO:tasks.workunit.client.1.smithi053.stdout:9/839: fdatasync d0/d28/f38 0 2024-09-21T04:35:19.189 INFO:tasks.workunit.client.0.smithi007.stdout:4/273: creat d2/d7/d55/d4f/f5c x:0 0 0 2024-09-21T04:35:19.189 INFO:tasks.workunit.client.1.smithi053.stdout:9/840: write d0/d2/d9/d34/f10e [488317,5804] 0 2024-09-21T04:35:19.190 INFO:tasks.workunit.client.1.smithi053.stdout:4/941: mkdir d1/dd/de/df5/d3f/d74/de1/d13b 0 2024-09-21T04:35:19.192 INFO:tasks.workunit.client.1.smithi053.stdout:7/748: mknod d0/d6f/dbf/cff 0 2024-09-21T04:35:19.192 INFO:tasks.workunit.client.1.smithi053.stdout:2/893: sync 2024-09-21T04:35:19.193 INFO:tasks.workunit.client.0.smithi007.stdout:0/246: chown d6/d16/c17 15 1 2024-09-21T04:35:19.193 INFO:tasks.workunit.client.1.smithi053.stdout:5/876: truncate d0/d8/db6/dbc/df4/f112 364697 0 2024-09-21T04:35:19.195 INFO:tasks.workunit.client.0.smithi007.stdout:8/178: mknod d0/d3/d8/c2d 0 2024-09-21T04:35:19.197 INFO:tasks.workunit.client.1.smithi053.stdout:0/825: truncate d5/d10/d4c/db1/dbc/dca/d21/d96/fbd 1588702 0 2024-09-21T04:35:19.199 INFO:tasks.workunit.client.0.smithi007.stdout:3/205: dread dc/f15 [0,4194304] 0 2024-09-21T04:35:19.200 INFO:tasks.workunit.client.0.smithi007.stdout:0/247: dwrite d6/d16/d18/d39/f54 [0,4194304] 0 2024-09-21T04:35:19.200 INFO:tasks.workunit.client.0.smithi007.stdout:6/243: link d3/de/d12/d19/f36 d3/d3e/f53 0 2024-09-21T04:35:19.201 INFO:tasks.workunit.client.0.smithi007.stdout:6/244: readlink d3/de/d12/l34 0 2024-09-21T04:35:19.202 INFO:tasks.workunit.client.0.smithi007.stdout:6/245: dread - d3/de/d12/f21 zero size 2024-09-21T04:35:19.202 INFO:tasks.workunit.client.0.smithi007.stdout:4/274: mkdir d2/d17/d2e/d45/d5d 0 2024-09-21T04:35:19.204 INFO:tasks.workunit.client.0.smithi007.stdout:4/275: fdatasync d2/d17/d2e/f3e 0 2024-09-21T04:35:19.205 INFO:tasks.workunit.client.1.smithi053.stdout:6/846: dread d2/d1b/d1e/d27/d41/f97 [0,4194304] 0 2024-09-21T04:35:19.205 INFO:tasks.workunit.client.0.smithi007.stdout:8/179: mknod d0/d3/d16/c2e 0 2024-09-21T04:35:19.206 INFO:tasks.workunit.client.0.smithi007.stdout:6/246: truncate d3/de/d12/d19/f51 620276 0 2024-09-21T04:35:19.207 INFO:tasks.workunit.client.1.smithi053.stdout:8/927: read - d3/d28/f7c zero size 2024-09-21T04:35:19.210 INFO:tasks.workunit.client.0.smithi007.stdout:6/247: dread d3/f6 [0,4194304] 0 2024-09-21T04:35:19.211 INFO:tasks.workunit.client.1.smithi053.stdout:9/841: creat d0/df/f125 x:0 0 0 2024-09-21T04:35:19.212 INFO:tasks.workunit.client.1.smithi053.stdout:4/942: mkdir d1/dd/de/df5/db6/d13c 0 2024-09-21T04:35:19.213 INFO:tasks.workunit.client.0.smithi007.stdout:3/206: truncate fb 2331060 0 2024-09-21T04:35:19.214 INFO:tasks.workunit.client.0.smithi007.stdout:0/248: mknod d6/d16/d18/d39/c5b 0 2024-09-21T04:35:19.215 INFO:tasks.workunit.client.1.smithi053.stdout:9/842: dwrite d0/d28/f9b [0,4194304] 0 2024-09-21T04:35:19.218 INFO:tasks.workunit.client.0.smithi007.stdout:4/276: mkdir d2/d17/d2e/d45/d5e 0 2024-09-21T04:35:19.219 INFO:tasks.workunit.client.0.smithi007.stdout:6/248: dwrite d3/d3e/f40 [0,4194304] 0 2024-09-21T04:35:19.220 INFO:tasks.workunit.client.0.smithi007.stdout:8/180: rmdir d0/d3/d8 39 2024-09-21T04:35:19.221 INFO:tasks.workunit.client.0.smithi007.stdout:6/249: fsync d3/de/f1a 0 2024-09-21T04:35:19.224 INFO:tasks.workunit.client.1.smithi053.stdout:2/894: dread d1/d18/d1f/d27/d58/d5c/d91/dd0/f80 [0,4194304] 0 2024-09-21T04:35:19.227 INFO:tasks.workunit.client.0.smithi007.stdout:3/207: rename dc/d22/d29/d2d/l2e to dc/d22/l3a 0 2024-09-21T04:35:19.227 INFO:tasks.workunit.client.0.smithi007.stdout:6/250: dread d3/de/d12/d19/f24 [0,4194304] 0 2024-09-21T04:35:19.229 INFO:tasks.workunit.client.1.smithi053.stdout:3/844: creat d16/d17/d21/d8b/d110/f136 x:0 0 0 2024-09-21T04:35:19.230 INFO:tasks.workunit.client.1.smithi053.stdout:3/845: write d16/d1b/dc3/fb2 [1688515,92694] 0 2024-09-21T04:35:19.231 INFO:tasks.workunit.client.1.smithi053.stdout:3/846: fdatasync f9 0 2024-09-21T04:35:19.232 INFO:tasks.workunit.client.1.smithi053.stdout:0/826: unlink d5/de/fb6 0 2024-09-21T04:35:19.234 INFO:tasks.workunit.client.0.smithi007.stdout:4/277: unlink d2/d7/f10 0 2024-09-21T04:35:19.235 INFO:tasks.workunit.client.1.smithi053.stdout:1/861: mknod d7/d9/d43/d54/da6/c117 0 2024-09-21T04:35:19.238 INFO:tasks.workunit.client.1.smithi053.stdout:6/847: rmdir d2/d1b/d3d/d49/de/d32/d80/dcd 39 2024-09-21T04:35:19.239 INFO:tasks.workunit.client.1.smithi053.stdout:6/848: write d2/d1b/f53 [828815,115385] 0 2024-09-21T04:35:19.241 INFO:tasks.workunit.client.0.smithi007.stdout:3/208: symlink dc/d22/l3b 0 2024-09-21T04:35:19.242 INFO:tasks.workunit.client.0.smithi007.stdout:3/209: fsync dc/d18/f28 0 2024-09-21T04:35:19.242 INFO:tasks.workunit.client.1.smithi053.stdout:8/928: symlink d3/d5/d38/d64/d9d/da9/l131 0 2024-09-21T04:35:19.246 INFO:tasks.workunit.client.1.smithi053.stdout:6/849: dwrite d2/d1b/d3d/d49/de/d32/d80/f82 [0,4194304] 0 2024-09-21T04:35:19.246 INFO:tasks.workunit.client.0.smithi007.stdout:4/278: mkdir d2/d5f 0 2024-09-21T04:35:19.246 INFO:tasks.workunit.client.0.smithi007.stdout:4/279: chown d2/d7/f59 3 1 2024-09-21T04:35:19.248 INFO:tasks.workunit.client.1.smithi053.stdout:7/749: rename d0/d2b/d36/d64/l9a to d0/d6f/dbf/d50/d84/l100 0 2024-09-21T04:35:19.250 INFO:tasks.workunit.client.1.smithi053.stdout:5/877: write d0/dd/d48/d8c/dad/fd8 [785996,114252] 0 2024-09-21T04:35:19.252 INFO:tasks.workunit.client.1.smithi053.stdout:5/878: write d0/dd/d48/d8c/dad/db9/fba [1212565,92367] 0 2024-09-21T04:35:19.253 INFO:tasks.workunit.client.1.smithi053.stdout:2/895: creat d1/d18/d1f/d27/d58/d5c/d91/dc1/f13c x:0 0 0 2024-09-21T04:35:19.257 INFO:tasks.workunit.client.1.smithi053.stdout:3/847: chown d16/d17/d21/d42/d43/d86/cbc 108 1 2024-09-21T04:35:19.258 INFO:tasks.workunit.client.1.smithi053.stdout:3/848: fsync d16/d17/d21/d8b/f66 0 2024-09-21T04:35:19.260 INFO:tasks.workunit.client.1.smithi053.stdout:0/827: mknod d5/de/d1c/da3/c111 0 2024-09-21T04:35:19.260 INFO:tasks.workunit.client.1.smithi053.stdout:3/849: read d16/d17/d21/d42/d43/d125/d127/f7e [1272638,129112] 0 2024-09-21T04:35:19.261 INFO:tasks.workunit.client.1.smithi053.stdout:1/862: unlink d7/d9/d43/d54/d7f/l82 0 2024-09-21T04:35:19.262 INFO:tasks.workunit.client.1.smithi053.stdout:3/850: write d16/d1b/dad/f117 [325237,75560] 0 2024-09-21T04:35:19.264 INFO:tasks.workunit.client.0.smithi007.stdout:3/210: creat dc/d18/d26/f3c x:0 0 0 2024-09-21T04:35:19.265 INFO:tasks.workunit.client.0.smithi007.stdout:3/211: dread - dc/d22/d29/d2d/f30 zero size 2024-09-21T04:35:19.266 INFO:tasks.workunit.client.1.smithi053.stdout:0/828: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/f91 [0,4194304] 0 2024-09-21T04:35:19.269 INFO:tasks.workunit.client.1.smithi053.stdout:4/943: creat d1/dd/d95/f13d x:0 0 0 2024-09-21T04:35:19.275 INFO:tasks.workunit.client.1.smithi053.stdout:5/879: symlink d0/d8/db/d68/l120 0 2024-09-21T04:35:19.278 INFO:tasks.workunit.client.1.smithi053.stdout:6/850: dwrite d2/d1b/d1e/d2a/d4b/d77/d88/fca [4194304,4194304] 0 2024-09-21T04:35:19.279 INFO:tasks.workunit.client.0.smithi007.stdout:6/251: creat d3/f54 x:0 0 0 2024-09-21T04:35:19.280 INFO:tasks.workunit.client.0.smithi007.stdout:3/212: write dc/d18/f28 [2284287,22309] 0 2024-09-21T04:35:19.280 INFO:tasks.workunit.client.0.smithi007.stdout:0/249: getdents d6/d16/d18 0 2024-09-21T04:35:19.280 INFO:tasks.workunit.client.1.smithi053.stdout:4/944: dwrite d1/dd/de/df5/d20/d4e/d51/da4/dc3/f109 [0,4194304] 0 2024-09-21T04:35:19.281 INFO:tasks.workunit.client.1.smithi053.stdout:6/851: chown d2/d1b/d1e/d2a/d4b/fd9 306598073 1 2024-09-21T04:35:19.281 INFO:tasks.workunit.client.0.smithi007.stdout:6/252: chown d3/de/lf 1264 1 2024-09-21T04:35:19.281 INFO:tasks.workunit.client.0.smithi007.stdout:0/250: read d6/d16/f28 [3435308,116786] 0 2024-09-21T04:35:19.282 INFO:tasks.workunit.client.0.smithi007.stdout:4/280: fdatasync d2/d3c/f53 0 2024-09-21T04:35:19.284 INFO:tasks.workunit.client.0.smithi007.stdout:4/281: truncate d2/d7/f59 299125 0 2024-09-21T04:35:19.286 INFO:tasks.workunit.client.0.smithi007.stdout:3/213: creat dc/d18/d26/f3d x:0 0 0 2024-09-21T04:35:19.286 INFO:tasks.workunit.client.1.smithi053.stdout:1/863: chown d7/d9/c72 4 1 2024-09-21T04:35:19.286 INFO:tasks.workunit.client.0.smithi007.stdout:3/214: readlink dc/l20 0 2024-09-21T04:35:19.287 INFO:tasks.workunit.client.1.smithi053.stdout:7/750: write d0/d2b/d36/d64/fb5 [4560630,130587] 0 2024-09-21T04:35:19.288 INFO:tasks.workunit.client.0.smithi007.stdout:6/253: dread d3/d3e/f43 [0,4194304] 0 2024-09-21T04:35:19.291 INFO:tasks.workunit.client.1.smithi053.stdout:2/896: write d1/d18/d1f/d22/d2d/f87 [712058,126887] 0 2024-09-21T04:35:19.292 INFO:tasks.workunit.client.1.smithi053.stdout:2/897: stat d1/d18/d1f/d27/d58/d5c/d91/dc1/ce0 0 2024-09-21T04:35:19.293 INFO:tasks.workunit.client.1.smithi053.stdout:0/829: write d5/d10/d4c/db1/dbc/dca/d21/fa0 [128219,86244] 0 2024-09-21T04:35:19.293 INFO:tasks.workunit.client.0.smithi007.stdout:4/282: dwrite d2/d17/d18/f47 [0,4194304] 0 2024-09-21T04:35:19.294 INFO:tasks.workunit.client.1.smithi053.stdout:9/843: creat d0/d1c/f126 x:0 0 0 2024-09-21T04:35:19.295 INFO:tasks.workunit.client.0.smithi007.stdout:3/215: symlink dc/d22/d29/l3e 0 2024-09-21T04:35:19.296 INFO:tasks.workunit.client.0.smithi007.stdout:3/216: rename dc/d22 to dc/d22/d29/d32/d37/d3f 22 2024-09-21T04:35:19.301 INFO:tasks.workunit.client.0.smithi007.stdout:4/283: rmdir d2/d17 39 2024-09-21T04:35:19.304 INFO:tasks.workunit.client.1.smithi053.stdout:9/844: dwrite d0/d1a/d4c/d8f/ff1 [0,4194304] 0 2024-09-21T04:35:19.306 INFO:tasks.workunit.client.0.smithi007.stdout:3/217: dread fb [0,4194304] 0 2024-09-21T04:35:19.308 INFO:tasks.workunit.client.1.smithi053.stdout:6/852: rmdir d2/d1b/d3d/d49/de/d32 39 2024-09-21T04:35:19.311 INFO:tasks.workunit.client.0.smithi007.stdout:6/254: write d3/de/d12/d19/f24 [1179909,49853] 0 2024-09-21T04:35:19.311 INFO:tasks.workunit.client.1.smithi053.stdout:3/851: creat d16/d17/d21/d42/d43/d125/d12e/f137 x:0 0 0 2024-09-21T04:35:19.314 INFO:tasks.workunit.client.0.smithi007.stdout:0/251: dwrite d6/d16/d18/d27/f35 [4194304,4194304] 0 2024-09-21T04:35:19.314 INFO:tasks.workunit.client.1.smithi053.stdout:3/852: write d16/d1b/dc3/f121 [991304,96485] 0 2024-09-21T04:35:19.316 INFO:tasks.workunit.client.1.smithi053.stdout:8/929: rmdir d3/df/d49/d103/d115 0 2024-09-21T04:35:19.316 INFO:tasks.workunit.client.1.smithi053.stdout:1/864: dwrite d7/d9/da/f50 [4194304,4194304] 0 2024-09-21T04:35:19.316 INFO:tasks.workunit.client.0.smithi007.stdout:3/218: write dc/d22/f24 [2259927,27079] 0 2024-09-21T04:35:19.317 INFO:tasks.workunit.client.0.smithi007.stdout:4/284: write d2/d17/d2e/d45/f4b [864972,33777] 0 2024-09-21T04:35:19.317 INFO:tasks.workunit.client.0.smithi007.stdout:3/219: readlink dc/l19 0 2024-09-21T04:35:19.318 INFO:tasks.workunit.client.0.smithi007.stdout:4/285: stat d2/d17/d2e 0 2024-09-21T04:35:19.318 INFO:tasks.workunit.client.1.smithi053.stdout:8/930: stat d3/d5/d4c/dd0 0 2024-09-21T04:35:19.320 INFO:tasks.workunit.client.0.smithi007.stdout:4/286: write d2/d17/d29/f58 [45629,115367] 0 2024-09-21T04:35:19.321 INFO:tasks.workunit.client.0.smithi007.stdout:6/255: fsync d3/de/f49 0 2024-09-21T04:35:19.322 INFO:tasks.workunit.client.1.smithi053.stdout:2/898: truncate d1/d18/d1f/d27/d41/d4b/d4d/d89/fd7 121124 0 2024-09-21T04:35:19.324 INFO:tasks.workunit.client.0.smithi007.stdout:3/220: dread dc/f13 [4194304,4194304] 0 2024-09-21T04:35:19.326 INFO:tasks.workunit.client.1.smithi053.stdout:4/945: mkdir d1/d27/d13e 0 2024-09-21T04:35:19.326 INFO:tasks.workunit.client.1.smithi053.stdout:7/751: dwrite d0/d2b/d36/d64/ddb/fb2 [0,4194304] 0 2024-09-21T04:35:19.328 INFO:tasks.workunit.client.0.smithi007.stdout:0/252: creat d6/d16/d52/f5c x:0 0 0 2024-09-21T04:35:19.328 INFO:tasks.workunit.client.0.smithi007.stdout:6/256: dread d3/d3e/f43 [0,4194304] 0 2024-09-21T04:35:19.329 INFO:tasks.workunit.client.1.smithi053.stdout:6/853: unlink d2/fed 0 2024-09-21T04:35:19.330 INFO:tasks.workunit.client.0.smithi007.stdout:6/257: write d3/de/f49 [4687733,81895] 0 2024-09-21T04:35:19.330 INFO:tasks.workunit.client.0.smithi007.stdout:4/287: unlink d2/d7/c39 0 2024-09-21T04:35:19.330 INFO:tasks.workunit.client.1.smithi053.stdout:3/853: creat d16/d17/d21/d8b/d110/f138 x:0 0 0 2024-09-21T04:35:19.332 INFO:tasks.workunit.client.1.smithi053.stdout:3/854: chown c1 18 1 2024-09-21T04:35:19.332 INFO:tasks.workunit.client.1.smithi053.stdout:1/865: dwrite d7/d9/da/d40/d49/fff [0,4194304] 0 2024-09-21T04:35:19.332 INFO:tasks.workunit.client.0.smithi007.stdout:3/221: chown dc/d22/l3a 8 1 2024-09-21T04:35:19.335 INFO:tasks.workunit.client.1.smithi053.stdout:1/866: chown d7/d9/da/d40/d49/l105 429973013 1 2024-09-21T04:35:19.335 INFO:tasks.workunit.client.0.smithi007.stdout:6/258: read d3/de/f42 [1270245,12100] 0 2024-09-21T04:35:19.336 INFO:tasks.workunit.client.0.smithi007.stdout:0/253: rename d6/d16/f25 to d6/d16/d18/f5d 0 2024-09-21T04:35:19.337 INFO:tasks.workunit.client.0.smithi007.stdout:6/259: write d3/f15 [573111,76552] 0 2024-09-21T04:35:19.339 INFO:tasks.workunit.client.0.smithi007.stdout:6/260: truncate d3/de/f42 4981619 0 2024-09-21T04:35:19.340 INFO:tasks.workunit.client.1.smithi053.stdout:5/880: rename d0/dd/d48/d8c/dad/fd8 to d0/d8/db6/f121 0 2024-09-21T04:35:19.342 INFO:tasks.workunit.client.1.smithi053.stdout:4/946: dwrite d1/dd/de/df5/d3f/f77 [0,4194304] 0 2024-09-21T04:35:19.344 INFO:tasks.workunit.client.1.smithi053.stdout:5/881: chown d0/dd/d48/d8c/dfd 565986 1 2024-09-21T04:35:19.345 INFO:tasks.workunit.client.0.smithi007.stdout:4/288: unlink d2/d17/d29/c49 0 2024-09-21T04:35:19.345 INFO:tasks.workunit.client.1.smithi053.stdout:5/882: fsync d0/dd/d5c/d89/fbd 0 2024-09-21T04:35:19.346 INFO:tasks.workunit.client.1.smithi053.stdout:1/867: dread d7/d47/d79/d99/fe4 [0,4194304] 0 2024-09-21T04:35:19.356 INFO:tasks.workunit.client.0.smithi007.stdout:4/289: dwrite d2/d17/d2e/f50 [0,4194304] 0 2024-09-21T04:35:19.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/290: readlink d2/d17/l35 0 2024-09-21T04:35:19.363 INFO:tasks.workunit.client.0.smithi007.stdout:6/261: rename d3/de/f3f to d3/d3e/f55 0 2024-09-21T04:35:19.363 INFO:tasks.workunit.client.1.smithi053.stdout:6/854: dread - d2/d1d/de0/fbf zero size 2024-09-21T04:35:19.365 INFO:tasks.workunit.client.0.smithi007.stdout:4/291: dwrite d2/d7/f59 [0,4194304] 0 2024-09-21T04:35:19.366 INFO:tasks.workunit.client.0.smithi007.stdout:4/292: readlink d2/d17/d2e/d45/l5b 0 2024-09-21T04:35:19.367 INFO:tasks.workunit.client.1.smithi053.stdout:0/830: write d5/dd/d50/fc7 [4094628,63064] 0 2024-09-21T04:35:19.375 INFO:tasks.workunit.client.1.smithi053.stdout:8/931: rename d3/df/fe8 to d3/d5/d38/d64/d73/d119/d12a/f132 0 2024-09-21T04:35:19.376 INFO:tasks.workunit.client.1.smithi053.stdout:4/947: fdatasync d1/d27/d53/ffc 0 2024-09-21T04:35:19.378 INFO:tasks.workunit.client.1.smithi053.stdout:7/752: write d0/d24/d46/f4a [4679473,18505] 0 2024-09-21T04:35:19.378 INFO:tasks.workunit.client.1.smithi053.stdout:3/855: truncate d16/d17/d21/d8b/d7d/d57/dc1/d53/f9a 228617 0 2024-09-21T04:35:19.379 INFO:tasks.workunit.client.1.smithi053.stdout:3/856: readlink d16/d17/d21/d26/d2b/l3f 0 2024-09-21T04:35:19.383 INFO:tasks.workunit.client.0.smithi007.stdout:3/222: truncate dc/d18/f28 3042797 0 2024-09-21T04:35:19.384 INFO:tasks.workunit.client.0.smithi007.stdout:3/223: read - dc/d18/f39 zero size 2024-09-21T04:35:19.384 INFO:tasks.workunit.client.0.smithi007.stdout:6/262: mknod d3/d3e/c56 0 2024-09-21T04:35:19.385 INFO:tasks.workunit.client.1.smithi053.stdout:9/845: creat d0/d2/d9/d34/f127 x:0 0 0 2024-09-21T04:35:19.386 INFO:tasks.workunit.client.1.smithi053.stdout:9/846: write d0/d28/fd6 [8637719,82342] 0 2024-09-21T04:35:19.386 INFO:tasks.workunit.client.0.smithi007.stdout:4/293: creat d2/d17/d2e/d45/f60 x:0 0 0 2024-09-21T04:35:19.388 INFO:tasks.workunit.client.0.smithi007.stdout:0/254: creat d6/d16/f5e x:0 0 0 2024-09-21T04:35:19.392 INFO:tasks.workunit.client.1.smithi053.stdout:1/868: write d7/d63/f7d [298145,72991] 0 2024-09-21T04:35:19.393 INFO:tasks.workunit.client.1.smithi053.stdout:6/855: creat d2/d1b/d1e/d27/d90/f10e x:0 0 0 2024-09-21T04:35:19.395 INFO:tasks.workunit.client.1.smithi053.stdout:6/856: write d2/d1b/d3d/d49/de/fef [263298,24414] 0 2024-09-21T04:35:19.396 INFO:tasks.workunit.client.0.smithi007.stdout:0/255: creat d6/d3d/f5f x:0 0 0 2024-09-21T04:35:19.397 INFO:tasks.workunit.client.1.smithi053.stdout:9/847: dread d0/d2/f55 [0,4194304] 0 2024-09-21T04:35:19.401 INFO:tasks.workunit.client.0.smithi007.stdout:3/224: getdents dc/d22/d34 0 2024-09-21T04:35:19.402 INFO:tasks.workunit.client.0.smithi007.stdout:3/225: write dc/d22/d29/d2d/f30 [537216,61311] 0 2024-09-21T04:35:19.404 INFO:tasks.workunit.client.1.smithi053.stdout:4/948: mknod d1/dd/de/c13f 0 2024-09-21T04:35:19.406 INFO:tasks.workunit.client.0.smithi007.stdout:4/294: dread d2/fd [0,4194304] 0 2024-09-21T04:35:19.408 INFO:tasks.workunit.client.1.smithi053.stdout:4/949: dread d1/dd/de/df5/d3f/f72 [0,4194304] 0 2024-09-21T04:35:19.408 INFO:tasks.workunit.client.0.smithi007.stdout:6/263: creat d3/de/d4e/f57 x:0 0 0 2024-09-21T04:35:19.410 INFO:tasks.workunit.client.0.smithi007.stdout:3/226: creat dc/d22/d29/d32/d37/f40 x:0 0 0 2024-09-21T04:35:19.410 INFO:tasks.workunit.client.1.smithi053.stdout:1/869: rmdir d7/d9/d43 39 2024-09-21T04:35:19.412 INFO:tasks.workunit.client.0.smithi007.stdout:4/295: mkdir d2/d7/d55/d61 0 2024-09-21T04:35:19.413 INFO:tasks.workunit.client.1.smithi053.stdout:6/857: fsync d2/d1d/f54 0 2024-09-21T04:35:19.413 INFO:tasks.workunit.client.0.smithi007.stdout:4/296: chown d2/d7/f59 118 1 2024-09-21T04:35:19.414 INFO:tasks.workunit.client.0.smithi007.stdout:4/297: dread - d2/d17/d18/f41 zero size 2024-09-21T04:35:19.415 INFO:tasks.workunit.client.1.smithi053.stdout:8/932: dread d3/d5/f13 [0,4194304] 0 2024-09-21T04:35:19.415 INFO:tasks.workunit.client.0.smithi007.stdout:0/256: rename d6/d3d to d6/d16/d18/d2e/d60 0 2024-09-21T04:35:19.416 INFO:tasks.workunit.client.1.smithi053.stdout:0/831: creat d5/d10/d4c/db1/dbc/dca/d21/d101/f112 x:0 0 0 2024-09-21T04:35:19.417 INFO:tasks.workunit.client.0.smithi007.stdout:0/257: chown d6/l24 148394 1 2024-09-21T04:35:19.417 INFO:tasks.workunit.client.0.smithi007.stdout:3/227: dwrite dc/d18/d26/f3d [0,4194304] 0 2024-09-21T04:35:19.419 INFO:tasks.workunit.client.0.smithi007.stdout:0/258: write d6/d16/f28 [3811014,106806] 0 2024-09-21T04:35:19.419 INFO:tasks.workunit.client.0.smithi007.stdout:6/264: read d3/de/d12/d19/f20 [326089,22896] 0 2024-09-21T04:35:19.421 INFO:tasks.workunit.client.0.smithi007.stdout:0/259: truncate d6/d16/d52/f5c 612207 0 2024-09-21T04:35:19.423 INFO:tasks.workunit.client.0.smithi007.stdout:3/228: mkdir dc/d18/d41 0 2024-09-21T04:35:19.429 INFO:tasks.workunit.client.1.smithi053.stdout:4/950: mknod d1/dd/de/df5/d20/d5b/d87/dd5/d115/c140 0 2024-09-21T04:35:19.430 INFO:tasks.workunit.client.1.smithi053.stdout:2/899: getdents d1/d18/d1f/d27/d58/d5c/d91/dc1 0 2024-09-21T04:35:19.431 INFO:tasks.workunit.client.0.smithi007.stdout:6/265: mkdir d3/de/d58 0 2024-09-21T04:35:19.431 INFO:tasks.workunit.client.1.smithi053.stdout:1/870: truncate d7/d63/f6a 6008841 0 2024-09-21T04:35:19.433 INFO:tasks.workunit.client.0.smithi007.stdout:6/266: write d3/de/d12/d19/f24 [1713798,115205] 0 2024-09-21T04:35:19.434 INFO:tasks.workunit.client.0.smithi007.stdout:6/267: truncate d3/d3e/f55 1470522 0 2024-09-21T04:35:19.435 INFO:tasks.workunit.client.1.smithi053.stdout:8/933: chown d3/df/d14/d16/dde/cf2 747950 1 2024-09-21T04:35:19.436 INFO:tasks.workunit.client.1.smithi053.stdout:8/934: chown d3/d5/d4b/lc6 27582 1 2024-09-21T04:35:19.439 INFO:tasks.workunit.client.1.smithi053.stdout:0/832: unlink d5/de/d1c/da3/f100 0 2024-09-21T04:35:19.439 INFO:tasks.workunit.client.0.smithi007.stdout:3/229: rename dc/d18/d26/f3d to dc/d18/f42 0 2024-09-21T04:35:19.440 INFO:tasks.workunit.client.0.smithi007.stdout:4/298: link d2/d7/l11 d2/d17/l62 0 2024-09-21T04:35:19.443 INFO:tasks.workunit.client.1.smithi053.stdout:5/883: getdents d0/d8 0 2024-09-21T04:35:19.444 INFO:tasks.workunit.client.1.smithi053.stdout:5/884: read - d0/dd/d5c/d80/fd7 zero size 2024-09-21T04:35:19.445 INFO:tasks.workunit.client.0.smithi007.stdout:4/299: dread d2/f1d [0,4194304] 0 2024-09-21T04:35:19.446 INFO:tasks.workunit.client.0.smithi007.stdout:3/230: rename dc/f14 to dc/d22/d29/d2d/f43 0 2024-09-21T04:35:19.446 INFO:tasks.workunit.client.0.smithi007.stdout:3/231: fdatasync dc/f23 0 2024-09-21T04:35:19.446 INFO:tasks.workunit.client.0.smithi007.stdout:4/300: chown d2/f6 1994 1 2024-09-21T04:35:19.447 INFO:tasks.workunit.client.1.smithi053.stdout:4/951: creat d1/dd/de/df5/d20/d73/d10b/f141 x:0 0 0 2024-09-21T04:35:19.448 INFO:tasks.workunit.client.0.smithi007.stdout:4/301: write d2/d17/d18/f2d [2498855,92646] 0 2024-09-21T04:35:19.450 INFO:tasks.workunit.client.1.smithi053.stdout:2/900: symlink d1/d18/d1f/d22/d54/l13d 0 2024-09-21T04:35:19.451 INFO:tasks.workunit.client.0.smithi007.stdout:6/268: mkdir d3/d59 0 2024-09-21T04:35:19.453 INFO:tasks.workunit.client.0.smithi007.stdout:0/260: write d6/f13 [3621572,933] 0 2024-09-21T04:35:19.456 INFO:tasks.workunit.client.1.smithi053.stdout:6/858: write d2/d1b/d3d/d49/f75 [2264459,550] 0 2024-09-21T04:35:19.459 INFO:tasks.workunit.client.0.smithi007.stdout:3/232: dread dc/f10 [4194304,4194304] 0 2024-09-21T04:35:19.460 INFO:tasks.workunit.client.0.smithi007.stdout:3/233: chown dc/f10 9696 1 2024-09-21T04:35:19.460 INFO:tasks.workunit.client.1.smithi053.stdout:8/935: rename d3/d5/d21/ld6 to d3/d5/d4b/d5c/d86/l133 0 2024-09-21T04:35:19.462 INFO:tasks.workunit.client.1.smithi053.stdout:9/848: write d0/d1c/d52/d6f/dfb/f11f [460700,45829] 0 2024-09-21T04:35:19.462 INFO:tasks.workunit.client.1.smithi053.stdout:9/849: chown d0/d2/f13 3056 1 2024-09-21T04:35:19.465 INFO:tasks.workunit.client.1.smithi053.stdout:7/753: getdents d0/dca 0 2024-09-21T04:35:19.466 INFO:tasks.workunit.client.1.smithi053.stdout:7/754: chown d0/d2b/d36/d64/ddb 36 1 2024-09-21T04:35:19.469 INFO:tasks.workunit.client.1.smithi053.stdout:4/952: symlink d1/dd/de/df5/d20/d4e/d51/da4/df1/l142 0 2024-09-21T04:35:19.470 INFO:tasks.workunit.client.1.smithi053.stdout:0/833: dread d5/d10/d4c/db1/dbc/dca/d21/d96/fbd [0,4194304] 0 2024-09-21T04:35:19.471 INFO:tasks.workunit.client.0.smithi007.stdout:4/302: mkdir d2/d3c/d63 0 2024-09-21T04:35:19.472 INFO:tasks.workunit.client.1.smithi053.stdout:1/871: symlink d7/da8/d74/l118 0 2024-09-21T04:35:19.473 INFO:tasks.workunit.client.1.smithi053.stdout:4/953: stat d1/dd/de/df5/d20/d4e/d51/da4/f118 0 2024-09-21T04:35:19.473 INFO:tasks.workunit.client.1.smithi053.stdout:6/859: creat d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8/f10f x:0 0 0 2024-09-21T04:35:19.474 INFO:tasks.workunit.client.1.smithi053.stdout:5/885: dwrite d0/dd/d6d/fa6 [4194304,4194304] 0 2024-09-21T04:35:19.475 INFO:tasks.workunit.client.1.smithi053.stdout:4/954: read - d1/dd/de/df5/d20/d73/d10b/f117 zero size 2024-09-21T04:35:19.475 INFO:tasks.workunit.client.0.smithi007.stdout:3/234: dwrite dc/d18/f31 [0,4194304] 0 2024-09-21T04:35:19.476 INFO:tasks.workunit.client.0.smithi007.stdout:0/261: unlink d6/d16/d18/l1f 0 2024-09-21T04:35:19.476 INFO:tasks.workunit.client.0.smithi007.stdout:3/235: chown dc/d22/f24 77844739 1 2024-09-21T04:35:19.476 INFO:tasks.workunit.client.0.smithi007.stdout:0/262: chown d6/d16/d52/f5c 1 1 2024-09-21T04:35:19.477 INFO:tasks.workunit.client.1.smithi053.stdout:8/936: mkdir d3/d5/d4c/dc3/ddb/d134 0 2024-09-21T04:35:19.478 INFO:tasks.workunit.client.0.smithi007.stdout:4/303: rmdir d2/d17 39 2024-09-21T04:35:19.484 INFO:tasks.workunit.client.0.smithi007.stdout:3/236: mknod dc/d22/c44 0 2024-09-21T04:35:19.485 INFO:tasks.workunit.client.1.smithi053.stdout:0/834: creat d5/d10/d4c/ddf/f113 x:0 0 0 2024-09-21T04:35:19.486 INFO:tasks.workunit.client.0.smithi007.stdout:4/304: write d2/f23 [827195,13508] 0 2024-09-21T04:35:19.487 INFO:tasks.workunit.client.1.smithi053.stdout:0/835: readlink d5/dd/d50/l7c 0 2024-09-21T04:35:19.487 INFO:tasks.workunit.client.0.smithi007.stdout:6/269: getdents d3/de/d12 0 2024-09-21T04:35:19.488 INFO:tasks.workunit.client.1.smithi053.stdout:2/901: dwrite d1/d18/d1f/d27/d58/d5c/d91/dd0/fb2 [0,4194304] 0 2024-09-21T04:35:19.489 INFO:tasks.workunit.client.0.smithi007.stdout:6/270: dread - d3/de/d4e/f57 zero size 2024-09-21T04:35:19.490 INFO:tasks.workunit.client.0.smithi007.stdout:0/263: rmdir d6/d16/d18/d39 39 2024-09-21T04:35:19.491 INFO:tasks.workunit.client.1.smithi053.stdout:2/902: chown d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/f122 12 1 2024-09-21T04:35:19.494 INFO:tasks.workunit.client.1.smithi053.stdout:0/836: dwrite d5/dd/d50/f9d [0,4194304] 0 2024-09-21T04:35:19.494 INFO:tasks.workunit.client.1.smithi053.stdout:3/857: sync 2024-09-21T04:35:19.494 INFO:tasks.workunit.client.1.smithi053.stdout:5/886: fdatasync d0/dd/d48/f54 0 2024-09-21T04:35:19.497 INFO:tasks.workunit.client.0.smithi007.stdout:6/271: dwrite d3/d3e/f40 [0,4194304] 0 2024-09-21T04:35:19.497 INFO:tasks.workunit.client.1.smithi053.stdout:5/887: chown d0/d8/f31 1 1 2024-09-21T04:35:19.499 INFO:tasks.workunit.client.0.smithi007.stdout:4/305: truncate d2/f25 297706 0 2024-09-21T04:35:19.501 INFO:tasks.workunit.client.1.smithi053.stdout:8/937: fsync d3/d28/f8b 0 2024-09-21T04:35:19.502 INFO:tasks.workunit.client.1.smithi053.stdout:9/850: dwrite d0/d1c/d25/f119 [0,4194304] 0 2024-09-21T04:35:19.505 INFO:tasks.workunit.client.0.smithi007.stdout:0/264: unlink d6/d16/d47/c49 0 2024-09-21T04:35:19.507 INFO:tasks.workunit.client.0.smithi007.stdout:3/237: link dc/d22/d29/d2d/f30 dc/d18/f45 0 2024-09-21T04:35:19.508 INFO:tasks.workunit.client.0.smithi007.stdout:3/238: truncate dc/d18/f42 4851415 0 2024-09-21T04:35:19.509 INFO:tasks.workunit.client.0.smithi007.stdout:3/239: chown dc/d22/d29/d2d/l36 0 1 2024-09-21T04:35:19.509 INFO:tasks.workunit.client.1.smithi053.stdout:7/755: unlink d0/d2b/d36/d64/dc0/fe6 0 2024-09-21T04:35:19.510 INFO:tasks.workunit.client.0.smithi007.stdout:6/272: mknod d3/d3e/d46/c5a 0 2024-09-21T04:35:19.514 INFO:tasks.workunit.client.1.smithi053.stdout:1/872: mknod d7/d63/d96/c119 0 2024-09-21T04:35:19.514 INFO:tasks.workunit.client.0.smithi007.stdout:0/265: symlink d6/d16/d18/d2e/d3f/l61 0 2024-09-21T04:35:19.516 INFO:tasks.workunit.client.1.smithi053.stdout:6/860: mkdir d2/d1d/d110 0 2024-09-21T04:35:19.517 INFO:tasks.workunit.client.1.smithi053.stdout:7/756: dwrite d0/d6f/dbf/d50/d8b/ff [0,4194304] 0 2024-09-21T04:35:19.518 INFO:tasks.workunit.client.0.smithi007.stdout:3/240: symlink dc/l46 0 2024-09-21T04:35:19.519 INFO:tasks.workunit.client.1.smithi053.stdout:1/873: truncate d7/d63/d96/dac/d106/f61 4688948 0 2024-09-21T04:35:19.520 INFO:tasks.workunit.client.1.smithi053.stdout:3/858: mknod d16/d1b/c139 0 2024-09-21T04:35:19.520 INFO:tasks.workunit.client.1.smithi053.stdout:1/874: chown d7/d9/da/d11/f33 3 1 2024-09-21T04:35:19.520 INFO:tasks.workunit.client.0.smithi007.stdout:0/266: symlink d6/d16/d57/l62 0 2024-09-21T04:35:19.522 INFO:tasks.workunit.client.1.smithi053.stdout:0/837: mkdir d5/de/d42/d78/d114 0 2024-09-21T04:35:19.527 INFO:tasks.workunit.client.1.smithi053.stdout:1/875: dwrite d7/d9/da/d4c/ffa [0,4194304] 0 2024-09-21T04:35:19.528 INFO:tasks.workunit.client.0.smithi007.stdout:3/241: rename c2 to dc/d22/d29/d32/d38/c47 0 2024-09-21T04:35:19.530 INFO:tasks.workunit.client.0.smithi007.stdout:0/267: chown d6/d16/d18/c3c 17778 1 2024-09-21T04:35:19.531 INFO:tasks.workunit.client.0.smithi007.stdout:6/273: write d3/de/d12/d19/f20 [975731,2591] 0 2024-09-21T04:35:19.535 INFO:tasks.workunit.client.0.smithi007.stdout:0/268: dread d6/d16/f22 [0,4194304] 0 2024-09-21T04:35:19.536 INFO:tasks.workunit.client.0.smithi007.stdout:3/242: creat dc/d22/d29/d2d/f48 x:0 0 0 2024-09-21T04:35:19.538 INFO:tasks.workunit.client.1.smithi053.stdout:4/955: truncate d1/d123/f108 399634 0 2024-09-21T04:35:19.539 INFO:tasks.workunit.client.1.smithi053.stdout:4/956: chown d1/dd/de/df5/d20/l21 17 1 2024-09-21T04:35:19.540 INFO:tasks.workunit.client.0.smithi007.stdout:4/306: truncate d2/d17/d18/f47 1449573 0 2024-09-21T04:35:19.540 INFO:tasks.workunit.client.0.smithi007.stdout:3/243: dread dc/d18/f31 [0,4194304] 0 2024-09-21T04:35:19.541 INFO:tasks.workunit.client.0.smithi007.stdout:3/244: chown dc/d22/f24 1 1 2024-09-21T04:35:19.544 INFO:tasks.workunit.client.1.smithi053.stdout:6/861: unlink d2/d1b/d1e/d2a/d4b/fd9 0 2024-09-21T04:35:19.545 INFO:tasks.workunit.client.1.smithi053.stdout:6/862: write d2/d1b/d1e/d2a/d4b/d77/d88/fca [7148489,98237] 0 2024-09-21T04:35:19.551 INFO:tasks.workunit.client.0.smithi007.stdout:4/307: truncate d2/d7/f8 411148 0 2024-09-21T04:35:19.554 INFO:tasks.workunit.client.0.smithi007.stdout:6/274: rename d3/de/f4d to d3/d3b/f5b 0 2024-09-21T04:35:19.555 INFO:tasks.workunit.client.1.smithi053.stdout:0/838: mkdir d5/d10/d4c/ddf/d115 0 2024-09-21T04:35:19.557 INFO:tasks.workunit.client.0.smithi007.stdout:3/245: rmdir dc/d22 39 2024-09-21T04:35:19.558 INFO:tasks.workunit.client.0.smithi007.stdout:6/275: dread d3/d3e/f43 [0,4194304] 0 2024-09-21T04:35:19.558 INFO:tasks.workunit.client.1.smithi053.stdout:1/876: creat d7/d47/f11a x:0 0 0 2024-09-21T04:35:19.559 INFO:tasks.workunit.client.0.smithi007.stdout:0/269: write d6/d16/d18/d27/f29 [338365,83778] 0 2024-09-21T04:35:19.560 INFO:tasks.workunit.client.0.smithi007.stdout:0/270: write d6/d16/d57/f59 [957655,30645] 0 2024-09-21T04:35:19.561 INFO:tasks.workunit.client.1.smithi053.stdout:7/757: dwrite d0/d6f/dbf/d50/d8b/f1b [0,4194304] 0 2024-09-21T04:35:19.563 INFO:tasks.workunit.client.1.smithi053.stdout:5/888: truncate d0/d8/f1d 3723656 0 2024-09-21T04:35:19.564 INFO:tasks.workunit.client.0.smithi007.stdout:4/308: mkdir d2/d7/d64 0 2024-09-21T04:35:19.565 INFO:tasks.workunit.client.1.smithi053.stdout:1/877: dread d7/d9/da/d40/d49/fff [0,4194304] 0 2024-09-21T04:35:19.566 INFO:tasks.workunit.client.0.smithi007.stdout:4/309: truncate d2/d17/d2e/d45/f60 586383 0 2024-09-21T04:35:19.569 INFO:tasks.workunit.client.1.smithi053.stdout:3/859: dwrite d16/d17/d21/d8b/d7d/d57/f6b [0,4194304] 0 2024-09-21T04:35:19.571 INFO:tasks.workunit.client.1.smithi053.stdout:4/957: mknod d1/d123/c143 0 2024-09-21T04:35:19.574 INFO:tasks.workunit.client.0.smithi007.stdout:4/310: dwrite d2/d7/d55/d4f/f5c [0,4194304] 0 2024-09-21T04:35:19.575 INFO:tasks.workunit.client.1.smithi053.stdout:2/903: rename d1/l2 to d1/d18/d1f/d27/l13e 0 2024-09-21T04:35:19.577 INFO:tasks.workunit.client.1.smithi053.stdout:1/878: dwrite d7/d47/d79/d99/db0/fe7 [0,4194304] 0 2024-09-21T04:35:19.580 INFO:tasks.workunit.client.0.smithi007.stdout:6/276: dwrite d3/d3e/f43 [0,4194304] 0 2024-09-21T04:35:19.581 INFO:tasks.workunit.client.0.smithi007.stdout:6/277: readlink d3/l4 0 2024-09-21T04:35:19.582 INFO:tasks.workunit.client.0.smithi007.stdout:0/271: rename f4 to d6/d16/d18/d2e/d60/f63 0 2024-09-21T04:35:19.583 INFO:tasks.workunit.client.0.smithi007.stdout:6/278: write d3/de/d4e/f57 [373573,123825] 0 2024-09-21T04:35:19.586 INFO:tasks.workunit.client.1.smithi053.stdout:6/863: truncate d2/d1b/d3d/d49/de/f85 5186821 0 2024-09-21T04:35:19.586 INFO:tasks.workunit.client.1.smithi053.stdout:2/904: dwrite d1/d18/d1f/d22/d2d/dcd/f12c [0,4194304] 0 2024-09-21T04:35:19.588 INFO:tasks.workunit.client.1.smithi053.stdout:5/889: dread d0/d8/f31 [4194304,4194304] 0 2024-09-21T04:35:19.593 INFO:tasks.workunit.client.0.smithi007.stdout:3/246: dwrite dc/f15 [0,4194304] 0 2024-09-21T04:35:19.593 INFO:tasks.workunit.client.0.smithi007.stdout:0/272: symlink d6/d16/d47/l64 0 2024-09-21T04:35:19.604 INFO:tasks.workunit.client.1.smithi053.stdout:4/958: unlink d1/dd/de/df5/d20/d73/d10b/c47 0 2024-09-21T04:35:19.605 INFO:tasks.workunit.client.0.smithi007.stdout:3/247: creat dc/d22/d29/d32/d37/f49 x:0 0 0 2024-09-21T04:35:19.606 INFO:tasks.workunit.client.1.smithi053.stdout:3/860: truncate d16/d17/d21/d26/d3e/f62 1049879 0 2024-09-21T04:35:19.606 INFO:tasks.workunit.client.1.smithi053.stdout:3/861: chown fc 178755451 1 2024-09-21T04:35:19.607 INFO:tasks.workunit.client.0.smithi007.stdout:0/273: mknod d6/d16/d18/d39/c65 0 2024-09-21T04:35:19.608 INFO:tasks.workunit.client.0.smithi007.stdout:0/274: write d6/d16/d57/f59 [1430520,48646] 0 2024-09-21T04:35:19.609 INFO:tasks.workunit.client.1.smithi053.stdout:8/938: sync 2024-09-21T04:35:19.610 INFO:tasks.workunit.client.1.smithi053.stdout:9/851: rename d0/d1a/d4c/l10c to d0/d2/d9/d34/d58/l128 0 2024-09-21T04:35:19.611 INFO:tasks.workunit.client.0.smithi007.stdout:3/248: rmdir dc/d22 39 2024-09-21T04:35:19.611 INFO:tasks.workunit.client.0.smithi007.stdout:4/311: write d2/f25 [1047929,22972] 0 2024-09-21T04:35:19.613 INFO:tasks.workunit.client.1.smithi053.stdout:1/879: mknod d7/d9/da/d40/d49/d77/c11b 0 2024-09-21T04:35:19.614 INFO:tasks.workunit.client.1.smithi053.stdout:1/880: write d7/d9/da/d11/f48 [1111691,107001] 0 2024-09-21T04:35:19.615 INFO:tasks.workunit.client.0.smithi007.stdout:0/275: symlink d6/d16/d57/l66 0 2024-09-21T04:35:19.616 INFO:tasks.workunit.client.0.smithi007.stdout:0/276: write d6/f38 [150319,1847] 0 2024-09-21T04:35:19.618 INFO:tasks.workunit.client.0.smithi007.stdout:6/279: getdents d3 0 2024-09-21T04:35:19.619 INFO:tasks.workunit.client.1.smithi053.stdout:5/890: creat d0/d8/db/db7/f122 x:0 0 0 2024-09-21T04:35:19.620 INFO:tasks.workunit.client.1.smithi053.stdout:2/905: dread d1/d18/d1f/d27/fa0 [0,4194304] 0 2024-09-21T04:35:19.620 INFO:tasks.workunit.client.1.smithi053.stdout:5/891: read d0/d8/f1f [582545,63635] 0 2024-09-21T04:35:19.621 INFO:tasks.workunit.client.1.smithi053.stdout:5/892: dread - d0/dd/d48/d8c/dad/ff8 zero size 2024-09-21T04:35:19.622 INFO:tasks.workunit.client.0.smithi007.stdout:3/249: creat dc/d18/d41/f4a x:0 0 0 2024-09-21T04:35:19.625 INFO:tasks.workunit.client.0.smithi007.stdout:0/277: creat d6/d16/d57/f67 x:0 0 0 2024-09-21T04:35:19.626 INFO:tasks.workunit.client.0.smithi007.stdout:6/280: creat d3/f5c x:0 0 0 2024-09-21T04:35:19.628 INFO:tasks.workunit.client.0.smithi007.stdout:4/312: symlink d2/d17/d2e/d45/d5e/l65 0 2024-09-21T04:35:19.631 INFO:tasks.workunit.client.0.smithi007.stdout:0/278: dwrite d6/d16/d18/d39/f54 [0,4194304] 0 2024-09-21T04:35:19.634 INFO:tasks.workunit.client.1.smithi053.stdout:6/864: dwrite d2/d1b/d1e/d27/d41/f46 [0,4194304] 0 2024-09-21T04:35:19.635 INFO:tasks.workunit.client.1.smithi053.stdout:5/893: dread d0/d8/d2a/fae [0,4194304] 0 2024-09-21T04:35:19.636 INFO:tasks.workunit.client.1.smithi053.stdout:8/939: dread d3/d5/d4b/d5c/d86/fc5 [0,4194304] 0 2024-09-21T04:35:19.639 INFO:tasks.workunit.client.1.smithi053.stdout:2/906: rmdir d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc 39 2024-09-21T04:35:19.639 INFO:tasks.workunit.client.1.smithi053.stdout:7/758: dwrite d0/f59 [0,4194304] 0 2024-09-21T04:35:19.640 INFO:tasks.workunit.client.1.smithi053.stdout:5/894: chown d0/d8/db6/dbc/df4/d119/fec 0 1 2024-09-21T04:35:19.641 INFO:tasks.workunit.client.1.smithi053.stdout:7/759: stat d0/d6f/dbf/d95/lbb 0 2024-09-21T04:35:19.641 INFO:tasks.workunit.client.1.smithi053.stdout:5/895: chown d0/d8/d2a/l34 1 1 2024-09-21T04:35:19.642 INFO:tasks.workunit.client.1.smithi053.stdout:7/760: fdatasync d0/d6f/dbf/d5c/ff8 0 2024-09-21T04:35:19.643 INFO:tasks.workunit.client.1.smithi053.stdout:5/896: chown d0/dd/d48/d8c 1621649499 1 2024-09-21T04:35:19.644 INFO:tasks.workunit.client.1.smithi053.stdout:1/881: stat d7/d9/d43/d54/d7f/d36/d81/d95/ccd 0 2024-09-21T04:35:19.645 INFO:tasks.workunit.client.1.smithi053.stdout:6/865: truncate d2/d1d/dbb/fac 898996 0 2024-09-21T04:35:19.647 INFO:tasks.workunit.client.1.smithi053.stdout:0/839: rename d5/d10/d4c/l7d to d5/de/d9e/l116 0 2024-09-21T04:35:19.647 INFO:tasks.workunit.client.1.smithi053.stdout:9/852: rmdir d0/d2/d2b/d105 0 2024-09-21T04:35:19.648 INFO:tasks.workunit.client.1.smithi053.stdout:8/940: symlink d3/df/l135 0 2024-09-21T04:35:19.649 INFO:tasks.workunit.client.1.smithi053.stdout:5/897: mknod d0/d8/db/c123 0 2024-09-21T04:35:19.650 INFO:tasks.workunit.client.1.smithi053.stdout:7/761: dread d0/d2b/d36/f80 [0,4194304] 0 2024-09-21T04:35:19.650 INFO:tasks.workunit.client.1.smithi053.stdout:5/898: fdatasync d0/fa4 0 2024-09-21T04:35:19.650 INFO:tasks.workunit.client.1.smithi053.stdout:1/882: unlink d7/d47/d79/dc1/le0 0 2024-09-21T04:35:19.651 INFO:tasks.workunit.client.1.smithi053.stdout:4/959: rename d1/d27/d53/fc2 to d1/dd/de/df5/d20/f144 0 2024-09-21T04:35:19.653 INFO:tasks.workunit.client.1.smithi053.stdout:7/762: chown d0/d6f/dbf/cff 52 1 2024-09-21T04:35:19.654 INFO:tasks.workunit.client.1.smithi053.stdout:2/907: truncate d1/d18/d1f/d22/f36 3772926 0 2024-09-21T04:35:19.655 INFO:tasks.workunit.client.1.smithi053.stdout:4/960: read d1/dd/de/df5/d20/d4e/d51/da4/df1/f85 [369977,20306] 0 2024-09-21T04:35:19.657 INFO:tasks.workunit.client.1.smithi053.stdout:9/853: truncate d0/df/f48 1585923 0 2024-09-21T04:35:19.659 INFO:tasks.workunit.client.1.smithi053.stdout:4/961: dread d1/dd/d95/fa2 [0,4194304] 0 2024-09-21T04:35:19.659 INFO:tasks.workunit.client.1.smithi053.stdout:2/908: dread d1/d18/d77/d9b/fed [0,4194304] 0 2024-09-21T04:35:19.660 INFO:tasks.workunit.client.1.smithi053.stdout:9/854: fsync d0/d1c/d52/d6f/dfb/f113 0 2024-09-21T04:35:19.661 INFO:tasks.workunit.client.1.smithi053.stdout:5/899: creat d0/dd/d5c/d92/ddb/d51/d52/f124 x:0 0 0 2024-09-21T04:35:19.661 INFO:tasks.workunit.client.1.smithi053.stdout:1/883: dwrite d7/d9/d43/d54/d7f/d59/f7b [0,4194304] 0 2024-09-21T04:35:19.662 INFO:tasks.workunit.client.1.smithi053.stdout:9/855: chown d0/d1a/d89/da1/dea 1 1 2024-09-21T04:35:19.663 INFO:tasks.workunit.client.1.smithi053.stdout:1/884: readlink d7/d63/d96/dac/lbb 0 2024-09-21T04:35:19.664 INFO:tasks.workunit.client.1.smithi053.stdout:3/862: rename d16/d17/d21/d42/d43/d125/d127/c5e to d16/d17/d21/d8b/d7d/d57/dc1/c13a 0 2024-09-21T04:35:19.666 INFO:tasks.workunit.client.1.smithi053.stdout:6/866: chown d2/d1b/d3d/d49/de/d32/d80/d2e/lce 15520 1 2024-09-21T04:35:19.666 INFO:tasks.workunit.client.1.smithi053.stdout:3/863: write d16/d17/d21/d26/d2b/f113 [234140,112599] 0 2024-09-21T04:35:19.668 INFO:tasks.workunit.client.1.smithi053.stdout:8/941: link d3/d5/d38/d64/f67 d3/df/d49/d100/f136 0 2024-09-21T04:35:19.670 INFO:tasks.workunit.client.1.smithi053.stdout:8/942: read d3/df/d10/fd3 [57695,41618] 0 2024-09-21T04:35:19.671 INFO:tasks.workunit.client.1.smithi053.stdout:4/962: creat d1/d123/f145 x:0 0 0 2024-09-21T04:35:19.672 INFO:tasks.workunit.client.1.smithi053.stdout:2/909: read d1/d18/d1f/d27/d41/d4b/d4d/f52 [3983400,122190] 0 2024-09-21T04:35:19.673 INFO:tasks.workunit.client.1.smithi053.stdout:5/900: creat d0/d8/d2a/dc8/f125 x:0 0 0 2024-09-21T04:35:19.675 INFO:tasks.workunit.client.1.smithi053.stdout:0/840: rename d5/de/d42/c62 to d5/de/d1c/c117 0 2024-09-21T04:35:19.675 INFO:tasks.workunit.client.1.smithi053.stdout:6/867: creat d2/d1b/d3d/de9/f111 x:0 0 0 2024-09-21T04:35:19.676 INFO:tasks.workunit.client.1.smithi053.stdout:3/864: fdatasync d16/d17/d21/d42/d43/d125/f128 0 2024-09-21T04:35:19.677 INFO:tasks.workunit.client.1.smithi053.stdout:6/868: chown d2/d1b/d3d/d49/de/d96/f7b 255328448 1 2024-09-21T04:35:19.677 INFO:tasks.workunit.client.1.smithi053.stdout:8/943: mknod d3/d5/db1/c137 0 2024-09-21T04:35:19.679 INFO:tasks.workunit.client.1.smithi053.stdout:5/901: symlink d0/dd/d48/d8c/dfd/l126 0 2024-09-21T04:35:19.679 INFO:tasks.workunit.client.1.smithi053.stdout:4/963: write d1/dd/de/f6d [3671675,87762] 0 2024-09-21T04:35:19.682 INFO:tasks.workunit.client.1.smithi053.stdout:8/944: dread - d3/d5/d4c/d8e/ff5 zero size 2024-09-21T04:35:19.683 INFO:tasks.workunit.client.1.smithi053.stdout:1/885: rename d7/d9/d43/d54/d7f/d36/d81/ff1 to d7/d9/d43/d54/d7f/d36/d81/f11c 0 2024-09-21T04:35:19.684 INFO:tasks.workunit.client.1.smithi053.stdout:7/763: link d0/c21 d0/d2b/c101 0 2024-09-21T04:35:19.686 INFO:tasks.workunit.client.1.smithi053.stdout:4/964: write d1/dd/de/df5/d20/d4e/faa [3085552,87231] 0 2024-09-21T04:35:19.687 INFO:tasks.workunit.client.1.smithi053.stdout:5/902: chown d0/dd/d5c/d92/ddb/d35/lcb 4557 1 2024-09-21T04:35:19.688 INFO:tasks.workunit.client.1.smithi053.stdout:9/856: dread d0/d1c/f39 [0,4194304] 0 2024-09-21T04:35:19.690 INFO:tasks.workunit.client.1.smithi053.stdout:1/886: readlink d7/d47/ldb 0 2024-09-21T04:35:19.691 INFO:tasks.workunit.client.1.smithi053.stdout:8/945: symlink d3/d5/d38/d64/d73/d119/d12a/l138 0 2024-09-21T04:35:19.693 INFO:tasks.workunit.client.1.smithi053.stdout:7/764: symlink d0/d6f/dbf/d50/d8b/d70/l102 0 2024-09-21T04:35:19.695 INFO:tasks.workunit.client.1.smithi053.stdout:5/903: fdatasync d0/f14 0 2024-09-21T04:35:19.698 INFO:tasks.workunit.client.1.smithi053.stdout:4/965: unlink d1/dd/de/df5/d20/d73/d10b/f141 0 2024-09-21T04:35:19.698 INFO:tasks.workunit.client.1.smithi053.stdout:6/869: dwrite d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8/f10f [0,4194304] 0 2024-09-21T04:35:19.701 INFO:tasks.workunit.client.1.smithi053.stdout:3/865: dread d16/d17/d21/d8b/d7d/dc6/f11a [0,4194304] 0 2024-09-21T04:35:19.702 INFO:tasks.workunit.client.1.smithi053.stdout:2/910: dread d1/d18/d1f/d27/d58/d5c/f62 [0,4194304] 0 2024-09-21T04:35:19.707 INFO:tasks.workunit.client.0.smithi007.stdout:4/313: dwrite d2/d17/d2e/f50 [0,4194304] 0 2024-09-21T04:35:19.707 INFO:tasks.workunit.client.1.smithi053.stdout:6/870: dread d2/fa [0,4194304] 0 2024-09-21T04:35:19.710 INFO:tasks.workunit.client.0.smithi007.stdout:0/279: rename d6/d16/d18/d39/c58 to d6/d16/d57/c68 0 2024-09-21T04:35:19.711 INFO:tasks.workunit.client.0.smithi007.stdout:0/280: chown d6/d16/d18/d27/f29 0 1 2024-09-21T04:35:19.713 INFO:tasks.workunit.client.0.smithi007.stdout:6/281: fdatasync d3/de/d12/d19/f20 0 2024-09-21T04:35:19.715 INFO:tasks.workunit.client.0.smithi007.stdout:0/281: read d6/f38 [85441,109536] 0 2024-09-21T04:35:19.723 INFO:tasks.workunit.client.0.smithi007.stdout:6/282: truncate d3/f2f 4671456 0 2024-09-21T04:35:19.726 INFO:tasks.workunit.client.1.smithi053.stdout:4/966: creat d1/dd/de/df5/d20/f146 x:0 0 0 2024-09-21T04:35:19.727 INFO:tasks.workunit.client.1.smithi053.stdout:5/904: dwrite d0/dd/f37 [0,4194304] 0 2024-09-21T04:35:19.727 INFO:tasks.workunit.client.1.smithi053.stdout:1/887: truncate d7/d63/d96/de2/f10c 1006678 0 2024-09-21T04:35:19.728 INFO:tasks.workunit.client.1.smithi053.stdout:9/857: link d0/d2/d2b/c3b d0/d1a/d89/c129 0 2024-09-21T04:35:19.728 INFO:tasks.workunit.client.1.smithi053.stdout:7/765: dwrite d0/d6f/dbf/d50/d8b/d70/dc1/ff6 [0,4194304] 0 2024-09-21T04:35:19.729 INFO:tasks.workunit.client.1.smithi053.stdout:2/911: rename d1/d18/d77 to d1/d18/d1f/d22/d2d/deb/d13f 0 2024-09-21T04:35:19.730 INFO:tasks.workunit.client.1.smithi053.stdout:0/841: write d5/f4a [799430,18715] 0 2024-09-21T04:35:19.730 INFO:tasks.workunit.client.1.smithi053.stdout:9/858: chown d0/d1c/d52/df7/d10d 111 1 2024-09-21T04:35:19.731 INFO:tasks.workunit.client.1.smithi053.stdout:4/967: truncate d1/d27/d90/fc0 4359624 0 2024-09-21T04:35:19.733 INFO:tasks.workunit.client.1.smithi053.stdout:4/968: chown d1/dd/f128 20246279 1 2024-09-21T04:35:19.734 INFO:tasks.workunit.client.1.smithi053.stdout:9/859: read d0/d2/d9/d34/f10e [461837,96081] 0 2024-09-21T04:35:19.735 INFO:tasks.workunit.client.1.smithi053.stdout:9/860: readlink d0/d1a/l8b 0 2024-09-21T04:35:19.735 INFO:tasks.workunit.client.1.smithi053.stdout:4/969: read d1/dd/de/f7a [6780408,95006] 0 2024-09-21T04:35:19.810 INFO:tasks.workunit.client.0.smithi007.stdout:4/314: fdatasync d2/d3c/f53 0 2024-09-21T04:35:19.811 INFO:tasks.workunit.client.0.smithi007.stdout:4/315: chown d2/d17/d2e/f50 0 1 2024-09-21T04:35:19.813 INFO:tasks.workunit.client.1.smithi053.stdout:8/946: link d3/d5/d38/d64/fe5 d3/d39/f139 0 2024-09-21T04:35:19.817 INFO:tasks.workunit.client.0.smithi007.stdout:6/283: dwrite d3/d3e/f55 [0,4194304] 0 2024-09-21T04:35:19.818 INFO:tasks.workunit.client.0.smithi007.stdout:6/284: fsync d3/de/d12/f1f 0 2024-09-21T04:35:19.835 INFO:tasks.workunit.client.1.smithi053.stdout:3/866: sync 2024-09-21T04:35:19.835 INFO:tasks.workunit.client.1.smithi053.stdout:3/867: chown d16/d17/d21/d42/d43/d125/d12e 756 1 2024-09-21T04:35:19.836 INFO:tasks.workunit.client.1.smithi053.stdout:3/868: stat d16/d17/d21/d26/d71 0 2024-09-21T04:35:20.020 INFO:tasks.workunit.client.0.smithi007.stdout:4/316: rename d2/d7/l11 to d2/d7/d64/l66 0 2024-09-21T04:35:20.025 INFO:tasks.workunit.client.0.smithi007.stdout:6/285: truncate d3/de/f4f 284960 0 2024-09-21T04:35:20.029 INFO:tasks.workunit.client.0.smithi007.stdout:4/317: dread d2/d17/d18/f47 [0,4194304] 0 2024-09-21T04:35:20.029 INFO:tasks.workunit.client.0.smithi007.stdout:4/318: stat d2/f6 0 2024-09-21T04:35:20.031 INFO:tasks.workunit.client.0.smithi007.stdout:6/286: rename d3/d3e/f53 to d3/de/d4e/f5d 0 2024-09-21T04:35:20.031 INFO:tasks.workunit.client.0.smithi007.stdout:6/287: dread - d3/f5c zero size 2024-09-21T04:35:20.032 INFO:tasks.workunit.client.0.smithi007.stdout:4/319: mknod d2/d17/d2e/d45/c67 0 2024-09-21T04:35:20.032 INFO:tasks.workunit.client.1.smithi053.stdout:6/871: write d2/d1d/f50 [1260884,25095] 0 2024-09-21T04:35:20.034 INFO:tasks.workunit.client.0.smithi007.stdout:4/320: write d2/d17/d18/f41 [487888,94412] 0 2024-09-21T04:35:20.034 INFO:tasks.workunit.client.0.smithi007.stdout:6/288: stat d3/d3b/f5b 0 2024-09-21T04:35:20.037 INFO:tasks.workunit.client.0.smithi007.stdout:4/321: rmdir d2/d17/d2e/d45/d5e 39 2024-09-21T04:35:20.038 INFO:tasks.workunit.client.0.smithi007.stdout:4/322: dread - d2/d17/f51 zero size 2024-09-21T04:35:20.038 INFO:tasks.workunit.client.1.smithi053.stdout:2/912: write d1/d18/d1f/d9d/f90 [5151641,78368] 0 2024-09-21T04:35:20.038 INFO:tasks.workunit.client.0.smithi007.stdout:4/323: fsync d2/f6 0 2024-09-21T04:35:20.039 INFO:tasks.workunit.client.0.smithi007.stdout:6/289: symlink d3/de/d4e/l5e 0 2024-09-21T04:35:20.040 INFO:tasks.workunit.client.1.smithi053.stdout:8/947: dwrite d3/d5/d38/d64/d73/d91/ffb [4194304,4194304] 0 2024-09-21T04:35:20.041 INFO:tasks.workunit.client.0.smithi007.stdout:6/290: write d3/de/f4f [576310,94448] 0 2024-09-21T04:35:20.041 INFO:tasks.workunit.client.1.smithi053.stdout:2/913: write d1/d18/d1f/d22/d43/da6/f100 [487821,67311] 0 2024-09-21T04:35:20.042 INFO:tasks.workunit.client.0.smithi007.stdout:4/324: creat d2/d7/d64/f68 x:0 0 0 2024-09-21T04:35:20.045 INFO:tasks.workunit.client.0.smithi007.stdout:6/291: unlink d3/d3e/d46/f4c 0 2024-09-21T04:35:20.046 INFO:tasks.workunit.client.0.smithi007.stdout:4/325: creat d2/d17/d29/f69 x:0 0 0 2024-09-21T04:35:20.052 INFO:tasks.workunit.client.0.smithi007.stdout:4/326: creat d2/f6a x:0 0 0 2024-09-21T04:35:20.055 INFO:tasks.workunit.client.0.smithi007.stdout:4/327: symlink d2/d17/d2e/d45/l6b 0 2024-09-21T04:35:20.060 INFO:tasks.workunit.client.0.smithi007.stdout:4/328: rename d2/d3c/f3f to d2/d7/f6c 0 2024-09-21T04:35:20.062 INFO:tasks.workunit.client.0.smithi007.stdout:4/329: symlink d2/d7/d64/l6d 0 2024-09-21T04:35:20.066 INFO:tasks.workunit.client.1.smithi053.stdout:5/905: mknod d0/dd/de4/c127 0 2024-09-21T04:35:20.067 INFO:tasks.workunit.client.0.smithi007.stdout:4/330: dread d2/d17/d2e/f33 [0,4194304] 0 2024-09-21T04:35:20.067 INFO:tasks.workunit.client.1.smithi053.stdout:1/888: creat d7/d9/d43/d54/d7f/d59/dc3/f11d x:0 0 0 2024-09-21T04:35:20.068 INFO:tasks.workunit.client.1.smithi053.stdout:1/889: readlink d7/d47/l92 0 2024-09-21T04:35:20.071 INFO:tasks.workunit.client.1.smithi053.stdout:0/842: creat d5/d10/d4c/db1/dbc/dca/db5/dda/f118 x:0 0 0 2024-09-21T04:35:20.072 INFO:tasks.workunit.client.1.smithi053.stdout:4/970: mknod d1/d27/d2e/d106/c147 0 2024-09-21T04:35:20.073 INFO:tasks.workunit.client.1.smithi053.stdout:6/872: readlink d2/d1b/d1e/d27/lb6 0 2024-09-21T04:35:20.074 INFO:tasks.workunit.client.1.smithi053.stdout:5/906: mknod d0/dd/d5c/d92/c128 0 2024-09-21T04:35:20.077 INFO:tasks.workunit.client.1.smithi053.stdout:1/890: truncate d7/d9/f1b 1032472 0 2024-09-21T04:35:20.078 INFO:tasks.workunit.client.1.smithi053.stdout:1/891: stat d7/d9/d43/d54/d7f/d36/d81/d95 0 2024-09-21T04:35:20.080 INFO:tasks.workunit.client.1.smithi053.stdout:4/971: mkdir d1/d123/d148 0 2024-09-21T04:35:20.083 INFO:tasks.workunit.client.1.smithi053.stdout:5/907: mkdir d0/d8/db/d68/d129 0 2024-09-21T04:35:20.084 INFO:tasks.workunit.client.1.smithi053.stdout:3/869: link d16/d17/d21/d26/d49/lb9 d16/d17/d21/d42/d43/d125/l13b 0 2024-09-21T04:35:20.085 INFO:tasks.workunit.client.1.smithi053.stdout:7/766: getdents d0/d6f/dbf/d50/d8b/d70/dc1 0 2024-09-21T04:35:20.089 INFO:tasks.workunit.client.1.smithi053.stdout:8/948: creat d3/d5/d38/d64/d73/f13a x:0 0 0 2024-09-21T04:35:20.091 INFO:tasks.workunit.client.1.smithi053.stdout:4/972: symlink d1/dd/de/df5/d20/d5b/d87/dd5/l149 0 2024-09-21T04:35:20.092 INFO:tasks.workunit.client.1.smithi053.stdout:5/908: mkdir d0/d12a 0 2024-09-21T04:35:20.094 INFO:tasks.workunit.client.1.smithi053.stdout:5/909: truncate d0/d8/d2a/f117 507156 0 2024-09-21T04:35:20.096 INFO:tasks.workunit.client.1.smithi053.stdout:2/914: getdents d1/d18/d1f/d27/d58/db7/d112/d115 0 2024-09-21T04:35:20.097 INFO:tasks.workunit.client.1.smithi053.stdout:2/915: fsync d1/d18/d1f/d27/d58/d5c/d91/dd0/f119 0 2024-09-21T04:35:20.098 INFO:tasks.workunit.client.1.smithi053.stdout:4/973: mkdir d1/d27/d53/d88/d14a 0 2024-09-21T04:35:20.099 INFO:tasks.workunit.client.1.smithi053.stdout:4/974: write d1/dd/de/f113 [218473,46578] 0 2024-09-21T04:35:20.099 INFO:tasks.workunit.client.1.smithi053.stdout:7/767: symlink d0/d2b/d36/d64/l103 0 2024-09-21T04:35:20.101 INFO:tasks.workunit.client.1.smithi053.stdout:1/892: rename d7/da8/d74/d91/fa2 to d7/d9/d43/d54/d7f/d36/d81/f11e 0 2024-09-21T04:35:20.101 INFO:tasks.workunit.client.1.smithi053.stdout:5/910: mknod d0/d8/db6/dbc/c12b 0 2024-09-21T04:35:20.102 INFO:tasks.workunit.client.1.smithi053.stdout:8/949: chown d3/d5/d1a/l3c 58472689 1 2024-09-21T04:35:20.103 INFO:tasks.workunit.client.1.smithi053.stdout:8/950: chown d3/df/d10/l2c 518162 1 2024-09-21T04:35:20.104 INFO:tasks.workunit.client.1.smithi053.stdout:8/951: write d3/df/d14/d16/d117/f128 [103759,3737] 0 2024-09-21T04:35:20.104 INFO:tasks.workunit.client.1.smithi053.stdout:3/870: creat d16/d1b/dc3/f13c x:0 0 0 2024-09-21T04:35:20.106 INFO:tasks.workunit.client.1.smithi053.stdout:4/975: mknod d1/dd/de/df5/d20/d5b/d87/dd5/c14b 0 2024-09-21T04:35:20.106 INFO:tasks.workunit.client.1.smithi053.stdout:3/871: chown d16/d17/d21/d8b/d7d/d57/df5/ffd 14056678 1 2024-09-21T04:35:20.106 INFO:tasks.workunit.client.1.smithi053.stdout:1/893: mkdir d7/d63/d96/de2/d11f 0 2024-09-21T04:35:20.107 INFO:tasks.workunit.client.1.smithi053.stdout:5/911: unlink d0/dd/d48/l55 0 2024-09-21T04:35:20.108 INFO:tasks.workunit.client.1.smithi053.stdout:1/894: chown d7/d47/c110 3686 1 2024-09-21T04:35:20.108 INFO:tasks.workunit.client.1.smithi053.stdout:2/916: symlink d1/d18/d1f/d22/l140 0 2024-09-21T04:35:20.111 INFO:tasks.workunit.client.1.smithi053.stdout:4/976: mkdir d1/d27/d2e/d106/d14c 0 2024-09-21T04:35:20.112 INFO:tasks.workunit.client.1.smithi053.stdout:3/872: write d16/d17/d21/d42/d43/d125/d127/f123 [973939,94022] 0 2024-09-21T04:35:20.112 INFO:tasks.workunit.client.1.smithi053.stdout:4/977: fdatasync d1/dd/de/df5/d3f/f77 0 2024-09-21T04:35:20.117 INFO:tasks.workunit.client.1.smithi053.stdout:7/768: link d0/fd d0/d24/f104 0 2024-09-21T04:35:20.117 INFO:tasks.workunit.client.1.smithi053.stdout:8/952: dread d3/df/dba/fca [4194304,4194304] 0 2024-09-21T04:35:20.118 INFO:tasks.workunit.client.1.smithi053.stdout:3/873: mknod d16/d17/d21/d8b/d110/c13d 0 2024-09-21T04:35:20.119 INFO:tasks.workunit.client.1.smithi053.stdout:5/912: creat d0/dd/d5c/d92/ddb/d35/f12c x:0 0 0 2024-09-21T04:35:20.122 INFO:tasks.workunit.client.1.smithi053.stdout:2/917: creat d1/d18/d1f/d27/d58/d5c/d91/f141 x:0 0 0 2024-09-21T04:35:20.123 INFO:tasks.workunit.client.1.smithi053.stdout:5/913: read d0/dd/d5c/d92/ddb/d51/d52/fc1 [438532,63745] 0 2024-09-21T04:35:20.124 INFO:tasks.workunit.client.1.smithi053.stdout:8/953: dwrite d3/df/d14/d16/d117/f128 [0,4194304] 0 2024-09-21T04:35:20.125 INFO:tasks.workunit.client.0.smithi007.stdout:4/331: truncate d2/d7/d55/d4f/f5c 794767 0 2024-09-21T04:35:20.128 INFO:tasks.workunit.client.1.smithi053.stdout:9/861: write d0/d2/d9/d1f/f26 [1283728,51439] 0 2024-09-21T04:35:20.129 INFO:tasks.workunit.client.1.smithi053.stdout:9/862: chown d0/d1a/d89/da1/dea/l116 27201225 1 2024-09-21T04:35:20.131 INFO:tasks.workunit.client.1.smithi053.stdout:7/769: dwrite d0/f13 [0,4194304] 0 2024-09-21T04:35:20.132 INFO:tasks.workunit.client.0.smithi007.stdout:4/332: unlink d2/d17/d18/c36 0 2024-09-21T04:35:20.133 INFO:tasks.workunit.client.1.smithi053.stdout:3/874: mknod d16/c13e 0 2024-09-21T04:35:20.136 INFO:tasks.workunit.client.0.smithi007.stdout:4/333: fsync d2/f1d 0 2024-09-21T04:35:20.141 INFO:tasks.workunit.client.1.smithi053.stdout:4/978: link d1/dd/de/df5/d20/d4e/d51/da4/df1/ld2 d1/d27/df8/l14d 0 2024-09-21T04:35:20.141 INFO:tasks.workunit.client.0.smithi007.stdout:4/334: rename d2/d17/d2e/d45 to d2/d17/d2e/d45/d5e/d6e 22 2024-09-21T04:35:20.144 INFO:tasks.workunit.client.1.smithi053.stdout:0/843: dwrite d5/d10/d4c/db1/dbc/dca/d21/fe0 [0,4194304] 0 2024-09-21T04:35:20.149 INFO:tasks.workunit.client.0.smithi007.stdout:4/335: symlink d2/d3c/l6f 0 2024-09-21T04:35:20.152 INFO:tasks.workunit.client.1.smithi053.stdout:8/954: truncate d3/df/f62 3227782 0 2024-09-21T04:35:20.154 INFO:tasks.workunit.client.1.smithi053.stdout:4/979: chown d1/cea 0 1 2024-09-21T04:35:20.155 INFO:tasks.workunit.client.0.smithi007.stdout:4/336: mknod d2/d7/d55/d61/c70 0 2024-09-21T04:35:20.155 INFO:tasks.workunit.client.1.smithi053.stdout:1/895: write d7/d47/d79/dc1/fc9 [74736,8285] 0 2024-09-21T04:35:20.155 INFO:tasks.workunit.client.1.smithi053.stdout:4/980: fdatasync d1/dd/de/df5/d20/dee/ff6 0 2024-09-21T04:35:20.158 INFO:tasks.workunit.client.0.smithi007.stdout:4/337: creat d2/d17/d2e/f71 x:0 0 0 2024-09-21T04:35:20.165 INFO:tasks.workunit.client.0.smithi007.stdout:4/338: rename d2/c42 to d2/d7/d64/c72 0 2024-09-21T04:35:20.169 INFO:tasks.workunit.client.1.smithi053.stdout:8/955: symlink d3/d5/d4b/d5c/d86/l13b 0 2024-09-21T04:35:20.170 INFO:tasks.workunit.client.0.smithi007.stdout:4/339: creat d2/d7/d55/d61/f73 x:0 0 0 2024-09-21T04:35:20.171 INFO:tasks.workunit.client.1.smithi053.stdout:1/896: stat d7/d63/d96/de2/f10c 0 2024-09-21T04:35:20.172 INFO:tasks.workunit.client.1.smithi053.stdout:7/770: mknod d0/d6f/dbf/d5d/d9f/c105 0 2024-09-21T04:35:20.172 INFO:tasks.workunit.client.1.smithi053.stdout:1/897: write d7/d9/d43/d54/d7f/d59/f7b [2622392,96341] 0 2024-09-21T04:35:20.176 INFO:tasks.workunit.client.1.smithi053.stdout:5/914: rename d0/d66/ce0 to d0/dd/d5c/c12d 0 2024-09-21T04:35:20.177 INFO:tasks.workunit.client.1.smithi053.stdout:7/771: dread d0/f59 [0,4194304] 0 2024-09-21T04:35:20.179 INFO:tasks.workunit.client.1.smithi053.stdout:9/863: getdents d0/d1c/db8 0 2024-09-21T04:35:20.180 INFO:tasks.workunit.client.1.smithi053.stdout:9/864: chown d0/df 37069812 1 2024-09-21T04:35:20.181 INFO:tasks.workunit.client.1.smithi053.stdout:3/875: write d16/d17/d21/d8b/d7d/d57/dc1/d53/f9a [1091642,82471] 0 2024-09-21T04:35:20.181 INFO:tasks.workunit.client.0.smithi007.stdout:7/290: sync 2024-09-21T04:35:20.181 INFO:tasks.workunit.client.0.smithi007.stdout:4/340: getdents d2/d7/d55 0 2024-09-21T04:35:20.181 INFO:tasks.workunit.client.1.smithi053.stdout:3/876: chown d16/d17/d21/d8b/d7d/l89 1 1 2024-09-21T04:35:20.183 INFO:tasks.workunit.client.1.smithi053.stdout:4/981: mkdir d1/dd/de/df5/d3f/d74/de1/d13b/d14e 0 2024-09-21T04:35:20.189 INFO:tasks.workunit.client.0.smithi007.stdout:7/291: dwrite d3/f5 [0,4194304] 0 2024-09-21T04:35:20.190 INFO:tasks.workunit.client.1.smithi053.stdout:9/865: symlink d0/d1c/d52/db5/l12a 0 2024-09-21T04:35:20.196 INFO:tasks.workunit.client.1.smithi053.stdout:4/982: dread d1/dd/de/df5/d20/d73/f79 [0,4194304] 0 2024-09-21T04:35:20.197 INFO:tasks.workunit.client.1.smithi053.stdout:8/956: rename d3/df/d14/l129 to d3/df/dba/l13c 0 2024-09-21T04:35:20.198 INFO:tasks.workunit.client.1.smithi053.stdout:2/918: getdents d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9 0 2024-09-21T04:35:20.199 INFO:tasks.workunit.client.0.smithi007.stdout:7/292: creat d3/d6/d2e/f6d x:0 0 0 2024-09-21T04:35:20.201 INFO:tasks.workunit.client.1.smithi053.stdout:7/772: link d0/d6f/dbf/d50/d8b/d70/dc1/ff6 d0/d6f/dbf/d95/f106 0 2024-09-21T04:35:20.201 INFO:tasks.workunit.client.1.smithi053.stdout:2/919: write d1/d18/d1f/d27/d41/f114 [1399412,6652] 0 2024-09-21T04:35:20.204 INFO:tasks.workunit.client.1.smithi053.stdout:3/877: write d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fbf [54555,109497] 0 2024-09-21T04:35:20.206 INFO:tasks.workunit.client.1.smithi053.stdout:5/915: rename d0/dd/d48/d8c to d0/d8/d12e 0 2024-09-21T04:35:20.207 INFO:tasks.workunit.client.0.smithi007.stdout:7/293: dwrite d3/d6/d13/f3c [0,4194304] 0 2024-09-21T04:35:20.207 INFO:tasks.workunit.client.0.smithi007.stdout:4/341: dwrite d2/f1d [0,4194304] 0 2024-09-21T04:35:20.208 INFO:tasks.workunit.client.1.smithi053.stdout:7/773: dread d0/d6f/ffa [0,4194304] 0 2024-09-21T04:35:20.208 INFO:tasks.workunit.client.1.smithi053.stdout:4/983: dwrite d1/dd/de/f6d [0,4194304] 0 2024-09-21T04:35:20.209 INFO:tasks.workunit.client.0.smithi007.stdout:4/342: chown d2/d7/cc 1878 1 2024-09-21T04:35:20.209 INFO:tasks.workunit.client.1.smithi053.stdout:8/957: chown d3/d5/d21/l4d 205787097 1 2024-09-21T04:35:20.209 INFO:tasks.workunit.client.1.smithi053.stdout:1/898: dwrite d7/d9/da/d40/d49/d77/feb [0,4194304] 0 2024-09-21T04:35:20.211 INFO:tasks.workunit.client.1.smithi053.stdout:3/878: chown d16/d17/d21/d8b/d7d/d57/dc1/d53/f9a 14641 1 2024-09-21T04:35:20.215 INFO:tasks.workunit.client.1.smithi053.stdout:7/774: read d0/d6f/dbf/d50/d8b/f42 [2352432,83272] 0 2024-09-21T04:35:20.215 INFO:tasks.workunit.client.1.smithi053.stdout:3/879: read - d16/d17/d21/d42/ff0 zero size 2024-09-21T04:35:20.216 INFO:tasks.workunit.client.1.smithi053.stdout:3/880: chown d16/d17/d21/d60/f9c 234362 1 2024-09-21T04:35:20.216 INFO:tasks.workunit.client.0.smithi007.stdout:4/343: unlink d2/d17/d18/f57 0 2024-09-21T04:35:20.216 INFO:tasks.workunit.client.1.smithi053.stdout:3/881: chown d16/d1b 88 1 2024-09-21T04:35:20.217 INFO:tasks.workunit.client.0.smithi007.stdout:7/294: dwrite d3/d6/f28 [8388608,4194304] 0 2024-09-21T04:35:20.218 INFO:tasks.workunit.client.1.smithi053.stdout:3/882: chown d16/d17/d21/d8b/d7d/d57/df5/f11d 6446 1 2024-09-21T04:35:20.219 INFO:tasks.workunit.client.0.smithi007.stdout:4/344: truncate d2/f27 3416982 0 2024-09-21T04:35:20.222 INFO:tasks.workunit.client.0.smithi007.stdout:7/295: creat d3/d6/d10/d51/f6e x:0 0 0 2024-09-21T04:35:20.223 INFO:tasks.workunit.client.0.smithi007.stdout:4/345: mknod d2/d17/d2e/d45/d5e/c74 0 2024-09-21T04:35:20.224 INFO:tasks.workunit.client.0.smithi007.stdout:4/346: chown d2/d17/d18/f54 1880718 1 2024-09-21T04:35:20.225 INFO:tasks.workunit.client.1.smithi053.stdout:2/920: creat d1/d18/d1f/d22/d2d/deb/f142 x:0 0 0 2024-09-21T04:35:20.226 INFO:tasks.workunit.client.1.smithi053.stdout:7/775: mknod d0/dcd/c107 0 2024-09-21T04:35:20.227 INFO:tasks.workunit.client.1.smithi053.stdout:2/921: chown d1/d18/d1f/d22/d43/f73 2 1 2024-09-21T04:35:20.228 INFO:tasks.workunit.client.1.smithi053.stdout:4/984: creat d1/d27/d53/d88/d14a/f14f x:0 0 0 2024-09-21T04:35:20.231 INFO:tasks.workunit.client.1.smithi053.stdout:3/883: symlink d16/d17/d21/d8b/d7d/d57/dc1/d53/l13f 0 2024-09-21T04:35:20.231 INFO:tasks.workunit.client.1.smithi053.stdout:7/776: write d0/d6f/dbf/d50/d8b/d70/fd6 [970236,12649] 0 2024-09-21T04:35:20.231 INFO:tasks.workunit.client.1.smithi053.stdout:1/899: rename d7/d9/d43/d54/d7f/d36/lfb to d7/d9/da/l120 0 2024-09-21T04:35:20.232 INFO:tasks.workunit.client.1.smithi053.stdout:6/873: sync 2024-09-21T04:35:20.232 INFO:tasks.workunit.client.0.smithi007.stdout:4/347: dwrite d2/f5a [0,4194304] 0 2024-09-21T04:35:20.235 INFO:tasks.workunit.client.1.smithi053.stdout:7/777: write d0/d6f/dbf/d95/feb [599699,56634] 0 2024-09-21T04:35:20.237 INFO:tasks.workunit.client.1.smithi053.stdout:9/866: write d0/d76/fc9 [1272828,50977] 0 2024-09-21T04:35:20.241 INFO:tasks.workunit.client.1.smithi053.stdout:9/867: chown d0/d2/c7 68 1 2024-09-21T04:35:20.241 INFO:tasks.workunit.client.1.smithi053.stdout:2/922: rename d1/d18/d1f/d27/d58/d5c/d91/cf3 to d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/d134/c143 0 2024-09-21T04:35:20.243 INFO:tasks.workunit.client.0.smithi007.stdout:4/348: dwrite d2/d17/d18/f54 [0,4194304] 0 2024-09-21T04:35:20.243 INFO:tasks.workunit.client.1.smithi053.stdout:1/900: fdatasync d7/d63/d96/de2/f10c 0 2024-09-21T04:35:20.244 INFO:tasks.workunit.client.0.smithi007.stdout:4/349: read d2/fd [1074225,101297] 0 2024-09-21T04:35:20.250 INFO:tasks.workunit.client.1.smithi053.stdout:1/901: chown d7/d47/cbf 4 1 2024-09-21T04:35:20.250 INFO:tasks.workunit.client.1.smithi053.stdout:3/884: dwrite d16/d17/d21/d42/d43/d86/fce [0,4194304] 0 2024-09-21T04:35:20.256 INFO:tasks.workunit.client.1.smithi053.stdout:5/916: dwrite d0/d8/db/f78 [4194304,4194304] 0 2024-09-21T04:35:20.257 INFO:tasks.workunit.client.1.smithi053.stdout:2/923: dread d1/d18/d1f/d27/d58/d5c/d91/dd0/f80 [0,4194304] 0 2024-09-21T04:35:20.258 INFO:tasks.workunit.client.1.smithi053.stdout:6/874: write d2/d1d/f54 [4572697,11954] 0 2024-09-21T04:35:20.259 INFO:tasks.workunit.client.1.smithi053.stdout:9/868: truncate d0/d2/d9/f43 3956372 0 2024-09-21T04:35:20.262 INFO:tasks.workunit.client.1.smithi053.stdout:4/985: link d1/dd/de/df5/d20/d5b/cbf d1/dd/de/df5/d3f/d74/c150 0 2024-09-21T04:35:20.263 INFO:tasks.workunit.client.1.smithi053.stdout:0/844: sync 2024-09-21T04:35:20.263 INFO:tasks.workunit.client.1.smithi053.stdout:1/902: write d7/d9/d43/d54/f80 [1514680,86672] 0 2024-09-21T04:35:20.266 INFO:tasks.workunit.client.1.smithi053.stdout:5/917: chown d0/dd/d5c/d92/ddb/d51/d52/f6c 4 1 2024-09-21T04:35:20.267 INFO:tasks.workunit.client.0.smithi007.stdout:4/350: symlink d2/l75 0 2024-09-21T04:35:20.268 INFO:tasks.workunit.client.0.smithi007.stdout:4/351: stat d2/d17/l4d 0 2024-09-21T04:35:20.269 INFO:tasks.workunit.client.0.smithi007.stdout:4/352: stat d2/d17/d2e/d45/d5e/c74 0 2024-09-21T04:35:20.270 INFO:tasks.workunit.client.0.smithi007.stdout:4/353: truncate d2/d17/d29/f3a 1690536 0 2024-09-21T04:35:20.271 INFO:tasks.workunit.client.1.smithi053.stdout:8/958: dwrite d3/df/d10/f94 [0,4194304] 0 2024-09-21T04:35:20.271 INFO:tasks.workunit.client.1.smithi053.stdout:4/986: chown d1/dd/de/df5/d20/d4e/fff 323308728 1 2024-09-21T04:35:20.272 INFO:tasks.workunit.client.1.smithi053.stdout:6/875: symlink d2/d1b/d1e/d2a/d4b/d77/d88/da4/l112 0 2024-09-21T04:35:20.273 INFO:tasks.workunit.client.0.smithi007.stdout:4/354: mknod d2/d7/c76 0 2024-09-21T04:35:20.273 INFO:tasks.workunit.client.1.smithi053.stdout:1/903: mknod d7/d9/da/d40/d49/d77/c121 0 2024-09-21T04:35:20.274 INFO:tasks.workunit.client.1.smithi053.stdout:3/885: fsync d16/d17/d21/d42/d43/d86/fce 0 2024-09-21T04:35:20.274 INFO:tasks.workunit.client.1.smithi053.stdout:5/918: read d0/dd/d6d/f11c [1240047,59901] 0 2024-09-21T04:35:20.279 INFO:tasks.workunit.client.1.smithi053.stdout:7/778: dread d0/fd [0,4194304] 0 2024-09-21T04:35:20.279 INFO:tasks.workunit.client.1.smithi053.stdout:4/987: chown d1/dd/de/df5/d20/d73/d10b/d41/ccb 26873913 1 2024-09-21T04:35:20.280 INFO:tasks.workunit.client.1.smithi053.stdout:6/876: mkdir d2/d1b/d1e/d27/d41/dc9/d113 0 2024-09-21T04:35:20.281 INFO:tasks.workunit.client.0.smithi007.stdout:4/355: dwrite d2/d17/d2e/f50 [0,4194304] 0 2024-09-21T04:35:20.282 INFO:tasks.workunit.client.1.smithi053.stdout:0/845: truncate d5/d10/d4c/db1/dbc/dca/db5/dda/f118 744331 0 2024-09-21T04:35:20.282 INFO:tasks.workunit.client.1.smithi053.stdout:2/924: dwrite d1/d18/d1f/d27/d58/d5c/f113 [0,4194304] 0 2024-09-21T04:35:20.285 INFO:tasks.workunit.client.0.smithi007.stdout:4/356: symlink d2/d17/d29/l77 0 2024-09-21T04:35:20.286 INFO:tasks.workunit.client.1.smithi053.stdout:1/904: fdatasync d7/d9/d43/d54/d7f/d36/f4e 0 2024-09-21T04:35:20.286 INFO:tasks.workunit.client.1.smithi053.stdout:8/959: mknod d3/d5/d38/d64/d9d/dff/c13d 0 2024-09-21T04:35:20.288 INFO:tasks.workunit.client.0.smithi007.stdout:4/357: creat d2/d17/d2e/f78 x:0 0 0 2024-09-21T04:35:20.290 INFO:tasks.workunit.client.1.smithi053.stdout:5/919: read d0/dd/d5c/d92/ddb/d51/d52/fc1 [387948,47444] 0 2024-09-21T04:35:20.290 INFO:tasks.workunit.client.1.smithi053.stdout:2/925: chown d1/d18/d1f/d22/d43/fd6 4855 1 2024-09-21T04:35:20.292 INFO:tasks.workunit.client.1.smithi053.stdout:7/779: mknod d0/d6f/dbf/d5c/c108 0 2024-09-21T04:35:20.293 INFO:tasks.workunit.client.1.smithi053.stdout:6/877: mknod d2/d1b/d3d/d49/d5c/c114 0 2024-09-21T04:35:20.295 INFO:tasks.workunit.client.0.smithi007.stdout:4/358: dwrite d2/f6a [0,4194304] 0 2024-09-21T04:35:20.297 INFO:tasks.workunit.client.1.smithi053.stdout:0/846: unlink d5/d10/d4c/db1/dbc/dca/d21/d96/fc2 0 2024-09-21T04:35:20.298 INFO:tasks.workunit.client.1.smithi053.stdout:1/905: mknod d7/d9/da/d40/d49/d77/c122 0 2024-09-21T04:35:20.300 INFO:tasks.workunit.client.1.smithi053.stdout:4/988: rename d1/dd/de/df5/d3f/d74/de1/fed to d1/d27/d53/d88/f151 0 2024-09-21T04:35:20.301 INFO:tasks.workunit.client.1.smithi053.stdout:2/926: read d1/f6d [907884,12815] 0 2024-09-21T04:35:20.302 INFO:tasks.workunit.client.1.smithi053.stdout:4/989: chown d1/dd/de/df5/d20/d73/d10b/d41 1473394 1 2024-09-21T04:35:20.304 INFO:tasks.workunit.client.0.smithi007.stdout:4/359: dwrite d2/d17/d2e/f71 [0,4194304] 0 2024-09-21T04:35:20.307 INFO:tasks.workunit.client.1.smithi053.stdout:8/960: dwrite d3/d5/d38/d64/d73/f13a [0,4194304] 0 2024-09-21T04:35:20.309 INFO:tasks.workunit.client.0.smithi007.stdout:4/360: unlink d2/d7/cc 0 2024-09-21T04:35:20.312 INFO:tasks.workunit.client.1.smithi053.stdout:3/886: dread d16/d17/d21/d42/d43/d86/fc7 [0,4194304] 0 2024-09-21T04:35:20.312 INFO:tasks.workunit.client.1.smithi053.stdout:9/869: dread d0/f15 [0,4194304] 0 2024-09-21T04:35:20.314 INFO:tasks.workunit.client.1.smithi053.stdout:7/780: dwrite d0/d24/fd9 [4194304,4194304] 0 2024-09-21T04:35:20.315 INFO:tasks.workunit.client.1.smithi053.stdout:3/887: fsync d16/d17/d21/d26/d2b/fb7 0 2024-09-21T04:35:20.320 INFO:tasks.workunit.client.0.smithi007.stdout:4/361: creat d2/d17/f79 x:0 0 0 2024-09-21T04:35:20.324 INFO:tasks.workunit.client.0.smithi007.stdout:4/362: unlink d2/d7/c34 0 2024-09-21T04:35:20.327 INFO:tasks.workunit.client.0.smithi007.stdout:4/363: mknod d2/d17/d2e/d45/d5d/c7a 0 2024-09-21T04:35:20.329 INFO:tasks.workunit.client.0.smithi007.stdout:4/364: mknod d2/d17/d2e/d45/c7b 0 2024-09-21T04:35:20.333 INFO:tasks.workunit.client.0.smithi007.stdout:4/365: dread d2/d17/d18/f54 [0,4194304] 0 2024-09-21T04:35:20.335 INFO:tasks.workunit.client.0.smithi007.stdout:4/366: write d2/d17/d18/f54 [1324328,38957] 0 2024-09-21T04:35:20.338 INFO:tasks.workunit.client.0.smithi007.stdout:4/367: creat d2/d17/d2e/f7c x:0 0 0 2024-09-21T04:35:20.339 INFO:tasks.workunit.client.0.smithi007.stdout:4/368: write d2/f1d [755641,50317] 0 2024-09-21T04:35:20.342 INFO:tasks.workunit.client.1.smithi053.stdout:0/847: mkdir d5/dd/d119 0 2024-09-21T04:35:20.342 INFO:tasks.workunit.client.1.smithi053.stdout:1/906: mkdir d7/d9/da/d40/d123 0 2024-09-21T04:35:20.343 INFO:tasks.workunit.client.1.smithi053.stdout:6/878: truncate d2/d1b/d1e/d2a/d4b/f58 648679 0 2024-09-21T04:35:20.345 INFO:tasks.workunit.client.1.smithi053.stdout:1/907: fdatasync d7/da8/d74/d91/f9a 0 2024-09-21T04:35:20.346 INFO:tasks.workunit.client.1.smithi053.stdout:2/927: fsync d1/d18/f67 0 2024-09-21T04:35:20.349 INFO:tasks.workunit.client.1.smithi053.stdout:5/920: dwrite d0/f14 [0,4194304] 0 2024-09-21T04:35:20.349 INFO:tasks.workunit.client.1.smithi053.stdout:2/928: stat d1/d18/d1f/d27/d58/d5c/d91/la1 0 2024-09-21T04:35:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:4/369: dread d2/d17/d2e/d45/f4b [0,4194304] 0 2024-09-21T04:35:20.366 INFO:tasks.workunit.client.0.smithi007.stdout:9/281: sync 2024-09-21T04:35:20.367 INFO:tasks.workunit.client.0.smithi007.stdout:5/519: sync 2024-09-21T04:35:20.367 INFO:tasks.workunit.client.1.smithi053.stdout:9/870: link d0/d1c/d21/d8e/fef d0/d1a/d89/f12b 0 2024-09-21T04:35:20.368 INFO:tasks.workunit.client.1.smithi053.stdout:0/848: mkdir d5/d10/d4c/db1/dbc/dca/d21/d101/d11a 0 2024-09-21T04:35:20.370 INFO:tasks.workunit.client.0.smithi007.stdout:4/370: stat d2/d3c/l4e 0 2024-09-21T04:35:20.371 INFO:tasks.workunit.client.0.smithi007.stdout:4/371: fdatasync d2/d17/d2e/f50 0 2024-09-21T04:35:20.371 INFO:tasks.workunit.client.1.smithi053.stdout:7/781: write d0/f44 [462188,119724] 0 2024-09-21T04:35:20.372 INFO:tasks.workunit.client.0.smithi007.stdout:4/372: write d2/d17/f79 [634712,96088] 0 2024-09-21T04:35:20.372 INFO:tasks.workunit.client.1.smithi053.stdout:0/849: dread d5/d10/d4c/db1/dbc/fcf [0,4194304] 0 2024-09-21T04:35:20.373 INFO:tasks.workunit.client.0.smithi007.stdout:5/520: fdatasync d2/d27/d3d/f69 0 2024-09-21T04:35:20.374 INFO:tasks.workunit.client.1.smithi053.stdout:4/990: creat d1/dd/de/df5/d20/d73/f152 x:0 0 0 2024-09-21T04:35:20.376 INFO:tasks.workunit.client.1.smithi053.stdout:6/879: mkdir d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/d115 0 2024-09-21T04:35:20.379 INFO:tasks.workunit.client.1.smithi053.stdout:4/991: dread d1/dd/de/df5/d3f/f77 [0,4194304] 0 2024-09-21T04:35:20.379 INFO:tasks.workunit.client.1.smithi053.stdout:1/908: rename d7/d9/da/d40/l53 to d7/d47/d79/dc1/l124 0 2024-09-21T04:35:20.380 INFO:tasks.workunit.client.1.smithi053.stdout:4/992: chown d1/d27/d2e/d106/c147 295 1 2024-09-21T04:35:20.382 INFO:tasks.workunit.client.0.smithi007.stdout:9/282: dread d13/d18/f34 [0,4194304] 0 2024-09-21T04:35:20.383 INFO:tasks.workunit.client.0.smithi007.stdout:4/373: chown d2/d7/d55/d61/f73 6677 1 2024-09-21T04:35:20.383 INFO:tasks.workunit.client.1.smithi053.stdout:2/929: truncate d1/d18/d1f/d22/d2d/deb/d13f/d83/f8d 2385061 0 2024-09-21T04:35:20.384 INFO:tasks.workunit.client.0.smithi007.stdout:9/283: write d13/d18/f2f [41319,2076] 0 2024-09-21T04:35:20.384 INFO:tasks.workunit.client.0.smithi007.stdout:5/521: mknod d2/d27/d3d/d9b/ca0 0 2024-09-21T04:35:20.385 INFO:tasks.workunit.client.1.smithi053.stdout:9/871: mkdir d0/d2/d4e/d12c 0 2024-09-21T04:35:20.386 INFO:tasks.workunit.client.0.smithi007.stdout:9/284: dread d13/d18/f1a [0,4194304] 0 2024-09-21T04:35:20.387 INFO:tasks.workunit.client.1.smithi053.stdout:1/909: dwrite d7/d63/d96/dc7/fed [0,4194304] 0 2024-09-21T04:35:20.387 INFO:tasks.workunit.client.1.smithi053.stdout:4/993: dwrite d1/dd/de/df5/d3f/f77 [0,4194304] 0 2024-09-21T04:35:20.388 INFO:tasks.workunit.client.0.smithi007.stdout:5/522: mkdir d2/dd/d23/da1 0 2024-09-21T04:35:20.390 INFO:tasks.workunit.client.1.smithi053.stdout:0/850: mknod d5/de/d42/d78/c11b 0 2024-09-21T04:35:20.394 INFO:tasks.workunit.client.1.smithi053.stdout:9/872: rename d0/d2/d9/fcc to d0/d1c/d52/df7/f12d 0 2024-09-21T04:35:20.394 INFO:tasks.workunit.client.0.smithi007.stdout:4/374: getdents d2/d7/d55 0 2024-09-21T04:35:20.396 INFO:tasks.workunit.client.1.smithi053.stdout:1/910: read - d7/d9/d43/d54/d7f/d36/f98 zero size 2024-09-21T04:35:20.396 INFO:tasks.workunit.client.1.smithi053.stdout:7/782: dwrite d0/d6f/dbf/d5d/d9f/fa6 [4194304,4194304] 0 2024-09-21T04:35:20.398 INFO:tasks.workunit.client.0.smithi007.stdout:4/375: read d2/d17/d29/f3a [417348,33475] 0 2024-09-21T04:35:20.405 INFO:tasks.workunit.client.0.smithi007.stdout:9/285: getdents d13 0 2024-09-21T04:35:20.407 INFO:tasks.workunit.client.0.smithi007.stdout:9/286: write d13/f5e [512191,123528] 0 2024-09-21T04:35:20.413 INFO:tasks.workunit.client.1.smithi053.stdout:7/783: creat d0/d6f/dbf/d5d/d9f/f109 x:0 0 0 2024-09-21T04:35:20.414 INFO:tasks.workunit.client.0.smithi007.stdout:9/287: fsync d13/f5b 0 2024-09-21T04:35:20.414 INFO:tasks.workunit.client.1.smithi053.stdout:7/784: chown d0/d6f/dbf/d5c/c108 0 1 2024-09-21T04:35:20.414 INFO:tasks.workunit.client.1.smithi053.stdout:4/994: creat d1/d27/d2e/d106/d14c/f153 x:0 0 0 2024-09-21T04:35:20.415 INFO:tasks.workunit.client.0.smithi007.stdout:9/288: chown d13/c48 363466 1 2024-09-21T04:35:20.415 INFO:tasks.workunit.client.0.smithi007.stdout:9/289: truncate fa 4955614 0 2024-09-21T04:35:20.415 INFO:tasks.workunit.client.1.smithi053.stdout:4/995: write d1/dd/de/df5/d20/d73/d10b/d41/fa3 [408005,5518] 0 2024-09-21T04:35:20.416 INFO:tasks.workunit.client.0.smithi007.stdout:9/290: chown d13/d18/c58 38062228 1 2024-09-21T04:35:20.416 INFO:tasks.workunit.client.0.smithi007.stdout:9/291: readlink d13/d18/l2e 0 2024-09-21T04:35:20.417 INFO:tasks.workunit.client.0.smithi007.stdout:9/292: dread - d13/f5b zero size 2024-09-21T04:35:20.417 INFO:tasks.workunit.client.1.smithi053.stdout:5/921: getdents d0/dd/d5c/d89 0 2024-09-21T04:35:20.418 INFO:tasks.workunit.client.0.smithi007.stdout:9/293: write d13/d2a/f2d [3718729,73654] 0 2024-09-21T04:35:20.419 INFO:tasks.workunit.client.1.smithi053.stdout:0/851: mknod d5/de/c11c 0 2024-09-21T04:35:20.420 INFO:tasks.workunit.client.0.smithi007.stdout:5/523: fdatasync d2/dd/d23/d2a/f2f 0 2024-09-21T04:35:20.422 INFO:tasks.workunit.client.1.smithi053.stdout:1/911: dwrite d7/d9/da/d11/f20 [0,4194304] 0 2024-09-21T04:35:20.424 INFO:tasks.workunit.client.1.smithi053.stdout:8/961: sync 2024-09-21T04:35:20.425 INFO:tasks.workunit.client.0.smithi007.stdout:4/376: dwrite d2/d7/f6c [0,4194304] 0 2024-09-21T04:35:20.425 INFO:tasks.workunit.client.1.smithi053.stdout:6/880: getdents d2/d1b/d3d 0 2024-09-21T04:35:20.426 INFO:tasks.workunit.client.0.smithi007.stdout:4/377: write d2/d17/f79 [465625,54138] 0 2024-09-21T04:35:20.427 INFO:tasks.workunit.client.1.smithi053.stdout:9/873: mkdir d0/d1a/dcd/d12e 0 2024-09-21T04:35:20.428 INFO:tasks.workunit.client.1.smithi053.stdout:9/874: dread - d0/d1a/d89/ff4 zero size 2024-09-21T04:35:20.428 INFO:tasks.workunit.client.0.smithi007.stdout:4/378: truncate d2/f43 1488750 0 2024-09-21T04:35:20.429 INFO:tasks.workunit.client.1.smithi053.stdout:6/881: write d2/d1b/d3d/de9/f111 [602715,47123] 0 2024-09-21T04:35:20.430 INFO:tasks.workunit.client.1.smithi053.stdout:6/882: readlink d2/d1b/d1e/d27/d41/lfc 0 2024-09-21T04:35:20.431 INFO:tasks.workunit.client.1.smithi053.stdout:9/875: write d0/d2/d2b/f124 [75828,5239] 0 2024-09-21T04:35:20.433 INFO:tasks.workunit.client.1.smithi053.stdout:1/912: dread d7/d9/d43/d54/d7f/d36/d81/f11c [0,4194304] 0 2024-09-21T04:35:20.434 INFO:tasks.workunit.client.0.smithi007.stdout:4/379: unlink d2/d3c/l4e 0 2024-09-21T04:35:20.435 INFO:tasks.workunit.client.0.smithi007.stdout:4/380: readlink d2/d7/d55/l56 0 2024-09-21T04:35:20.437 INFO:tasks.workunit.client.0.smithi007.stdout:5/524: symlink d2/dd/d23/d49/la2 0 2024-09-21T04:35:20.440 INFO:tasks.workunit.client.0.smithi007.stdout:9/294: dread d13/d2a/f3d [0,4194304] 0 2024-09-21T04:35:20.441 INFO:tasks.workunit.client.0.smithi007.stdout:9/295: fdatasync fa 0 2024-09-21T04:35:20.441 INFO:tasks.workunit.client.0.smithi007.stdout:9/296: dread - d13/f5b zero size 2024-09-21T04:35:20.442 INFO:tasks.workunit.client.1.smithi053.stdout:5/922: fsync d0/dd/d5c/d80/fd7 0 2024-09-21T04:35:20.442 INFO:tasks.workunit.client.1.smithi053.stdout:0/852: symlink d5/de/d42/d78/d94/l11d 0 2024-09-21T04:35:20.443 INFO:tasks.workunit.client.1.smithi053.stdout:4/996: dread d1/dd/de/df5/d20/d73/d10b/f65 [0,4194304] 0 2024-09-21T04:35:20.443 INFO:tasks.workunit.client.0.smithi007.stdout:9/297: readlink d13/l3e 0 2024-09-21T04:35:20.443 INFO:tasks.workunit.client.0.smithi007.stdout:9/298: chown f10 0 1 2024-09-21T04:35:20.443 INFO:tasks.workunit.client.0.smithi007.stdout:4/381: creat d2/d7/d64/f7d x:0 0 0 2024-09-21T04:35:20.444 INFO:tasks.workunit.client.1.smithi053.stdout:5/923: stat d0/dd/d5c/l96 0 2024-09-21T04:35:20.450 INFO:tasks.workunit.client.1.smithi053.stdout:2/930: link d1/d18/d1f/d22/l40 d1/df2/l144 0 2024-09-21T04:35:20.451 INFO:tasks.workunit.client.1.smithi053.stdout:8/962: getdents d3/d5/d4c/d8e/df6 0 2024-09-21T04:35:20.452 INFO:tasks.workunit.client.1.smithi053.stdout:4/997: rmdir d1/dd/de/df5/d20/d73/d10b 39 2024-09-21T04:35:20.453 INFO:tasks.workunit.client.0.smithi007.stdout:5/525: rename d2/dd to d2/d27/d42/da3 0 2024-09-21T04:35:20.453 INFO:tasks.workunit.client.1.smithi053.stdout:9/876: dread d0/d2/d9/d34/f56 [0,4194304] 0 2024-09-21T04:35:20.454 INFO:tasks.workunit.client.0.smithi007.stdout:5/526: fdatasync d2/d27/d42/da3/d28/f97 0 2024-09-21T04:35:20.454 INFO:tasks.workunit.client.1.smithi053.stdout:9/877: fsync d0/d1c/d25/f119 0 2024-09-21T04:35:20.455 INFO:tasks.workunit.client.1.smithi053.stdout:8/963: dread d3/d5/d4b/d5c/d86/f108 [0,4194304] 0 2024-09-21T04:35:20.455 INFO:tasks.workunit.client.1.smithi053.stdout:0/853: rename d5/d10/d4c/db1/dbc/dca/db5/dd5 to d5/d10/d4c/ddf/d103/d11e 0 2024-09-21T04:35:20.457 INFO:tasks.workunit.client.0.smithi007.stdout:4/382: rename d2/d7/d64/l6d to d2/d17/d29/l7e 0 2024-09-21T04:35:20.460 INFO:tasks.workunit.client.1.smithi053.stdout:2/931: creat d1/d18/d1f/da2/dac/f145 x:0 0 0 2024-09-21T04:35:20.463 INFO:tasks.workunit.client.0.smithi007.stdout:4/383: symlink d2/d7/d55/d4f/l7f 0 2024-09-21T04:35:20.464 INFO:tasks.workunit.client.1.smithi053.stdout:1/913: write d7/d9/da/d40/d49/d77/f100 [2812702,94268] 0 2024-09-21T04:35:20.465 INFO:tasks.workunit.client.1.smithi053.stdout:1/914: write d7/d24/fe1 [347558,40874] 0 2024-09-21T04:35:20.465 INFO:tasks.workunit.client.0.smithi007.stdout:9/299: write d13/f28 [2120155,7265] 0 2024-09-21T04:35:20.466 INFO:tasks.workunit.client.1.smithi053.stdout:9/878: rmdir d0/d2/d2b 39 2024-09-21T04:35:20.467 INFO:tasks.workunit.client.1.smithi053.stdout:9/879: dread - d0/d2/d4e/d5c/ffd zero size 2024-09-21T04:35:20.467 INFO:tasks.workunit.client.0.smithi007.stdout:5/527: dwrite d2/fc [0,4194304] 0 2024-09-21T04:35:20.468 INFO:tasks.workunit.client.0.smithi007.stdout:9/300: chown fa 2 1 2024-09-21T04:35:20.469 INFO:tasks.workunit.client.0.smithi007.stdout:9/301: chown fe 332 1 2024-09-21T04:35:20.469 INFO:tasks.workunit.client.1.smithi053.stdout:0/854: mknod d5/d10/d4c/db1/dbc/dca/db5/d99/c11f 0 2024-09-21T04:35:20.470 INFO:tasks.workunit.client.1.smithi053.stdout:2/932: fsync d1/d18/d1f/d22/d54/f85 0 2024-09-21T04:35:20.472 INFO:tasks.workunit.client.1.smithi053.stdout:2/933: dread - d1/d18/d1f/fb8 zero size 2024-09-21T04:35:20.472 INFO:tasks.workunit.client.1.smithi053.stdout:4/998: mknod d1/dd/de/df5/d20/d73/d10b/dec/c154 0 2024-09-21T04:35:20.473 INFO:tasks.workunit.client.0.smithi007.stdout:4/384: dwrite d2/f23 [0,4194304] 0 2024-09-21T04:35:20.475 INFO:tasks.workunit.client.1.smithi053.stdout:9/880: dwrite d0/d1c/d52/daa/ffa [0,4194304] 0 2024-09-21T04:35:20.475 INFO:tasks.workunit.client.1.smithi053.stdout:4/999: truncate d1/d27/d2e/f10d 1045257 0 2024-09-21T04:35:20.477 INFO:tasks.workunit.client.1.smithi053.stdout:1/915: unlink d7/d9/da/c23 0 2024-09-21T04:35:20.481 INFO:tasks.workunit.client.0.smithi007.stdout:9/302: mkdir d13/d18/d60 0 2024-09-21T04:35:20.483 INFO:tasks.workunit.client.0.smithi007.stdout:4/385: dwrite d2/f6 [0,4194304] 0 2024-09-21T04:35:20.483 INFO:tasks.workunit.client.0.smithi007.stdout:9/303: readlink d13/d2a/l4a 0 2024-09-21T04:35:20.484 INFO:tasks.workunit.client.1.smithi053.stdout:8/964: fsync d3/d39/f139 0 2024-09-21T04:35:20.484 INFO:tasks.workunit.client.0.smithi007.stdout:4/386: chown d2/d17/d2e/f71 927252866 1 2024-09-21T04:35:20.486 INFO:tasks.workunit.client.0.smithi007.stdout:9/304: read d13/d18/f3c [1165040,17487] 0 2024-09-21T04:35:20.486 INFO:tasks.workunit.client.0.smithi007.stdout:5/528: creat d2/d27/d42/da3/d23/d5d/d61/fa4 x:0 0 0 2024-09-21T04:35:20.492 INFO:tasks.workunit.client.1.smithi053.stdout:2/934: mkdir d1/d18/d1f/d27/d58/d5c/d91/dc1/d146 0 2024-09-21T04:35:20.493 INFO:tasks.workunit.client.1.smithi053.stdout:2/935: rename d1/d18/d1f/d27/d58/db7/d112 to d1/d18/d1f/d27/d58/db7/d112/d147 22 2024-09-21T04:35:20.494 INFO:tasks.workunit.client.0.smithi007.stdout:4/387: creat d2/d7/d55/f80 x:0 0 0 2024-09-21T04:35:20.494 INFO:tasks.workunit.client.0.smithi007.stdout:4/388: stat d2/f44 0 2024-09-21T04:35:20.495 INFO:tasks.workunit.client.0.smithi007.stdout:2/210: sync 2024-09-21T04:35:20.495 INFO:tasks.workunit.client.0.smithi007.stdout:1/451: sync 2024-09-21T04:35:20.495 INFO:tasks.workunit.client.0.smithi007.stdout:8/181: sync 2024-09-21T04:35:20.495 INFO:tasks.workunit.client.0.smithi007.stdout:3/250: sync 2024-09-21T04:35:20.496 INFO:tasks.workunit.client.0.smithi007.stdout:4/389: chown d2/d17/d2e/d45/d5e/c74 348 1 2024-09-21T04:35:20.497 INFO:tasks.workunit.client.0.smithi007.stdout:9/305: unlink d13/d18/f34 0 2024-09-21T04:35:20.501 INFO:tasks.workunit.client.0.smithi007.stdout:4/390: dread d2/d17/d18/f54 [0,4194304] 0 2024-09-21T04:35:20.501 INFO:tasks.workunit.client.0.smithi007.stdout:4/391: stat d2/d17/d29 0 2024-09-21T04:35:20.504 INFO:tasks.workunit.client.0.smithi007.stdout:8/182: write d0/d3/d8/de/f17 [3931224,81150] 0 2024-09-21T04:35:20.505 INFO:tasks.workunit.client.1.smithi053.stdout:2/936: stat d1/d18/d1f/d22/cda 0 2024-09-21T04:35:20.506 INFO:tasks.workunit.client.0.smithi007.stdout:2/211: write d0/d2/f2b [2972978,9503] 0 2024-09-21T04:35:20.507 INFO:tasks.workunit.client.0.smithi007.stdout:5/529: dread d2/d27/d42/da3/d1d/d20/f6d [0,4194304] 0 2024-09-21T04:35:20.508 INFO:tasks.workunit.client.1.smithi053.stdout:9/881: unlink d0/d2/ddf/le8 0 2024-09-21T04:35:20.509 INFO:tasks.workunit.client.1.smithi053.stdout:9/882: readlink d0/d2/d9/d34/d58/d92/l96 0 2024-09-21T04:35:20.514 INFO:tasks.workunit.client.1.smithi053.stdout:2/937: symlink d1/d18/d1f/d22/d2d/dcd/l148 0 2024-09-21T04:35:20.515 INFO:tasks.workunit.client.1.smithi053.stdout:1/916: truncate d7/d9/d43/d54/f80 1284327 0 2024-09-21T04:35:20.515 INFO:tasks.workunit.client.1.smithi053.stdout:0/855: dread d5/dd/d50/fc7 [0,4194304] 0 2024-09-21T04:35:20.516 INFO:tasks.workunit.client.0.smithi007.stdout:3/251: dwrite fb [0,4194304] 0 2024-09-21T04:35:20.517 INFO:tasks.workunit.client.0.smithi007.stdout:8/183: fsync d0/d3/d16/f2c 0 2024-09-21T04:35:20.517 INFO:tasks.workunit.client.1.smithi053.stdout:2/938: stat d1/d18/d1f/d22/d2d/deb/d13f/l99 0 2024-09-21T04:35:20.518 INFO:tasks.workunit.client.1.smithi053.stdout:8/965: dwrite d3/d28/f65 [0,4194304] 0 2024-09-21T04:35:20.520 INFO:tasks.workunit.client.1.smithi053.stdout:9/883: mkdir d0/d1a/dcd/d12e/d12f 0 2024-09-21T04:35:20.521 INFO:tasks.workunit.client.1.smithi053.stdout:8/966: chown d3/d5/d38/d64/d73/d119/d12a/l138 41 1 2024-09-21T04:35:20.521 INFO:tasks.workunit.client.1.smithi053.stdout:1/917: creat d7/da8/d74/d75/f125 x:0 0 0 2024-09-21T04:35:20.524 INFO:tasks.workunit.client.1.smithi053.stdout:0/856: mknod d5/de/d1c/c120 0 2024-09-21T04:35:20.525 INFO:tasks.workunit.client.0.smithi007.stdout:1/452: truncate d5/d6/d21/d2b/f51 3000507 0 2024-09-21T04:35:20.529 INFO:tasks.workunit.client.0.smithi007.stdout:2/212: write d0/d11/f28 [632458,58753] 0 2024-09-21T04:35:20.531 INFO:tasks.workunit.client.0.smithi007.stdout:9/306: write d13/d18/f54 [4685565,66555] 0 2024-09-21T04:35:20.532 INFO:tasks.workunit.client.0.smithi007.stdout:3/252: creat dc/d18/d26/f4b x:0 0 0 2024-09-21T04:35:20.534 INFO:tasks.workunit.client.1.smithi053.stdout:0/857: stat d5/de/l8f 0 2024-09-21T04:35:20.534 INFO:tasks.workunit.client.1.smithi053.stdout:3/888: sync 2024-09-21T04:35:20.534 INFO:tasks.workunit.client.1.smithi053.stdout:0/858: fsync d5/de/d1c/ff1 0 2024-09-21T04:35:20.535 INFO:tasks.workunit.client.1.smithi053.stdout:3/889: readlink d16/lbe 0 2024-09-21T04:35:20.535 INFO:tasks.workunit.client.1.smithi053.stdout:2/939: symlink d1/d18/d1f/d22/d2d/deb/d13f/d9b/d13a/l149 0 2024-09-21T04:35:20.537 INFO:tasks.workunit.client.1.smithi053.stdout:9/884: link d0/d1c/d52/d6d/f117 d0/d2/ddf/f130 0 2024-09-21T04:35:20.538 INFO:tasks.workunit.client.1.smithi053.stdout:9/885: chown d0/d6c/fa7 1 1 2024-09-21T04:35:20.539 INFO:tasks.workunit.client.0.smithi007.stdout:3/253: dwrite dc/d18/f39 [0,4194304] 0 2024-09-21T04:35:20.540 INFO:tasks.workunit.client.1.smithi053.stdout:9/886: write d0/d1a/d4c/d8f/f97 [1114051,70690] 0 2024-09-21T04:35:20.540 INFO:tasks.workunit.client.1.smithi053.stdout:0/859: dread d5/dd/d45/f6c [0,4194304] 0 2024-09-21T04:35:20.541 INFO:tasks.workunit.client.0.smithi007.stdout:1/453: rename d5/c22 to d5/d45/d5e/d55/d58/c91 0 2024-09-21T04:35:20.542 INFO:tasks.workunit.client.0.smithi007.stdout:4/392: link d2/d17/d2e/f78 d2/f81 0 2024-09-21T04:35:20.543 INFO:tasks.workunit.client.0.smithi007.stdout:4/393: chown d2/d7 335600 1 2024-09-21T04:35:20.543 INFO:tasks.workunit.client.0.smithi007.stdout:4/394: readlink d2/d7/d55/l56 0 2024-09-21T04:35:20.546 INFO:tasks.workunit.client.1.smithi053.stdout:3/890: creat d16/d1b/dad/f140 x:0 0 0 2024-09-21T04:35:20.549 INFO:tasks.workunit.client.0.smithi007.stdout:9/307: write d13/d2a/f3d [1011999,61470] 0 2024-09-21T04:35:20.549 INFO:tasks.workunit.client.0.smithi007.stdout:9/308: chown d13/f1d 671792 1 2024-09-21T04:35:20.551 INFO:tasks.workunit.client.0.smithi007.stdout:8/184: symlink d0/d3/d8/l2f 0 2024-09-21T04:35:20.551 INFO:tasks.workunit.client.0.smithi007.stdout:9/309: read d13/d18/f3f [789250,9759] 0 2024-09-21T04:35:20.552 INFO:tasks.workunit.client.1.smithi053.stdout:8/967: creat d3/d5/d38/d64/f13e x:0 0 0 2024-09-21T04:35:20.552 INFO:tasks.workunit.client.1.smithi053.stdout:3/891: dread d16/d17/d21/d8b/d7d/d57/df5/f11d [0,4194304] 0 2024-09-21T04:35:20.553 INFO:tasks.workunit.client.1.smithi053.stdout:8/968: readlink d3/d5/lb8 0 2024-09-21T04:35:20.554 INFO:tasks.workunit.client.1.smithi053.stdout:1/918: dwrite d7/d9/d43/d54/d7f/d36/d81/f11e [0,4194304] 0 2024-09-21T04:35:20.555 INFO:tasks.workunit.client.0.smithi007.stdout:3/254: chown dc/d22/d29/d2d/f30 956813 1 2024-09-21T04:35:20.557 INFO:tasks.workunit.client.1.smithi053.stdout:9/887: creat d0/d76/d79/f131 x:0 0 0 2024-09-21T04:35:20.558 INFO:tasks.workunit.client.1.smithi053.stdout:9/888: fdatasync d0/d1c/d52/df7/d10d/f11a 0 2024-09-21T04:35:20.559 INFO:tasks.workunit.client.0.smithi007.stdout:1/454: unlink d5/f17 0 2024-09-21T04:35:20.559 INFO:tasks.workunit.client.1.smithi053.stdout:9/889: chown d0/df/faf 571718457 1 2024-09-21T04:35:20.559 INFO:tasks.workunit.client.1.smithi053.stdout:9/890: write d0/d1c/d21/d8e/fba [3109075,54563] 0 2024-09-21T04:35:20.560 INFO:tasks.workunit.client.1.smithi053.stdout:0/860: mknod d5/d10/d4c/ddf/c121 0 2024-09-21T04:35:20.562 INFO:tasks.workunit.client.0.smithi007.stdout:4/395: fsync d2/d3c/f53 0 2024-09-21T04:35:20.564 INFO:tasks.workunit.client.1.smithi053.stdout:2/940: dwrite d1/d18/d1f/d27/d41/f64 [0,4194304] 0 2024-09-21T04:35:20.566 INFO:tasks.workunit.client.0.smithi007.stdout:9/310: creat d13/d5a/f61 x:0 0 0 2024-09-21T04:35:20.567 INFO:tasks.workunit.client.1.smithi053.stdout:3/892: symlink d16/d1b/l141 0 2024-09-21T04:35:20.569 INFO:tasks.workunit.client.0.smithi007.stdout:3/255: mknod dc/d18/c4c 0 2024-09-21T04:35:20.571 INFO:tasks.workunit.client.1.smithi053.stdout:7/785: sync 2024-09-21T04:35:20.574 INFO:tasks.workunit.client.0.smithi007.stdout:9/311: dwrite d13/d2a/f2d [0,4194304] 0 2024-09-21T04:35:20.574 INFO:tasks.workunit.client.1.smithi053.stdout:2/941: dwrite d1/d18/d1f/d27/d58/d5c/d91/dc1/f13c [0,4194304] 0 2024-09-21T04:35:20.575 INFO:tasks.workunit.client.1.smithi053.stdout:2/942: readlink d1/d18/l7d 0 2024-09-21T04:35:20.578 INFO:tasks.workunit.client.0.smithi007.stdout:4/396: creat d2/d17/d2e/d45/d5e/f82 x:0 0 0 2024-09-21T04:35:20.579 INFO:tasks.workunit.client.0.smithi007.stdout:2/213: link d0/d2/f6 d0/d32/f34 0 2024-09-21T04:35:20.579 INFO:tasks.workunit.client.0.smithi007.stdout:4/397: readlink d2/d17/l4d 0 2024-09-21T04:35:20.580 INFO:tasks.workunit.client.0.smithi007.stdout:2/214: write d0/d11/f28 [1498752,95191] 0 2024-09-21T04:35:20.580 INFO:tasks.workunit.client.1.smithi053.stdout:3/893: mkdir d16/d17/d21/d42/da0/dfb/d142 0 2024-09-21T04:35:20.587 INFO:tasks.workunit.client.0.smithi007.stdout:1/455: write d5/d45/d5e/d55/f80 [2113144,43164] 0 2024-09-21T04:35:20.588 INFO:tasks.workunit.client.1.smithi053.stdout:3/894: dwrite d16/d17/d21/d26/d3e/fd3 [0,4194304] 0 2024-09-21T04:35:20.590 INFO:tasks.workunit.client.0.smithi007.stdout:3/256: fdatasync dc/d22/d29/d2d/f30 0 2024-09-21T04:35:20.592 INFO:tasks.workunit.client.1.smithi053.stdout:8/969: creat d3/d5/d38/f13f x:0 0 0 2024-09-21T04:35:20.594 INFO:tasks.workunit.client.0.smithi007.stdout:2/215: dwrite d0/f2e [0,4194304] 0 2024-09-21T04:35:20.595 INFO:tasks.workunit.client.0.smithi007.stdout:8/185: dwrite d0/d3/d8/f1d [4194304,4194304] 0 2024-09-21T04:35:20.596 INFO:tasks.workunit.client.0.smithi007.stdout:8/186: stat d0/d3/d8/de/f17 0 2024-09-21T04:35:20.597 INFO:tasks.workunit.client.0.smithi007.stdout:2/216: chown d0/f8 16326151 1 2024-09-21T04:35:20.597 INFO:tasks.workunit.client.1.smithi053.stdout:1/919: creat d7/d9/d43/d54/d7f/f126 x:0 0 0 2024-09-21T04:35:20.599 INFO:tasks.workunit.client.1.smithi053.stdout:3/895: fdatasync d16/d17/d21/f6d 0 2024-09-21T04:35:20.599 INFO:tasks.workunit.client.1.smithi053.stdout:0/861: dwrite d5/f39 [0,4194304] 0 2024-09-21T04:35:20.599 INFO:tasks.workunit.client.0.smithi007.stdout:3/257: dread dc/d18/f31 [0,4194304] 0 2024-09-21T04:35:20.601 INFO:tasks.workunit.client.1.smithi053.stdout:1/920: readlink d7/d9/da/d40/d49/l111 0 2024-09-21T04:35:20.602 INFO:tasks.workunit.client.1.smithi053.stdout:3/896: stat d16/d1b/dc3 0 2024-09-21T04:35:20.605 INFO:tasks.workunit.client.1.smithi053.stdout:1/921: read d7/d9/d43/f6d [3182163,66730] 0 2024-09-21T04:35:20.606 INFO:tasks.workunit.client.0.smithi007.stdout:9/312: dread d13/d2a/f47 [0,4194304] 0 2024-09-21T04:35:20.606 INFO:tasks.workunit.client.1.smithi053.stdout:8/970: dwrite d3/df/dba/f120 [0,4194304] 0 2024-09-21T04:35:20.608 INFO:tasks.workunit.client.1.smithi053.stdout:1/922: chown d7/d9/da/d40/d58 2748359 1 2024-09-21T04:35:20.609 INFO:tasks.workunit.client.1.smithi053.stdout:7/786: dread d0/d6f/dbf/d50/d8b/d70/f7f [0,4194304] 0 2024-09-21T04:35:20.610 INFO:tasks.workunit.client.1.smithi053.stdout:9/891: getdents d0/d2 0 2024-09-21T04:35:20.612 INFO:tasks.workunit.client.0.smithi007.stdout:2/217: mknod d0/d11/c35 0 2024-09-21T04:35:20.612 INFO:tasks.workunit.client.1.smithi053.stdout:5/924: sync 2024-09-21T04:35:20.613 INFO:tasks.workunit.client.1.smithi053.stdout:6/883: sync 2024-09-21T04:35:20.613 INFO:tasks.workunit.client.0.smithi007.stdout:2/218: chown d0/d2/l1f 9521 1 2024-09-21T04:35:20.614 INFO:tasks.workunit.client.0.smithi007.stdout:2/219: read d0/fd [3918597,5267] 0 2024-09-21T04:35:20.615 INFO:tasks.workunit.client.0.smithi007.stdout:2/220: fdatasync d0/f19 0 2024-09-21T04:35:20.616 INFO:tasks.workunit.client.0.smithi007.stdout:2/221: readlink d0/d11/l1b 0 2024-09-21T04:35:20.616 INFO:tasks.workunit.client.0.smithi007.stdout:4/398: rename d2/d7/c14 to d2/d17/d2e/d45/d5e/c83 0 2024-09-21T04:35:20.616 INFO:tasks.workunit.client.1.smithi053.stdout:2/943: dwrite d1/d18/d1f/d27/d58/d5c/fcc [0,4194304] 0 2024-09-21T04:35:20.617 INFO:tasks.workunit.client.1.smithi053.stdout:2/944: dread - d1/d18/d1f/fb8 zero size 2024-09-21T04:35:20.617 INFO:tasks.workunit.client.0.smithi007.stdout:2/222: stat d0/c1d 0 2024-09-21T04:35:20.621 INFO:tasks.workunit.client.0.smithi007.stdout:4/399: creat d2/d17/d18/f84 x:0 0 0 2024-09-21T04:35:20.622 INFO:tasks.workunit.client.0.smithi007.stdout:4/400: stat d2/d7/d64/f7d 0 2024-09-21T04:35:20.623 INFO:tasks.workunit.client.1.smithi053.stdout:7/787: dread d0/d2b/f32 [0,4194304] 0 2024-09-21T04:35:20.624 INFO:tasks.workunit.client.0.smithi007.stdout:9/313: link d13/f23 d13/d18/d60/f62 0 2024-09-21T04:35:20.626 INFO:tasks.workunit.client.0.smithi007.stdout:2/223: mkdir d0/d36 0 2024-09-21T04:35:20.626 INFO:tasks.workunit.client.1.smithi053.stdout:9/892: creat d0/d1c/d52/d53/f132 x:0 0 0 2024-09-21T04:35:20.628 INFO:tasks.workunit.client.1.smithi053.stdout:5/925: fsync d0/dd/d5c/d89/fdd 0 2024-09-21T04:35:20.630 INFO:tasks.workunit.client.1.smithi053.stdout:2/945: dwrite d1/d18/d1f/d27/d58/d5c/d91/dc1/f13c [0,4194304] 0 2024-09-21T04:35:20.631 INFO:tasks.workunit.client.1.smithi053.stdout:2/946: chown d1/d18/d1f/d22/d54/cbe 6422 1 2024-09-21T04:35:20.633 INFO:tasks.workunit.client.1.smithi053.stdout:2/947: dread d1/d18/d1f/d22/d43/f59 [0,4194304] 0 2024-09-21T04:35:20.634 INFO:tasks.workunit.client.1.smithi053.stdout:2/948: fdatasync d1/d18/d1f/d22/d2d/dcd/dcf/f103 0 2024-09-21T04:35:20.634 INFO:tasks.workunit.client.0.smithi007.stdout:4/401: dwrite d2/d7/fa [4194304,4194304] 0 2024-09-21T04:35:20.635 INFO:tasks.workunit.client.0.smithi007.stdout:9/314: creat d13/d18/d60/f63 x:0 0 0 2024-09-21T04:35:20.638 INFO:tasks.workunit.client.1.smithi053.stdout:8/971: mkdir d3/d5/d1a/d140 0 2024-09-21T04:35:20.639 INFO:tasks.workunit.client.0.smithi007.stdout:3/258: dwrite dc/f13 [0,4194304] 0 2024-09-21T04:35:20.641 INFO:tasks.workunit.client.0.smithi007.stdout:2/224: creat d0/f37 x:0 0 0 2024-09-21T04:35:20.643 INFO:tasks.workunit.client.1.smithi053.stdout:9/893: read - d0/d2/d9/f7a zero size 2024-09-21T04:35:20.650 INFO:tasks.workunit.client.0.smithi007.stdout:3/259: mkdir dc/d22/d29/d32/d4d 0 2024-09-21T04:35:20.652 INFO:tasks.workunit.client.1.smithi053.stdout:0/862: rename d5/de/f82 to d5/de/f122 0 2024-09-21T04:35:20.655 INFO:tasks.workunit.client.1.smithi053.stdout:1/923: creat d7/da8/f127 x:0 0 0 2024-09-21T04:35:20.655 INFO:tasks.workunit.client.0.smithi007.stdout:3/260: write dc/f10 [8524773,111135] 0 2024-09-21T04:35:20.656 INFO:tasks.workunit.client.1.smithi053.stdout:6/884: dwrite d2/d1b/d3d/fd6 [0,4194304] 0 2024-09-21T04:35:20.656 INFO:tasks.workunit.client.0.smithi007.stdout:2/225: symlink d0/d32/l38 0 2024-09-21T04:35:20.657 INFO:tasks.workunit.client.0.smithi007.stdout:2/226: chown d0/c10 1013 1 2024-09-21T04:35:20.658 INFO:tasks.workunit.client.0.smithi007.stdout:2/227: chown d0/d2/f16 8 1 2024-09-21T04:35:20.658 INFO:tasks.workunit.client.0.smithi007.stdout:2/228: chown d0/c1d 12 1 2024-09-21T04:35:20.658 INFO:tasks.workunit.client.1.smithi053.stdout:2/949: rename d1/d18/d1f/d22/d2d/deb/d13f/d9b/d13a to d1/d18/d1f/d22/d43/d14a 0 2024-09-21T04:35:20.661 INFO:tasks.workunit.client.1.smithi053.stdout:9/894: unlink d0/d2/c7 0 2024-09-21T04:35:20.662 INFO:tasks.workunit.client.1.smithi053.stdout:5/926: unlink d0/d8/d2a/l43 0 2024-09-21T04:35:20.664 INFO:tasks.workunit.client.1.smithi053.stdout:0/863: dwrite d5/de/d9e/dd8/ff8 [0,4194304] 0 2024-09-21T04:35:20.664 INFO:tasks.workunit.client.0.smithi007.stdout:4/402: dread d2/d7/fa [0,4194304] 0 2024-09-21T04:35:20.666 INFO:tasks.workunit.client.1.smithi053.stdout:3/897: rename d16/d1b/dc3/c98 to d16/d1b/dc3/c143 0 2024-09-21T04:35:20.669 INFO:tasks.workunit.client.1.smithi053.stdout:3/898: truncate d16/d17/d21/d8b/d7d/d57/dc1/f132 582398 0 2024-09-21T04:35:20.673 INFO:tasks.workunit.client.0.smithi007.stdout:9/315: dread d13/d18/f3c [0,4194304] 0 2024-09-21T04:35:20.676 INFO:tasks.workunit.client.0.smithi007.stdout:3/261: getdents dc/d22/d29 0 2024-09-21T04:35:20.677 INFO:tasks.workunit.client.0.smithi007.stdout:4/403: dread d2/d17/d18/f54 [0,4194304] 0 2024-09-21T04:35:20.677 INFO:tasks.workunit.client.1.smithi053.stdout:5/927: dread d0/dd/d5c/d92/f10d [0,4194304] 0 2024-09-21T04:35:20.677 INFO:tasks.workunit.client.0.smithi007.stdout:2/229: dread d0/d2/f16 [0,4194304] 0 2024-09-21T04:35:20.678 INFO:tasks.workunit.client.1.smithi053.stdout:2/950: creat d1/d18/d1f/da2/f14b x:0 0 0 2024-09-21T04:35:20.679 INFO:tasks.workunit.client.1.smithi053.stdout:5/928: fdatasync d0/dd/d48/f11f 0 2024-09-21T04:35:20.681 INFO:tasks.workunit.client.1.smithi053.stdout:2/951: truncate d1/d18/d1f/d27/d58/db7/d112/d115/f130 428320 0 2024-09-21T04:35:20.682 INFO:tasks.workunit.client.1.smithi053.stdout:9/895: dread d0/d1a/d89/da1/dea/f10f [0,4194304] 0 2024-09-21T04:35:20.682 INFO:tasks.workunit.client.1.smithi053.stdout:2/952: chown d1/d18/c135 1 1 2024-09-21T04:35:20.683 INFO:tasks.workunit.client.0.smithi007.stdout:3/262: unlink dc/d18/d26/f3c 0 2024-09-21T04:35:20.683 INFO:tasks.workunit.client.1.smithi053.stdout:9/896: write d0/d1a/d4c/fab [2075915,101197] 0 2024-09-21T04:35:20.684 INFO:tasks.workunit.client.1.smithi053.stdout:2/953: fdatasync d1/d18/d1f/d27/d58/d5c/fd4 0 2024-09-21T04:35:20.684 INFO:tasks.workunit.client.1.smithi053.stdout:1/924: unlink d7/d9/da/l120 0 2024-09-21T04:35:20.685 INFO:tasks.workunit.client.1.smithi053.stdout:6/885: dread d2/d1b/d3d/d49/f75 [0,4194304] 0 2024-09-21T04:35:20.686 INFO:tasks.workunit.client.0.smithi007.stdout:4/404: creat d2/d17/d29/f85 x:0 0 0 2024-09-21T04:35:20.687 INFO:tasks.workunit.client.1.smithi053.stdout:6/886: stat d2/d1b/f6f 0 2024-09-21T04:35:20.687 INFO:tasks.workunit.client.1.smithi053.stdout:5/929: dread d0/dd/fcd [0,4194304] 0 2024-09-21T04:35:20.687 INFO:tasks.workunit.client.1.smithi053.stdout:1/925: fsync d7/d9/d43/d54/d7f/f126 0 2024-09-21T04:35:20.687 INFO:tasks.workunit.client.1.smithi053.stdout:1/926: chown d7/d9/da/f86 1538050154 1 2024-09-21T04:35:20.690 INFO:tasks.workunit.client.1.smithi053.stdout:8/972: creat d3/d5/d21/f141 x:0 0 0 2024-09-21T04:35:20.692 INFO:tasks.workunit.client.1.smithi053.stdout:8/973: chown d3/d5/lbc 5734232 1 2024-09-21T04:35:20.692 INFO:tasks.workunit.client.0.smithi007.stdout:3/263: rename dc/d18/c4c to dc/d22/d29/d2d/c4e 0 2024-09-21T04:35:20.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:20 smithi053 ceph-mon[35554]: pgmap v31: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 71 MiB/s rd, 240 MiB/s wr, 494 op/s 2024-09-21T04:35:20.693 INFO:tasks.workunit.client.0.smithi007.stdout:4/405: rmdir d2/d17/d2e/d45/d5d 39 2024-09-21T04:35:20.693 INFO:tasks.workunit.client.1.smithi053.stdout:1/927: dwrite d7/d63/f7d [0,4194304] 0 2024-09-21T04:35:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:20 smithi007 ceph-mon[27954]: pgmap v31: 65 pgs: 65 active+clean; 2.1 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 71 MiB/s rd, 240 MiB/s wr, 494 op/s 2024-09-21T04:35:20.695 INFO:tasks.workunit.client.0.smithi007.stdout:4/406: chown d2/d17/d2e/d45/d5e/c74 7125 1 2024-09-21T04:35:20.695 INFO:tasks.workunit.client.0.smithi007.stdout:2/230: mkdir d0/d11/d31/d39 0 2024-09-21T04:35:20.695 INFO:tasks.workunit.client.1.smithi053.stdout:0/864: creat d5/d10/d4c/db1/dbc/dca/db5/d10b/f123 x:0 0 0 2024-09-21T04:35:20.695 INFO:tasks.workunit.client.1.smithi053.stdout:1/928: dread d7/d63/d96/de2/f10c [0,4194304] 0 2024-09-21T04:35:20.696 INFO:tasks.workunit.client.0.smithi007.stdout:9/316: link d13/d2a/c2b d13/d18/d60/c64 0 2024-09-21T04:35:20.696 INFO:tasks.workunit.client.0.smithi007.stdout:2/231: chown d0/f37 3326706 1 2024-09-21T04:35:20.697 INFO:tasks.workunit.client.0.smithi007.stdout:3/264: dread dc/d18/f31 [0,4194304] 0 2024-09-21T04:35:20.697 INFO:tasks.workunit.client.1.smithi053.stdout:2/954: dread - d1/d18/d1f/d22/d2d/deb/d13f/fef zero size 2024-09-21T04:35:20.698 INFO:tasks.workunit.client.1.smithi053.stdout:0/865: truncate d5/d10/d4c/ddf/f113 534538 0 2024-09-21T04:35:20.698 INFO:tasks.workunit.client.0.smithi007.stdout:4/407: dread d2/d17/f26 [0,4194304] 0 2024-09-21T04:35:20.698 INFO:tasks.workunit.client.0.smithi007.stdout:9/317: read d13/d2a/f40 [3306579,29046] 0 2024-09-21T04:35:20.699 INFO:tasks.workunit.client.1.smithi053.stdout:5/930: creat d0/d8/db/db7/f12f x:0 0 0 2024-09-21T04:35:20.700 INFO:tasks.workunit.client.0.smithi007.stdout:2/232: chown d0/d2/l33 73290 1 2024-09-21T04:35:20.701 INFO:tasks.workunit.client.1.smithi053.stdout:6/887: mknod d2/d1b/d1e/d2a/d4b/d77/d88/c116 0 2024-09-21T04:35:20.701 INFO:tasks.workunit.client.1.smithi053.stdout:5/931: fsync d0/d8/d2a/f117 0 2024-09-21T04:35:20.701 INFO:tasks.workunit.client.1.smithi053.stdout:2/955: write d1/d18/d1f/d27/d41/d4b/d4d/f88 [5135505,18770] 0 2024-09-21T04:35:20.702 INFO:tasks.workunit.client.0.smithi007.stdout:3/265: mkdir dc/d22/d29/d32/d4f 0 2024-09-21T04:35:20.704 INFO:tasks.workunit.client.0.smithi007.stdout:4/408: creat d2/d17/d29/f86 x:0 0 0 2024-09-21T04:35:20.706 INFO:tasks.workunit.client.1.smithi053.stdout:7/788: sync 2024-09-21T04:35:20.707 INFO:tasks.workunit.client.0.smithi007.stdout:2/233: dwrite d0/d2/f2b [0,4194304] 0 2024-09-21T04:35:20.709 INFO:tasks.workunit.client.0.smithi007.stdout:2/234: dread d0/d2/f25 [0,4194304] 0 2024-09-21T04:35:20.711 INFO:tasks.workunit.client.0.smithi007.stdout:2/235: chown d0/d32/l38 28445 1 2024-09-21T04:35:20.717 INFO:tasks.workunit.client.1.smithi053.stdout:0/866: creat d5/de/d9e/dd2/f124 x:0 0 0 2024-09-21T04:35:20.717 INFO:tasks.workunit.client.0.smithi007.stdout:3/266: dread dc/d18/f28 [0,4194304] 0 2024-09-21T04:35:20.718 INFO:tasks.workunit.client.0.smithi007.stdout:3/267: readlink dc/l20 0 2024-09-21T04:35:20.718 INFO:tasks.workunit.client.0.smithi007.stdout:4/409: unlink d2/d17/d18/f41 0 2024-09-21T04:35:20.720 INFO:tasks.workunit.client.0.smithi007.stdout:7/296: sync 2024-09-21T04:35:20.720 INFO:tasks.workunit.client.0.smithi007.stdout:0/282: sync 2024-09-21T04:35:20.720 INFO:tasks.workunit.client.0.smithi007.stdout:6/292: sync 2024-09-21T04:35:20.720 INFO:tasks.workunit.client.0.smithi007.stdout:6/293: stat d3/de/d4e 0 2024-09-21T04:35:20.721 INFO:tasks.workunit.client.1.smithi053.stdout:3/899: write d16/d17/f1e [2464682,75633] 0 2024-09-21T04:35:20.721 INFO:tasks.workunit.client.0.smithi007.stdout:6/294: stat d3/de/d12/d19/f1b 0 2024-09-21T04:35:20.726 INFO:tasks.workunit.client.1.smithi053.stdout:2/956: chown d1/d18/d1f/d22/c53 204469 1 2024-09-21T04:35:20.728 INFO:tasks.workunit.client.0.smithi007.stdout:2/236: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:20.728 INFO:tasks.workunit.client.1.smithi053.stdout:5/932: rename d0/l12 to d0/d8/db/l130 0 2024-09-21T04:35:20.728 INFO:tasks.workunit.client.1.smithi053.stdout:8/974: write d3/d5/d4b/d5c/fa6 [37463,2695] 0 2024-09-21T04:35:20.729 INFO:tasks.workunit.client.1.smithi053.stdout:6/888: truncate d2/d1b/d1e/d2a/f4d 242297 0 2024-09-21T04:35:20.731 INFO:tasks.workunit.client.1.smithi053.stdout:7/789: creat d0/d6f/dbf/d50/d8b/f10a x:0 0 0 2024-09-21T04:35:20.731 INFO:tasks.workunit.client.1.smithi053.stdout:6/889: read d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8/f10f [1952808,33143] 0 2024-09-21T04:35:20.732 INFO:tasks.workunit.client.0.smithi007.stdout:7/297: creat d3/d6/d10/d43/f6f x:0 0 0 2024-09-21T04:35:20.733 INFO:tasks.workunit.client.0.smithi007.stdout:9/318: dwrite d13/d18/f1a [0,4194304] 0 2024-09-21T04:35:20.735 INFO:tasks.workunit.client.0.smithi007.stdout:9/319: chown d13/d18/c3a 33 1 2024-09-21T04:35:20.735 INFO:tasks.workunit.client.1.smithi053.stdout:9/897: mkdir d0/d2/d9/de7/d133 0 2024-09-21T04:35:20.736 INFO:tasks.workunit.client.0.smithi007.stdout:9/320: fdatasync d13/f21 0 2024-09-21T04:35:20.737 INFO:tasks.workunit.client.0.smithi007.stdout:3/268: mknod dc/c50 0 2024-09-21T04:35:20.739 INFO:tasks.workunit.client.1.smithi053.stdout:1/929: link d7/d9/d43/d54/d7f/f126 d7/da8/d74/d91/d9f/f128 0 2024-09-21T04:35:20.740 INFO:tasks.workunit.client.0.smithi007.stdout:4/410: creat d2/d5f/f87 x:0 0 0 2024-09-21T04:35:20.740 INFO:tasks.workunit.client.1.smithi053.stdout:3/900: mknod d16/d17/d21/d42/da0/c144 0 2024-09-21T04:35:20.741 INFO:tasks.workunit.client.0.smithi007.stdout:4/411: fsync d2/d7/d55/f80 0 2024-09-21T04:35:20.742 INFO:tasks.workunit.client.0.smithi007.stdout:2/237: unlink d0/d11/l1b 0 2024-09-21T04:35:20.744 INFO:tasks.workunit.client.1.smithi053.stdout:5/933: truncate d0/dd/d5c/d89/dc5/fce 782814 0 2024-09-21T04:35:20.747 INFO:tasks.workunit.client.1.smithi053.stdout:7/790: mknod d0/dcd/c10b 0 2024-09-21T04:35:20.748 INFO:tasks.workunit.client.0.smithi007.stdout:2/238: dwrite d0/d2/f2b [0,4194304] 0 2024-09-21T04:35:20.750 INFO:tasks.workunit.client.1.smithi053.stdout:0/867: dread d5/d10/fb9 [0,4194304] 0 2024-09-21T04:35:20.750 INFO:tasks.workunit.client.0.smithi007.stdout:9/321: symlink d13/d18/l65 0 2024-09-21T04:35:20.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/269: creat dc/d18/d26/f51 x:0 0 0 2024-09-21T04:35:20.754 INFO:tasks.workunit.client.1.smithi053.stdout:3/901: dread d16/d17/d21/d42/d43/d86/fb3 [0,4194304] 0 2024-09-21T04:35:20.760 INFO:tasks.workunit.client.0.smithi007.stdout:6/295: creat d3/d59/f5f x:0 0 0 2024-09-21T04:35:20.763 INFO:tasks.workunit.client.0.smithi007.stdout:2/239: rmdir d0/d11/d31 39 2024-09-21T04:35:20.765 INFO:tasks.workunit.client.0.smithi007.stdout:7/298: dread d3/d6/d14/d20/f35 [0,4194304] 0 2024-09-21T04:35:20.766 INFO:tasks.workunit.client.1.smithi053.stdout:5/934: unlink d0/d8/db/f78 0 2024-09-21T04:35:20.768 INFO:tasks.workunit.client.0.smithi007.stdout:9/322: creat d13/f66 x:0 0 0 2024-09-21T04:35:20.770 INFO:tasks.workunit.client.1.smithi053.stdout:9/898: dread d0/d76/f7e [0,4194304] 0 2024-09-21T04:35:20.770 INFO:tasks.workunit.client.0.smithi007.stdout:3/270: symlink dc/d18/d26/l52 0 2024-09-21T04:35:20.770 INFO:tasks.workunit.client.0.smithi007.stdout:3/271: write dc/f23 [4719759,95626] 0 2024-09-21T04:35:20.771 INFO:tasks.workunit.client.0.smithi007.stdout:4/412: mkdir d2/d88 0 2024-09-21T04:35:20.771 INFO:tasks.workunit.client.0.smithi007.stdout:3/272: fsync dc/d18/f39 0 2024-09-21T04:35:20.773 INFO:tasks.workunit.client.0.smithi007.stdout:6/296: mkdir d3/d3e/d46/d60 0 2024-09-21T04:35:20.775 INFO:tasks.workunit.client.1.smithi053.stdout:0/868: creat d5/d10/d4c/db1/dbc/dca/db5/f125 x:0 0 0 2024-09-21T04:35:20.775 INFO:tasks.workunit.client.1.smithi053.stdout:0/869: chown d5/c40 418834 1 2024-09-21T04:35:20.775 INFO:tasks.workunit.client.0.smithi007.stdout:2/240: fsync d0/f8 0 2024-09-21T04:35:20.776 INFO:tasks.workunit.client.0.smithi007.stdout:2/241: stat d0/d32 0 2024-09-21T04:35:20.776 INFO:tasks.workunit.client.1.smithi053.stdout:1/930: getdents d7/da8/d74/dce 0 2024-09-21T04:35:20.824 INFO:tasks.workunit.client.0.smithi007.stdout:0/283: truncate d6/d16/d18/d27/f35 3009242 0 2024-09-21T04:35:20.829 INFO:tasks.workunit.client.1.smithi053.stdout:3/902: truncate d16/d17/d21/d8b/d7d/d9d/fb4 77876 0 2024-09-21T04:35:20.829 INFO:tasks.workunit.client.1.smithi053.stdout:6/890: write d2/d1d/dbb/f84 [1688489,81380] 0 2024-09-21T04:35:20.835 INFO:tasks.workunit.client.0.smithi007.stdout:6/297: fsync d3/de/d12/d19/f36 0 2024-09-21T04:35:20.837 INFO:tasks.workunit.client.1.smithi053.stdout:5/935: creat d0/dd/d5c/d80/f131 x:0 0 0 2024-09-21T04:35:20.845 INFO:tasks.workunit.client.0.smithi007.stdout:0/284: write d6/f12 [8309486,108232] 0 2024-09-21T04:35:20.847 INFO:tasks.workunit.client.1.smithi053.stdout:1/931: unlink d7/d9/d43/d54/l10a 0 2024-09-21T04:35:20.852 INFO:tasks.workunit.client.1.smithi053.stdout:9/899: dread d0/d2/fe [4194304,4194304] 0 2024-09-21T04:35:20.853 INFO:tasks.workunit.client.0.smithi007.stdout:6/298: creat d3/d59/f61 x:0 0 0 2024-09-21T04:35:20.855 INFO:tasks.workunit.client.0.smithi007.stdout:9/323: write d13/f5b [940944,35769] 0 2024-09-21T04:35:20.857 INFO:tasks.workunit.client.1.smithi053.stdout:0/870: dread d5/de/f5b [0,4194304] 0 2024-09-21T04:35:20.859 INFO:tasks.workunit.client.1.smithi053.stdout:0/871: write d5/d10/d4c/db1/dbc/dca/d21/d96/f98 [6033569,97712] 0 2024-09-21T04:35:20.861 INFO:tasks.workunit.client.1.smithi053.stdout:0/872: readlink d5/d10/d4c/db1/dbc/dca/db5/la1 0 2024-09-21T04:35:20.864 INFO:tasks.workunit.client.0.smithi007.stdout:4/413: creat d2/d3c/f89 x:0 0 0 2024-09-21T04:35:20.871 INFO:tasks.workunit.client.0.smithi007.stdout:9/324: read d13/d18/f35 [869401,126208] 0 2024-09-21T04:35:20.881 INFO:tasks.workunit.client.0.smithi007.stdout:6/299: write d3/f28 [2178542,87507] 0 2024-09-21T04:35:20.891 INFO:tasks.workunit.client.0.smithi007.stdout:9/325: dwrite d13/d2a/f3d [0,4194304] 0 2024-09-21T04:35:20.901 INFO:tasks.workunit.client.0.smithi007.stdout:6/300: mkdir d3/d3e/d46/d60/d62 0 2024-09-21T04:35:20.905 INFO:tasks.workunit.client.0.smithi007.stdout:6/301: read d3/de/f42 [1422421,101743] 0 2024-09-21T04:35:20.914 INFO:tasks.workunit.client.0.smithi007.stdout:6/302: getdents d3 0 2024-09-21T04:35:20.919 INFO:tasks.workunit.client.0.smithi007.stdout:6/303: dread d3/de/d12/d19/f20 [0,4194304] 0 2024-09-21T04:35:20.923 INFO:tasks.workunit.client.0.smithi007.stdout:6/304: dread d3/d3e/f40 [0,4194304] 0 2024-09-21T04:35:20.926 INFO:tasks.workunit.client.0.smithi007.stdout:6/305: dread d3/fc [0,4194304] 0 2024-09-21T04:35:20.927 INFO:tasks.workunit.client.0.smithi007.stdout:6/306: stat d3/de/d12/f29 0 2024-09-21T04:35:20.927 INFO:tasks.workunit.client.0.smithi007.stdout:6/307: truncate d3/de/d12/f29 391595 0 2024-09-21T04:35:20.928 INFO:tasks.workunit.client.0.smithi007.stdout:6/308: chown d3/d3e/d46/c5a 2523 1 2024-09-21T04:35:20.931 INFO:tasks.workunit.client.0.smithi007.stdout:6/309: dread d3/fc [0,4194304] 0 2024-09-21T04:35:20.932 INFO:tasks.workunit.client.1.smithi053.stdout:2/957: getdents d1/d18/d1f/d27/d58 0 2024-09-21T04:35:20.933 INFO:tasks.workunit.client.1.smithi053.stdout:8/975: getdents d3/d5/d38 0 2024-09-21T04:35:20.934 INFO:tasks.workunit.client.1.smithi053.stdout:3/903: mknod d16/d17/d21/d8b/d7d/d57/d90/c145 0 2024-09-21T04:35:20.936 INFO:tasks.workunit.client.1.smithi053.stdout:3/904: truncate d16/d17/d21/d42/d43/d125/f128 471582 0 2024-09-21T04:35:20.937 INFO:tasks.workunit.client.1.smithi053.stdout:1/932: symlink d7/d9/da/d11/de5/l129 0 2024-09-21T04:35:20.940 INFO:tasks.workunit.client.1.smithi053.stdout:0/873: fdatasync d5/d10/d4c/db1/dbc/dca/d21/d96/feb 0 2024-09-21T04:35:20.943 INFO:tasks.workunit.client.1.smithi053.stdout:2/958: mknod d1/d18/d1f/d27/d41/d4b/ddc/c14c 0 2024-09-21T04:35:20.943 INFO:tasks.workunit.client.1.smithi053.stdout:2/959: chown d1/d18/d1f/d27/l4e 1848645277 1 2024-09-21T04:35:20.944 INFO:tasks.workunit.client.1.smithi053.stdout:5/936: creat d0/d8/db6/dbc/df4/f132 x:0 0 0 2024-09-21T04:35:20.945 INFO:tasks.workunit.client.1.smithi053.stdout:5/937: dread - d0/dd/d67/fe9 zero size 2024-09-21T04:35:20.946 INFO:tasks.workunit.client.1.smithi053.stdout:9/900: creat d0/d2/ddf/f134 x:0 0 0 2024-09-21T04:35:20.947 INFO:tasks.workunit.client.1.smithi053.stdout:5/938: fsync d0/dd/d48/f11f 0 2024-09-21T04:35:20.948 INFO:tasks.workunit.client.1.smithi053.stdout:2/960: symlink d1/d18/d1f/d27/d71/de3/l14d 0 2024-09-21T04:35:20.951 INFO:tasks.workunit.client.1.smithi053.stdout:8/976: dread d3/d5/d38/d64/d9d/da9/fe3 [0,4194304] 0 2024-09-21T04:35:20.951 INFO:tasks.workunit.client.1.smithi053.stdout:5/939: truncate d0/f14 8493569 0 2024-09-21T04:35:20.951 INFO:tasks.workunit.client.0.smithi007.stdout:7/299: dwrite d3/d6/d14/f26 [0,4194304] 0 2024-09-21T04:35:20.953 INFO:tasks.workunit.client.1.smithi053.stdout:2/961: dread - d1/d18/d1f/d22/d2d/deb/f142 zero size 2024-09-21T04:35:20.954 INFO:tasks.workunit.client.0.smithi007.stdout:7/300: truncate d3/d6/f2f 4390006 0 2024-09-21T04:35:20.954 INFO:tasks.workunit.client.1.smithi053.stdout:9/901: write d0/d2/d2b/f124 [420,74638] 0 2024-09-21T04:35:20.954 INFO:tasks.workunit.client.1.smithi053.stdout:3/905: dwrite d16/d17/d21/d8b/d7d/d57/df5/f11d [0,4194304] 0 2024-09-21T04:35:20.955 INFO:tasks.workunit.client.1.smithi053.stdout:1/933: getdents d7/d9/da 0 2024-09-21T04:35:20.956 INFO:tasks.workunit.client.1.smithi053.stdout:5/940: chown d0/dd/d5c/d92/ddb/d35/d61 114890 1 2024-09-21T04:35:20.957 INFO:tasks.workunit.client.1.smithi053.stdout:6/891: creat d2/d1d/f117 x:0 0 0 2024-09-21T04:35:20.958 INFO:tasks.workunit.client.0.smithi007.stdout:3/273: write dc/d22/d29/d2d/f30 [34834,122618] 0 2024-09-21T04:35:20.959 INFO:tasks.workunit.client.1.smithi053.stdout:5/941: chown d0/dd/d6d/caf 16125323 1 2024-09-21T04:35:20.960 INFO:tasks.workunit.client.1.smithi053.stdout:3/906: chown d16/d17/d21/d26 347187 1 2024-09-21T04:35:20.961 INFO:tasks.workunit.client.1.smithi053.stdout:6/892: fdatasync d2/d1b/d3d/de9/f111 0 2024-09-21T04:35:20.961 INFO:tasks.workunit.client.1.smithi053.stdout:3/907: stat d16/d17/d21/d42 0 2024-09-21T04:35:20.961 INFO:tasks.workunit.client.1.smithi053.stdout:5/942: readlink d0/d8/d2a/l50 0 2024-09-21T04:35:20.961 INFO:tasks.workunit.client.1.smithi053.stdout:1/934: creat d7/d47/dae/f12a x:0 0 0 2024-09-21T04:35:20.963 INFO:tasks.workunit.client.0.smithi007.stdout:3/274: write dc/d18/f42 [3461399,56306] 0 2024-09-21T04:35:20.963 INFO:tasks.workunit.client.0.smithi007.stdout:3/275: stat fa 0 2024-09-21T04:35:20.965 INFO:tasks.workunit.client.1.smithi053.stdout:3/908: fsync d16/d17/d21/d26/d2b/faa 0 2024-09-21T04:35:20.968 INFO:tasks.workunit.client.1.smithi053.stdout:5/943: symlink d0/d66/l133 0 2024-09-21T04:35:20.968 INFO:tasks.workunit.client.1.smithi053.stdout:9/902: dwrite d0/d2/d9/f7b [0,4194304] 0 2024-09-21T04:35:20.969 INFO:tasks.workunit.client.0.smithi007.stdout:3/276: fsync dc/d18/f28 0 2024-09-21T04:35:20.969 INFO:tasks.workunit.client.1.smithi053.stdout:5/944: read - d0/dd/d5c/d92/ddb/d35/f12c zero size 2024-09-21T04:35:20.969 INFO:tasks.workunit.client.0.smithi007.stdout:3/277: stat dc/d18/f31 0 2024-09-21T04:35:20.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/278: write dc/d18/f42 [4477390,113935] 0 2024-09-21T04:35:20.972 INFO:tasks.workunit.client.0.smithi007.stdout:3/279: write dc/f13 [6804023,96318] 0 2024-09-21T04:35:20.974 INFO:tasks.workunit.client.1.smithi053.stdout:6/893: creat d2/d1b/d3d/d49/d5c/f118 x:0 0 0 2024-09-21T04:35:20.974 INFO:tasks.workunit.client.1.smithi053.stdout:7/791: truncate d0/d6f/dbf/f8f 2211312 0 2024-09-21T04:35:20.980 INFO:tasks.workunit.client.1.smithi053.stdout:3/909: symlink d16/d17/d21/d8b/d7d/d57/dc1/d53/d12a/l146 0 2024-09-21T04:35:20.981 INFO:tasks.workunit.client.0.smithi007.stdout:3/280: write dc/d22/d29/d2d/f43 [2035786,7645] 0 2024-09-21T04:35:20.982 INFO:tasks.workunit.client.1.smithi053.stdout:5/945: dwrite d0/dd/d5c/d80/f131 [0,4194304] 0 2024-09-21T04:35:20.983 INFO:tasks.workunit.client.0.smithi007.stdout:3/281: dread - dc/d18/d41/f4a zero size 2024-09-21T04:35:20.984 INFO:tasks.workunit.client.1.smithi053.stdout:9/903: write d0/d2/fe [3758254,125971] 0 2024-09-21T04:35:20.986 INFO:tasks.workunit.client.0.smithi007.stdout:3/282: symlink dc/d22/d29/l53 0 2024-09-21T04:35:20.988 INFO:tasks.workunit.client.1.smithi053.stdout:6/894: write d2/d1b/d3d/f5e [623180,44139] 0 2024-09-21T04:35:20.990 INFO:tasks.workunit.client.1.smithi053.stdout:8/977: dread d3/f85 [0,4194304] 0 2024-09-21T04:35:20.991 INFO:tasks.workunit.client.1.smithi053.stdout:2/962: rename d1/l23 to d1/d18/d1f/d27/d58/d5c/l14e 0 2024-09-21T04:35:20.991 INFO:tasks.workunit.client.1.smithi053.stdout:8/978: chown d3/df/d14 1309113 1 2024-09-21T04:35:20.991 INFO:tasks.workunit.client.1.smithi053.stdout:3/910: rmdir d16/d17/d21/d8b/d7d/dc6 39 2024-09-21T04:35:20.993 INFO:tasks.workunit.client.0.smithi007.stdout:3/283: symlink dc/d22/d34/l54 0 2024-09-21T04:35:20.993 INFO:tasks.workunit.client.1.smithi053.stdout:8/979: write d3/d39/f87 [702857,36925] 0 2024-09-21T04:35:20.994 INFO:tasks.workunit.client.0.smithi007.stdout:3/284: write dc/d22/f24 [12573945,87939] 0 2024-09-21T04:35:20.996 INFO:tasks.workunit.client.1.smithi053.stdout:5/946: unlink d0/dd/d48/fa9 0 2024-09-21T04:35:20.997 INFO:tasks.workunit.client.1.smithi053.stdout:6/895: dread d2/d1b/d1e/d27/d41/dc9/ff1 [0,4194304] 0 2024-09-21T04:35:20.999 INFO:tasks.workunit.client.0.smithi007.stdout:3/285: creat dc/d22/d29/f55 x:0 0 0 2024-09-21T04:35:21.000 INFO:tasks.workunit.client.1.smithi053.stdout:5/947: dread - d0/dd/d5c/d92/ddb/d51/d52/f124 zero size 2024-09-21T04:35:21.001 INFO:tasks.workunit.client.1.smithi053.stdout:3/911: dwrite f9 [0,4194304] 0 2024-09-21T04:35:21.004 INFO:tasks.workunit.client.1.smithi053.stdout:8/980: mknod d3/d5/d4b/d5c/d86/dcd/d11b/c142 0 2024-09-21T04:35:21.009 INFO:tasks.workunit.client.1.smithi053.stdout:2/963: unlink d1/d18/d1f/d9d/c12b 0 2024-09-21T04:35:21.011 INFO:tasks.workunit.client.1.smithi053.stdout:3/912: creat d16/d1b/f147 x:0 0 0 2024-09-21T04:35:21.012 INFO:tasks.workunit.client.1.smithi053.stdout:8/981: mknod d3/d5/d4b/d5c/c143 0 2024-09-21T04:35:21.012 INFO:tasks.workunit.client.1.smithi053.stdout:7/792: link d0/c2d d0/d2b/d36/d64/ddb/c10c 0 2024-09-21T04:35:21.013 INFO:tasks.workunit.client.1.smithi053.stdout:6/896: getdents d2/d1b/d1e/dfb 0 2024-09-21T04:35:21.014 INFO:tasks.workunit.client.1.smithi053.stdout:5/948: truncate d0/d8/f31 5253501 0 2024-09-21T04:35:21.014 INFO:tasks.workunit.client.1.smithi053.stdout:7/793: read - d0/d6f/dbf/d50/d8b/d70/dc1/ffd zero size 2024-09-21T04:35:21.015 INFO:tasks.workunit.client.1.smithi053.stdout:5/949: dread - d0/dd/d48/f11f zero size 2024-09-21T04:35:21.015 INFO:tasks.workunit.client.1.smithi053.stdout:8/982: symlink d3/l144 0 2024-09-21T04:35:21.016 INFO:tasks.workunit.client.1.smithi053.stdout:3/913: creat d16/d17/d21/d8b/d7d/d57/d90/dfe/f148 x:0 0 0 2024-09-21T04:35:21.017 INFO:tasks.workunit.client.1.smithi053.stdout:7/794: stat d0/d6f/dbf/d5d/d9f/cb4 0 2024-09-21T04:35:21.018 INFO:tasks.workunit.client.1.smithi053.stdout:6/897: symlink d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8/l119 0 2024-09-21T04:35:21.018 INFO:tasks.workunit.client.1.smithi053.stdout:8/983: dread d3/d5/d4b/d5c/fa6 [0,4194304] 0 2024-09-21T04:35:21.019 INFO:tasks.workunit.client.1.smithi053.stdout:3/914: creat d16/d17/d21/d42/d43/d86/f149 x:0 0 0 2024-09-21T04:35:21.022 INFO:tasks.workunit.client.1.smithi053.stdout:3/915: dread - d16/d17/d21/d42/d43/d86/f119 zero size 2024-09-21T04:35:21.024 INFO:tasks.workunit.client.1.smithi053.stdout:8/984: creat d3/df/d49/d100/f145 x:0 0 0 2024-09-21T04:35:21.027 INFO:tasks.workunit.client.1.smithi053.stdout:9/904: symlink d0/d1a/l135 0 2024-09-21T04:35:21.030 INFO:tasks.workunit.client.0.smithi007.stdout:4/414: truncate d2/d7/f59 3053458 0 2024-09-21T04:35:21.031 INFO:tasks.workunit.client.1.smithi053.stdout:3/916: stat d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fda 0 2024-09-21T04:35:21.031 INFO:tasks.workunit.client.1.smithi053.stdout:5/950: dwrite d0/d8/d12e/dad/db9/fcf [0,4194304] 0 2024-09-21T04:35:21.032 INFO:tasks.workunit.client.1.smithi053.stdout:6/898: dwrite d2/d1b/d3d/f6c [0,4194304] 0 2024-09-21T04:35:21.032 INFO:tasks.workunit.client.1.smithi053.stdout:8/985: creat d3/d5/d4c/d8e/df6/f146 x:0 0 0 2024-09-21T04:35:21.034 INFO:tasks.workunit.client.1.smithi053.stdout:7/795: dwrite d0/d2b/d36/d64/ddb/f78 [0,4194304] 0 2024-09-21T04:35:21.035 INFO:tasks.workunit.client.1.smithi053.stdout:8/986: chown d3/df/d10/fbb 690143 1 2024-09-21T04:35:21.037 INFO:tasks.workunit.client.1.smithi053.stdout:9/905: fsync d0/d1a/d4c/f94 0 2024-09-21T04:35:21.037 INFO:tasks.workunit.client.0.smithi007.stdout:4/415: mkdir d2/d17/d2e/d45/d5d/d8a 0 2024-09-21T04:35:21.037 INFO:tasks.workunit.client.0.smithi007.stdout:5/530: sync 2024-09-21T04:35:21.039 INFO:tasks.workunit.client.1.smithi053.stdout:7/796: truncate d0/d6f/dbf/d5c/ff8 657727 0 2024-09-21T04:35:21.046 INFO:tasks.workunit.client.0.smithi007.stdout:4/416: mknod d2/d17/c8b 0 2024-09-21T04:35:21.047 INFO:tasks.workunit.client.1.smithi053.stdout:3/917: dread d16/d1b/dc3/f121 [0,4194304] 0 2024-09-21T04:35:21.049 INFO:tasks.workunit.client.1.smithi053.stdout:9/906: fdatasync d0/fcf 0 2024-09-21T04:35:21.050 INFO:tasks.workunit.client.0.smithi007.stdout:4/417: getdents d2/d17/d2e/d45/d5d/d8a 0 2024-09-21T04:35:21.051 INFO:tasks.workunit.client.0.smithi007.stdout:5/531: rmdir d2/d27/d42/da3/d23/d2a 39 2024-09-21T04:35:21.053 INFO:tasks.workunit.client.1.smithi053.stdout:8/987: fdatasync d3/f9c 0 2024-09-21T04:35:21.055 INFO:tasks.workunit.client.0.smithi007.stdout:5/532: creat d2/d27/d42/da3/d28/fa5 x:0 0 0 2024-09-21T04:35:21.061 INFO:tasks.workunit.client.1.smithi053.stdout:8/988: rename d3/d4f/fc9 to d3/d5/d38/d64/d73/d119/d12a/f147 0 2024-09-21T04:35:21.062 INFO:tasks.workunit.client.1.smithi053.stdout:7/797: link d0/d6f/dbf/d95/feb d0/d6f/dbf/f10d 0 2024-09-21T04:35:21.063 INFO:tasks.workunit.client.1.smithi053.stdout:9/907: mkdir d0/d2/d9/de7/d133/d136 0 2024-09-21T04:35:21.063 INFO:tasks.workunit.client.0.smithi007.stdout:5/533: mkdir d2/d27/d42/da3/d23/d5d/d9a/da6 0 2024-09-21T04:35:21.064 INFO:tasks.workunit.client.1.smithi053.stdout:5/951: getdents d0/d8/d12e/dfd 0 2024-09-21T04:35:21.070 INFO:tasks.workunit.client.0.smithi007.stdout:5/534: getdents d2/d27/d3d 0 2024-09-21T04:35:21.073 INFO:tasks.workunit.client.1.smithi053.stdout:5/952: link d0/dd/d5c/d89/dc5/fce d0/dd/d5c/d92/ddb/d51/f134 0 2024-09-21T04:35:21.075 INFO:tasks.workunit.client.1.smithi053.stdout:5/953: mkdir d0/dd/d6d/d135 0 2024-09-21T04:35:21.075 INFO:tasks.workunit.client.1.smithi053.stdout:5/954: readlink d0/d8/db/l20 0 2024-09-21T04:35:21.084 INFO:tasks.workunit.client.1.smithi053.stdout:1/935: write d7/d63/d96/dac/d106/f51 [1083617,66252] 0 2024-09-21T04:35:21.085 INFO:tasks.workunit.client.1.smithi053.stdout:1/936: write d7/d9/d43/f114 [725355,30263] 0 2024-09-21T04:35:21.087 INFO:tasks.workunit.client.0.smithi007.stdout:3/286: mkdir dc/d56 0 2024-09-21T04:35:21.088 INFO:tasks.workunit.client.0.smithi007.stdout:0/285: creat d6/d16/d18/f69 x:0 0 0 2024-09-21T04:35:21.088 INFO:tasks.workunit.client.0.smithi007.stdout:9/326: rmdir d13 39 2024-09-21T04:35:21.089 INFO:tasks.workunit.client.0.smithi007.stdout:2/242: rename d0/f30 to d0/d11/d31/d39/f3a 0 2024-09-21T04:35:21.091 INFO:tasks.workunit.client.0.smithi007.stdout:6/310: dwrite d3/de/d12/d19/f35 [0,4194304] 0 2024-09-21T04:35:21.092 INFO:tasks.workunit.client.0.smithi007.stdout:6/311: fsync d3/de/d12/f50 0 2024-09-21T04:35:21.096 INFO:tasks.workunit.client.1.smithi053.stdout:1/937: rename d7/d9/da/d40/d49/d77/l84 to d7/d47/d79/l12b 0 2024-09-21T04:35:21.097 INFO:tasks.workunit.client.0.smithi007.stdout:2/243: dread d0/d2/fc [0,4194304] 0 2024-09-21T04:35:21.099 INFO:tasks.workunit.client.0.smithi007.stdout:7/301: rename d3/d6/d14/d20/d58/f67 to d3/d6/d2e/d63/d6a/f70 0 2024-09-21T04:35:21.101 INFO:tasks.workunit.client.0.smithi007.stdout:9/327: dwrite d13/d2a/f31 [0,4194304] 0 2024-09-21T04:35:21.102 INFO:tasks.workunit.client.0.smithi007.stdout:9/328: write fa [2199764,50757] 0 2024-09-21T04:35:21.103 INFO:tasks.workunit.client.1.smithi053.stdout:1/938: creat d7/d24/f12c x:0 0 0 2024-09-21T04:35:21.103 INFO:tasks.workunit.client.0.smithi007.stdout:6/312: rmdir d3/d59 39 2024-09-21T04:35:21.111 INFO:tasks.workunit.client.0.smithi007.stdout:2/244: rename d0/l24 to d0/d11/d31/d39/l3b 0 2024-09-21T04:35:21.115 INFO:tasks.workunit.client.0.smithi007.stdout:9/329: creat d13/d18/f67 x:0 0 0 2024-09-21T04:35:21.117 INFO:tasks.workunit.client.1.smithi053.stdout:1/939: getdents d7/d9/d43/d54/d7f/d36/d81/d95 0 2024-09-21T04:35:21.118 INFO:tasks.workunit.client.1.smithi053.stdout:8/989: getdents d3/df/d49/d100 0 2024-09-21T04:35:21.119 INFO:tasks.workunit.client.1.smithi053.stdout:8/990: readlink d3/df/d10/l63 0 2024-09-21T04:35:21.120 INFO:tasks.workunit.client.1.smithi053.stdout:1/940: write d7/d47/d79/d99/fe4 [499846,88497] 0 2024-09-21T04:35:21.122 INFO:tasks.workunit.client.0.smithi007.stdout:9/330: write d13/f4f [615886,12848] 0 2024-09-21T04:35:21.123 INFO:tasks.workunit.client.1.smithi053.stdout:1/941: read d7/d9/d43/d54/d7f/fcc [50770,85028] 0 2024-09-21T04:35:21.124 INFO:tasks.workunit.client.0.smithi007.stdout:3/287: getdents dc/d22/d29/d2d 0 2024-09-21T04:35:21.126 INFO:tasks.workunit.client.0.smithi007.stdout:0/286: getdents d6/d16/d18/d2e/d3f 0 2024-09-21T04:35:21.126 INFO:tasks.workunit.client.0.smithi007.stdout:3/288: truncate dc/d22/d29/d32/d37/f40 568550 0 2024-09-21T04:35:21.128 INFO:tasks.workunit.client.0.smithi007.stdout:3/289: write dc/d22/d29/d32/d37/f40 [1310032,116269] 0 2024-09-21T04:35:21.129 INFO:tasks.workunit.client.0.smithi007.stdout:2/245: mkdir d0/d3c 0 2024-09-21T04:35:21.131 INFO:tasks.workunit.client.0.smithi007.stdout:3/290: chown dc/d22/d29/d2d/f43 8 1 2024-09-21T04:35:21.134 INFO:tasks.workunit.client.0.smithi007.stdout:9/331: creat d13/d18/f68 x:0 0 0 2024-09-21T04:35:21.134 INFO:tasks.workunit.client.0.smithi007.stdout:0/287: dwrite d6/d16/d18/d27/f33 [0,4194304] 0 2024-09-21T04:35:21.136 INFO:tasks.workunit.client.0.smithi007.stdout:0/288: fdatasync d6/d16/d57/f67 0 2024-09-21T04:35:21.141 INFO:tasks.workunit.client.0.smithi007.stdout:3/291: dwrite dc/d22/d29/d32/d37/f40 [0,4194304] 0 2024-09-21T04:35:21.142 INFO:tasks.workunit.client.1.smithi053.stdout:1/942: dread d7/d9/d43/d54/d7f/d36/f4e [0,4194304] 0 2024-09-21T04:35:21.142 INFO:tasks.workunit.client.0.smithi007.stdout:3/292: read - dc/d18/d41/f4a zero size 2024-09-21T04:35:21.144 INFO:tasks.workunit.client.0.smithi007.stdout:6/313: getdents d3/de/d4e 0 2024-09-21T04:35:21.147 INFO:tasks.workunit.client.1.smithi053.stdout:6/899: dwrite d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8/f10f [4194304,4194304] 0 2024-09-21T04:35:21.151 INFO:tasks.workunit.client.0.smithi007.stdout:6/314: dread - d3/de/f1a zero size 2024-09-21T04:35:21.152 INFO:tasks.workunit.client.1.smithi053.stdout:3/918: dwrite d16/d17/d21/d8b/d7d/d57/fac [8388608,4194304] 0 2024-09-21T04:35:21.153 INFO:tasks.workunit.client.0.smithi007.stdout:6/315: truncate d3/de/d12/f17 4304473 0 2024-09-21T04:35:21.155 INFO:tasks.workunit.client.0.smithi007.stdout:5/535: write d2/d1e/f58 [828371,78417] 0 2024-09-21T04:35:21.156 INFO:tasks.workunit.client.0.smithi007.stdout:3/293: creat dc/d22/d29/d32/d4d/f57 x:0 0 0 2024-09-21T04:35:21.156 INFO:tasks.workunit.client.1.smithi053.stdout:1/943: mkdir d7/d9/da/d40/d12d 0 2024-09-21T04:35:21.160 INFO:tasks.workunit.client.0.smithi007.stdout:3/294: write dc/d22/d29/f55 [517743,62967] 0 2024-09-21T04:35:21.160 INFO:tasks.workunit.client.1.smithi053.stdout:3/919: unlink d16/d17/d21/d42/d43/d125/d12e/f137 0 2024-09-21T04:35:21.161 INFO:tasks.workunit.client.1.smithi053.stdout:0/874: sync 2024-09-21T04:35:21.161 INFO:tasks.workunit.client.0.smithi007.stdout:6/316: write d3/d3e/f55 [2216647,65125] 0 2024-09-21T04:35:21.166 INFO:tasks.workunit.client.0.smithi007.stdout:9/332: dread d13/d18/f5c [0,4194304] 0 2024-09-21T04:35:21.172 INFO:tasks.workunit.client.1.smithi053.stdout:6/900: dread d2/d1b/f53 [0,4194304] 0 2024-09-21T04:35:21.174 INFO:tasks.workunit.client.0.smithi007.stdout:3/295: creat dc/d18/f58 x:0 0 0 2024-09-21T04:35:21.178 INFO:tasks.workunit.client.1.smithi053.stdout:5/955: write d0/d8/f31 [4576001,96939] 0 2024-09-21T04:35:21.179 INFO:tasks.workunit.client.0.smithi007.stdout:2/246: getdents d0/d11/d31/d39 0 2024-09-21T04:35:21.181 INFO:tasks.workunit.client.0.smithi007.stdout:5/536: dread d2/d27/d42/da3/d23/d5d/d61/f65 [0,4194304] 0 2024-09-21T04:35:21.184 INFO:tasks.workunit.client.0.smithi007.stdout:5/537: chown d2/d27/d3d/f69 77016471 1 2024-09-21T04:35:21.184 INFO:tasks.workunit.client.0.smithi007.stdout:3/296: rename dc/f15 to dc/d22/f59 0 2024-09-21T04:35:21.186 INFO:tasks.workunit.client.0.smithi007.stdout:6/317: getdents d3/de/d4e 0 2024-09-21T04:35:21.187 INFO:tasks.workunit.client.1.smithi053.stdout:3/920: mkdir d16/d17/d21/d8b/d14a 0 2024-09-21T04:35:21.188 INFO:tasks.workunit.client.0.smithi007.stdout:3/297: write dc/d18/f39 [1374196,42136] 0 2024-09-21T04:35:21.188 INFO:tasks.workunit.client.0.smithi007.stdout:7/302: dwrite d3/d6/d14/d20/d33/f49 [0,4194304] 0 2024-09-21T04:35:21.188 INFO:tasks.workunit.client.1.smithi053.stdout:8/991: truncate d3/d5/db1/f10c 80488 0 2024-09-21T04:35:21.189 INFO:tasks.workunit.client.0.smithi007.stdout:2/247: symlink d0/d32/l3d 0 2024-09-21T04:35:21.190 INFO:tasks.workunit.client.0.smithi007.stdout:2/248: readlink d0/d2/le 0 2024-09-21T04:35:21.195 INFO:tasks.workunit.client.0.smithi007.stdout:9/333: dread d13/d18/f35 [0,4194304] 0 2024-09-21T04:35:21.198 INFO:tasks.workunit.client.0.smithi007.stdout:5/538: rmdir d2/d27/d42/da3/d23/d5d 39 2024-09-21T04:35:21.199 INFO:tasks.workunit.client.1.smithi053.stdout:3/921: symlink d16/d17/d21/d8b/d7d/d9d/l14b 0 2024-09-21T04:35:21.202 INFO:tasks.workunit.client.1.smithi053.stdout:8/992: mknod d3/d5/db1/c148 0 2024-09-21T04:35:21.203 INFO:tasks.workunit.client.0.smithi007.stdout:6/318: dread d3/f2f [4194304,4194304] 0 2024-09-21T04:35:21.203 INFO:tasks.workunit.client.0.smithi007.stdout:7/303: symlink d3/d6/d10/d4f/l71 0 2024-09-21T04:35:21.206 INFO:tasks.workunit.client.1.smithi053.stdout:2/964: sync 2024-09-21T04:35:21.206 INFO:tasks.workunit.client.0.smithi007.stdout:2/249: creat d0/d11/d31/d39/f3e x:0 0 0 2024-09-21T04:35:21.207 INFO:tasks.workunit.client.1.smithi053.stdout:2/965: chown d1/d18/d1f/d22/d43 40120 1 2024-09-21T04:35:21.207 INFO:tasks.workunit.client.0.smithi007.stdout:0/289: dwrite d6/f5a [0,4194304] 0 2024-09-21T04:35:21.212 INFO:tasks.workunit.client.0.smithi007.stdout:5/539: creat d2/d27/d3d/d9b/fa7 x:0 0 0 2024-09-21T04:35:21.215 INFO:tasks.workunit.client.1.smithi053.stdout:0/875: link d5/de/d42/f89 d5/de/d9e/f126 0 2024-09-21T04:35:21.216 INFO:tasks.workunit.client.1.smithi053.stdout:0/876: dread - d5/d10/d4c/db1/dbc/dca/db5/f125 zero size 2024-09-21T04:35:21.218 INFO:tasks.workunit.client.1.smithi053.stdout:6/901: link d2/d1b/d1e/d27/d90/la2 d2/d1b/d3d/d49/de/d32/d80/l11a 0 2024-09-21T04:35:21.219 INFO:tasks.workunit.client.0.smithi007.stdout:2/250: chown d0/d2/l26 4079 1 2024-09-21T04:35:21.220 INFO:tasks.workunit.client.1.smithi053.stdout:3/922: creat d16/d17/d21/d8b/d7d/dc6/dec/f14c x:0 0 0 2024-09-21T04:35:21.221 INFO:tasks.workunit.client.0.smithi007.stdout:0/290: creat d6/d16/d18/d39/f6a x:0 0 0 2024-09-21T04:35:21.223 INFO:tasks.workunit.client.0.smithi007.stdout:8/187: sync 2024-09-21T04:35:21.223 INFO:tasks.workunit.client.0.smithi007.stdout:1/456: sync 2024-09-21T04:35:21.223 INFO:tasks.workunit.client.0.smithi007.stdout:0/291: write d6/d16/d18/d27/f29 [598418,19288] 0 2024-09-21T04:35:21.224 INFO:tasks.workunit.client.1.smithi053.stdout:1/944: dwrite d7/d47/ff2 [0,4194304] 0 2024-09-21T04:35:21.224 INFO:tasks.workunit.client.1.smithi053.stdout:2/966: link d1/d18/d1f/d27/d58/d5c/d91/dc1/f10e d1/d18/d1f/d22/d2d/dcd/dee/d121/f14f 0 2024-09-21T04:35:21.225 INFO:tasks.workunit.client.0.smithi007.stdout:2/251: dread d0/fd [0,4194304] 0 2024-09-21T04:35:21.226 INFO:tasks.workunit.client.0.smithi007.stdout:9/334: mknod d13/c69 0 2024-09-21T04:35:21.227 INFO:tasks.workunit.client.0.smithi007.stdout:2/252: readlink d0/d2/l4 0 2024-09-21T04:35:21.227 INFO:tasks.workunit.client.1.smithi053.stdout:2/967: dread d1/d18/d1f/d27/d41/d4b/fb4 [0,4194304] 0 2024-09-21T04:35:21.228 INFO:tasks.workunit.client.1.smithi053.stdout:2/968: stat d1/d18/d1f/d27/d58/c6e 0 2024-09-21T04:35:21.229 INFO:tasks.workunit.client.1.smithi053.stdout:6/902: creat d2/d1d/de0/df2/f11b x:0 0 0 2024-09-21T04:35:21.230 INFO:tasks.workunit.client.1.smithi053.stdout:6/903: fsync d2/d1b/d1e/dc1/fff 0 2024-09-21T04:35:21.233 INFO:tasks.workunit.client.0.smithi007.stdout:1/457: dwrite d5/d45/d5e/d55/d58/f67 [0,4194304] 0 2024-09-21T04:35:21.236 INFO:tasks.workunit.client.0.smithi007.stdout:7/304: rename d3/d6/d2e/l31 to d3/d6/d40/l72 0 2024-09-21T04:35:21.240 INFO:tasks.workunit.client.1.smithi053.stdout:3/923: rename d16/d17/d21/d42/da0/dfb/d11e/f105 to d16/d17/d21/d8b/d7d/d57/d90/f14d 0 2024-09-21T04:35:21.240 INFO:tasks.workunit.client.1.smithi053.stdout:3/924: stat c1 0 2024-09-21T04:35:21.240 INFO:tasks.workunit.client.0.smithi007.stdout:6/319: dread d3/f38 [0,4194304] 0 2024-09-21T04:35:21.243 INFO:tasks.workunit.client.0.smithi007.stdout:0/292: chown f1 1585 1 2024-09-21T04:35:21.246 INFO:tasks.workunit.client.1.smithi053.stdout:2/969: mknod d1/d18/d1f/d27/d58/db7/c150 0 2024-09-21T04:35:21.248 INFO:tasks.workunit.client.1.smithi053.stdout:3/925: dwrite d16/d17/d21/d42/da0/ffc [0,4194304] 0 2024-09-21T04:35:21.250 INFO:tasks.workunit.client.0.smithi007.stdout:9/335: rmdir d13/d18/d60 39 2024-09-21T04:35:21.253 INFO:tasks.workunit.client.0.smithi007.stdout:1/458: rmdir d5/d6/d21 39 2024-09-21T04:35:21.256 INFO:tasks.workunit.client.0.smithi007.stdout:7/305: rename d3/d6/d13/f3c to d3/d6/d2e/d63/f73 0 2024-09-21T04:35:21.257 INFO:tasks.workunit.client.1.smithi053.stdout:2/970: fdatasync d1/d18/d1f/d27/f7a 0 2024-09-21T04:35:21.258 INFO:tasks.workunit.client.1.smithi053.stdout:5/956: write d0/d8/f10 [1668358,34987] 0 2024-09-21T04:35:21.260 INFO:tasks.workunit.client.1.smithi053.stdout:5/957: truncate d0/dd/d5c/d92/f10d 4910843 0 2024-09-21T04:35:21.260 INFO:tasks.workunit.client.1.smithi053.stdout:3/926: mknod d16/d17/d21/d8b/d7d/d57/d90/dfe/c14e 0 2024-09-21T04:35:21.263 INFO:tasks.workunit.client.1.smithi053.stdout:2/971: fsync d1/d18/f5b 0 2024-09-21T04:35:21.265 INFO:tasks.workunit.client.1.smithi053.stdout:5/958: symlink d0/d8/d12e/dad/db9/l136 0 2024-09-21T04:35:21.267 INFO:tasks.workunit.client.1.smithi053.stdout:3/927: mknod d16/d17/d21/d26/d2b/c14f 0 2024-09-21T04:35:21.272 INFO:tasks.workunit.client.1.smithi053.stdout:5/959: symlink d0/d8/d12e/dad/db9/l137 0 2024-09-21T04:35:21.273 INFO:tasks.workunit.client.1.smithi053.stdout:1/945: write d7/d63/d96/dac/d106/fd1 [648079,111199] 0 2024-09-21T04:35:21.273 INFO:tasks.workunit.client.1.smithi053.stdout:3/928: mknod d16/d17/d21/d42/da0/dfb/c150 0 2024-09-21T04:35:21.274 INFO:tasks.workunit.client.1.smithi053.stdout:0/877: write d5/de/d9e/faf [759414,80016] 0 2024-09-21T04:35:21.276 INFO:tasks.workunit.client.1.smithi053.stdout:0/878: write d5/d10/d4c/db1/dbc/dca/d21/fe0 [252168,72594] 0 2024-09-21T04:35:21.276 INFO:tasks.workunit.client.1.smithi053.stdout:6/904: write d2/d1b/d3d/d49/de/d32/d80/d2e/d38/f9b [3873006,87721] 0 2024-09-21T04:35:21.277 INFO:tasks.workunit.client.0.smithi007.stdout:5/540: truncate d2/f55 4069566 0 2024-09-21T04:35:21.278 INFO:tasks.workunit.client.1.smithi053.stdout:6/905: chown d2/d1b/d3d/l66 585062 1 2024-09-21T04:35:21.278 INFO:tasks.workunit.client.0.smithi007.stdout:2/253: write d0/d11/d31/d39/f3a [2010905,125785] 0 2024-09-21T04:35:21.279 INFO:tasks.workunit.client.0.smithi007.stdout:2/254: chown d0/d32 112181560 1 2024-09-21T04:35:21.280 INFO:tasks.workunit.client.1.smithi053.stdout:1/946: mknod d7/d9/d43/d54/d7f/d36/d81/d95/c12e 0 2024-09-21T04:35:21.284 INFO:tasks.workunit.client.1.smithi053.stdout:2/972: creat d1/d18/d1f/f151 x:0 0 0 2024-09-21T04:35:21.285 INFO:tasks.workunit.client.1.smithi053.stdout:0/879: creat d5/d10/d4c/db1/dbc/dca/db5/d99/f127 x:0 0 0 2024-09-21T04:35:21.286 INFO:tasks.workunit.client.0.smithi007.stdout:9/336: fdatasync d13/d18/f3c 0 2024-09-21T04:35:21.286 INFO:tasks.workunit.client.1.smithi053.stdout:7/798: sync 2024-09-21T04:35:21.286 INFO:tasks.workunit.client.1.smithi053.stdout:9/908: sync 2024-09-21T04:35:21.287 INFO:tasks.workunit.client.0.smithi007.stdout:1/459: rmdir d5/d45/d5e/d42 39 2024-09-21T04:35:21.287 INFO:tasks.workunit.client.1.smithi053.stdout:9/909: chown d0/d1a/c8c 0 1 2024-09-21T04:35:21.288 INFO:tasks.workunit.client.1.smithi053.stdout:7/799: write d0/d6f/dbf/d50/d8b/f31 [238743,77251] 0 2024-09-21T04:35:21.288 INFO:tasks.workunit.client.1.smithi053.stdout:9/910: fdatasync d0/d1a/d89/fb7 0 2024-09-21T04:35:21.290 INFO:tasks.workunit.client.1.smithi053.stdout:9/911: write d0/d1c/dcb/fd9 [439403,24743] 0 2024-09-21T04:35:21.290 INFO:tasks.workunit.client.1.smithi053.stdout:1/947: mkdir d7/d63/d96/de2/d12f 0 2024-09-21T04:35:21.291 INFO:tasks.workunit.client.0.smithi007.stdout:7/306: write d3/f5 [9435163,122607] 0 2024-09-21T04:35:21.292 INFO:tasks.workunit.client.0.smithi007.stdout:6/320: rmdir d3/d59 39 2024-09-21T04:35:21.294 INFO:tasks.workunit.client.1.smithi053.stdout:9/912: dread d0/d1c/dcb/fd9 [0,4194304] 0 2024-09-21T04:35:21.294 INFO:tasks.workunit.client.0.smithi007.stdout:0/293: link d6/d16/d57/f59 d6/d16/d47/f6b 0 2024-09-21T04:35:21.296 INFO:tasks.workunit.client.1.smithi053.stdout:5/960: creat d0/dd/d48/f138 x:0 0 0 2024-09-21T04:35:21.296 INFO:tasks.workunit.client.0.smithi007.stdout:8/188: link d0/d3/d8/l14 d0/d3/d16/l30 0 2024-09-21T04:35:21.297 INFO:tasks.workunit.client.0.smithi007.stdout:8/189: fsync d0/d3/d8/de/f17 0 2024-09-21T04:35:21.303 INFO:tasks.workunit.client.1.smithi053.stdout:7/800: mknod d0/d6f/dbf/da2/c10e 0 2024-09-21T04:35:21.305 INFO:tasks.workunit.client.0.smithi007.stdout:5/541: dwrite d2/d27/d42/da3/d23/d5d/d61/f74 [0,4194304] 0 2024-09-21T04:35:21.308 INFO:tasks.workunit.client.0.smithi007.stdout:2/255: dwrite d0/fd [0,4194304] 0 2024-09-21T04:35:21.309 INFO:tasks.workunit.client.1.smithi053.stdout:2/973: creat d1/d18/d1f/d22/d12e/f152 x:0 0 0 2024-09-21T04:35:21.311 INFO:tasks.workunit.client.0.smithi007.stdout:6/321: write d3/f33 [688242,129907] 0 2024-09-21T04:35:21.313 INFO:tasks.workunit.client.0.smithi007.stdout:0/294: rename d6/d16/d18/d27/c41 to d6/d16/d18/d2e/d60/c6c 0 2024-09-21T04:35:21.316 INFO:tasks.workunit.client.0.smithi007.stdout:0/295: truncate d6/d16/d18/d39/f54 5033822 0 2024-09-21T04:35:21.316 INFO:tasks.workunit.client.1.smithi053.stdout:0/880: fsync d5/d10/d4c/db1/dbc/dca/d21/fc4 0 2024-09-21T04:35:21.317 INFO:tasks.workunit.client.0.smithi007.stdout:0/296: fsync d6/d16/d18/d39/f6a 0 2024-09-21T04:35:21.318 INFO:tasks.workunit.client.0.smithi007.stdout:0/297: chown d6/d16/d18/d27/c2c 17516057 1 2024-09-21T04:35:21.319 INFO:tasks.workunit.client.0.smithi007.stdout:0/298: write d6/f13 [1458931,8337] 0 2024-09-21T04:35:21.319 INFO:tasks.workunit.client.1.smithi053.stdout:1/948: dwrite d7/d9/da/d40/d49/fff [4194304,4194304] 0 2024-09-21T04:35:21.322 INFO:tasks.workunit.client.0.smithi007.stdout:1/460: symlink d5/d6/d21/d2b/d61/l92 0 2024-09-21T04:35:21.323 INFO:tasks.workunit.client.1.smithi053.stdout:7/801: rmdir d0/dca 39 2024-09-21T04:35:21.324 INFO:tasks.workunit.client.0.smithi007.stdout:7/307: write d3/d6/d14/d20/f37 [5031483,121410] 0 2024-09-21T04:35:21.327 INFO:tasks.workunit.client.1.smithi053.stdout:2/974: chown d1/d18/d1f/da2/dac/c139 45802222 1 2024-09-21T04:35:21.329 INFO:tasks.workunit.client.0.smithi007.stdout:2/256: creat d0/d32/f3f x:0 0 0 2024-09-21T04:35:21.330 INFO:tasks.workunit.client.0.smithi007.stdout:2/257: write d0/fd [1007247,14804] 0 2024-09-21T04:35:21.331 INFO:tasks.workunit.client.0.smithi007.stdout:2/258: readlink d0/d2/l4 0 2024-09-21T04:35:21.333 INFO:tasks.workunit.client.1.smithi053.stdout:0/881: creat d5/d10/d4c/db1/dbc/dca/d53/f128 x:0 0 0 2024-09-21T04:35:21.333 INFO:tasks.workunit.client.1.smithi053.stdout:5/961: write d0/d8/db/fb1 [817768,87468] 0 2024-09-21T04:35:21.334 INFO:tasks.workunit.client.0.smithi007.stdout:0/299: mkdir d6/d16/d52/d6d 0 2024-09-21T04:35:21.334 INFO:tasks.workunit.client.1.smithi053.stdout:5/962: read - d0/d8/d12e/dad/ff8 zero size 2024-09-21T04:35:21.335 INFO:tasks.workunit.client.0.smithi007.stdout:0/300: write d6/d16/d57/f67 [691867,90322] 0 2024-09-21T04:35:21.337 INFO:tasks.workunit.client.1.smithi053.stdout:6/906: getdents d2/d1d/de0/df2 0 2024-09-21T04:35:21.338 INFO:tasks.workunit.client.0.smithi007.stdout:5/542: symlink d2/d27/d3d/d9e/la8 0 2024-09-21T04:35:21.339 INFO:tasks.workunit.client.0.smithi007.stdout:7/308: mknod d3/d6/d2e/d63/d6a/c74 0 2024-09-21T04:35:21.340 INFO:tasks.workunit.client.0.smithi007.stdout:7/309: chown d3/d6/d10/d43/f64 1 1 2024-09-21T04:35:21.340 INFO:tasks.workunit.client.1.smithi053.stdout:5/963: dread d0/d8/d12e/dad/db9/fba [0,4194304] 0 2024-09-21T04:35:21.342 INFO:tasks.workunit.client.1.smithi053.stdout:5/964: stat d0/d8/d12e/dad/db9/fcf 0 2024-09-21T04:35:21.344 INFO:tasks.workunit.client.0.smithi007.stdout:0/301: read d6/d16/d18/f5d [4076264,102072] 0 2024-09-21T04:35:21.344 INFO:tasks.workunit.client.1.smithi053.stdout:6/907: dwrite d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8/f10f [0,4194304] 0 2024-09-21T04:35:21.347 INFO:tasks.workunit.client.1.smithi053.stdout:6/908: read d2/d1b/d1e/d27/d41/f97 [641172,15322] 0 2024-09-21T04:35:21.347 INFO:tasks.workunit.client.0.smithi007.stdout:7/310: dwrite d3/d6/d14/d20/f37 [0,4194304] 0 2024-09-21T04:35:21.348 INFO:tasks.workunit.client.0.smithi007.stdout:6/322: link d3/de/d12/d19/f35 d3/d59/f63 0 2024-09-21T04:35:21.349 INFO:tasks.workunit.client.0.smithi007.stdout:7/311: write d3/d6/d2e/f6d [319130,129587] 0 2024-09-21T04:35:21.350 INFO:tasks.workunit.client.0.smithi007.stdout:2/259: symlink d0/d11/l40 0 2024-09-21T04:35:21.353 INFO:tasks.workunit.client.1.smithi053.stdout:7/802: symlink d0/d2b/d36/d64/l10f 0 2024-09-21T04:35:21.354 INFO:tasks.workunit.client.0.smithi007.stdout:1/461: creat d5/d6/f93 x:0 0 0 2024-09-21T04:35:21.355 INFO:tasks.workunit.client.1.smithi053.stdout:5/965: dread d0/d8/d2a/fae [0,4194304] 0 2024-09-21T04:35:21.356 INFO:tasks.workunit.client.1.smithi053.stdout:8/993: sync 2024-09-21T04:35:21.356 INFO:tasks.workunit.client.1.smithi053.stdout:2/975: unlink d1/d18/d1f/d22/d2d/deb/l10b 0 2024-09-21T04:35:21.357 INFO:tasks.workunit.client.1.smithi053.stdout:8/994: chown d3/d4f/l72 3 1 2024-09-21T04:35:21.358 INFO:tasks.workunit.client.1.smithi053.stdout:9/913: getdents d0/d6c 0 2024-09-21T04:35:21.359 INFO:tasks.workunit.client.1.smithi053.stdout:8/995: fsync d3/d5/d4b/d5c/f10d 0 2024-09-21T04:35:21.359 INFO:tasks.workunit.client.0.smithi007.stdout:0/302: mkdir d6/d16/d57/d6e 0 2024-09-21T04:35:21.360 INFO:tasks.workunit.client.1.smithi053.stdout:1/949: truncate d7/d9/da/f86 1497341 0 2024-09-21T04:35:21.362 INFO:tasks.workunit.client.1.smithi053.stdout:8/996: truncate d3/d5/d21/f141 484262 0 2024-09-21T04:35:21.363 INFO:tasks.workunit.client.1.smithi053.stdout:8/997: chown d3/d5/l84 0 1 2024-09-21T04:35:21.366 INFO:tasks.workunit.client.0.smithi007.stdout:1/462: truncate d5/d6/f20 1123669 0 2024-09-21T04:35:21.367 INFO:tasks.workunit.client.0.smithi007.stdout:5/543: dwrite d2/d27/f87 [0,4194304] 0 2024-09-21T04:35:21.369 INFO:tasks.workunit.client.1.smithi053.stdout:0/882: creat d5/d10/d4c/db1/dbc/dca/f129 x:0 0 0 2024-09-21T04:35:21.370 INFO:tasks.workunit.client.1.smithi053.stdout:0/883: chown d5/de 0 1 2024-09-21T04:35:21.370 INFO:tasks.workunit.client.0.smithi007.stdout:6/323: dwrite d3/de/f49 [4194304,4194304] 0 2024-09-21T04:35:21.376 INFO:tasks.workunit.client.0.smithi007.stdout:0/303: creat d6/d16/d52/d6d/f6f x:0 0 0 2024-09-21T04:35:21.378 INFO:tasks.workunit.client.1.smithi053.stdout:7/803: mkdir d0/d6f/dbf/d50/d110 0 2024-09-21T04:35:21.379 INFO:tasks.workunit.client.1.smithi053.stdout:8/998: read d3/df/d10/fd3 [234055,120812] 0 2024-09-21T04:35:21.380 INFO:tasks.workunit.client.1.smithi053.stdout:1/950: dread d7/d47/d79/d99/fe4 [0,4194304] 0 2024-09-21T04:35:21.380 INFO:tasks.workunit.client.1.smithi053.stdout:2/976: dread d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/f107 [0,4194304] 0 2024-09-21T04:35:21.382 INFO:tasks.workunit.client.0.smithi007.stdout:2/260: rmdir d0/d3c 0 2024-09-21T04:35:21.383 INFO:tasks.workunit.client.1.smithi053.stdout:2/977: readlink d1/d18/d1f/d22/d54/lfe 0 2024-09-21T04:35:21.383 INFO:tasks.workunit.client.1.smithi053.stdout:5/966: dread d0/d66/f82 [0,4194304] 0 2024-09-21T04:35:21.384 INFO:tasks.workunit.client.0.smithi007.stdout:0/304: dwrite d6/d16/d18/d27/f3a [0,4194304] 0 2024-09-21T04:35:21.384 INFO:tasks.workunit.client.0.smithi007.stdout:7/312: link d3/d6/d10/d43/f64 d3/d6/d13/f75 0 2024-09-21T04:35:21.385 INFO:tasks.workunit.client.0.smithi007.stdout:2/261: truncate d0/f23 293214 0 2024-09-21T04:35:21.389 INFO:tasks.workunit.client.1.smithi053.stdout:9/914: link d0/d1c/d21/d46/fe5 d0/d2/d9/de7/d133/d136/f137 0 2024-09-21T04:35:21.390 INFO:tasks.workunit.client.1.smithi053.stdout:3/929: sync 2024-09-21T04:35:21.393 INFO:tasks.workunit.client.1.smithi053.stdout:6/909: creat d2/d1b/d3d/d49/f11c x:0 0 0 2024-09-21T04:35:21.398 INFO:tasks.workunit.client.1.smithi053.stdout:5/967: dwrite d0/dd/d5c/d92/ddb/d51/fe2 [0,4194304] 0 2024-09-21T04:35:21.398 INFO:tasks.workunit.client.1.smithi053.stdout:1/951: mkdir d7/d47/d130 0 2024-09-21T04:35:21.402 INFO:tasks.workunit.client.1.smithi053.stdout:8/999: symlink d3/d4f/d102/l149 0 2024-09-21T04:35:21.403 INFO:tasks.workunit.client.1.smithi053.stdout:5/968: fsync d0/d8/d2a/df5/fff 0 2024-09-21T04:35:21.404 INFO:tasks.workunit.client.1.smithi053.stdout:0/884: symlink d5/d10/d4c/ddf/d103/d11e/de5/l12a 0 2024-09-21T04:35:21.404 INFO:tasks.workunit.client.0.smithi007.stdout:0/305: truncate d6/d16/d57/f59 437781 0 2024-09-21T04:35:21.405 INFO:tasks.workunit.client.0.smithi007.stdout:5/544: dread d2/f5a [0,4194304] 0 2024-09-21T04:35:21.405 INFO:tasks.workunit.client.1.smithi053.stdout:5/969: stat d0/dd/d48/faa 0 2024-09-21T04:35:21.405 INFO:tasks.workunit.client.1.smithi053.stdout:7/804: dread d0/d6f/dbf/d95/f106 [0,4194304] 0 2024-09-21T04:35:21.407 INFO:tasks.workunit.client.0.smithi007.stdout:7/313: mknod d3/d6/d2e/d5e/c76 0 2024-09-21T04:35:21.409 INFO:tasks.workunit.client.0.smithi007.stdout:2/262: dread d0/d32/f34 [0,4194304] 0 2024-09-21T04:35:21.409 INFO:tasks.workunit.client.1.smithi053.stdout:2/978: unlink d1/d18/d1f/d27/d41/f114 0 2024-09-21T04:35:21.410 INFO:tasks.workunit.client.0.smithi007.stdout:2/263: write d0/d32/f3f [584543,17268] 0 2024-09-21T04:35:21.412 INFO:tasks.workunit.client.0.smithi007.stdout:1/463: rename d5/d45/d5e/d42 to d5/d45/d94 0 2024-09-21T04:35:21.413 INFO:tasks.workunit.client.0.smithi007.stdout:2/264: dread d0/f23 [0,4194304] 0 2024-09-21T04:35:21.414 INFO:tasks.workunit.client.1.smithi053.stdout:9/915: creat d0/d6c/f138 x:0 0 0 2024-09-21T04:35:21.415 INFO:tasks.workunit.client.1.smithi053.stdout:2/979: dwrite d1/d18/d1f/f151 [0,4194304] 0 2024-09-21T04:35:21.416 INFO:tasks.workunit.client.0.smithi007.stdout:5/545: write d2/d27/d42/da3/d23/d49/d44/d54/f88 [137422,66535] 0 2024-09-21T04:35:21.417 INFO:tasks.workunit.client.0.smithi007.stdout:7/314: mknod d3/d6/d10/d4f/c77 0 2024-09-21T04:35:21.426 INFO:tasks.workunit.client.0.smithi007.stdout:5/546: dwrite d2/d27/d42/da3/d28/f98 [0,4194304] 0 2024-09-21T04:35:21.426 INFO:tasks.workunit.client.1.smithi053.stdout:1/952: rename d7/da8/d74/d75/ccf to d7/d9/d43/c131 0 2024-09-21T04:35:21.428 INFO:tasks.workunit.client.0.smithi007.stdout:7/315: dwrite d3/d6/d14/f26 [0,4194304] 0 2024-09-21T04:35:21.429 INFO:tasks.workunit.client.0.smithi007.stdout:6/324: rename d3/d59/f61 to d3/de/d12/d19/f64 0 2024-09-21T04:35:21.431 INFO:tasks.workunit.client.1.smithi053.stdout:7/805: mkdir d0/d6f/dbf/d95/d111 0 2024-09-21T04:35:21.435 INFO:tasks.workunit.client.1.smithi053.stdout:5/970: creat d0/dd/d5c/d92/ddb/d51/d52/f139 x:0 0 0 2024-09-21T04:35:21.438 INFO:tasks.workunit.client.1.smithi053.stdout:2/980: dread d1/d18/d1f/d27/d58/d5c/fd4 [0,4194304] 0 2024-09-21T04:35:21.441 INFO:tasks.workunit.client.1.smithi053.stdout:2/981: read - d1/d18/d1f/d27/d41/d4b/d4d/d89/dbb/f122 zero size 2024-09-21T04:35:21.443 INFO:tasks.workunit.client.1.smithi053.stdout:9/916: creat d0/d76/d79/f139 x:0 0 0 2024-09-21T04:35:21.444 INFO:tasks.workunit.client.1.smithi053.stdout:3/930: write d16/d17/d21/d8b/d7d/d57/df5/ffd [257143,4438] 0 2024-09-21T04:35:21.445 INFO:tasks.workunit.client.1.smithi053.stdout:6/910: write d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f9d [550382,79335] 0 2024-09-21T04:35:21.445 INFO:tasks.workunit.client.1.smithi053.stdout:3/931: write d16/d17/d21/d26/d3e/fdb [2138236,15773] 0 2024-09-21T04:35:21.448 INFO:tasks.workunit.client.0.smithi007.stdout:4/418: sync 2024-09-21T04:35:21.449 INFO:tasks.workunit.client.0.smithi007.stdout:4/419: read d2/d17/d2e/d45/f4b [2290880,16948] 0 2024-09-21T04:35:21.450 INFO:tasks.workunit.client.0.smithi007.stdout:7/316: write d3/d6/d14/f18 [1636673,29184] 0 2024-09-21T04:35:21.450 INFO:tasks.workunit.client.1.smithi053.stdout:6/911: dread d2/d1b/f7d [0,4194304] 0 2024-09-21T04:35:21.451 INFO:tasks.workunit.client.1.smithi053.stdout:9/917: dwrite d0/d1c/d52/d53/f80 [0,4194304] 0 2024-09-21T04:35:21.454 INFO:tasks.workunit.client.0.smithi007.stdout:5/547: symlink d2/d27/d42/da3/la9 0 2024-09-21T04:35:21.455 INFO:tasks.workunit.client.1.smithi053.stdout:7/806: readlink d0/l38 0 2024-09-21T04:35:21.456 INFO:tasks.workunit.client.0.smithi007.stdout:6/325: symlink d3/de/l65 0 2024-09-21T04:35:21.458 INFO:tasks.workunit.client.1.smithi053.stdout:2/982: rmdir d1/d18/d1f/d27/d58/d5c/d91 39 2024-09-21T04:35:21.459 INFO:tasks.workunit.client.0.smithi007.stdout:7/317: symlink d3/d6/d14/d20/l78 0 2024-09-21T04:35:21.461 INFO:tasks.workunit.client.0.smithi007.stdout:1/464: link d5/d6/f8 d5/d2f/f95 0 2024-09-21T04:35:21.464 INFO:tasks.workunit.client.0.smithi007.stdout:5/548: creat d2/d27/d42/da3/d23/d5d/d9a/faa x:0 0 0 2024-09-21T04:35:21.465 INFO:tasks.workunit.client.1.smithi053.stdout:3/932: read d16/d17/d21/d26/d3e/fd5 [593141,42221] 0 2024-09-21T04:35:21.466 INFO:tasks.workunit.client.0.smithi007.stdout:6/326: chown d3/d3b/f45 68429019 1 2024-09-21T04:35:21.467 INFO:tasks.workunit.client.1.smithi053.stdout:6/912: mkdir d2/d1b/d11d 0 2024-09-21T04:35:21.467 INFO:tasks.workunit.client.1.smithi053.stdout:3/933: chown d16/d17/d21/d8b/ce9 13 1 2024-09-21T04:35:21.468 INFO:tasks.workunit.client.1.smithi053.stdout:0/885: dwrite d5/de/d9e/f126 [0,4194304] 0 2024-09-21T04:35:21.468 INFO:tasks.workunit.client.1.smithi053.stdout:9/918: symlink d0/d1c/d52/db5/l13a 0 2024-09-21T04:35:21.469 INFO:tasks.workunit.client.0.smithi007.stdout:4/420: mkdir d2/d3c/d63/d8c 0 2024-09-21T04:35:21.470 INFO:tasks.workunit.client.0.smithi007.stdout:4/421: write d2/d7/f6c [4451894,123025] 0 2024-09-21T04:35:21.470 INFO:tasks.workunit.client.1.smithi053.stdout:9/919: truncate d0/df/fee 550434 0 2024-09-21T04:35:21.471 INFO:tasks.workunit.client.0.smithi007.stdout:4/422: stat d2/d17/d2e/d45/f60 0 2024-09-21T04:35:21.472 INFO:tasks.workunit.client.1.smithi053.stdout:5/971: mkdir d0/dd/d5c/d13a 0 2024-09-21T04:35:21.473 INFO:tasks.workunit.client.1.smithi053.stdout:2/983: mknod d1/d18/d1f/d22/d2d/dcd/dee/d121/c153 0 2024-09-21T04:35:21.474 INFO:tasks.workunit.client.0.smithi007.stdout:6/327: rename d3/de/f42 to d3/de/d4e/f66 0 2024-09-21T04:35:21.475 INFO:tasks.workunit.client.0.smithi007.stdout:7/318: mknod d3/d6/c79 0 2024-09-21T04:35:21.475 INFO:tasks.workunit.client.0.smithi007.stdout:4/423: dread d2/d17/f52 [0,4194304] 0 2024-09-21T04:35:21.477 INFO:tasks.workunit.client.0.smithi007.stdout:4/424: read d2/d17/d2e/d45/f60 [360636,94519] 0 2024-09-21T04:35:21.478 INFO:tasks.workunit.client.1.smithi053.stdout:3/934: dwrite d16/d17/d21/d42/d43/d125/d127/f123 [4194304,4194304] 0 2024-09-21T04:35:21.482 INFO:tasks.workunit.client.1.smithi053.stdout:7/807: write d0/d24/f4d [1974018,54174] 0 2024-09-21T04:35:21.484 INFO:tasks.workunit.client.0.smithi007.stdout:5/549: mkdir d2/d27/d42/da3/d23/d5d/d9a/da6/dab 0 2024-09-21T04:35:21.485 INFO:tasks.workunit.client.0.smithi007.stdout:5/550: chown d2/d1e/f37 1171868773 1 2024-09-21T04:35:21.486 INFO:tasks.workunit.client.1.smithi053.stdout:0/886: symlink d5/d10/d4c/ddf/d103/d11e/de5/l12b 0 2024-09-21T04:35:21.488 INFO:tasks.workunit.client.0.smithi007.stdout:6/328: mkdir d3/de/d4e/d67 0 2024-09-21T04:35:21.489 INFO:tasks.workunit.client.0.smithi007.stdout:6/329: write d3/de/d4e/f57 [408509,109319] 0 2024-09-21T04:35:21.490 INFO:tasks.workunit.client.1.smithi053.stdout:9/920: truncate d0/d1a/f2c 12655634 0 2024-09-21T04:35:21.491 INFO:tasks.workunit.client.1.smithi053.stdout:6/913: write d2/d1b/f5a [1107191,104728] 0 2024-09-21T04:35:21.492 INFO:tasks.workunit.client.1.smithi053.stdout:1/953: getdents d7/d63/d96/dac 0 2024-09-21T04:35:21.493 INFO:tasks.workunit.client.1.smithi053.stdout:1/954: readlink d7/da8/d74/l118 0 2024-09-21T04:35:21.493 INFO:tasks.workunit.client.0.smithi007.stdout:4/425: mknod d2/d17/d29/c8d 0 2024-09-21T04:35:21.494 INFO:tasks.workunit.client.1.smithi053.stdout:1/955: stat d7/d9/da/d40/d49/f5e 0 2024-09-21T04:35:21.497 INFO:tasks.workunit.client.1.smithi053.stdout:3/935: unlink d16/d17/d21/d26/d3e/fdb 0 2024-09-21T04:35:21.498 INFO:tasks.workunit.client.0.smithi007.stdout:5/551: fdatasync d2/d1e/f37 0 2024-09-21T04:35:21.501 INFO:tasks.workunit.client.1.smithi053.stdout:7/808: stat d0/d6f/dbf/d5d/l7c 0 2024-09-21T04:35:21.510 INFO:tasks.workunit.client.1.smithi053.stdout:2/984: dwrite d1/d18/d1f/d9d/fa8 [0,4194304] 0 2024-09-21T04:35:21.510 INFO:tasks.workunit.client.1.smithi053.stdout:5/972: link d0/d8/d2a/df5/fff d0/d8/db/db7/f13b 0 2024-09-21T04:35:21.510 INFO:tasks.workunit.client.0.smithi007.stdout:1/465: getdents d5/d2f 0 2024-09-21T04:35:21.513 INFO:tasks.workunit.client.1.smithi053.stdout:1/956: dwrite d7/d47/d79/d99/fe4 [0,4194304] 0 2024-09-21T04:35:21.515 INFO:tasks.workunit.client.1.smithi053.stdout:3/936: readlink d16/d1b/dc3/l5d 0 2024-09-21T04:35:21.516 INFO:tasks.workunit.client.0.smithi007.stdout:6/330: dwrite d3/d59/f63 [0,4194304] 0 2024-09-21T04:35:21.516 INFO:tasks.workunit.client.0.smithi007.stdout:7/319: link d3/l6c d3/d6/d10/d62/d66/l7a 0 2024-09-21T04:35:21.518 INFO:tasks.workunit.client.1.smithi053.stdout:0/887: mknod d5/de/dfc/c12c 0 2024-09-21T04:35:21.519 INFO:tasks.workunit.client.0.smithi007.stdout:4/426: mkdir d2/d17/d2e/d45/d8e 0 2024-09-21T04:35:21.520 INFO:tasks.workunit.client.1.smithi053.stdout:0/888: write d5/d10/d4c/db1/dbc/d8c/fee [501234,38087] 0 2024-09-21T04:35:21.524 INFO:tasks.workunit.client.1.smithi053.stdout:2/985: rmdir d1/d18/d1f/d27/d71 39 2024-09-21T04:35:21.528 INFO:tasks.workunit.client.1.smithi053.stdout:5/973: rename d0/dd/cb3 to d0/dd/d5c/d92/dfa/c13c 0 2024-09-21T04:35:21.532 INFO:tasks.workunit.client.1.smithi053.stdout:3/937: rmdir d16/d17/d21/d42/d43/d125 39 2024-09-21T04:35:21.532 INFO:tasks.workunit.client.1.smithi053.stdout:3/938: chown d16/d1b/dc3/f121 1136777 1 2024-09-21T04:35:21.534 INFO:tasks.workunit.client.1.smithi053.stdout:7/809: creat d0/dca/f112 x:0 0 0 2024-09-21T04:35:21.535 INFO:tasks.workunit.client.1.smithi053.stdout:7/810: chown d0/d24 449 1 2024-09-21T04:35:21.537 INFO:tasks.workunit.client.1.smithi053.stdout:0/889: creat d5/de/dfc/f12d x:0 0 0 2024-09-21T04:35:21.537 INFO:tasks.workunit.client.0.smithi007.stdout:6/331: dwrite d3/f28 [0,4194304] 0 2024-09-21T04:35:21.539 INFO:tasks.workunit.client.0.smithi007.stdout:5/552: truncate d2/d27/d42/f62 778629 0 2024-09-21T04:35:21.541 INFO:tasks.workunit.client.1.smithi053.stdout:9/921: dwrite d0/df/f48 [0,4194304] 0 2024-09-21T04:35:21.543 INFO:tasks.workunit.client.0.smithi007.stdout:1/466: link d5/d45/d5e/l7f d5/d6/d21/l96 0 2024-09-21T04:35:21.546 INFO:tasks.workunit.client.0.smithi007.stdout:7/320: write d3/d6/d10/d43/f64 [714854,1736] 0 2024-09-21T04:35:21.547 INFO:tasks.workunit.client.0.smithi007.stdout:7/321: readlink d3/d6/d14/l55 0 2024-09-21T04:35:21.548 INFO:tasks.workunit.client.1.smithi053.stdout:1/957: creat d7/d47/d79/d10e/f132 x:0 0 0 2024-09-21T04:35:21.548 INFO:tasks.workunit.client.0.smithi007.stdout:7/322: chown d3/d6/fa 806 1 2024-09-21T04:35:21.548 INFO:tasks.workunit.client.0.smithi007.stdout:5/553: mknod d2/d27/d3d/d9b/cac 0 2024-09-21T04:35:21.551 INFO:tasks.workunit.client.0.smithi007.stdout:7/323: truncate d3/d6/d10/d4f/f65 652620 0 2024-09-21T04:35:21.551 INFO:tasks.workunit.client.1.smithi053.stdout:1/958: truncate d7/d9/d43/d54/d7f/d36/f97 2606518 0 2024-09-21T04:35:21.551 INFO:tasks.workunit.client.0.smithi007.stdout:4/427: rename d2/d17/d18/f47 to d2/d17/d2e/d45/f8f 0 2024-09-21T04:35:21.552 INFO:tasks.workunit.client.1.smithi053.stdout:1/959: write d7/d9/d43/d54/d7f/f116 [491213,59667] 0 2024-09-21T04:35:21.552 INFO:tasks.workunit.client.0.smithi007.stdout:4/428: write d2/f6a [999665,20920] 0 2024-09-21T04:35:21.553 INFO:tasks.workunit.client.0.smithi007.stdout:5/554: symlink d2/d27/d42/lad 0 2024-09-21T04:35:21.554 INFO:tasks.workunit.client.0.smithi007.stdout:4/429: chown d2/d17/d2e/f33 1812555927 1 2024-09-21T04:35:21.556 INFO:tasks.workunit.client.1.smithi053.stdout:0/890: rename d5/d10/d4c/ddf to d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/d12e 0 2024-09-21T04:35:21.558 INFO:tasks.workunit.client.0.smithi007.stdout:1/467: dread d5/f39 [0,4194304] 0 2024-09-21T04:35:21.558 INFO:tasks.workunit.client.0.smithi007.stdout:5/555: mkdir d2/d27/dae 0 2024-09-21T04:35:21.559 INFO:tasks.workunit.client.0.smithi007.stdout:1/468: fsync d5/f48 0 2024-09-21T04:35:21.560 INFO:tasks.workunit.client.0.smithi007.stdout:4/430: mkdir d2/d17/d2e/d45/d5e/d90 0 2024-09-21T04:35:21.562 INFO:tasks.workunit.client.1.smithi053.stdout:9/922: dread d0/df/faf [0,4194304] 0 2024-09-21T04:35:21.567 INFO:tasks.workunit.client.0.smithi007.stdout:7/324: creat d3/f7b x:0 0 0 2024-09-21T04:35:21.568 INFO:tasks.workunit.client.1.smithi053.stdout:1/960: rmdir d7/d63/d96/dc7 39 2024-09-21T04:35:21.569 INFO:tasks.workunit.client.0.smithi007.stdout:4/431: dwrite d2/d17/d29/f86 [0,4194304] 0 2024-09-21T04:35:21.569 INFO:tasks.workunit.client.1.smithi053.stdout:5/974: dwrite d0/f6a [0,4194304] 0 2024-09-21T04:35:21.570 INFO:tasks.workunit.client.1.smithi053.stdout:5/975: readlink d0/d8/d12e/dad/db9/l136 0 2024-09-21T04:35:21.572 INFO:tasks.workunit.client.0.smithi007.stdout:1/469: creat d5/d6/d21/d2b/d4b/d8a/f97 x:0 0 0 2024-09-21T04:35:21.574 INFO:tasks.workunit.client.0.smithi007.stdout:5/556: rename d2/d27/d42/da3/d23/d5d/d9a/da6/dab to d2/d27/d42/d53/daf 0 2024-09-21T04:35:21.580 INFO:tasks.workunit.client.1.smithi053.stdout:2/986: creat d1/d18/d1f/d27/d41/f154 x:0 0 0 2024-09-21T04:35:21.581 INFO:tasks.workunit.client.1.smithi053.stdout:9/923: dread d0/d1a/d4c/f94 [0,4194304] 0 2024-09-21T04:35:21.581 INFO:tasks.workunit.client.0.smithi007.stdout:7/325: dread - d3/f4c zero size 2024-09-21T04:35:21.582 INFO:tasks.workunit.client.0.smithi007.stdout:7/326: stat d3/d6/d2e/f6d 0 2024-09-21T04:35:21.582 INFO:tasks.workunit.client.1.smithi053.stdout:2/987: stat d1/d18/d1f/d22/d2d/deb/d13f/l99 0 2024-09-21T04:35:21.584 INFO:tasks.workunit.client.1.smithi053.stdout:3/939: truncate f9 245291 0 2024-09-21T04:35:21.592 INFO:tasks.workunit.client.1.smithi053.stdout:0/891: symlink d5/dd/d50/l12f 0 2024-09-21T04:35:21.595 INFO:tasks.workunit.client.0.smithi007.stdout:7/327: symlink d3/d6/d10/d51/l7c 0 2024-09-21T04:35:21.596 INFO:tasks.workunit.client.1.smithi053.stdout:9/924: dread d0/d2/d9/d1f/f4f [0,4194304] 0 2024-09-21T04:35:21.598 INFO:tasks.workunit.client.1.smithi053.stdout:3/940: creat d16/d17/d21/d26/d49/f151 x:0 0 0 2024-09-21T04:35:21.599 INFO:tasks.workunit.client.0.smithi007.stdout:4/432: rename d2/d17/d2e/d45/c7b to d2/d17/d2e/d45/c91 0 2024-09-21T04:35:21.600 INFO:tasks.workunit.client.1.smithi053.stdout:6/914: sync 2024-09-21T04:35:21.600 INFO:tasks.workunit.client.0.smithi007.stdout:5/557: dwrite d2/d27/d42/da3/d1d/f1f [0,4194304] 0 2024-09-21T04:35:21.603 INFO:tasks.workunit.client.1.smithi053.stdout:5/976: dwrite d0/dd/fcd [0,4194304] 0 2024-09-21T04:35:21.604 INFO:tasks.workunit.client.1.smithi053.stdout:7/811: link d0/f4 d0/d6f/dbf/f113 0 2024-09-21T04:35:21.605 INFO:tasks.workunit.client.0.smithi007.stdout:1/470: getdents d5 0 2024-09-21T04:35:21.606 INFO:tasks.workunit.client.1.smithi053.stdout:9/925: mkdir d0/d1a/d4c/d13b 0 2024-09-21T04:35:21.606 INFO:tasks.workunit.client.1.smithi053.stdout:0/892: read d5/de/d42/d78/fb8 [2328231,70199] 0 2024-09-21T04:35:21.607 INFO:tasks.workunit.client.1.smithi053.stdout:3/941: mkdir d16/d17/d21/d42/d43/d86/d152 0 2024-09-21T04:35:21.608 INFO:tasks.workunit.client.1.smithi053.stdout:9/926: chown d0/d2/d9/d34 125247483 1 2024-09-21T04:35:21.610 INFO:tasks.workunit.client.1.smithi053.stdout:5/977: mkdir d0/d8/db/d13d 0 2024-09-21T04:35:21.610 INFO:tasks.workunit.client.1.smithi053.stdout:9/927: chown d0/d1c/d25 262698 1 2024-09-21T04:35:21.611 INFO:tasks.workunit.client.1.smithi053.stdout:7/812: mknod d0/d6f/dbf/d50/d8b/d70/dc1/c114 0 2024-09-21T04:35:21.611 INFO:tasks.workunit.client.1.smithi053.stdout:6/915: dwrite d2/d1d/f50 [0,4194304] 0 2024-09-21T04:35:21.614 INFO:tasks.workunit.client.1.smithi053.stdout:2/988: dread d1/d18/d1f/d27/f6a [0,4194304] 0 2024-09-21T04:35:21.615 INFO:tasks.workunit.client.1.smithi053.stdout:3/942: truncate d16/d1b/dc3/f83 5107539 0 2024-09-21T04:35:21.617 INFO:tasks.workunit.client.1.smithi053.stdout:5/978: creat d0/d8/db6/dbc/df4/f13e x:0 0 0 2024-09-21T04:35:21.617 INFO:tasks.workunit.client.1.smithi053.stdout:9/928: symlink d0/d1c/d52/d53/l13c 0 2024-09-21T04:35:21.618 INFO:tasks.workunit.client.1.smithi053.stdout:7/813: mkdir d0/d2b/d36/d115 0 2024-09-21T04:35:21.619 INFO:tasks.workunit.client.1.smithi053.stdout:3/943: chown d16/d17/d21/d8b/d7d/d57/f122 244000 1 2024-09-21T04:35:21.620 INFO:tasks.workunit.client.1.smithi053.stdout:0/893: creat d5/d10/d4c/db1/dbc/dca/f130 x:0 0 0 2024-09-21T04:35:21.620 INFO:tasks.workunit.client.0.smithi007.stdout:5/558: mkdir d2/d27/d42/d53/daf/db0 0 2024-09-21T04:35:21.623 INFO:tasks.workunit.client.1.smithi053.stdout:2/989: mknod d1/d18/d1f/d27/d58/db7/d112/c155 0 2024-09-21T04:35:21.624 INFO:tasks.workunit.client.0.smithi007.stdout:1/471: dread d5/d45/d5e/d55/f65 [0,4194304] 0 2024-09-21T04:35:21.624 INFO:tasks.workunit.client.1.smithi053.stdout:7/814: write d0/ff0 [204111,67949] 0 2024-09-21T04:35:21.626 INFO:tasks.workunit.client.1.smithi053.stdout:0/894: chown d5/dd/c51 4295 1 2024-09-21T04:35:21.627 INFO:tasks.workunit.client.0.smithi007.stdout:1/472: fsync d5/d2f/f3a 0 2024-09-21T04:35:21.627 INFO:tasks.workunit.client.1.smithi053.stdout:6/916: dwrite d2/d1b/d3d/d49/d5c/d8d/fbc [4194304,4194304] 0 2024-09-21T04:35:21.630 INFO:tasks.workunit.client.0.smithi007.stdout:4/433: dread d2/d17/d2e/d45/f8f [0,4194304] 0 2024-09-21T04:35:21.630 INFO:tasks.workunit.client.0.smithi007.stdout:1/473: symlink d5/d6/l98 0 2024-09-21T04:35:21.631 INFO:tasks.workunit.client.0.smithi007.stdout:1/474: fdatasync d5/fe 0 2024-09-21T04:35:21.631 INFO:tasks.workunit.client.1.smithi053.stdout:7/815: chown d0/d2b/d36/d64/ddb/fb2 97126 1 2024-09-21T04:35:21.633 INFO:tasks.workunit.client.1.smithi053.stdout:2/990: creat d1/d18/d1f/d27/d41/d4b/f156 x:0 0 0 2024-09-21T04:35:21.634 INFO:tasks.workunit.client.1.smithi053.stdout:1/961: truncate d7/d63/d96/dac/d106/f61 697162 0 2024-09-21T04:35:21.634 INFO:tasks.workunit.client.0.smithi007.stdout:7/328: write d3/d6/d10/f34 [174218,80396] 0 2024-09-21T04:35:21.635 INFO:tasks.workunit.client.0.smithi007.stdout:5/559: dread d2/d27/d42/da3/d23/d2a/f2f [0,4194304] 0 2024-09-21T04:35:21.636 INFO:tasks.workunit.client.1.smithi053.stdout:0/895: creat d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/f131 x:0 0 0 2024-09-21T04:35:21.638 INFO:tasks.workunit.client.1.smithi053.stdout:3/944: dwrite d16/d17/d21/d26/d2b/dbb/f130 [0,4194304] 0 2024-09-21T04:35:21.639 INFO:tasks.workunit.client.1.smithi053.stdout:0/896: write d5/f39 [5025722,88369] 0 2024-09-21T04:35:21.640 INFO:tasks.workunit.client.1.smithi053.stdout:7/816: truncate d0/d6f/dbf/d50/d8b/d70/fd6 1009126 0 2024-09-21T04:35:21.641 INFO:tasks.workunit.client.1.smithi053.stdout:6/917: rename d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/d115 to d2/d1b/d11d/d11e 0 2024-09-21T04:35:21.642 INFO:tasks.workunit.client.0.smithi007.stdout:4/434: fdatasync d2/d7/f8 0 2024-09-21T04:35:21.643 INFO:tasks.workunit.client.0.smithi007.stdout:7/329: dwrite d3/d6/d10/d43/f64 [0,4194304] 0 2024-09-21T04:35:21.643 INFO:tasks.workunit.client.1.smithi053.stdout:6/918: write d2/fa [1860234,126596] 0 2024-09-21T04:35:21.648 INFO:tasks.workunit.client.0.smithi007.stdout:5/560: rmdir d2/d27/d3d 39 2024-09-21T04:35:21.652 INFO:tasks.workunit.client.0.smithi007.stdout:1/475: truncate d5/d6/d21/d2b/f51 3146408 0 2024-09-21T04:35:21.654 INFO:tasks.workunit.client.0.smithi007.stdout:1/476: write d5/d6/d21/d2b/d4b/f70 [716113,45280] 0 2024-09-21T04:35:21.656 INFO:tasks.workunit.client.0.smithi007.stdout:5/561: dwrite d2/d27/d42/da3/d28/f98 [0,4194304] 0 2024-09-21T04:35:21.659 INFO:tasks.workunit.client.0.smithi007.stdout:7/330: read - d3/d6/d40/f48 zero size 2024-09-21T04:35:21.662 INFO:tasks.workunit.client.1.smithi053.stdout:9/929: dwrite d0/d1c/fb0 [0,4194304] 0 2024-09-21T04:35:21.662 INFO:tasks.workunit.client.0.smithi007.stdout:1/477: dread d5/d6/d21/d2b/d4b/f74 [0,4194304] 0 2024-09-21T04:35:21.664 INFO:tasks.workunit.client.0.smithi007.stdout:1/478: write d5/f1f [2422891,108374] 0 2024-09-21T04:35:21.665 INFO:tasks.workunit.client.1.smithi053.stdout:0/897: truncate d5/de/d42/d78/fb8 3024972 0 2024-09-21T04:35:21.669 INFO:tasks.workunit.client.1.smithi053.stdout:6/919: mkdir d2/d1b/d1e/d27/d41/d11f 0 2024-09-21T04:35:21.674 INFO:tasks.workunit.client.0.smithi007.stdout:1/479: dwrite d5/d45/d94/f62 [0,4194304] 0 2024-09-21T04:35:21.674 INFO:tasks.workunit.client.0.smithi007.stdout:4/435: dread d2/f27 [0,4194304] 0 2024-09-21T04:35:21.675 INFO:tasks.workunit.client.1.smithi053.stdout:1/962: rename d7/d9/da/d40/d49/d77/c11b to d7/d63/d96/dc7/d107/c133 0 2024-09-21T04:35:21.682 INFO:tasks.workunit.client.1.smithi053.stdout:3/945: write d16/d17/d21/d8b/d7d/d57/dc1/f70 [2774684,41856] 0 2024-09-21T04:35:21.684 INFO:tasks.workunit.client.0.smithi007.stdout:7/331: dread d3/d6/f2d [0,4194304] 0 2024-09-21T04:35:21.685 INFO:tasks.workunit.client.1.smithi053.stdout:2/991: link d1/f5 d1/d18/d1f/d22/f157 0 2024-09-21T04:35:21.686 INFO:tasks.workunit.client.1.smithi053.stdout:7/817: dwrite d0/d24/fcc [0,4194304] 0 2024-09-21T04:35:21.686 INFO:tasks.workunit.client.0.smithi007.stdout:5/562: mknod d2/d27/d42/d53/daf/db0/cb1 0 2024-09-21T04:35:21.689 INFO:tasks.workunit.client.0.smithi007.stdout:4/436: creat d2/d7/d55/f92 x:0 0 0 2024-09-21T04:35:21.690 INFO:tasks.workunit.client.1.smithi053.stdout:0/898: rmdir d5/d10/d4c/db1/dbc/dca 39 2024-09-21T04:35:21.694 INFO:tasks.workunit.client.0.smithi007.stdout:1/480: unlink d5/d45/d94/c50 0 2024-09-21T04:35:21.694 INFO:tasks.workunit.client.1.smithi053.stdout:9/930: dwrite d0/d2/d4e/d5c/f78 [0,4194304] 0 2024-09-21T04:35:21.698 INFO:tasks.workunit.client.1.smithi053.stdout:1/963: creat d7/d9/da/d40/d49/d77/f134 x:0 0 0 2024-09-21T04:35:21.698 INFO:tasks.workunit.client.0.smithi007.stdout:4/437: dwrite d2/d17/d29/f58 [0,4194304] 0 2024-09-21T04:35:21.700 INFO:tasks.workunit.client.0.smithi007.stdout:7/332: readlink d3/d6/d2e/l3e 0 2024-09-21T04:35:21.706 INFO:tasks.workunit.client.0.smithi007.stdout:1/481: mkdir d5/d6/d21/d64/d71/d99 0 2024-09-21T04:35:21.707 INFO:tasks.workunit.client.0.smithi007.stdout:1/482: fsync d5/f5d 0 2024-09-21T04:35:21.708 INFO:tasks.workunit.client.0.smithi007.stdout:1/483: fsync d5/f90 0 2024-09-21T04:35:21.709 INFO:tasks.workunit.client.0.smithi007.stdout:4/438: rename d2/f1d to d2/d5f/f93 0 2024-09-21T04:35:21.710 INFO:tasks.workunit.client.0.smithi007.stdout:7/333: truncate d3/d6/fa 389902 0 2024-09-21T04:35:21.710 INFO:tasks.workunit.client.0.smithi007.stdout:4/439: dread - d2/d7/d55/d61/f73 zero size 2024-09-21T04:35:21.711 INFO:tasks.workunit.client.0.smithi007.stdout:4/440: stat d2/d17/d2e/f3e 0 2024-09-21T04:35:21.711 INFO:tasks.workunit.client.1.smithi053.stdout:0/899: fsync d5/de/f5b 0 2024-09-21T04:35:21.714 INFO:tasks.workunit.client.0.smithi007.stdout:4/441: dread d2/f12 [4194304,4194304] 0 2024-09-21T04:35:21.715 INFO:tasks.workunit.client.0.smithi007.stdout:1/484: mknod d5/d2f/c9a 0 2024-09-21T04:35:21.716 INFO:tasks.workunit.client.0.smithi007.stdout:1/485: write d5/f26 [4021267,18638] 0 2024-09-21T04:35:21.718 INFO:tasks.workunit.client.1.smithi053.stdout:3/946: truncate d16/f1d 803657 0 2024-09-21T04:35:21.720 INFO:tasks.workunit.client.1.smithi053.stdout:7/818: mknod d0/c116 0 2024-09-21T04:35:21.721 INFO:tasks.workunit.client.1.smithi053.stdout:7/819: chown d0/d6f/dbf/d50 62085 1 2024-09-21T04:35:21.723 INFO:tasks.workunit.client.0.smithi007.stdout:1/486: rmdir d5/d6/d21/d64 39 2024-09-21T04:35:21.726 INFO:tasks.workunit.client.0.smithi007.stdout:5/563: link d2/d27/d42/da3/d23/d49/l5f d2/lb2 0 2024-09-21T04:35:21.728 INFO:tasks.workunit.client.1.smithi053.stdout:9/931: truncate d0/d1a/d89/fbb 1810938 0 2024-09-21T04:35:21.729 INFO:tasks.workunit.client.0.smithi007.stdout:4/442: symlink d2/d7/l94 0 2024-09-21T04:35:21.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/487: dwrite d5/d45/d5e/f76 [0,4194304] 0 2024-09-21T04:35:21.730 INFO:tasks.workunit.client.0.smithi007.stdout:4/443: chown d2/d7/d55/d61 10159 1 2024-09-21T04:35:21.731 INFO:tasks.workunit.client.0.smithi007.stdout:1/488: chown d5/d45/d5e/d55/d58/f73 29 1 2024-09-21T04:35:21.732 INFO:tasks.workunit.client.1.smithi053.stdout:6/920: rename d2/d1b/d1e/d2a/d4b/d77/d88/da4/da8 to d2/d120 0 2024-09-21T04:35:21.736 INFO:tasks.workunit.client.0.smithi007.stdout:7/334: link d3/d6/d13/f2c d3/d6/d13/d4b/f7d 0 2024-09-21T04:35:21.737 INFO:tasks.workunit.client.0.smithi007.stdout:1/489: dread d5/d6/d21/d2b/f2d [0,4194304] 0 2024-09-21T04:35:21.739 INFO:tasks.workunit.client.0.smithi007.stdout:4/444: dwrite d2/d7/d55/f80 [0,4194304] 0 2024-09-21T04:35:21.742 INFO:tasks.workunit.client.1.smithi053.stdout:0/900: mknod d5/d10/c132 0 2024-09-21T04:35:21.744 INFO:tasks.workunit.client.0.smithi007.stdout:7/335: dread d3/d6/f7 [4194304,4194304] 0 2024-09-21T04:35:21.748 INFO:tasks.workunit.client.1.smithi053.stdout:9/932: creat d0/d1c/d52/f13d x:0 0 0 2024-09-21T04:35:21.749 INFO:tasks.workunit.client.0.smithi007.stdout:4/445: creat d2/d17/d2e/f95 x:0 0 0 2024-09-21T04:35:21.750 INFO:tasks.workunit.client.0.smithi007.stdout:4/446: write d2/d17/d2e/f33 [1768300,16112] 0 2024-09-21T04:35:21.751 INFO:tasks.workunit.client.0.smithi007.stdout:4/447: stat d2/d17/d2e/f78 0 2024-09-21T04:35:21.752 INFO:tasks.workunit.client.1.smithi053.stdout:1/964: link d7/d9/d43/d54/d7f/d36/d81/lfe d7/d9/da/d11/de5/l135 0 2024-09-21T04:35:21.753 INFO:tasks.workunit.client.0.smithi007.stdout:5/564: creat d2/d27/d42/da3/fb3 x:0 0 0 2024-09-21T04:35:21.756 INFO:tasks.workunit.client.0.smithi007.stdout:1/490: rename d5/d45/d94/c49 to d5/d2f/c9b 0 2024-09-21T04:35:21.756 INFO:tasks.workunit.client.1.smithi053.stdout:5/979: sync 2024-09-21T04:35:21.763 INFO:tasks.workunit.client.1.smithi053.stdout:1/965: mkdir d7/d9/da/d40/d49/d136 0 2024-09-21T04:35:21.764 INFO:tasks.workunit.client.1.smithi053.stdout:7/820: write d0/f49 [1205342,48492] 0 2024-09-21T04:35:21.765 INFO:tasks.workunit.client.0.smithi007.stdout:7/336: truncate d3/d6/d10/d43/f64 512698 0 2024-09-21T04:35:21.767 INFO:tasks.workunit.client.1.smithi053.stdout:6/921: dread d2/fb [0,4194304] 0 2024-09-21T04:35:21.767 INFO:tasks.workunit.client.1.smithi053.stdout:0/901: truncate d5/de/d1c/f7e 3470186 0 2024-09-21T04:35:21.768 INFO:tasks.workunit.client.0.smithi007.stdout:1/491: link d5/d6/d21/f2c d5/d45/d5e/d55/f9c 0 2024-09-21T04:35:21.769 INFO:tasks.workunit.client.1.smithi053.stdout:5/980: rename d0/d8/db6/dbc/c12b to d0/d8/db/d68/c13f 0 2024-09-21T04:35:21.770 INFO:tasks.workunit.client.1.smithi053.stdout:3/947: dwrite d16/d17/d21/d42/d43/d125/d127/f103 [0,4194304] 0 2024-09-21T04:35:21.771 INFO:tasks.workunit.client.0.smithi007.stdout:4/448: creat d2/d7/f96 x:0 0 0 2024-09-21T04:35:21.773 INFO:tasks.workunit.client.1.smithi053.stdout:9/933: dwrite d0/d6c/fa7 [0,4194304] 0 2024-09-21T04:35:21.773 INFO:tasks.workunit.client.0.smithi007.stdout:5/565: rename d2/d27/d42/da3/d23/d2a/c3c to d2/d27/d42/da3/cb4 0 2024-09-21T04:35:21.776 INFO:tasks.workunit.client.0.smithi007.stdout:1/492: unlink d5/d45/d5e/f8f 0 2024-09-21T04:35:21.783 INFO:tasks.workunit.client.1.smithi053.stdout:2/992: sync 2024-09-21T04:35:21.784 INFO:tasks.workunit.client.0.smithi007.stdout:1/493: dread d5/d6/d21/d2b/d4b/f70 [0,4194304] 0 2024-09-21T04:35:21.786 INFO:tasks.workunit.client.1.smithi053.stdout:0/902: write d5/dd/ff [5193078,106876] 0 2024-09-21T04:35:21.787 INFO:tasks.workunit.client.0.smithi007.stdout:7/337: rename d3/d6/cc to d3/d6/d14/d20/d58/c7e 0 2024-09-21T04:35:21.789 INFO:tasks.workunit.client.1.smithi053.stdout:1/966: rename d7/d47/d79/dc1/ld8 to d7/d47/l137 0 2024-09-21T04:35:21.790 INFO:tasks.workunit.client.1.smithi053.stdout:1/967: fsync d7/d9/da/d40/d49/d77/f134 0 2024-09-21T04:35:21.791 INFO:tasks.workunit.client.1.smithi053.stdout:1/968: dread - d7/d47/d79/d99/fe9 zero size 2024-09-21T04:35:21.792 INFO:tasks.workunit.client.0.smithi007.stdout:4/449: mkdir d2/d97 0 2024-09-21T04:35:21.793 INFO:tasks.workunit.client.0.smithi007.stdout:4/450: chown d2/d5f/f93 0 1 2024-09-21T04:35:21.793 INFO:tasks.workunit.client.1.smithi053.stdout:7/821: write d0/d6f/dbf/d50/f96 [1196968,123255] 0 2024-09-21T04:35:21.794 INFO:tasks.workunit.client.1.smithi053.stdout:3/948: truncate d16/d17/d21/d60/f9c 741685 0 2024-09-21T04:35:21.796 INFO:tasks.workunit.client.0.smithi007.stdout:1/494: creat d5/d45/d5e/d55/f9d x:0 0 0 2024-09-21T04:35:21.796 INFO:tasks.workunit.client.1.smithi053.stdout:9/934: rmdir d0/d1c/d21 39 2024-09-21T04:35:21.797 INFO:tasks.workunit.client.0.smithi007.stdout:7/338: creat d3/d6/d10/f7f x:0 0 0 2024-09-21T04:35:21.799 INFO:tasks.workunit.client.0.smithi007.stdout:4/451: creat d2/d17/d2e/d45/d5d/f98 x:0 0 0 2024-09-21T04:35:21.800 INFO:tasks.workunit.client.0.smithi007.stdout:4/452: chown d2/d7/d55/d4f/l7f 210499 1 2024-09-21T04:35:21.801 INFO:tasks.workunit.client.0.smithi007.stdout:7/339: rename d3/d6/d10/d43 to d3/d6/d14/d20/d80 0 2024-09-21T04:35:21.802 INFO:tasks.workunit.client.0.smithi007.stdout:1/495: dwrite d5/d45/d5e/d55/f80 [0,4194304] 0 2024-09-21T04:35:21.802 INFO:tasks.workunit.client.1.smithi053.stdout:6/922: mknod d2/d1b/d1e/d2a/d4b/deb/df5/c121 0 2024-09-21T04:35:21.803 INFO:tasks.workunit.client.0.smithi007.stdout:4/453: truncate d2/d17/d29/f69 720887 0 2024-09-21T04:35:21.804 INFO:tasks.workunit.client.0.smithi007.stdout:7/340: write d3/d6/d10/f34 [127919,60530] 0 2024-09-21T04:35:21.805 INFO:tasks.workunit.client.0.smithi007.stdout:7/341: fdatasync d3/d6/d2e/f6d 0 2024-09-21T04:35:21.807 INFO:tasks.workunit.client.1.smithi053.stdout:3/949: creat d16/d17/d21/d8b/d7d/dc6/dec/f153 x:0 0 0 2024-09-21T04:35:21.809 INFO:tasks.workunit.client.0.smithi007.stdout:1/496: creat d5/d45/d94/f9e x:0 0 0 2024-09-21T04:35:21.810 INFO:tasks.workunit.client.1.smithi053.stdout:9/935: rename d0/d2 to d0/d1c/d52/d6d/d13e 0 2024-09-21T04:35:21.811 INFO:tasks.workunit.client.1.smithi053.stdout:9/936: chown d0/d1c/d52/d6f/cbd 0 1 2024-09-21T04:35:21.812 INFO:tasks.workunit.client.0.smithi007.stdout:1/497: write d5/d6/d21/d2b/d4b/d8a/f97 [31174,116740] 0 2024-09-21T04:35:21.813 INFO:tasks.workunit.client.0.smithi007.stdout:4/454: dwrite d2/d7/d64/f7d [0,4194304] 0 2024-09-21T04:35:21.815 INFO:tasks.workunit.client.1.smithi053.stdout:2/993: symlink d1/d18/d1f/d27/d58/d5c/l158 0 2024-09-21T04:35:21.815 INFO:tasks.workunit.client.1.smithi053.stdout:7/822: dread d0/d6f/dbf/d50/d8b/f8 [0,4194304] 0 2024-09-21T04:35:21.817 INFO:tasks.workunit.client.1.smithi053.stdout:2/994: chown d1/d18/d1f/d22/d43/f4f 127 1 2024-09-21T04:35:21.821 INFO:tasks.workunit.client.0.smithi007.stdout:1/498: rename d5/c33 to d5/c9f 0 2024-09-21T04:35:21.823 INFO:tasks.workunit.client.0.smithi007.stdout:4/455: mknod d2/d5f/c99 0 2024-09-21T04:35:21.824 INFO:tasks.workunit.client.0.smithi007.stdout:7/342: dread d3/d6/d14/d20/d33/f25 [0,4194304] 0 2024-09-21T04:35:21.828 INFO:tasks.workunit.client.0.smithi007.stdout:1/499: dread d5/d45/d5e/d55/f65 [0,4194304] 0 2024-09-21T04:35:21.829 INFO:tasks.workunit.client.0.smithi007.stdout:4/456: unlink d2/d17/d2e/f50 0 2024-09-21T04:35:21.831 INFO:tasks.workunit.client.0.smithi007.stdout:3/298: sync 2024-09-21T04:35:21.833 INFO:tasks.workunit.client.0.smithi007.stdout:4/457: dread d2/d7/f8 [0,4194304] 0 2024-09-21T04:35:21.833 INFO:tasks.workunit.client.0.smithi007.stdout:3/299: readlink dc/d22/d29/l53 0 2024-09-21T04:35:21.834 INFO:tasks.workunit.client.1.smithi053.stdout:3/950: rename d16/d17/d21/d8b/d7d/ce6 to d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/c154 0 2024-09-21T04:35:21.834 INFO:tasks.workunit.client.0.smithi007.stdout:7/343: dread d3/d6/d13/f75 [0,4194304] 0 2024-09-21T04:35:21.837 INFO:tasks.workunit.client.1.smithi053.stdout:7/823: unlink d0/f4 0 2024-09-21T04:35:21.837 INFO:tasks.workunit.client.1.smithi053.stdout:0/903: dwrite d5/de/d1c/f29 [0,4194304] 0 2024-09-21T04:35:21.839 INFO:tasks.workunit.client.0.smithi007.stdout:4/458: dwrite d2/d17/d29/f58 [0,4194304] 0 2024-09-21T04:35:21.843 INFO:tasks.workunit.client.0.smithi007.stdout:3/300: mkdir dc/d22/d5a 0 2024-09-21T04:35:21.845 INFO:tasks.workunit.client.1.smithi053.stdout:1/969: truncate d7/d47/d79/d99/db0/fe7 1318148 0 2024-09-21T04:35:21.846 INFO:tasks.workunit.client.1.smithi053.stdout:1/970: fdatasync d7/d9/da/d3c/f65 0 2024-09-21T04:35:21.846 INFO:tasks.workunit.client.1.smithi053.stdout:5/981: link d0/dd/d48/f54 d0/d8/db6/f140 0 2024-09-21T04:35:21.848 INFO:tasks.workunit.client.0.smithi007.stdout:7/344: mkdir d3/d6/d2e/d63/d6a/d81 0 2024-09-21T04:35:21.849 INFO:tasks.workunit.client.0.smithi007.stdout:7/345: chown d3/d6/d10/c52 174899006 1 2024-09-21T04:35:21.854 INFO:tasks.workunit.client.0.smithi007.stdout:3/301: mknod dc/d22/d29/d32/d38/c5b 0 2024-09-21T04:35:21.855 INFO:tasks.workunit.client.0.smithi007.stdout:4/459: fsync d2/d17/d29/f58 0 2024-09-21T04:35:21.856 INFO:tasks.workunit.client.0.smithi007.stdout:4/460: write d2/d7/f96 [296318,73502] 0 2024-09-21T04:35:21.857 INFO:tasks.workunit.client.0.smithi007.stdout:4/461: fdatasync d2/d17/d2e/d45/d5e/f82 0 2024-09-21T04:35:21.858 INFO:tasks.workunit.client.0.smithi007.stdout:1/500: rename d5/d6/d21/d64/d71 to d5/d45/d94/da0 0 2024-09-21T04:35:21.859 INFO:tasks.workunit.client.1.smithi053.stdout:2/995: write d1/d18/d1f/d27/d41/d4b/d4d/d89/dfc/de9/fff [1137551,78216] 0 2024-09-21T04:35:21.860 INFO:tasks.workunit.client.0.smithi007.stdout:7/346: truncate d3/d6/f7 5717104 0 2024-09-21T04:35:21.866 INFO:tasks.workunit.client.0.smithi007.stdout:1/501: dwrite d5/d6/f93 [0,4194304] 0 2024-09-21T04:35:21.868 INFO:tasks.workunit.client.0.smithi007.stdout:1/502: dread - d5/d6/d21/d2b/f6f zero size 2024-09-21T04:35:21.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/302: dread dc/f13 [4194304,4194304] 0 2024-09-21T04:35:21.871 INFO:tasks.workunit.client.0.smithi007.stdout:3/303: write dc/f23 [408500,60412] 0 2024-09-21T04:35:21.875 INFO:tasks.workunit.client.0.smithi007.stdout:1/503: symlink d5/d45/d94/la1 0 2024-09-21T04:35:21.878 INFO:tasks.workunit.client.0.smithi007.stdout:3/304: dwrite dc/f23 [4194304,4194304] 0 2024-09-21T04:35:21.882 INFO:tasks.workunit.client.0.smithi007.stdout:1/504: chown d5/f28 55290 1 2024-09-21T04:35:21.883 INFO:tasks.workunit.client.0.smithi007.stdout:1/505: chown d5/d45/d94 326 1 2024-09-21T04:35:21.883 INFO:tasks.workunit.client.0.smithi007.stdout:3/305: creat dc/d22/d29/d32/d38/f5c x:0 0 0 2024-09-21T04:35:21.885 INFO:tasks.workunit.client.0.smithi007.stdout:3/306: rename dc/d22/d29/d32 to dc/d22/d29/d32/d4f/d5d 22 2024-09-21T04:35:21.885 INFO:tasks.workunit.client.0.smithi007.stdout:7/347: dwrite d3/f4c [0,4194304] 0 2024-09-21T04:35:21.886 INFO:tasks.workunit.client.0.smithi007.stdout:3/307: stat dc/f23 0 2024-09-21T04:35:21.890 INFO:tasks.workunit.client.0.smithi007.stdout:7/348: symlink d3/d6/d2e/d63/d6a/l82 0 2024-09-21T04:35:21.895 INFO:tasks.workunit.client.0.smithi007.stdout:7/349: unlink d3/d6/d13/c16 0 2024-09-21T04:35:21.896 INFO:tasks.workunit.client.0.smithi007.stdout:7/350: chown d3/d6/cb 17315 1 2024-09-21T04:35:21.897 INFO:tasks.workunit.client.0.smithi007.stdout:3/308: creat dc/d22/d29/d32/d4f/f5e x:0 0 0 2024-09-21T04:35:21.898 INFO:tasks.workunit.client.0.smithi007.stdout:7/351: rmdir d3/d6/d10 39 2024-09-21T04:35:21.900 INFO:tasks.workunit.client.0.smithi007.stdout:3/309: creat dc/d18/d26/f5f x:0 0 0 2024-09-21T04:35:21.904 INFO:tasks.workunit.client.0.smithi007.stdout:7/352: symlink d3/d6/d10/d62/d66/d69/l83 0 2024-09-21T04:35:21.910 INFO:tasks.workunit.client.0.smithi007.stdout:7/353: dread d3/d6/d13/f29 [0,4194304] 0 2024-09-21T04:35:21.911 INFO:tasks.workunit.client.0.smithi007.stdout:7/354: write d3/d6/d14/d20/d80/f6f [932776,94501] 0 2024-09-21T04:35:21.912 INFO:tasks.workunit.client.1.smithi053.stdout:3/951: symlink d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/l155 0 2024-09-21T04:35:21.913 INFO:tasks.workunit.client.1.smithi053.stdout:7/824: symlink d0/d6f/dbf/d50/d8b/d70/dc1/l117 0 2024-09-21T04:35:21.914 INFO:tasks.workunit.client.1.smithi053.stdout:9/937: dread d0/d1c/d21/f9f [0,4194304] 0 2024-09-21T04:35:21.915 INFO:tasks.workunit.client.1.smithi053.stdout:6/923: creat d2/d1b/d1e/d2a/f122 x:0 0 0 2024-09-21T04:35:21.916 INFO:tasks.workunit.client.1.smithi053.stdout:6/924: chown d2/d1b/d3d/d49/d5c/d8d/cb9 185273831 1 2024-09-21T04:35:21.917 INFO:tasks.workunit.client.1.smithi053.stdout:5/982: unlink d0/dd/d48/faa 0 2024-09-21T04:35:21.918 INFO:tasks.workunit.client.1.smithi053.stdout:1/971: symlink d7/d9/da/d4c/l138 0 2024-09-21T04:35:21.922 INFO:tasks.workunit.client.1.smithi053.stdout:3/952: creat d16/d17/d21/d42/da0/dfb/d11e/f156 x:0 0 0 2024-09-21T04:35:21.923 INFO:tasks.workunit.client.1.smithi053.stdout:0/904: read - d5/d10/d4c/db1/dbc/dca/d21/d96/fea zero size 2024-09-21T04:35:21.923 INFO:tasks.workunit.client.0.smithi007.stdout:7/355: dread d3/d6/f28 [0,4194304] 0 2024-09-21T04:35:21.924 INFO:tasks.workunit.client.1.smithi053.stdout:0/905: write d5/de/d1c/ff1 [3295590,84721] 0 2024-09-21T04:35:21.925 INFO:tasks.workunit.client.1.smithi053.stdout:0/906: dread - d5/d10/d4c/db1/dbc/dca/db5/f125 zero size 2024-09-21T04:35:21.925 INFO:tasks.workunit.client.1.smithi053.stdout:9/938: symlink d0/d1a/d4c/d8f/l13f 0 2024-09-21T04:35:21.932 INFO:tasks.workunit.client.1.smithi053.stdout:7/825: creat d0/d2b/d36/d115/f118 x:0 0 0 2024-09-21T04:35:21.934 INFO:tasks.workunit.client.1.smithi053.stdout:0/907: read d5/dd/d45/f6c [822531,3446] 0 2024-09-21T04:35:21.935 INFO:tasks.workunit.client.0.smithi007.stdout:7/356: truncate d3/d6/d2e/f6d 45749 0 2024-09-21T04:35:21.935 INFO:tasks.workunit.client.1.smithi053.stdout:0/908: chown d5/d10/d4c/db1/dbc/dca/db5/lef 49567210 1 2024-09-21T04:35:21.936 INFO:tasks.workunit.client.1.smithi053.stdout:9/939: fdatasync d0/d1c/d52/d6d/d13e/ddf/f130 0 2024-09-21T04:35:21.937 INFO:tasks.workunit.client.1.smithi053.stdout:6/925: fsync d2/d1b/d3d/d49/de/d32/d80/f83 0 2024-09-21T04:35:21.939 INFO:tasks.workunit.client.1.smithi053.stdout:2/996: creat d1/d18/d1f/d27/d41/f159 x:0 0 0 2024-09-21T04:35:21.940 INFO:tasks.workunit.client.1.smithi053.stdout:5/983: truncate d0/d8/f2b 3833473 0 2024-09-21T04:35:21.942 INFO:tasks.workunit.client.1.smithi053.stdout:3/953: symlink d16/d17/d21/d8b/d7d/d57/dc1/d53/l157 0 2024-09-21T04:35:21.944 INFO:tasks.workunit.client.1.smithi053.stdout:0/909: chown d5/d10/cf2 789237957 1 2024-09-21T04:35:21.945 INFO:tasks.workunit.client.1.smithi053.stdout:9/940: mkdir d0/df/d140 0 2024-09-21T04:35:21.947 INFO:tasks.workunit.client.1.smithi053.stdout:2/997: creat d1/d18/d1f/d27/d41/d4b/ddc/f15a x:0 0 0 2024-09-21T04:35:21.947 INFO:tasks.workunit.client.1.smithi053.stdout:5/984: readlink d0/d8/l27 0 2024-09-21T04:35:21.949 INFO:tasks.workunit.client.1.smithi053.stdout:5/985: truncate d0/d8/db6/dbc/f115 987998 0 2024-09-21T04:35:21.952 INFO:tasks.workunit.client.1.smithi053.stdout:3/954: symlink d16/d17/l158 0 2024-09-21T04:35:21.953 INFO:tasks.workunit.client.1.smithi053.stdout:7/826: link d0/d6f/dbf/d5c/cda d0/d6f/dbf/d5c/dac/c119 0 2024-09-21T04:35:21.956 INFO:tasks.workunit.client.1.smithi053.stdout:2/998: mknod d1/d18/d1f/d22/c15b 0 2024-09-21T04:35:21.957 INFO:tasks.workunit.client.1.smithi053.stdout:3/955: unlink d16/d17/d21/d8b/d7d/d57/df5/ffd 0 2024-09-21T04:35:21.958 INFO:tasks.workunit.client.1.smithi053.stdout:7/827: chown d0/d2b/d36/d64/fb7 729152 1 2024-09-21T04:35:21.958 INFO:tasks.workunit.client.1.smithi053.stdout:3/956: chown d16/d1b/dad/lcb 56 1 2024-09-21T04:35:21.959 INFO:tasks.workunit.client.1.smithi053.stdout:6/926: rename d2/d1b/d3d/d49/de/d32/d80/dcd to d2/d1b/d3d/d49/de/d32/d80/d123 0 2024-09-21T04:35:21.964 INFO:tasks.workunit.client.1.smithi053.stdout:3/957: creat d16/d17/d21/d8b/d7d/d57/f159 x:0 0 0 2024-09-21T04:35:21.967 INFO:tasks.workunit.client.1.smithi053.stdout:7/828: symlink d0/d6f/dbf/d50/l11a 0 2024-09-21T04:35:21.967 INFO:tasks.workunit.client.1.smithi053.stdout:1/972: dwrite d7/d9/da/d40/d49/d77/f90 [0,4194304] 0 2024-09-21T04:35:21.969 INFO:tasks.workunit.client.1.smithi053.stdout:1/973: dread - d7/d47/f11a zero size 2024-09-21T04:35:21.969 INFO:tasks.workunit.client.1.smithi053.stdout:3/958: chown d16/d17/d21/d8b/d7d/d57/d90/l10c 186209 1 2024-09-21T04:35:21.969 INFO:tasks.workunit.client.1.smithi053.stdout:5/986: link d0/dd/d5c/c12d d0/d8/d12e/dad/db9/deb/d111/c141 0 2024-09-21T04:35:21.970 INFO:tasks.workunit.client.1.smithi053.stdout:6/927: dwrite d2/d1d/f54 [0,4194304] 0 2024-09-21T04:35:21.971 INFO:tasks.workunit.client.0.smithi007.stdout:7/357: dwrite d3/d6/f7 [0,4194304] 0 2024-09-21T04:35:21.971 INFO:tasks.workunit.client.1.smithi053.stdout:1/974: stat d7/d63/d96/dc7 0 2024-09-21T04:35:21.972 INFO:tasks.workunit.client.0.smithi007.stdout:7/358: chown d3/d6/d10/d62 61 1 2024-09-21T04:35:21.973 INFO:tasks.workunit.client.0.smithi007.stdout:7/359: chown d3/f5 164459589 1 2024-09-21T04:35:21.974 INFO:tasks.workunit.client.1.smithi053.stdout:1/975: dread - d7/d9/d43/f10f zero size 2024-09-21T04:35:21.977 INFO:tasks.workunit.client.0.smithi007.stdout:7/360: fsync d3/d6/d14/d20/d80/f64 0 2024-09-21T04:35:21.978 INFO:tasks.workunit.client.1.smithi053.stdout:0/910: write d5/d10/d4c/db1/dbc/fcf [593158,70148] 0 2024-09-21T04:35:21.981 INFO:tasks.workunit.client.1.smithi053.stdout:6/928: creat d2/d1b/d1e/dc1/f124 x:0 0 0 2024-09-21T04:35:21.985 INFO:tasks.workunit.client.1.smithi053.stdout:6/929: stat d2/d1b/d3d/d49/de/cda 0 2024-09-21T04:35:21.988 INFO:tasks.workunit.client.1.smithi053.stdout:7/829: dwrite d0/f13 [0,4194304] 0 2024-09-21T04:35:21.990 INFO:tasks.workunit.client.1.smithi053.stdout:3/959: dwrite d16/d17/d21/d42/d43/d125/d127/f123 [4194304,4194304] 0 2024-09-21T04:35:21.991 INFO:tasks.workunit.client.1.smithi053.stdout:2/999: truncate d1/d18/d1f/d27/d58/d5c/d91/dd0/fb2 3684822 0 2024-09-21T04:35:21.992 INFO:tasks.workunit.client.0.smithi007.stdout:7/361: truncate d3/d6/d2e/d63/d6a/f70 6497011 0 2024-09-21T04:35:21.993 INFO:tasks.workunit.client.1.smithi053.stdout:9/941: dwrite d0/d1c/d52/df7/ffc [0,4194304] 0 2024-09-21T04:35:21.993 INFO:tasks.workunit.client.1.smithi053.stdout:3/960: dread - d16/d1b/dad/f140 zero size 2024-09-21T04:35:21.996 INFO:tasks.workunit.client.1.smithi053.stdout:1/976: write d7/d9/d43/d54/d7f/d36/f98 [74003,76571] 0 2024-09-21T04:35:21.998 INFO:tasks.workunit.client.0.smithi007.stdout:7/362: link d3/d6/d13/l6b d3/d6/d10/d62/d66/l84 0 2024-09-21T04:35:21.999 INFO:tasks.workunit.client.1.smithi053.stdout:1/977: dread - d7/d47/f11a zero size 2024-09-21T04:35:22.002 INFO:tasks.workunit.client.1.smithi053.stdout:9/942: creat d0/d1c/d52/daa/f141 x:0 0 0 2024-09-21T04:35:22.003 INFO:tasks.workunit.client.0.smithi007.stdout:7/363: rename d3/d6/d14/d20/d33 to d3/d6/d85 0 2024-09-21T04:35:22.003 INFO:tasks.workunit.client.0.smithi007.stdout:7/364: chown d3/d6/d14/l2a 431804 1 2024-09-21T04:35:22.005 INFO:tasks.workunit.client.1.smithi053.stdout:7/830: dwrite d0/d6f/dbf/d95/fc9 [0,4194304] 0 2024-09-21T04:35:22.006 INFO:tasks.workunit.client.1.smithi053.stdout:7/831: write d0/ff0 [907699,125770] 0 2024-09-21T04:35:22.008 INFO:tasks.workunit.client.1.smithi053.stdout:1/978: creat d7/d9/da/d40/d123/f139 x:0 0 0 2024-09-21T04:35:22.009 INFO:tasks.workunit.client.0.smithi007.stdout:7/365: symlink d3/d6/d10/d62/d66/l86 0 2024-09-21T04:35:22.012 INFO:tasks.workunit.client.1.smithi053.stdout:9/943: symlink d0/d1c/d52/d6d/d13e/d4e/l142 0 2024-09-21T04:35:22.015 INFO:tasks.workunit.client.0.smithi007.stdout:7/366: rename d3/f7b to d3/d6/d14/d20/d58/f87 0 2024-09-21T04:35:22.015 INFO:tasks.workunit.client.1.smithi053.stdout:7/832: dwrite d0/d6f/dbf/d5d/d9f/f109 [0,4194304] 0 2024-09-21T04:35:22.017 INFO:tasks.workunit.client.1.smithi053.stdout:1/979: rename d7/d9/da/d11/de5 to d7/d47/d79/d99/db0/d13a 0 2024-09-21T04:35:22.021 INFO:tasks.workunit.client.1.smithi053.stdout:6/930: truncate d2/d1b/d1e/d2a/d4b/d77/d88/fab 1589128 0 2024-09-21T04:35:22.023 INFO:tasks.workunit.client.0.smithi007.stdout:7/367: creat d3/f88 x:0 0 0 2024-09-21T04:35:22.026 INFO:tasks.workunit.client.1.smithi053.stdout:3/961: dwrite d16/d17/d21/d8b/ff2 [0,4194304] 0 2024-09-21T04:35:22.026 INFO:tasks.workunit.client.1.smithi053.stdout:1/980: creat d7/da8/d74/f13b x:0 0 0 2024-09-21T04:35:22.030 INFO:tasks.workunit.client.1.smithi053.stdout:9/944: dread d0/d1c/d52/d6d/f87 [0,4194304] 0 2024-09-21T04:35:22.031 INFO:tasks.workunit.client.1.smithi053.stdout:3/962: creat d16/d17/d21/d42/d43/d125/d127/f15a x:0 0 0 2024-09-21T04:35:22.037 INFO:tasks.workunit.client.1.smithi053.stdout:1/981: dwrite d7/f56 [0,4194304] 0 2024-09-21T04:35:22.038 INFO:tasks.workunit.client.1.smithi053.stdout:1/982: chown d7/d9/da/d40/d49 43862 1 2024-09-21T04:35:22.040 INFO:tasks.workunit.client.1.smithi053.stdout:1/983: dread - d7/d47/fd6 zero size 2024-09-21T04:35:22.042 INFO:tasks.workunit.client.1.smithi053.stdout:7/833: dread d0/d6f/dbf/d50/d8b/f83 [0,4194304] 0 2024-09-21T04:35:22.044 INFO:tasks.workunit.client.1.smithi053.stdout:1/984: write d7/d9/da/d4c/f7e [2831588,56876] 0 2024-09-21T04:35:22.049 INFO:tasks.workunit.client.1.smithi053.stdout:6/931: dwrite d2/d1b/d3d/d49/de/d32/d80/d2e/f79 [0,4194304] 0 2024-09-21T04:35:22.049 INFO:tasks.workunit.client.1.smithi053.stdout:1/985: chown d7/d63/d6b/lc8 28467450 1 2024-09-21T04:35:22.054 INFO:tasks.workunit.client.1.smithi053.stdout:6/932: stat d2/d1b/d1e/d27/d41/f10c 0 2024-09-21T04:35:22.054 INFO:tasks.workunit.client.1.smithi053.stdout:9/945: dwrite d0/d1a/f2c [8388608,4194304] 0 2024-09-21T04:35:22.054 INFO:tasks.workunit.client.1.smithi053.stdout:1/986: readlink d7/d9/da/d40/d49/d77/l103 0 2024-09-21T04:35:22.055 INFO:tasks.workunit.client.1.smithi053.stdout:3/963: dwrite d16/d17/d21/d60/f9c [0,4194304] 0 2024-09-21T04:35:22.058 INFO:tasks.workunit.client.1.smithi053.stdout:7/834: link d0/d6f/dbf/d5c/f8a d0/d6f/dbf/d50/dc7/f11b 0 2024-09-21T04:35:22.061 INFO:tasks.workunit.client.1.smithi053.stdout:3/964: creat d16/d17/d21/d60/f15b x:0 0 0 2024-09-21T04:35:22.062 INFO:tasks.workunit.client.1.smithi053.stdout:1/987: rename c2 to d7/d9/da/d40/d49/d136/c13c 0 2024-09-21T04:35:22.063 INFO:tasks.workunit.client.1.smithi053.stdout:3/965: symlink d16/d17/d21/d42/da0/dfb/l15c 0 2024-09-21T04:35:22.066 INFO:tasks.workunit.client.1.smithi053.stdout:9/946: getdents d0/d1a 0 2024-09-21T04:35:22.069 INFO:tasks.workunit.client.1.smithi053.stdout:3/966: truncate d16/d17/d21/d8b/d110/f138 31592 0 2024-09-21T04:35:22.069 INFO:tasks.workunit.client.1.smithi053.stdout:6/933: dwrite d2/fd [0,4194304] 0 2024-09-21T04:35:22.071 INFO:tasks.workunit.client.1.smithi053.stdout:1/988: rename d7/d9/d43/d54/f80 to d7/d47/d79/d99/db0/f13d 0 2024-09-21T04:35:22.074 INFO:tasks.workunit.client.1.smithi053.stdout:3/967: symlink d16/d17/d21/d8b/d7d/d57/dc1/l15d 0 2024-09-21T04:35:22.076 INFO:tasks.workunit.client.1.smithi053.stdout:7/835: write d0/d6f/dbf/d50/d8b/f83 [2164595,41287] 0 2024-09-21T04:35:22.080 INFO:tasks.workunit.client.1.smithi053.stdout:3/968: creat d16/d17/d21/d8b/d7d/d57/dc1/f15e x:0 0 0 2024-09-21T04:35:22.080 INFO:tasks.workunit.client.1.smithi053.stdout:6/934: mkdir d2/d1b/d3d/d49/de/d32/d80/d123/d10d/d125 0 2024-09-21T04:35:22.083 INFO:tasks.workunit.client.1.smithi053.stdout:1/989: getdents d7/d9 0 2024-09-21T04:35:22.083 INFO:tasks.workunit.client.1.smithi053.stdout:6/935: stat d2/d1b/d1e/dc1/f124 0 2024-09-21T04:35:22.084 INFO:tasks.workunit.client.1.smithi053.stdout:7/836: read d0/d6f/dbf/f8f [1114911,113256] 0 2024-09-21T04:35:22.085 INFO:tasks.workunit.client.1.smithi053.stdout:9/947: dwrite d0/d1c/d52/d6d/d13e/d9/f95 [0,4194304] 0 2024-09-21T04:35:22.086 INFO:tasks.workunit.client.1.smithi053.stdout:3/969: truncate d16/d17/d21/d42/ff0 193465 0 2024-09-21T04:35:22.089 INFO:tasks.workunit.client.1.smithi053.stdout:1/990: symlink d7/d47/d79/d99/l13e 0 2024-09-21T04:35:22.090 INFO:tasks.workunit.client.1.smithi053.stdout:9/948: rmdir d0/d1a/dcd/d12e 39 2024-09-21T04:35:22.090 INFO:tasks.workunit.client.1.smithi053.stdout:7/837: fdatasync d0/d6f/dbf/d50/fa5 0 2024-09-21T04:35:22.092 INFO:tasks.workunit.client.1.smithi053.stdout:3/970: creat d16/d17/d21/d42/d43/d125/d12e/f15f x:0 0 0 2024-09-21T04:35:22.094 INFO:tasks.workunit.client.1.smithi053.stdout:1/991: write d7/d9/d43/d54/d7f/d59/dc3/f11d [411204,85285] 0 2024-09-21T04:35:22.094 INFO:tasks.workunit.client.1.smithi053.stdout:7/838: fdatasync d0/f61 0 2024-09-21T04:35:22.099 INFO:tasks.workunit.client.1.smithi053.stdout:7/839: creat d0/d6f/dbf/d50/dc7/f11c x:0 0 0 2024-09-21T04:35:22.100 INFO:tasks.workunit.client.1.smithi053.stdout:1/992: fdatasync d7/d9/d43/d54/d7f/fcc 0 2024-09-21T04:35:22.102 INFO:tasks.workunit.client.1.smithi053.stdout:9/949: dwrite d0/d6c/f138 [0,4194304] 0 2024-09-21T04:35:22.107 INFO:tasks.workunit.client.1.smithi053.stdout:9/950: mkdir d0/d1c/d52/df7/d10d/d143 0 2024-09-21T04:35:22.107 INFO:tasks.workunit.client.1.smithi053.stdout:7/840: write d0/d6f/dbf/d50/d8b/f42 [274075,27144] 0 2024-09-21T04:35:22.110 INFO:tasks.workunit.client.1.smithi053.stdout:6/936: dread d2/d1b/d1e/d2a/d4b/d77/d88/da4/fcc [0,4194304] 0 2024-09-21T04:35:22.112 INFO:tasks.workunit.client.1.smithi053.stdout:9/951: mkdir d0/d1a/dcd/d144 0 2024-09-21T04:35:22.114 INFO:tasks.workunit.client.1.smithi053.stdout:3/971: dread fa [4194304,4194304] 0 2024-09-21T04:35:22.115 INFO:tasks.workunit.client.1.smithi053.stdout:9/952: readlink d0/d1a/l135 0 2024-09-21T04:35:22.117 INFO:tasks.workunit.client.1.smithi053.stdout:1/993: dwrite d7/d9/d43/d54/d7f/d36/d81/f11c [0,4194304] 0 2024-09-21T04:35:22.118 INFO:tasks.workunit.client.1.smithi053.stdout:5/987: sync 2024-09-21T04:35:22.119 INFO:tasks.workunit.client.1.smithi053.stdout:6/937: write d2/d1b/d3d/d49/de/d32/fd2 [817658,9830] 0 2024-09-21T04:35:22.122 INFO:tasks.workunit.client.1.smithi053.stdout:7/841: dwrite d0/d6f/f72 [4194304,4194304] 0 2024-09-21T04:35:22.125 INFO:tasks.workunit.client.1.smithi053.stdout:7/842: readlink d0/d24/l8e 0 2024-09-21T04:35:22.126 INFO:tasks.workunit.client.1.smithi053.stdout:1/994: dread d7/da8/d74/d75/f88 [0,4194304] 0 2024-09-21T04:35:22.126 INFO:tasks.workunit.client.1.smithi053.stdout:5/988: chown d0/dd/d5c/d92/ddb/d35/c41 42 1 2024-09-21T04:35:22.126 INFO:tasks.workunit.client.1.smithi053.stdout:3/972: truncate d16/d17/d21/d42/d43/d86/f116 20768 0 2024-09-21T04:35:22.128 INFO:tasks.workunit.client.1.smithi053.stdout:6/938: readlink d2/d1b/d3d/d49/de/d32/lb7 0 2024-09-21T04:35:22.131 INFO:tasks.workunit.client.1.smithi053.stdout:1/995: write d7/d47/ff2 [3263338,35924] 0 2024-09-21T04:35:22.131 INFO:tasks.workunit.client.1.smithi053.stdout:9/953: dwrite d0/d1c/d25/f119 [0,4194304] 0 2024-09-21T04:35:22.132 INFO:tasks.workunit.client.1.smithi053.stdout:6/939: dread - d2/d1b/d1e/d27/d41/f10b zero size 2024-09-21T04:35:22.134 INFO:tasks.workunit.client.1.smithi053.stdout:7/843: read d0/d6f/dbf/f113 [3670264,80640] 0 2024-09-21T04:35:22.134 INFO:tasks.workunit.client.1.smithi053.stdout:3/973: unlink d16/d1b/l4b 0 2024-09-21T04:35:22.138 INFO:tasks.workunit.client.1.smithi053.stdout:6/940: unlink d2/d1b/d1e/d2a/d4b/deb/df5/c121 0 2024-09-21T04:35:22.138 INFO:tasks.workunit.client.1.smithi053.stdout:3/974: write d16/d17/d21/d8b/d7d/dc6/dec/f14c [658188,21867] 0 2024-09-21T04:35:22.139 INFO:tasks.workunit.client.1.smithi053.stdout:7/844: getdents d0/dca/df5 0 2024-09-21T04:35:22.193 INFO:tasks.workunit.client.1.smithi053.stdout:5/989: getdents d0/dd/d5c/d92/ddb/d35 0 2024-09-21T04:35:22.193 INFO:tasks.workunit.client.1.smithi053.stdout:6/941: creat d2/d1b/d1e/dc1/f126 x:0 0 0 2024-09-21T04:35:22.195 INFO:tasks.workunit.client.1.smithi053.stdout:5/990: mknod d0/c142 0 2024-09-21T04:35:22.195 INFO:tasks.workunit.client.1.smithi053.stdout:6/942: chown d2/d1b/d3d/d49/de/f73 354641849 1 2024-09-21T04:35:22.197 INFO:tasks.workunit.client.1.smithi053.stdout:5/991: chown d0/dd/d5c/d80/f131 2580 1 2024-09-21T04:35:22.198 INFO:tasks.workunit.client.1.smithi053.stdout:5/992: rmdir d0/dd/d5c/d92/ddb/d51/d52 39 2024-09-21T04:35:22.201 INFO:tasks.workunit.client.1.smithi053.stdout:1/996: dread d7/d9/d43/f6e [0,4194304] 0 2024-09-21T04:35:22.201 INFO:tasks.workunit.client.1.smithi053.stdout:6/943: rename d2/d1b/d3d/d49/de/f73 to d2/d1b/d1e/d2a/d4b/deb/df5/f127 0 2024-09-21T04:35:22.201 INFO:tasks.workunit.client.1.smithi053.stdout:1/997: fsync d7/d24/fe1 0 2024-09-21T04:35:22.202 INFO:tasks.workunit.client.1.smithi053.stdout:5/993: creat d0/dd/d5c/d92/ddb/d35/d61/f143 x:0 0 0 2024-09-21T04:35:22.205 INFO:tasks.workunit.client.1.smithi053.stdout:6/944: creat d2/d1b/d1e/d27/d41/f128 x:0 0 0 2024-09-21T04:35:22.206 INFO:tasks.workunit.client.1.smithi053.stdout:7/845: write d0/d6f/f86 [236611,3739] 0 2024-09-21T04:35:22.207 INFO:tasks.workunit.client.1.smithi053.stdout:3/975: write d16/d17/d21/d8b/d7d/d9d/fb4 [541982,67838] 0 2024-09-21T04:35:22.211 INFO:tasks.workunit.client.1.smithi053.stdout:1/998: write d7/da8/fc4 [419401,65980] 0 2024-09-21T04:35:22.211 INFO:tasks.workunit.client.1.smithi053.stdout:5/994: dwrite d0/d8/d12e/dad/db9/deb/f11d [0,4194304] 0 2024-09-21T04:35:22.213 INFO:tasks.workunit.client.0.smithi007.stdout:9/337: sync 2024-09-21T04:35:22.213 INFO:tasks.workunit.client.1.smithi053.stdout:3/976: symlink d16/d17/d21/d42/l160 0 2024-09-21T04:35:22.215 INFO:tasks.workunit.client.1.smithi053.stdout:6/945: rename d2/d1b/d3d/d49/lc3 to d2/d1b/d3d/d49/de/d32/d80/d2e/l129 0 2024-09-21T04:35:22.218 INFO:tasks.workunit.client.0.smithi007.stdout:9/338: symlink d13/d18/l6a 0 2024-09-21T04:35:22.219 INFO:tasks.workunit.client.1.smithi053.stdout:5/995: rename d0/d8/d12e/lfc to d0/dd/d5c/d89/l144 0 2024-09-21T04:35:22.221 INFO:tasks.workunit.client.0.smithi007.stdout:9/339: dread d13/f4f [0,4194304] 0 2024-09-21T04:35:22.222 INFO:tasks.workunit.client.1.smithi053.stdout:6/946: mknod d2/d1d/c12a 0 2024-09-21T04:35:22.222 INFO:tasks.workunit.client.1.smithi053.stdout:6/947: stat d2/d1b/d1e/d2a/f99 0 2024-09-21T04:35:22.223 INFO:tasks.workunit.client.1.smithi053.stdout:3/977: rename d16/d17/d21/d8b/d7d/dc6/l129 to d16/d17/d21/d42/d43/d125/d12e/l161 0 2024-09-21T04:35:22.225 INFO:tasks.workunit.client.0.smithi007.stdout:9/340: read - d13/d5a/f61 zero size 2024-09-21T04:35:22.225 INFO:tasks.workunit.client.0.smithi007.stdout:9/341: dread - d13/d18/f55 zero size 2024-09-21T04:35:22.226 INFO:tasks.workunit.client.0.smithi007.stdout:9/342: chown d13/d2a/f3d 52267 1 2024-09-21T04:35:22.226 INFO:tasks.workunit.client.1.smithi053.stdout:1/999: link d7/d9/d43/d54/d7f/f126 d7/d47/d79/dc1/f13f 0 2024-09-21T04:35:22.226 INFO:tasks.workunit.client.1.smithi053.stdout:7/846: dread d0/d6f/dbf/d5c/f85 [0,4194304] 0 2024-09-21T04:35:22.226 INFO:tasks.workunit.client.1.smithi053.stdout:6/948: symlink d2/d1b/d3d/l12b 0 2024-09-21T04:35:22.226 INFO:tasks.workunit.client.1.smithi053.stdout:5/996: mknod d0/d8/db/d68/d10a/c145 0 2024-09-21T04:35:22.227 INFO:tasks.workunit.client.1.smithi053.stdout:3/978: chown d16/d17/d21/cc4 6 1 2024-09-21T04:35:22.228 INFO:tasks.workunit.client.1.smithi053.stdout:7/847: read - d0/d6f/dbf/d50/dc7/fef zero size 2024-09-21T04:35:22.228 INFO:tasks.workunit.client.1.smithi053.stdout:3/979: readlink d16/d1b/l2c 0 2024-09-21T04:35:22.229 INFO:tasks.workunit.client.1.smithi053.stdout:5/997: mknod d0/d8/db/d68/c146 0 2024-09-21T04:35:22.232 INFO:tasks.workunit.client.1.smithi053.stdout:0/911: sync 2024-09-21T04:35:22.232 INFO:tasks.workunit.client.1.smithi053.stdout:5/998: fdatasync d0/d8/db6/dbc/df4/d119/fec 0 2024-09-21T04:35:22.236 INFO:tasks.workunit.client.1.smithi053.stdout:7/848: dwrite d0/d6f/dbf/d50/d8b/f31 [0,4194304] 0 2024-09-21T04:35:22.237 INFO:tasks.workunit.client.1.smithi053.stdout:7/849: fdatasync d0/f13 0 2024-09-21T04:35:22.237 INFO:tasks.workunit.client.1.smithi053.stdout:3/980: link d16/d17/d21/d8b/d7d/d57/dc1/d53/d82/fbf d16/d17/d21/d26/d2b/dbb/f162 0 2024-09-21T04:35:22.238 INFO:tasks.workunit.client.1.smithi053.stdout:7/850: dread - d0/d2b/d36/d64/fb7 zero size 2024-09-21T04:35:22.239 INFO:tasks.workunit.client.1.smithi053.stdout:0/912: mknod d5/d10/d4c/db1/dbc/dca/d21/d101/d11a/c133 0 2024-09-21T04:35:22.240 INFO:tasks.workunit.client.1.smithi053.stdout:0/913: fsync d5/de/d9e/faf 0 2024-09-21T04:35:22.242 INFO:tasks.workunit.client.1.smithi053.stdout:3/981: chown d16/d17/d21/d8b/d7d/d57/dc1/d53/c9e 161892155 1 2024-09-21T04:35:22.243 INFO:tasks.workunit.client.1.smithi053.stdout:6/949: truncate d2/d1b/d3d/f6c 2090055 0 2024-09-21T04:35:22.245 INFO:tasks.workunit.client.1.smithi053.stdout:7/851: dread d0/d24/d46/f4a [4194304,4194304] 0 2024-09-21T04:35:22.247 INFO:tasks.workunit.client.1.smithi053.stdout:0/914: rename d5/dd/d45/f6c to d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/d12e/d103/d11e/de5/f134 0 2024-09-21T04:35:22.249 INFO:tasks.workunit.client.0.smithi007.stdout:9/343: dread d13/d18/d60/f62 [0,4194304] 0 2024-09-21T04:35:22.250 INFO:tasks.workunit.client.1.smithi053.stdout:7/852: truncate d0/d6f/dbf/d50/d8b/f10a 962094 0 2024-09-21T04:35:22.250 INFO:tasks.workunit.client.1.smithi053.stdout:3/982: dwrite d16/d17/f31 [4194304,4194304] 0 2024-09-21T04:35:22.250 INFO:tasks.workunit.client.0.smithi007.stdout:9/344: chown d13/d18/f68 6010 1 2024-09-21T04:35:22.251 INFO:tasks.workunit.client.1.smithi053.stdout:0/915: mkdir d5/de/d1c/d135 0 2024-09-21T04:35:22.252 INFO:tasks.workunit.client.1.smithi053.stdout:3/983: dread - d16/d17/d21/d8b/d7d/dc6/dec/f153 zero size 2024-09-21T04:35:22.260 INFO:tasks.workunit.client.1.smithi053.stdout:0/916: dwrite d5/d10/d4c/db1/dbc/dca/d53/f128 [0,4194304] 0 2024-09-21T04:35:22.262 INFO:tasks.workunit.client.1.smithi053.stdout:0/917: write d5/d10/d4c/db1/dbc/dca/f129 [920429,87055] 0 2024-09-21T04:35:22.263 INFO:tasks.workunit.client.0.smithi007.stdout:9/345: creat d13/f6b x:0 0 0 2024-09-21T04:35:22.264 INFO:tasks.workunit.client.0.smithi007.stdout:9/346: readlink d13/d18/l2e 0 2024-09-21T04:35:22.265 INFO:tasks.workunit.client.0.smithi007.stdout:9/347: mknod d13/d2a/c6c 0 2024-09-21T04:35:22.266 INFO:tasks.workunit.client.0.smithi007.stdout:9/348: chown d13/c4c 1741 1 2024-09-21T04:35:22.269 INFO:tasks.workunit.client.0.smithi007.stdout:9/349: mkdir d13/d18/d60/d6d 0 2024-09-21T04:35:22.272 INFO:tasks.workunit.client.1.smithi053.stdout:0/918: getdents d5/d10/d4c/db1/dbc/dca/d21/d101/d11a 0 2024-09-21T04:35:22.275 INFO:tasks.workunit.client.0.smithi007.stdout:9/350: link d13/d2a/f3d d13/d18/f6e 0 2024-09-21T04:35:22.276 INFO:tasks.workunit.client.1.smithi053.stdout:6/950: dwrite d2/d1b/d3d/d49/de/d9c/fbd [4194304,4194304] 0 2024-09-21T04:35:22.276 INFO:tasks.workunit.client.1.smithi053.stdout:7/853: dwrite d0/d6f/dbf/d50/d8b/d70/f88 [0,4194304] 0 2024-09-21T04:35:22.278 INFO:tasks.workunit.client.1.smithi053.stdout:6/951: chown d2/d1b/d1e/d2a/d4b 7 1 2024-09-21T04:35:22.279 INFO:tasks.workunit.client.0.smithi007.stdout:9/351: creat d13/d18/d60/d6d/f6f x:0 0 0 2024-09-21T04:35:22.280 INFO:tasks.workunit.client.0.smithi007.stdout:9/352: chown d13/d2a/f56 54392509 1 2024-09-21T04:35:22.281 INFO:tasks.workunit.client.1.smithi053.stdout:3/984: write f9 [200414,58430] 0 2024-09-21T04:35:22.283 INFO:tasks.workunit.client.1.smithi053.stdout:7/854: symlink d0/d6f/dbf/d50/d8b/d70/dc1/l11d 0 2024-09-21T04:35:22.283 INFO:tasks.workunit.client.0.smithi007.stdout:9/353: fdatasync d13/f1e 0 2024-09-21T04:35:22.284 INFO:tasks.workunit.client.0.smithi007.stdout:9/354: fdatasync d13/d18/f67 0 2024-09-21T04:35:22.290 INFO:tasks.workunit.client.1.smithi053.stdout:6/952: dwrite d2/d1b/d1e/d2a/d4b/deb/df5/f127 [0,4194304] 0 2024-09-21T04:35:22.293 INFO:tasks.workunit.client.1.smithi053.stdout:9/954: sync 2024-09-21T04:35:22.295 INFO:tasks.workunit.client.0.smithi007.stdout:9/355: truncate d13/f1e 370988 0 2024-09-21T04:35:22.296 INFO:tasks.workunit.client.0.smithi007.stdout:9/356: rename d13/d18/d60 to d13/d18/d60/d6d/d70 22 2024-09-21T04:35:22.300 INFO:tasks.workunit.client.1.smithi053.stdout:9/955: read d0/d1c/d52/d6d/d13e/f13 [7423922,341] 0 2024-09-21T04:35:22.300 INFO:tasks.workunit.client.1.smithi053.stdout:0/919: dwrite d5/f97 [4194304,4194304] 0 2024-09-21T04:35:22.300 INFO:tasks.workunit.client.0.smithi007.stdout:9/357: creat d13/d18/f71 x:0 0 0 2024-09-21T04:35:22.301 INFO:tasks.workunit.client.1.smithi053.stdout:6/953: dwrite d2/d1b/d3d/d49/de/fef [0,4194304] 0 2024-09-21T04:35:22.303 INFO:tasks.workunit.client.1.smithi053.stdout:9/956: chown d0/d1a/f2c 17906 1 2024-09-21T04:35:22.310 INFO:tasks.workunit.client.1.smithi053.stdout:7/855: dread d0/f23 [0,4194304] 0 2024-09-21T04:35:22.311 INFO:tasks.workunit.client.1.smithi053.stdout:0/920: rename d5/d10/d4c/db1/dbc/dca/d21/d96/lf7 to d5/de/d9e/dd2/l136 0 2024-09-21T04:35:22.312 INFO:tasks.workunit.client.0.smithi007.stdout:9/358: write d13/d18/f35 [4422818,76346] 0 2024-09-21T04:35:22.315 INFO:tasks.workunit.client.1.smithi053.stdout:6/954: dwrite d2/d1b/d1e/d27/d41/f128 [0,4194304] 0 2024-09-21T04:35:22.320 INFO:tasks.workunit.client.1.smithi053.stdout:6/955: creat d2/d1b/d1e/d27/d90/f12c x:0 0 0 2024-09-21T04:35:22.324 INFO:tasks.workunit.client.1.smithi053.stdout:7/856: dwrite d0/d2b/d36/f89 [0,4194304] 0 2024-09-21T04:35:22.324 INFO:tasks.workunit.client.1.smithi053.stdout:0/921: link d5/dd/d50/c57 d5/dd/d45/c137 0 2024-09-21T04:35:22.324 INFO:tasks.workunit.client.1.smithi053.stdout:6/956: write d2/d1b/d1e/d27/d41/f10b [556372,57541] 0 2024-09-21T04:35:22.325 INFO:tasks.workunit.client.0.smithi007.stdout:9/359: truncate d13/f23 4034690 0 2024-09-21T04:35:22.327 INFO:tasks.workunit.client.1.smithi053.stdout:7/857: write d0/d6f/dbf/d50/dc7/fef [588078,28433] 0 2024-09-21T04:35:22.327 INFO:tasks.workunit.client.1.smithi053.stdout:6/957: write d2/d1b/d3d/d49/d5c/d8d/fbc [2172749,35938] 0 2024-09-21T04:35:22.330 INFO:tasks.workunit.client.1.smithi053.stdout:9/957: dread d0/d1c/d52/d6d/d13e/d9/f19 [0,4194304] 0 2024-09-21T04:35:22.331 INFO:tasks.workunit.client.1.smithi053.stdout:7/858: fsync d0/d6f/dbf/d50/d8b/d70/faf 0 2024-09-21T04:35:22.333 INFO:tasks.workunit.client.0.smithi007.stdout:9/360: link d13/d2a/l57 d13/d18/d60/d6d/l72 0 2024-09-21T04:35:22.335 INFO:tasks.workunit.client.1.smithi053.stdout:7/859: readlink d0/d6f/dbf/ld4 0 2024-09-21T04:35:22.337 INFO:tasks.workunit.client.1.smithi053.stdout:7/860: write d0/d6f/dbf/d50/f96 [3863887,55735] 0 2024-09-21T04:35:22.338 INFO:tasks.workunit.client.1.smithi053.stdout:7/861: truncate d0/f44 4275611 0 2024-09-21T04:35:22.339 INFO:tasks.workunit.client.0.smithi007.stdout:9/361: dwrite fa [0,4194304] 0 2024-09-21T04:35:22.340 INFO:tasks.workunit.client.1.smithi053.stdout:9/958: rename d0/d1c/d52/d6d/d13e/d9/d34/d58/d92 to d0/d1c/d21/d8e/d145 0 2024-09-21T04:35:22.340 INFO:tasks.workunit.client.1.smithi053.stdout:0/922: write d5/d10/d4c/db1/dbc/fa7 [1487921,80313] 0 2024-09-21T04:35:22.341 INFO:tasks.workunit.client.1.smithi053.stdout:0/923: readlink d5/d47/l66 0 2024-09-21T04:35:22.344 INFO:tasks.workunit.client.1.smithi053.stdout:7/862: dread d0/d6f/dbf/d5c/f8a [0,4194304] 0 2024-09-21T04:35:22.346 INFO:tasks.workunit.client.1.smithi053.stdout:6/958: rmdir d2/d1b/d1e/d27/d41/dc9/d113 0 2024-09-21T04:35:22.348 INFO:tasks.workunit.client.1.smithi053.stdout:9/959: creat d0/d1a/d4c/d8f/f146 x:0 0 0 2024-09-21T04:35:22.349 INFO:tasks.workunit.client.0.smithi007.stdout:9/362: symlink d13/d18/l73 0 2024-09-21T04:35:22.355 INFO:tasks.workunit.client.0.smithi007.stdout:9/363: unlink d13/f28 0 2024-09-21T04:35:22.357 INFO:tasks.workunit.client.1.smithi053.stdout:7/863: rename d0/d6f/dbf/d5d/l93 to d0/d6f/dbf/d50/d8b/d70/df2/l11e 0 2024-09-21T04:35:22.358 INFO:tasks.workunit.client.1.smithi053.stdout:7/864: chown d0/c116 48 1 2024-09-21T04:35:22.359 INFO:tasks.workunit.client.1.smithi053.stdout:9/960: mkdir d0/d1c/d52/d6d/d13e/d4e/d114/d147 0 2024-09-21T04:35:22.360 INFO:tasks.workunit.client.0.smithi007.stdout:9/364: mkdir d13/d74 0 2024-09-21T04:35:22.361 INFO:tasks.workunit.client.0.smithi007.stdout:9/365: chown d13/d18/d60/d6d/f6f 515346 1 2024-09-21T04:35:22.362 INFO:tasks.workunit.client.1.smithi053.stdout:6/959: getdents d2/d1d 0 2024-09-21T04:35:22.363 INFO:tasks.workunit.client.1.smithi053.stdout:6/960: chown d2/d1d/de0/fa6 3451 1 2024-09-21T04:35:22.364 INFO:tasks.workunit.client.0.smithi007.stdout:9/366: truncate f10 913921 0 2024-09-21T04:35:22.365 INFO:tasks.workunit.client.1.smithi053.stdout:6/961: creat d2/d1b/d3d/d49/d5c/d8d/f12d x:0 0 0 2024-09-21T04:35:22.366 INFO:tasks.workunit.client.0.smithi007.stdout:9/367: mkdir d13/d18/d60/d6d/d75 0 2024-09-21T04:35:22.368 INFO:tasks.workunit.client.0.smithi007.stdout:9/368: fsync d13/d2a/f3d 0 2024-09-21T04:35:22.372 INFO:tasks.workunit.client.1.smithi053.stdout:6/962: dwrite d2/d1b/d3d/d49/d5c/f74 [0,4194304] 0 2024-09-21T04:35:22.373 INFO:tasks.workunit.client.1.smithi053.stdout:9/961: dread d0/d1c/d52/d6d/d13e/d9/d34/f56 [0,4194304] 0 2024-09-21T04:35:22.376 INFO:tasks.workunit.client.1.smithi053.stdout:9/962: symlink d0/d1c/d21/l148 0 2024-09-21T04:35:22.376 INFO:tasks.workunit.client.1.smithi053.stdout:7/865: write d0/f59 [1532033,17217] 0 2024-09-21T04:35:22.378 INFO:tasks.workunit.client.1.smithi053.stdout:6/963: truncate d2/d1b/d1e/d27/d90/fd3 2895585 0 2024-09-21T04:35:22.380 INFO:tasks.workunit.client.1.smithi053.stdout:7/866: mkdir d0/d6f/dbf/d50/d8b/d70/df2/d11f 0 2024-09-21T04:35:22.383 INFO:tasks.workunit.client.1.smithi053.stdout:6/964: mkdir d2/d1d/dbb/d12e 0 2024-09-21T04:35:22.383 INFO:tasks.workunit.client.1.smithi053.stdout:7/867: symlink d0/dcd/l120 0 2024-09-21T04:35:22.384 INFO:tasks.workunit.client.0.smithi007.stdout:9/369: write d13/d18/f45 [233922,107511] 0 2024-09-21T04:35:22.384 INFO:tasks.workunit.client.1.smithi053.stdout:6/965: chown d2/d1b/d1e/d2a 0 1 2024-09-21T04:35:22.384 INFO:tasks.workunit.client.1.smithi053.stdout:9/963: dread d0/d1c/d52/daa/fbc [0,4194304] 0 2024-09-21T04:35:22.385 INFO:tasks.workunit.client.1.smithi053.stdout:7/868: stat d0/d6f/dbf/d5d/d9f/fa6 0 2024-09-21T04:35:22.386 INFO:tasks.workunit.client.0.smithi007.stdout:9/370: fdatasync d13/d18/d60/f63 0 2024-09-21T04:35:22.387 INFO:tasks.workunit.client.1.smithi053.stdout:7/869: read d0/d24/d46/fe3 [1097046,127067] 0 2024-09-21T04:35:22.388 INFO:tasks.workunit.client.0.smithi007.stdout:9/371: write d13/d18/f45 [1279517,30048] 0 2024-09-21T04:35:22.388 INFO:tasks.workunit.client.1.smithi053.stdout:6/966: mknod d2/d1b/d1e/d27/d41/dc9/c12f 0 2024-09-21T04:35:22.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/372: symlink d13/d18/d60/l76 0 2024-09-21T04:35:22.392 INFO:tasks.workunit.client.1.smithi053.stdout:6/967: symlink d2/d1b/d3d/d49/d5c/d8d/l130 0 2024-09-21T04:35:22.393 INFO:tasks.workunit.client.1.smithi053.stdout:6/968: fdatasync d2/d1b/d1e/d2a/d4b/d77/f100 0 2024-09-21T04:35:22.395 INFO:tasks.workunit.client.1.smithi053.stdout:6/969: write d2/d1b/d1e/dc1/f124 [721923,43268] 0 2024-09-21T04:35:22.396 INFO:tasks.workunit.client.0.smithi007.stdout:9/373: rename d13/c14 to d13/d18/c77 0 2024-09-21T04:35:22.398 INFO:tasks.workunit.client.0.smithi007.stdout:9/374: dread - d13/d18/f68 zero size 2024-09-21T04:35:22.399 INFO:tasks.workunit.client.1.smithi053.stdout:9/964: dwrite d0/d1c/d52/f93 [0,4194304] 0 2024-09-21T04:35:22.401 INFO:tasks.workunit.client.0.smithi007.stdout:9/375: creat d13/d74/f78 x:0 0 0 2024-09-21T04:35:22.403 INFO:tasks.workunit.client.0.smithi007.stdout:9/376: symlink d13/l79 0 2024-09-21T04:35:22.403 INFO:tasks.workunit.client.0.smithi007.stdout:9/377: chown d13/d2a/f56 0 1 2024-09-21T04:35:22.404 INFO:tasks.workunit.client.0.smithi007.stdout:9/378: dread - d13/d74/f78 zero size 2024-09-21T04:35:22.404 INFO:tasks.workunit.client.1.smithi053.stdout:6/970: write d2/d1b/d3d/d49/de/d32/d80/f61 [2075,88092] 0 2024-09-21T04:35:22.406 INFO:tasks.workunit.client.0.smithi007.stdout:9/379: symlink d13/d2a/l7a 0 2024-09-21T04:35:22.408 INFO:tasks.workunit.client.1.smithi053.stdout:9/965: dwrite d0/d76/d79/f131 [0,4194304] 0 2024-09-21T04:35:22.411 INFO:tasks.workunit.client.0.smithi007.stdout:9/380: dread d13/d18/f45 [0,4194304] 0 2024-09-21T04:35:22.413 INFO:tasks.workunit.client.1.smithi053.stdout:6/971: creat d2/d1b/d1e/d27/f131 x:0 0 0 2024-09-21T04:35:22.413 INFO:tasks.workunit.client.1.smithi053.stdout:5/999: sync 2024-09-21T04:35:22.415 INFO:tasks.workunit.client.1.smithi053.stdout:6/972: mknod d2/d1b/d3d/d49/c132 0 2024-09-21T04:35:22.418 INFO:tasks.workunit.client.1.smithi053.stdout:6/973: mkdir d2/d1d/dbb/d12e/d133 0 2024-09-21T04:35:22.419 INFO:tasks.workunit.client.1.smithi053.stdout:6/974: mknod d2/d1d/c134 0 2024-09-21T04:35:22.420 INFO:tasks.workunit.client.0.smithi007.stdout:9/381: write d13/d2a/f3d [3396607,59157] 0 2024-09-21T04:35:22.421 INFO:tasks.workunit.client.1.smithi053.stdout:6/975: mkdir d2/d1b/d1e/d27/d41/dc9/d135 0 2024-09-21T04:35:22.423 INFO:tasks.workunit.client.0.smithi007.stdout:9/382: rename d13/d2a/c49 to d13/d74/c7b 0 2024-09-21T04:35:22.425 INFO:tasks.workunit.client.1.smithi053.stdout:6/976: readlink d2/d1b/d3d/d49/de/l70 0 2024-09-21T04:35:22.425 INFO:tasks.workunit.client.1.smithi053.stdout:9/966: dwrite d0/fc3 [0,4194304] 0 2024-09-21T04:35:22.425 INFO:tasks.workunit.client.0.smithi007.stdout:9/383: mknod d13/d5a/c7c 0 2024-09-21T04:35:22.427 INFO:tasks.workunit.client.1.smithi053.stdout:6/977: mkdir d2/d120/d136 0 2024-09-21T04:35:22.427 INFO:tasks.workunit.client.1.smithi053.stdout:6/978: dread - d2/d1b/d1e/d2a/f122 zero size 2024-09-21T04:35:22.430 INFO:tasks.workunit.client.0.smithi007.stdout:9/384: rename d13/d2a/f31 to d13/d18/d60/f7d 0 2024-09-21T04:35:22.438 INFO:tasks.workunit.client.1.smithi053.stdout:6/979: fdatasync d2/d1b/d1e/d2a/d4b/d77/d88/fab 0 2024-09-21T04:35:22.442 INFO:tasks.workunit.client.1.smithi053.stdout:9/967: truncate d0/d1c/fb0 1834755 0 2024-09-21T04:35:22.443 INFO:tasks.workunit.client.0.smithi007.stdout:9/385: truncate d13/d18/f6e 296048 0 2024-09-21T04:35:22.444 INFO:tasks.workunit.client.1.smithi053.stdout:9/968: symlink d0/d1c/d52/d6d/d13e/d9/de7/d133/l149 0 2024-09-21T04:35:22.445 INFO:tasks.workunit.client.1.smithi053.stdout:6/980: dread d2/d1b/d1e/d2a/d4b/f58 [0,4194304] 0 2024-09-21T04:35:22.446 INFO:tasks.workunit.client.1.smithi053.stdout:9/969: creat d0/d6c/f14a x:0 0 0 2024-09-21T04:35:22.446 INFO:tasks.workunit.client.1.smithi053.stdout:6/981: dread - d2/d1b/d1e/dc1/f126 zero size 2024-09-21T04:35:22.447 INFO:tasks.workunit.client.1.smithi053.stdout:6/982: readlink d2/d1b/d3d/d49/de/l98 0 2024-09-21T04:35:22.447 INFO:tasks.workunit.client.0.smithi007.stdout:9/386: rename d13/d18/d60/f62 to d13/d18/d60/d6d/d75/f7e 0 2024-09-21T04:35:22.449 INFO:tasks.workunit.client.0.smithi007.stdout:9/387: mkdir d13/d18/d60/d6d/d75/d7f 0 2024-09-21T04:35:22.449 INFO:tasks.workunit.client.0.smithi007.stdout:9/388: fsync fa 0 2024-09-21T04:35:22.450 INFO:tasks.workunit.client.1.smithi053.stdout:6/983: creat d2/d1b/d3d/d49/de/d32/f137 x:0 0 0 2024-09-21T04:35:22.451 INFO:tasks.workunit.client.1.smithi053.stdout:9/970: link d0/d1c/d52/d6d/d13e/d9/d34/d58/c60 d0/d1c/d52/db5/df8/c14b 0 2024-09-21T04:35:22.452 INFO:tasks.workunit.client.1.smithi053.stdout:9/971: read d0/d1c/d52/f93 [2692031,52136] 0 2024-09-21T04:35:22.452 INFO:tasks.workunit.client.0.smithi007.stdout:9/389: mkdir d13/d18/d80 0 2024-09-21T04:35:22.453 INFO:tasks.workunit.client.1.smithi053.stdout:9/972: read d0/d1a/d89/f12b [132129,7205] 0 2024-09-21T04:35:22.456 INFO:tasks.workunit.client.0.smithi007.stdout:9/390: link d13/d2a/l7a d13/d74/l81 0 2024-09-21T04:35:22.462 INFO:tasks.workunit.client.1.smithi053.stdout:9/973: dwrite d0/d1c/d52/d6d/f87 [0,4194304] 0 2024-09-21T04:35:22.462 INFO:tasks.workunit.client.0.smithi007.stdout:9/391: dread d13/f1e [0,4194304] 0 2024-09-21T04:35:22.462 INFO:tasks.workunit.client.1.smithi053.stdout:6/984: dread d2/d1b/f53 [0,4194304] 0 2024-09-21T04:35:22.464 INFO:tasks.workunit.client.1.smithi053.stdout:6/985: write d2/d1b/d1e/d2a/d4b/deb/df5/f127 [304991,74560] 0 2024-09-21T04:35:22.468 INFO:tasks.workunit.client.1.smithi053.stdout:9/974: unlink d0/d1c/d52/d6d/d13e/d9/d34/d58/c60 0 2024-09-21T04:35:22.469 INFO:tasks.workunit.client.1.smithi053.stdout:6/986: creat d2/d1b/d3d/f138 x:0 0 0 2024-09-21T04:35:22.470 INFO:tasks.workunit.client.0.smithi007.stdout:9/392: dwrite d13/d18/f71 [0,4194304] 0 2024-09-21T04:35:22.471 INFO:tasks.workunit.client.1.smithi053.stdout:9/975: fsync d0/d1c/f39 0 2024-09-21T04:35:22.472 INFO:tasks.workunit.client.1.smithi053.stdout:9/976: chown d0/d1c 13778668 1 2024-09-21T04:35:22.474 INFO:tasks.workunit.client.1.smithi053.stdout:9/977: truncate d0/d1c/d52/d6d/d13e/ddf/f134 419643 0 2024-09-21T04:35:22.474 INFO:tasks.workunit.client.0.smithi007.stdout:9/393: fdatasync d13/d18/f35 0 2024-09-21T04:35:22.474 INFO:tasks.workunit.client.1.smithi053.stdout:3/985: sync 2024-09-21T04:35:22.479 INFO:tasks.workunit.client.0.smithi007.stdout:9/394: mknod d13/d5a/c82 0 2024-09-21T04:35:22.482 INFO:tasks.workunit.client.1.smithi053.stdout:6/987: creat d2/d1b/d1e/d2a/d4b/d77/f139 x:0 0 0 2024-09-21T04:35:22.483 INFO:tasks.workunit.client.0.smithi007.stdout:9/395: mkdir d13/d18/d60/d83 0 2024-09-21T04:35:22.483 INFO:tasks.workunit.client.1.smithi053.stdout:3/986: mknod d16/d17/d21/d26/d71/c163 0 2024-09-21T04:35:22.485 INFO:tasks.workunit.client.0.smithi007.stdout:9/396: mknod d13/d5a/c84 0 2024-09-21T04:35:22.487 INFO:tasks.workunit.client.0.smithi007.stdout:9/397: dread d13/d18/f45 [0,4194304] 0 2024-09-21T04:35:22.489 INFO:tasks.workunit.client.1.smithi053.stdout:9/978: dwrite d0/d1c/d21/f9f [0,4194304] 0 2024-09-21T04:35:22.489 INFO:tasks.workunit.client.1.smithi053.stdout:6/988: fsync d2/d1d/f7e 0 2024-09-21T04:35:22.492 INFO:tasks.workunit.client.1.smithi053.stdout:9/979: chown d0/d1c/d21/c51 97237 1 2024-09-21T04:35:22.492 INFO:tasks.workunit.client.1.smithi053.stdout:3/987: dwrite d16/d17/d21/d26/d2b/dbb/f130 [0,4194304] 0 2024-09-21T04:35:22.492 INFO:tasks.workunit.client.1.smithi053.stdout:0/924: sync 2024-09-21T04:35:22.492 INFO:tasks.workunit.client.1.smithi053.stdout:7/870: sync 2024-09-21T04:35:22.494 INFO:tasks.workunit.client.0.smithi007.stdout:9/398: dread d13/d2a/f47 [0,4194304] 0 2024-09-21T04:35:22.495 INFO:tasks.workunit.client.0.smithi007.stdout:9/399: read - d13/d18/f68 zero size 2024-09-21T04:35:22.498 INFO:tasks.workunit.client.1.smithi053.stdout:9/980: creat d0/d1a/dcd/d12e/d12f/f14c x:0 0 0 2024-09-21T04:35:22.500 INFO:tasks.workunit.client.1.smithi053.stdout:6/989: truncate d2/d1b/d1e/d2a/d4b/d77/d88/da4/fcf 773628 0 2024-09-21T04:35:22.502 INFO:tasks.workunit.client.0.smithi007.stdout:9/400: truncate d13/d18/f6e 368961 0 2024-09-21T04:35:22.503 INFO:tasks.workunit.client.0.smithi007.stdout:9/401: truncate d13/d2a/f56 401618 0 2024-09-21T04:35:22.503 INFO:tasks.workunit.client.1.smithi053.stdout:6/990: readlink d2/d1b/d1e/d27/lb6 0 2024-09-21T04:35:22.504 INFO:tasks.workunit.client.1.smithi053.stdout:3/988: dwrite d16/d1b/f1f [0,4194304] 0 2024-09-21T04:35:22.504 INFO:tasks.workunit.client.1.smithi053.stdout:0/925: symlink d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/d12e/d103/d11e/l138 0 2024-09-21T04:35:22.505 INFO:tasks.workunit.client.0.smithi007.stdout:9/402: truncate d13/f21 5053340 0 2024-09-21T04:35:22.506 INFO:tasks.workunit.client.1.smithi053.stdout:7/871: dwrite d0/d6f/dbf/d50/d8b/f42 [0,4194304] 0 2024-09-21T04:35:22.508 INFO:tasks.workunit.client.1.smithi053.stdout:0/926: readlink d5/d10/d4c/db1/dbc/dca/db5/d99/le2 0 2024-09-21T04:35:22.512 INFO:tasks.workunit.client.1.smithi053.stdout:7/872: dread d0/d6f/f86 [0,4194304] 0 2024-09-21T04:35:22.513 INFO:tasks.workunit.client.1.smithi053.stdout:0/927: truncate d5/dd/f1a 1807786 0 2024-09-21T04:35:22.514 INFO:tasks.workunit.client.1.smithi053.stdout:3/989: link d16/d1b/c139 d16/d17/d21/d26/d49/c164 0 2024-09-21T04:35:22.518 INFO:tasks.workunit.client.1.smithi053.stdout:3/990: dread d16/d17/d21/d26/d3e/f62 [0,4194304] 0 2024-09-21T04:35:22.519 INFO:tasks.workunit.client.1.smithi053.stdout:0/928: rename d5/d10/d4c/db1/dbc/dca/d53/fbe to d5/de/d42/d78/f139 0 2024-09-21T04:35:22.522 INFO:tasks.workunit.client.1.smithi053.stdout:9/981: dwrite d0/d1a/d4c/fa4 [0,4194304] 0 2024-09-21T04:35:22.525 INFO:tasks.workunit.client.1.smithi053.stdout:0/929: mknod d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/c13a 0 2024-09-21T04:35:22.525 INFO:tasks.workunit.client.1.smithi053.stdout:3/991: getdents d16/d17/d21/d8b/d7d 0 2024-09-21T04:35:22.529 INFO:tasks.workunit.client.1.smithi053.stdout:3/992: unlink d16/d17/d21/d8b/d7d/d57/dc1/d53/l8f 0 2024-09-21T04:35:22.530 INFO:tasks.workunit.client.1.smithi053.stdout:3/993: dread - d16/d17/d21/d8b/d7d/f88 zero size 2024-09-21T04:35:22.530 INFO:tasks.workunit.client.1.smithi053.stdout:0/930: truncate d5/d10/d4c/db1/dbc/dca/f61 3723176 0 2024-09-21T04:35:22.531 INFO:tasks.workunit.client.1.smithi053.stdout:9/982: truncate d0/d1c/d52/d6d/d13e/d9/d1f/f26 2321179 0 2024-09-21T04:35:22.532 INFO:tasks.workunit.client.1.smithi053.stdout:3/994: rmdir d16/d17/d21/d8b/d7d/d57/d90/dfe 39 2024-09-21T04:35:22.533 INFO:tasks.workunit.client.1.smithi053.stdout:0/931: symlink d5/de/d1c/da3/l13b 0 2024-09-21T04:35:22.534 INFO:tasks.workunit.client.1.smithi053.stdout:0/932: chown d5/de/d42/f48 3 1 2024-09-21T04:35:22.535 INFO:tasks.workunit.client.1.smithi053.stdout:3/995: symlink d16/d17/d21/d8b/l165 0 2024-09-21T04:35:22.535 INFO:tasks.workunit.client.1.smithi053.stdout:3/996: chown d16/d1b/dc3 5 1 2024-09-21T04:35:22.536 INFO:tasks.workunit.client.1.smithi053.stdout:9/983: mknod d0/d1c/d52/db5/df8/c14d 0 2024-09-21T04:35:22.541 INFO:tasks.workunit.client.1.smithi053.stdout:9/984: creat d0/d1a/dcd/f14e x:0 0 0 2024-09-21T04:35:22.541 INFO:tasks.workunit.client.1.smithi053.stdout:9/985: chown d0 563901 1 2024-09-21T04:35:22.544 INFO:tasks.workunit.client.1.smithi053.stdout:0/933: write d5/d10/d4c/db1/dbc/dca/d53/fa9 [2024417,100243] 0 2024-09-21T04:35:22.545 INFO:tasks.workunit.client.1.smithi053.stdout:3/997: write d16/d17/d21/d8b/d7d/f88 [307719,44971] 0 2024-09-21T04:35:22.548 INFO:tasks.workunit.client.1.smithi053.stdout:9/986: dwrite d0/d1c/d52/daa/ffa [0,4194304] 0 2024-09-21T04:35:22.552 INFO:tasks.workunit.client.1.smithi053.stdout:0/934: truncate d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/d12e/d103/f104 96933 0 2024-09-21T04:35:22.555 INFO:tasks.workunit.client.1.smithi053.stdout:0/935: write d5/d10/d4c/db1/dbc/dca/d21/d96/f98 [7775215,30133] 0 2024-09-21T04:35:22.556 INFO:tasks.workunit.client.1.smithi053.stdout:9/987: truncate d0/d1c/d52/d6d/d13e/d9/fb 2316088 0 2024-09-21T04:35:22.557 INFO:tasks.workunit.client.1.smithi053.stdout:3/998: dwrite d16/d17/d21/f64 [0,4194304] 0 2024-09-21T04:35:22.559 INFO:tasks.workunit.client.1.smithi053.stdout:3/999: chown d16/d17/d21/d26/f102 1 1 2024-09-21T04:35:22.563 INFO:tasks.workunit.client.1.smithi053.stdout:9/988: rename d0/d1c/d52/db5/df8 to d0/d1c/d52/d6d/d13e/d4e/d12c/d14f 0 2024-09-21T04:35:22.565 INFO:tasks.workunit.client.1.smithi053.stdout:9/989: mkdir d0/d150 0 2024-09-21T04:35:22.568 INFO:tasks.workunit.client.1.smithi053.stdout:9/990: symlink d0/d1c/d52/d6d/d13e/d4e/d114/d147/l151 0 2024-09-21T04:35:22.570 INFO:tasks.workunit.client.1.smithi053.stdout:9/991: chown d0/d76/f7e 148531 1 2024-09-21T04:35:22.571 INFO:tasks.workunit.client.1.smithi053.stdout:0/936: dwrite d5/dd/d50/f6e [0,4194304] 0 2024-09-21T04:35:22.572 INFO:tasks.workunit.client.1.smithi053.stdout:9/992: rmdir d0/d1c/d52/d6f/dfb 39 2024-09-21T04:35:22.578 INFO:tasks.workunit.client.1.smithi053.stdout:9/993: getdents d0/d1a/dcd 0 2024-09-21T04:35:22.580 INFO:tasks.workunit.client.1.smithi053.stdout:9/994: dread - d0/d1c/d52/d6f/dfb/f113 zero size 2024-09-21T04:35:22.586 INFO:tasks.workunit.client.1.smithi053.stdout:0/937: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/fbd [0,4194304] 0 2024-09-21T04:35:22.586 INFO:tasks.workunit.client.1.smithi053.stdout:9/995: dread d0/d1a/f2c [8388608,4194304] 0 2024-09-21T04:35:22.586 INFO:tasks.workunit.client.1.smithi053.stdout:0/938: chown d5/dd/d45 98900 1 2024-09-21T04:35:22.588 INFO:tasks.workunit.client.1.smithi053.stdout:0/939: write d5/d10/d4c/db1/fe9 [184090,58059] 0 2024-09-21T04:35:22.590 INFO:tasks.workunit.client.1.smithi053.stdout:9/996: symlink d0/d1c/d52/d6d/d13e/d9/de7/d133/l152 0 2024-09-21T04:35:22.593 INFO:tasks.workunit.client.0.smithi007.stdout:8/190: sync 2024-09-21T04:35:22.596 INFO:tasks.workunit.client.1.smithi053.stdout:9/997: dwrite d0/d76/fc9 [0,4194304] 0 2024-09-21T04:35:22.597 INFO:tasks.workunit.client.0.smithi007.stdout:8/191: creat d0/d3/d16/f31 x:0 0 0 2024-09-21T04:35:22.597 INFO:tasks.workunit.client.0.smithi007.stdout:8/192: fdatasync d0/d3/d8/fa 0 2024-09-21T04:35:22.598 INFO:tasks.workunit.client.1.smithi053.stdout:9/998: rmdir d0/d1c 39 2024-09-21T04:35:22.598 INFO:tasks.workunit.client.0.smithi007.stdout:8/193: readlink d0/d3/d16/l30 0 2024-09-21T04:35:22.599 INFO:tasks.workunit.client.0.smithi007.stdout:8/194: stat d0/d3/l22 0 2024-09-21T04:35:22.602 INFO:tasks.workunit.client.1.smithi053.stdout:0/940: truncate d5/de/d42/d78/f139 1161565 0 2024-09-21T04:35:22.602 INFO:tasks.workunit.client.1.smithi053.stdout:9/999: fdatasync d0/d1c/d52/daa/fbc 0 2024-09-21T04:35:22.602 INFO:tasks.workunit.client.0.smithi007.stdout:8/195: fsync d0/d3/d8/de/f18 0 2024-09-21T04:35:22.605 INFO:tasks.workunit.client.1.smithi053.stdout:0/941: creat d5/d10/d4c/db1/dbc/dca/d21/d96/f13c x:0 0 0 2024-09-21T04:35:22.607 INFO:tasks.workunit.client.0.smithi007.stdout:8/196: mknod d0/d3/d8/de/d26/c32 0 2024-09-21T04:35:22.607 INFO:tasks.workunit.client.1.smithi053.stdout:0/942: read - d5/de/f122 zero size 2024-09-21T04:35:22.608 INFO:tasks.workunit.client.0.smithi007.stdout:8/197: write d0/d3/d8/de/f17 [1159227,93895] 0 2024-09-21T04:35:22.618 INFO:tasks.workunit.client.1.smithi053.stdout:0/943: dread d5/de/f3e [0,4194304] 0 2024-09-21T04:35:22.620 INFO:tasks.workunit.client.0.smithi007.stdout:8/198: dread d0/d3/d8/f9 [4194304,4194304] 0 2024-09-21T04:35:22.621 INFO:tasks.workunit.client.1.smithi053.stdout:0/944: creat d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/d12e/d103/f13d x:0 0 0 2024-09-21T04:35:22.624 INFO:tasks.workunit.client.0.smithi007.stdout:8/199: dread d0/d3/d16/f21 [4194304,4194304] 0 2024-09-21T04:35:22.626 INFO:tasks.workunit.client.1.smithi053.stdout:6/991: sync 2024-09-21T04:35:22.626 INFO:tasks.workunit.client.1.smithi053.stdout:7/873: sync 2024-09-21T04:35:22.628 INFO:tasks.workunit.client.1.smithi053.stdout:6/992: fdatasync d2/f8 0 2024-09-21T04:35:22.629 INFO:tasks.workunit.client.1.smithi053.stdout:0/945: dwrite d5/de/ffa [0,4194304] 0 2024-09-21T04:35:22.630 INFO:tasks.workunit.client.1.smithi053.stdout:7/874: chown d0/d6f/dbf/d5d/cd5 202 1 2024-09-21T04:35:22.630 INFO:tasks.workunit.client.0.smithi007.stdout:8/200: dwrite d0/f20 [4194304,4194304] 0 2024-09-21T04:35:22.631 INFO:tasks.workunit.client.1.smithi053.stdout:7/875: fsync d0/d6f/dbf/d50/fa5 0 2024-09-21T04:35:22.632 INFO:tasks.workunit.client.0.smithi007.stdout:8/201: readlink d0/d3/d16/l30 0 2024-09-21T04:35:22.633 INFO:tasks.workunit.client.1.smithi053.stdout:6/993: mknod d2/d1d/de0/c13a 0 2024-09-21T04:35:22.633 INFO:tasks.workunit.client.1.smithi053.stdout:0/946: symlink d5/de/d42/d78/l13e 0 2024-09-21T04:35:22.635 INFO:tasks.workunit.client.1.smithi053.stdout:7/876: rmdir d0/d6f/dbf/d5d/d9f 39 2024-09-21T04:35:22.635 INFO:tasks.workunit.client.1.smithi053.stdout:0/947: stat d5/dd/d50/l7c 0 2024-09-21T04:35:22.637 INFO:tasks.workunit.client.1.smithi053.stdout:7/877: mknod d0/d24/c121 0 2024-09-21T04:35:22.637 INFO:tasks.workunit.client.1.smithi053.stdout:7/878: readlink d0/l38 0 2024-09-21T04:35:22.639 INFO:tasks.workunit.client.1.smithi053.stdout:0/948: rename d5/dd/d45/c76 to d5/d10/d4c/db1/dbc/dca/d21/d96/d9f/c13f 0 2024-09-21T04:35:22.642 INFO:tasks.workunit.client.1.smithi053.stdout:0/949: chown d5/d10/d4c/db1/dbc/dca/d21/d96/lae 227101563 1 2024-09-21T04:35:22.650 INFO:tasks.workunit.client.0.smithi007.stdout:8/202: write d0/d3/d16/f1b [908746,69709] 0 2024-09-21T04:35:22.651 INFO:tasks.workunit.client.0.smithi007.stdout:8/203: truncate d0/d3/f2b 19872 0 2024-09-21T04:35:22.653 INFO:tasks.workunit.client.1.smithi053.stdout:7/879: dwrite d0/d6f/dbf/d50/d8b/ff [0,4194304] 0 2024-09-21T04:35:22.653 INFO:tasks.workunit.client.1.smithi053.stdout:0/950: chown d5/de/d9e/dd2/cff 2807 1 2024-09-21T04:35:22.654 INFO:tasks.workunit.client.1.smithi053.stdout:7/880: chown d0/d6f/dbf/d5c/f8a 983486060 1 2024-09-21T04:35:22.655 INFO:tasks.workunit.client.1.smithi053.stdout:0/951: chown d5/d10/d4c/db1/dbc/dca/db5/d99/f127 3356 1 2024-09-21T04:35:22.655 INFO:tasks.workunit.client.0.smithi007.stdout:8/204: truncate d0/d3/d16/f21 5875283 0 2024-09-21T04:35:22.656 INFO:tasks.workunit.client.0.smithi007.stdout:8/205: chown d0/d3/d8/f10 4864 1 2024-09-21T04:35:22.657 INFO:tasks.workunit.client.1.smithi053.stdout:7/881: symlink d0/d2b/d36/d64/dc0/l122 0 2024-09-21T04:35:22.657 INFO:tasks.workunit.client.1.smithi053.stdout:7/882: stat d0/f59 0 2024-09-21T04:35:22.657 INFO:tasks.workunit.client.1.smithi053.stdout:0/952: rmdir d5/d10/d4c/db1/dbc/dca/d21/d96 39 2024-09-21T04:35:22.658 INFO:tasks.workunit.client.0.smithi007.stdout:8/206: mknod d0/d3/d16/c33 0 2024-09-21T04:35:22.658 INFO:tasks.workunit.client.0.smithi007.stdout:8/207: stat d0/d3/d8/l2f 0 2024-09-21T04:35:22.659 INFO:tasks.workunit.client.1.smithi053.stdout:7/883: fsync d0/d2b/d36/f3c 0 2024-09-21T04:35:22.660 INFO:tasks.workunit.client.1.smithi053.stdout:0/953: truncate d5/de/f5b 2424288 0 2024-09-21T04:35:22.660 INFO:tasks.workunit.client.0.smithi007.stdout:8/208: dread d0/d3/f2b [0,4194304] 0 2024-09-21T04:35:22.665 INFO:tasks.workunit.client.0.smithi007.stdout:8/209: getdents d0/d3/d8 0 2024-09-21T04:35:22.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/210: fsync d0/d3/d8/f10 0 2024-09-21T04:35:22.672 INFO:tasks.workunit.client.1.smithi053.stdout:7/884: write d0/d2b/d36/fa9 [739593,129664] 0 2024-09-21T04:35:22.673 INFO:tasks.workunit.client.1.smithi053.stdout:0/954: truncate d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/d12e/fe6 3787480 0 2024-09-21T04:35:22.675 INFO:tasks.workunit.client.1.smithi053.stdout:7/885: truncate d0/f23 1000506 0 2024-09-21T04:35:22.675 INFO:tasks.workunit.client.0.smithi007.stdout:8/211: write d0/d3/d16/f21 [4530244,113056] 0 2024-09-21T04:35:22.677 INFO:tasks.workunit.client.1.smithi053.stdout:7/886: truncate d0/f61 5141154 0 2024-09-21T04:35:22.685 INFO:tasks.workunit.client.1.smithi053.stdout:7/887: dwrite d0/d6f/dbf/d50/fa5 [0,4194304] 0 2024-09-21T04:35:22.687 INFO:tasks.workunit.client.1.smithi053.stdout:0/955: dwrite d5/fa [0,4194304] 0 2024-09-21T04:35:22.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:22 smithi053 ceph-mon[35554]: pgmap v32: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 94 MiB/s rd, 294 MiB/s wr, 554 op/s 2024-09-21T04:35:22.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:22 smithi007 ceph-mon[27954]: pgmap v32: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 94 MiB/s rd, 294 MiB/s wr, 554 op/s 2024-09-21T04:35:22.695 INFO:tasks.workunit.client.1.smithi053.stdout:0/956: dwrite d5/d47/f107 [0,4194304] 0 2024-09-21T04:35:22.696 INFO:tasks.workunit.client.1.smithi053.stdout:0/957: fdatasync d5/f8d 0 2024-09-21T04:35:22.697 INFO:tasks.workunit.client.1.smithi053.stdout:0/958: chown d5/de/l8f 134 1 2024-09-21T04:35:22.743 INFO:tasks.workunit.client.1.smithi053.stdout:6/994: sync 2024-09-21T04:35:22.746 INFO:tasks.workunit.client.1.smithi053.stdout:7/888: rename d0/d6f/dbf/da2/c10e to d0/d2b/c123 0 2024-09-21T04:35:22.748 INFO:tasks.workunit.client.1.smithi053.stdout:6/995: symlink d2/d1b/d3d/d104/l13b 0 2024-09-21T04:35:22.750 INFO:tasks.workunit.client.1.smithi053.stdout:7/889: unlink d0/d2b/d36/d115/f118 0 2024-09-21T04:35:22.750 INFO:tasks.workunit.client.1.smithi053.stdout:0/959: dwrite d5/de/f3e [0,4194304] 0 2024-09-21T04:35:22.750 INFO:tasks.workunit.client.1.smithi053.stdout:6/996: read d2/d1b/d3d/d49/de/d32/d80/d2e/d5b/d89/f8a [2530670,101175] 0 2024-09-21T04:35:22.751 INFO:tasks.workunit.client.1.smithi053.stdout:7/890: chown d0/c116 3530128 1 2024-09-21T04:35:22.751 INFO:tasks.workunit.client.1.smithi053.stdout:6/997: stat d2/d1b/d1e/d27/d41 0 2024-09-21T04:35:22.753 INFO:tasks.workunit.client.1.smithi053.stdout:7/891: truncate d0/d2b/f43 2424379 0 2024-09-21T04:35:22.758 INFO:tasks.workunit.client.1.smithi053.stdout:0/960: dwrite d5/de/f3e [0,4194304] 0 2024-09-21T04:35:22.763 INFO:tasks.workunit.client.1.smithi053.stdout:7/892: getdents d0/d6f/dbf/da2 0 2024-09-21T04:35:22.766 INFO:tasks.workunit.client.1.smithi053.stdout:0/961: dread d5/d10/d4c/db1/dbc/dca/f129 [0,4194304] 0 2024-09-21T04:35:22.768 INFO:tasks.workunit.client.1.smithi053.stdout:7/893: creat d0/d2b/d36/d64/ddb/f124 x:0 0 0 2024-09-21T04:35:22.769 INFO:tasks.workunit.client.1.smithi053.stdout:7/894: symlink d0/d6f/l125 0 2024-09-21T04:35:22.770 INFO:tasks.workunit.client.1.smithi053.stdout:7/895: readlink d0/l38 0 2024-09-21T04:35:22.771 INFO:tasks.workunit.client.1.smithi053.stdout:6/998: dread d2/d1b/d3d/d49/de/d96/f7b [0,4194304] 0 2024-09-21T04:35:22.772 INFO:tasks.workunit.client.1.smithi053.stdout:7/896: creat d0/dcd/f126 x:0 0 0 2024-09-21T04:35:22.773 INFO:tasks.workunit.client.1.smithi053.stdout:6/999: mknod d2/d1b/d1e/dc1/c13c 0 2024-09-21T04:35:22.774 INFO:tasks.workunit.client.1.smithi053.stdout:0/962: truncate d5/d10/d4c/db1/dbc/dca/d21/fa0 1267281 0 2024-09-21T04:35:22.776 INFO:tasks.workunit.client.1.smithi053.stdout:0/963: symlink d5/dd/d119/l140 0 2024-09-21T04:35:22.777 INFO:tasks.workunit.client.1.smithi053.stdout:0/964: write d5/dd/d50/f9d [3233985,43618] 0 2024-09-21T04:35:22.786 INFO:tasks.workunit.client.1.smithi053.stdout:7/897: dread d0/d6f/dbf/f6b [0,4194304] 0 2024-09-21T04:35:22.788 INFO:tasks.workunit.client.1.smithi053.stdout:7/898: mknod d0/d2b/c127 0 2024-09-21T04:35:22.790 INFO:tasks.workunit.client.1.smithi053.stdout:0/965: dwrite d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/d12e/fe6 [0,4194304] 0 2024-09-21T04:35:22.793 INFO:tasks.workunit.client.1.smithi053.stdout:7/899: link d0/d2b/fa8 d0/d6f/dbf/f128 0 2024-09-21T04:35:22.794 INFO:tasks.workunit.client.1.smithi053.stdout:0/966: mkdir d5/d10/d4c/db1/dbc/dca/d141 0 2024-09-21T04:35:22.795 INFO:tasks.workunit.client.1.smithi053.stdout:7/900: chown d0/d6f/dbf/d5d/d9f/c105 123 1 2024-09-21T04:35:22.795 INFO:tasks.workunit.client.1.smithi053.stdout:0/967: readlink d5/d10/d4c/db1/dbc/dca/d53/lb3 0 2024-09-21T04:35:22.797 INFO:tasks.workunit.client.1.smithi053.stdout:0/968: fsync d5/d10/d4c/db1/dbc/dca/d21/d96/dbb/f131 0 2024-09-21T04:35:22.799 INFO:tasks.workunit.client.1.smithi053.stdout:0/969: fsync d5/de/d42/fa4 0 2024-09-21T04:35:22.801 INFO:tasks.workunit.client.1.smithi053.stdout:0/970: creat d5/de/dfc/f142 x:0 0 0 2024-09-21T04:35:22.803 INFO:tasks.workunit.client.1.smithi053.stdout:7/901: dwrite d0/d6f/dbf/d5c/f85 [0,4194304] 0 2024-09-21T04:35:22.815 INFO:tasks.workunit.client.1.smithi053.stdout:0/971: dwrite d5/d10/d4c/db1/fdd [0,4194304] 0 2024-09-21T04:35:22.820 INFO:tasks.workunit.client.1.smithi053.stdout:0/972: unlink d5/d10/d4c/db1/dbc/dca/d21/f33 0 2024-09-21T04:35:22.824 INFO:tasks.workunit.client.1.smithi053.stdout:0/973: rename d5/d10/d4c/db1/dbc/dca/db5/d10b to d5/de/dfc/d143 0 2024-09-21T04:35:22.826 INFO:tasks.workunit.client.1.smithi053.stdout:0/974: mknod d5/d10/c144 0 2024-09-21T04:35:22.827 INFO:tasks.workunit.client.0.smithi007.stdout:0/306: sync 2024-09-21T04:35:22.827 INFO:tasks.workunit.client.0.smithi007.stdout:2/265: sync 2024-09-21T04:35:22.827 INFO:tasks.workunit.client.1.smithi053.stdout:0/975: write d5/de/d1c/f29 [195234,81299] 0 2024-09-21T04:35:22.828 INFO:tasks.workunit.client.1.smithi053.stdout:0/976: chown d5 1006265 1 2024-09-21T04:35:22.829 INFO:tasks.workunit.client.1.smithi053.stdout:7/902: dread d0/d6f/dbf/d50/d8b/d70/dc1/ff6 [0,4194304] 0 2024-09-21T04:35:22.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/307: symlink d6/d16/d18/d27/l70 0 2024-09-21T04:35:22.832 INFO:tasks.workunit.client.1.smithi053.stdout:7/903: creat d0/d6f/dbf/da2/f129 x:0 0 0 2024-09-21T04:35:22.832 INFO:tasks.workunit.client.1.smithi053.stdout:0/977: dread d5/d47/f107 [0,4194304] 0 2024-09-21T04:35:22.833 INFO:tasks.workunit.client.0.smithi007.stdout:0/308: dread - d6/d16/d18/d2e/f37 zero size 2024-09-21T04:35:22.833 INFO:tasks.workunit.client.1.smithi053.stdout:0/978: fdatasync d5/de/d42/f89 0 2024-09-21T04:35:22.834 INFO:tasks.workunit.client.0.smithi007.stdout:0/309: chown d6/d16/d18 0 1 2024-09-21T04:35:22.834 INFO:tasks.workunit.client.0.smithi007.stdout:2/266: dwrite d0/f19 [0,4194304] 0 2024-09-21T04:35:22.835 INFO:tasks.workunit.client.0.smithi007.stdout:2/267: chown d0/d2/f6 625 1 2024-09-21T04:35:22.836 INFO:tasks.workunit.client.0.smithi007.stdout:2/268: fsync d0/fd 0 2024-09-21T04:35:22.839 INFO:tasks.workunit.client.1.smithi053.stdout:7/904: mknod d0/d2b/d36/d64/c12a 0 2024-09-21T04:35:22.840 INFO:tasks.workunit.client.1.smithi053.stdout:0/979: read d5/de/d42/f89 [486597,111990] 0 2024-09-21T04:35:22.841 INFO:tasks.workunit.client.0.smithi007.stdout:0/310: symlink d6/d16/d18/d2e/d60/l71 0 2024-09-21T04:35:22.844 INFO:tasks.workunit.client.0.smithi007.stdout:2/269: symlink d0/d11/d31/l41 0 2024-09-21T04:35:22.846 INFO:tasks.workunit.client.0.smithi007.stdout:2/270: write d0/d2/f16 [408992,62122] 0 2024-09-21T04:35:22.847 INFO:tasks.workunit.client.0.smithi007.stdout:0/311: dwrite d6/d16/d18/d27/f29 [0,4194304] 0 2024-09-21T04:35:22.849 INFO:tasks.workunit.client.0.smithi007.stdout:2/271: chown d0/d2/l4 116 1 2024-09-21T04:35:22.850 INFO:tasks.workunit.client.0.smithi007.stdout:2/272: fdatasync d0/d11/d31/d39/f3a 0 2024-09-21T04:35:22.851 INFO:tasks.workunit.client.0.smithi007.stdout:2/273: readlink d0/d11/d31/d39/l3b 0 2024-09-21T04:35:22.853 INFO:tasks.workunit.client.0.smithi007.stdout:2/274: write d0/f18 [3266401,74692] 0 2024-09-21T04:35:22.858 INFO:tasks.workunit.client.0.smithi007.stdout:2/275: rename d0/d36 to d0/d11/d31/d42 0 2024-09-21T04:35:22.859 INFO:tasks.workunit.client.0.smithi007.stdout:2/276: write d0/d11/f28 [3010517,2115] 0 2024-09-21T04:35:22.863 INFO:tasks.workunit.client.0.smithi007.stdout:0/312: getdents d6/d16/d18/d39 0 2024-09-21T04:35:22.866 INFO:tasks.workunit.client.0.smithi007.stdout:2/277: symlink d0/d11/d31/l43 0 2024-09-21T04:35:22.868 INFO:tasks.workunit.client.0.smithi007.stdout:0/313: symlink d6/d16/d18/d27/l72 0 2024-09-21T04:35:22.871 INFO:tasks.workunit.client.0.smithi007.stdout:2/278: creat d0/d11/d31/d39/f44 x:0 0 0 2024-09-21T04:35:22.872 INFO:tasks.workunit.client.0.smithi007.stdout:0/314: mknod d6/d16/d57/c73 0 2024-09-21T04:35:22.873 INFO:tasks.workunit.client.0.smithi007.stdout:0/315: dread - d6/d16/d18/d2e/f56 zero size 2024-09-21T04:35:22.874 INFO:tasks.workunit.client.0.smithi007.stdout:2/279: mknod d0/d11/d31/c45 0 2024-09-21T04:35:22.875 INFO:tasks.workunit.client.0.smithi007.stdout:0/316: write d6/d16/d18/d2e/f4a [1570703,6746] 0 2024-09-21T04:35:22.875 INFO:tasks.workunit.client.0.smithi007.stdout:2/280: fdatasync d0/d2/f2b 0 2024-09-21T04:35:22.876 INFO:tasks.workunit.client.0.smithi007.stdout:0/317: chown d6/l15 707631 1 2024-09-21T04:35:22.876 INFO:tasks.workunit.client.0.smithi007.stdout:2/281: dread - d0/f37 zero size 2024-09-21T04:35:22.878 INFO:tasks.workunit.client.0.smithi007.stdout:0/318: truncate d6/d16/d18/d2e/f56 822213 0 2024-09-21T04:35:22.881 INFO:tasks.workunit.client.0.smithi007.stdout:2/282: mkdir d0/d11/d31/d46 0 2024-09-21T04:35:22.884 INFO:tasks.workunit.client.0.smithi007.stdout:5/566: sync 2024-09-21T04:35:22.884 INFO:tasks.workunit.client.0.smithi007.stdout:6/332: sync 2024-09-21T04:35:22.885 INFO:tasks.workunit.client.0.smithi007.stdout:0/319: dwrite d6/d16/d18/d2e/f56 [0,4194304] 0 2024-09-21T04:35:22.885 INFO:tasks.workunit.client.0.smithi007.stdout:6/333: stat d3/de/d58 0 2024-09-21T04:35:22.893 INFO:tasks.workunit.client.0.smithi007.stdout:6/334: creat d3/d3e/d46/f68 x:0 0 0 2024-09-21T04:35:22.895 INFO:tasks.workunit.client.0.smithi007.stdout:5/567: symlink d2/d27/d42/da3/d23/d5d/lb5 0 2024-09-21T04:35:22.897 INFO:tasks.workunit.client.0.smithi007.stdout:0/320: symlink d6/d16/d57/l74 0 2024-09-21T04:35:22.900 INFO:tasks.workunit.client.0.smithi007.stdout:6/335: creat d3/d59/f69 x:0 0 0 2024-09-21T04:35:22.902 INFO:tasks.workunit.client.0.smithi007.stdout:5/568: rename d2/d27/d42/da3/d23/d5d/f89 to d2/d27/d42/da3/d23/d5d/fb6 0 2024-09-21T04:35:22.906 INFO:tasks.workunit.client.0.smithi007.stdout:2/283: getdents d0/d11 0 2024-09-21T04:35:22.921 INFO:tasks.workunit.client.0.smithi007.stdout:6/336: creat d3/de/f6a x:0 0 0 2024-09-21T04:35:22.922 INFO:tasks.workunit.client.0.smithi007.stdout:6/337: write d3/d3e/f43 [1669817,94435] 0 2024-09-21T04:35:22.927 INFO:tasks.workunit.client.0.smithi007.stdout:0/321: write d6/f38 [406955,85461] 0 2024-09-21T04:35:22.928 INFO:tasks.workunit.client.0.smithi007.stdout:0/322: fsync d6/d16/d18/f69 0 2024-09-21T04:35:22.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/284: write d0/d2/f25 [18807,9959] 0 2024-09-21T04:35:22.935 INFO:tasks.workunit.client.1.smithi053.stdout:7/905: sync 2024-09-21T04:35:22.941 INFO:tasks.workunit.client.0.smithi007.stdout:0/323: write d6/fd [2740700,88309] 0 2024-09-21T04:35:22.942 INFO:tasks.workunit.client.0.smithi007.stdout:6/338: truncate d3/fc 2780598 0 2024-09-21T04:35:22.944 INFO:tasks.workunit.client.0.smithi007.stdout:0/324: mkdir d6/d75 0 2024-09-21T04:35:22.945 INFO:tasks.workunit.client.1.smithi053.stdout:0/980: creat d5/d10/d4c/db1/dbc/dca/d141/f145 x:0 0 0 2024-09-21T04:35:22.945 INFO:tasks.workunit.client.1.smithi053.stdout:0/981: readlink d5/de/d42/l54 0 2024-09-21T04:35:22.946 INFO:tasks.workunit.client.0.smithi007.stdout:6/339: mkdir d3/d59/d6b 0 2024-09-21T04:35:22.946 INFO:tasks.workunit.client.0.smithi007.stdout:6/340: stat d3/de/d12/c3d 0 2024-09-21T04:35:22.946 INFO:tasks.workunit.client.0.smithi007.stdout:0/325: mkdir d6/d16/d18/d39/d76 0 2024-09-21T04:35:22.947 INFO:tasks.workunit.client.0.smithi007.stdout:0/326: dread - d6/d16/d18/f69 zero size 2024-09-21T04:35:22.948 INFO:tasks.workunit.client.1.smithi053.stdout:0/982: mknod d5/de/d1c/da3/c146 0 2024-09-21T04:35:22.949 INFO:tasks.workunit.client.0.smithi007.stdout:2/285: truncate d0/d2/f16 2918608 0 2024-09-21T04:35:22.952 INFO:tasks.workunit.client.0.smithi007.stdout:0/327: dread d6/d16/f4d [0,4194304] 0 2024-09-21T04:35:22.955 INFO:tasks.workunit.client.0.smithi007.stdout:2/286: creat d0/d11/d31/d46/f47 x:0 0 0 2024-09-21T04:35:22.956 INFO:tasks.workunit.client.0.smithi007.stdout:2/287: write d0/f27 [54165,119985] 0 2024-09-21T04:35:22.960 INFO:tasks.workunit.client.0.smithi007.stdout:2/288: mkdir d0/d11/d31/d46/d48 0 2024-09-21T04:35:22.960 INFO:tasks.workunit.client.0.smithi007.stdout:0/328: dwrite d6/d16/d18/d2e/f4a [0,4194304] 0 2024-09-21T04:35:22.961 INFO:tasks.workunit.client.0.smithi007.stdout:6/341: write d3/de/d4e/f5d [4697670,108994] 0 2024-09-21T04:35:22.962 INFO:tasks.workunit.client.0.smithi007.stdout:2/289: dread - d0/d11/d31/d39/f3e zero size 2024-09-21T04:35:22.963 INFO:tasks.workunit.client.0.smithi007.stdout:6/342: stat d3/de/d4e/d67 0 2024-09-21T04:35:22.967 INFO:tasks.workunit.client.1.smithi053.stdout:0/983: dwrite d5/d10/d4c/db1/dbc/dca/f36 [0,4194304] 0 2024-09-21T04:35:22.968 INFO:tasks.workunit.client.0.smithi007.stdout:6/343: mknod d3/de/d4e/c6c 0 2024-09-21T04:35:22.970 INFO:tasks.workunit.client.0.smithi007.stdout:2/290: dwrite d0/d11/d31/d39/f3a [0,4194304] 0 2024-09-21T04:35:22.971 INFO:tasks.workunit.client.0.smithi007.stdout:2/291: stat d0/c10 0 2024-09-21T04:35:22.972 INFO:tasks.workunit.client.0.smithi007.stdout:2/292: read - d0/d11/d31/d46/f47 zero size 2024-09-21T04:35:22.973 INFO:tasks.workunit.client.0.smithi007.stdout:6/344: creat d3/d3e/d46/f6d x:0 0 0 2024-09-21T04:35:22.975 INFO:tasks.workunit.client.0.smithi007.stdout:0/329: getdents d6/d16/d57 0 2024-09-21T04:35:22.977 INFO:tasks.workunit.client.1.smithi053.stdout:0/984: unlink d5/de/f5b 0 2024-09-21T04:35:22.978 INFO:tasks.workunit.client.0.smithi007.stdout:6/345: creat d3/d3e/d46/d60/f6e x:0 0 0 2024-09-21T04:35:22.981 INFO:tasks.workunit.client.0.smithi007.stdout:0/330: mknod d6/d16/d18/d39/d76/c77 0 2024-09-21T04:35:22.984 INFO:tasks.workunit.client.0.smithi007.stdout:2/293: dwrite d0/d11/d31/d46/f47 [0,4194304] 0 2024-09-21T04:35:22.984 INFO:tasks.workunit.client.0.smithi007.stdout:6/346: getdents d3/de/d58 0 2024-09-21T04:35:22.985 INFO:tasks.workunit.client.0.smithi007.stdout:0/331: write d6/d16/f2a [5015507,112570] 0 2024-09-21T04:35:22.987 INFO:tasks.workunit.client.0.smithi007.stdout:6/347: write d3/de/f6a [374947,56228] 0 2024-09-21T04:35:22.988 INFO:tasks.workunit.client.1.smithi053.stdout:0/985: link d5/d47/f60 d5/d10/d4c/db1/dbc/dca/db5/f147 0 2024-09-21T04:35:22.988 INFO:tasks.workunit.client.0.smithi007.stdout:6/348: read - d3/d3e/d46/f68 zero size 2024-09-21T04:35:22.990 INFO:tasks.workunit.client.0.smithi007.stdout:6/349: fdatasync d3/d3b/f45 0 2024-09-21T04:35:22.992 INFO:tasks.workunit.client.1.smithi053.stdout:0/986: creat d5/d10/d4c/db1/dbc/dca/db5/f148 x:0 0 0 2024-09-21T04:35:22.998 INFO:tasks.workunit.client.1.smithi053.stdout:0/987: creat d5/d10/d4c/db1/dbc/dca/db5/d99/f149 x:0 0 0 2024-09-21T04:35:23.001 INFO:tasks.workunit.client.1.smithi053.stdout:7/906: sync 2024-09-21T04:35:23.002 INFO:tasks.workunit.client.0.smithi007.stdout:2/294: dread d0/d2/f2b [0,4194304] 0 2024-09-21T04:35:23.003 INFO:tasks.workunit.client.0.smithi007.stdout:6/350: link d3/c7 d3/de/d4e/d67/c6f 0 2024-09-21T04:35:23.005 INFO:tasks.workunit.client.0.smithi007.stdout:0/332: truncate d6/d16/d18/d2e/d3f/f43 2200105 0 2024-09-21T04:35:23.006 INFO:tasks.workunit.client.1.smithi053.stdout:7/907: mkdir d0/d6f/dbf/da2/d12b 0 2024-09-21T04:35:23.007 INFO:tasks.workunit.client.0.smithi007.stdout:2/295: mknod d0/d11/d31/c49 0 2024-09-21T04:35:23.010 INFO:tasks.workunit.client.0.smithi007.stdout:0/333: creat d6/d16/d52/f78 x:0 0 0 2024-09-21T04:35:23.012 INFO:tasks.workunit.client.1.smithi053.stdout:0/988: getdents d5/d10/d4c/db1/dbc/dca 0 2024-09-21T04:35:23.013 INFO:tasks.workunit.client.1.smithi053.stdout:7/908: mkdir d0/d2b/d36/d64/d9e/d12c 0 2024-09-21T04:35:23.014 INFO:tasks.workunit.client.0.smithi007.stdout:0/334: mknod d6/d16/d18/d39/c79 0 2024-09-21T04:35:23.015 INFO:tasks.workunit.client.0.smithi007.stdout:0/335: read - d6/d16/d18/f69 zero size 2024-09-21T04:35:23.016 INFO:tasks.workunit.client.1.smithi053.stdout:0/989: creat d5/d10/d4c/db1/dbc/dca/d53/f14a x:0 0 0 2024-09-21T04:35:23.016 INFO:tasks.workunit.client.1.smithi053.stdout:0/990: dread - d5/de/d9e/dd2/f124 zero size 2024-09-21T04:35:23.017 INFO:tasks.workunit.client.0.smithi007.stdout:6/351: rmdir d3/d3e/d46/d60/d62 0 2024-09-21T04:35:23.017 INFO:tasks.workunit.client.1.smithi053.stdout:0/991: chown d5/d10/d4c/db1/dbc/dca/d21/d96/fea 45754867 1 2024-09-21T04:35:23.018 INFO:tasks.workunit.client.1.smithi053.stdout:7/909: mkdir d0/d6f/dbf/d50/d8b/d70/dc1/d12d 0 2024-09-21T04:35:23.019 INFO:tasks.workunit.client.0.smithi007.stdout:6/352: mkdir d3/d3e/d46/d70 0 2024-09-21T04:35:23.022 INFO:tasks.workunit.client.1.smithi053.stdout:0/992: rename d5/de/d9e/cd4 to d5/de/c14b 0 2024-09-21T04:35:23.023 INFO:tasks.workunit.client.0.smithi007.stdout:0/336: link d6/d16/c17 d6/d16/d18/c7a 0 2024-09-21T04:35:23.023 INFO:tasks.workunit.client.1.smithi053.stdout:0/993: write d5/d10/d4c/db1/dbc/fcf [1965033,90888] 0 2024-09-21T04:35:23.024 INFO:tasks.workunit.client.0.smithi007.stdout:6/353: symlink d3/d59/d6b/l71 0 2024-09-21T04:35:23.026 INFO:tasks.workunit.client.0.smithi007.stdout:0/337: mknod d6/d16/d18/d39/c7b 0 2024-09-21T04:35:23.027 INFO:tasks.workunit.client.0.smithi007.stdout:6/354: symlink d3/d3e/d46/d60/l72 0 2024-09-21T04:35:23.027 INFO:tasks.workunit.client.0.smithi007.stdout:0/338: truncate d6/d16/f4d 1085002 0 2024-09-21T04:35:23.027 INFO:tasks.workunit.client.1.smithi053.stdout:7/910: link d0/d6f/dbf/d50/d8b/f18 d0/d6f/dbf/d50/f12e 0 2024-09-21T04:35:23.028 INFO:tasks.workunit.client.1.smithi053.stdout:0/994: mknod d5/dd/c14c 0 2024-09-21T04:35:23.029 INFO:tasks.workunit.client.1.smithi053.stdout:0/995: dread - d5/d10/d4c/db1/dbc/dca/f130 zero size 2024-09-21T04:35:23.029 INFO:tasks.workunit.client.1.smithi053.stdout:7/911: mknod d0/d6f/dbf/da2/dd7/c12f 0 2024-09-21T04:35:23.030 INFO:tasks.workunit.client.0.smithi007.stdout:0/339: unlink d6/d16/d18/d2e/d60/l71 0 2024-09-21T04:35:23.032 INFO:tasks.workunit.client.0.smithi007.stdout:0/340: read d6/d16/d57/f67 [259131,97492] 0 2024-09-21T04:35:23.034 INFO:tasks.workunit.client.0.smithi007.stdout:6/355: link d3/de/d12/d19/f1c d3/d59/d6b/f73 0 2024-09-21T04:35:23.034 INFO:tasks.workunit.client.0.smithi007.stdout:6/356: read d3/f28 [3510084,36202] 0 2024-09-21T04:35:23.037 INFO:tasks.workunit.client.1.smithi053.stdout:0/996: creat d5/d10/d4c/db1/dbc/f14d x:0 0 0 2024-09-21T04:35:23.039 INFO:tasks.workunit.client.1.smithi053.stdout:7/912: mknod d0/d2b/d36/d64/ddb/c130 0 2024-09-21T04:35:23.042 INFO:tasks.workunit.client.1.smithi053.stdout:0/997: rename d5/f15 to d5/de/d42/d78/d94/f14e 0 2024-09-21T04:35:23.044 INFO:tasks.workunit.client.0.smithi007.stdout:0/341: dread d6/d16/f28 [0,4194304] 0 2024-09-21T04:35:23.044 INFO:tasks.workunit.client.1.smithi053.stdout:7/913: dwrite d0/f13 [0,4194304] 0 2024-09-21T04:35:23.045 INFO:tasks.workunit.client.0.smithi007.stdout:0/342: chown d6/d16/d18/d2e/d3f/l46 5801 1 2024-09-21T04:35:23.045 INFO:tasks.workunit.client.1.smithi053.stdout:7/914: readlink d0/d24/lc5 0 2024-09-21T04:35:23.045 INFO:tasks.workunit.client.0.smithi007.stdout:6/357: rename d3/de/d12/d19/c4a to d3/d3e/d46/d60/c74 0 2024-09-21T04:35:23.047 INFO:tasks.workunit.client.1.smithi053.stdout:0/998: mkdir d5/d10/d4c/db1/dbc/dca/d141/d14f 0 2024-09-21T04:35:23.048 INFO:tasks.workunit.client.0.smithi007.stdout:6/358: rmdir d3/d59 39 2024-09-21T04:35:23.055 INFO:tasks.workunit.client.0.smithi007.stdout:6/359: dwrite d3/de/d4e/f57 [0,4194304] 0 2024-09-21T04:35:23.057 INFO:tasks.workunit.client.0.smithi007.stdout:3/310: sync 2024-09-21T04:35:23.058 INFO:tasks.workunit.client.0.smithi007.stdout:4/462: sync 2024-09-21T04:35:23.058 INFO:tasks.workunit.client.0.smithi007.stdout:1/506: sync 2024-09-21T04:35:23.058 INFO:tasks.workunit.client.0.smithi007.stdout:7/368: sync 2024-09-21T04:35:23.058 INFO:tasks.workunit.client.0.smithi007.stdout:9/403: sync 2024-09-21T04:35:23.058 INFO:tasks.workunit.client.0.smithi007.stdout:8/212: sync 2024-09-21T04:35:23.058 INFO:tasks.workunit.client.1.smithi053.stdout:0/999: link d5/d10/d4c/db1/dbc/dca/d21/d95/f9b d5/d10/d4c/f150 0 2024-09-21T04:35:23.062 INFO:tasks.workunit.client.0.smithi007.stdout:1/507: write d5/d6/d21/d2b/d4b/f66 [715885,15760] 0 2024-09-21T04:35:23.066 INFO:tasks.workunit.client.0.smithi007.stdout:3/311: write dc/d18/f31 [1932705,4360] 0 2024-09-21T04:35:23.069 INFO:tasks.workunit.client.0.smithi007.stdout:3/312: dread - dc/d18/d26/f51 zero size 2024-09-21T04:35:23.070 INFO:tasks.workunit.client.0.smithi007.stdout:9/404: mkdir d13/d18/d60/d6d/d85 0 2024-09-21T04:35:23.071 INFO:tasks.workunit.client.0.smithi007.stdout:8/213: dwrite d0/d3/d8/de/f17 [0,4194304] 0 2024-09-21T04:35:23.073 INFO:tasks.workunit.client.0.smithi007.stdout:4/463: dwrite d2/d17/d18/f84 [0,4194304] 0 2024-09-21T04:35:23.073 INFO:tasks.workunit.client.0.smithi007.stdout:3/313: fdatasync dc/d18/d26/f4b 0 2024-09-21T04:35:23.073 INFO:tasks.workunit.client.0.smithi007.stdout:6/360: creat d3/de/d4e/d67/f75 x:0 0 0 2024-09-21T04:35:23.076 INFO:tasks.workunit.client.0.smithi007.stdout:8/214: chown d0/d3/d8/l2f 4078 1 2024-09-21T04:35:23.076 INFO:tasks.workunit.client.0.smithi007.stdout:6/361: readlink d3/de/d12/l2e 0 2024-09-21T04:35:23.077 INFO:tasks.workunit.client.0.smithi007.stdout:8/215: readlink d0/d3/d8/lc 0 2024-09-21T04:35:23.077 INFO:tasks.workunit.client.0.smithi007.stdout:1/508: stat d5/l2e 0 2024-09-21T04:35:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:4/464: readlink d2/d7/l46 0 2024-09-21T04:35:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:1/509: dread d5/d6/d21/d2b/d4b/f66 [0,4194304] 0 2024-09-21T04:35:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:1/510: chown d5/f28 133 1 2024-09-21T04:35:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:3/314: creat dc/d22/d29/d32/d38/f60 x:0 0 0 2024-09-21T04:35:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:3/315: chown dc/d22/d29/f55 1258071605 1 2024-09-21T04:35:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:9/405: getdents d13/d18/d80 0 2024-09-21T04:35:23.095 INFO:tasks.workunit.client.0.smithi007.stdout:3/316: dwrite dc/d18/f45 [0,4194304] 0 2024-09-21T04:35:23.096 INFO:tasks.workunit.client.0.smithi007.stdout:3/317: fsync dc/d22/d29/d32/d37/f49 0 2024-09-21T04:35:23.100 INFO:tasks.workunit.client.0.smithi007.stdout:6/362: link d3/de/d12/f50 d3/d59/f76 0 2024-09-21T04:35:23.102 INFO:tasks.workunit.client.0.smithi007.stdout:1/511: mkdir d5/d6/da2 0 2024-09-21T04:35:23.103 INFO:tasks.workunit.client.0.smithi007.stdout:9/406: mknod d13/d18/c86 0 2024-09-21T04:35:23.106 INFO:tasks.workunit.client.0.smithi007.stdout:9/407: stat d13/d2a/c30 0 2024-09-21T04:35:23.106 INFO:tasks.workunit.client.0.smithi007.stdout:3/318: dwrite dc/d18/f58 [0,4194304] 0 2024-09-21T04:35:23.106 INFO:tasks.workunit.client.0.smithi007.stdout:8/216: getdents d0/d3/d16 0 2024-09-21T04:35:23.110 INFO:tasks.workunit.client.1.smithi053.stdout:7/915: write d0/d6f/dbf/d50/d84/fc8 [774109,57366] 0 2024-09-21T04:35:23.112 INFO:tasks.workunit.client.0.smithi007.stdout:9/408: readlink d13/d2a/l33 0 2024-09-21T04:35:23.113 INFO:tasks.workunit.client.1.smithi053.stdout:7/916: creat d0/d6f/dbf/d5d/d9f/f131 x:0 0 0 2024-09-21T04:35:23.114 INFO:tasks.workunit.client.0.smithi007.stdout:8/217: read d0/d3/d8/fa [1977122,49778] 0 2024-09-21T04:35:23.115 INFO:tasks.workunit.client.0.smithi007.stdout:1/512: dwrite d5/d45/d5e/f24 [4194304,4194304] 0 2024-09-21T04:35:23.116 INFO:tasks.workunit.client.0.smithi007.stdout:9/409: mkdir d13/d74/d87 0 2024-09-21T04:35:23.118 INFO:tasks.workunit.client.0.smithi007.stdout:8/218: fdatasync d0/d3/d8/f1d 0 2024-09-21T04:35:23.119 INFO:tasks.workunit.client.0.smithi007.stdout:6/363: dread d3/f28 [0,4194304] 0 2024-09-21T04:35:23.129 INFO:tasks.workunit.client.0.smithi007.stdout:7/369: dwrite d3/d6/d14/d20/f3f [0,4194304] 0 2024-09-21T04:35:23.130 INFO:tasks.workunit.client.0.smithi007.stdout:3/319: link dc/d22/d29/d2d/f48 dc/d56/f61 0 2024-09-21T04:35:23.136 INFO:tasks.workunit.client.0.smithi007.stdout:6/364: rename d3/d59 to d3/d3e/d46/d60/d77 0 2024-09-21T04:35:23.144 INFO:tasks.workunit.client.0.smithi007.stdout:1/513: dwrite d5/d45/d5e/d55/f80 [0,4194304] 0 2024-09-21T04:35:23.144 INFO:tasks.workunit.client.0.smithi007.stdout:9/410: dwrite d13/f4f [0,4194304] 0 2024-09-21T04:35:23.149 INFO:tasks.workunit.client.0.smithi007.stdout:8/219: dwrite d0/d3/d8/f1d [4194304,4194304] 0 2024-09-21T04:35:23.149 INFO:tasks.workunit.client.1.smithi053.stdout:7/917: dwrite d0/d6f/dbf/d50/dc7/f11b [0,4194304] 0 2024-09-21T04:35:23.156 INFO:tasks.workunit.client.0.smithi007.stdout:9/411: creat d13/d18/d60/f88 x:0 0 0 2024-09-21T04:35:23.157 INFO:tasks.workunit.client.0.smithi007.stdout:8/220: mknod d0/d3/d8/de/d26/c34 0 2024-09-21T04:35:23.159 INFO:tasks.workunit.client.0.smithi007.stdout:7/370: dread d3/d6/d10/f24 [0,4194304] 0 2024-09-21T04:35:23.159 INFO:tasks.workunit.client.1.smithi053.stdout:7/918: dread d0/d24/f104 [0,4194304] 0 2024-09-21T04:35:23.160 INFO:tasks.workunit.client.1.smithi053.stdout:7/919: dread - d0/d2b/d36/d64/fb7 zero size 2024-09-21T04:35:23.161 INFO:tasks.workunit.client.0.smithi007.stdout:3/320: truncate dc/f1c 1794457 0 2024-09-21T04:35:23.165 INFO:tasks.workunit.client.0.smithi007.stdout:1/514: truncate d5/d6/d21/d2b/f51 1348712 0 2024-09-21T04:35:23.166 INFO:tasks.workunit.client.0.smithi007.stdout:8/221: truncate d0/d3/f2b 677490 0 2024-09-21T04:35:23.169 INFO:tasks.workunit.client.0.smithi007.stdout:6/365: dwrite d3/f2f [0,4194304] 0 2024-09-21T04:35:23.169 INFO:tasks.workunit.client.0.smithi007.stdout:3/321: symlink dc/l62 0 2024-09-21T04:35:23.171 INFO:tasks.workunit.client.0.smithi007.stdout:8/222: readlink d0/d3/d8/de/l1c 0 2024-09-21T04:35:23.173 INFO:tasks.workunit.client.0.smithi007.stdout:1/515: write d5/d6/d21/d2b/f6f [254782,98371] 0 2024-09-21T04:35:23.175 INFO:tasks.workunit.client.0.smithi007.stdout:6/366: write d3/f54 [925232,107475] 0 2024-09-21T04:35:23.176 INFO:tasks.workunit.client.0.smithi007.stdout:3/322: read dc/d18/f31 [4089156,25459] 0 2024-09-21T04:35:23.181 INFO:tasks.workunit.client.0.smithi007.stdout:8/223: dwrite d0/d3/d8/de/f17 [0,4194304] 0 2024-09-21T04:35:23.181 INFO:tasks.workunit.client.0.smithi007.stdout:3/323: unlink fb 0 2024-09-21T04:35:23.184 INFO:tasks.workunit.client.0.smithi007.stdout:8/224: write d0/d3/d8/f10 [5144756,45669] 0 2024-09-21T04:35:23.189 INFO:tasks.workunit.client.0.smithi007.stdout:6/367: dwrite d3/d3e/f55 [0,4194304] 0 2024-09-21T04:35:23.205 INFO:tasks.workunit.client.0.smithi007.stdout:3/324: dread fa [0,4194304] 0 2024-09-21T04:35:23.210 INFO:tasks.workunit.client.0.smithi007.stdout:3/325: rename c7 to dc/d18/c63 0 2024-09-21T04:35:23.215 INFO:tasks.workunit.client.0.smithi007.stdout:3/326: symlink dc/d22/d29/d32/d37/l64 0 2024-09-21T04:35:23.216 INFO:tasks.workunit.client.0.smithi007.stdout:8/225: dread d0/d3/d8/f1d [0,4194304] 0 2024-09-21T04:35:23.217 INFO:tasks.workunit.client.0.smithi007.stdout:6/368: write d3/fc [2615840,130819] 0 2024-09-21T04:35:23.219 INFO:tasks.workunit.client.0.smithi007.stdout:3/327: creat dc/d18/f65 x:0 0 0 2024-09-21T04:35:23.220 INFO:tasks.workunit.client.0.smithi007.stdout:3/328: stat dc/c27 0 2024-09-21T04:35:23.226 INFO:tasks.workunit.client.0.smithi007.stdout:6/369: dwrite d3/de/d12/f1f [0,4194304] 0 2024-09-21T04:35:23.227 INFO:tasks.workunit.client.0.smithi007.stdout:6/370: write d3/de/f6a [575151,31894] 0 2024-09-21T04:35:23.230 INFO:tasks.workunit.client.0.smithi007.stdout:6/371: chown d3/de/d4e/d67/f75 5216663 1 2024-09-21T04:35:23.232 INFO:tasks.workunit.client.0.smithi007.stdout:8/226: truncate d0/d3/d16/f2c 3096791 0 2024-09-21T04:35:23.235 INFO:tasks.workunit.client.0.smithi007.stdout:6/372: rename d3/d3e/d46/d60/d77/d6b to d3/d3e/d46/d70/d78 0 2024-09-21T04:35:23.237 INFO:tasks.workunit.client.0.smithi007.stdout:8/227: mkdir d0/d3/d8/de/d26/d35 0 2024-09-21T04:35:23.239 INFO:tasks.workunit.client.0.smithi007.stdout:3/329: dread dc/f23 [0,4194304] 0 2024-09-21T04:35:23.240 INFO:tasks.workunit.client.0.smithi007.stdout:6/373: creat d3/d3b/f79 x:0 0 0 2024-09-21T04:35:23.241 INFO:tasks.workunit.client.0.smithi007.stdout:3/330: write dc/d18/f45 [1988469,57814] 0 2024-09-21T04:35:23.244 INFO:tasks.workunit.client.0.smithi007.stdout:3/331: write dc/d22/d29/d32/d38/f60 [1006464,74367] 0 2024-09-21T04:35:23.246 INFO:tasks.workunit.client.0.smithi007.stdout:3/332: stat dc/d22/d29/d2d/l36 0 2024-09-21T04:35:23.249 INFO:tasks.workunit.client.0.smithi007.stdout:3/333: mknod dc/d22/c66 0 2024-09-21T04:35:23.250 INFO:tasks.workunit.client.0.smithi007.stdout:8/228: write d0/d3/d8/fa [7473275,94980] 0 2024-09-21T04:35:23.250 INFO:tasks.workunit.client.0.smithi007.stdout:3/334: rmdir dc/d22/d29 39 2024-09-21T04:35:23.253 INFO:tasks.workunit.client.0.smithi007.stdout:6/374: write d3/d3e/d46/d60/d77/f76 [588523,124063] 0 2024-09-21T04:35:23.262 INFO:tasks.workunit.client.1.smithi053.stdout:7/920: sync 2024-09-21T04:35:23.263 INFO:tasks.workunit.client.1.smithi053.stdout:7/921: write d0/d6f/dbf/d50/dc7/f11c [887798,107065] 0 2024-09-21T04:35:23.266 INFO:tasks.workunit.client.1.smithi053.stdout:7/922: mknod d0/d6f/dbf/d50/d8b/d70/c132 0 2024-09-21T04:35:23.274 INFO:tasks.workunit.client.0.smithi007.stdout:5/569: sync 2024-09-21T04:35:23.278 INFO:tasks.workunit.client.0.smithi007.stdout:6/375: dread d3/de/d12/f50 [0,4194304] 0 2024-09-21T04:35:23.279 INFO:tasks.workunit.client.0.smithi007.stdout:5/570: symlink d2/d27/d42/da3/d23/d2a/lb7 0 2024-09-21T04:35:23.280 INFO:tasks.workunit.client.0.smithi007.stdout:3/335: dwrite dc/d22/f24 [4194304,4194304] 0 2024-09-21T04:35:23.282 INFO:tasks.workunit.client.0.smithi007.stdout:6/376: chown d3/de/d12/d19/f51 0 1 2024-09-21T04:35:23.282 INFO:tasks.workunit.client.0.smithi007.stdout:3/336: fsync dc/d18/f65 0 2024-09-21T04:35:23.284 INFO:tasks.workunit.client.0.smithi007.stdout:3/337: dread - dc/d18/d26/f5f zero size 2024-09-21T04:35:23.285 INFO:tasks.workunit.client.0.smithi007.stdout:6/377: creat d3/d3b/f7a x:0 0 0 2024-09-21T04:35:23.287 INFO:tasks.workunit.client.0.smithi007.stdout:3/338: creat dc/d18/f67 x:0 0 0 2024-09-21T04:35:23.288 INFO:tasks.workunit.client.0.smithi007.stdout:8/229: dwrite d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:23.289 INFO:tasks.workunit.client.0.smithi007.stdout:6/378: chown d3/d3e/d46/d70/d78 215088 1 2024-09-21T04:35:23.290 INFO:tasks.workunit.client.1.smithi053.stdout:7/923: dwrite d0/d6f/dbf/f8f [0,4194304] 0 2024-09-21T04:35:23.291 INFO:tasks.workunit.client.0.smithi007.stdout:5/571: rename d2/d27/d42/da3/d1d/c2b to d2/d27/d42/da3/d23/d49/d44/d54/cb8 0 2024-09-21T04:35:23.294 INFO:tasks.workunit.client.0.smithi007.stdout:5/572: chown d2/d27/d42/da3/le 0 1 2024-09-21T04:35:23.294 INFO:tasks.workunit.client.0.smithi007.stdout:6/379: creat d3/d3e/f7b x:0 0 0 2024-09-21T04:35:23.294 INFO:tasks.workunit.client.0.smithi007.stdout:3/339: write dc/d22/d29/d2d/f30 [1405921,40312] 0 2024-09-21T04:35:23.296 INFO:tasks.workunit.client.1.smithi053.stdout:7/924: mkdir d0/dca/d133 0 2024-09-21T04:35:23.301 INFO:tasks.workunit.client.0.smithi007.stdout:6/380: creat d3/de/d12/f7c x:0 0 0 2024-09-21T04:35:23.302 INFO:tasks.workunit.client.0.smithi007.stdout:5/573: read - d2/d27/d42/da3/d23/d2a/f9f zero size 2024-09-21T04:35:23.304 INFO:tasks.workunit.client.0.smithi007.stdout:8/230: dwrite d0/d3/d8/de/f17 [0,4194304] 0 2024-09-21T04:35:23.307 INFO:tasks.workunit.client.0.smithi007.stdout:6/381: mkdir d3/de/d7d 0 2024-09-21T04:35:23.310 INFO:tasks.workunit.client.0.smithi007.stdout:6/382: truncate d3/de/f4f 1710786 0 2024-09-21T04:35:23.311 INFO:tasks.workunit.client.0.smithi007.stdout:5/574: readlink d2/d27/d42/da3/d23/d2a/lb7 0 2024-09-21T04:35:23.312 INFO:tasks.workunit.client.0.smithi007.stdout:8/231: symlink d0/l36 0 2024-09-21T04:35:23.313 INFO:tasks.workunit.client.0.smithi007.stdout:5/575: write d2/d27/d42/da3/d23/d2a/f75 [1141796,78471] 0 2024-09-21T04:35:23.316 INFO:tasks.workunit.client.1.smithi053.stdout:7/925: creat d0/d2b/f134 x:0 0 0 2024-09-21T04:35:23.317 INFO:tasks.workunit.client.0.smithi007.stdout:5/576: write d2/d27/d42/da3/d23/d49/f80 [2795164,107198] 0 2024-09-21T04:35:23.322 INFO:tasks.workunit.client.0.smithi007.stdout:8/232: dwrite d0/d3/d8/f9 [0,4194304] 0 2024-09-21T04:35:23.324 INFO:tasks.workunit.client.0.smithi007.stdout:6/383: dwrite d3/de/d12/d19/f24 [0,4194304] 0 2024-09-21T04:35:23.325 INFO:tasks.workunit.client.1.smithi053.stdout:7/926: truncate d0/d2b/d36/d64/fd1 2208830 0 2024-09-21T04:35:23.326 INFO:tasks.workunit.client.0.smithi007.stdout:3/340: write dc/d56/f61 [313579,65676] 0 2024-09-21T04:35:23.327 INFO:tasks.workunit.client.0.smithi007.stdout:8/233: dread d0/d3/d16/f21 [4194304,4194304] 0 2024-09-21T04:35:23.328 INFO:tasks.workunit.client.0.smithi007.stdout:3/341: dread - dc/d18/d26/f51 zero size 2024-09-21T04:35:23.332 INFO:tasks.workunit.client.1.smithi053.stdout:7/927: dwrite d0/f13 [0,4194304] 0 2024-09-21T04:35:23.335 INFO:tasks.workunit.client.0.smithi007.stdout:5/577: link d2/fc d2/d27/d42/da3/fb9 0 2024-09-21T04:35:23.337 INFO:tasks.workunit.client.0.smithi007.stdout:5/578: fdatasync d2/d27/d42/da3/d28/fa5 0 2024-09-21T04:35:23.337 INFO:tasks.workunit.client.0.smithi007.stdout:3/342: creat dc/d18/d26/f68 x:0 0 0 2024-09-21T04:35:23.340 INFO:tasks.workunit.client.0.smithi007.stdout:3/343: truncate dc/d22/d29/d2d/f48 830074 0 2024-09-21T04:35:23.340 INFO:tasks.workunit.client.0.smithi007.stdout:3/344: readlink dc/d18/l1b 0 2024-09-21T04:35:23.342 INFO:tasks.workunit.client.0.smithi007.stdout:3/345: write dc/d22/f24 [12476908,28601] 0 2024-09-21T04:35:23.345 INFO:tasks.workunit.client.1.smithi053.stdout:7/928: creat d0/f135 x:0 0 0 2024-09-21T04:35:23.346 INFO:tasks.workunit.client.0.smithi007.stdout:3/346: rename dc/d22/d29/d32/d37 to dc/d22/d29/d32/d69 0 2024-09-21T04:35:23.347 INFO:tasks.workunit.client.0.smithi007.stdout:3/347: dread - dc/d18/f65 zero size 2024-09-21T04:35:23.348 INFO:tasks.workunit.client.1.smithi053.stdout:7/929: mknod d0/d24/d46/ddd/c136 0 2024-09-21T04:35:23.351 INFO:tasks.workunit.client.0.smithi007.stdout:3/348: creat dc/d22/d29/f6a x:0 0 0 2024-09-21T04:35:23.351 INFO:tasks.workunit.client.0.smithi007.stdout:3/349: readlink dc/l62 0 2024-09-21T04:35:23.353 INFO:tasks.workunit.client.0.smithi007.stdout:3/350: fsync dc/d22/d29/d32/d38/f60 0 2024-09-21T04:35:23.355 INFO:tasks.workunit.client.0.smithi007.stdout:3/351: unlink dc/ld 0 2024-09-21T04:35:23.355 INFO:tasks.workunit.client.0.smithi007.stdout:6/384: truncate d3/de/d12/f1f 4102164 0 2024-09-21T04:35:23.356 INFO:tasks.workunit.client.0.smithi007.stdout:5/579: write d2/f51 [889019,60623] 0 2024-09-21T04:35:23.357 INFO:tasks.workunit.client.0.smithi007.stdout:3/352: write dc/d22/d29/d2d/f43 [2724742,3110] 0 2024-09-21T04:35:23.357 INFO:tasks.workunit.client.0.smithi007.stdout:6/385: readlink d3/d3e/d46/d70/d78/l71 0 2024-09-21T04:35:23.359 INFO:tasks.workunit.client.1.smithi053.stdout:7/930: chown d0/d2b/d36/d64/fd1 184556 1 2024-09-21T04:35:23.359 INFO:tasks.workunit.client.0.smithi007.stdout:3/353: fdatasync dc/d18/f39 0 2024-09-21T04:35:23.363 INFO:tasks.workunit.client.0.smithi007.stdout:6/386: mknod d3/de/d58/c7e 0 2024-09-21T04:35:23.365 INFO:tasks.workunit.client.0.smithi007.stdout:5/580: rename d2/d27/d42/da3/d23/d49/d44/d54/f88 to d2/d27/d42/da3/d23/d5d/d9a/da6/fba 0 2024-09-21T04:35:23.370 INFO:tasks.workunit.client.0.smithi007.stdout:6/387: rmdir d3/de/d12/d19 39 2024-09-21T04:35:23.371 INFO:tasks.workunit.client.1.smithi053.stdout:7/931: write d0/d6f/dbf/f53 [3076023,18714] 0 2024-09-21T04:35:23.371 INFO:tasks.workunit.client.0.smithi007.stdout:5/581: rmdir d2/d27/d42/da3/d23/d5d/d61 39 2024-09-21T04:35:23.375 INFO:tasks.workunit.client.0.smithi007.stdout:6/388: symlink d3/d3e/d46/d60/l7f 0 2024-09-21T04:35:23.376 INFO:tasks.workunit.client.0.smithi007.stdout:5/582: mknod d2/d27/d42/da3/d23/d49/d44/cbb 0 2024-09-21T04:35:23.378 INFO:tasks.workunit.client.1.smithi053.stdout:7/932: mkdir d0/d2b/d36/d64/d9e/d12c/d137 0 2024-09-21T04:35:23.378 INFO:tasks.workunit.client.0.smithi007.stdout:3/354: truncate dc/d22/d29/d2d/f30 2705385 0 2024-09-21T04:35:23.380 INFO:tasks.workunit.client.0.smithi007.stdout:5/583: stat d2/d27/d42/da3/c21 0 2024-09-21T04:35:23.381 INFO:tasks.workunit.client.1.smithi053.stdout:7/933: dread - d0/d2b/d36/d64/ddb/f124 zero size 2024-09-21T04:35:23.386 INFO:tasks.workunit.client.1.smithi053.stdout:7/934: symlink d0/d6f/dbf/d50/d84/db8/l138 0 2024-09-21T04:35:23.386 INFO:tasks.workunit.client.0.smithi007.stdout:6/389: dwrite d3/f1e [0,4194304] 0 2024-09-21T04:35:23.387 INFO:tasks.workunit.client.1.smithi053.stdout:7/935: write d0/d6f/dbf/d50/fa5 [2823587,103897] 0 2024-09-21T04:35:23.387 INFO:tasks.workunit.client.0.smithi007.stdout:6/390: dread - d3/de/f1a zero size 2024-09-21T04:35:23.393 INFO:tasks.workunit.client.0.smithi007.stdout:5/584: rename d2/d27/d42/da3/d23/d5d/lb5 to d2/d27/d42/da3/d23/d5d/d9a/lbc 0 2024-09-21T04:35:23.394 INFO:tasks.workunit.client.0.smithi007.stdout:3/355: write dc/d18/f67 [72030,73096] 0 2024-09-21T04:35:23.394 INFO:tasks.workunit.client.1.smithi053.stdout:7/936: creat d0/d6f/dbf/da2/f139 x:0 0 0 2024-09-21T04:35:23.398 INFO:tasks.workunit.client.1.smithi053.stdout:7/937: creat d0/d6f/dbf/d95/f13a x:0 0 0 2024-09-21T04:35:23.402 INFO:tasks.workunit.client.0.smithi007.stdout:3/356: unlink dc/d56/f61 0 2024-09-21T04:35:23.403 INFO:tasks.workunit.client.1.smithi053.stdout:7/938: dwrite d0/d24/fcc [0,4194304] 0 2024-09-21T04:35:23.403 INFO:tasks.workunit.client.0.smithi007.stdout:6/391: getdents d3/d3e/d46/d60/d77 0 2024-09-21T04:35:23.404 INFO:tasks.workunit.client.0.smithi007.stdout:6/392: fdatasync d3/d3b/f79 0 2024-09-21T04:35:23.408 INFO:tasks.workunit.client.1.smithi053.stdout:7/939: mkdir d0/d6f/dbf/d5c/d13b 0 2024-09-21T04:35:23.411 INFO:tasks.workunit.client.0.smithi007.stdout:3/357: chown dc/d22/d29/d2d/c4e 36491 1 2024-09-21T04:35:23.412 INFO:tasks.workunit.client.0.smithi007.stdout:6/393: symlink d3/d3e/l80 0 2024-09-21T04:35:23.413 INFO:tasks.workunit.client.1.smithi053.stdout:7/940: symlink d0/d6f/dbf/d50/d8b/l13c 0 2024-09-21T04:35:23.414 INFO:tasks.workunit.client.0.smithi007.stdout:5/585: link d2/d27/d42/da3/d28/f4d d2/d27/d42/da3/d23/d5d/d9a/fbd 0 2024-09-21T04:35:23.416 INFO:tasks.workunit.client.0.smithi007.stdout:3/358: symlink dc/d22/d29/d32/d69/l6b 0 2024-09-21T04:35:23.416 INFO:tasks.workunit.client.0.smithi007.stdout:6/394: creat d3/d3e/d46/f81 x:0 0 0 2024-09-21T04:35:23.417 INFO:tasks.workunit.client.1.smithi053.stdout:7/941: rmdir d0/d6f/dbf/d50/d8b/d70/df2 39 2024-09-21T04:35:23.419 INFO:tasks.workunit.client.0.smithi007.stdout:5/586: rename d2/d27/d42/da3/d23/d5d/d9a/faa to d2/d27/d42/d53/daf/db0/fbe 0 2024-09-21T04:35:23.420 INFO:tasks.workunit.client.0.smithi007.stdout:6/395: mknod d3/d3b/c82 0 2024-09-21T04:35:23.421 INFO:tasks.workunit.client.0.smithi007.stdout:3/359: fdatasync dc/f21 0 2024-09-21T04:35:23.422 INFO:tasks.workunit.client.1.smithi053.stdout:7/942: mkdir d0/d6f/dbf/d50/d8b/d70/dc1/d12d/d13d 0 2024-09-21T04:35:23.424 INFO:tasks.workunit.client.0.smithi007.stdout:6/396: write d3/de/d12/d19/f39 [624045,2070] 0 2024-09-21T04:35:23.430 INFO:tasks.workunit.client.0.smithi007.stdout:3/360: mkdir dc/d6c 0 2024-09-21T04:35:23.431 INFO:tasks.workunit.client.0.smithi007.stdout:6/397: unlink d3/d3e/f43 0 2024-09-21T04:35:23.432 INFO:tasks.workunit.client.0.smithi007.stdout:6/398: chown d3/de/d12/l34 145 1 2024-09-21T04:35:23.436 INFO:tasks.workunit.client.0.smithi007.stdout:3/361: rmdir dc/d22/d29 39 2024-09-21T04:35:23.437 INFO:tasks.workunit.client.1.smithi053.stdout:7/943: truncate d0/d6f/dbf/d5c/f8a 1308814 0 2024-09-21T04:35:23.439 INFO:tasks.workunit.client.0.smithi007.stdout:6/399: dwrite d3/d3e/f55 [4194304,4194304] 0 2024-09-21T04:35:23.440 INFO:tasks.workunit.client.1.smithi053.stdout:7/944: creat d0/d6f/dbf/d95/f13e x:0 0 0 2024-09-21T04:35:23.443 INFO:tasks.workunit.client.0.smithi007.stdout:5/587: dwrite d2/d27/d42/f62 [0,4194304] 0 2024-09-21T04:35:23.444 INFO:tasks.workunit.client.1.smithi053.stdout:7/945: dread d0/d6f/dbf/d50/d8b/f31 [0,4194304] 0 2024-09-21T04:35:23.445 INFO:tasks.workunit.client.0.smithi007.stdout:5/588: stat d2/d27/d42/da3/d23/d5d 0 2024-09-21T04:35:23.446 INFO:tasks.workunit.client.0.smithi007.stdout:6/400: creat d3/d3b/f83 x:0 0 0 2024-09-21T04:35:23.446 INFO:tasks.workunit.client.0.smithi007.stdout:3/362: dwrite dc/d18/f67 [0,4194304] 0 2024-09-21T04:35:23.449 INFO:tasks.workunit.client.0.smithi007.stdout:5/589: stat d2/d1e/f58 0 2024-09-21T04:35:23.451 INFO:tasks.workunit.client.1.smithi053.stdout:7/946: unlink d0/d2b/d36/d64/dc0/lee 0 2024-09-21T04:35:23.454 INFO:tasks.workunit.client.0.smithi007.stdout:5/590: dread - d2/d27/d3d/d9b/fa7 zero size 2024-09-21T04:35:23.456 INFO:tasks.workunit.client.0.smithi007.stdout:6/401: creat d3/d3e/d46/d70/d78/f84 x:0 0 0 2024-09-21T04:35:23.459 INFO:tasks.workunit.client.0.smithi007.stdout:3/363: rename dc/d22/d29/d2d to dc/d22/d29/d32/d69/d6d 0 2024-09-21T04:35:23.459 INFO:tasks.workunit.client.1.smithi053.stdout:7/947: dwrite d0/d6f/dbf/d5d/d9f/fa6 [0,4194304] 0 2024-09-21T04:35:23.462 INFO:tasks.workunit.client.0.smithi007.stdout:6/402: rmdir d3/d3e/d46/d70 39 2024-09-21T04:35:23.464 INFO:tasks.workunit.client.1.smithi053.stdout:7/948: dread - d0/d2b/f45 zero size 2024-09-21T04:35:23.466 INFO:tasks.workunit.client.0.smithi007.stdout:5/591: mknod d2/d27/d42/da3/d23/d5d/cbf 0 2024-09-21T04:35:23.469 INFO:tasks.workunit.client.0.smithi007.stdout:6/403: truncate d3/d3e/d46/d70/d78/f84 470555 0 2024-09-21T04:35:23.470 INFO:tasks.workunit.client.0.smithi007.stdout:5/592: fsync d2/d27/d42/da3/d23/d2a/f2f 0 2024-09-21T04:35:23.472 INFO:tasks.workunit.client.0.smithi007.stdout:5/593: write d2/d27/d42/da3/d23/d49/d6f/f7f [3104929,9619] 0 2024-09-21T04:35:23.474 INFO:tasks.workunit.client.0.smithi007.stdout:6/404: rename d3/d3e/d46/d70 to d3/d3e/d85 0 2024-09-21T04:35:23.478 INFO:tasks.workunit.client.0.smithi007.stdout:5/594: write d2/d27/d42/da3/d23/d5d/d61/fa4 [301710,110409] 0 2024-09-21T04:35:23.479 INFO:tasks.workunit.client.0.smithi007.stdout:3/364: write dc/f1c [721343,83394] 0 2024-09-21T04:35:23.479 INFO:tasks.workunit.client.0.smithi007.stdout:6/405: dread d3/f28 [0,4194304] 0 2024-09-21T04:35:23.483 INFO:tasks.workunit.client.0.smithi007.stdout:3/365: stat dc/f10 0 2024-09-21T04:35:23.485 INFO:tasks.workunit.client.0.smithi007.stdout:3/366: stat dc/l12 0 2024-09-21T04:35:23.487 INFO:tasks.workunit.client.0.smithi007.stdout:3/367: fdatasync dc/d22/d29/f6a 0 2024-09-21T04:35:23.488 INFO:tasks.workunit.client.0.smithi007.stdout:5/595: dwrite d2/d1e/f24 [0,4194304] 0 2024-09-21T04:35:23.488 INFO:tasks.workunit.client.0.smithi007.stdout:6/406: dwrite d3/f2f [0,4194304] 0 2024-09-21T04:35:23.489 INFO:tasks.workunit.client.0.smithi007.stdout:3/368: stat dc/d18/d26 0 2024-09-21T04:35:23.490 INFO:tasks.workunit.client.0.smithi007.stdout:5/596: mknod d2/d27/d3d/d9b/cc0 0 2024-09-21T04:35:23.492 INFO:tasks.workunit.client.0.smithi007.stdout:3/369: symlink dc/d22/d29/d32/d4d/l6e 0 2024-09-21T04:35:23.495 INFO:tasks.workunit.client.0.smithi007.stdout:5/597: dread d2/d27/d42/d53/f90 [0,4194304] 0 2024-09-21T04:35:23.496 INFO:tasks.workunit.client.0.smithi007.stdout:3/370: mknod dc/d22/d29/d32/d4f/c6f 0 2024-09-21T04:35:23.499 INFO:tasks.workunit.client.0.smithi007.stdout:5/598: mknod d2/d27/d42/da3/d23/da1/cc1 0 2024-09-21T04:35:23.501 INFO:tasks.workunit.client.0.smithi007.stdout:3/371: getdents dc/d22/d29/d32/d38 0 2024-09-21T04:35:23.502 INFO:tasks.workunit.client.0.smithi007.stdout:5/599: write d2/d27/d42/da3/d23/d5d/fb6 [2754433,66021] 0 2024-09-21T04:35:23.503 INFO:tasks.workunit.client.0.smithi007.stdout:3/372: dread - dc/d22/d29/f6a zero size 2024-09-21T04:35:23.507 INFO:tasks.workunit.client.0.smithi007.stdout:3/373: mknod dc/d22/d29/d32/d69/d6d/c70 0 2024-09-21T04:35:23.510 INFO:tasks.workunit.client.0.smithi007.stdout:3/374: truncate dc/d18/f28 1787556 0 2024-09-21T04:35:23.512 INFO:tasks.workunit.client.0.smithi007.stdout:3/375: mkdir dc/d22/d5a/d71 0 2024-09-21T04:35:23.515 INFO:tasks.workunit.client.0.smithi007.stdout:3/376: symlink dc/d22/d29/d32/d38/l72 0 2024-09-21T04:35:23.517 INFO:tasks.workunit.client.0.smithi007.stdout:5/600: dwrite d2/d27/d3d/d56/f96 [0,4194304] 0 2024-09-21T04:35:23.517 INFO:tasks.workunit.client.0.smithi007.stdout:3/377: creat dc/d22/d29/d32/d4d/f73 x:0 0 0 2024-09-21T04:35:23.519 INFO:tasks.workunit.client.0.smithi007.stdout:3/378: dread - dc/d22/d29/d32/d4d/f57 zero size 2024-09-21T04:35:23.520 INFO:tasks.workunit.client.0.smithi007.stdout:5/601: mkdir d2/d27/d3d/dc2 0 2024-09-21T04:35:23.523 INFO:tasks.workunit.client.0.smithi007.stdout:3/379: mkdir dc/d22/d5a/d71/d74 0 2024-09-21T04:35:23.526 INFO:tasks.workunit.client.0.smithi007.stdout:3/380: creat dc/d22/d5a/f75 x:0 0 0 2024-09-21T04:35:23.530 INFO:tasks.workunit.client.0.smithi007.stdout:3/381: link dc/d22/f24 dc/d18/d26/f76 0 2024-09-21T04:35:23.531 INFO:tasks.workunit.client.0.smithi007.stdout:3/382: dread - dc/d22/d29/d32/d4d/f73 zero size 2024-09-21T04:35:23.537 INFO:tasks.workunit.client.1.smithi053.stdout:7/949: sync 2024-09-21T04:35:23.539 INFO:tasks.workunit.client.0.smithi007.stdout:5/602: dwrite d2/d27/d42/da3/d1d/d20/f6d [0,4194304] 0 2024-09-21T04:35:23.539 INFO:tasks.workunit.client.1.smithi053.stdout:7/950: creat d0/d24/f13f x:0 0 0 2024-09-21T04:35:23.542 INFO:tasks.workunit.client.0.smithi007.stdout:3/383: dwrite dc/d22/f24 [12582912,4194304] 0 2024-09-21T04:35:23.546 INFO:tasks.workunit.client.1.smithi053.stdout:7/951: getdents d0/d2b/d36 0 2024-09-21T04:35:23.547 INFO:tasks.workunit.client.0.smithi007.stdout:3/384: creat dc/d22/d29/d32/f77 x:0 0 0 2024-09-21T04:35:23.548 INFO:tasks.workunit.client.1.smithi053.stdout:7/952: chown d0/d6f/dbf/d5d/d9f/fa6 392422162 1 2024-09-21T04:35:23.550 INFO:tasks.workunit.client.0.smithi007.stdout:3/385: write dc/d22/f24 [664610,24681] 0 2024-09-21T04:35:23.552 INFO:tasks.workunit.client.0.smithi007.stdout:5/603: dwrite d2/d27/d42/da3/f14 [0,4194304] 0 2024-09-21T04:35:23.552 INFO:tasks.workunit.client.1.smithi053.stdout:7/953: symlink d0/d2b/l140 0 2024-09-21T04:35:23.554 INFO:tasks.workunit.client.0.smithi007.stdout:5/604: symlink d2/d27/d42/da3/d23/d2a/lc3 0 2024-09-21T04:35:23.556 INFO:tasks.workunit.client.1.smithi053.stdout:7/954: mkdir d0/d6f/dbf/d50/d8b/d70/d141 0 2024-09-21T04:35:23.559 INFO:tasks.workunit.client.0.smithi007.stdout:3/386: rename dc/d22/d29/d32/d69/d6d/f43 to dc/f78 0 2024-09-21T04:35:23.560 INFO:tasks.workunit.client.0.smithi007.stdout:5/605: mknod d2/d27/d42/da3/d23/d49/d44/d54/cc4 0 2024-09-21T04:35:23.561 INFO:tasks.workunit.client.0.smithi007.stdout:5/606: readlink d2/d1e/l8a 0 2024-09-21T04:35:23.564 INFO:tasks.workunit.client.0.smithi007.stdout:3/387: creat dc/d22/d29/d32/d4d/f79 x:0 0 0 2024-09-21T04:35:23.566 INFO:tasks.workunit.client.0.smithi007.stdout:3/388: dread - dc/d22/d5a/f75 zero size 2024-09-21T04:35:23.566 INFO:tasks.workunit.client.1.smithi053.stdout:7/955: chown d0/d6f/dbf/d50/d8b/d70/df2/l11e 225496 1 2024-09-21T04:35:23.566 INFO:tasks.workunit.client.0.smithi007.stdout:3/389: write dc/d18/d41/f4a [702594,98856] 0 2024-09-21T04:35:23.569 INFO:tasks.workunit.client.1.smithi053.stdout:7/956: symlink d0/d2b/d36/d64/l142 0 2024-09-21T04:35:23.569 INFO:tasks.workunit.client.1.smithi053.stdout:7/957: readlink d0/d6f/dbf/d50/d84/db8/lcf 0 2024-09-21T04:35:23.570 INFO:tasks.workunit.client.1.smithi053.stdout:7/958: chown d0/d6f/dbf/d50/fa4 36193372 1 2024-09-21T04:35:23.571 INFO:tasks.workunit.client.1.smithi053.stdout:7/959: write d0/d6f/dbf/d95/f13e [26066,130057] 0 2024-09-21T04:35:23.571 INFO:tasks.workunit.client.0.smithi007.stdout:5/607: rename d2/d27/d42/da3/d23/d5d/d9a/fbd to d2/d27/d42/da3/d23/fc5 0 2024-09-21T04:35:23.571 INFO:tasks.workunit.client.0.smithi007.stdout:3/390: rename dc/d22/d29/d32/d4d to dc/d22/d29/d32/d4d/d7a 22 2024-09-21T04:35:23.573 INFO:tasks.workunit.client.1.smithi053.stdout:7/960: fsync d0/d2b/d36/d64/ddb/f78 0 2024-09-21T04:35:23.576 INFO:tasks.workunit.client.0.smithi007.stdout:5/608: truncate d2/d27/d42/da3/d28/f64 1234114 0 2024-09-21T04:35:23.577 INFO:tasks.workunit.client.1.smithi053.stdout:7/961: dread d0/d24/fb3 [0,4194304] 0 2024-09-21T04:35:23.579 INFO:tasks.workunit.client.1.smithi053.stdout:7/962: chown d0/d6f/dbf/d50/d84/f9c 598104 1 2024-09-21T04:35:23.579 INFO:tasks.workunit.client.0.smithi007.stdout:5/609: fdatasync d2/d1e/f6c 0 2024-09-21T04:35:23.580 INFO:tasks.workunit.client.1.smithi053.stdout:7/963: truncate d0/d2b/d36/f5e 326944 0 2024-09-21T04:35:23.585 INFO:tasks.workunit.client.1.smithi053.stdout:7/964: link d0/d6f/dbf/d50/d8b/l13c d0/dcd/l143 0 2024-09-21T04:35:23.587 INFO:tasks.workunit.client.0.smithi007.stdout:3/391: write dc/d18/f45 [1151101,47751] 0 2024-09-21T04:35:23.589 INFO:tasks.workunit.client.1.smithi053.stdout:7/965: creat d0/d6f/dbf/d50/d110/f144 x:0 0 0 2024-09-21T04:35:23.590 INFO:tasks.workunit.client.0.smithi007.stdout:3/392: creat dc/d18/f7b x:0 0 0 2024-09-21T04:35:23.590 INFO:tasks.workunit.client.1.smithi053.stdout:7/966: write d0/d2b/d36/f89 [349118,66150] 0 2024-09-21T04:35:23.590 INFO:tasks.workunit.client.0.smithi007.stdout:5/610: write d2/d27/d42/da3/d1d/f39 [4190442,26089] 0 2024-09-21T04:35:23.591 INFO:tasks.workunit.client.1.smithi053.stdout:7/967: fsync d0/d24/d46/fe3 0 2024-09-21T04:35:23.592 INFO:tasks.workunit.client.0.smithi007.stdout:3/393: read dc/d22/d29/d32/d69/f40 [2930240,58161] 0 2024-09-21T04:35:23.597 INFO:tasks.workunit.client.0.smithi007.stdout:3/394: dread dc/d18/f67 [0,4194304] 0 2024-09-21T04:35:23.598 INFO:tasks.workunit.client.1.smithi053.stdout:7/968: link d0/d24/fd9 d0/dca/f145 0 2024-09-21T04:35:23.600 INFO:tasks.workunit.client.0.smithi007.stdout:5/611: mkdir d2/d27/d42/da3/d23/d5d/dc6 0 2024-09-21T04:35:23.602 INFO:tasks.workunit.client.0.smithi007.stdout:5/612: mknod d2/d27/d3d/d56/cc7 0 2024-09-21T04:35:23.604 INFO:tasks.workunit.client.0.smithi007.stdout:3/395: link dc/d22/d34/l54 dc/d22/d29/d32/d4d/l7c 0 2024-09-21T04:35:23.604 INFO:tasks.workunit.client.1.smithi053.stdout:7/969: dwrite d0/d6f/dbf/d50/d84/fc8 [0,4194304] 0 2024-09-21T04:35:23.605 INFO:tasks.workunit.client.0.smithi007.stdout:2/296: sync 2024-09-21T04:35:23.605 INFO:tasks.workunit.client.0.smithi007.stdout:3/396: readlink dc/d22/d29/d32/d38/l72 0 2024-09-21T04:35:23.605 INFO:tasks.workunit.client.0.smithi007.stdout:5/613: unlink d2/d27/d42/da3/d23/d5d/d61/f65 0 2024-09-21T04:35:23.610 INFO:tasks.workunit.client.0.smithi007.stdout:2/297: fdatasync d0/f18 0 2024-09-21T04:35:23.612 INFO:tasks.workunit.client.0.smithi007.stdout:3/397: mkdir dc/d22/d29/d32/d4d/d7d 0 2024-09-21T04:35:23.612 INFO:tasks.workunit.client.0.smithi007.stdout:5/614: fsync d2/f66 0 2024-09-21T04:35:23.615 INFO:tasks.workunit.client.0.smithi007.stdout:2/298: rmdir d0/d2 39 2024-09-21T04:35:23.620 INFO:tasks.workunit.client.0.smithi007.stdout:3/398: rename dc/d22/c44 to dc/d22/d29/d32/d4f/c7e 0 2024-09-21T04:35:23.621 INFO:tasks.workunit.client.0.smithi007.stdout:3/399: write dc/d18/f65 [911723,98113] 0 2024-09-21T04:35:23.622 INFO:tasks.workunit.client.0.smithi007.stdout:2/299: symlink d0/d11/d31/d46/d48/l4a 0 2024-09-21T04:35:23.624 INFO:tasks.workunit.client.1.smithi053.stdout:7/970: write d0/d6f/dbf/da2/dd7/fea [351710,10114] 0 2024-09-21T04:35:23.629 INFO:tasks.workunit.client.0.smithi007.stdout:2/300: dwrite d0/d11/d31/d39/f3a [0,4194304] 0 2024-09-21T04:35:23.629 INFO:tasks.workunit.client.0.smithi007.stdout:3/400: getdents dc/d22/d29 0 2024-09-21T04:35:23.630 INFO:tasks.workunit.client.0.smithi007.stdout:5/615: write d2/d1e/f37 [57282,96808] 0 2024-09-21T04:35:23.633 INFO:tasks.workunit.client.0.smithi007.stdout:2/301: write d0/d11/d31/d46/f47 [721649,113119] 0 2024-09-21T04:35:23.636 INFO:tasks.workunit.client.0.smithi007.stdout:3/401: creat dc/d18/f7f x:0 0 0 2024-09-21T04:35:23.636 INFO:tasks.workunit.client.1.smithi053.stdout:7/971: link d0/dcd/l143 d0/d6f/dbf/d50/d8b/d70/d141/l146 0 2024-09-21T04:35:23.637 INFO:tasks.workunit.client.0.smithi007.stdout:3/402: chown dc/d22/f24 0 1 2024-09-21T04:35:23.637 INFO:tasks.workunit.client.1.smithi053.stdout:7/972: fsync d0/d2b/f134 0 2024-09-21T04:35:23.638 INFO:tasks.workunit.client.0.smithi007.stdout:2/302: creat d0/d11/f4b x:0 0 0 2024-09-21T04:35:23.639 INFO:tasks.workunit.client.0.smithi007.stdout:3/403: mkdir dc/d22/d29/d32/d38/d80 0 2024-09-21T04:35:23.640 INFO:tasks.workunit.client.0.smithi007.stdout:5/616: creat d2/d27/fc8 x:0 0 0 2024-09-21T04:35:23.642 INFO:tasks.workunit.client.1.smithi053.stdout:7/973: creat d0/d6f/dbf/d5c/f147 x:0 0 0 2024-09-21T04:35:23.647 INFO:tasks.workunit.client.1.smithi053.stdout:7/974: rename d0/d6f/l125 to d0/d6f/dbf/d50/d8b/d70/dc1/d12d/d13d/l148 0 2024-09-21T04:35:23.647 INFO:tasks.workunit.client.0.smithi007.stdout:3/404: mknod dc/d6c/c81 0 2024-09-21T04:35:23.648 INFO:tasks.workunit.client.0.smithi007.stdout:3/405: chown dc/d18/f31 0 1 2024-09-21T04:35:23.650 INFO:tasks.workunit.client.0.smithi007.stdout:3/406: mknod dc/d22/d29/d32/d69/d6d/c82 0 2024-09-21T04:35:23.655 INFO:tasks.workunit.client.0.smithi007.stdout:2/303: write d0/d11/d31/d39/f44 [1012201,72015] 0 2024-09-21T04:35:23.657 INFO:tasks.workunit.client.0.smithi007.stdout:5/617: write d2/d27/d42/da3/d1d/d20/f82 [472991,100363] 0 2024-09-21T04:35:23.660 INFO:tasks.workunit.client.0.smithi007.stdout:3/407: getdents dc/d22/d29/d32/d69 0 2024-09-21T04:35:23.661 INFO:tasks.workunit.client.0.smithi007.stdout:3/408: read - dc/d18/f7b zero size 2024-09-21T04:35:23.666 INFO:tasks.workunit.client.1.smithi053.stdout:7/975: dwrite d0/d6f/dbf/d5c/dac/fce [0,4194304] 0 2024-09-21T04:35:23.669 INFO:tasks.workunit.client.1.smithi053.stdout:7/976: creat d0/d6f/dbf/d50/d8b/d70/dc1/f149 x:0 0 0 2024-09-21T04:35:23.671 INFO:tasks.workunit.client.0.smithi007.stdout:2/304: dwrite d0/d2/f6 [0,4194304] 0 2024-09-21T04:35:23.674 INFO:tasks.workunit.client.0.smithi007.stdout:5/618: truncate d2/d27/d42/da3/d1d/d20/f6d 1306628 0 2024-09-21T04:35:23.679 INFO:tasks.workunit.client.1.smithi053.stdout:7/977: link d0/f13 d0/d6f/dbf/d95/f14a 0 2024-09-21T04:35:23.681 INFO:tasks.workunit.client.0.smithi007.stdout:3/409: dread dc/f10 [4194304,4194304] 0 2024-09-21T04:35:23.683 INFO:tasks.workunit.client.0.smithi007.stdout:5/619: unlink d2/f66 0 2024-09-21T04:35:23.686 INFO:tasks.workunit.client.1.smithi053.stdout:7/978: fsync d0/d24/fd9 0 2024-09-21T04:35:23.687 INFO:tasks.workunit.client.0.smithi007.stdout:2/305: rename d0/d11/l2d to d0/d32/l4c 0 2024-09-21T04:35:23.691 INFO:tasks.workunit.client.0.smithi007.stdout:3/410: dwrite dc/d22/d29/d32/d4d/f57 [0,4194304] 0 2024-09-21T04:35:23.695 INFO:tasks.workunit.client.0.smithi007.stdout:3/411: creat dc/d22/d34/f83 x:0 0 0 2024-09-21T04:35:23.698 INFO:tasks.workunit.client.0.smithi007.stdout:5/620: rename d2/d27/d42/da3/d1d/l5c to d2/lc9 0 2024-09-21T04:35:23.699 INFO:tasks.workunit.client.0.smithi007.stdout:3/412: chown dc/d18/c63 97509 1 2024-09-21T04:35:23.704 INFO:tasks.workunit.client.0.smithi007.stdout:5/621: symlink d2/d27/d42/da3/d23/d5d/lca 0 2024-09-21T04:35:23.709 INFO:tasks.workunit.client.0.smithi007.stdout:5/622: chown d2/d27/d42/d53/c7c 76 1 2024-09-21T04:35:23.710 INFO:tasks.workunit.client.0.smithi007.stdout:3/413: dwrite dc/d22/f59 [4194304,4194304] 0 2024-09-21T04:35:23.715 INFO:tasks.workunit.client.0.smithi007.stdout:3/414: mkdir dc/d22/d29/d32/d4d/d84 0 2024-09-21T04:35:23.718 INFO:tasks.workunit.client.0.smithi007.stdout:2/306: write d0/f7 [1764604,123258] 0 2024-09-21T04:35:23.723 INFO:tasks.workunit.client.1.smithi053.stdout:7/979: dwrite d0/d6f/dbf/d50/f12e [4194304,4194304] 0 2024-09-21T04:35:23.727 INFO:tasks.workunit.client.0.smithi007.stdout:5/623: dread d2/f5a [0,4194304] 0 2024-09-21T04:35:23.727 INFO:tasks.workunit.client.0.smithi007.stdout:3/415: link dc/d22/d29/d32/d69/f49 dc/d22/d5a/d71/f85 0 2024-09-21T04:35:23.728 INFO:tasks.workunit.client.1.smithi053.stdout:7/980: chown d0/d2b/l4c 188240 1 2024-09-21T04:35:23.732 INFO:tasks.workunit.client.0.smithi007.stdout:5/624: write d2/d1e/f37 [728297,75941] 0 2024-09-21T04:35:23.733 INFO:tasks.workunit.client.0.smithi007.stdout:3/416: readlink dc/d22/d29/d32/d4d/l7c 0 2024-09-21T04:35:23.734 INFO:tasks.workunit.client.1.smithi053.stdout:7/981: link d0/d2b/d36/f89 d0/dcd/f14b 0 2024-09-21T04:35:23.736 INFO:tasks.workunit.client.0.smithi007.stdout:2/307: dwrite d0/d11/d31/d39/f3e [0,4194304] 0 2024-09-21T04:35:23.736 INFO:tasks.workunit.client.0.smithi007.stdout:3/417: write dc/d18/f58 [4602480,111593] 0 2024-09-21T04:35:23.738 INFO:tasks.workunit.client.1.smithi053.stdout:7/982: unlink d0/d6f/dbf/d50/d84/ffe 0 2024-09-21T04:35:23.739 INFO:tasks.workunit.client.0.smithi007.stdout:2/308: write d0/d2/f6 [1449029,32706] 0 2024-09-21T04:35:23.739 INFO:tasks.workunit.client.0.smithi007.stdout:5/625: dread d2/d27/d42/da3/d23/d49/d44/f84 [0,4194304] 0 2024-09-21T04:35:23.743 INFO:tasks.workunit.client.0.smithi007.stdout:2/309: symlink d0/d11/l4d 0 2024-09-21T04:35:23.745 INFO:tasks.workunit.client.0.smithi007.stdout:2/310: stat d0/d11/d31/d46/f47 0 2024-09-21T04:35:23.745 INFO:tasks.workunit.client.1.smithi053.stdout:7/983: mknod d0/d2b/d36/d64/d9e/d12c/c14c 0 2024-09-21T04:35:23.746 INFO:tasks.workunit.client.0.smithi007.stdout:5/626: read d2/f55 [2602504,59937] 0 2024-09-21T04:35:23.746 INFO:tasks.workunit.client.1.smithi053.stdout:7/984: stat d0/d2b/d36/d64/d9e/d12c/c14c 0 2024-09-21T04:35:23.747 INFO:tasks.workunit.client.0.smithi007.stdout:3/418: link dc/d22/d29/d32/d4d/l7c dc/d22/d29/d32/d38/l86 0 2024-09-21T04:35:23.750 INFO:tasks.workunit.client.0.smithi007.stdout:3/419: truncate dc/d22/d29/d32/f77 672662 0 2024-09-21T04:35:23.753 INFO:tasks.workunit.client.0.smithi007.stdout:2/311: creat d0/d11/f4e x:0 0 0 2024-09-21T04:35:23.753 INFO:tasks.workunit.client.0.smithi007.stdout:5/627: dwrite d2/d27/d42/da3/d28/f98 [0,4194304] 0 2024-09-21T04:35:23.756 INFO:tasks.workunit.client.0.smithi007.stdout:3/420: mknod dc/d22/d5a/c87 0 2024-09-21T04:35:23.758 INFO:tasks.workunit.client.0.smithi007.stdout:3/421: write dc/d22/f59 [4586971,82292] 0 2024-09-21T04:35:23.760 INFO:tasks.workunit.client.0.smithi007.stdout:2/312: rename d0/f19 to d0/d11/d31/d39/f4f 0 2024-09-21T04:35:23.762 INFO:tasks.workunit.client.0.smithi007.stdout:5/628: mknod d2/d27/d3d/dc2/ccb 0 2024-09-21T04:35:23.765 INFO:tasks.workunit.client.0.smithi007.stdout:2/313: dread d0/fd [4194304,4194304] 0 2024-09-21T04:35:23.766 INFO:tasks.workunit.client.0.smithi007.stdout:5/629: dread d2/d27/d42/da3/d1d/d20/f82 [0,4194304] 0 2024-09-21T04:35:23.772 INFO:tasks.workunit.client.1.smithi053.stdout:7/985: write d0/d6f/dbf/d5c/f8a [1691196,46945] 0 2024-09-21T04:35:23.773 INFO:tasks.workunit.client.1.smithi053.stdout:7/986: write d0/d24/d46/fe3 [4974842,68550] 0 2024-09-21T04:35:23.773 INFO:tasks.workunit.client.1.smithi053.stdout:7/987: fsync d0/dca/ff7 0 2024-09-21T04:35:23.779 INFO:tasks.workunit.client.1.smithi053.stdout:7/988: getdents d0/d2b/d36/d64/ddb 0 2024-09-21T04:35:23.780 INFO:tasks.workunit.client.1.smithi053.stdout:7/989: fsync d0/fb 0 2024-09-21T04:35:23.780 INFO:tasks.workunit.client.1.smithi053.stdout:7/990: stat d0/d2b/d36/f3c 0 2024-09-21T04:35:23.781 INFO:tasks.workunit.client.1.smithi053.stdout:7/991: chown d0/fd 0 1 2024-09-21T04:35:23.781 INFO:tasks.workunit.client.0.smithi007.stdout:5/630: dread d2/f55 [0,4194304] 0 2024-09-21T04:35:23.782 INFO:tasks.workunit.client.1.smithi053.stdout:7/992: dread - d0/d2b/d36/d64/dc0/fcb zero size 2024-09-21T04:35:23.783 INFO:tasks.workunit.client.0.smithi007.stdout:3/422: dwrite dc/d18/f28 [0,4194304] 0 2024-09-21T04:35:23.783 INFO:tasks.workunit.client.1.smithi053.stdout:7/993: write d0/d6f/dbf/da2/f139 [403920,55529] 0 2024-09-21T04:35:23.784 INFO:tasks.workunit.client.0.smithi007.stdout:2/314: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:23.785 INFO:tasks.workunit.client.1.smithi053.stdout:7/994: readlink d0/d2b/lbc 0 2024-09-21T04:35:23.787 INFO:tasks.workunit.client.1.smithi053.stdout:7/995: read d0/dca/f145 [273250,88905] 0 2024-09-21T04:35:23.787 INFO:tasks.workunit.client.0.smithi007.stdout:5/631: mknod d2/d27/d42/da3/d23/da1/ccc 0 2024-09-21T04:35:23.790 INFO:tasks.workunit.client.0.smithi007.stdout:2/315: creat d0/d11/d31/d39/f50 x:0 0 0 2024-09-21T04:35:23.791 INFO:tasks.workunit.client.1.smithi053.stdout:7/996: creat d0/d6f/dbf/d50/d8b/d70/dc1/f14d x:0 0 0 2024-09-21T04:35:23.792 INFO:tasks.workunit.client.0.smithi007.stdout:5/632: mkdir d2/d27/d42/d53/daf/dcd 0 2024-09-21T04:35:23.794 INFO:tasks.workunit.client.0.smithi007.stdout:5/633: chown d2/d27/d42/d53/c7c 27 1 2024-09-21T04:35:23.795 INFO:tasks.workunit.client.1.smithi053.stdout:7/997: mknod d0/d6f/dbf/c14e 0 2024-09-21T04:35:23.798 INFO:tasks.workunit.client.1.smithi053.stdout:7/998: creat d0/d6f/dbf/d50/d84/f14f x:0 0 0 2024-09-21T04:35:23.800 INFO:tasks.workunit.client.1.smithi053.stdout:7/999: unlink d0/l38 0 2024-09-21T04:35:23.801 INFO:tasks.workunit.client.0.smithi007.stdout:3/423: truncate dc/d22/f59 1092637 0 2024-09-21T04:35:23.803 INFO:tasks.workunit.client.0.smithi007.stdout:3/424: mknod dc/d22/d34/c88 0 2024-09-21T04:35:23.803 INFO:tasks.workunit.client.1.smithi053.stderr:+ rm -rf -- ./tmp.92DWlWLYWz 2024-09-21T04:35:23.810 INFO:tasks.workunit.client.0.smithi007.stdout:3/425: dwrite dc/d18/d26/f51 [0,4194304] 0 2024-09-21T04:35:23.814 INFO:tasks.workunit.client.0.smithi007.stdout:3/426: mkdir dc/d22/d5a/d71/d89 0 2024-09-21T04:35:23.819 INFO:tasks.workunit.client.0.smithi007.stdout:0/343: sync 2024-09-21T04:35:23.820 INFO:tasks.workunit.client.0.smithi007.stdout:3/427: unlink dc/l46 0 2024-09-21T04:35:23.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/344: dwrite d6/d16/d47/f6b [0,4194304] 0 2024-09-21T04:35:23.832 INFO:tasks.workunit.client.0.smithi007.stdout:0/345: chown d6/d16/f28 26354628 1 2024-09-21T04:35:23.834 INFO:tasks.workunit.client.0.smithi007.stdout:0/346: unlink d6/f12 0 2024-09-21T04:35:23.874 INFO:tasks.workunit.client.0.smithi007.stdout:0/347: rename d6/d16/d18/d2e/f4a to d6/d16/d18/d2e/d3f/f7c 0 2024-09-21T04:35:23.875 INFO:tasks.workunit.client.0.smithi007.stdout:0/348: write d6/d16/d52/f78 [121679,7503] 0 2024-09-21T04:35:23.887 INFO:tasks.workunit.client.0.smithi007.stdout:0/349: dread d6/f5a [0,4194304] 0 2024-09-21T04:35:23.892 INFO:tasks.workunit.client.0.smithi007.stdout:0/350: link d6/d16/d18/d39/d76/c77 d6/d16/d18/d27/c7d 0 2024-09-21T04:35:23.892 INFO:tasks.workunit.client.0.smithi007.stdout:0/351: readlink d6/d16/d57/l74 0 2024-09-21T04:35:23.894 INFO:tasks.workunit.client.0.smithi007.stdout:0/352: symlink d6/d16/d47/l7e 0 2024-09-21T04:35:23.899 INFO:tasks.workunit.client.0.smithi007.stdout:0/353: link d6/d16/d18/d27/c7d d6/d16/d18/c7f 0 2024-09-21T04:35:23.901 INFO:tasks.workunit.client.0.smithi007.stdout:0/354: rename d6/d16/d18/d27/l70 to d6/d16/d18/d39/l80 0 2024-09-21T04:35:23.903 INFO:tasks.workunit.client.0.smithi007.stdout:0/355: creat d6/d16/d52/f81 x:0 0 0 2024-09-21T04:35:23.938 INFO:tasks.workunit.client.0.smithi007.stdout:0/356: getdents d6/d16/d18/d27 0 2024-09-21T04:35:23.940 INFO:tasks.workunit.client.0.smithi007.stdout:0/357: mknod d6/d16/d47/c82 0 2024-09-21T04:35:23.988 INFO:tasks.workunit.client.0.smithi007.stdout:0/358: dwrite d6/d16/d52/f5c [0,4194304] 0 2024-09-21T04:35:23.989 INFO:tasks.workunit.client.0.smithi007.stdout:0/359: chown d6/d16/d47/f4b 3182866 1 2024-09-21T04:35:23.992 INFO:tasks.workunit.client.0.smithi007.stdout:0/360: symlink d6/d16/d18/d2e/l83 0 2024-09-21T04:35:23.994 INFO:tasks.workunit.client.0.smithi007.stdout:0/361: write d6/d16/d52/d6d/f6f [324196,121363] 0 2024-09-21T04:35:23.997 INFO:tasks.workunit.client.0.smithi007.stdout:0/362: rename d6/d16/d18/c3c to d6/c84 0 2024-09-21T04:35:24.004 INFO:tasks.workunit.client.0.smithi007.stdout:0/363: dwrite d6/d16/d47/f6b [0,4194304] 0 2024-09-21T04:35:24.005 INFO:tasks.workunit.client.0.smithi007.stdout:0/364: stat d6/d16/d52 0 2024-09-21T04:35:24.007 INFO:tasks.workunit.client.0.smithi007.stdout:0/365: fdatasync d6/f5a 0 2024-09-21T04:35:24.013 INFO:tasks.workunit.client.0.smithi007.stdout:0/366: dwrite d6/d16/d18/d27/f29 [0,4194304] 0 2024-09-21T04:35:24.020 INFO:tasks.workunit.client.0.smithi007.stdout:0/367: dwrite d6/fd [0,4194304] 0 2024-09-21T04:35:24.024 INFO:tasks.workunit.client.0.smithi007.stdout:0/368: symlink d6/d16/d52/d6d/l85 0 2024-09-21T04:35:24.026 INFO:tasks.workunit.client.0.smithi007.stdout:0/369: readlink d6/d16/d18/d39/l80 0 2024-09-21T04:35:24.028 INFO:tasks.workunit.client.0.smithi007.stdout:0/370: getdents d6/d75 0 2024-09-21T04:35:24.034 INFO:tasks.workunit.client.0.smithi007.stdout:0/371: dwrite d6/f38 [0,4194304] 0 2024-09-21T04:35:24.038 INFO:tasks.workunit.client.0.smithi007.stdout:0/372: creat d6/d16/f86 x:0 0 0 2024-09-21T04:35:24.040 INFO:tasks.workunit.client.0.smithi007.stdout:0/373: chown d6/d16/d18/d39/d76/c77 1094 1 2024-09-21T04:35:24.041 INFO:tasks.workunit.client.0.smithi007.stdout:0/374: readlink d6/d16/d18/d27/l36 0 2024-09-21T04:35:24.042 INFO:tasks.workunit.client.0.smithi007.stdout:0/375: fsync d6/d16/d18/d2e/f56 0 2024-09-21T04:35:24.042 INFO:tasks.workunit.client.0.smithi007.stdout:7/371: sync 2024-09-21T04:35:24.043 INFO:tasks.workunit.client.0.smithi007.stdout:0/376: read - d6/d16/f5e zero size 2024-09-21T04:35:24.043 INFO:tasks.workunit.client.0.smithi007.stdout:7/372: stat d3/d6/d10/d62/d66 0 2024-09-21T04:35:24.048 INFO:tasks.workunit.client.0.smithi007.stdout:0/377: creat d6/d16/d18/d2e/f87 x:0 0 0 2024-09-21T04:35:24.050 INFO:tasks.workunit.client.0.smithi007.stdout:7/373: truncate d3/d6/d13/d4b/f7d 887965 0 2024-09-21T04:35:24.056 INFO:tasks.workunit.client.0.smithi007.stdout:7/374: fsync d3/f5 0 2024-09-21T04:35:24.057 INFO:tasks.workunit.client.0.smithi007.stdout:7/375: fdatasync d3/d6/d85/f49 0 2024-09-21T04:35:24.063 INFO:tasks.workunit.client.0.smithi007.stdout:0/378: dread d6/d16/f22 [0,4194304] 0 2024-09-21T04:35:24.065 INFO:tasks.workunit.client.0.smithi007.stdout:7/376: dwrite d3/d6/d14/d20/f3f [0,4194304] 0 2024-09-21T04:35:24.073 INFO:tasks.workunit.client.0.smithi007.stdout:7/377: rename d3/d6/d13/l1e to d3/d6/d10/d51/l89 0 2024-09-21T04:35:24.077 INFO:tasks.workunit.client.0.smithi007.stdout:7/378: rename d3/d6/d14/d20/f3f to d3/d6/d2e/d5e/f8a 0 2024-09-21T04:35:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:0/379: truncate d6/d16/f28 1684389 0 2024-09-21T04:35:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:7/379: link d3/d6/d14/d20/d58/c5b d3/d6/c8b 0 2024-09-21T04:35:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:7/380: rmdir d3/d6/d2e/d63/d6a 39 2024-09-21T04:35:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:0/380: dwrite d6/d16/d47/f6b [0,4194304] 0 2024-09-21T04:35:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:0/381: creat d6/f88 x:0 0 0 2024-09-21T04:35:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:0/382: fdatasync d6/d16/d52/d6d/f6f 0 2024-09-21T04:35:24.095 INFO:tasks.workunit.client.0.smithi007.stdout:7/381: creat d3/d6/d2e/d63/d6a/d81/f8c x:0 0 0 2024-09-21T04:35:24.096 INFO:tasks.workunit.client.0.smithi007.stdout:0/383: symlink d6/d16/d57/l89 0 2024-09-21T04:35:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:7/382: creat d3/d6/d10/d4f/f8d x:0 0 0 2024-09-21T04:35:24.115 INFO:tasks.workunit.client.0.smithi007.stdout:0/384: dwrite d6/d16/d18/d27/f35 [0,4194304] 0 2024-09-21T04:35:24.116 INFO:tasks.workunit.client.0.smithi007.stdout:7/383: dread d3/d6/d14/f1b [0,4194304] 0 2024-09-21T04:35:24.118 INFO:tasks.workunit.client.0.smithi007.stdout:0/385: fdatasync d6/d16/d18/d39/f3b 0 2024-09-21T04:35:24.120 INFO:tasks.workunit.client.0.smithi007.stdout:7/384: symlink d3/d6/d10/l8e 0 2024-09-21T04:35:24.125 INFO:tasks.workunit.client.0.smithi007.stdout:0/386: getdents d6/d16 0 2024-09-21T04:35:24.126 INFO:tasks.workunit.client.0.smithi007.stdout:0/387: read d6/d16/d57/f59 [1913092,98844] 0 2024-09-21T04:35:24.130 INFO:tasks.workunit.client.0.smithi007.stdout:0/388: mkdir d6/d16/d47/d8a 0 2024-09-21T04:35:24.133 INFO:tasks.workunit.client.0.smithi007.stdout:0/389: read d6/d16/d47/f6b [3755706,83665] 0 2024-09-21T04:35:24.134 INFO:tasks.workunit.client.0.smithi007.stdout:7/385: truncate d3/d6/f2d 2897 0 2024-09-21T04:35:24.134 INFO:tasks.workunit.client.0.smithi007.stdout:7/386: chown d3/d6/d2e 73 1 2024-09-21T04:35:24.135 INFO:tasks.workunit.client.0.smithi007.stdout:0/390: truncate d6/d16/d57/f59 4945621 0 2024-09-21T04:35:24.139 INFO:tasks.workunit.client.0.smithi007.stdout:0/391: creat d6/d16/d18/d2e/f8b x:0 0 0 2024-09-21T04:35:24.139 INFO:tasks.workunit.client.0.smithi007.stdout:7/387: dread d3/d6/d13/f75 [0,4194304] 0 2024-09-21T04:35:24.141 INFO:tasks.workunit.client.0.smithi007.stdout:0/392: read d6/d16/d18/f5d [1205225,53754] 0 2024-09-21T04:35:24.143 INFO:tasks.workunit.client.0.smithi007.stdout:0/393: creat d6/d16/d18/d39/d76/f8c x:0 0 0 2024-09-21T04:35:24.145 INFO:tasks.workunit.client.0.smithi007.stdout:0/394: mkdir d6/d16/d52/d8d 0 2024-09-21T04:35:24.146 INFO:tasks.workunit.client.0.smithi007.stdout:0/395: chown d6/d16/d18/d39/c40 900751 1 2024-09-21T04:35:24.148 INFO:tasks.workunit.client.0.smithi007.stdout:0/396: chown d6/d16/d18/d2e/f87 26 1 2024-09-21T04:35:24.152 INFO:tasks.workunit.client.0.smithi007.stdout:7/388: truncate d3/d6/d14/d20/f35 4090164 0 2024-09-21T04:35:24.157 INFO:tasks.workunit.client.0.smithi007.stdout:7/389: rmdir d3/d6/d2e/d5e 39 2024-09-21T04:35:24.162 INFO:tasks.workunit.client.0.smithi007.stdout:0/397: truncate d6/f38 307689 0 2024-09-21T04:35:24.167 INFO:tasks.workunit.client.0.smithi007.stdout:7/390: symlink d3/d6/d2e/d5e/l8f 0 2024-09-21T04:35:24.168 INFO:tasks.workunit.client.0.smithi007.stdout:7/391: fsync d3/d6/d10/d4f/f65 0 2024-09-21T04:35:24.171 INFO:tasks.workunit.client.0.smithi007.stdout:0/398: creat d6/d16/d57/d6e/f8e x:0 0 0 2024-09-21T04:35:24.174 INFO:tasks.workunit.client.0.smithi007.stdout:7/392: mkdir d3/d6/d14/d20/d58/d90 0 2024-09-21T04:35:24.176 INFO:tasks.workunit.client.0.smithi007.stdout:7/393: truncate d3/d6/d14/f1b 4982985 0 2024-09-21T04:35:24.176 INFO:tasks.workunit.client.0.smithi007.stdout:7/394: stat d3/d6/d10/c52 0 2024-09-21T04:35:24.179 INFO:tasks.workunit.client.0.smithi007.stdout:0/399: link d6/d16/f5e d6/d75/f8f 0 2024-09-21T04:35:24.179 INFO:tasks.workunit.client.0.smithi007.stdout:7/395: fdatasync d3/d6/d14/f1b 0 2024-09-21T04:35:24.182 INFO:tasks.workunit.client.0.smithi007.stdout:0/400: creat d6/d16/d18/d2e/d3f/f90 x:0 0 0 2024-09-21T04:35:24.184 INFO:tasks.workunit.client.0.smithi007.stdout:0/401: mkdir d6/d91 0 2024-09-21T04:35:24.184 INFO:tasks.workunit.client.0.smithi007.stdout:7/396: mknod d3/d6/d2e/d63/c91 0 2024-09-21T04:35:24.185 INFO:tasks.workunit.client.0.smithi007.stdout:7/397: stat d3/d6/d10/f7f 0 2024-09-21T04:35:24.187 INFO:tasks.workunit.client.0.smithi007.stdout:0/402: creat d6/d16/d18/d2e/f92 x:0 0 0 2024-09-21T04:35:24.188 INFO:tasks.workunit.client.0.smithi007.stdout:0/403: stat d6/d16/l26 0 2024-09-21T04:35:24.191 INFO:tasks.workunit.client.0.smithi007.stdout:7/398: rename d3/d6/d14/d20/l78 to d3/d6/d14/d20/d80/l92 0 2024-09-21T04:35:24.195 INFO:tasks.workunit.client.0.smithi007.stdout:7/399: write d3/d6/d2e/d63/f73 [5071293,21397] 0 2024-09-21T04:35:24.197 INFO:tasks.workunit.client.0.smithi007.stdout:7/400: truncate d3/d6/d2e/f38 853694 0 2024-09-21T04:35:24.206 INFO:tasks.workunit.client.0.smithi007.stdout:0/404: dread d6/d16/d18/d2e/d60/f63 [0,4194304] 0 2024-09-21T04:35:24.215 INFO:tasks.workunit.client.0.smithi007.stdout:0/405: dread d6/f13 [0,4194304] 0 2024-09-21T04:35:24.216 INFO:tasks.workunit.client.0.smithi007.stdout:0/406: symlink d6/d91/l93 0 2024-09-21T04:35:24.229 INFO:tasks.workunit.client.0.smithi007.stdout:0/407: dread d6/d16/d18/d27/f3a [0,4194304] 0 2024-09-21T04:35:24.232 INFO:tasks.workunit.client.0.smithi007.stdout:0/408: rename d6/d16/d18/c7f to d6/d16/d52/c94 0 2024-09-21T04:35:24.251 INFO:tasks.workunit.client.0.smithi007.stdout:0/409: dwrite d6/f38 [0,4194304] 0 2024-09-21T04:35:24.265 INFO:tasks.workunit.client.0.smithi007.stdout:4/465: sync 2024-09-21T04:35:24.266 INFO:tasks.workunit.client.0.smithi007.stdout:4/466: stat d2/d7/c1a 0 2024-09-21T04:35:24.268 INFO:tasks.workunit.client.0.smithi007.stdout:0/410: write d6/d16/d47/f6b [4110573,125592] 0 2024-09-21T04:35:24.269 INFO:tasks.workunit.client.0.smithi007.stdout:0/411: write d6/fd [1455324,21727] 0 2024-09-21T04:35:24.270 INFO:tasks.workunit.client.0.smithi007.stdout:4/467: creat d2/d7/d55/d61/f9a x:0 0 0 2024-09-21T04:35:24.273 INFO:tasks.workunit.client.0.smithi007.stdout:0/412: rename d6/d16/l26 to d6/d16/d18/d39/l95 0 2024-09-21T04:35:24.274 INFO:tasks.workunit.client.0.smithi007.stdout:0/413: write d6/f5a [335970,12635] 0 2024-09-21T04:35:24.279 INFO:tasks.workunit.client.0.smithi007.stdout:4/468: rename d2/d17/d2e/d45/l6b to d2/d5f/l9b 0 2024-09-21T04:35:24.285 INFO:tasks.workunit.client.0.smithi007.stdout:4/469: dwrite d2/d3c/f89 [0,4194304] 0 2024-09-21T04:35:24.303 INFO:tasks.workunit.client.0.smithi007.stdout:0/414: readlink d6/d16/d57/l89 0 2024-09-21T04:35:24.304 INFO:tasks.workunit.client.0.smithi007.stdout:4/470: read d2/d17/d2e/d45/f60 [95067,78669] 0 2024-09-21T04:35:24.307 INFO:tasks.workunit.client.0.smithi007.stdout:4/471: creat d2/d17/d2e/d45/d5e/f9c x:0 0 0 2024-09-21T04:35:24.310 INFO:tasks.workunit.client.0.smithi007.stdout:4/472: rename d2/d17/f52 to d2/d7/d55/d4f/f9d 0 2024-09-21T04:35:24.311 INFO:tasks.workunit.client.0.smithi007.stdout:4/473: stat d2/d7/d64/f7d 0 2024-09-21T04:35:24.312 INFO:tasks.workunit.client.0.smithi007.stdout:4/474: rename d2/d3c to d2/d3c/d9e 22 2024-09-21T04:35:24.318 INFO:tasks.workunit.client.0.smithi007.stdout:4/475: dwrite d2/f23 [0,4194304] 0 2024-09-21T04:35:24.320 INFO:tasks.workunit.client.0.smithi007.stdout:0/415: dread d6/d16/d18/d39/f3b [4194304,4194304] 0 2024-09-21T04:35:24.327 INFO:tasks.workunit.client.0.smithi007.stdout:0/416: write d6/d16/f22 [1528184,56666] 0 2024-09-21T04:35:24.334 INFO:tasks.workunit.client.0.smithi007.stdout:0/417: dread f1 [0,4194304] 0 2024-09-21T04:35:24.337 INFO:tasks.workunit.client.0.smithi007.stdout:0/418: chown d6/d16/c34 10 1 2024-09-21T04:35:24.339 INFO:tasks.workunit.client.0.smithi007.stdout:0/419: write d6/d16/d18/d39/f6a [937593,104667] 0 2024-09-21T04:35:24.340 INFO:tasks.workunit.client.0.smithi007.stdout:0/420: truncate d6/d75/f8f 1013425 0 2024-09-21T04:35:24.343 INFO:tasks.workunit.client.0.smithi007.stdout:0/421: mknod d6/d16/d18/d2e/c96 0 2024-09-21T04:35:24.345 INFO:tasks.workunit.client.0.smithi007.stdout:0/422: write d6/d16/d18/d27/f3a [1725287,38103] 0 2024-09-21T04:35:24.349 INFO:tasks.workunit.client.0.smithi007.stdout:0/423: rename f1 to d6/d16/d18/d27/f97 0 2024-09-21T04:35:24.349 INFO:tasks.workunit.client.0.smithi007.stdout:0/424: stat d6/f5a 0 2024-09-21T04:35:24.353 INFO:tasks.workunit.client.0.smithi007.stdout:0/425: symlink d6/d16/d18/d2e/d3f/l98 0 2024-09-21T04:35:24.357 INFO:tasks.workunit.client.0.smithi007.stdout:0/426: chown d6/c84 579 1 2024-09-21T04:35:24.358 INFO:tasks.workunit.client.0.smithi007.stdout:0/427: fsync d6/d16/d18/f69 0 2024-09-21T04:35:24.366 INFO:tasks.workunit.client.0.smithi007.stdout:0/428: dread d6/d16/f28 [0,4194304] 0 2024-09-21T04:35:24.377 INFO:tasks.workunit.client.0.smithi007.stdout:0/429: dwrite d6/d16/d47/f4b [0,4194304] 0 2024-09-21T04:35:24.379 INFO:tasks.workunit.client.0.smithi007.stdout:0/430: mkdir d6/d16/d47/d99 0 2024-09-21T04:35:24.385 INFO:tasks.workunit.client.0.smithi007.stdout:0/431: dread d6/d16/d47/f4b [0,4194304] 0 2024-09-21T04:35:24.390 INFO:tasks.workunit.client.0.smithi007.stdout:0/432: rename d6/d16/f2a to d6/d16/d18/d39/d76/f9a 0 2024-09-21T04:35:24.402 INFO:tasks.workunit.client.0.smithi007.stdout:0/433: creat d6/d16/d18/d27/f9b x:0 0 0 2024-09-21T04:35:24.402 INFO:tasks.workunit.client.0.smithi007.stdout:0/434: fsync d6/d16/d18/d2e/f8b 0 2024-09-21T04:35:24.410 INFO:tasks.workunit.client.0.smithi007.stdout:0/435: dwrite d6/d16/f86 [0,4194304] 0 2024-09-21T04:35:24.420 INFO:tasks.workunit.client.0.smithi007.stdout:0/436: mkdir d6/d16/d47/d8a/d9c 0 2024-09-21T04:35:24.421 INFO:tasks.workunit.client.0.smithi007.stdout:0/437: truncate d6/d16/d18/f69 950804 0 2024-09-21T04:35:24.425 INFO:tasks.workunit.client.0.smithi007.stdout:0/438: dread d6/fd [0,4194304] 0 2024-09-21T04:35:24.432 INFO:tasks.workunit.client.0.smithi007.stdout:0/439: mkdir d6/d16/d57/d9d 0 2024-09-21T04:35:24.438 INFO:tasks.workunit.client.0.smithi007.stdout:0/440: link d6/d16/d18/d2e/d60/f63 d6/d16/d18/f9e 0 2024-09-21T04:35:24.440 INFO:tasks.workunit.client.0.smithi007.stdout:0/441: symlink d6/d16/d57/d9d/l9f 0 2024-09-21T04:35:24.441 INFO:tasks.workunit.client.0.smithi007.stdout:0/442: creat d6/d16/d47/fa0 x:0 0 0 2024-09-21T04:35:24.442 INFO:tasks.workunit.client.0.smithi007.stdout:0/443: truncate d6/d16/d18/f69 1282959 0 2024-09-21T04:35:24.445 INFO:tasks.workunit.client.0.smithi007.stdout:0/444: readlink d6/d16/d52/d6d/l85 0 2024-09-21T04:35:24.448 INFO:tasks.workunit.client.0.smithi007.stdout:0/445: creat d6/d16/d18/d27/fa1 x:0 0 0 2024-09-21T04:35:24.450 INFO:tasks.workunit.client.0.smithi007.stdout:0/446: symlink d6/d91/la2 0 2024-09-21T04:35:24.457 INFO:tasks.workunit.client.0.smithi007.stdout:0/447: rename d6/d16/d18/d39/d76 to d6/d91/da3 0 2024-09-21T04:35:24.460 INFO:tasks.workunit.client.0.smithi007.stdout:0/448: creat d6/d16/fa4 x:0 0 0 2024-09-21T04:35:24.464 INFO:tasks.workunit.client.0.smithi007.stdout:0/449: rmdir d6/d16/d52/d8d 0 2024-09-21T04:35:24.466 INFO:tasks.workunit.client.0.smithi007.stdout:0/450: dread d6/d16/d52/f78 [0,4194304] 0 2024-09-21T04:35:24.466 INFO:tasks.workunit.client.0.smithi007.stdout:0/451: dread - d6/d16/fa4 zero size 2024-09-21T04:35:24.468 INFO:tasks.workunit.client.0.smithi007.stdout:0/452: write d6/d16/f4d [126992,26399] 0 2024-09-21T04:35:24.470 INFO:tasks.workunit.client.0.smithi007.stdout:0/453: chown d6/d16/c17 207330933 1 2024-09-21T04:35:24.478 INFO:tasks.workunit.client.0.smithi007.stdout:0/454: dwrite d6/d16/d18/d2e/f87 [0,4194304] 0 2024-09-21T04:35:24.495 INFO:tasks.workunit.client.0.smithi007.stdout:0/455: dwrite d6/d16/d18/d2e/d60/f63 [0,4194304] 0 2024-09-21T04:35:24.502 INFO:tasks.workunit.client.0.smithi007.stdout:0/456: creat d6/d16/d57/fa5 x:0 0 0 2024-09-21T04:35:24.503 INFO:tasks.workunit.client.0.smithi007.stdout:0/457: dread - d6/d16/d18/d2e/d3f/f90 zero size 2024-09-21T04:35:24.504 INFO:tasks.workunit.client.0.smithi007.stdout:0/458: dread - d6/d16/d57/fa5 zero size 2024-09-21T04:35:24.508 INFO:tasks.workunit.client.0.smithi007.stdout:0/459: creat d6/d16/d52/d6d/fa6 x:0 0 0 2024-09-21T04:35:24.512 INFO:tasks.workunit.client.0.smithi007.stdout:0/460: rename d6/d16/d18/d27/f33 to d6/d16/d18/d39/fa7 0 2024-09-21T04:35:24.513 INFO:tasks.workunit.client.0.smithi007.stdout:0/461: dread - d6/d16/d18/d27/f9b zero size 2024-09-21T04:35:24.514 INFO:tasks.workunit.client.0.smithi007.stdout:0/462: readlink d6/d16/d18/d2e/d3f/l61 0 2024-09-21T04:35:24.520 INFO:tasks.workunit.client.0.smithi007.stdout:0/463: link d6/d16/d18/d2e/d60/l44 d6/d16/d18/d2e/d3f/la8 0 2024-09-21T04:35:24.522 INFO:tasks.workunit.client.0.smithi007.stdout:0/464: mknod d6/d16/d18/d27/ca9 0 2024-09-21T04:35:24.528 INFO:tasks.workunit.client.0.smithi007.stdout:0/465: dwrite d6/d16/d18/f9e [4194304,4194304] 0 2024-09-21T04:35:24.529 INFO:tasks.workunit.client.0.smithi007.stdout:0/466: write d6/d16/d52/f5c [197758,111964] 0 2024-09-21T04:35:24.535 INFO:tasks.workunit.client.0.smithi007.stdout:0/467: creat d6/d16/d47/d8a/faa x:0 0 0 2024-09-21T04:35:24.546 INFO:tasks.workunit.client.0.smithi007.stdout:0/468: dread d6/d16/d18/d2e/d3f/f43 [0,4194304] 0 2024-09-21T04:35:24.547 INFO:tasks.workunit.client.0.smithi007.stdout:0/469: write d6/d16/fa4 [185073,49522] 0 2024-09-21T04:35:24.548 INFO:tasks.workunit.client.0.smithi007.stdout:9/412: sync 2024-09-21T04:35:24.549 INFO:tasks.workunit.client.0.smithi007.stdout:0/470: write d6/d16/d18/f69 [286775,90764] 0 2024-09-21T04:35:24.557 INFO:tasks.workunit.client.0.smithi007.stdout:9/413: dwrite d13/d18/f1a [4194304,4194304] 0 2024-09-21T04:35:24.558 INFO:tasks.workunit.client.0.smithi007.stdout:0/471: creat d6/d16/d18/fab x:0 0 0 2024-09-21T04:35:24.562 INFO:tasks.workunit.client.0.smithi007.stdout:0/472: rename d6/d16/d18/d39/f54 to d6/d16/d18/d39/fac 0 2024-09-21T04:35:24.564 INFO:tasks.workunit.client.0.smithi007.stdout:9/414: mkdir d13/d89 0 2024-09-21T04:35:24.565 INFO:tasks.workunit.client.0.smithi007.stdout:9/415: chown d13/d18/d60/f88 396 1 2024-09-21T04:35:24.568 INFO:tasks.workunit.client.0.smithi007.stdout:9/416: chown d13/d18/f67 1 1 2024-09-21T04:35:24.570 INFO:tasks.workunit.client.0.smithi007.stdout:0/473: dwrite d6/d16/d18/f69 [0,4194304] 0 2024-09-21T04:35:24.570 INFO:tasks.workunit.client.0.smithi007.stdout:9/417: read - d13/d18/d60/f63 zero size 2024-09-21T04:35:24.575 INFO:tasks.workunit.client.0.smithi007.stdout:9/418: readlink d13/l3e 0 2024-09-21T04:35:24.577 INFO:tasks.workunit.client.0.smithi007.stdout:0/474: symlink d6/d16/d18/d2e/d60/lad 0 2024-09-21T04:35:24.581 INFO:tasks.workunit.client.0.smithi007.stdout:9/419: rename d13/d2a/c2b to d13/d18/d60/d83/c8a 0 2024-09-21T04:35:24.585 INFO:tasks.workunit.client.0.smithi007.stdout:0/475: getdents d6/d16/d57/d6e 0 2024-09-21T04:35:24.587 INFO:tasks.workunit.client.0.smithi007.stdout:0/476: write d6/d16/d18/d2e/d60/f5f [667828,89507] 0 2024-09-21T04:35:24.588 INFO:tasks.workunit.client.0.smithi007.stdout:0/477: truncate d6/d16/d52/d6d/f6f 643424 0 2024-09-21T04:35:24.590 INFO:tasks.workunit.client.0.smithi007.stdout:9/420: write d13/d18/f36 [4063975,98981] 0 2024-09-21T04:35:24.590 INFO:tasks.workunit.client.0.smithi007.stdout:0/478: rmdir d6/d16/d18/d2e/d3f 39 2024-09-21T04:35:24.594 INFO:tasks.workunit.client.0.smithi007.stdout:9/421: truncate d13/d18/f55 419614 0 2024-09-21T04:35:24.598 INFO:tasks.workunit.client.0.smithi007.stdout:0/479: getdents d6/d16/d18/d2e 0 2024-09-21T04:35:24.599 INFO:tasks.workunit.client.0.smithi007.stdout:8/234: sync 2024-09-21T04:35:24.599 INFO:tasks.workunit.client.0.smithi007.stdout:2/316: sync 2024-09-21T04:35:24.599 INFO:tasks.workunit.client.0.smithi007.stdout:1/516: sync 2024-09-21T04:35:24.601 INFO:tasks.workunit.client.0.smithi007.stdout:0/480: dread d6/d16/d18/d2e/d3f/f43 [0,4194304] 0 2024-09-21T04:35:24.602 INFO:tasks.workunit.client.0.smithi007.stdout:2/317: symlink d0/d32/l51 0 2024-09-21T04:35:24.605 INFO:tasks.workunit.client.0.smithi007.stdout:9/422: dread d13/d18/f3c [0,4194304] 0 2024-09-21T04:35:24.607 INFO:tasks.workunit.client.0.smithi007.stdout:0/481: creat d6/d16/d47/d99/fae x:0 0 0 2024-09-21T04:35:24.610 INFO:tasks.workunit.client.0.smithi007.stdout:0/482: mkdir d6/d16/d57/d9d/daf 0 2024-09-21T04:35:24.613 INFO:tasks.workunit.client.0.smithi007.stdout:2/318: rmdir d0/d11/d31/d42 0 2024-09-21T04:35:24.615 INFO:tasks.workunit.client.0.smithi007.stdout:2/319: fdatasync d0/d11/d31/d39/f50 0 2024-09-21T04:35:24.616 INFO:tasks.workunit.client.0.smithi007.stdout:0/483: dwrite d6/f38 [0,4194304] 0 2024-09-21T04:35:24.618 INFO:tasks.workunit.client.0.smithi007.stdout:2/320: write d0/d11/f4b [137816,120088] 0 2024-09-21T04:35:24.618 INFO:tasks.workunit.client.0.smithi007.stdout:1/517: write d5/d6/d21/d2b/d4b/f70 [1773409,117977] 0 2024-09-21T04:35:24.620 INFO:tasks.workunit.client.0.smithi007.stdout:9/423: link d13/d18/l65 d13/d18/d60/d6d/l8b 0 2024-09-21T04:35:24.625 INFO:tasks.workunit.client.0.smithi007.stdout:8/235: dwrite d0/d3/f2b [0,4194304] 0 2024-09-21T04:35:24.627 INFO:tasks.workunit.client.0.smithi007.stdout:8/236: chown d0/d3/d8/de/l1c 69 1 2024-09-21T04:35:24.627 INFO:tasks.workunit.client.0.smithi007.stdout:1/518: creat d5/d6/d21/d3c/fa3 x:0 0 0 2024-09-21T04:35:24.630 INFO:tasks.workunit.client.0.smithi007.stdout:2/321: dread d0/d11/f28 [0,4194304] 0 2024-09-21T04:35:24.633 INFO:tasks.workunit.client.0.smithi007.stdout:8/237: rmdir d0 39 2024-09-21T04:35:24.635 INFO:tasks.workunit.client.0.smithi007.stdout:2/322: rmdir d0/d11 39 2024-09-21T04:35:24.636 INFO:tasks.workunit.client.0.smithi007.stdout:2/323: fsync d0/f2e 0 2024-09-21T04:35:24.638 INFO:tasks.workunit.client.0.smithi007.stdout:0/484: getdents d6/d16/d47/d99 0 2024-09-21T04:35:24.638 INFO:tasks.workunit.client.0.smithi007.stdout:9/424: dread d13/d2a/f3d [0,4194304] 0 2024-09-21T04:35:24.640 INFO:tasks.workunit.client.0.smithi007.stdout:0/485: write d6/d16/d47/f6b [812866,45863] 0 2024-09-21T04:35:24.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/519: mkdir d5/d6/d21/da4 0 2024-09-21T04:35:24.642 INFO:tasks.workunit.client.0.smithi007.stdout:0/486: write d6/d16/d18/d2e/f92 [278071,104099] 0 2024-09-21T04:35:24.646 INFO:tasks.workunit.client.0.smithi007.stdout:8/238: dwrite d0/d3/d16/f2c [0,4194304] 0 2024-09-21T04:35:24.649 INFO:tasks.workunit.client.0.smithi007.stdout:2/324: mkdir d0/d11/d31/d52 0 2024-09-21T04:35:24.652 INFO:tasks.workunit.client.0.smithi007.stdout:1/520: rmdir d5/d2f 39 2024-09-21T04:35:24.652 INFO:tasks.workunit.client.0.smithi007.stdout:0/487: dwrite d6/d16/d18/fab [0,4194304] 0 2024-09-21T04:35:24.654 INFO:tasks.workunit.client.0.smithi007.stdout:9/425: rmdir d13/d74/d87 0 2024-09-21T04:35:24.656 INFO:tasks.workunit.client.0.smithi007.stdout:2/325: rename d0/d2/l4 to d0/d2/l53 0 2024-09-21T04:35:24.658 INFO:tasks.workunit.client.0.smithi007.stdout:1/521: creat d5/d45/d5e/d55/d58/fa5 x:0 0 0 2024-09-21T04:35:24.659 INFO:tasks.workunit.client.0.smithi007.stdout:0/488: dread d6/d16/d52/d6d/f6f [0,4194304] 0 2024-09-21T04:35:24.662 INFO:tasks.workunit.client.0.smithi007.stdout:8/239: creat d0/d3/d8/de/d26/d35/f37 x:0 0 0 2024-09-21T04:35:24.662 INFO:tasks.workunit.client.0.smithi007.stdout:8/240: readlink d0/d3/d16/l30 0 2024-09-21T04:35:24.663 INFO:tasks.workunit.client.0.smithi007.stdout:2/326: rename d0/d32/f3f to d0/d11/d31/d46/d48/f54 0 2024-09-21T04:35:24.666 INFO:tasks.workunit.client.0.smithi007.stdout:0/489: creat d6/d16/d57/d6e/fb0 x:0 0 0 2024-09-21T04:35:24.667 INFO:tasks.workunit.client.0.smithi007.stdout:0/490: dread - d6/d16/d18/d2e/f37 zero size 2024-09-21T04:35:24.670 INFO:tasks.workunit.client.0.smithi007.stdout:2/327: creat d0/d11/d31/d46/f55 x:0 0 0 2024-09-21T04:35:24.672 INFO:tasks.workunit.client.0.smithi007.stdout:0/491: creat d6/d16/d57/fb1 x:0 0 0 2024-09-21T04:35:24.673 INFO:tasks.workunit.client.0.smithi007.stdout:8/241: symlink d0/l38 0 2024-09-21T04:35:24.673 INFO:tasks.workunit.client.0.smithi007.stdout:0/492: read d6/d16/d18/d27/f29 [966005,15856] 0 2024-09-21T04:35:24.675 INFO:tasks.workunit.client.0.smithi007.stdout:1/522: rename d5/d45/d5e/d55 to d5/d6/d21/d2b/d4b/da6 0 2024-09-21T04:35:24.677 INFO:tasks.workunit.client.0.smithi007.stdout:2/328: symlink d0/d32/l56 0 2024-09-21T04:35:24.679 INFO:tasks.workunit.client.0.smithi007.stdout:8/242: mknod d0/d3/d8/c39 0 2024-09-21T04:35:24.683 INFO:tasks.workunit.client.0.smithi007.stdout:2/329: mkdir d0/d2/d57 0 2024-09-21T04:35:24.686 INFO:tasks.workunit.client.0.smithi007.stdout:1/523: dwrite d5/d6/d21/d2b/d61/f6e [0,4194304] 0 2024-09-21T04:35:24.689 INFO:tasks.workunit.client.0.smithi007.stdout:2/330: unlink d0/c1d 0 2024-09-21T04:35:24.690 INFO:tasks.workunit.client.0.smithi007.stdout:2/331: write d0/d11/f4e [392317,105610] 0 2024-09-21T04:35:24.690 INFO:tasks.workunit.client.0.smithi007.stdout:0/493: link d6/d16/d18/d2e/d3f/l98 d6/d16/d57/lb2 0 2024-09-21T04:35:24.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:24 smithi053 ceph-mon[35554]: pgmap v33: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 74 MiB/s rd, 226 MiB/s wr, 426 op/s 2024-09-21T04:35:24.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:24 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:24.693 INFO:tasks.workunit.client.0.smithi007.stdout:1/524: rename d5/d6/l98 to d5/d45/d94/da0/la7 0 2024-09-21T04:35:24.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:24 smithi007 ceph-mon[27954]: pgmap v33: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 74 MiB/s rd, 226 MiB/s wr, 426 op/s 2024-09-21T04:35:24.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:24 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:24.695 INFO:tasks.workunit.client.0.smithi007.stdout:2/332: symlink d0/d2/l58 0 2024-09-21T04:35:24.701 INFO:tasks.workunit.client.0.smithi007.stdout:0/494: mknod d6/d75/cb3 0 2024-09-21T04:35:24.701 INFO:tasks.workunit.client.0.smithi007.stdout:2/333: truncate d0/f1c 1222203 0 2024-09-21T04:35:24.703 INFO:tasks.workunit.client.0.smithi007.stdout:1/525: mknod d5/d6/da2/ca8 0 2024-09-21T04:35:24.708 INFO:tasks.workunit.client.0.smithi007.stdout:2/334: rename d0/d32/l51 to d0/d11/l59 0 2024-09-21T04:35:24.713 INFO:tasks.workunit.client.0.smithi007.stdout:8/243: getdents d0/d3/d8/de/d26/d35 0 2024-09-21T04:35:24.716 INFO:tasks.workunit.client.0.smithi007.stdout:1/526: link d5/fe d5/d6/fa9 0 2024-09-21T04:35:24.716 INFO:tasks.workunit.client.0.smithi007.stdout:0/495: truncate d6/d16/d47/f4b 3169760 0 2024-09-21T04:35:24.717 INFO:tasks.workunit.client.0.smithi007.stdout:2/335: creat d0/d2/d57/f5a x:0 0 0 2024-09-21T04:35:24.720 INFO:tasks.workunit.client.0.smithi007.stdout:8/244: write d0/d3/d8/fa [8623619,6912] 0 2024-09-21T04:35:24.724 INFO:tasks.workunit.client.0.smithi007.stdout:0/496: rename d6/d16/d57/d9d/l9f to d6/d16/d18/d2e/lb4 0 2024-09-21T04:35:24.725 INFO:tasks.workunit.client.0.smithi007.stdout:1/527: dread d5/d6/d21/d2b/f5c [0,4194304] 0 2024-09-21T04:35:24.732 INFO:tasks.workunit.client.0.smithi007.stdout:1/528: unlink d5/c9f 0 2024-09-21T04:35:24.739 INFO:tasks.workunit.client.0.smithi007.stdout:8/245: link d0/d3/d8/de/c2a d0/d3/d8/de/c3a 0 2024-09-21T04:35:24.744 INFO:tasks.workunit.client.0.smithi007.stdout:8/246: write d0/d3/d8/f10 [6110524,28555] 0 2024-09-21T04:35:24.755 INFO:tasks.workunit.client.0.smithi007.stdout:2/336: dwrite d0/d11/f20 [0,4194304] 0 2024-09-21T04:35:24.757 INFO:tasks.workunit.client.0.smithi007.stdout:0/497: dwrite d6/d16/d18/d39/f3b [4194304,4194304] 0 2024-09-21T04:35:24.764 INFO:tasks.workunit.client.0.smithi007.stdout:2/337: rename d0/d11/d31/c49 to d0/d11/d31/d52/c5b 0 2024-09-21T04:35:24.765 INFO:tasks.workunit.client.0.smithi007.stdout:8/247: dread d0/d3/d8/de/f17 [0,4194304] 0 2024-09-21T04:35:24.766 INFO:tasks.workunit.client.0.smithi007.stdout:1/529: dwrite d5/d6/d21/d2b/f2d [0,4194304] 0 2024-09-21T04:35:24.783 INFO:tasks.workunit.client.0.smithi007.stdout:2/338: rmdir d0/d11/d31/d52 39 2024-09-21T04:35:24.783 INFO:tasks.workunit.client.0.smithi007.stdout:0/498: link d6/d16/d18/d39/l80 d6/d16/lb5 0 2024-09-21T04:35:24.785 INFO:tasks.workunit.client.0.smithi007.stdout:2/339: stat d0/d11/d31/l43 0 2024-09-21T04:35:24.787 INFO:tasks.workunit.client.0.smithi007.stdout:8/248: mkdir d0/d3/d16/d3b 0 2024-09-21T04:35:24.796 INFO:tasks.workunit.client.0.smithi007.stdout:8/249: symlink d0/d3/l3c 0 2024-09-21T04:35:24.797 INFO:tasks.workunit.client.0.smithi007.stdout:0/499: mknod d6/d16/d18/d2e/d60/cb6 0 2024-09-21T04:35:24.798 INFO:tasks.workunit.client.0.smithi007.stdout:8/250: write d0/d3/d8/f10 [68208,111078] 0 2024-09-21T04:35:24.799 INFO:tasks.workunit.client.0.smithi007.stdout:8/251: write d0/f20 [7040650,19117] 0 2024-09-21T04:35:24.800 INFO:tasks.workunit.client.0.smithi007.stdout:1/530: link d5/d6/d21/d2b/c3d d5/d6/da2/caa 0 2024-09-21T04:35:24.803 INFO:tasks.workunit.client.0.smithi007.stdout:8/252: read d0/f20 [7334303,2894] 0 2024-09-21T04:35:24.803 INFO:tasks.workunit.client.0.smithi007.stdout:0/500: creat d6/d16/d18/d27/fb7 x:0 0 0 2024-09-21T04:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout:0/501: stat d6/d16/d18/d39/l50 0 2024-09-21T04:35:24.808 INFO:tasks.workunit.client.0.smithi007.stdout:8/253: unlink d0/d3/d8/f10 0 2024-09-21T04:35:24.812 INFO:tasks.workunit.client.0.smithi007.stdout:0/502: dwrite d6/d16/d18/fab [4194304,4194304] 0 2024-09-21T04:35:24.814 INFO:tasks.workunit.client.0.smithi007.stdout:1/531: truncate d5/f48 2710757 0 2024-09-21T04:35:24.818 INFO:tasks.workunit.client.0.smithi007.stdout:0/503: creat d6/d16/d52/fb8 x:0 0 0 2024-09-21T04:35:24.822 INFO:tasks.workunit.client.0.smithi007.stdout:1/532: creat d5/d45/d94/fab x:0 0 0 2024-09-21T04:35:24.823 INFO:tasks.workunit.client.0.smithi007.stdout:1/533: fdatasync d5/d45/d5e/f24 0 2024-09-21T04:35:24.826 INFO:tasks.workunit.client.0.smithi007.stdout:0/504: mknod d6/d16/d47/d8a/d9c/cb9 0 2024-09-21T04:35:24.827 INFO:tasks.workunit.client.0.smithi007.stdout:0/505: write d6/f38 [890442,90655] 0 2024-09-21T04:35:24.834 INFO:tasks.workunit.client.0.smithi007.stdout:0/506: dwrite d6/d16/d18/d39/f6a [0,4194304] 0 2024-09-21T04:35:24.839 INFO:tasks.workunit.client.0.smithi007.stdout:1/534: write d5/d45/d94/f7b [352054,27471] 0 2024-09-21T04:35:24.841 INFO:tasks.workunit.client.0.smithi007.stdout:0/507: creat d6/d16/fba x:0 0 0 2024-09-21T04:35:24.842 INFO:tasks.workunit.client.0.smithi007.stdout:0/508: chown d6/d16/d18/d27/f3a 1830 1 2024-09-21T04:35:24.844 INFO:tasks.workunit.client.0.smithi007.stdout:1/535: creat d5/d6/d21/d2b/d4b/da6/d58/fac x:0 0 0 2024-09-21T04:35:24.848 INFO:tasks.workunit.client.0.smithi007.stdout:0/509: creat d6/d16/d18/d2e/d3f/fbb x:0 0 0 2024-09-21T04:35:24.852 INFO:tasks.workunit.client.0.smithi007.stdout:0/510: creat d6/d16/d47/d99/fbc x:0 0 0 2024-09-21T04:35:24.853 INFO:tasks.workunit.client.0.smithi007.stdout:1/536: dread d5/d6/d21/d2b/d4b/f74 [0,4194304] 0 2024-09-21T04:35:24.854 INFO:tasks.workunit.client.0.smithi007.stdout:1/537: dread - d5/d45/d94/f9e zero size 2024-09-21T04:35:24.859 INFO:tasks.workunit.client.0.smithi007.stdout:0/511: rename d6/d16/lb5 to d6/d16/d47/d99/lbd 0 2024-09-21T04:35:24.859 INFO:tasks.workunit.client.0.smithi007.stdout:0/512: write d6/d16/d18/fab [523938,15273] 0 2024-09-21T04:35:24.860 INFO:tasks.workunit.client.0.smithi007.stdout:1/538: fsync d5/d45/d94/f56 0 2024-09-21T04:35:24.864 INFO:tasks.workunit.client.0.smithi007.stdout:0/513: symlink d6/d91/lbe 0 2024-09-21T04:35:24.868 INFO:tasks.workunit.client.0.smithi007.stdout:0/514: mkdir d6/d91/da3/dbf 0 2024-09-21T04:35:24.868 INFO:tasks.workunit.client.0.smithi007.stdout:0/515: dread - d6/d16/d47/d99/fae zero size 2024-09-21T04:35:24.871 INFO:tasks.workunit.client.0.smithi007.stdout:1/539: link d5/d6/d21/d2b/f2d d5/d45/d94/da0/fad 0 2024-09-21T04:35:24.873 INFO:tasks.workunit.client.0.smithi007.stdout:1/540: write d5/d6/d21/d2b/d4b/da6/d58/fa5 [742240,117452] 0 2024-09-21T04:35:24.874 INFO:tasks.workunit.client.0.smithi007.stdout:0/516: creat d6/d16/d18/d39/fc0 x:0 0 0 2024-09-21T04:35:24.883 INFO:tasks.workunit.client.0.smithi007.stdout:1/541: mkdir d5/d45/d94/da0/d99/dae 0 2024-09-21T04:35:24.883 INFO:tasks.workunit.client.0.smithi007.stdout:0/517: dwrite d6/d16/f22 [0,4194304] 0 2024-09-21T04:35:24.884 INFO:tasks.workunit.client.0.smithi007.stdout:1/542: chown d5/d6/d21/d2b/d4b/d8a/d8d 0 1 2024-09-21T04:35:24.887 INFO:tasks.workunit.client.0.smithi007.stdout:1/543: mknod d5/d6/d21/d2b/d4b/da6/d58/caf 0 2024-09-21T04:35:24.891 INFO:tasks.workunit.client.0.smithi007.stdout:0/518: dwrite d6/d16/d18/d39/fac [0,4194304] 0 2024-09-21T04:35:24.891 INFO:tasks.workunit.client.0.smithi007.stdout:1/544: fdatasync d5/d45/f7d 0 2024-09-21T04:35:24.901 INFO:tasks.workunit.client.0.smithi007.stdout:0/519: dwrite d6/d16/d57/d6e/fb0 [0,4194304] 0 2024-09-21T04:35:24.902 INFO:tasks.workunit.client.0.smithi007.stdout:0/520: stat d6/d16/f4d 0 2024-09-21T04:35:24.906 INFO:tasks.workunit.client.0.smithi007.stdout:1/545: creat d5/d45/fb0 x:0 0 0 2024-09-21T04:35:24.910 INFO:tasks.workunit.client.0.smithi007.stdout:1/546: dread - d5/d6/d21/d2b/d4b/f75 zero size 2024-09-21T04:35:24.913 INFO:tasks.workunit.client.0.smithi007.stdout:1/547: creat d5/d45/d94/fb1 x:0 0 0 2024-09-21T04:35:24.917 INFO:tasks.workunit.client.0.smithi007.stdout:0/521: link d6/d16/d52/c94 d6/d16/d47/d8a/cc1 0 2024-09-21T04:35:24.918 INFO:tasks.workunit.client.0.smithi007.stdout:0/522: write d6/d16/d52/d6d/fa6 [1004210,26575] 0 2024-09-21T04:35:24.921 INFO:tasks.workunit.client.0.smithi007.stdout:7/401: sync 2024-09-21T04:35:24.921 INFO:tasks.workunit.client.0.smithi007.stdout:5/634: sync 2024-09-21T04:35:24.921 INFO:tasks.workunit.client.0.smithi007.stdout:6/407: sync 2024-09-21T04:35:24.921 INFO:tasks.workunit.client.0.smithi007.stdout:3/428: sync 2024-09-21T04:35:24.921 INFO:tasks.workunit.client.0.smithi007.stdout:4/476: sync 2024-09-21T04:35:24.922 INFO:tasks.workunit.client.0.smithi007.stdout:1/548: rename d5/d6/d21/d2b/d4b/l52 to d5/d6/lb2 0 2024-09-21T04:35:24.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/523: read d6/d16/d18/d2e/f92 [43068,33726] 0 2024-09-21T04:35:24.926 INFO:tasks.workunit.client.0.smithi007.stdout:6/408: dread d3/f33 [0,4194304] 0 2024-09-21T04:35:24.928 INFO:tasks.workunit.client.0.smithi007.stdout:4/477: mkdir d2/d17/d2e/d9f 0 2024-09-21T04:35:24.929 INFO:tasks.workunit.client.0.smithi007.stdout:4/478: truncate d2/d7/d55/f92 392076 0 2024-09-21T04:35:24.930 INFO:tasks.workunit.client.0.smithi007.stdout:4/479: read d2/f6 [3571804,105848] 0 2024-09-21T04:35:24.935 INFO:tasks.workunit.client.0.smithi007.stdout:7/402: rename d3/d6/d10/d4f/f8d to d3/d6/d2e/d5e/f93 0 2024-09-21T04:35:24.938 INFO:tasks.workunit.client.0.smithi007.stdout:0/524: rmdir d6/d16/d52 39 2024-09-21T04:35:24.938 INFO:tasks.workunit.client.0.smithi007.stdout:0/525: chown d6/d16/d57/c73 207 1 2024-09-21T04:35:24.943 INFO:tasks.workunit.client.0.smithi007.stdout:3/429: creat dc/d22/d5a/d71/d89/f8a x:0 0 0 2024-09-21T04:35:24.944 INFO:tasks.workunit.client.0.smithi007.stdout:3/430: write dc/d22/f24 [440769,3141] 0 2024-09-21T04:35:24.945 INFO:tasks.workunit.client.0.smithi007.stdout:3/431: read dc/d18/f58 [3905159,14047] 0 2024-09-21T04:35:24.952 INFO:tasks.workunit.client.0.smithi007.stdout:6/409: symlink d3/l86 0 2024-09-21T04:35:24.952 INFO:tasks.workunit.client.0.smithi007.stdout:6/410: readlink d3/l86 0 2024-09-21T04:35:24.956 INFO:tasks.workunit.client.0.smithi007.stdout:1/549: write d5/d6/d21/f2c [5611262,103162] 0 2024-09-21T04:35:24.957 INFO:tasks.workunit.client.0.smithi007.stdout:1/550: stat d5/d6/d21/d2b/d61/f6e 0 2024-09-21T04:35:24.959 INFO:tasks.workunit.client.0.smithi007.stdout:4/480: symlink d2/d17/d2e/d45/d8e/la0 0 2024-09-21T04:35:24.960 INFO:tasks.workunit.client.0.smithi007.stdout:4/481: read d2/d17/d18/f84 [2976816,3521] 0 2024-09-21T04:35:24.964 INFO:tasks.workunit.client.0.smithi007.stdout:3/432: creat dc/d22/d29/d32/d4d/f8b x:0 0 0 2024-09-21T04:35:24.965 INFO:tasks.workunit.client.0.smithi007.stdout:3/433: write dc/d22/d29/d32/d38/f60 [1738955,124991] 0 2024-09-21T04:35:24.967 INFO:tasks.workunit.client.0.smithi007.stdout:0/526: link d6/d16/d18/d39/f3b d6/d16/d57/d9d/fc2 0 2024-09-21T04:35:24.969 INFO:tasks.workunit.client.0.smithi007.stdout:3/434: dread dc/d18/f65 [0,4194304] 0 2024-09-21T04:35:24.971 INFO:tasks.workunit.client.0.smithi007.stdout:1/551: mknod d5/d6/d21/d2b/d61/cb3 0 2024-09-21T04:35:24.976 INFO:tasks.workunit.client.0.smithi007.stdout:1/552: dread d5/d6/d21/d2b/d4b/da6/d58/fa5 [0,4194304] 0 2024-09-21T04:35:24.976 INFO:tasks.workunit.client.0.smithi007.stdout:5/635: getdents d2/d27/d42/d53/daf/db0 0 2024-09-21T04:35:24.977 INFO:tasks.workunit.client.0.smithi007.stdout:5/636: chown d2/d1e/f5b 0 1 2024-09-21T04:35:24.981 INFO:tasks.workunit.client.0.smithi007.stdout:0/527: fdatasync d6/d16/f86 0 2024-09-21T04:35:24.982 INFO:tasks.workunit.client.0.smithi007.stdout:0/528: truncate d6/d16/fba 833113 0 2024-09-21T04:35:24.983 INFO:tasks.workunit.client.0.smithi007.stdout:7/403: truncate d3/d6/d2e/d63/f73 2157143 0 2024-09-21T04:35:24.985 INFO:tasks.workunit.client.0.smithi007.stdout:3/435: fsync dc/d22/d5a/d71/f85 0 2024-09-21T04:35:24.986 INFO:tasks.workunit.client.0.smithi007.stdout:3/436: write dc/d18/f42 [184423,118239] 0 2024-09-21T04:35:24.987 INFO:tasks.workunit.client.0.smithi007.stdout:3/437: chown dc/d18/f28 31 1 2024-09-21T04:35:24.990 INFO:tasks.workunit.client.0.smithi007.stdout:1/553: creat d5/d6/d21/d2b/d61/fb4 x:0 0 0 2024-09-21T04:35:24.992 INFO:tasks.workunit.client.0.smithi007.stdout:6/411: truncate d3/de/d12/d19/f20 3048437 0 2024-09-21T04:35:24.997 INFO:tasks.workunit.client.0.smithi007.stdout:0/529: rmdir d6 39 2024-09-21T04:35:24.999 INFO:tasks.workunit.client.0.smithi007.stdout:5/637: write d2/d27/d42/da3/d23/d49/f6a [221570,30998] 0 2024-09-21T04:35:25.003 INFO:tasks.workunit.client.0.smithi007.stdout:5/638: dread d2/f55 [0,4194304] 0 2024-09-21T04:35:25.004 INFO:tasks.workunit.client.0.smithi007.stdout:3/438: write dc/f10 [5358086,43926] 0 2024-09-21T04:35:25.007 INFO:tasks.workunit.client.0.smithi007.stdout:6/412: mknod d3/d3e/c87 0 2024-09-21T04:35:25.011 INFO:tasks.workunit.client.0.smithi007.stdout:3/439: dwrite dc/f10 [0,4194304] 0 2024-09-21T04:35:25.014 INFO:tasks.workunit.client.0.smithi007.stdout:5/639: rename d2/d27/d42/da3/d28/f98 to d2/d27/d42/da3/d1d/d20/fce 0 2024-09-21T04:35:25.016 INFO:tasks.workunit.client.0.smithi007.stdout:3/440: dread dc/d18/f28 [0,4194304] 0 2024-09-21T04:35:25.017 INFO:tasks.workunit.client.0.smithi007.stdout:6/413: unlink d3/d3e/d46/c5a 0 2024-09-21T04:35:25.019 INFO:tasks.workunit.client.0.smithi007.stdout:6/414: dread - d3/d3e/d46/f68 zero size 2024-09-21T04:35:25.024 INFO:tasks.workunit.client.0.smithi007.stdout:5/640: mkdir d2/d27/d42/d53/dcf 0 2024-09-21T04:35:25.026 INFO:tasks.workunit.client.0.smithi007.stdout:7/404: getdents d3/d6/d2e/d63/d6a/d81 0 2024-09-21T04:35:25.026 INFO:tasks.workunit.client.0.smithi007.stdout:0/530: dwrite d6/f5a [4194304,4194304] 0 2024-09-21T04:35:25.027 INFO:tasks.workunit.client.0.smithi007.stdout:6/415: mkdir d3/d3e/d85/d88 0 2024-09-21T04:35:25.029 INFO:tasks.workunit.client.0.smithi007.stdout:1/554: getdents d5/d6/d21 0 2024-09-21T04:35:25.032 INFO:tasks.workunit.client.0.smithi007.stdout:3/441: dwrite dc/d22/d34/f83 [0,4194304] 0 2024-09-21T04:35:25.035 INFO:tasks.workunit.client.0.smithi007.stdout:7/405: write d3/d6/d2e/d5e/f93 [493932,80071] 0 2024-09-21T04:35:25.039 INFO:tasks.workunit.client.0.smithi007.stdout:7/406: fsync d3/d6/d10/d4f/f65 0 2024-09-21T04:35:25.041 INFO:tasks.workunit.client.0.smithi007.stdout:3/442: rmdir dc/d22/d29/d32/d4f 39 2024-09-21T04:35:25.043 INFO:tasks.workunit.client.0.smithi007.stdout:5/641: dwrite d2/f51 [0,4194304] 0 2024-09-21T04:35:25.045 INFO:tasks.workunit.client.0.smithi007.stdout:1/555: rename d5/d6/d21/d2b/d61/c89 to d5/d45/d94/da0/d99/cb5 0 2024-09-21T04:35:25.048 INFO:tasks.workunit.client.0.smithi007.stdout:7/407: creat d3/d6/d10/d62/d66/f94 x:0 0 0 2024-09-21T04:35:25.049 INFO:tasks.workunit.client.0.smithi007.stdout:5/642: fsync d2/f8d 0 2024-09-21T04:35:25.051 INFO:tasks.workunit.client.0.smithi007.stdout:7/408: write d3/d6/d14/f1b [983307,82784] 0 2024-09-21T04:35:25.052 INFO:tasks.workunit.client.0.smithi007.stdout:5/643: stat d2/d27/d42/da3/d28/fa5 0 2024-09-21T04:35:25.053 INFO:tasks.workunit.client.0.smithi007.stdout:3/443: dwrite dc/d22/d29/d32/d69/d6d/f30 [0,4194304] 0 2024-09-21T04:35:25.059 INFO:tasks.workunit.client.0.smithi007.stdout:1/556: mkdir d5/d6/d21/d3c/db6 0 2024-09-21T04:35:25.062 INFO:tasks.workunit.client.0.smithi007.stdout:0/531: dread d6/d16/d18/d2e/d3f/f7c [0,4194304] 0 2024-09-21T04:35:25.063 INFO:tasks.workunit.client.0.smithi007.stdout:5/644: rmdir d2/d27 39 2024-09-21T04:35:25.064 INFO:tasks.workunit.client.0.smithi007.stdout:7/409: creat d3/d6/d14/d20/d58/f95 x:0 0 0 2024-09-21T04:35:25.066 INFO:tasks.workunit.client.0.smithi007.stdout:0/532: chown d6/d16/d18/d27/fa1 912655 1 2024-09-21T04:35:25.066 INFO:tasks.workunit.client.0.smithi007.stdout:3/444: dwrite dc/d22/d29/d32/d4d/f73 [0,4194304] 0 2024-09-21T04:35:25.067 INFO:tasks.workunit.client.0.smithi007.stdout:7/410: chown d3/d6/d13/f47 3264 1 2024-09-21T04:35:25.067 INFO:tasks.workunit.client.0.smithi007.stdout:1/557: creat d5/d6/d21/d2b/d4b/da6/d58/fb7 x:0 0 0 2024-09-21T04:35:25.071 INFO:tasks.workunit.client.0.smithi007.stdout:5/645: truncate d2/d27/d42/da3/d1d/f1f 5130257 0 2024-09-21T04:35:25.071 INFO:tasks.workunit.client.0.smithi007.stdout:1/558: readlink d5/d6/lb2 0 2024-09-21T04:35:25.073 INFO:tasks.workunit.client.0.smithi007.stdout:7/411: rmdir d3/d6 39 2024-09-21T04:35:25.078 INFO:tasks.workunit.client.0.smithi007.stdout:5/646: unlink d2/d27/d42/da3/d1d/d20/c8f 0 2024-09-21T04:35:25.082 INFO:tasks.workunit.client.0.smithi007.stdout:0/533: link d6/d16/d18/d27/f35 d6/d91/da3/dbf/fc3 0 2024-09-21T04:35:25.082 INFO:tasks.workunit.client.0.smithi007.stdout:1/559: symlink d5/d45/d94/da0/d99/dae/lb8 0 2024-09-21T04:35:25.082 INFO:tasks.workunit.client.0.smithi007.stdout:1/560: chown d5/d45/d94/da0/fad 18 1 2024-09-21T04:35:25.083 INFO:tasks.workunit.client.0.smithi007.stdout:7/412: creat d3/d6/d2e/d63/d6a/d81/f96 x:0 0 0 2024-09-21T04:35:25.085 INFO:tasks.workunit.client.0.smithi007.stdout:3/445: dread dc/d22/f24 [12582912,4194304] 0 2024-09-21T04:35:25.087 INFO:tasks.workunit.client.0.smithi007.stdout:7/413: truncate d3/d6/d2e/d63/d6a/d81/f8c 277712 0 2024-09-21T04:35:25.092 INFO:tasks.workunit.client.0.smithi007.stdout:7/414: creat d3/d6/d10/d62/f97 x:0 0 0 2024-09-21T04:35:25.094 INFO:tasks.workunit.client.0.smithi007.stdout:5/647: write d2/d1e/f5b [797383,119241] 0 2024-09-21T04:35:25.095 INFO:tasks.workunit.client.0.smithi007.stdout:5/648: write d2/d1e/f24 [5921585,17191] 0 2024-09-21T04:35:25.102 INFO:tasks.workunit.client.0.smithi007.stdout:1/561: truncate d5/d6/d21/d2b/d4b/da6/f80 3315331 0 2024-09-21T04:35:25.102 INFO:tasks.workunit.client.0.smithi007.stdout:0/534: dwrite d6/d75/f8f [0,4194304] 0 2024-09-21T04:35:25.102 INFO:tasks.workunit.client.0.smithi007.stdout:3/446: dwrite dc/d18/f58 [0,4194304] 0 2024-09-21T04:35:25.104 INFO:tasks.workunit.client.0.smithi007.stdout:0/535: truncate d6/d16/d47/d99/fbc 6541 0 2024-09-21T04:35:25.106 INFO:tasks.workunit.client.0.smithi007.stdout:5/649: mknod d2/d27/d3d/d56/cd0 0 2024-09-21T04:35:25.109 INFO:tasks.workunit.client.0.smithi007.stdout:0/536: creat d6/d16/d47/d8a/fc4 x:0 0 0 2024-09-21T04:35:25.109 INFO:tasks.workunit.client.0.smithi007.stdout:3/447: dread dc/d18/f45 [0,4194304] 0 2024-09-21T04:35:25.110 INFO:tasks.workunit.client.0.smithi007.stdout:7/415: link d3/d6/d10/d51/l7c d3/d6/d14/d20/l98 0 2024-09-21T04:35:25.117 INFO:tasks.workunit.client.0.smithi007.stdout:1/562: truncate d5/d45/d5e/f24 7715229 0 2024-09-21T04:35:25.125 INFO:tasks.workunit.client.0.smithi007.stdout:5/650: creat d2/d27/d42/da3/fd1 x:0 0 0 2024-09-21T04:35:25.129 INFO:tasks.workunit.client.0.smithi007.stdout:5/651: creat d2/d27/d3d/d9e/fd2 x:0 0 0 2024-09-21T04:35:25.130 INFO:tasks.workunit.client.0.smithi007.stdout:7/416: link d3/d6/d2e/l3e d3/l99 0 2024-09-21T04:35:25.133 INFO:tasks.workunit.client.0.smithi007.stdout:0/537: rename d6/d16 to d6/d75/dc5 0 2024-09-21T04:35:25.134 INFO:tasks.workunit.client.0.smithi007.stdout:5/652: symlink d2/d1e/ld3 0 2024-09-21T04:35:25.134 INFO:tasks.workunit.client.0.smithi007.stdout:0/538: write d6/d75/dc5/d18/d27/fb7 [848210,67828] 0 2024-09-21T04:35:25.135 INFO:tasks.workunit.client.0.smithi007.stdout:5/653: dread - d2/d27/d3d/f69 zero size 2024-09-21T04:35:25.136 INFO:tasks.workunit.client.0.smithi007.stdout:7/417: unlink d3/d6/d14/d20/d58/c61 0 2024-09-21T04:35:25.138 INFO:tasks.workunit.client.0.smithi007.stdout:5/654: chown d2/d27/d42/da3/d1d/f6e 66 1 2024-09-21T04:35:25.138 INFO:tasks.workunit.client.0.smithi007.stdout:0/539: mknod d6/d75/dc5/d47/d8a/d9c/cc6 0 2024-09-21T04:35:25.140 INFO:tasks.workunit.client.0.smithi007.stdout:5/655: write d2/d27/d42/da3/d1d/f1f [4014690,36243] 0 2024-09-21T04:35:25.141 INFO:tasks.workunit.client.0.smithi007.stdout:0/540: dread - d6/d91/da3/f8c zero size 2024-09-21T04:35:25.141 INFO:tasks.workunit.client.0.smithi007.stdout:7/418: symlink d3/d6/d14/d20/d80/l9a 0 2024-09-21T04:35:25.142 INFO:tasks.workunit.client.0.smithi007.stdout:1/563: write d5/fe [5818579,13778] 0 2024-09-21T04:35:25.147 INFO:tasks.workunit.client.0.smithi007.stdout:0/541: mknod d6/d75/dc5/d18/d2e/d3f/cc7 0 2024-09-21T04:35:25.148 INFO:tasks.workunit.client.0.smithi007.stdout:1/564: fsync d5/fc 0 2024-09-21T04:35:25.149 INFO:tasks.workunit.client.0.smithi007.stdout:3/448: dwrite dc/d22/d29/d32/d69/d6d/f48 [0,4194304] 0 2024-09-21T04:35:25.150 INFO:tasks.workunit.client.0.smithi007.stdout:0/542: stat d6/d75/dc5/d18/d39/fc0 0 2024-09-21T04:35:25.150 INFO:tasks.workunit.client.0.smithi007.stdout:7/419: chown d3/d6/d14/d20/d58/c5b 3 1 2024-09-21T04:35:25.154 INFO:tasks.workunit.client.0.smithi007.stdout:5/656: rename d2/d27/d42/da3/d23/d2a/d94 to d2/d27/d42/da3/d23/d49/dd4 0 2024-09-21T04:35:25.156 INFO:tasks.workunit.client.0.smithi007.stdout:1/565: rmdir d5/d6 39 2024-09-21T04:35:25.157 INFO:tasks.workunit.client.0.smithi007.stdout:3/449: symlink dc/d22/d5a/l8c 0 2024-09-21T04:35:25.159 INFO:tasks.workunit.client.0.smithi007.stdout:0/543: creat d6/d75/dc5/d57/d6e/fc8 x:0 0 0 2024-09-21T04:35:25.160 INFO:tasks.workunit.client.0.smithi007.stdout:5/657: mkdir d2/d27/d42/da3/d23/d49/dd4/dd5 0 2024-09-21T04:35:25.162 INFO:tasks.workunit.client.0.smithi007.stdout:5/658: write d2/d27/d42/da3/d23/d5d/d61/fa4 [1132446,26377] 0 2024-09-21T04:35:25.164 INFO:tasks.workunit.client.0.smithi007.stdout:1/566: dread d5/d6/d21/d2b/d4b/f74 [0,4194304] 0 2024-09-21T04:35:25.165 INFO:tasks.workunit.client.0.smithi007.stdout:0/544: unlink d6/d75/dc5/d18/d2e/d60/cb6 0 2024-09-21T04:35:25.169 INFO:tasks.workunit.client.0.smithi007.stdout:0/545: dread d6/d75/dc5/d18/d39/f6a [0,4194304] 0 2024-09-21T04:35:25.171 INFO:tasks.workunit.client.0.smithi007.stdout:0/546: write d6/d75/dc5/d18/d27/f9b [687254,67255] 0 2024-09-21T04:35:25.171 INFO:tasks.workunit.client.0.smithi007.stdout:1/567: symlink d5/d6/d21/d2b/lb9 0 2024-09-21T04:35:25.179 INFO:tasks.workunit.client.0.smithi007.stdout:7/420: write d3/d6/d10/d4f/f5c [102977,82094] 0 2024-09-21T04:35:25.180 INFO:tasks.workunit.client.0.smithi007.stdout:1/568: symlink d5/d6/d21/d64/lba 0 2024-09-21T04:35:25.184 INFO:tasks.workunit.client.0.smithi007.stdout:0/547: rmdir d6/d75/dc5/d52/d6d 39 2024-09-21T04:35:25.187 INFO:tasks.workunit.client.0.smithi007.stdout:1/569: rmdir d5 39 2024-09-21T04:35:25.187 INFO:tasks.workunit.client.0.smithi007.stdout:7/421: rmdir d3/d6/d14/d20/d58 39 2024-09-21T04:35:25.188 INFO:tasks.workunit.client.0.smithi007.stdout:5/659: getdents d2/d27/d42/da3/d23/d49/d6f 0 2024-09-21T04:35:25.190 INFO:tasks.workunit.client.0.smithi007.stdout:3/450: dwrite dc/f21 [0,4194304] 0 2024-09-21T04:35:25.190 INFO:tasks.workunit.client.0.smithi007.stdout:0/548: mkdir d6/d75/dc9 0 2024-09-21T04:35:25.193 INFO:tasks.workunit.client.0.smithi007.stdout:1/570: dread - d5/d6/d21/d2b/d4b/da6/d58/fac zero size 2024-09-21T04:35:25.193 INFO:tasks.workunit.client.0.smithi007.stdout:7/422: chown d3/d6/d10/d51/l7c 817 1 2024-09-21T04:35:25.194 INFO:tasks.workunit.client.0.smithi007.stdout:5/660: mknod d2/d27/d42/da3/d1d/cd6 0 2024-09-21T04:35:25.194 INFO:tasks.workunit.client.0.smithi007.stdout:7/423: fsync d3/f4c 0 2024-09-21T04:35:25.201 INFO:tasks.workunit.client.0.smithi007.stdout:1/571: chown d5/d45/d94/da0/d99/cb5 125 1 2024-09-21T04:35:25.204 INFO:tasks.workunit.client.0.smithi007.stdout:3/451: chown dc/d22/d29/d32/d4f/c7e 7251107 1 2024-09-21T04:35:25.204 INFO:tasks.workunit.client.0.smithi007.stdout:0/549: rmdir d6/d75/dc5/d52/d6d 39 2024-09-21T04:35:25.206 INFO:tasks.workunit.client.0.smithi007.stdout:3/452: write dc/d18/d26/f68 [244469,57715] 0 2024-09-21T04:35:25.211 INFO:tasks.workunit.client.0.smithi007.stdout:0/550: mknod d6/d75/dc5/d57/cca 0 2024-09-21T04:35:25.214 INFO:tasks.workunit.client.0.smithi007.stdout:1/572: dread d5/d45/d94/f62 [0,4194304] 0 2024-09-21T04:35:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:7/424: dread d3/d6/d13/f47 [0,4194304] 0 2024-09-21T04:35:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:3/453: dwrite dc/d18/d26/f76 [4194304,4194304] 0 2024-09-21T04:35:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:7/425: creat d3/d6/d10/d4f/f9b x:0 0 0 2024-09-21T04:35:25.226 INFO:tasks.workunit.client.0.smithi007.stdout:0/551: getdents d6/d75/dc5/d47/d8a 0 2024-09-21T04:35:25.227 INFO:tasks.workunit.client.0.smithi007.stdout:0/552: dread - d6/d75/dc5/d52/fb8 zero size 2024-09-21T04:35:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:3/454: dread dc/d22/d29/d32/d69/d6d/f48 [0,4194304] 0 2024-09-21T04:35:25.228 INFO:tasks.workunit.client.0.smithi007.stdout:1/573: link d5/d6/d21/d2b/c6d d5/d6/cbb 0 2024-09-21T04:35:25.230 INFO:tasks.workunit.client.0.smithi007.stdout:3/455: chown dc/d22/d29/d32/d69/l64 63 1 2024-09-21T04:35:25.231 INFO:tasks.workunit.client.0.smithi007.stdout:0/553: symlink d6/d75/dc5/d47/lcb 0 2024-09-21T04:35:25.232 INFO:tasks.workunit.client.0.smithi007.stdout:1/574: read d5/d6/d21/d2b/d4b/da6/f9c [4665412,33649] 0 2024-09-21T04:35:25.233 INFO:tasks.workunit.client.0.smithi007.stdout:0/554: dread - d6/d75/dc5/d47/fa0 zero size 2024-09-21T04:35:25.233 INFO:tasks.workunit.client.0.smithi007.stdout:3/456: mkdir dc/d22/d8d 0 2024-09-21T04:35:25.236 INFO:tasks.workunit.client.0.smithi007.stdout:1/575: mknod d5/d6/d21/da4/cbc 0 2024-09-21T04:35:25.238 INFO:tasks.workunit.client.0.smithi007.stdout:3/457: creat dc/d22/d29/d32/d4d/f8e x:0 0 0 2024-09-21T04:35:25.242 INFO:tasks.workunit.client.0.smithi007.stdout:0/555: dwrite d6/d75/dc5/d18/d2e/d3f/f90 [0,4194304] 0 2024-09-21T04:35:25.243 INFO:tasks.workunit.client.0.smithi007.stdout:1/576: dread d5/d6/d21/d2b/d4b/f66 [0,4194304] 0 2024-09-21T04:35:25.250 INFO:tasks.workunit.client.0.smithi007.stdout:0/556: mknod d6/d91/da3/dbf/ccc 0 2024-09-21T04:35:25.252 INFO:tasks.workunit.client.0.smithi007.stdout:7/426: dwrite d3/d6/d2e/d63/f73 [0,4194304] 0 2024-09-21T04:35:25.253 INFO:tasks.workunit.client.0.smithi007.stdout:7/427: chown d3/d6/d13/l53 198347 1 2024-09-21T04:35:25.258 INFO:tasks.workunit.client.0.smithi007.stdout:3/458: truncate dc/d22/d29/d32/d4d/f73 960407 0 2024-09-21T04:35:25.259 INFO:tasks.workunit.client.0.smithi007.stdout:1/577: creat d5/fbd x:0 0 0 2024-09-21T04:35:25.260 INFO:tasks.workunit.client.0.smithi007.stdout:3/459: truncate dc/d22/d34/f83 5233285 0 2024-09-21T04:35:25.264 INFO:tasks.workunit.client.0.smithi007.stdout:3/460: dread dc/d22/d29/d32/f77 [0,4194304] 0 2024-09-21T04:35:25.268 INFO:tasks.workunit.client.0.smithi007.stdout:7/428: dwrite d3/d6/d14/d20/d58/f87 [0,4194304] 0 2024-09-21T04:35:25.272 INFO:tasks.workunit.client.0.smithi007.stdout:3/461: dread dc/d22/d34/f83 [0,4194304] 0 2024-09-21T04:35:25.284 INFO:tasks.workunit.client.0.smithi007.stdout:3/462: mkdir dc/d22/d29/d32/d8f 0 2024-09-21T04:35:25.285 INFO:tasks.workunit.client.0.smithi007.stdout:7/429: dwrite d3/d6/d2e/f38 [0,4194304] 0 2024-09-21T04:35:25.286 INFO:tasks.workunit.client.0.smithi007.stdout:1/578: dread d5/f5d [0,4194304] 0 2024-09-21T04:35:25.294 INFO:tasks.workunit.client.0.smithi007.stdout:7/430: rename d3/d6/d14/d20/f37 to d3/d6/d14/d20/d80/f9c 0 2024-09-21T04:35:25.299 INFO:tasks.workunit.client.0.smithi007.stdout:7/431: creat d3/d6/d14/d20/d80/f9d x:0 0 0 2024-09-21T04:35:25.300 INFO:tasks.workunit.client.0.smithi007.stdout:7/432: dread - d3/d6/d10/d62/d66/f94 zero size 2024-09-21T04:35:25.301 INFO:tasks.workunit.client.0.smithi007.stdout:3/463: dwrite dc/f23 [0,4194304] 0 2024-09-21T04:35:25.303 INFO:tasks.workunit.client.0.smithi007.stdout:3/464: write dc/d18/f39 [3338017,29250] 0 2024-09-21T04:35:25.303 INFO:tasks.workunit.client.0.smithi007.stdout:1/579: link d5/d6/d21/d2b/d4b/da6/c5a d5/d45/d94/da0/d99/cbe 0 2024-09-21T04:35:25.310 INFO:tasks.workunit.client.0.smithi007.stdout:1/580: symlink d5/d45/d94/lbf 0 2024-09-21T04:35:25.316 INFO:tasks.workunit.client.0.smithi007.stdout:7/433: creat d3/d6/d2e/d63/f9e x:0 0 0 2024-09-21T04:35:25.321 INFO:tasks.workunit.client.0.smithi007.stdout:1/581: rename d5/d45/d94/da0/d99/cbe to d5/d6/d21/d3c/cc0 0 2024-09-21T04:35:25.325 INFO:tasks.workunit.client.0.smithi007.stdout:3/465: getdents dc/d56 0 2024-09-21T04:35:25.327 INFO:tasks.workunit.client.0.smithi007.stdout:7/434: dread d3/d6/d85/f25 [0,4194304] 0 2024-09-21T04:35:25.328 INFO:tasks.workunit.client.0.smithi007.stdout:7/435: chown d3/d6 166 1 2024-09-21T04:35:25.330 INFO:tasks.workunit.client.0.smithi007.stdout:1/582: creat d5/d45/d5e/fc1 x:0 0 0 2024-09-21T04:35:25.331 INFO:tasks.workunit.client.0.smithi007.stdout:1/583: chown d5/d6/d21/d64 1080869 1 2024-09-21T04:35:25.333 INFO:tasks.workunit.client.0.smithi007.stdout:7/436: mknod d3/d6/d10/c9f 0 2024-09-21T04:35:25.333 INFO:tasks.workunit.client.0.smithi007.stdout:1/584: fsync d5/d6/d21/d2b/d4b/da6/f9d 0 2024-09-21T04:35:25.336 INFO:tasks.workunit.client.0.smithi007.stdout:7/437: rmdir d3/d6/d13/d4b 39 2024-09-21T04:35:25.339 INFO:tasks.workunit.client.0.smithi007.stdout:7/438: rmdir d3/d6/d2e/d63/d6a/d81 39 2024-09-21T04:35:25.346 INFO:tasks.workunit.client.0.smithi007.stdout:3/466: write dc/d22/f59 [1224722,120077] 0 2024-09-21T04:35:25.346 INFO:tasks.workunit.client.0.smithi007.stdout:7/439: dwrite d3/d6/d85/f25 [4194304,4194304] 0 2024-09-21T04:35:25.346 INFO:tasks.workunit.client.0.smithi007.stdout:1/585: write d5/d45/d94/f56 [762809,71823] 0 2024-09-21T04:35:25.349 INFO:tasks.workunit.client.0.smithi007.stdout:3/467: chown dc/l20 686425773 1 2024-09-21T04:35:25.351 INFO:tasks.workunit.client.0.smithi007.stdout:3/468: write dc/d22/d29/d32/d4d/f79 [949812,43992] 0 2024-09-21T04:35:25.351 INFO:tasks.workunit.client.0.smithi007.stdout:7/440: creat d3/d6/d2e/d63/d6a/d81/fa0 x:0 0 0 2024-09-21T04:35:25.354 INFO:tasks.workunit.client.0.smithi007.stdout:3/469: write dc/f78 [2354611,77371] 0 2024-09-21T04:35:25.357 INFO:tasks.workunit.client.0.smithi007.stdout:1/586: dwrite d5/d6/d21/f2c [0,4194304] 0 2024-09-21T04:35:25.361 INFO:tasks.workunit.client.0.smithi007.stdout:1/587: creat d5/d6/d21/d64/fc2 x:0 0 0 2024-09-21T04:35:25.365 INFO:tasks.workunit.client.0.smithi007.stdout:3/470: link dc/d22/d29/d32/d69/f49 dc/d22/d5a/f90 0 2024-09-21T04:35:25.368 INFO:tasks.workunit.client.0.smithi007.stdout:3/471: mknod dc/d22/d34/c91 0 2024-09-21T04:35:25.369 INFO:tasks.workunit.client.0.smithi007.stdout:1/588: rename d5/f1f to d5/fc3 0 2024-09-21T04:35:25.374 INFO:tasks.workunit.client.0.smithi007.stdout:1/589: dread d5/d6/d21/d2b/d4b/da6/d58/fa5 [0,4194304] 0 2024-09-21T04:35:25.376 INFO:tasks.workunit.client.0.smithi007.stdout:3/472: dwrite dc/d18/f31 [4194304,4194304] 0 2024-09-21T04:35:25.377 INFO:tasks.workunit.client.0.smithi007.stdout:1/590: symlink d5/d6/d21/d3c/db6/lc4 0 2024-09-21T04:35:25.379 INFO:tasks.workunit.client.0.smithi007.stdout:3/473: mkdir dc/d22/d29/d92 0 2024-09-21T04:35:25.385 INFO:tasks.workunit.client.0.smithi007.stdout:3/474: link dc/d22/f59 dc/d22/d5a/f93 0 2024-09-21T04:35:25.391 INFO:tasks.workunit.client.0.smithi007.stdout:3/475: symlink dc/d22/d29/d32/d38/d80/l94 0 2024-09-21T04:35:25.403 INFO:tasks.workunit.client.0.smithi007.stdout:1/591: write d5/d6/d21/d2b/f51 [802844,100457] 0 2024-09-21T04:35:25.404 INFO:tasks.workunit.client.0.smithi007.stdout:1/592: chown d5/d6/d21/d2b/d4b/f74 115640830 1 2024-09-21T04:35:25.409 INFO:tasks.workunit.client.0.smithi007.stdout:3/476: truncate dc/d18/d26/f51 1765037 0 2024-09-21T04:35:25.411 INFO:tasks.workunit.client.0.smithi007.stdout:3/477: mknod dc/d22/d34/c95 0 2024-09-21T04:35:25.412 INFO:tasks.workunit.client.0.smithi007.stdout:1/593: getdents d5/d6/d21/da4 0 2024-09-21T04:35:25.414 INFO:tasks.workunit.client.0.smithi007.stdout:3/478: mkdir dc/d18/d41/d96 0 2024-09-21T04:35:25.415 INFO:tasks.workunit.client.0.smithi007.stdout:1/594: mkdir d5/d6/dc5 0 2024-09-21T04:35:25.417 INFO:tasks.workunit.client.0.smithi007.stdout:3/479: getdents dc/d22/d29/d32/d69/d6d 0 2024-09-21T04:35:25.419 INFO:tasks.workunit.client.0.smithi007.stdout:3/480: truncate dc/d22/d29/d32/d38/f5c 243096 0 2024-09-21T04:35:25.424 INFO:tasks.workunit.client.0.smithi007.stdout:3/481: dwrite dc/d22/f24 [8388608,4194304] 0 2024-09-21T04:35:25.426 INFO:tasks.workunit.client.0.smithi007.stdout:3/482: write dc/d22/d29/d32/d4f/f5e [1012451,79011] 0 2024-09-21T04:35:25.426 INFO:tasks.workunit.client.0.smithi007.stdout:3/483: readlink dc/d18/l1b 0 2024-09-21T04:35:25.431 INFO:tasks.workunit.client.0.smithi007.stdout:3/484: dread dc/d22/d29/d32/d69/d6d/f48 [0,4194304] 0 2024-09-21T04:35:25.435 INFO:tasks.workunit.client.0.smithi007.stdout:3/485: creat dc/d22/d29/d32/d38/d80/f97 x:0 0 0 2024-09-21T04:35:25.437 INFO:tasks.workunit.client.0.smithi007.stdout:3/486: getdents dc/d22/d5a/d71/d74 0 2024-09-21T04:35:25.440 INFO:tasks.workunit.client.0.smithi007.stdout:3/487: creat dc/d18/d41/d96/f98 x:0 0 0 2024-09-21T04:35:25.441 INFO:tasks.workunit.client.0.smithi007.stdout:3/488: write dc/f23 [5008090,122604] 0 2024-09-21T04:35:25.441 INFO:tasks.workunit.client.0.smithi007.stdout:3/489: stat dc/d22/d8d 0 2024-09-21T04:35:25.449 INFO:tasks.workunit.client.0.smithi007.stdout:3/490: dwrite dc/d22/d29/d32/d69/d6d/f30 [4194304,4194304] 0 2024-09-21T04:35:25.451 INFO:tasks.workunit.client.0.smithi007.stdout:3/491: read dc/d22/d29/d32/d38/f60 [222871,114176] 0 2024-09-21T04:35:25.453 INFO:tasks.workunit.client.0.smithi007.stdout:3/492: fsync dc/d22/d29/d32/d4d/f8e 0 2024-09-21T04:35:25.460 INFO:tasks.workunit.client.0.smithi007.stdout:3/493: link dc/d22/d29/d32/d38/l72 dc/d6c/l99 0 2024-09-21T04:35:25.464 INFO:tasks.workunit.client.0.smithi007.stdout:3/494: mkdir dc/d22/d29/d32/d4d/d7d/d9a 0 2024-09-21T04:35:25.467 INFO:tasks.workunit.client.0.smithi007.stdout:3/495: rename dc/d22/d29/d32/d4f/f5e to dc/d22/d29/d32/d69/d6d/f9b 0 2024-09-21T04:35:25.469 INFO:tasks.workunit.client.0.smithi007.stdout:3/496: creat dc/d22/d5a/f9c x:0 0 0 2024-09-21T04:35:25.472 INFO:tasks.workunit.client.0.smithi007.stdout:3/497: truncate dc/d22/d29/d32/d38/f60 884504 0 2024-09-21T04:35:25.472 INFO:tasks.workunit.client.0.smithi007.stdout:3/498: dread - dc/d18/f7f zero size 2024-09-21T04:35:25.476 INFO:tasks.workunit.client.0.smithi007.stdout:3/499: rename dc/d22/d29/d32/d4d/f57 to dc/d22/d5a/d71/f9d 0 2024-09-21T04:35:25.490 INFO:tasks.workunit.client.0.smithi007.stdout:3/500: write dc/d22/d5a/d71/f85 [903812,37411] 0 2024-09-21T04:35:25.496 INFO:tasks.workunit.client.0.smithi007.stdout:3/501: dwrite dc/d18/f7f [0,4194304] 0 2024-09-21T04:35:25.503 INFO:tasks.workunit.client.0.smithi007.stdout:3/502: creat dc/d22/d29/d32/f9e x:0 0 0 2024-09-21T04:35:25.504 INFO:tasks.workunit.client.0.smithi007.stdout:3/503: truncate dc/d22/f59 1402581 0 2024-09-21T04:35:25.504 INFO:tasks.workunit.client.0.smithi007.stdout:3/504: chown dc/d18/f45 409739 1 2024-09-21T04:35:25.505 INFO:tasks.workunit.client.0.smithi007.stdout:3/505: write dc/d18/f39 [1116043,1341] 0 2024-09-21T04:35:25.508 INFO:tasks.workunit.client.0.smithi007.stdout:3/506: write dc/d22/d5a/d71/f9d [4170001,20017] 0 2024-09-21T04:35:25.509 INFO:tasks.workunit.client.0.smithi007.stdout:3/507: rmdir dc/d22 39 2024-09-21T04:35:25.511 INFO:tasks.workunit.client.0.smithi007.stdout:3/508: symlink dc/d6c/l9f 0 2024-09-21T04:35:25.514 INFO:tasks.workunit.client.0.smithi007.stdout:3/509: creat dc/d22/d29/d32/d4d/fa0 x:0 0 0 2024-09-21T04:35:25.515 INFO:tasks.workunit.client.0.smithi007.stdout:3/510: readlink dc/d22/l3a 0 2024-09-21T04:35:25.527 INFO:tasks.workunit.client.0.smithi007.stdout:3/511: dwrite dc/d22/d5a/f93 [0,4194304] 0 2024-09-21T04:35:25.531 INFO:tasks.workunit.client.0.smithi007.stdout:3/512: mknod dc/d22/d29/d32/d4d/ca1 0 2024-09-21T04:35:25.532 INFO:tasks.workunit.client.0.smithi007.stdout:3/513: write dc/f1c [2756365,10963] 0 2024-09-21T04:35:25.535 INFO:tasks.workunit.client.0.smithi007.stdout:3/514: truncate dc/d22/d29/d32/f77 713113 0 2024-09-21T04:35:25.541 INFO:tasks.workunit.client.0.smithi007.stdout:3/515: dwrite dc/d22/f59 [0,4194304] 0 2024-09-21T04:35:25.546 INFO:tasks.workunit.client.0.smithi007.stdout:3/516: mknod dc/ca2 0 2024-09-21T04:35:25.552 INFO:tasks.workunit.client.0.smithi007.stdout:3/517: rename dc/d18/d26/f76 to dc/d22/d29/d32/fa3 0 2024-09-21T04:35:25.554 INFO:tasks.workunit.client.0.smithi007.stdout:3/518: mkdir dc/d6c/da4 0 2024-09-21T04:35:25.554 INFO:tasks.workunit.client.0.smithi007.stdout:3/519: chown dc/d22/d29/d32/d8f 1145462681 1 2024-09-21T04:35:25.556 INFO:tasks.workunit.client.0.smithi007.stdout:3/520: truncate dc/d22/d29/d32/d4d/f8b 670391 0 2024-09-21T04:35:25.556 INFO:tasks.workunit.client.0.smithi007.stdout:3/521: write dc/d18/f39 [713380,117281] 0 2024-09-21T04:35:25.559 INFO:tasks.workunit.client.0.smithi007.stdout:3/522: creat dc/d18/d41/fa5 x:0 0 0 2024-09-21T04:35:25.562 INFO:tasks.workunit.client.0.smithi007.stdout:3/523: rename dc/d22/d29/d32/d38 to dc/d22/d34/da6 0 2024-09-21T04:35:25.563 INFO:tasks.workunit.client.0.smithi007.stdout:3/524: write dc/d22/d29/d32/d4d/f8b [395436,26397] 0 2024-09-21T04:35:25.576 INFO:tasks.workunit.client.0.smithi007.stdout:3/525: dread dc/d22/d29/d32/d69/f40 [0,4194304] 0 2024-09-21T04:35:25.578 INFO:tasks.workunit.client.0.smithi007.stdout:3/526: mkdir dc/d6c/da7 0 2024-09-21T04:35:25.582 INFO:tasks.workunit.client.0.smithi007.stdout:3/527: dread dc/d18/f58 [0,4194304] 0 2024-09-21T04:35:25.585 INFO:tasks.workunit.client.0.smithi007.stdout:3/528: mkdir dc/d22/d34/da6/d80/da8 0 2024-09-21T04:35:25.586 INFO:tasks.workunit.client.0.smithi007.stdout:3/529: readlink dc/d22/d29/d32/d69/l6b 0 2024-09-21T04:35:25.588 INFO:tasks.workunit.client.0.smithi007.stdout:3/530: mknod dc/d18/d26/ca9 0 2024-09-21T04:35:25.593 INFO:tasks.workunit.client.0.smithi007.stdout:3/531: dwrite dc/d22/d5a/f90 [0,4194304] 0 2024-09-21T04:35:25.594 INFO:tasks.workunit.client.0.smithi007.stdout:3/532: chown dc/l12 241298 1 2024-09-21T04:35:25.595 INFO:tasks.workunit.client.0.smithi007.stdout:3/533: dread - dc/d22/d5a/d71/d89/f8a zero size 2024-09-21T04:35:25.597 INFO:tasks.workunit.client.0.smithi007.stdout:3/534: creat dc/d56/faa x:0 0 0 2024-09-21T04:35:25.598 INFO:tasks.workunit.client.0.smithi007.stdout:3/535: chown dc/d22/d5a/f93 76889214 1 2024-09-21T04:35:25.598 INFO:tasks.workunit.client.0.smithi007.stdout:3/536: chown dc/d22/d29/d32/d69 40061290 1 2024-09-21T04:35:25.601 INFO:tasks.workunit.client.0.smithi007.stdout:3/537: rename dc/d18/d41/d96/f98 to dc/d18/fab 0 2024-09-21T04:35:25.605 INFO:tasks.workunit.client.0.smithi007.stdout:3/538: dread dc/d22/d5a/d71/f85 [0,4194304] 0 2024-09-21T04:35:25.606 INFO:tasks.workunit.client.0.smithi007.stdout:3/539: chown dc/d22/d29/d32/d69/f49 3636 1 2024-09-21T04:35:25.608 INFO:tasks.workunit.client.0.smithi007.stdout:3/540: mkdir dc/d22/d29/dac 0 2024-09-21T04:35:25.608 INFO:tasks.workunit.client.0.smithi007.stdout:3/541: readlink dc/d6c/l9f 0 2024-09-21T04:35:25.610 INFO:tasks.workunit.client.0.smithi007.stdout:3/542: write dc/d22/d29/d32/d69/d6d/f30 [6646356,29949] 0 2024-09-21T04:35:25.614 INFO:tasks.workunit.client.0.smithi007.stdout:3/543: symlink dc/d22/lad 0 2024-09-21T04:35:25.615 INFO:tasks.workunit.client.0.smithi007.stdout:3/544: getdents dc/d6c/da4 0 2024-09-21T04:35:25.616 INFO:tasks.workunit.client.0.smithi007.stdout:3/545: chown dc/d22/d29/d32 55 1 2024-09-21T04:35:25.620 INFO:tasks.workunit.client.0.smithi007.stdout:3/546: getdents dc/d22/d34/da6 0 2024-09-21T04:35:25.622 INFO:tasks.workunit.client.0.smithi007.stdout:3/547: truncate dc/d18/d26/f4b 1036667 0 2024-09-21T04:35:25.623 INFO:tasks.workunit.client.0.smithi007.stdout:3/548: mknod dc/d56/cae 0 2024-09-21T04:35:25.625 INFO:tasks.workunit.client.0.smithi007.stdout:3/549: symlink dc/d6c/da4/laf 0 2024-09-21T04:35:25.626 INFO:tasks.workunit.client.0.smithi007.stdout:3/550: fsync dc/d22/d29/f55 0 2024-09-21T04:35:25.629 INFO:tasks.workunit.client.0.smithi007.stdout:3/551: mknod dc/d22/cb0 0 2024-09-21T04:35:25.631 INFO:tasks.workunit.client.0.smithi007.stdout:3/552: creat dc/d6c/fb1 x:0 0 0 2024-09-21T04:35:25.634 INFO:tasks.workunit.client.0.smithi007.stdout:3/553: creat dc/d18/d41/fb2 x:0 0 0 2024-09-21T04:35:25.636 INFO:tasks.workunit.client.0.smithi007.stdout:3/554: creat dc/d22/d29/d32/d69/d6d/fb3 x:0 0 0 2024-09-21T04:35:25.638 INFO:tasks.workunit.client.0.smithi007.stdout:3/555: unlink dc/ca2 0 2024-09-21T04:35:25.640 INFO:tasks.workunit.client.0.smithi007.stdout:3/556: creat dc/d22/d29/d32/d4d/d7d/d9a/fb4 x:0 0 0 2024-09-21T04:35:25.646 INFO:tasks.workunit.client.0.smithi007.stdout:3/557: dwrite dc/d18/d26/f4b [0,4194304] 0 2024-09-21T04:35:25.649 INFO:tasks.workunit.client.0.smithi007.stdout:3/558: symlink dc/d22/d5a/lb5 0 2024-09-21T04:35:25.650 INFO:tasks.workunit.client.0.smithi007.stdout:3/559: read - dc/d22/d34/da6/d80/f97 zero size 2024-09-21T04:35:25.658 INFO:tasks.workunit.client.0.smithi007.stdout:3/560: dwrite dc/d22/d29/f55 [0,4194304] 0 2024-09-21T04:35:25.659 INFO:tasks.workunit.client.0.smithi007.stdout:3/561: read - dc/d22/d29/d32/d4d/fa0 zero size 2024-09-21T04:35:25.660 INFO:tasks.workunit.client.0.smithi007.stdout:3/562: stat dc/d22/d5a/d71 0 2024-09-21T04:35:25.661 INFO:tasks.workunit.client.0.smithi007.stdout:3/563: dread - dc/d22/d29/d32/d69/d6d/fb3 zero size 2024-09-21T04:35:25.667 INFO:tasks.workunit.client.0.smithi007.stdout:3/564: dread dc/f21 [0,4194304] 0 2024-09-21T04:35:25.667 INFO:tasks.workunit.client.0.smithi007.stdout:3/565: chown dc/d22/d5a/f90 8079309 1 2024-09-21T04:35:25.679 INFO:tasks.workunit.client.0.smithi007.stdout:3/566: dwrite dc/d18/f28 [0,4194304] 0 2024-09-21T04:35:25.694 INFO:tasks.workunit.client.0.smithi007.stdout:3/567: truncate dc/d22/d5a/f90 1773444 0 2024-09-21T04:35:25.696 INFO:tasks.workunit.client.0.smithi007.stdout:3/568: rename dc/c16 to dc/d18/d41/d96/cb6 0 2024-09-21T04:35:25.698 INFO:tasks.workunit.client.0.smithi007.stdout:3/569: dread dc/d22/d34/f83 [4194304,4194304] 0 2024-09-21T04:35:25.699 INFO:tasks.workunit.client.0.smithi007.stdout:3/570: readlink dc/d6c/l99 0 2024-09-21T04:35:25.701 INFO:tasks.workunit.client.0.smithi007.stdout:3/571: mknod dc/d22/d29/d32/d69/cb7 0 2024-09-21T04:35:25.703 INFO:tasks.workunit.client.0.smithi007.stdout:3/572: creat dc/d22/d29/d32/d4d/d7d/d9a/fb8 x:0 0 0 2024-09-21T04:35:25.705 INFO:tasks.workunit.client.0.smithi007.stdout:3/573: mkdir dc/d22/d29/d92/db9 0 2024-09-21T04:35:25.723 INFO:tasks.workunit.client.0.smithi007.stdout:3/574: write dc/d22/d34/da6/f60 [1724275,115328] 0 2024-09-21T04:35:25.728 INFO:tasks.workunit.client.0.smithi007.stdout:3/575: rename dc/d22/d29/d32/d69/d6d/c70 to dc/d18/d41/cba 0 2024-09-21T04:35:25.731 INFO:tasks.workunit.client.0.smithi007.stdout:3/576: mkdir dc/d22/d29/d32/d4d/d84/dbb 0 2024-09-21T04:35:25.733 INFO:tasks.workunit.client.0.smithi007.stdout:3/577: mkdir dc/d22/d34/dbc 0 2024-09-21T04:35:25.734 INFO:tasks.workunit.client.0.smithi007.stdout:3/578: write dc/d18/d41/fa5 [193241,37162] 0 2024-09-21T04:35:25.735 INFO:tasks.workunit.client.0.smithi007.stdout:3/579: stat dc/d22/d5a/l8c 0 2024-09-21T04:35:25.735 INFO:tasks.workunit.client.0.smithi007.stdout:3/580: fsync dc/f78 0 2024-09-21T04:35:25.736 INFO:tasks.workunit.client.0.smithi007.stdout:3/581: stat dc/d18/d41/f4a 0 2024-09-21T04:35:26.167 INFO:tasks.workunit.client.0.smithi007.stdout:9/426: sync 2024-09-21T04:35:26.167 INFO:tasks.workunit.client.0.smithi007.stdout:2/340: sync 2024-09-21T04:35:26.168 INFO:tasks.workunit.client.0.smithi007.stdout:2/341: readlink d0/d2/l33 0 2024-09-21T04:35:26.168 INFO:tasks.workunit.client.0.smithi007.stdout:9/427: read d13/f5b [647410,73346] 0 2024-09-21T04:35:26.173 INFO:tasks.workunit.client.0.smithi007.stdout:9/428: symlink d13/d18/d60/d6d/d75/d7f/l8c 0 2024-09-21T04:35:26.174 INFO:tasks.workunit.client.0.smithi007.stdout:9/429: chown d13/f23 6628 1 2024-09-21T04:35:26.175 INFO:tasks.workunit.client.0.smithi007.stdout:2/342: getdents d0/d11/d31/d39 0 2024-09-21T04:35:26.177 INFO:tasks.workunit.client.0.smithi007.stdout:2/343: read d0/d2/f16 [569287,103114] 0 2024-09-21T04:35:26.179 INFO:tasks.workunit.client.0.smithi007.stdout:2/344: truncate d0/d11/d31/d39/f3e 4615835 0 2024-09-21T04:35:26.183 INFO:tasks.workunit.client.0.smithi007.stdout:2/345: dwrite d0/d11/d31/d39/f3e [0,4194304] 0 2024-09-21T04:35:26.196 INFO:tasks.workunit.client.0.smithi007.stdout:9/430: unlink d13/d18/c77 0 2024-09-21T04:35:26.197 INFO:tasks.workunit.client.0.smithi007.stdout:9/431: dread - d13/d18/f67 zero size 2024-09-21T04:35:26.197 INFO:tasks.workunit.client.0.smithi007.stdout:9/432: chown d13/d5a/c82 19 1 2024-09-21T04:35:26.197 INFO:tasks.workunit.client.0.smithi007.stdout:9/433: readlink d13/l3e 0 2024-09-21T04:35:26.198 INFO:tasks.workunit.client.0.smithi007.stdout:9/434: fdatasync d13/d18/d60/f88 0 2024-09-21T04:35:26.199 INFO:tasks.workunit.client.0.smithi007.stdout:9/435: fsync d13/d18/d60/d6d/f6f 0 2024-09-21T04:35:26.200 INFO:tasks.workunit.client.0.smithi007.stdout:9/436: fdatasync d13/d18/f35 0 2024-09-21T04:35:26.207 INFO:tasks.workunit.client.0.smithi007.stdout:2/346: rename d0/d2/l53 to d0/d11/d31/d52/l5c 0 2024-09-21T04:35:26.220 INFO:tasks.workunit.client.0.smithi007.stdout:2/347: rename d0/d11/d31/l41 to d0/d11/l5d 0 2024-09-21T04:35:26.221 INFO:tasks.workunit.client.0.smithi007.stdout:2/348: write d0/d2/f6 [1077111,130880] 0 2024-09-21T04:35:26.221 INFO:tasks.workunit.client.0.smithi007.stdout:2/349: readlink d0/d2/le 0 2024-09-21T04:35:26.222 INFO:tasks.workunit.client.0.smithi007.stdout:2/350: chown d0/d11/d31 1379 1 2024-09-21T04:35:26.236 INFO:tasks.workunit.client.0.smithi007.stdout:2/351: dread d0/d2/f6 [0,4194304] 0 2024-09-21T04:35:26.241 INFO:tasks.workunit.client.0.smithi007.stdout:2/352: chown d0/d11/d31/d52/c5b 41 1 2024-09-21T04:35:26.241 INFO:tasks.workunit.client.0.smithi007.stdout:9/437: dwrite d13/d18/f4b [0,4194304] 0 2024-09-21T04:35:26.243 INFO:tasks.workunit.client.0.smithi007.stdout:2/353: rmdir d0/d2 39 2024-09-21T04:35:26.246 INFO:tasks.workunit.client.0.smithi007.stdout:2/354: read d0/d11/d31/d46/f47 [2211708,51404] 0 2024-09-21T04:35:26.265 INFO:tasks.workunit.client.0.smithi007.stdout:6/416: sync 2024-09-21T04:35:26.265 INFO:tasks.workunit.client.0.smithi007.stdout:8/254: sync 2024-09-21T04:35:26.265 INFO:tasks.workunit.client.0.smithi007.stdout:4/482: sync 2024-09-21T04:35:26.266 INFO:tasks.workunit.client.0.smithi007.stdout:6/417: chown d3/d3e/f55 14 1 2024-09-21T04:35:26.267 INFO:tasks.workunit.client.0.smithi007.stdout:6/418: write d3/de/f1a [1040092,43909] 0 2024-09-21T04:35:26.267 INFO:tasks.workunit.client.0.smithi007.stdout:4/483: truncate d2/d17/d2e/d45/d5e/f82 172455 0 2024-09-21T04:35:26.269 INFO:tasks.workunit.client.0.smithi007.stdout:6/419: read d3/de/d12/d19/f35 [2067286,100072] 0 2024-09-21T04:35:26.276 INFO:tasks.workunit.client.0.smithi007.stdout:8/255: dwrite d0/d3/d8/de/f17 [0,4194304] 0 2024-09-21T04:35:26.276 INFO:tasks.workunit.client.0.smithi007.stdout:9/438: truncate d13/d18/f71 2385345 0 2024-09-21T04:35:26.287 INFO:tasks.workunit.client.0.smithi007.stdout:2/355: write d0/f8 [1224227,28713] 0 2024-09-21T04:35:26.287 INFO:tasks.workunit.client.0.smithi007.stdout:4/484: getdents d2/d7/d55 0 2024-09-21T04:35:26.289 INFO:tasks.workunit.client.0.smithi007.stdout:9/439: dread fe [0,4194304] 0 2024-09-21T04:35:26.323 INFO:tasks.workunit.client.0.smithi007.stdout:6/420: write d3/f1e [151702,9904] 0 2024-09-21T04:35:26.328 INFO:tasks.workunit.client.0.smithi007.stdout:9/440: dwrite d13/d18/f67 [0,4194304] 0 2024-09-21T04:35:26.332 INFO:tasks.workunit.client.0.smithi007.stdout:6/421: dwrite d3/d3e/f55 [4194304,4194304] 0 2024-09-21T04:35:26.334 INFO:tasks.workunit.client.0.smithi007.stdout:2/356: write d0/d11/d31/d46/f47 [638624,118621] 0 2024-09-21T04:35:26.336 INFO:tasks.workunit.client.0.smithi007.stdout:6/422: stat d3/de/c3a 0 2024-09-21T04:35:26.340 INFO:tasks.workunit.client.0.smithi007.stdout:6/423: chown d3/f15 8470 1 2024-09-21T04:35:26.344 INFO:tasks.workunit.client.0.smithi007.stdout:2/357: dwrite d0/d11/d31/d46/f55 [0,4194304] 0 2024-09-21T04:35:26.346 INFO:tasks.workunit.client.0.smithi007.stdout:4/485: dwrite d2/d17/d2e/d45/f8f [0,4194304] 0 2024-09-21T04:35:26.347 INFO:tasks.workunit.client.0.smithi007.stdout:9/441: dread d13/d18/f6e [0,4194304] 0 2024-09-21T04:35:26.347 INFO:tasks.workunit.client.0.smithi007.stdout:4/486: chown d2/d17/d2e 1012 1 2024-09-21T04:35:26.348 INFO:tasks.workunit.client.0.smithi007.stdout:9/442: write d13/f6b [331095,42028] 0 2024-09-21T04:35:26.351 INFO:tasks.workunit.client.0.smithi007.stdout:6/424: fsync d3/d3e/d46/d60/d77/f63 0 2024-09-21T04:35:26.354 INFO:tasks.workunit.client.0.smithi007.stdout:9/443: dread d13/d18/f67 [0,4194304] 0 2024-09-21T04:35:26.355 INFO:tasks.workunit.client.0.smithi007.stdout:9/444: chown d13/d89 151 1 2024-09-21T04:35:26.356 INFO:tasks.workunit.client.0.smithi007.stdout:9/445: truncate d13/d5a/f61 67950 0 2024-09-21T04:35:26.357 INFO:tasks.workunit.client.0.smithi007.stdout:9/446: write d13/d18/f1a [2835087,41427] 0 2024-09-21T04:35:26.358 INFO:tasks.workunit.client.0.smithi007.stdout:4/487: unlink d2/d17/l35 0 2024-09-21T04:35:26.364 INFO:tasks.workunit.client.0.smithi007.stdout:6/425: symlink d3/de/d58/l89 0 2024-09-21T04:35:26.366 INFO:tasks.workunit.client.0.smithi007.stdout:9/447: rmdir d13/d18/d60/d6d/d75/d7f 39 2024-09-21T04:35:26.366 INFO:tasks.workunit.client.0.smithi007.stdout:4/488: rmdir d2/d7/d55 39 2024-09-21T04:35:26.372 INFO:tasks.workunit.client.0.smithi007.stdout:6/426: getdents d3/d3e/d85/d78 0 2024-09-21T04:35:26.379 INFO:tasks.workunit.client.0.smithi007.stdout:4/489: dread d2/f27 [0,4194304] 0 2024-09-21T04:35:26.392 INFO:tasks.workunit.client.0.smithi007.stdout:2/358: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:26.393 INFO:tasks.workunit.client.0.smithi007.stdout:6/427: dread d3/de/d12/d19/f39 [0,4194304] 0 2024-09-21T04:35:26.400 INFO:tasks.workunit.client.0.smithi007.stdout:9/448: fsync d13/d18/f71 0 2024-09-21T04:35:26.404 INFO:tasks.workunit.client.0.smithi007.stdout:2/359: mknod d0/d2/c5e 0 2024-09-21T04:35:26.405 INFO:tasks.workunit.client.0.smithi007.stdout:6/428: creat d3/de/d7d/f8a x:0 0 0 2024-09-21T04:35:26.407 INFO:tasks.workunit.client.0.smithi007.stdout:2/360: creat d0/d11/d31/d39/f5f x:0 0 0 2024-09-21T04:35:26.407 INFO:tasks.workunit.client.0.smithi007.stdout:6/429: write d3/de/d12/f7c [241808,72906] 0 2024-09-21T04:35:26.409 INFO:tasks.workunit.client.0.smithi007.stdout:2/361: fdatasync d0/f29 0 2024-09-21T04:35:26.414 INFO:tasks.workunit.client.0.smithi007.stdout:2/362: creat d0/d11/d31/d46/f60 x:0 0 0 2024-09-21T04:35:26.415 INFO:tasks.workunit.client.0.smithi007.stdout:6/430: mknod d3/c8b 0 2024-09-21T04:35:26.415 INFO:tasks.workunit.client.0.smithi007.stdout:9/449: dread fa [0,4194304] 0 2024-09-21T04:35:26.420 INFO:tasks.workunit.client.0.smithi007.stdout:9/450: symlink d13/d18/d80/l8d 0 2024-09-21T04:35:26.423 INFO:tasks.workunit.client.0.smithi007.stdout:2/363: dwrite d0/f7 [0,4194304] 0 2024-09-21T04:35:26.423 INFO:tasks.workunit.client.0.smithi007.stdout:9/451: read d13/f1e [762188,122087] 0 2024-09-21T04:35:26.424 INFO:tasks.workunit.client.0.smithi007.stdout:2/364: chown d0/d2/l26 718287726 1 2024-09-21T04:35:26.430 INFO:tasks.workunit.client.0.smithi007.stdout:2/365: mknod d0/d11/d31/d39/c61 0 2024-09-21T04:35:26.430 INFO:tasks.workunit.client.0.smithi007.stdout:9/452: symlink d13/d18/l8e 0 2024-09-21T04:35:26.434 INFO:tasks.workunit.client.0.smithi007.stdout:2/366: creat d0/d11/d31/d46/d48/f62 x:0 0 0 2024-09-21T04:35:26.436 INFO:tasks.workunit.client.0.smithi007.stdout:2/367: dread - d0/d11/d31/d39/f50 zero size 2024-09-21T04:35:26.439 INFO:tasks.workunit.client.0.smithi007.stdout:9/453: dwrite d13/d18/f36 [0,4194304] 0 2024-09-21T04:35:26.442 INFO:tasks.workunit.client.0.smithi007.stdout:6/431: truncate d3/d3e/f55 1183207 0 2024-09-21T04:35:26.442 INFO:tasks.workunit.client.0.smithi007.stdout:2/368: dread d0/d11/d31/d39/f3e [4194304,4194304] 0 2024-09-21T04:35:26.444 INFO:tasks.workunit.client.0.smithi007.stdout:2/369: readlink d0/d11/d31/l43 0 2024-09-21T04:35:26.447 INFO:tasks.workunit.client.0.smithi007.stdout:6/432: mknod d3/de/d4e/d67/c8c 0 2024-09-21T04:35:26.453 INFO:tasks.workunit.client.0.smithi007.stdout:7/441: sync 2024-09-21T04:35:26.453 INFO:tasks.workunit.client.0.smithi007.stdout:1/595: sync 2024-09-21T04:35:26.453 INFO:tasks.workunit.client.0.smithi007.stdout:3/582: sync 2024-09-21T04:35:26.453 INFO:tasks.workunit.client.0.smithi007.stdout:5/661: sync 2024-09-21T04:35:26.453 INFO:tasks.workunit.client.0.smithi007.stdout:0/557: sync 2024-09-21T04:35:26.454 INFO:tasks.workunit.client.0.smithi007.stdout:9/454: link d13/d18/f5c d13/d18/d60/d83/f8f 0 2024-09-21T04:35:26.456 INFO:tasks.workunit.client.0.smithi007.stdout:1/596: chown d5/d6/f20 12 1 2024-09-21T04:35:26.458 INFO:tasks.workunit.client.0.smithi007.stdout:3/583: creat dc/d18/fbd x:0 0 0 2024-09-21T04:35:26.459 INFO:tasks.workunit.client.0.smithi007.stdout:3/584: stat dc/l12 0 2024-09-21T04:35:26.460 INFO:tasks.workunit.client.0.smithi007.stdout:6/433: creat d3/de/d12/f8d x:0 0 0 2024-09-21T04:35:26.463 INFO:tasks.workunit.client.0.smithi007.stdout:9/455: chown d13/c50 0 1 2024-09-21T04:35:26.463 INFO:tasks.workunit.client.0.smithi007.stdout:5/662: rmdir d2/d27/d42/da3/d23/da1 39 2024-09-21T04:35:26.465 INFO:tasks.workunit.client.0.smithi007.stdout:9/456: fdatasync d13/d2a/f56 0 2024-09-21T04:35:26.470 INFO:tasks.workunit.client.0.smithi007.stdout:3/585: truncate dc/d22/d29/d32/d69/d6d/fb3 881957 0 2024-09-21T04:35:26.470 INFO:tasks.workunit.client.0.smithi007.stdout:0/558: rename d6/d75/dc5/d47/d8a/faa to d6/d75/dc5/d18/d2e/d3f/fcd 0 2024-09-21T04:35:26.471 INFO:tasks.workunit.client.0.smithi007.stdout:3/586: chown dc/d22/d5a/d71/d89 6053 1 2024-09-21T04:35:26.472 INFO:tasks.workunit.client.0.smithi007.stdout:3/587: fdatasync dc/d22/f59 0 2024-09-21T04:35:26.473 INFO:tasks.workunit.client.0.smithi007.stdout:7/442: truncate d3/d6/d13/f47 458988 0 2024-09-21T04:35:26.475 INFO:tasks.workunit.client.0.smithi007.stdout:7/443: write d3/d6/d14/d20/d58/f95 [623020,92106] 0 2024-09-21T04:35:26.476 INFO:tasks.workunit.client.0.smithi007.stdout:3/588: dread dc/d22/d34/da6/f5c [0,4194304] 0 2024-09-21T04:35:26.482 INFO:tasks.workunit.client.0.smithi007.stdout:2/370: dread d0/fd [0,4194304] 0 2024-09-21T04:35:26.483 INFO:tasks.workunit.client.0.smithi007.stdout:3/589: truncate dc/d18/d41/fb2 776874 0 2024-09-21T04:35:26.488 INFO:tasks.workunit.client.0.smithi007.stdout:7/444: dwrite d3/d6/d2e/d63/d6a/d81/f8c [0,4194304] 0 2024-09-21T04:35:26.491 INFO:tasks.workunit.client.0.smithi007.stdout:9/457: chown d13/d18/c3a 313243890 1 2024-09-21T04:35:26.492 INFO:tasks.workunit.client.0.smithi007.stdout:2/371: write d0/d2/fc [3766162,33678] 0 2024-09-21T04:35:26.492 INFO:tasks.workunit.client.0.smithi007.stdout:0/559: symlink d6/lce 0 2024-09-21T04:35:26.494 INFO:tasks.workunit.client.0.smithi007.stdout:6/434: mkdir d3/d8e 0 2024-09-21T04:35:26.498 INFO:tasks.workunit.client.0.smithi007.stdout:1/597: write d5/d45/d94/f59 [1341552,51038] 0 2024-09-21T04:35:26.508 INFO:tasks.workunit.client.0.smithi007.stdout:5/663: write d2/d27/d42/da3/f16 [6027292,103590] 0 2024-09-21T04:35:26.509 INFO:tasks.workunit.client.0.smithi007.stdout:9/458: readlink d13/l5d 0 2024-09-21T04:35:26.512 INFO:tasks.workunit.client.0.smithi007.stdout:2/372: fsync d0/f23 0 2024-09-21T04:35:26.515 INFO:tasks.workunit.client.0.smithi007.stdout:6/435: write d3/f6 [565904,130870] 0 2024-09-21T04:35:26.518 INFO:tasks.workunit.client.0.smithi007.stdout:7/445: write d3/d6/d14/d20/d80/f9c [4301494,107783] 0 2024-09-21T04:35:26.522 INFO:tasks.workunit.client.0.smithi007.stdout:2/373: dwrite d0/d2/d57/f5a [0,4194304] 0 2024-09-21T04:35:26.529 INFO:tasks.workunit.client.0.smithi007.stdout:5/664: mknod d2/d27/d3d/d9b/cd7 0 2024-09-21T04:35:26.530 INFO:tasks.workunit.client.0.smithi007.stdout:1/598: write d5/d45/f7d [738692,13651] 0 2024-09-21T04:35:26.532 INFO:tasks.workunit.client.0.smithi007.stdout:3/590: write dc/d18/d26/f51 [2664476,85553] 0 2024-09-21T04:35:26.532 INFO:tasks.workunit.client.0.smithi007.stdout:6/436: rename d3/de to d3/de/d8f 22 2024-09-21T04:35:26.535 INFO:tasks.workunit.client.0.smithi007.stdout:7/446: mkdir d3/d6/d40/da1 0 2024-09-21T04:35:26.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/560: mknod d6/d75/dc5/d57/d9d/daf/ccf 0 2024-09-21T04:35:26.537 INFO:tasks.workunit.client.0.smithi007.stdout:0/561: chown d6/d75/dc5/d18/d2e/d3f/fcd 12 1 2024-09-21T04:35:26.538 INFO:tasks.workunit.client.0.smithi007.stdout:0/562: write d6/d75/dc5/d18/d2e/d60/f5f [989320,31161] 0 2024-09-21T04:35:26.542 INFO:tasks.workunit.client.0.smithi007.stdout:6/437: rename d3/d3e/f7b to d3/de/d58/f90 0 2024-09-21T04:35:26.543 INFO:tasks.workunit.client.0.smithi007.stdout:1/599: truncate d5/d6/d21/d2b/d4b/da6/d58/fa5 308058 0 2024-09-21T04:35:26.544 INFO:tasks.workunit.client.0.smithi007.stdout:7/447: unlink d3/d6/d14/d20/d58/c5b 0 2024-09-21T04:35:26.546 INFO:tasks.workunit.client.0.smithi007.stdout:6/438: dread d3/de/d4e/f5d [4194304,4194304] 0 2024-09-21T04:35:26.546 INFO:tasks.workunit.client.0.smithi007.stdout:2/374: link d0/d32/l56 d0/d32/l63 0 2024-09-21T04:35:26.553 INFO:tasks.workunit.client.0.smithi007.stdout:2/375: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:26.555 INFO:tasks.workunit.client.0.smithi007.stdout:7/448: symlink d3/d6/d10/la2 0 2024-09-21T04:35:26.562 INFO:tasks.workunit.client.0.smithi007.stdout:5/665: truncate d2/d27/d42/da3/d1d/f1f 3701026 0 2024-09-21T04:35:26.563 INFO:tasks.workunit.client.0.smithi007.stdout:5/666: stat d2/d27/d42/da3/d23/d5d/lca 0 2024-09-21T04:35:26.564 INFO:tasks.workunit.client.0.smithi007.stdout:1/600: dread d5/d6/d21/d2b/d4b/da6/f80 [0,4194304] 0 2024-09-21T04:35:26.565 INFO:tasks.workunit.client.0.smithi007.stdout:5/667: truncate d2/d27/d42/da3/fb3 566430 0 2024-09-21T04:35:26.566 INFO:tasks.workunit.client.0.smithi007.stdout:3/591: truncate dc/d22/d34/da6/f60 1432338 0 2024-09-21T04:35:26.569 INFO:tasks.workunit.client.0.smithi007.stdout:0/563: rename d6/d75/dc5/d18/d2e/f37 to d6/d75/dc5/d47/d8a/d9c/fd0 0 2024-09-21T04:35:26.570 INFO:tasks.workunit.client.0.smithi007.stdout:2/376: mknod d0/d2/c64 0 2024-09-21T04:35:26.571 INFO:tasks.workunit.client.0.smithi007.stdout:0/564: read d6/d75/dc5/d18/d2e/d3f/f7c [3956376,3942] 0 2024-09-21T04:35:26.572 INFO:tasks.workunit.client.0.smithi007.stdout:3/592: dread dc/d18/f7f [0,4194304] 0 2024-09-21T04:35:26.573 INFO:tasks.workunit.client.0.smithi007.stdout:7/449: creat d3/d6/d10/d62/d66/fa3 x:0 0 0 2024-09-21T04:35:26.575 INFO:tasks.workunit.client.0.smithi007.stdout:6/439: unlink d3/d3e/d46/d60/c74 0 2024-09-21T04:35:26.578 INFO:tasks.workunit.client.0.smithi007.stdout:6/440: chown d3/d3b/f79 873232613 1 2024-09-21T04:35:26.578 INFO:tasks.workunit.client.0.smithi007.stdout:6/441: dread - d3/d3b/f83 zero size 2024-09-21T04:35:26.581 INFO:tasks.workunit.client.0.smithi007.stdout:0/565: dread d6/d75/dc5/d18/d27/f9b [0,4194304] 0 2024-09-21T04:35:26.582 INFO:tasks.workunit.client.0.smithi007.stdout:5/668: dread d2/d27/d42/da3/d23/d5d/fb6 [0,4194304] 0 2024-09-21T04:35:26.583 INFO:tasks.workunit.client.0.smithi007.stdout:5/669: stat d2/d27/d42/da3/f3e 0 2024-09-21T04:35:26.587 INFO:tasks.workunit.client.0.smithi007.stdout:2/377: dread d0/f2e [0,4194304] 0 2024-09-21T04:35:26.588 INFO:tasks.workunit.client.0.smithi007.stdout:1/601: dwrite d5/f26 [4194304,4194304] 0 2024-09-21T04:35:26.595 INFO:tasks.workunit.client.0.smithi007.stdout:2/378: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:26.598 INFO:tasks.workunit.client.0.smithi007.stdout:2/379: write d0/d11/d31/d39/f50 [238566,50443] 0 2024-09-21T04:35:26.600 INFO:tasks.workunit.client.0.smithi007.stdout:2/380: fdatasync d0/d2/f2b 0 2024-09-21T04:35:26.602 INFO:tasks.workunit.client.0.smithi007.stdout:0/566: read d6/d75/dc5/d18/d39/f6a [3388338,19943] 0 2024-09-21T04:35:26.603 INFO:tasks.workunit.client.0.smithi007.stdout:5/670: write d2/d27/d42/d53/f90 [271216,13729] 0 2024-09-21T04:35:26.608 INFO:tasks.workunit.client.0.smithi007.stdout:6/442: write d3/d3e/f40 [1270967,49908] 0 2024-09-21T04:35:26.610 INFO:tasks.workunit.client.0.smithi007.stdout:3/593: symlink dc/d22/d34/dbc/lbe 0 2024-09-21T04:35:26.612 INFO:tasks.workunit.client.0.smithi007.stdout:6/443: write d3/de/d12/d19/f24 [386925,91001] 0 2024-09-21T04:35:26.614 INFO:tasks.workunit.client.0.smithi007.stdout:7/450: chown d3/d6/d10/d4f/c77 7008641 1 2024-09-21T04:35:26.614 INFO:tasks.workunit.client.0.smithi007.stdout:2/381: creat d0/d11/d31/d52/f65 x:0 0 0 2024-09-21T04:35:26.616 INFO:tasks.workunit.client.0.smithi007.stdout:3/594: truncate dc/d22/d29/d32/d4d/fa0 661326 0 2024-09-21T04:35:26.616 INFO:tasks.workunit.client.0.smithi007.stdout:0/567: mknod d6/d75/dc5/d57/d9d/daf/cd1 0 2024-09-21T04:35:26.621 INFO:tasks.workunit.client.0.smithi007.stdout:5/671: dread d2/d27/d3d/d56/f96 [0,4194304] 0 2024-09-21T04:35:26.623 INFO:tasks.workunit.client.0.smithi007.stdout:1/602: dread d5/d6/d21/d2b/d4b/f79 [0,4194304] 0 2024-09-21T04:35:26.625 INFO:tasks.workunit.client.0.smithi007.stdout:0/568: rmdir d6/d91/da3/dbf 39 2024-09-21T04:35:26.626 INFO:tasks.workunit.client.0.smithi007.stdout:6/444: symlink d3/d3e/d85/d88/l91 0 2024-09-21T04:35:26.627 INFO:tasks.workunit.client.0.smithi007.stdout:0/569: chown d6/d75/dc5/d18/d39/c40 7837797 1 2024-09-21T04:35:26.629 INFO:tasks.workunit.client.0.smithi007.stdout:6/445: truncate d3/de/d4e/d67/f75 625400 0 2024-09-21T04:35:26.629 INFO:tasks.workunit.client.0.smithi007.stdout:3/595: dread dc/f1c [0,4194304] 0 2024-09-21T04:35:26.630 INFO:tasks.workunit.client.0.smithi007.stdout:7/451: mknod d3/d6/d40/ca4 0 2024-09-21T04:35:26.631 INFO:tasks.workunit.client.0.smithi007.stdout:6/446: readlink d3/d3e/d46/d60/l72 0 2024-09-21T04:35:26.632 INFO:tasks.workunit.client.0.smithi007.stdout:7/452: chown d3/d6/d10/c9f 1297 1 2024-09-21T04:35:26.633 INFO:tasks.workunit.client.0.smithi007.stdout:6/447: readlink d3/de/d12/l34 0 2024-09-21T04:35:26.635 INFO:tasks.workunit.client.0.smithi007.stdout:0/570: dwrite d6/d75/dc5/d57/d6e/f8e [0,4194304] 0 2024-09-21T04:35:26.639 INFO:tasks.workunit.client.0.smithi007.stdout:3/596: creat dc/d22/d34/fbf x:0 0 0 2024-09-21T04:35:26.640 INFO:tasks.workunit.client.0.smithi007.stdout:6/448: fdatasync d3/d3e/d46/d60/d77/f63 0 2024-09-21T04:35:26.644 INFO:tasks.workunit.client.0.smithi007.stdout:2/382: getdents d0/d2 0 2024-09-21T04:35:26.649 INFO:tasks.workunit.client.0.smithi007.stdout:5/672: write d2/d27/d42/da3/f3e [4597570,97253] 0 2024-09-21T04:35:26.649 INFO:tasks.workunit.client.0.smithi007.stdout:6/449: chown d3/d3e/f55 321808 1 2024-09-21T04:35:26.650 INFO:tasks.workunit.client.0.smithi007.stdout:1/603: write d5/d6/f4a [2611526,26253] 0 2024-09-21T04:35:26.652 INFO:tasks.workunit.client.0.smithi007.stdout:3/597: dwrite dc/d22/d5a/d71/f9d [0,4194304] 0 2024-09-21T04:35:26.653 INFO:tasks.workunit.client.0.smithi007.stdout:7/453: dread d3/d6/d10/f24 [0,4194304] 0 2024-09-21T04:35:26.658 INFO:tasks.workunit.client.0.smithi007.stdout:1/604: symlink d5/d6/d21/d2b/lc6 0 2024-09-21T04:35:26.660 INFO:tasks.workunit.client.0.smithi007.stdout:2/383: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:26.668 INFO:tasks.workunit.client.0.smithi007.stdout:0/571: dwrite d6/d75/dc5/d18/d2e/f92 [0,4194304] 0 2024-09-21T04:35:26.672 INFO:tasks.workunit.client.0.smithi007.stdout:7/454: mknod d3/d6/d14/d20/d80/ca5 0 2024-09-21T04:35:26.672 INFO:tasks.workunit.client.0.smithi007.stdout:0/572: chown d6/d75/dc5/d57/l89 2373 1 2024-09-21T04:35:26.673 INFO:tasks.workunit.client.0.smithi007.stdout:2/384: mknod d0/c66 0 2024-09-21T04:35:26.674 INFO:tasks.workunit.client.0.smithi007.stdout:6/450: dwrite d3/de/d7d/f8a [0,4194304] 0 2024-09-21T04:35:26.678 INFO:tasks.workunit.client.0.smithi007.stdout:1/605: creat d5/d6/d21/fc7 x:0 0 0 2024-09-21T04:35:26.682 INFO:tasks.workunit.client.0.smithi007.stdout:3/598: dwrite dc/d22/d29/d32/d4d/f8e [0,4194304] 0 2024-09-21T04:35:26.685 INFO:tasks.workunit.client.0.smithi007.stdout:1/606: dread - d5/d6/d21/d2b/d4b/da6/d58/fb7 zero size 2024-09-21T04:35:26.686 INFO:tasks.workunit.client.0.smithi007.stdout:6/451: chown d3/de/d4e/d67/f75 49834 1 2024-09-21T04:35:26.687 INFO:tasks.workunit.client.0.smithi007.stdout:0/573: unlink d6/d75/dc5/d57/f59 0 2024-09-21T04:35:26.692 INFO:tasks.workunit.client.0.smithi007.stdout:0/574: chown d6/d75/dc5/d18/d27/l36 1027916 1 2024-09-21T04:35:26.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:26 smithi053 ceph-mon[35554]: pgmap v34: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 110 MiB/s rd, 320 MiB/s wr, 625 op/s 2024-09-21T04:35:26.692 INFO:tasks.workunit.client.0.smithi007.stdout:2/385: dwrite d0/fd [0,4194304] 0 2024-09-21T04:35:26.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:26 smithi007 ceph-mon[27954]: pgmap v34: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 110 MiB/s rd, 320 MiB/s wr, 625 op/s 2024-09-21T04:35:26.694 INFO:tasks.workunit.client.0.smithi007.stdout:7/455: symlink d3/d6/d10/d62/d66/d69/la6 0 2024-09-21T04:35:26.694 INFO:tasks.workunit.client.0.smithi007.stdout:0/575: fdatasync d6/d75/dc5/d18/d27/fb7 0 2024-09-21T04:35:26.699 INFO:tasks.workunit.client.0.smithi007.stdout:2/386: dread d0/d11/f4e [0,4194304] 0 2024-09-21T04:35:26.705 INFO:tasks.workunit.client.0.smithi007.stdout:2/387: dwrite d0/f37 [0,4194304] 0 2024-09-21T04:35:26.709 INFO:tasks.workunit.client.0.smithi007.stdout:6/452: creat d3/d3e/d85/f92 x:0 0 0 2024-09-21T04:35:26.711 INFO:tasks.workunit.client.0.smithi007.stdout:2/388: fsync d0/fd 0 2024-09-21T04:35:26.718 INFO:tasks.workunit.client.0.smithi007.stdout:7/456: readlink d3/d6/d13/l59 0 2024-09-21T04:35:26.718 INFO:tasks.workunit.client.0.smithi007.stdout:0/576: symlink d6/d75/dc5/d18/d39/ld2 0 2024-09-21T04:35:26.721 INFO:tasks.workunit.client.0.smithi007.stdout:2/389: dwrite d0/d11/d31/d39/f5f [0,4194304] 0 2024-09-21T04:35:26.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/457: chown d3/d6/d14/d20 73 1 2024-09-21T04:35:26.723 INFO:tasks.workunit.client.0.smithi007.stdout:1/607: creat d5/d6/d21/d2b/d86/fc8 x:0 0 0 2024-09-21T04:35:26.725 INFO:tasks.workunit.client.0.smithi007.stdout:7/458: write d3/d6/d14/d20/d58/f87 [2802281,101783] 0 2024-09-21T04:35:26.729 INFO:tasks.workunit.client.0.smithi007.stdout:6/453: creat d3/d3e/d46/d60/f93 x:0 0 0 2024-09-21T04:35:26.734 INFO:tasks.workunit.client.0.smithi007.stdout:3/599: creat dc/d22/fc0 x:0 0 0 2024-09-21T04:35:26.746 INFO:tasks.workunit.client.0.smithi007.stdout:1/608: creat d5/d2f/fc9 x:0 0 0 2024-09-21T04:35:26.747 INFO:tasks.workunit.client.0.smithi007.stdout:0/577: link d6/d75/dc5/fba d6/d75/dc5/d18/d27/fd3 0 2024-09-21T04:35:26.759 INFO:tasks.workunit.client.0.smithi007.stdout:6/454: rename d3/de/d7d to d3/d94 0 2024-09-21T04:35:26.759 INFO:tasks.workunit.client.0.smithi007.stdout:2/390: truncate d0/d2/f16 2890194 0 2024-09-21T04:35:26.760 INFO:tasks.workunit.client.0.smithi007.stdout:7/459: dwrite d3/d6/d14/d20/f35 [0,4194304] 0 2024-09-21T04:35:26.764 INFO:tasks.workunit.client.0.smithi007.stdout:6/455: dread - d3/f5c zero size 2024-09-21T04:35:26.768 INFO:tasks.workunit.client.0.smithi007.stdout:0/578: truncate d6/d75/dc5/d18/f69 160974 0 2024-09-21T04:35:26.774 INFO:tasks.workunit.client.0.smithi007.stdout:1/609: rename d5/f15 to d5/d6/d21/d2b/d4b/d8a/d8d/fca 0 2024-09-21T04:35:26.775 INFO:tasks.workunit.client.0.smithi007.stdout:6/456: mkdir d3/de/d4e/d95 0 2024-09-21T04:35:26.777 INFO:tasks.workunit.client.0.smithi007.stdout:2/391: symlink d0/l67 0 2024-09-21T04:35:26.779 INFO:tasks.workunit.client.0.smithi007.stdout:7/460: dwrite d3/d6/d2e/d63/d6a/d81/f96 [0,4194304] 0 2024-09-21T04:35:26.780 INFO:tasks.workunit.client.0.smithi007.stdout:2/392: chown d0/f2e 16691 1 2024-09-21T04:35:26.780 INFO:tasks.workunit.client.0.smithi007.stdout:6/457: chown d3/de/d12/d19/f35 10 1 2024-09-21T04:35:26.781 INFO:tasks.workunit.client.0.smithi007.stdout:7/461: dread - d3/d6/d10/d62/d66/fa3 zero size 2024-09-21T04:35:26.783 INFO:tasks.workunit.client.0.smithi007.stdout:1/610: mknod d5/d6/d21/d2b/d4b/d8a/d8d/ccb 0 2024-09-21T04:35:26.785 INFO:tasks.workunit.client.0.smithi007.stdout:1/611: stat d5/d45/d94/f7b 0 2024-09-21T04:35:26.785 INFO:tasks.workunit.client.0.smithi007.stdout:0/579: write d6/d75/dc5/d47/d8a/fc4 [319136,16675] 0 2024-09-21T04:35:26.786 INFO:tasks.workunit.client.0.smithi007.stdout:6/458: chown d3/de/d4e/d67/c6f 0 1 2024-09-21T04:35:26.789 INFO:tasks.workunit.client.0.smithi007.stdout:7/462: truncate d3/d6/d2e/d63/d6a/d81/fa0 688234 0 2024-09-21T04:35:26.789 INFO:tasks.workunit.client.0.smithi007.stdout:0/580: chown d6/d75/dc5/d57/l89 74075062 1 2024-09-21T04:35:26.797 INFO:tasks.workunit.client.0.smithi007.stdout:1/612: write d5/d6/d21/d2b/d4b/d8a/d8d/fca [1116346,122934] 0 2024-09-21T04:35:26.798 INFO:tasks.workunit.client.0.smithi007.stdout:7/463: mknod d3/d6/d10/d62/d66/d69/ca7 0 2024-09-21T04:35:26.800 INFO:tasks.workunit.client.0.smithi007.stdout:2/393: link d0/d11/f28 d0/d2/f68 0 2024-09-21T04:35:26.801 INFO:tasks.workunit.client.0.smithi007.stdout:2/394: fdatasync d0/d2/fc 0 2024-09-21T04:35:26.806 INFO:tasks.workunit.client.0.smithi007.stdout:6/459: write d3/d3b/f5b [2396966,96732] 0 2024-09-21T04:35:26.812 INFO:tasks.workunit.client.0.smithi007.stdout:1/613: truncate d5/fe 1836664 0 2024-09-21T04:35:26.814 INFO:tasks.workunit.client.0.smithi007.stdout:2/395: symlink d0/d11/l69 0 2024-09-21T04:35:26.816 INFO:tasks.workunit.client.0.smithi007.stdout:0/581: link d6/d75/dc5/d47/f4b d6/fd4 0 2024-09-21T04:35:26.818 INFO:tasks.workunit.client.0.smithi007.stdout:6/460: dread d3/d3e/d46/d60/d77/f63 [0,4194304] 0 2024-09-21T04:35:26.819 INFO:tasks.workunit.client.0.smithi007.stdout:7/464: rmdir d3/d6/d14/d20/d58/d90 0 2024-09-21T04:35:26.820 INFO:tasks.workunit.client.0.smithi007.stdout:1/614: symlink d5/d6/d21/d2b/d4b/d8a/lcc 0 2024-09-21T04:35:26.821 INFO:tasks.workunit.client.0.smithi007.stdout:2/396: creat d0/d11/d31/d46/f6a x:0 0 0 2024-09-21T04:35:26.823 INFO:tasks.workunit.client.0.smithi007.stdout:0/582: mkdir d6/d75/dc5/d18/d39/dd5 0 2024-09-21T04:35:26.825 INFO:tasks.workunit.client.0.smithi007.stdout:7/465: mkdir d3/d6/d14/d20/d58/da8 0 2024-09-21T04:35:26.828 INFO:tasks.workunit.client.0.smithi007.stdout:1/615: rename d5/d6/d21/da4 to d5/d45/d94/da0/d99/dcd 0 2024-09-21T04:35:26.829 INFO:tasks.workunit.client.0.smithi007.stdout:7/466: dread d3/d6/d2e/d5e/f8a [0,4194304] 0 2024-09-21T04:35:26.833 INFO:tasks.workunit.client.0.smithi007.stdout:2/397: mkdir d0/d11/d6b 0 2024-09-21T04:35:26.836 INFO:tasks.workunit.client.0.smithi007.stdout:2/398: write d0/d11/f4b [67802,82861] 0 2024-09-21T04:35:26.838 INFO:tasks.workunit.client.0.smithi007.stdout:7/467: symlink d3/d6/d14/la9 0 2024-09-21T04:35:26.841 INFO:tasks.workunit.client.0.smithi007.stdout:1/616: dread d5/d6/f4a [0,4194304] 0 2024-09-21T04:35:26.892 INFO:tasks.workunit.client.0.smithi007.stdout:8/256: sync 2024-09-21T04:35:26.892 INFO:tasks.workunit.client.0.smithi007.stdout:0/583: link d6/f88 d6/d75/dc5/d47/d8a/fd6 0 2024-09-21T04:35:26.899 INFO:tasks.workunit.client.0.smithi007.stdout:6/461: getdents d3/d3e/d46/d60 0 2024-09-21T04:35:26.902 INFO:tasks.workunit.client.0.smithi007.stdout:2/399: mkdir d0/d11/d31/d46/d6c 0 2024-09-21T04:35:26.905 INFO:tasks.workunit.client.0.smithi007.stdout:1/617: truncate d5/d6/d21/d2b/f63 1042588 0 2024-09-21T04:35:26.907 INFO:tasks.workunit.client.0.smithi007.stdout:1/618: read d5/d6/d21/d2b/d61/f6e [228940,74398] 0 2024-09-21T04:35:26.908 INFO:tasks.workunit.client.0.smithi007.stdout:1/619: fdatasync d5/d45/d94/f9e 0 2024-09-21T04:35:26.910 INFO:tasks.workunit.client.0.smithi007.stdout:6/462: fdatasync d3/f28 0 2024-09-21T04:35:26.912 INFO:tasks.workunit.client.0.smithi007.stdout:2/400: mkdir d0/d11/d31/d46/d6d 0 2024-09-21T04:35:26.920 INFO:tasks.workunit.client.0.smithi007.stdout:8/257: fdatasync d0/d3/d8/f1d 0 2024-09-21T04:35:26.931 INFO:tasks.workunit.client.0.smithi007.stdout:6/463: write d3/de/d12/d19/f64 [413679,99659] 0 2024-09-21T04:35:26.935 INFO:tasks.workunit.client.0.smithi007.stdout:2/401: chown d0/d11/l40 22 1 2024-09-21T04:35:26.936 INFO:tasks.workunit.client.0.smithi007.stdout:8/258: dread d0/d3/d8/de/f18 [0,4194304] 0 2024-09-21T04:35:26.936 INFO:tasks.workunit.client.0.smithi007.stdout:1/620: dwrite d5/f39 [0,4194304] 0 2024-09-21T04:35:26.939 INFO:tasks.workunit.client.0.smithi007.stdout:0/584: link d6/d75/dc5/d18/d27/l72 d6/d75/dc5/d18/d2e/d3f/ld7 0 2024-09-21T04:35:26.943 INFO:tasks.workunit.client.0.smithi007.stdout:2/402: dread d0/f2e [0,4194304] 0 2024-09-21T04:35:26.944 INFO:tasks.workunit.client.0.smithi007.stdout:8/259: chown d0/d3/d8/f1d 220986330 1 2024-09-21T04:35:26.947 INFO:tasks.workunit.client.0.smithi007.stdout:1/621: mknod d5/d45/d94/cce 0 2024-09-21T04:35:26.949 INFO:tasks.workunit.client.0.smithi007.stdout:7/468: truncate d3/d6/d14/d20/d80/f6f 296540 0 2024-09-21T04:35:26.950 INFO:tasks.workunit.client.0.smithi007.stdout:0/585: mknod d6/d75/dc5/d47/d8a/d9c/cd8 0 2024-09-21T04:35:26.951 INFO:tasks.workunit.client.0.smithi007.stdout:7/469: truncate d3/d6/d10/d4f/f9b 141000 0 2024-09-21T04:35:26.952 INFO:tasks.workunit.client.0.smithi007.stdout:0/586: chown d6/d75/dc5/d18/d39/ld2 278229939 1 2024-09-21T04:35:26.955 INFO:tasks.workunit.client.0.smithi007.stdout:6/464: dread d3/de/d12/d19/f24 [0,4194304] 0 2024-09-21T04:35:26.955 INFO:tasks.workunit.client.0.smithi007.stdout:1/622: mkdir d5/d6/d21/d2b/d4b/da6/d58/dcf 0 2024-09-21T04:35:26.957 INFO:tasks.workunit.client.0.smithi007.stdout:7/470: creat d3/d6/d40/faa x:0 0 0 2024-09-21T04:35:26.960 INFO:tasks.workunit.client.0.smithi007.stdout:7/471: write d3/d6/d10/d62/d66/f94 [1030591,49583] 0 2024-09-21T04:35:26.960 INFO:tasks.workunit.client.0.smithi007.stdout:8/260: mknod d0/d3/d16/d3b/c3d 0 2024-09-21T04:35:26.962 INFO:tasks.workunit.client.0.smithi007.stdout:2/403: mkdir d0/d11/d6e 0 2024-09-21T04:35:26.964 INFO:tasks.workunit.client.0.smithi007.stdout:7/472: chown d3/d6/d14/l3d 248135165 1 2024-09-21T04:35:26.965 INFO:tasks.workunit.client.0.smithi007.stdout:8/261: read d0/d3/d16/f1b [296981,81665] 0 2024-09-21T04:35:26.966 INFO:tasks.workunit.client.0.smithi007.stdout:2/404: write d0/f29 [4642657,12248] 0 2024-09-21T04:35:26.968 INFO:tasks.workunit.client.0.smithi007.stdout:1/623: dwrite d5/d6/fa [0,4194304] 0 2024-09-21T04:35:26.986 INFO:tasks.workunit.client.0.smithi007.stdout:7/473: fdatasync d3/d6/d13/d4b/f7d 0 2024-09-21T04:35:26.994 INFO:tasks.workunit.client.0.smithi007.stdout:0/587: link d6/d75/dc5/d52/c94 d6/d75/dc5/d52/cd9 0 2024-09-21T04:35:27.001 INFO:tasks.workunit.client.0.smithi007.stdout:6/465: write d3/de/d4e/f5d [5217712,84273] 0 2024-09-21T04:35:27.003 INFO:tasks.workunit.client.0.smithi007.stdout:8/262: link d0/d3/d8/de/d26/c32 d0/d3/d8/c3e 0 2024-09-21T04:35:27.006 INFO:tasks.workunit.client.0.smithi007.stdout:0/588: creat d6/d75/dc5/d18/d39/dd5/fda x:0 0 0 2024-09-21T04:35:27.007 INFO:tasks.workunit.client.0.smithi007.stdout:0/589: fsync d6/d75/dc5/d52/fb8 0 2024-09-21T04:35:27.008 INFO:tasks.workunit.client.0.smithi007.stdout:7/474: creat d3/d6/d2e/fab x:0 0 0 2024-09-21T04:35:27.015 INFO:tasks.workunit.client.0.smithi007.stdout:0/590: rename d6/d75/dc5/d57/d6e to d6/ddb 0 2024-09-21T04:35:27.016 INFO:tasks.workunit.client.0.smithi007.stdout:7/475: dwrite d3/f88 [0,4194304] 0 2024-09-21T04:35:27.017 INFO:tasks.workunit.client.0.smithi007.stdout:7/476: chown d3/d6/d40/ca4 1 1 2024-09-21T04:35:27.026 INFO:tasks.workunit.client.0.smithi007.stdout:0/591: rename d6/d75/dc5/f4d to d6/d75/dc5/d57/d9d/fdc 0 2024-09-21T04:35:27.027 INFO:tasks.workunit.client.0.smithi007.stdout:0/592: fdatasync d6/f5a 0 2024-09-21T04:35:27.028 INFO:tasks.workunit.client.0.smithi007.stdout:2/405: truncate d0/d11/d31/d46/f55 454759 0 2024-09-21T04:35:27.030 INFO:tasks.workunit.client.0.smithi007.stdout:8/263: creat d0/d3/d8/f3f x:0 0 0 2024-09-21T04:35:27.031 INFO:tasks.workunit.client.0.smithi007.stdout:8/264: fdatasync d0/d3/d8/de/f17 0 2024-09-21T04:35:27.032 INFO:tasks.workunit.client.0.smithi007.stdout:6/466: getdents d3/de/d12 0 2024-09-21T04:35:27.041 INFO:tasks.workunit.client.0.smithi007.stdout:8/265: dwrite d0/d3/d8/de/d26/d35/f37 [0,4194304] 0 2024-09-21T04:35:27.042 INFO:tasks.workunit.client.0.smithi007.stdout:8/266: chown d0/l36 10 1 2024-09-21T04:35:27.044 INFO:tasks.workunit.client.0.smithi007.stdout:2/406: dread d0/d2/f2b [0,4194304] 0 2024-09-21T04:35:27.045 INFO:tasks.workunit.client.0.smithi007.stdout:6/467: creat d3/d3e/d85/d78/f96 x:0 0 0 2024-09-21T04:35:27.046 INFO:tasks.workunit.client.0.smithi007.stdout:2/407: fsync d0/d2/f2b 0 2024-09-21T04:35:27.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/408: chown d0/d11/d31/d52/c5b 7 1 2024-09-21T04:35:27.050 INFO:tasks.workunit.client.0.smithi007.stdout:2/409: chown d0/d2/f25 1 1 2024-09-21T04:35:27.051 INFO:tasks.workunit.client.0.smithi007.stdout:0/593: symlink d6/d75/dc5/d52/ldd 0 2024-09-21T04:35:27.053 INFO:tasks.workunit.client.0.smithi007.stdout:0/594: truncate d6/f88 298264 0 2024-09-21T04:35:27.053 INFO:tasks.workunit.client.0.smithi007.stdout:8/267: dwrite d0/d3/d8/de/d26/d35/f37 [0,4194304] 0 2024-09-21T04:35:27.059 INFO:tasks.workunit.client.0.smithi007.stdout:8/268: mkdir d0/d3/d8/de/d40 0 2024-09-21T04:35:27.079 INFO:tasks.workunit.client.0.smithi007.stdout:2/410: dwrite d0/d11/d31/d39/f3a [0,4194304] 0 2024-09-21T04:35:27.087 INFO:tasks.workunit.client.0.smithi007.stdout:2/411: dwrite d0/f27 [0,4194304] 0 2024-09-21T04:35:27.197 INFO:tasks.workunit.client.0.smithi007.stdout:4/490: sync 2024-09-21T04:35:27.198 INFO:tasks.workunit.client.0.smithi007.stdout:4/491: chown d2/d17/d29/f85 394440406 1 2024-09-21T04:35:27.209 INFO:tasks.workunit.client.0.smithi007.stdout:4/492: dread d2/d17/f2a [0,4194304] 0 2024-09-21T04:35:27.211 INFO:tasks.workunit.client.0.smithi007.stdout:4/493: getdents d2/d17/d2e/d45/d5d/d8a 0 2024-09-21T04:35:27.221 INFO:tasks.workunit.client.0.smithi007.stdout:4/494: truncate d2/f6a 2064720 0 2024-09-21T04:35:27.224 INFO:tasks.workunit.client.0.smithi007.stdout:4/495: creat d2/d7/d55/d4f/fa1 x:0 0 0 2024-09-21T04:35:27.227 INFO:tasks.workunit.client.0.smithi007.stdout:4/496: creat d2/d17/d2e/d45/d5e/d90/fa2 x:0 0 0 2024-09-21T04:35:27.230 INFO:tasks.workunit.client.0.smithi007.stdout:4/497: creat d2/d3c/d63/fa3 x:0 0 0 2024-09-21T04:35:27.230 INFO:tasks.workunit.client.0.smithi007.stdout:4/498: read - d2/d7/d55/d61/f9a zero size 2024-09-21T04:35:27.237 INFO:tasks.workunit.client.0.smithi007.stdout:4/499: link d2/d7/d64/f68 d2/d88/fa4 0 2024-09-21T04:35:27.240 INFO:tasks.workunit.client.0.smithi007.stdout:4/500: fdatasync d2/d17/d2e/f7c 0 2024-09-21T04:35:27.240 INFO:tasks.workunit.client.0.smithi007.stdout:4/501: stat d2/f12 0 2024-09-21T04:35:27.246 INFO:tasks.workunit.client.0.smithi007.stdout:4/502: write d2/d7/f24 [1852782,24431] 0 2024-09-21T04:35:27.260 INFO:tasks.workunit.client.0.smithi007.stdout:4/503: write d2/f44 [660544,99924] 0 2024-09-21T04:35:27.261 INFO:tasks.workunit.client.0.smithi007.stdout:4/504: write d2/d3c/f89 [4208602,29999] 0 2024-09-21T04:35:27.272 INFO:tasks.workunit.client.0.smithi007.stdout:4/505: dread d2/d17/d2e/d45/f4b [0,4194304] 0 2024-09-21T04:35:27.273 INFO:tasks.workunit.client.0.smithi007.stdout:4/506: fsync d2/d3c/f89 0 2024-09-21T04:35:27.277 INFO:tasks.workunit.client.0.smithi007.stdout:4/507: rmdir d2/d3c/d63/d8c 0 2024-09-21T04:35:27.282 INFO:tasks.workunit.client.0.smithi007.stdout:4/508: dwrite d2/d5f/f93 [0,4194304] 0 2024-09-21T04:35:27.290 INFO:tasks.workunit.client.0.smithi007.stdout:4/509: mkdir d2/d97/da5 0 2024-09-21T04:35:27.291 INFO:tasks.workunit.client.0.smithi007.stdout:4/510: read - d2/f81 zero size 2024-09-21T04:35:27.299 INFO:tasks.workunit.client.0.smithi007.stdout:3/600: sync 2024-09-21T04:35:27.299 INFO:tasks.workunit.client.0.smithi007.stdout:9/459: sync 2024-09-21T04:35:27.299 INFO:tasks.workunit.client.0.smithi007.stdout:5/673: sync 2024-09-21T04:35:27.311 INFO:tasks.workunit.client.0.smithi007.stdout:9/460: mknod d13/d18/d60/c90 0 2024-09-21T04:35:27.316 INFO:tasks.workunit.client.0.smithi007.stdout:3/601: mkdir dc/d22/d29/d32/d4d/d84/dbb/dc1 0 2024-09-21T04:35:27.320 INFO:tasks.workunit.client.0.smithi007.stdout:5/674: creat d2/d27/d42/da3/fd8 x:0 0 0 2024-09-21T04:35:27.322 INFO:tasks.workunit.client.0.smithi007.stdout:9/461: symlink d13/d18/d60/d6d/d85/l91 0 2024-09-21T04:35:27.323 INFO:tasks.workunit.client.0.smithi007.stdout:3/602: creat dc/d22/d29/d32/d4d/fc2 x:0 0 0 2024-09-21T04:35:27.324 INFO:tasks.workunit.client.0.smithi007.stdout:4/511: getdents d2/d17/d2e/d45 0 2024-09-21T04:35:27.326 INFO:tasks.workunit.client.0.smithi007.stdout:3/603: rmdir dc/d22/d34/da6 39 2024-09-21T04:35:27.328 INFO:tasks.workunit.client.0.smithi007.stdout:3/604: truncate dc/d18/d26/f5f 720926 0 2024-09-21T04:35:27.330 INFO:tasks.workunit.client.0.smithi007.stdout:4/512: link d2/d7/f6c d2/d88/fa6 0 2024-09-21T04:35:27.330 INFO:tasks.workunit.client.0.smithi007.stdout:3/605: mknod dc/d22/d34/cc3 0 2024-09-21T04:35:27.331 INFO:tasks.workunit.client.0.smithi007.stdout:4/513: dread - d2/d17/f51 zero size 2024-09-21T04:35:27.339 INFO:tasks.workunit.client.0.smithi007.stdout:9/462: dread d13/d2a/f32 [0,4194304] 0 2024-09-21T04:35:27.340 INFO:tasks.workunit.client.0.smithi007.stdout:4/514: dread d2/d17/d2e/f33 [0,4194304] 0 2024-09-21T04:35:27.341 INFO:tasks.workunit.client.0.smithi007.stdout:9/463: chown d13/d18/d60/d6d/d85/l91 14 1 2024-09-21T04:35:27.342 INFO:tasks.workunit.client.0.smithi007.stdout:9/464: write d13/d18/d60/f88 [361773,12503] 0 2024-09-21T04:35:27.344 INFO:tasks.workunit.client.0.smithi007.stdout:5/675: write d2/d27/d42/da3/d23/d2a/f2d [5044786,89802] 0 2024-09-21T04:35:27.346 INFO:tasks.workunit.client.0.smithi007.stdout:3/606: dread fa [8388608,4194304] 0 2024-09-21T04:35:27.348 INFO:tasks.workunit.client.0.smithi007.stdout:4/515: fsync d2/d7/d55/d4f/f9d 0 2024-09-21T04:35:27.349 INFO:tasks.workunit.client.0.smithi007.stdout:4/516: fsync d2/d17/d2e/d45/d5e/f9c 0 2024-09-21T04:35:27.352 INFO:tasks.workunit.client.0.smithi007.stdout:5/676: rename d2/f51 to d2/d27/d3d/d9e/fd9 0 2024-09-21T04:35:27.357 INFO:tasks.workunit.client.0.smithi007.stdout:3/607: unlink dc/d22/d29/l3e 0 2024-09-21T04:35:27.358 INFO:tasks.workunit.client.0.smithi007.stdout:3/608: chown dc/d22/d29/d32/d69/d6d/f9b 1 1 2024-09-21T04:35:27.361 INFO:tasks.workunit.client.0.smithi007.stdout:4/517: unlink d2/d17/d2e/d45/f60 0 2024-09-21T04:35:27.362 INFO:tasks.workunit.client.0.smithi007.stdout:4/518: truncate d2/d7/d55/d4f/fa1 489395 0 2024-09-21T04:35:27.364 INFO:tasks.workunit.client.0.smithi007.stdout:5/677: mkdir d2/d27/d3d/dc2/dda 0 2024-09-21T04:35:27.366 INFO:tasks.workunit.client.0.smithi007.stdout:9/465: link d13/d5a/c7c d13/d2a/c92 0 2024-09-21T04:35:27.368 INFO:tasks.workunit.client.0.smithi007.stdout:4/519: rmdir d2/d7/d55/d4f 39 2024-09-21T04:35:27.370 INFO:tasks.workunit.client.0.smithi007.stdout:5/678: mkdir d2/d27/d42/da3/d23/d49/d6f/ddb 0 2024-09-21T04:35:27.371 INFO:tasks.workunit.client.0.smithi007.stdout:5/679: stat d2/d27/d42/da3/d23/d49/dd4/dd5 0 2024-09-21T04:35:27.373 INFO:tasks.workunit.client.0.smithi007.stdout:5/680: chown d2/c57 177998540 1 2024-09-21T04:35:27.374 INFO:tasks.workunit.client.0.smithi007.stdout:9/466: write d13/d18/f3f [569662,96951] 0 2024-09-21T04:35:27.380 INFO:tasks.workunit.client.0.smithi007.stdout:9/467: fsync d13/d18/f6e 0 2024-09-21T04:35:27.381 INFO:tasks.workunit.client.0.smithi007.stdout:9/468: stat d13/d18/d80/l8d 0 2024-09-21T04:35:27.381 INFO:tasks.workunit.client.0.smithi007.stdout:4/520: mknod d2/d17/d2e/d45/d5d/d8a/ca7 0 2024-09-21T04:35:27.382 INFO:tasks.workunit.client.0.smithi007.stdout:9/469: fsync d13/d5a/f61 0 2024-09-21T04:35:27.387 INFO:tasks.workunit.client.0.smithi007.stdout:9/470: symlink d13/d2a/l93 0 2024-09-21T04:35:27.387 INFO:tasks.workunit.client.0.smithi007.stdout:5/681: dwrite d2/d27/d42/da3/d23/d49/d44/d54/f59 [0,4194304] 0 2024-09-21T04:35:27.387 INFO:tasks.workunit.client.0.smithi007.stdout:4/521: mkdir d2/d88/da8 0 2024-09-21T04:35:27.388 INFO:tasks.workunit.client.0.smithi007.stdout:9/471: rename d13 to d13/d18/d80/d94 22 2024-09-21T04:35:27.389 INFO:tasks.workunit.client.0.smithi007.stdout:9/472: fdatasync d13/f4f 0 2024-09-21T04:35:27.392 INFO:tasks.workunit.client.0.smithi007.stdout:4/522: truncate d2/d17/d18/f2d 665687 0 2024-09-21T04:35:27.396 INFO:tasks.workunit.client.0.smithi007.stdout:4/523: creat d2/d7/d64/fa9 x:0 0 0 2024-09-21T04:35:27.400 INFO:tasks.workunit.client.0.smithi007.stdout:5/682: rename d2/d27/d42/da3/d23/d5d/d61/fa4 to d2/d27/d42/da3/d23/fdc 0 2024-09-21T04:35:27.403 INFO:tasks.workunit.client.0.smithi007.stdout:9/473: write d13/d18/f5c [561459,29306] 0 2024-09-21T04:35:27.405 INFO:tasks.workunit.client.0.smithi007.stdout:4/524: creat d2/d17/d2e/d45/d5e/faa x:0 0 0 2024-09-21T04:35:27.406 INFO:tasks.workunit.client.0.smithi007.stdout:9/474: creat d13/d18/d60/f95 x:0 0 0 2024-09-21T04:35:27.408 INFO:tasks.workunit.client.0.smithi007.stdout:5/683: rename d2/d27/d42/d53/daf/db0/fbe to d2/fdd 0 2024-09-21T04:35:27.410 INFO:tasks.workunit.client.0.smithi007.stdout:4/525: creat d2/d17/d2e/d9f/fab x:0 0 0 2024-09-21T04:35:27.411 INFO:tasks.workunit.client.0.smithi007.stdout:4/526: rmdir d2/d7/d64 39 2024-09-21T04:35:27.412 INFO:tasks.workunit.client.0.smithi007.stdout:9/475: dwrite d13/d18/d60/d83/f8f [0,4194304] 0 2024-09-21T04:35:27.413 INFO:tasks.workunit.client.0.smithi007.stdout:9/476: dread fa [4194304,4194304] 0 2024-09-21T04:35:27.415 INFO:tasks.workunit.client.0.smithi007.stdout:4/527: creat d2/d5f/fac x:0 0 0 2024-09-21T04:35:27.416 INFO:tasks.workunit.client.0.smithi007.stdout:9/477: fdatasync d13/d2a/f40 0 2024-09-21T04:35:27.418 INFO:tasks.workunit.client.0.smithi007.stdout:4/528: rename d2/f44 to d2/d97/da5/fad 0 2024-09-21T04:35:27.436 INFO:tasks.workunit.client.0.smithi007.stdout:9/478: write d13/d18/f3c [582867,75249] 0 2024-09-21T04:35:27.438 INFO:tasks.workunit.client.0.smithi007.stdout:9/479: symlink d13/d18/d60/d83/l96 0 2024-09-21T04:35:27.439 INFO:tasks.workunit.client.0.smithi007.stdout:9/480: write d13/d2a/f47 [958635,70055] 0 2024-09-21T04:35:27.441 INFO:tasks.workunit.client.0.smithi007.stdout:9/481: read d13/d18/d60/d6d/d75/f7e [2915091,32141] 0 2024-09-21T04:35:27.442 INFO:tasks.workunit.client.0.smithi007.stdout:9/482: write d13/d18/d60/f88 [746826,60528] 0 2024-09-21T04:35:27.446 INFO:tasks.workunit.client.0.smithi007.stdout:9/483: creat d13/d18/d60/d6d/d75/d7f/f97 x:0 0 0 2024-09-21T04:35:27.447 INFO:tasks.workunit.client.0.smithi007.stdout:9/484: rmdir d13/d2a 39 2024-09-21T04:35:27.448 INFO:tasks.workunit.client.0.smithi007.stdout:9/485: dread - d13/d18/d60/f63 zero size 2024-09-21T04:35:27.450 INFO:tasks.workunit.client.0.smithi007.stdout:9/486: creat d13/d74/f98 x:0 0 0 2024-09-21T04:35:27.453 INFO:tasks.workunit.client.0.smithi007.stdout:9/487: rename d13/d2a to d13/d74/d99 0 2024-09-21T04:35:27.471 INFO:tasks.workunit.client.0.smithi007.stdout:9/488: write d13/d74/d99/f2d [2433898,31545] 0 2024-09-21T04:35:27.476 INFO:tasks.workunit.client.0.smithi007.stdout:9/489: dread d13/d18/d60/f88 [0,4194304] 0 2024-09-21T04:35:27.478 INFO:tasks.workunit.client.0.smithi007.stdout:9/490: mknod d13/d74/c9a 0 2024-09-21T04:35:27.482 INFO:tasks.workunit.client.0.smithi007.stdout:9/491: dread d13/d18/f45 [0,4194304] 0 2024-09-21T04:35:27.483 INFO:tasks.workunit.client.0.smithi007.stdout:9/492: read d13/d18/f4b [1205930,104056] 0 2024-09-21T04:35:27.495 INFO:tasks.workunit.client.0.smithi007.stdout:9/493: dread d13/d18/f4b [0,4194304] 0 2024-09-21T04:35:27.497 INFO:tasks.workunit.client.0.smithi007.stdout:9/494: unlink d13/d74/d99/c59 0 2024-09-21T04:35:27.500 INFO:tasks.workunit.client.0.smithi007.stdout:9/495: rename d13/c29 to d13/d18/d60/d6d/c9b 0 2024-09-21T04:35:27.501 INFO:tasks.workunit.client.0.smithi007.stdout:9/496: readlink d13/d18/l65 0 2024-09-21T04:35:27.515 INFO:tasks.workunit.client.0.smithi007.stdout:9/497: dwrite d13/f1d [0,4194304] 0 2024-09-21T04:35:27.523 INFO:tasks.workunit.client.0.smithi007.stdout:9/498: dwrite d13/d74/d99/f56 [0,4194304] 0 2024-09-21T04:35:27.525 INFO:tasks.workunit.client.0.smithi007.stdout:9/499: write d13/d18/f35 [4455620,125104] 0 2024-09-21T04:35:27.526 INFO:tasks.workunit.client.0.smithi007.stdout:9/500: chown d13/d74/c9a 806745 1 2024-09-21T04:35:27.533 INFO:tasks.workunit.client.0.smithi007.stdout:9/501: link d13/d18/f45 d13/d18/d60/d6d/d75/f9c 0 2024-09-21T04:35:27.535 INFO:tasks.workunit.client.0.smithi007.stdout:9/502: fdatasync d13/d18/f2f 0 2024-09-21T04:35:27.535 INFO:tasks.workunit.client.0.smithi007.stdout:9/503: stat d13/d18/d60/d83/f8f 0 2024-09-21T04:35:27.538 INFO:tasks.workunit.client.0.smithi007.stdout:9/504: link d13/d18/f55 d13/d18/d60/d6d/f9d 0 2024-09-21T04:35:27.915 INFO:tasks.workunit.client.0.smithi007.stdout:1/624: sync 2024-09-21T04:35:27.916 INFO:tasks.workunit.client.0.smithi007.stdout:7/477: sync 2024-09-21T04:35:27.928 INFO:tasks.workunit.client.0.smithi007.stdout:1/625: write d5/d45/d5e/f24 [5812987,124906] 0 2024-09-21T04:35:27.928 INFO:tasks.workunit.client.0.smithi007.stdout:7/478: write d3/d6/d13/f2c [577095,40378] 0 2024-09-21T04:35:27.936 INFO:tasks.workunit.client.0.smithi007.stdout:7/479: link d3/d6/d2e/d63/c91 d3/d6/d2e/cac 0 2024-09-21T04:35:27.937 INFO:tasks.workunit.client.0.smithi007.stdout:1/626: dwrite d5/d6/d21/f8b [0,4194304] 0 2024-09-21T04:35:27.941 INFO:tasks.workunit.client.0.smithi007.stdout:7/480: mkdir d3/d6/d85/dad 0 2024-09-21T04:35:27.942 INFO:tasks.workunit.client.0.smithi007.stdout:7/481: chown d3/d6/d2e/d63/d6a/d81/fa0 4124080 1 2024-09-21T04:35:27.946 INFO:tasks.workunit.client.0.smithi007.stdout:7/482: unlink d3/d6/d10/d51/l89 0 2024-09-21T04:35:27.948 INFO:tasks.workunit.client.0.smithi007.stdout:7/483: mknod d3/d6/d14/d20/d80/cae 0 2024-09-21T04:35:27.950 INFO:tasks.workunit.client.0.smithi007.stdout:7/484: chown d3/d6/d2e/d5e/c76 14 1 2024-09-21T04:35:27.951 INFO:tasks.workunit.client.0.smithi007.stdout:7/485: write d3/d6/d85/f25 [663711,36854] 0 2024-09-21T04:35:27.951 INFO:tasks.workunit.client.0.smithi007.stdout:1/627: dread d5/d45/d5e/f76 [0,4194304] 0 2024-09-21T04:35:27.956 INFO:tasks.workunit.client.0.smithi007.stdout:7/486: rename d3/d6/d10/f24 to d3/d6/d40/faf 0 2024-09-21T04:35:27.959 INFO:tasks.workunit.client.0.smithi007.stdout:7/487: truncate d3/d6/d2e/d5e/f8a 1733623 0 2024-09-21T04:35:27.960 INFO:tasks.workunit.client.0.smithi007.stdout:7/488: rmdir d3/d6/d10/d62/d66 39 2024-09-21T04:35:27.963 INFO:tasks.workunit.client.0.smithi007.stdout:1/628: dwrite d5/d6/d21/d2b/d4b/da6/d58/f73 [4194304,4194304] 0 2024-09-21T04:35:27.967 INFO:tasks.workunit.client.0.smithi007.stdout:1/629: truncate d5/d6/d21/d3c/f43 1278811 0 2024-09-21T04:35:27.970 INFO:tasks.workunit.client.0.smithi007.stdout:1/630: creat d5/fd0 x:0 0 0 2024-09-21T04:35:27.977 INFO:tasks.workunit.client.0.smithi007.stdout:1/631: dwrite d5/d45/d94/f56 [0,4194304] 0 2024-09-21T04:35:27.987 INFO:tasks.workunit.client.0.smithi007.stdout:7/489: dwrite d3/f5 [0,4194304] 0 2024-09-21T04:35:27.988 INFO:tasks.workunit.client.0.smithi007.stdout:7/490: dread - d3/d6/d10/d62/f97 zero size 2024-09-21T04:35:27.990 INFO:tasks.workunit.client.0.smithi007.stdout:1/632: write d5/d6/d21/d2b/d4b/f79 [3609599,88777] 0 2024-09-21T04:35:27.994 INFO:tasks.workunit.client.0.smithi007.stdout:1/633: creat d5/d45/d94/da0/d99/dae/fd1 x:0 0 0 2024-09-21T04:35:27.995 INFO:tasks.workunit.client.0.smithi007.stdout:1/634: stat d5/d45/d5e/f24 0 2024-09-21T04:35:27.996 INFO:tasks.workunit.client.0.smithi007.stdout:7/491: write d3/d6/d85/f41 [2191648,93933] 0 2024-09-21T04:35:27.997 INFO:tasks.workunit.client.0.smithi007.stdout:7/492: stat d3/d6/d13/l59 0 2024-09-21T04:35:28.003 INFO:tasks.workunit.client.0.smithi007.stdout:7/493: rename d3/d6/d10/l8e to d3/d6/d10/d62/lb0 0 2024-09-21T04:35:28.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/635: dwrite d5/fe [0,4194304] 0 2024-09-21T04:35:28.020 INFO:tasks.workunit.client.0.smithi007.stdout:7/494: dwrite d3/d6/d85/f50 [0,4194304] 0 2024-09-21T04:35:28.020 INFO:tasks.workunit.client.0.smithi007.stdout:1/636: write d5/d6/d21/d2b/f2d [2157314,73696] 0 2024-09-21T04:35:28.022 INFO:tasks.workunit.client.0.smithi007.stdout:1/637: chown d5/d45/d94/cce 1 1 2024-09-21T04:35:28.022 INFO:tasks.workunit.client.0.smithi007.stdout:7/495: write d3/d6/d2e/d5e/f8a [1551100,97313] 0 2024-09-21T04:35:28.029 INFO:tasks.workunit.client.0.smithi007.stdout:1/638: fsync d5/d6/fa9 0 2024-09-21T04:35:28.038 INFO:tasks.workunit.client.0.smithi007.stdout:7/496: dwrite d3/d6/d13/f5d [0,4194304] 0 2024-09-21T04:35:28.040 INFO:tasks.workunit.client.0.smithi007.stdout:1/639: dwrite d5/d6/fa9 [4194304,4194304] 0 2024-09-21T04:35:28.044 INFO:tasks.workunit.client.0.smithi007.stdout:1/640: mknod d5/d6/d21/d3c/db6/cd2 0 2024-09-21T04:35:28.046 INFO:tasks.workunit.client.0.smithi007.stdout:7/497: creat d3/d6/d10/d62/d66/d69/fb1 x:0 0 0 2024-09-21T04:35:28.053 INFO:tasks.workunit.client.0.smithi007.stdout:1/641: dwrite d5/d45/f7d [0,4194304] 0 2024-09-21T04:35:28.057 INFO:tasks.workunit.client.0.smithi007.stdout:7/498: dread d3/d6/d14/d20/d80/f9c [4194304,4194304] 0 2024-09-21T04:35:28.059 INFO:tasks.workunit.client.0.smithi007.stdout:1/642: rename d5/d6/d21/d64/lba to d5/d6/d21/d3c/db6/ld3 0 2024-09-21T04:35:28.061 INFO:tasks.workunit.client.0.smithi007.stdout:1/643: chown d5/f48 1201 1 2024-09-21T04:35:28.064 INFO:tasks.workunit.client.0.smithi007.stdout:7/499: dread d3/d6/d13/f47 [0,4194304] 0 2024-09-21T04:35:28.065 INFO:tasks.workunit.client.0.smithi007.stdout:7/500: stat d3/d6/d10/d62 0 2024-09-21T04:35:28.073 INFO:tasks.workunit.client.0.smithi007.stdout:7/501: dwrite d3/d6/d10/d62/d66/d69/fb1 [0,4194304] 0 2024-09-21T04:35:28.076 INFO:tasks.workunit.client.0.smithi007.stdout:7/502: write d3/d6/d10/d62/d66/fa3 [677807,10220] 0 2024-09-21T04:35:28.080 INFO:tasks.workunit.client.0.smithi007.stdout:1/644: dread d5/d6/d21/d2b/d4b/d8a/d8d/fca [0,4194304] 0 2024-09-21T04:35:28.082 INFO:tasks.workunit.client.0.smithi007.stdout:7/503: unlink d3/d6/d2e/d63/c91 0 2024-09-21T04:35:28.085 INFO:tasks.workunit.client.0.smithi007.stdout:1/645: mkdir d5/d6/d21/dd4 0 2024-09-21T04:35:28.089 INFO:tasks.workunit.client.0.smithi007.stdout:1/646: mkdir d5/d6/d21/d3c/dd5 0 2024-09-21T04:35:28.091 INFO:tasks.workunit.client.0.smithi007.stdout:7/504: dwrite d3/d6/d10/d62/d66/fa3 [0,4194304] 0 2024-09-21T04:35:28.092 INFO:tasks.workunit.client.0.smithi007.stdout:1/647: truncate d5/d6/d21/d2b/d4b/f79 4337705 0 2024-09-21T04:35:28.095 INFO:tasks.workunit.client.0.smithi007.stdout:7/505: fsync d3/d6/d40/f48 0 2024-09-21T04:35:28.101 INFO:tasks.workunit.client.0.smithi007.stdout:7/506: rename d3/d6/d10/d62/d66/d69 to d3/d6/d14/d20/d80/db2 0 2024-09-21T04:35:28.105 INFO:tasks.workunit.client.0.smithi007.stdout:7/507: creat d3/d6/d10/d51/fb3 x:0 0 0 2024-09-21T04:35:28.113 INFO:tasks.workunit.client.0.smithi007.stdout:7/508: mknod d3/d6/d40/da1/cb4 0 2024-09-21T04:35:28.119 INFO:tasks.workunit.client.0.smithi007.stdout:1/648: dwrite d5/d45/d94/f78 [0,4194304] 0 2024-09-21T04:35:28.123 INFO:tasks.workunit.client.0.smithi007.stdout:7/509: creat d3/d6/d85/dad/fb5 x:0 0 0 2024-09-21T04:35:28.126 INFO:tasks.workunit.client.0.smithi007.stdout:1/649: rename d5/d6/d21/d2b/d4b/da6/f9d to d5/d6/d21/d2b/d4b/d8a/d8d/fd6 0 2024-09-21T04:35:28.127 INFO:tasks.workunit.client.0.smithi007.stdout:1/650: fsync d5/d45/d5e/f24 0 2024-09-21T04:35:28.130 INFO:tasks.workunit.client.0.smithi007.stdout:7/510: mkdir d3/d6/d14/d20/d80/db2/db6 0 2024-09-21T04:35:28.131 INFO:tasks.workunit.client.0.smithi007.stdout:7/511: fsync d3/d6/d14/d20/d80/f9d 0 2024-09-21T04:35:28.133 INFO:tasks.workunit.client.0.smithi007.stdout:1/651: creat d5/d6/d21/d2b/fd7 x:0 0 0 2024-09-21T04:35:28.137 INFO:tasks.workunit.client.0.smithi007.stdout:1/652: stat d5/f25 0 2024-09-21T04:35:28.141 INFO:tasks.workunit.client.0.smithi007.stdout:7/512: getdents d3/d6 0 2024-09-21T04:35:28.141 INFO:tasks.workunit.client.0.smithi007.stdout:1/653: truncate d5/d6/d21/f47 443197 0 2024-09-21T04:35:28.143 INFO:tasks.workunit.client.0.smithi007.stdout:1/654: readlink d5/l1e 0 2024-09-21T04:35:28.143 INFO:tasks.workunit.client.0.smithi007.stdout:7/513: dread d3/d6/d13/f47 [0,4194304] 0 2024-09-21T04:35:28.144 INFO:tasks.workunit.client.0.smithi007.stdout:1/655: write d5/d6/d21/f8b [4781381,39337] 0 2024-09-21T04:35:28.149 INFO:tasks.workunit.client.0.smithi007.stdout:7/514: truncate d3/d6/d10/f34 541263 0 2024-09-21T04:35:28.149 INFO:tasks.workunit.client.0.smithi007.stdout:7/515: chown d3/d6/d85/f50 13367541 1 2024-09-21T04:35:28.153 INFO:tasks.workunit.client.0.smithi007.stdout:7/516: rename d3/d6/d14/c60 to d3/d6/d14/d20/d58/da8/cb7 0 2024-09-21T04:35:28.153 INFO:tasks.workunit.client.0.smithi007.stdout:7/517: chown d3/d6 3041484 1 2024-09-21T04:35:28.161 INFO:tasks.workunit.client.0.smithi007.stdout:1/656: dread d5/d45/f46 [0,4194304] 0 2024-09-21T04:35:28.168 INFO:tasks.workunit.client.0.smithi007.stdout:6/468: sync 2024-09-21T04:35:28.172 INFO:tasks.workunit.client.0.smithi007.stdout:7/518: write d3/d6/d2e/d5e/f8a [955024,10075] 0 2024-09-21T04:35:28.180 INFO:tasks.workunit.client.0.smithi007.stdout:6/469: rename d3/d3e/d46/d60/d77/f76 to d3/f97 0 2024-09-21T04:35:28.182 INFO:tasks.workunit.client.0.smithi007.stdout:7/519: rename d3/d6/d10/d4f/l71 to d3/d6/d13/d4b/lb8 0 2024-09-21T04:35:28.182 INFO:tasks.workunit.client.0.smithi007.stdout:1/657: truncate d5/f39 1426325 0 2024-09-21T04:35:28.184 INFO:tasks.workunit.client.0.smithi007.stdout:1/658: chown d5/f90 7800693 1 2024-09-21T04:35:28.185 INFO:tasks.workunit.client.0.smithi007.stdout:7/520: dread d3/d6/d13/f47 [0,4194304] 0 2024-09-21T04:35:28.187 INFO:tasks.workunit.client.0.smithi007.stdout:6/470: dwrite d3/d3e/d46/f6d [0,4194304] 0 2024-09-21T04:35:28.189 INFO:tasks.workunit.client.0.smithi007.stdout:6/471: stat d3/d3b/c82 0 2024-09-21T04:35:28.191 INFO:tasks.workunit.client.0.smithi007.stdout:1/659: dread d5/d2f/f3a [0,4194304] 0 2024-09-21T04:35:28.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:27 smithi053 ceph-mon[35554]: Upgrade: Updating mgr.smithi007.huohfl 2024-09-21T04:35:28.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:27 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:28.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:27 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.huohfl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:35:28.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:27 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.huohfl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:35:28.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:27 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:35:28.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:27 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:35:28.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:27 smithi053 ceph-mon[35554]: Deploying daemon mgr.smithi007.huohfl on smithi007 2024-09-21T04:35:28.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:27 smithi053 ceph-mon[35554]: pgmap v35: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 79 MiB/s rd, 203 MiB/s wr, 418 op/s 2024-09-21T04:35:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:27 smithi007 ceph-mon[27954]: Upgrade: Updating mgr.smithi007.huohfl 2024-09-21T04:35:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:27 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:27 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.huohfl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:35:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:27 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.huohfl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:35:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:27 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:35:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:27 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:35:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:27 smithi007 ceph-mon[27954]: Deploying daemon mgr.smithi007.huohfl on smithi007 2024-09-21T04:35:28.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:27 smithi007 ceph-mon[27954]: pgmap v35: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 79 MiB/s rd, 203 MiB/s wr, 418 op/s 2024-09-21T04:35:28.207 INFO:tasks.workunit.client.0.smithi007.stdout:7/521: unlink d3/d6/d14/d20/d80/f6f 0 2024-09-21T04:35:28.209 INFO:tasks.workunit.client.0.smithi007.stdout:7/522: read d3/d6/d2e/d5e/f93 [303057,88255] 0 2024-09-21T04:35:28.212 INFO:tasks.workunit.client.0.smithi007.stdout:1/660: chown d5/d6/d21/d2b/d4b/da6/f65 234623640 1 2024-09-21T04:35:28.217 INFO:tasks.workunit.client.0.smithi007.stdout:6/472: truncate d3/d3e/d46/f6d 2631249 0 2024-09-21T04:35:28.218 INFO:tasks.workunit.client.0.smithi007.stdout:7/523: dwrite d3/d6/f28 [8388608,4194304] 0 2024-09-21T04:35:28.226 INFO:tasks.workunit.client.0.smithi007.stdout:6/473: dwrite d3/de/d12/d19/f39 [0,4194304] 0 2024-09-21T04:35:28.233 INFO:tasks.workunit.client.0.smithi007.stdout:6/474: dread d3/f33 [0,4194304] 0 2024-09-21T04:35:28.237 INFO:tasks.workunit.client.0.smithi007.stdout:6/475: mkdir d3/d3e/d46/d60/d98 0 2024-09-21T04:35:28.239 INFO:tasks.workunit.client.0.smithi007.stdout:1/661: dread d5/d6/d21/d2b/d61/f6e [0,4194304] 0 2024-09-21T04:35:28.242 INFO:tasks.workunit.client.0.smithi007.stdout:6/476: dread d3/d94/f8a [0,4194304] 0 2024-09-21T04:35:28.243 INFO:tasks.workunit.client.0.smithi007.stdout:1/662: rename d5/d6/d21/d2b/d61 to d5/d6/d21/d2b/d86/dd8 0 2024-09-21T04:35:28.243 INFO:tasks.workunit.client.0.smithi007.stdout:1/663: stat d5/d45/d94/f9e 0 2024-09-21T04:35:28.245 INFO:tasks.workunit.client.0.smithi007.stdout:6/477: mkdir d3/d3e/d99 0 2024-09-21T04:35:28.245 INFO:tasks.workunit.client.0.smithi007.stdout:1/664: dread - d5/d45/d5e/fc1 zero size 2024-09-21T04:35:28.246 INFO:tasks.workunit.client.0.smithi007.stdout:1/665: dread - d5/d45/d5e/fc1 zero size 2024-09-21T04:35:28.247 INFO:tasks.workunit.client.0.smithi007.stdout:6/478: dread d3/de/d12/d19/f51 [0,4194304] 0 2024-09-21T04:35:28.248 INFO:tasks.workunit.client.0.smithi007.stdout:6/479: fsync d3/d3e/d46/f81 0 2024-09-21T04:35:28.249 INFO:tasks.workunit.client.0.smithi007.stdout:1/666: mkdir d5/d6/dd9 0 2024-09-21T04:35:28.250 INFO:tasks.workunit.client.0.smithi007.stdout:6/480: rmdir d3/d3e/d46 39 2024-09-21T04:35:28.256 INFO:tasks.workunit.client.0.smithi007.stdout:6/481: link d3/d3e/d46/d60/f93 d3/de/d4e/f9a 0 2024-09-21T04:35:28.259 INFO:tasks.workunit.client.0.smithi007.stdout:6/482: mkdir d3/de/d4e/d9b 0 2024-09-21T04:35:28.261 INFO:tasks.workunit.client.0.smithi007.stdout:6/483: mknod d3/d3e/d99/c9c 0 2024-09-21T04:35:28.264 INFO:tasks.workunit.client.0.smithi007.stdout:6/484: rename d3/de/d4e/d9b to d3/d3e/d85/d9d 0 2024-09-21T04:35:28.268 INFO:tasks.workunit.client.0.smithi007.stdout:6/485: rename d3/de/d4e/d67/f75 to d3/f9e 0 2024-09-21T04:35:28.268 INFO:tasks.workunit.client.0.smithi007.stdout:6/486: dread - d3/d3e/d46/f81 zero size 2024-09-21T04:35:28.271 INFO:tasks.workunit.client.0.smithi007.stdout:6/487: creat d3/de/d4e/f9f x:0 0 0 2024-09-21T04:35:28.275 INFO:tasks.workunit.client.0.smithi007.stdout:6/488: mknod d3/d3e/ca0 0 2024-09-21T04:35:28.278 INFO:tasks.workunit.client.0.smithi007.stdout:6/489: symlink d3/de/d12/d19/la1 0 2024-09-21T04:35:28.279 INFO:tasks.workunit.client.0.smithi007.stdout:6/490: stat d3/de/d4e/f9a 0 2024-09-21T04:35:28.281 INFO:tasks.workunit.client.0.smithi007.stdout:6/491: mknod d3/d8e/ca2 0 2024-09-21T04:35:28.282 INFO:tasks.workunit.client.0.smithi007.stdout:6/492: write d3/d3e/d46/d60/d77/f69 [655635,46349] 0 2024-09-21T04:35:28.283 INFO:tasks.workunit.client.0.smithi007.stdout:6/493: readlink d3/d3e/l80 0 2024-09-21T04:35:28.284 INFO:tasks.workunit.client.0.smithi007.stdout:6/494: chown d3/de/d12/f8d 92253478 1 2024-09-21T04:35:28.310 INFO:tasks.workunit.client.0.smithi007.stdout:1/667: dwrite d5/d2f/f95 [0,4194304] 0 2024-09-21T04:35:28.315 INFO:tasks.workunit.client.0.smithi007.stdout:1/668: fdatasync d5/d6/f40 0 2024-09-21T04:35:28.316 INFO:tasks.workunit.client.0.smithi007.stdout:1/669: dread - d5/d6/d21/d3c/fa3 zero size 2024-09-21T04:35:28.320 INFO:tasks.workunit.client.0.smithi007.stdout:1/670: mkdir d5/d6/d21/d3c/dda 0 2024-09-21T04:35:28.325 INFO:tasks.workunit.client.0.smithi007.stdout:1/671: fdatasync d5/d45/f46 0 2024-09-21T04:35:28.334 INFO:tasks.workunit.client.0.smithi007.stdout:6/495: write d3/f38 [4592758,102346] 0 2024-09-21T04:35:28.339 INFO:tasks.workunit.client.0.smithi007.stdout:6/496: mkdir d3/de/d12/d19/da3 0 2024-09-21T04:35:28.345 INFO:tasks.workunit.client.0.smithi007.stdout:1/672: dread d5/d45/d94/f62 [0,4194304] 0 2024-09-21T04:35:28.350 INFO:tasks.workunit.client.0.smithi007.stdout:1/673: unlink d5/d45/f46 0 2024-09-21T04:35:28.353 INFO:tasks.workunit.client.0.smithi007.stdout:1/674: symlink d5/d6/d21/d2b/d4b/da6/ldb 0 2024-09-21T04:35:28.354 INFO:tasks.workunit.client.0.smithi007.stdout:1/675: dread - d5/d45/fb0 zero size 2024-09-21T04:35:28.359 INFO:tasks.workunit.client.0.smithi007.stdout:1/676: write d5/f39 [1509909,46868] 0 2024-09-21T04:35:28.361 INFO:tasks.workunit.client.0.smithi007.stdout:1/677: read d5/d6/d21/d2b/d4b/f70 [1584548,46781] 0 2024-09-21T04:35:28.362 INFO:tasks.workunit.client.0.smithi007.stdout:1/678: fdatasync d5/d45/d94/f78 0 2024-09-21T04:35:28.372 INFO:tasks.workunit.client.0.smithi007.stdout:1/679: dwrite d5/d6/d21/d2b/fd7 [0,4194304] 0 2024-09-21T04:35:28.372 INFO:tasks.workunit.client.0.smithi007.stdout:6/497: dwrite d3/de/d12/f2b [0,4194304] 0 2024-09-21T04:35:28.375 INFO:tasks.workunit.client.0.smithi007.stdout:1/680: stat d5/d6/d21/d2b/fd7 0 2024-09-21T04:35:28.419 INFO:tasks.workunit.client.0.smithi007.stdout:6/498: fdatasync d3/d3e/f55 0 2024-09-21T04:35:28.419 INFO:tasks.workunit.client.0.smithi007.stdout:6/499: chown d3/d3e/d85/d9d 7816054 1 2024-09-21T04:35:28.419 INFO:tasks.workunit.client.0.smithi007.stdout:1/681: getdents d5/d6/dc5 0 2024-09-21T04:35:28.419 INFO:tasks.workunit.client.0.smithi007.stdout:6/500: write d3/de/d12/d19/f39 [713885,83874] 0 2024-09-21T04:35:28.419 INFO:tasks.workunit.client.0.smithi007.stdout:6/501: write d3/f2f [2720841,17625] 0 2024-09-21T04:35:28.421 INFO:tasks.workunit.client.0.smithi007.stdout:6/502: truncate d3/de/d12/f2b 1872738 0 2024-09-21T04:35:28.422 INFO:tasks.workunit.client.0.smithi007.stdout:6/503: dread - d3/d3e/d85/f92 zero size 2024-09-21T04:35:28.427 INFO:tasks.workunit.client.0.smithi007.stdout:1/682: write d5/d6/d21/f47 [1181616,123212] 0 2024-09-21T04:35:28.429 INFO:tasks.workunit.client.0.smithi007.stdout:1/683: chown d5/d6/d21/d2b/d86/dd8/fb4 1412 1 2024-09-21T04:35:28.430 INFO:tasks.workunit.client.0.smithi007.stdout:1/684: write d5/f5d [4333726,67324] 0 2024-09-21T04:35:28.432 INFO:tasks.workunit.client.0.smithi007.stdout:6/504: link d3/de/f49 d3/d3e/d85/fa4 0 2024-09-21T04:35:28.436 INFO:tasks.workunit.client.0.smithi007.stdout:1/685: rename d5/d6/d21/l96 to d5/d6/d21/d3c/dda/ldc 0 2024-09-21T04:35:28.439 INFO:tasks.workunit.client.0.smithi007.stdout:6/505: creat d3/d3e/d99/fa5 x:0 0 0 2024-09-21T04:35:28.439 INFO:tasks.workunit.client.0.smithi007.stdout:1/686: mkdir d5/d45/d94/ddd 0 2024-09-21T04:35:28.440 INFO:tasks.workunit.client.0.smithi007.stdout:1/687: chown d5/d45/d94/f78 100 1 2024-09-21T04:35:28.441 INFO:tasks.workunit.client.0.smithi007.stdout:6/506: chown d3/de/d12/d19/f1c 1 1 2024-09-21T04:35:28.448 INFO:tasks.workunit.client.0.smithi007.stdout:6/507: creat d3/d3e/d85/fa6 x:0 0 0 2024-09-21T04:35:28.451 INFO:tasks.workunit.client.0.smithi007.stdout:6/508: stat d3/de/lf 0 2024-09-21T04:35:28.455 INFO:tasks.workunit.client.0.smithi007.stdout:6/509: rename d3/de/d12/d19/c2a to d3/d94/ca7 0 2024-09-21T04:35:28.456 INFO:tasks.workunit.client.0.smithi007.stdout:1/688: dread d5/f5b [0,4194304] 0 2024-09-21T04:35:28.458 INFO:tasks.workunit.client.0.smithi007.stdout:6/510: read d3/d3e/f40 [3469239,25800] 0 2024-09-21T04:35:28.464 INFO:tasks.workunit.client.0.smithi007.stdout:6/511: symlink d3/d94/la8 0 2024-09-21T04:35:28.465 INFO:tasks.workunit.client.0.smithi007.stdout:1/689: dread d5/d6/d21/d2b/d4b/da6/d58/fa5 [0,4194304] 0 2024-09-21T04:35:28.467 INFO:tasks.workunit.client.0.smithi007.stdout:1/690: write d5/fd0 [651904,37631] 0 2024-09-21T04:35:28.467 INFO:tasks.workunit.client.0.smithi007.stdout:6/512: truncate d3/d3e/d46/f81 1024612 0 2024-09-21T04:35:28.470 INFO:tasks.workunit.client.0.smithi007.stdout:1/691: dread d5/d6/d21/d2b/d4b/f79 [4194304,4194304] 0 2024-09-21T04:35:28.472 INFO:tasks.workunit.client.0.smithi007.stdout:6/513: dread d3/de/d12/d19/f64 [0,4194304] 0 2024-09-21T04:35:28.474 INFO:tasks.workunit.client.0.smithi007.stdout:1/692: mkdir d5/d6/d21/d2b/d4b/da6/d58/dde 0 2024-09-21T04:35:28.487 INFO:tasks.workunit.client.0.smithi007.stdout:6/514: write d3/f28 [4510956,11241] 0 2024-09-21T04:35:28.487 INFO:tasks.workunit.client.0.smithi007.stdout:1/693: write d5/d6/d21/d2b/f63 [80089,16834] 0 2024-09-21T04:35:28.489 INFO:tasks.workunit.client.0.smithi007.stdout:1/694: write d5/d45/d94/da0/fad [1146350,80418] 0 2024-09-21T04:35:28.489 INFO:tasks.workunit.client.0.smithi007.stdout:6/515: truncate d3/d3e/d46/f81 1458856 0 2024-09-21T04:35:28.493 INFO:tasks.workunit.client.0.smithi007.stdout:6/516: creat d3/d3e/d85/fa9 x:0 0 0 2024-09-21T04:35:28.498 INFO:tasks.workunit.client.0.smithi007.stdout:6/517: rename d3/d3e/d85/fa4 to d3/de/d58/faa 0 2024-09-21T04:35:28.501 INFO:tasks.workunit.client.0.smithi007.stdout:1/695: dread d5/d6/d21/d2b/d4b/da6/f9c [0,4194304] 0 2024-09-21T04:35:28.502 INFO:tasks.workunit.client.0.smithi007.stdout:1/696: chown d5/d45/d94/da0/d99/cb5 702952 1 2024-09-21T04:35:28.504 INFO:tasks.workunit.client.0.smithi007.stdout:0/595: sync 2024-09-21T04:35:28.504 INFO:tasks.workunit.client.0.smithi007.stdout:5/684: sync 2024-09-21T04:35:28.504 INFO:tasks.workunit.client.0.smithi007.stdout:2/412: sync 2024-09-21T04:35:28.504 INFO:tasks.workunit.client.0.smithi007.stdout:8/269: sync 2024-09-21T04:35:28.504 INFO:tasks.workunit.client.0.smithi007.stdout:4/529: sync 2024-09-21T04:35:28.504 INFO:tasks.workunit.client.0.smithi007.stdout:3/609: sync 2024-09-21T04:35:28.506 INFO:tasks.workunit.client.0.smithi007.stdout:0/596: write d6/f38 [338789,67702] 0 2024-09-21T04:35:28.507 INFO:tasks.workunit.client.0.smithi007.stdout:1/697: creat d5/d45/d94/ddd/fdf x:0 0 0 2024-09-21T04:35:28.508 INFO:tasks.workunit.client.0.smithi007.stdout:2/413: chown d0/d11/d31/d39/f44 201 1 2024-09-21T04:35:28.509 INFO:tasks.workunit.client.0.smithi007.stdout:8/270: symlink d0/d3/d16/l41 0 2024-09-21T04:35:28.510 INFO:tasks.workunit.client.0.smithi007.stdout:5/685: truncate d2/d27/d42/d53/f90 614284 0 2024-09-21T04:35:28.511 INFO:tasks.workunit.client.0.smithi007.stdout:6/518: write d3/f5c [18618,75702] 0 2024-09-21T04:35:28.512 INFO:tasks.workunit.client.0.smithi007.stdout:4/530: readlink d2/d17/l62 0 2024-09-21T04:35:28.515 INFO:tasks.workunit.client.0.smithi007.stdout:2/414: fsync d0/d11/d31/d39/f4f 0 2024-09-21T04:35:28.520 INFO:tasks.workunit.client.0.smithi007.stdout:1/698: dwrite d5/fd0 [0,4194304] 0 2024-09-21T04:35:28.527 INFO:tasks.workunit.client.0.smithi007.stdout:4/531: mkdir d2/d17/d2e/d45/d5d/dae 0 2024-09-21T04:35:28.528 INFO:tasks.workunit.client.0.smithi007.stdout:3/610: truncate dc/d22/d29/d32/d4d/f8e 2287562 0 2024-09-21T04:35:28.530 INFO:tasks.workunit.client.0.smithi007.stdout:4/532: dread d2/d7/d55/f92 [0,4194304] 0 2024-09-21T04:35:28.530 INFO:tasks.workunit.client.0.smithi007.stdout:6/519: dwrite d3/d3e/f55 [0,4194304] 0 2024-09-21T04:35:28.534 INFO:tasks.workunit.client.0.smithi007.stdout:0/597: creat d6/d75/dc5/d18/fde x:0 0 0 2024-09-21T04:35:28.551 INFO:tasks.workunit.client.0.smithi007.stdout:8/271: getdents d0/d3/d16 0 2024-09-21T04:35:28.552 INFO:tasks.workunit.client.0.smithi007.stdout:1/699: write d5/d6/d21/d2b/f6f [1281396,84843] 0 2024-09-21T04:35:28.558 INFO:tasks.workunit.client.0.smithi007.stdout:3/611: creat dc/d22/d34/da6/fc4 x:0 0 0 2024-09-21T04:35:28.559 INFO:tasks.workunit.client.0.smithi007.stdout:4/533: mknod d2/d7/d64/caf 0 2024-09-21T04:35:28.559 INFO:tasks.workunit.client.0.smithi007.stdout:5/686: dwrite d2/d27/d42/da3/d23/fc5 [0,4194304] 0 2024-09-21T04:35:28.561 INFO:tasks.workunit.client.0.smithi007.stdout:6/520: dwrite d3/de/d12/f29 [0,4194304] 0 2024-09-21T04:35:28.564 INFO:tasks.workunit.client.0.smithi007.stdout:3/612: truncate dc/d18/fbd 30600 0 2024-09-21T04:35:28.573 INFO:tasks.workunit.client.0.smithi007.stdout:1/700: unlink d5/d6/d21/d2b/d86/fc8 0 2024-09-21T04:35:28.574 INFO:tasks.workunit.client.0.smithi007.stdout:1/701: stat d5/f39 0 2024-09-21T04:35:28.576 INFO:tasks.workunit.client.0.smithi007.stdout:0/598: write d6/d75/dc5/d18/f69 [1187320,103764] 0 2024-09-21T04:35:28.578 INFO:tasks.workunit.client.0.smithi007.stdout:2/415: rename d0/d11/d31/d46/f55 to d0/d11/d31/f6f 0 2024-09-21T04:35:28.579 INFO:tasks.workunit.client.0.smithi007.stdout:2/416: chown d0/d11/f4b 18426377 1 2024-09-21T04:35:28.579 INFO:tasks.workunit.client.0.smithi007.stdout:5/687: unlink d2/d27/d3d/d9b/cd7 0 2024-09-21T04:35:28.580 INFO:tasks.workunit.client.0.smithi007.stdout:2/417: chown d0/d2/d57/f5a 118 1 2024-09-21T04:35:28.580 INFO:tasks.workunit.client.0.smithi007.stdout:2/418: fsync d0/f27 0 2024-09-21T04:35:28.581 INFO:tasks.workunit.client.0.smithi007.stdout:5/688: write d2/d27/d42/da3/d23/d2a/f75 [161665,5547] 0 2024-09-21T04:35:28.582 INFO:tasks.workunit.client.0.smithi007.stdout:4/534: creat d2/d97/da5/fb0 x:0 0 0 2024-09-21T04:35:28.583 INFO:tasks.workunit.client.0.smithi007.stdout:4/535: stat d2/d17/d18/c28 0 2024-09-21T04:35:28.584 INFO:tasks.workunit.client.0.smithi007.stdout:4/536: chown d2/d17 7743095 1 2024-09-21T04:35:28.594 INFO:tasks.workunit.client.0.smithi007.stdout:5/689: unlink d2/d27/d42/da3/d23/d2a/lc3 0 2024-09-21T04:35:28.595 INFO:tasks.workunit.client.0.smithi007.stdout:2/419: dread d0/d2/fc [0,4194304] 0 2024-09-21T04:35:28.595 INFO:tasks.workunit.client.0.smithi007.stdout:6/521: dwrite d3/de/d12/f21 [0,4194304] 0 2024-09-21T04:35:28.597 INFO:tasks.workunit.client.0.smithi007.stdout:4/537: truncate d2/d3c/f53 1321191 0 2024-09-21T04:35:28.602 INFO:tasks.workunit.client.0.smithi007.stdout:0/599: unlink d6/d75/dc5/d18/d2e/d3f/ld7 0 2024-09-21T04:35:28.606 INFO:tasks.workunit.client.0.smithi007.stdout:2/420: mknod d0/d11/d31/d52/c70 0 2024-09-21T04:35:28.608 INFO:tasks.workunit.client.0.smithi007.stdout:8/272: getdents d0/d3/d8/de 0 2024-09-21T04:35:28.609 INFO:tasks.workunit.client.0.smithi007.stdout:1/702: link d5/d45/d94/lbf d5/d6/d21/dd4/le0 0 2024-09-21T04:35:28.611 INFO:tasks.workunit.client.0.smithi007.stdout:4/538: creat d2/d17/d2e/d45/d5d/fb1 x:0 0 0 2024-09-21T04:35:28.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/600: mknod d6/d75/dc5/d57/d9d/daf/cdf 0 2024-09-21T04:35:28.615 INFO:tasks.workunit.client.0.smithi007.stdout:6/522: symlink d3/lab 0 2024-09-21T04:35:28.616 INFO:tasks.workunit.client.0.smithi007.stdout:5/690: write d2/f50 [1001629,97281] 0 2024-09-21T04:35:28.617 INFO:tasks.workunit.client.0.smithi007.stdout:3/613: getdents dc/d22/d29/d32/d69/d6d 0 2024-09-21T04:35:28.619 INFO:tasks.workunit.client.0.smithi007.stdout:3/614: write dc/d18/d26/f5f [1419858,48641] 0 2024-09-21T04:35:28.621 INFO:tasks.workunit.client.0.smithi007.stdout:6/523: dread d3/de/d12/f2b [0,4194304] 0 2024-09-21T04:35:28.625 INFO:tasks.workunit.client.0.smithi007.stdout:0/601: write d6/d75/dc5/d47/d99/fbc [701729,27428] 0 2024-09-21T04:35:28.626 INFO:tasks.workunit.client.0.smithi007.stdout:1/703: dread d5/d6/d21/d3c/f43 [0,4194304] 0 2024-09-21T04:35:28.627 INFO:tasks.workunit.client.0.smithi007.stdout:6/524: write d3/d3e/d99/fa5 [666918,108495] 0 2024-09-21T04:35:28.628 INFO:tasks.workunit.client.0.smithi007.stdout:8/273: truncate d0/d3/d8/fa 1518729 0 2024-09-21T04:35:28.629 INFO:tasks.workunit.client.0.smithi007.stdout:4/539: mkdir d2/d88/da8/db2 0 2024-09-21T04:35:28.630 INFO:tasks.workunit.client.0.smithi007.stdout:5/691: dwrite d2/f5a [0,4194304] 0 2024-09-21T04:35:28.631 INFO:tasks.workunit.client.0.smithi007.stdout:6/525: dread - d3/d3e/d85/fa9 zero size 2024-09-21T04:35:28.632 INFO:tasks.workunit.client.0.smithi007.stdout:3/615: creat dc/d22/d29/d32/d4d/d7d/d9a/fc5 x:0 0 0 2024-09-21T04:35:28.633 INFO:tasks.workunit.client.0.smithi007.stdout:6/526: stat d3/d3e/d46/f68 0 2024-09-21T04:35:28.633 INFO:tasks.workunit.client.0.smithi007.stdout:8/274: truncate d0/d3/d8/f3f 192696 0 2024-09-21T04:35:28.638 INFO:tasks.workunit.client.0.smithi007.stdout:4/540: unlink d2/d5f/f87 0 2024-09-21T04:35:28.642 INFO:tasks.workunit.client.0.smithi007.stdout:3/616: creat dc/d22/d5a/fc6 x:0 0 0 2024-09-21T04:35:28.648 INFO:tasks.workunit.client.0.smithi007.stdout:2/421: dwrite d0/d11/d31/f6f [0,4194304] 0 2024-09-21T04:35:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:0/602: rename d6/d75/dc5/d47/f6b to d6/d75/dc5/d18/d39/fe0 0 2024-09-21T04:35:28.657 INFO:tasks.workunit.client.0.smithi007.stdout:1/704: dwrite d5/f28 [0,4194304] 0 2024-09-21T04:35:28.659 INFO:tasks.workunit.client.0.smithi007.stdout:6/527: creat d3/de/d12/d19/da3/fac x:0 0 0 2024-09-21T04:35:28.663 INFO:tasks.workunit.client.0.smithi007.stdout:1/705: fsync d5/d45/d94/fb1 0 2024-09-21T04:35:28.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/275: dwrite d0/d3/d8/de/f18 [4194304,4194304] 0 2024-09-21T04:35:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:1/706: write d5/d45/d94/f56 [3209842,103521] 0 2024-09-21T04:35:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:3/617: rename dc/d22/c66 to dc/d18/d41/cc7 0 2024-09-21T04:35:28.674 INFO:tasks.workunit.client.0.smithi007.stdout:6/528: fsync d3/f15 0 2024-09-21T04:35:28.675 INFO:tasks.workunit.client.0.smithi007.stdout:0/603: dread d6/d75/f8f [0,4194304] 0 2024-09-21T04:35:28.676 INFO:tasks.workunit.client.0.smithi007.stdout:4/541: dwrite d2/d7/fa [0,4194304] 0 2024-09-21T04:35:28.677 INFO:tasks.workunit.client.0.smithi007.stdout:4/542: read - d2/f81 zero size 2024-09-21T04:35:28.677 INFO:tasks.workunit.client.0.smithi007.stdout:2/422: unlink d0/d2/f16 0 2024-09-21T04:35:28.678 INFO:tasks.workunit.client.0.smithi007.stdout:1/707: truncate d5/d6/d21/d3c/fa3 559385 0 2024-09-21T04:35:28.679 INFO:tasks.workunit.client.0.smithi007.stdout:8/276: mkdir d0/d3/d16/d3b/d42 0 2024-09-21T04:35:28.680 INFO:tasks.workunit.client.0.smithi007.stdout:4/543: read d2/f43 [4108321,74747] 0 2024-09-21T04:35:28.683 INFO:tasks.workunit.client.0.smithi007.stdout:3/618: mkdir dc/d22/dc8 0 2024-09-21T04:35:28.684 INFO:tasks.workunit.client.0.smithi007.stdout:0/604: stat d6/d75/dc5/d18/d39/fe0 0 2024-09-21T04:35:28.689 INFO:tasks.workunit.client.0.smithi007.stdout:2/423: symlink d0/d11/d31/d46/d48/l71 0 2024-09-21T04:35:28.689 INFO:tasks.workunit.client.0.smithi007.stdout:4/544: read d2/f6a [1224781,35663] 0 2024-09-21T04:35:28.690 INFO:tasks.workunit.client.0.smithi007.stdout:2/424: chown d0/d11/d31/d52/c5b 4 1 2024-09-21T04:35:28.696 INFO:tasks.workunit.client.0.smithi007.stdout:1/708: rename d5/d6/d21/d2b/d4b/d8a/f97 to d5/d6/d21/fe1 0 2024-09-21T04:35:28.700 INFO:tasks.workunit.client.0.smithi007.stdout:2/425: mknod d0/d11/d31/d46/c72 0 2024-09-21T04:35:28.704 INFO:tasks.workunit.client.0.smithi007.stdout:8/277: unlink d0/d3/d8/de/d26/c32 0 2024-09-21T04:35:28.707 INFO:tasks.workunit.client.0.smithi007.stdout:3/619: unlink dc/d22/d29/d32/d4d/f73 0 2024-09-21T04:35:28.708 INFO:tasks.workunit.client.0.smithi007.stdout:1/709: mkdir d5/d45/d94/da0/d99/de2 0 2024-09-21T04:35:28.709 INFO:tasks.workunit.client.0.smithi007.stdout:0/605: mkdir d6/d75/dc5/d52/d6d/de1 0 2024-09-21T04:35:28.710 INFO:tasks.workunit.client.0.smithi007.stdout:6/529: link d3/d3b/c82 d3/de/d4e/cad 0 2024-09-21T04:35:28.711 INFO:tasks.workunit.client.0.smithi007.stdout:6/530: fsync d3/de/f4f 0 2024-09-21T04:35:28.713 INFO:tasks.workunit.client.0.smithi007.stdout:4/545: symlink d2/d17/d2e/d45/d5d/dae/lb3 0 2024-09-21T04:35:28.713 INFO:tasks.workunit.client.0.smithi007.stdout:4/546: readlink d2/d17/l4d 0 2024-09-21T04:35:28.718 INFO:tasks.workunit.client.0.smithi007.stdout:3/620: symlink dc/d56/lc9 0 2024-09-21T04:35:28.724 INFO:tasks.workunit.client.0.smithi007.stdout:1/710: creat d5/d6/da2/fe3 x:0 0 0 2024-09-21T04:35:28.725 INFO:tasks.workunit.client.0.smithi007.stdout:3/621: dwrite dc/d18/d41/f4a [0,4194304] 0 2024-09-21T04:35:28.727 INFO:tasks.workunit.client.0.smithi007.stdout:0/606: rename d6/ca to d6/d75/dc5/d18/d27/ce2 0 2024-09-21T04:35:28.732 INFO:tasks.workunit.client.0.smithi007.stdout:8/278: write d0/d3/d16/f1b [1896207,80378] 0 2024-09-21T04:35:28.732 INFO:tasks.workunit.client.0.smithi007.stdout:3/622: dwrite dc/d22/d29/f6a [0,4194304] 0 2024-09-21T04:35:28.735 INFO:tasks.workunit.client.0.smithi007.stdout:6/531: mkdir d3/d8e/dae 0 2024-09-21T04:35:28.736 INFO:tasks.workunit.client.0.smithi007.stdout:4/547: unlink d2/d17/d2e/f78 0 2024-09-21T04:35:28.739 INFO:tasks.workunit.client.0.smithi007.stdout:1/711: mknod d5/d6/d21/d64/ce4 0 2024-09-21T04:35:28.746 INFO:tasks.workunit.client.0.smithi007.stdout:2/426: dwrite d0/d11/f28 [0,4194304] 0 2024-09-21T04:35:28.747 INFO:tasks.workunit.client.0.smithi007.stdout:4/548: rmdir d2/d7/d64 39 2024-09-21T04:35:28.750 INFO:tasks.workunit.client.0.smithi007.stdout:1/712: creat d5/d6/d21/d3c/dda/fe5 x:0 0 0 2024-09-21T04:35:28.752 INFO:tasks.workunit.client.0.smithi007.stdout:1/713: chown d5/d6/d21/d2b/d4b/da6/l82 18614945 1 2024-09-21T04:35:28.752 INFO:tasks.workunit.client.0.smithi007.stdout:8/279: dread d0/d3/f2b [0,4194304] 0 2024-09-21T04:35:28.753 INFO:tasks.workunit.client.0.smithi007.stdout:1/714: chown d5/fc3 64533 1 2024-09-21T04:35:28.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/623: mkdir dc/d22/d29/d32/d4d/d84/dbb/dc1/dca 0 2024-09-21T04:35:28.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/624: chown dc 30987943 1 2024-09-21T04:35:28.755 INFO:tasks.workunit.client.0.smithi007.stdout:6/532: creat d3/d3e/d46/d60/d98/faf x:0 0 0 2024-09-21T04:35:28.758 INFO:tasks.workunit.client.0.smithi007.stdout:4/549: rename d2/d3c/f89 to d2/d17/d2e/d45/d5e/fb4 0 2024-09-21T04:35:28.763 INFO:tasks.workunit.client.0.smithi007.stdout:3/625: symlink dc/d6c/lcb 0 2024-09-21T04:35:28.763 INFO:tasks.workunit.client.0.smithi007.stdout:6/533: dwrite d3/d3b/f83 [0,4194304] 0 2024-09-21T04:35:28.765 INFO:tasks.workunit.client.0.smithi007.stdout:2/427: mkdir d0/d11/d6e/d73 0 2024-09-21T04:35:28.777 INFO:tasks.workunit.client.0.smithi007.stdout:6/534: rename d3/de/d12/f29 to d3/d3e/d85/d78/fb0 0 2024-09-21T04:35:28.782 INFO:tasks.workunit.client.0.smithi007.stdout:8/280: creat d0/f43 x:0 0 0 2024-09-21T04:35:28.783 INFO:tasks.workunit.client.0.smithi007.stdout:1/715: dwrite d5/d45/d5e/f60 [0,4194304] 0 2024-09-21T04:35:28.785 INFO:tasks.workunit.client.0.smithi007.stdout:2/428: rmdir d0/d32 39 2024-09-21T04:35:28.790 INFO:tasks.workunit.client.0.smithi007.stdout:4/550: dwrite d2/d7/d55/d61/f73 [0,4194304] 0 2024-09-21T04:35:28.792 INFO:tasks.workunit.client.0.smithi007.stdout:6/535: creat d3/d3e/d85/fb1 x:0 0 0 2024-09-21T04:35:28.794 INFO:tasks.workunit.client.0.smithi007.stdout:1/716: dwrite d5/f5d [4194304,4194304] 0 2024-09-21T04:35:28.801 INFO:tasks.workunit.client.0.smithi007.stdout:3/626: truncate dc/d22/d29/d32/d69/d6d/f9b 321998 0 2024-09-21T04:35:28.806 INFO:tasks.workunit.client.0.smithi007.stdout:6/536: symlink d3/d3e/d85/d78/lb2 0 2024-09-21T04:35:28.809 INFO:tasks.workunit.client.0.smithi007.stdout:8/281: fsync d0/d3/d8/f1d 0 2024-09-21T04:35:28.810 INFO:tasks.workunit.client.0.smithi007.stdout:3/627: dwrite dc/d22/d29/d32/d4d/fa0 [0,4194304] 0 2024-09-21T04:35:28.814 INFO:tasks.workunit.client.0.smithi007.stdout:3/628: truncate dc/d22/d34/da6/d80/f97 803252 0 2024-09-21T04:35:28.815 INFO:tasks.workunit.client.0.smithi007.stdout:3/629: truncate dc/f21 4683756 0 2024-09-21T04:35:28.818 INFO:tasks.workunit.client.0.smithi007.stdout:3/630: write dc/d22/d29/d32/d4d/fc2 [42334,92937] 0 2024-09-21T04:35:28.819 INFO:tasks.workunit.client.0.smithi007.stdout:1/717: mkdir d5/d6/dc5/de6 0 2024-09-21T04:35:28.822 INFO:tasks.workunit.client.0.smithi007.stdout:2/429: unlink d0/f27 0 2024-09-21T04:35:28.826 INFO:tasks.workunit.client.0.smithi007.stdout:6/537: dwrite d3/f9e [0,4194304] 0 2024-09-21T04:35:28.829 INFO:tasks.workunit.client.0.smithi007.stdout:4/551: creat d2/d17/d2e/d45/d5d/d8a/fb5 x:0 0 0 2024-09-21T04:35:28.830 INFO:tasks.workunit.client.0.smithi007.stdout:3/631: dwrite dc/d22/d29/d32/d4d/f79 [0,4194304] 0 2024-09-21T04:35:28.830 INFO:tasks.workunit.client.0.smithi007.stdout:8/282: dread d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:28.831 INFO:tasks.workunit.client.0.smithi007.stdout:1/718: unlink d5/d6/f4a 0 2024-09-21T04:35:28.832 INFO:tasks.workunit.client.0.smithi007.stdout:3/632: chown dc/d22/d29/f55 1 1 2024-09-21T04:35:28.835 INFO:tasks.workunit.client.0.smithi007.stdout:2/430: stat d0/fd 0 2024-09-21T04:35:28.839 INFO:tasks.workunit.client.0.smithi007.stdout:3/633: dwrite dc/d18/f39 [0,4194304] 0 2024-09-21T04:35:28.843 INFO:tasks.workunit.client.0.smithi007.stdout:1/719: rename d5/d45/d94/da0/d99/cb5 to d5/d6/d21/ce7 0 2024-09-21T04:35:28.846 INFO:tasks.workunit.client.0.smithi007.stdout:4/552: fsync d2/f81 0 2024-09-21T04:35:28.851 INFO:tasks.workunit.client.0.smithi007.stdout:1/720: symlink d5/d45/d94/da0/d99/le8 0 2024-09-21T04:35:28.852 INFO:tasks.workunit.client.0.smithi007.stdout:8/283: dwrite d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:28.853 INFO:tasks.workunit.client.0.smithi007.stdout:1/721: chown d5/d6/dc5/de6 7 1 2024-09-21T04:35:28.856 INFO:tasks.workunit.client.0.smithi007.stdout:6/538: rmdir d3/d3e/d85/d9d 0 2024-09-21T04:35:28.858 INFO:tasks.workunit.client.0.smithi007.stdout:3/634: dwrite dc/d22/d29/d32/d4d/f8b [0,4194304] 0 2024-09-21T04:35:28.860 INFO:tasks.workunit.client.0.smithi007.stdout:4/553: rmdir d2/d17/d2e/d9f 39 2024-09-21T04:35:28.860 INFO:tasks.workunit.client.0.smithi007.stdout:6/539: write d3/d3e/d85/d78/f96 [404309,112911] 0 2024-09-21T04:35:28.862 INFO:tasks.workunit.client.0.smithi007.stdout:3/635: chown dc/d22/d29/d32/d4d/d84 16426517 1 2024-09-21T04:35:28.864 INFO:tasks.workunit.client.0.smithi007.stdout:1/722: dwrite d5/d45/d94/da0/fad [0,4194304] 0 2024-09-21T04:35:28.865 INFO:tasks.workunit.client.0.smithi007.stdout:1/723: read - d5/d45/d94/fb1 zero size 2024-09-21T04:35:28.869 INFO:tasks.workunit.client.0.smithi007.stdout:2/431: symlink d0/d11/d6b/l74 0 2024-09-21T04:35:28.872 INFO:tasks.workunit.client.0.smithi007.stdout:1/724: creat d5/d6/d21/d64/fe9 x:0 0 0 2024-09-21T04:35:28.875 INFO:tasks.workunit.client.0.smithi007.stdout:3/636: creat dc/d22/d29/d92/db9/fcc x:0 0 0 2024-09-21T04:35:28.880 INFO:tasks.workunit.client.0.smithi007.stdout:4/554: rmdir d2/d7/d55/d61 39 2024-09-21T04:35:28.883 INFO:tasks.workunit.client.0.smithi007.stdout:2/432: rmdir d0/d32 39 2024-09-21T04:35:28.887 INFO:tasks.workunit.client.0.smithi007.stdout:8/284: truncate d0/d3/d8/de/f17 1383764 0 2024-09-21T04:35:28.887 INFO:tasks.workunit.client.0.smithi007.stdout:3/637: dwrite dc/d18/f7f [0,4194304] 0 2024-09-21T04:35:28.889 INFO:tasks.workunit.client.0.smithi007.stdout:3/638: chown dc/d18/f39 3 1 2024-09-21T04:35:28.893 INFO:tasks.workunit.client.0.smithi007.stdout:8/285: symlink d0/d3/d8/de/d26/l44 0 2024-09-21T04:35:28.894 INFO:tasks.workunit.client.0.smithi007.stdout:3/639: mknod dc/d22/d34/da6/ccd 0 2024-09-21T04:35:28.896 INFO:tasks.workunit.client.0.smithi007.stdout:3/640: write dc/d22/d29/d32/d4d/d7d/d9a/fb4 [529066,107621] 0 2024-09-21T04:35:28.897 INFO:tasks.workunit.client.0.smithi007.stdout:3/641: chown dc/d18/c63 1749220 1 2024-09-21T04:35:28.899 INFO:tasks.workunit.client.0.smithi007.stdout:1/725: write d5/d6/d21/d2b/d4b/f74 [929397,33214] 0 2024-09-21T04:35:28.924 INFO:tasks.workunit.client.0.smithi007.stdout:4/555: creat d2/fb6 x:0 0 0 2024-09-21T04:35:28.956 INFO:tasks.workunit.client.0.smithi007.stdout:8/286: rmdir d0/d3/d16/d3b/d42 0 2024-09-21T04:35:28.958 INFO:tasks.workunit.client.0.smithi007.stdout:1/726: getdents d5/d6/d21/d3c/dd5 0 2024-09-21T04:35:28.958 INFO:tasks.workunit.client.0.smithi007.stdout:2/433: getdents d0/d2/d57 0 2024-09-21T04:35:28.959 INFO:tasks.workunit.client.0.smithi007.stdout:4/556: creat d2/d17/d2e/d45/d8e/fb7 x:0 0 0 2024-09-21T04:35:28.961 INFO:tasks.workunit.client.0.smithi007.stdout:8/287: fdatasync d0/d3/d8/f3f 0 2024-09-21T04:35:28.962 INFO:tasks.workunit.client.0.smithi007.stdout:3/642: link dc/d22/d29/d32/d4d/ca1 dc/d22/d5a/d71/cce 0 2024-09-21T04:35:28.964 INFO:tasks.workunit.client.0.smithi007.stdout:2/434: unlink d0/d11/d31/d46/c72 0 2024-09-21T04:35:28.966 INFO:tasks.workunit.client.0.smithi007.stdout:4/557: chown d2/d17/d29/l7e 23 1 2024-09-21T04:35:28.969 INFO:tasks.workunit.client.0.smithi007.stdout:3/643: dwrite dc/f78 [0,4194304] 0 2024-09-21T04:35:28.970 INFO:tasks.workunit.client.0.smithi007.stdout:3/644: chown dc/d22/d29/d32/d69/l6b 69014 1 2024-09-21T04:35:28.970 INFO:tasks.workunit.client.0.smithi007.stdout:8/288: fdatasync d0/d3/d16/f21 0 2024-09-21T04:35:28.970 INFO:tasks.workunit.client.0.smithi007.stdout:3/645: chown dc/d22/d29 156431410 1 2024-09-21T04:35:28.971 INFO:tasks.workunit.client.0.smithi007.stdout:8/289: fsync d0/d3/d8/f1d 0 2024-09-21T04:35:28.975 INFO:tasks.workunit.client.0.smithi007.stdout:3/646: creat dc/d22/d5a/d71/fcf x:0 0 0 2024-09-21T04:35:28.978 INFO:tasks.workunit.client.0.smithi007.stdout:8/290: read d0/f20 [1469656,113295] 0 2024-09-21T04:35:28.979 INFO:tasks.workunit.client.0.smithi007.stdout:2/435: symlink d0/d11/d31/l75 0 2024-09-21T04:35:28.979 INFO:tasks.workunit.client.0.smithi007.stdout:8/291: chown d0 2314 1 2024-09-21T04:35:28.981 INFO:tasks.workunit.client.0.smithi007.stdout:4/558: write d2/d17/d29/f85 [360266,125513] 0 2024-09-21T04:35:28.982 INFO:tasks.workunit.client.0.smithi007.stdout:3/647: fsync dc/d22/d34/f83 0 2024-09-21T04:35:28.986 INFO:tasks.workunit.client.0.smithi007.stdout:2/436: symlink d0/l76 0 2024-09-21T04:35:28.987 INFO:tasks.workunit.client.0.smithi007.stdout:3/648: rmdir dc/d22/d29/d32/d4d/d7d/d9a 39 2024-09-21T04:35:28.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/649: rename dc/d22/d34 to dc/d22/d29/d32/d4d/d84/dd0 0 2024-09-21T04:35:28.995 INFO:tasks.workunit.client.0.smithi007.stdout:2/437: dread d0/d11/f4e [0,4194304] 0 2024-09-21T04:35:28.997 INFO:tasks.workunit.client.0.smithi007.stdout:3/650: unlink dc/d22/d5a/f9c 0 2024-09-21T04:35:28.998 INFO:tasks.workunit.client.0.smithi007.stdout:8/292: dwrite d0/f20 [0,4194304] 0 2024-09-21T04:35:29.000 INFO:tasks.workunit.client.0.smithi007.stdout:4/559: dread d2/d17/d2e/d45/f8f [0,4194304] 0 2024-09-21T04:35:29.003 INFO:tasks.workunit.client.0.smithi007.stdout:2/438: mkdir d0/d11/d31/d46/d77 0 2024-09-21T04:35:29.004 INFO:tasks.workunit.client.0.smithi007.stdout:3/651: mkdir dc/d22/d29/d32/d4d/d84/dd0/dbc/dd1 0 2024-09-21T04:35:29.004 INFO:tasks.workunit.client.0.smithi007.stdout:4/560: chown d2/d17/d29/f3a 2 1 2024-09-21T04:35:29.008 INFO:tasks.workunit.client.0.smithi007.stdout:8/293: dwrite d0/d3/d8/de/f18 [0,4194304] 0 2024-09-21T04:35:29.010 INFO:tasks.workunit.client.0.smithi007.stdout:8/294: write d0/d3/d8/de/f18 [6144472,86369] 0 2024-09-21T04:35:29.015 INFO:tasks.workunit.client.0.smithi007.stdout:3/652: creat dc/d22/dc8/fd2 x:0 0 0 2024-09-21T04:35:29.016 INFO:tasks.workunit.client.0.smithi007.stdout:3/653: write dc/d18/fbd [120779,73312] 0 2024-09-21T04:35:29.018 INFO:tasks.workunit.client.0.smithi007.stdout:4/561: truncate d2/d17/d18/f2d 48175 0 2024-09-21T04:35:29.020 INFO:tasks.workunit.client.0.smithi007.stdout:8/295: creat d0/d3/d8/de/d40/f45 x:0 0 0 2024-09-21T04:35:29.026 INFO:tasks.workunit.client.0.smithi007.stdout:3/654: dwrite dc/d22/d29/d32/d4d/d84/dd0/da6/d80/f97 [0,4194304] 0 2024-09-21T04:35:29.026 INFO:tasks.workunit.client.0.smithi007.stdout:4/562: fsync d2/d7/d64/f68 0 2024-09-21T04:35:29.028 INFO:tasks.workunit.client.0.smithi007.stdout:8/296: dread d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:29.029 INFO:tasks.workunit.client.0.smithi007.stdout:8/297: write d0/d3/d8/f1d [7567505,96127] 0 2024-09-21T04:35:29.030 INFO:tasks.workunit.client.0.smithi007.stdout:4/563: stat d2/l75 0 2024-09-21T04:35:29.032 INFO:tasks.workunit.client.0.smithi007.stdout:3/655: symlink dc/d22/d29/d92/ld3 0 2024-09-21T04:35:29.036 INFO:tasks.workunit.client.0.smithi007.stdout:8/298: write d0/d3/f2b [3225580,50244] 0 2024-09-21T04:35:29.048 INFO:tasks.workunit.client.0.smithi007.stdout:3/656: truncate dc/d22/d29/d32/d4d/f79 3207771 0 2024-09-21T04:35:29.048 INFO:tasks.workunit.client.0.smithi007.stdout:4/564: dread d2/d5f/f93 [0,4194304] 0 2024-09-21T04:35:29.049 INFO:tasks.workunit.client.0.smithi007.stdout:4/565: readlink d2/d17/l62 0 2024-09-21T04:35:29.050 INFO:tasks.workunit.client.0.smithi007.stdout:8/299: truncate d0/d3/d16/f1b 2797008 0 2024-09-21T04:35:29.053 INFO:tasks.workunit.client.0.smithi007.stdout:4/566: write d2/f12 [1153574,25043] 0 2024-09-21T04:35:29.053 INFO:tasks.workunit.client.0.smithi007.stdout:3/657: creat dc/fd4 x:0 0 0 2024-09-21T04:35:29.054 INFO:tasks.workunit.client.0.smithi007.stdout:4/567: read d2/d5f/f93 [2797865,108045] 0 2024-09-21T04:35:29.057 INFO:tasks.workunit.client.0.smithi007.stdout:8/300: mkdir d0/d3/d8/d46 0 2024-09-21T04:35:29.059 INFO:tasks.workunit.client.0.smithi007.stdout:3/658: rmdir dc/d18/d41/d96 39 2024-09-21T04:35:29.063 INFO:tasks.workunit.client.0.smithi007.stdout:3/659: unlink dc/d22/d5a/fc6 0 2024-09-21T04:35:29.070 INFO:tasks.workunit.client.0.smithi007.stdout:3/660: dwrite dc/d18/f39 [0,4194304] 0 2024-09-21T04:35:29.073 INFO:tasks.workunit.client.0.smithi007.stdout:8/301: creat d0/d3/d8/f47 x:0 0 0 2024-09-21T04:35:29.074 INFO:tasks.workunit.client.0.smithi007.stdout:8/302: write d0/d3/d8/f47 [180292,1529] 0 2024-09-21T04:35:29.074 INFO:tasks.workunit.client.0.smithi007.stdout:3/661: fdatasync dc/f1c 0 2024-09-21T04:35:29.075 INFO:tasks.workunit.client.0.smithi007.stdout:4/568: write d2/d17/d18/f54 [4578154,112918] 0 2024-09-21T04:35:29.077 INFO:tasks.workunit.client.0.smithi007.stdout:3/662: write dc/d56/faa [96636,109702] 0 2024-09-21T04:35:29.078 INFO:tasks.workunit.client.0.smithi007.stdout:8/303: symlink d0/d3/d8/de/l48 0 2024-09-21T04:35:29.078 INFO:tasks.workunit.client.0.smithi007.stdout:8/304: chown d0/d3 219677 1 2024-09-21T04:35:29.079 INFO:tasks.workunit.client.0.smithi007.stdout:3/663: truncate dc/d22/d5a/f75 229057 0 2024-09-21T04:35:29.081 INFO:tasks.workunit.client.0.smithi007.stdout:3/664: chown dc/d18/d41 61 1 2024-09-21T04:35:29.087 INFO:tasks.workunit.client.0.smithi007.stdout:4/569: dwrite d2/d7/d55/d61/f9a [0,4194304] 0 2024-09-21T04:35:29.089 INFO:tasks.workunit.client.0.smithi007.stdout:3/665: mknod dc/d6c/cd5 0 2024-09-21T04:35:29.090 INFO:tasks.workunit.client.0.smithi007.stdout:8/305: fdatasync d0/d3/d16/f2c 0 2024-09-21T04:35:29.091 INFO:tasks.workunit.client.0.smithi007.stdout:4/570: mkdir d2/d7/d64/db8 0 2024-09-21T04:35:29.096 INFO:tasks.workunit.client.0.smithi007.stdout:4/571: readlink d2/d17/l3b 0 2024-09-21T04:35:29.098 INFO:tasks.workunit.client.0.smithi007.stdout:8/306: creat d0/d3/f49 x:0 0 0 2024-09-21T04:35:29.099 INFO:tasks.workunit.client.0.smithi007.stdout:4/572: rename d2 to d2/d7/d55/d4f/db9 22 2024-09-21T04:35:29.101 INFO:tasks.workunit.client.0.smithi007.stdout:4/573: fsync d2/d7/f96 0 2024-09-21T04:35:29.103 INFO:tasks.workunit.client.0.smithi007.stdout:8/307: creat d0/d3/d8/de/d26/f4a x:0 0 0 2024-09-21T04:35:29.103 INFO:tasks.workunit.client.0.smithi007.stdout:8/308: dread - d0/f43 zero size 2024-09-21T04:35:29.106 INFO:tasks.workunit.client.0.smithi007.stdout:3/666: dread dc/d22/d5a/d71/f85 [0,4194304] 0 2024-09-21T04:35:29.107 INFO:tasks.workunit.client.0.smithi007.stdout:8/309: dread d0/d3/d16/f2c [0,4194304] 0 2024-09-21T04:35:29.107 INFO:tasks.workunit.client.0.smithi007.stdout:4/574: mknod d2/d17/d2e/d45/cba 0 2024-09-21T04:35:29.109 INFO:tasks.workunit.client.0.smithi007.stdout:3/667: rmdir dc/d22/d29/d32/d4d/d84/dd0 39 2024-09-21T04:35:29.111 INFO:tasks.workunit.client.0.smithi007.stdout:8/310: rename d0/l28 to d0/d3/d8/de/d26/d35/l4b 0 2024-09-21T04:35:29.112 INFO:tasks.workunit.client.0.smithi007.stdout:3/668: mknod dc/d22/d5a/d71/d89/cd6 0 2024-09-21T04:35:29.113 INFO:tasks.workunit.client.0.smithi007.stdout:4/575: mknod d2/d17/cbb 0 2024-09-21T04:35:29.114 INFO:tasks.workunit.client.0.smithi007.stdout:3/669: unlink dc/d56/lc9 0 2024-09-21T04:35:29.124 INFO:tasks.workunit.client.0.smithi007.stdout:3/670: getdents dc/d22/d29/d32/d4f 0 2024-09-21T04:35:29.126 INFO:tasks.workunit.client.0.smithi007.stdout:3/671: write dc/d22/d29/d32/d69/d6d/f48 [3804548,3689] 0 2024-09-21T04:35:29.131 INFO:tasks.workunit.client.0.smithi007.stdout:4/576: dwrite d2/f25 [0,4194304] 0 2024-09-21T04:35:29.138 INFO:tasks.workunit.client.0.smithi007.stdout:4/577: creat d2/d7/d55/d4f/fbc x:0 0 0 2024-09-21T04:35:29.142 INFO:tasks.workunit.client.0.smithi007.stdout:3/672: getdents dc/d22 0 2024-09-21T04:35:29.143 INFO:tasks.workunit.client.0.smithi007.stdout:4/578: fdatasync d2/d7/f24 0 2024-09-21T04:35:29.144 INFO:tasks.workunit.client.0.smithi007.stdout:3/673: stat dc/d22/d29/d32/d69/f40 0 2024-09-21T04:35:29.146 INFO:tasks.workunit.client.0.smithi007.stdout:4/579: read d2/d17/d2e/f71 [3172115,16853] 0 2024-09-21T04:35:29.146 INFO:tasks.workunit.client.0.smithi007.stdout:3/674: write dc/d22/d5a/d71/fcf [90215,99998] 0 2024-09-21T04:35:29.147 INFO:tasks.workunit.client.0.smithi007.stdout:4/580: stat d2/d17/d2e/d45/d5d/d8a/ca7 0 2024-09-21T04:35:29.148 INFO:tasks.workunit.client.0.smithi007.stdout:3/675: mknod dc/d22/d29/cd7 0 2024-09-21T04:35:29.148 INFO:tasks.workunit.client.0.smithi007.stdout:3/676: fsync dc/f10 0 2024-09-21T04:35:29.149 INFO:tasks.workunit.client.0.smithi007.stdout:9/505: sync 2024-09-21T04:35:29.151 INFO:tasks.workunit.client.0.smithi007.stdout:3/677: creat dc/d22/d5a/fd8 x:0 0 0 2024-09-21T04:35:29.153 INFO:tasks.workunit.client.0.smithi007.stdout:9/506: creat d13/d74/d99/f9e x:0 0 0 2024-09-21T04:35:29.154 INFO:tasks.workunit.client.0.smithi007.stdout:4/581: getdents d2/d17 0 2024-09-21T04:35:29.156 INFO:tasks.workunit.client.0.smithi007.stdout:3/678: creat dc/d18/d41/d96/fd9 x:0 0 0 2024-09-21T04:35:29.157 INFO:tasks.workunit.client.0.smithi007.stdout:3/679: chown dc/d22/d29/d32/d4d/d84/dbb/dc1 4387 1 2024-09-21T04:35:29.158 INFO:tasks.workunit.client.0.smithi007.stdout:4/582: mknod d2/d17/d18/cbd 0 2024-09-21T04:35:29.159 INFO:tasks.workunit.client.0.smithi007.stdout:9/507: mkdir d13/d18/d60/d6d/d75/d9f 0 2024-09-21T04:35:29.161 INFO:tasks.workunit.client.0.smithi007.stdout:9/508: truncate d13/d18/f35 5014189 0 2024-09-21T04:35:29.163 INFO:tasks.workunit.client.0.smithi007.stdout:3/680: dwrite dc/f23 [0,4194304] 0 2024-09-21T04:35:29.165 INFO:tasks.workunit.client.0.smithi007.stdout:3/681: truncate dc/d18/fab 40162 0 2024-09-21T04:35:29.299 INFO:tasks.workunit.client.0.smithi007.stdout:3/682: truncate dc/d22/d29/d32/d69/f49 1651629 0 2024-09-21T04:35:29.300 INFO:tasks.workunit.client.0.smithi007.stdout:4/583: dread - d2/d17/d2e/d9f/fab zero size 2024-09-21T04:35:29.301 INFO:tasks.workunit.client.0.smithi007.stdout:4/584: write d2/d5f/fac [376156,125752] 0 2024-09-21T04:35:29.317 INFO:tasks.workunit.client.0.smithi007.stdout:9/509: creat d13/d18/d60/d6d/d75/fa0 x:0 0 0 2024-09-21T04:35:29.320 INFO:tasks.workunit.client.0.smithi007.stdout:7/524: sync 2024-09-21T04:35:29.322 INFO:tasks.workunit.client.0.smithi007.stdout:7/525: read d3/d6/d14/d20/d80/f9c [1225385,32779] 0 2024-09-21T04:35:29.323 INFO:tasks.workunit.client.0.smithi007.stdout:9/510: dread d13/d74/d99/f47 [0,4194304] 0 2024-09-21T04:35:29.324 INFO:tasks.workunit.client.0.smithi007.stdout:9/511: dread - d13/d74/d99/f9e zero size 2024-09-21T04:35:29.325 INFO:tasks.workunit.client.0.smithi007.stdout:7/526: read d3/d6/d10/d62/d66/fa3 [781131,51490] 0 2024-09-21T04:35:29.325 INFO:tasks.workunit.client.0.smithi007.stdout:7/527: chown d3/d6/d10/d62/d66/f94 54742745 1 2024-09-21T04:35:29.326 INFO:tasks.workunit.client.0.smithi007.stdout:7/528: dread - d3/d6/d14/d20/d80/f9d zero size 2024-09-21T04:35:29.327 INFO:tasks.workunit.client.0.smithi007.stdout:7/529: chown d3/d6/d10/la2 1260 1 2024-09-21T04:35:29.331 INFO:tasks.workunit.client.0.smithi007.stdout:9/512: mkdir d13/d18/d60/d6d/d85/da1 0 2024-09-21T04:35:29.334 INFO:tasks.workunit.client.0.smithi007.stdout:7/530: mknod d3/d6/d40/cb9 0 2024-09-21T04:35:29.334 INFO:tasks.workunit.client.0.smithi007.stdout:9/513: stat d13/d18/d60/c64 0 2024-09-21T04:35:29.335 INFO:tasks.workunit.client.0.smithi007.stdout:7/531: write d3/d6/d14/d20/d80/f64 [4130331,59182] 0 2024-09-21T04:35:29.336 INFO:tasks.workunit.client.0.smithi007.stdout:7/532: readlink d3/d6/d13/l4a 0 2024-09-21T04:35:29.336 INFO:tasks.workunit.client.0.smithi007.stdout:9/514: read d13/d18/d60/d6d/d75/f7e [2953407,121246] 0 2024-09-21T04:35:29.338 INFO:tasks.workunit.client.0.smithi007.stdout:7/533: read d3/f5 [556432,97098] 0 2024-09-21T04:35:29.338 INFO:tasks.workunit.client.0.smithi007.stdout:9/515: write d13/d74/f98 [85759,83906] 0 2024-09-21T04:35:29.340 INFO:tasks.workunit.client.0.smithi007.stdout:9/516: dread d13/d18/f2f [0,4194304] 0 2024-09-21T04:35:29.342 INFO:tasks.workunit.client.0.smithi007.stdout:9/517: write d13/d18/f5c [5319340,25613] 0 2024-09-21T04:35:29.346 INFO:tasks.workunit.client.0.smithi007.stdout:9/518: symlink d13/d5a/la2 0 2024-09-21T04:35:29.347 INFO:tasks.workunit.client.0.smithi007.stdout:9/519: chown d13/d18 79685 1 2024-09-21T04:35:29.348 INFO:tasks.workunit.client.0.smithi007.stdout:7/534: creat d3/d6/d13/fba x:0 0 0 2024-09-21T04:35:29.349 INFO:tasks.workunit.client.0.smithi007.stdout:9/520: write d13/d18/d60/d6d/f6f [102692,100478] 0 2024-09-21T04:35:29.350 INFO:tasks.workunit.client.0.smithi007.stdout:4/585: dwrite d2/d7/f59 [0,4194304] 0 2024-09-21T04:35:29.357 INFO:tasks.workunit.client.0.smithi007.stdout:7/535: mknod d3/d6/d13/cbb 0 2024-09-21T04:35:29.358 INFO:tasks.workunit.client.0.smithi007.stdout:9/521: mknod d13/d18/ca3 0 2024-09-21T04:35:29.359 INFO:tasks.workunit.client.0.smithi007.stdout:7/536: creat d3/d6/d10/d4f/fbc x:0 0 0 2024-09-21T04:35:29.361 INFO:tasks.workunit.client.0.smithi007.stdout:4/586: mkdir d2/d3c/dbe 0 2024-09-21T04:35:29.361 INFO:tasks.workunit.client.0.smithi007.stdout:9/522: rmdir d13/d5a 39 2024-09-21T04:35:29.364 INFO:tasks.workunit.client.0.smithi007.stdout:7/537: rename d3/d6/d14/la9 to d3/d6/d10/d51/lbd 0 2024-09-21T04:35:29.368 INFO:tasks.workunit.client.0.smithi007.stdout:7/538: creat d3/d6/d10/d4f/fbe x:0 0 0 2024-09-21T04:35:29.370 INFO:tasks.workunit.client.0.smithi007.stdout:4/587: rename d2/f5a to d2/d17/d29/fbf 0 2024-09-21T04:35:29.374 INFO:tasks.workunit.client.0.smithi007.stdout:4/588: mkdir d2/d17/dc0 0 2024-09-21T04:35:29.379 INFO:tasks.workunit.client.0.smithi007.stdout:7/539: dread d3/d6/d2e/d63/d6a/d81/f8c [0,4194304] 0 2024-09-21T04:35:29.386 INFO:tasks.workunit.client.0.smithi007.stdout:9/523: dwrite d13/d18/f45 [0,4194304] 0 2024-09-21T04:35:29.390 INFO:tasks.workunit.client.0.smithi007.stdout:7/540: rmdir d3/d6/d85/d5f 0 2024-09-21T04:35:29.392 INFO:tasks.workunit.client.0.smithi007.stdout:9/524: fsync fe 0 2024-09-21T04:35:29.399 INFO:tasks.workunit.client.0.smithi007.stdout:7/541: creat d3/d6/d2e/fbf x:0 0 0 2024-09-21T04:35:29.402 INFO:tasks.workunit.client.0.smithi007.stdout:7/542: truncate d3/d6/d13/f29 2019505 0 2024-09-21T04:35:29.403 INFO:tasks.workunit.client.0.smithi007.stdout:7/543: write d3/d6/d85/dad/fb5 [903937,51869] 0 2024-09-21T04:35:29.406 INFO:tasks.workunit.client.0.smithi007.stdout:4/589: dread d2/d17/d2e/d45/d5e/fb4 [0,4194304] 0 2024-09-21T04:35:29.410 INFO:tasks.workunit.client.0.smithi007.stdout:4/590: unlink d2/d7/f6c 0 2024-09-21T04:35:29.412 INFO:tasks.workunit.client.0.smithi007.stdout:4/591: mknod d2/d17/cc1 0 2024-09-21T04:35:29.414 INFO:tasks.workunit.client.0.smithi007.stdout:4/592: fdatasync d2/d17/d18/f84 0 2024-09-21T04:35:29.419 INFO:tasks.workunit.client.0.smithi007.stdout:4/593: mknod d2/d17/d2e/d45/d5d/d8a/cc2 0 2024-09-21T04:35:29.419 INFO:tasks.workunit.client.0.smithi007.stdout:4/594: stat d2/d17/d2e/d45 0 2024-09-21T04:35:29.423 INFO:tasks.workunit.client.0.smithi007.stdout:4/595: creat d2/d3c/dbe/fc3 x:0 0 0 2024-09-21T04:35:29.485 INFO:tasks.workunit.client.0.smithi007.stdout:5/692: sync 2024-09-21T04:35:29.487 INFO:tasks.workunit.client.0.smithi007.stdout:5/693: creat d2/d27/d42/da3/d23/d49/dd4/fde x:0 0 0 2024-09-21T04:35:29.487 INFO:tasks.workunit.client.0.smithi007.stdout:5/694: chown d2/d1e/ld3 2998874 1 2024-09-21T04:35:29.489 INFO:tasks.workunit.client.0.smithi007.stdout:5/695: read d2/d27/d42/da3/d23/d5d/d9a/da6/fba [37103,34699] 0 2024-09-21T04:35:29.495 INFO:tasks.workunit.client.0.smithi007.stdout:5/696: link d2/d27/d3d/d56/cd0 d2/d27/d42/da3/d23/da1/cdf 0 2024-09-21T04:35:29.502 INFO:tasks.workunit.client.0.smithi007.stdout:5/697: write d2/d27/d42/da3/d23/d5d/d61/f74 [1569897,7465] 0 2024-09-21T04:35:29.507 INFO:tasks.workunit.client.0.smithi007.stdout:5/698: getdents d2/d27/d42/da3/d23/d2a 0 2024-09-21T04:35:29.508 INFO:tasks.workunit.client.0.smithi007.stdout:5/699: dread - d2/d27/d42/da3/d23/d2a/f9f zero size 2024-09-21T04:35:29.509 INFO:tasks.workunit.client.0.smithi007.stdout:5/700: chown d2/d27/d3d/d56/cd0 838 1 2024-09-21T04:35:29.510 INFO:tasks.workunit.client.0.smithi007.stdout:5/701: write d2/d27/d42/da3/fd8 [912864,60845] 0 2024-09-21T04:35:29.512 INFO:tasks.workunit.client.0.smithi007.stdout:5/702: mknod d2/d27/d3d/ce0 0 2024-09-21T04:35:29.516 INFO:tasks.workunit.client.0.smithi007.stdout:5/703: creat d2/d27/d42/da3/fe1 x:0 0 0 2024-09-21T04:35:29.518 INFO:tasks.workunit.client.0.smithi007.stdout:5/704: mkdir d2/d27/d42/da3/d28/de2 0 2024-09-21T04:35:29.524 INFO:tasks.workunit.client.0.smithi007.stdout:5/705: dwrite d2/d27/d3d/d9e/fd9 [0,4194304] 0 2024-09-21T04:35:29.718 INFO:tasks.workunit.client.0.smithi007.stdout:6/540: sync 2024-09-21T04:35:29.718 INFO:tasks.workunit.client.0.smithi007.stdout:0/607: sync 2024-09-21T04:35:29.720 INFO:tasks.workunit.client.0.smithi007.stdout:0/608: read d6/d75/dc5/d18/d27/fb7 [911674,113006] 0 2024-09-21T04:35:29.721 INFO:tasks.workunit.client.0.smithi007.stdout:6/541: chown d3/de/d12/c25 406504 1 2024-09-21T04:35:29.722 INFO:tasks.workunit.client.0.smithi007.stdout:0/609: write d6/d75/dc5/d57/d9d/fdc [679941,36158] 0 2024-09-21T04:35:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:0/610: write d6/d75/dc5/d18/d2e/d3f/f90 [1497968,92009] 0 2024-09-21T04:35:29.728 INFO:tasks.workunit.client.0.smithi007.stdout:6/542: dwrite d3/f38 [0,4194304] 0 2024-09-21T04:35:29.730 INFO:tasks.workunit.client.0.smithi007.stdout:0/611: creat d6/d75/dc5/d57/fe3 x:0 0 0 2024-09-21T04:35:29.731 INFO:tasks.workunit.client.0.smithi007.stdout:0/612: readlink d6/d75/dc5/d18/d27/l36 0 2024-09-21T04:35:29.734 INFO:tasks.workunit.client.0.smithi007.stdout:6/543: creat d3/d94/fb3 x:0 0 0 2024-09-21T04:35:29.735 INFO:tasks.workunit.client.0.smithi007.stdout:0/613: mknod d6/d75/dc5/d18/d2e/d60/ce4 0 2024-09-21T04:35:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:6/544: mknod d3/de/d4e/d67/cb4 0 2024-09-21T04:35:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:0/614: write d6/d75/dc5/d18/d39/fa7 [1276458,93824] 0 2024-09-21T04:35:29.740 INFO:tasks.workunit.client.0.smithi007.stdout:6/545: write d3/d3b/f5b [1490350,118010] 0 2024-09-21T04:35:29.743 INFO:tasks.workunit.client.0.smithi007.stdout:0/615: rename d6/d91/l93 to d6/d75/dc5/d18/d2e/d60/le5 0 2024-09-21T04:35:29.747 INFO:tasks.workunit.client.0.smithi007.stdout:0/616: creat d6/d75/dc5/d18/d2e/d3f/fe6 x:0 0 0 2024-09-21T04:35:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:0/617: creat d6/d75/dc5/d52/d6d/de1/fe7 x:0 0 0 2024-09-21T04:35:29.755 INFO:tasks.workunit.client.0.smithi007.stdout:0/618: fsync d6/f13 0 2024-09-21T04:35:29.756 INFO:tasks.workunit.client.0.smithi007.stdout:6/546: dwrite d3/d3b/f79 [0,4194304] 0 2024-09-21T04:35:29.763 INFO:tasks.workunit.client.0.smithi007.stdout:0/619: symlink d6/d75/dc5/d18/d2e/d60/le8 0 2024-09-21T04:35:29.766 INFO:tasks.workunit.client.0.smithi007.stdout:0/620: mkdir d6/d75/dc5/d47/d8a/de9 0 2024-09-21T04:35:29.768 INFO:tasks.workunit.client.0.smithi007.stdout:0/621: rmdir d6/d75/dc5/d52 39 2024-09-21T04:35:29.771 INFO:tasks.workunit.client.0.smithi007.stdout:0/622: chown d6/d75/dc5/d18/d39/l80 0 1 2024-09-21T04:35:29.773 INFO:tasks.workunit.client.0.smithi007.stdout:6/547: dread d3/f2f [0,4194304] 0 2024-09-21T04:35:29.774 INFO:tasks.workunit.client.0.smithi007.stdout:0/623: mknod d6/d75/dc5/d52/d6d/de1/cea 0 2024-09-21T04:35:29.775 INFO:tasks.workunit.client.0.smithi007.stdout:0/624: chown d6/d75/dc5/d18 0 1 2024-09-21T04:35:29.777 INFO:tasks.workunit.client.0.smithi007.stdout:6/548: creat d3/de/fb5 x:0 0 0 2024-09-21T04:35:29.782 INFO:tasks.workunit.client.0.smithi007.stdout:6/549: symlink d3/d3e/d46/lb6 0 2024-09-21T04:35:29.787 INFO:tasks.workunit.client.0.smithi007.stdout:6/550: mknod d3/d3e/cb7 0 2024-09-21T04:35:29.789 INFO:tasks.workunit.client.0.smithi007.stdout:1/727: sync 2024-09-21T04:35:29.789 INFO:tasks.workunit.client.0.smithi007.stdout:6/551: rmdir d3 39 2024-09-21T04:35:29.792 INFO:tasks.workunit.client.0.smithi007.stdout:0/625: dwrite d6/d75/dc5/d18/d2e/d3f/f43 [0,4194304] 0 2024-09-21T04:35:29.801 INFO:tasks.workunit.client.0.smithi007.stdout:0/626: dwrite d6/d75/dc5/d52/d6d/fa6 [0,4194304] 0 2024-09-21T04:35:29.802 INFO:tasks.workunit.client.0.smithi007.stdout:6/552: link d3/d3e/d99/c9c d3/d3e/d85/d78/cb8 0 2024-09-21T04:35:29.806 INFO:tasks.workunit.client.0.smithi007.stdout:1/728: truncate d5/f26 4340466 0 2024-09-21T04:35:29.815 INFO:tasks.workunit.client.0.smithi007.stdout:1/729: mkdir d5/d6/d21/d3c/dd5/dea 0 2024-09-21T04:35:29.816 INFO:tasks.workunit.client.0.smithi007.stdout:0/627: creat d6/d75/dc5/d18/d2e/feb x:0 0 0 2024-09-21T04:35:29.817 INFO:tasks.workunit.client.0.smithi007.stdout:1/730: chown d5/d6/d21/d2b/d86/dd8/cb3 6 1 2024-09-21T04:35:29.819 INFO:tasks.workunit.client.0.smithi007.stdout:6/553: dwrite d3/f54 [0,4194304] 0 2024-09-21T04:35:29.820 INFO:tasks.workunit.client.0.smithi007.stdout:0/628: write d6/d75/dc5/d52/f78 [703388,117459] 0 2024-09-21T04:35:29.822 INFO:tasks.workunit.client.0.smithi007.stdout:0/629: chown d6/d75/dc5/d57/c73 2 1 2024-09-21T04:35:29.825 INFO:tasks.workunit.client.0.smithi007.stdout:9/525: sync 2024-09-21T04:35:29.825 INFO:tasks.workunit.client.0.smithi007.stdout:5/706: sync 2024-09-21T04:35:29.825 INFO:tasks.workunit.client.0.smithi007.stdout:2/439: sync 2024-09-21T04:35:29.825 INFO:tasks.workunit.client.0.smithi007.stdout:8/311: sync 2024-09-21T04:35:29.826 INFO:tasks.workunit.client.0.smithi007.stdout:7/544: sync 2024-09-21T04:35:29.826 INFO:tasks.workunit.client.0.smithi007.stdout:3/683: sync 2024-09-21T04:35:29.826 INFO:tasks.workunit.client.0.smithi007.stdout:4/596: sync 2024-09-21T04:35:29.828 INFO:tasks.workunit.client.0.smithi007.stdout:8/312: chown d0/d3/c11 31 1 2024-09-21T04:35:29.828 INFO:tasks.workunit.client.0.smithi007.stdout:2/440: chown d0/d11/f20 64566592 1 2024-09-21T04:35:29.831 INFO:tasks.workunit.client.0.smithi007.stdout:1/731: truncate d5/d6/d21/d2b/d4b/f75 346361 0 2024-09-21T04:35:29.833 INFO:tasks.workunit.client.0.smithi007.stdout:1/732: fdatasync d5/d6/d21/d2b/f2d 0 2024-09-21T04:35:29.836 INFO:tasks.workunit.client.0.smithi007.stdout:6/554: dread d3/fc [0,4194304] 0 2024-09-21T04:35:29.837 INFO:tasks.workunit.client.0.smithi007.stdout:7/545: symlink d3/d6/d14/d20/d58/da8/lc0 0 2024-09-21T04:35:29.838 INFO:tasks.workunit.client.0.smithi007.stdout:8/313: dwrite d0/d3/d8/de/d26/f4a [0,4194304] 0 2024-09-21T04:35:29.839 INFO:tasks.workunit.client.0.smithi007.stdout:6/555: write d3/d3e/d85/f92 [324034,49227] 0 2024-09-21T04:35:29.842 INFO:tasks.workunit.client.0.smithi007.stdout:6/556: read d3/d3e/d99/fa5 [18011,14969] 0 2024-09-21T04:35:29.844 INFO:tasks.workunit.client.0.smithi007.stdout:3/684: rename dc/d22/d29/d32/d69/l6b to dc/d22/d5a/lda 0 2024-09-21T04:35:29.845 INFO:tasks.workunit.client.0.smithi007.stdout:0/630: mkdir d6/d91/da3/dbf/dec 0 2024-09-21T04:35:29.847 INFO:tasks.workunit.client.0.smithi007.stdout:0/631: write d6/d75/dc5/d52/d6d/fa6 [3627341,92318] 0 2024-09-21T04:35:29.853 INFO:tasks.workunit.client.0.smithi007.stdout:9/526: mkdir d13/d18/d60/d6d/d75/d9f/da4 0 2024-09-21T04:35:29.856 INFO:tasks.workunit.client.0.smithi007.stdout:2/441: rmdir d0/d32 39 2024-09-21T04:35:29.863 INFO:tasks.workunit.client.0.smithi007.stdout:8/314: mknod d0/c4c 0 2024-09-21T04:35:29.863 INFO:tasks.workunit.client.0.smithi007.stdout:7/546: dread d3/d6/d2e/f38 [0,4194304] 0 2024-09-21T04:35:29.863 INFO:tasks.workunit.client.0.smithi007.stdout:8/315: fdatasync d0/d3/d16/f21 0 2024-09-21T04:35:29.870 INFO:tasks.workunit.client.0.smithi007.stdout:6/557: mknod d3/de/d12/d19/da3/cb9 0 2024-09-21T04:35:29.875 INFO:tasks.workunit.client.0.smithi007.stdout:9/527: dwrite d13/d18/d60/f7d [0,4194304] 0 2024-09-21T04:35:29.875 INFO:tasks.workunit.client.0.smithi007.stdout:7/547: mknod d3/d6/d14/d20/d80/db2/cc1 0 2024-09-21T04:35:29.877 INFO:tasks.workunit.client.0.smithi007.stdout:2/442: creat d0/d11/d31/d46/d6c/f78 x:0 0 0 2024-09-21T04:35:29.878 INFO:tasks.workunit.client.0.smithi007.stdout:1/733: dwrite d5/d6/f31 [0,4194304] 0 2024-09-21T04:35:29.881 INFO:tasks.workunit.client.0.smithi007.stdout:3/685: mkdir dc/d22/d29/d32/d4d/d84/dd0/ddb 0 2024-09-21T04:35:29.884 INFO:tasks.workunit.client.0.smithi007.stdout:0/632: rename d6/d75/dc5/d18/d39/fc0 to d6/d75/dc5/d18/d2e/fed 0 2024-09-21T04:35:29.884 INFO:tasks.workunit.client.0.smithi007.stdout:4/597: dwrite d2/d7/f48 [0,4194304] 0 2024-09-21T04:35:29.891 INFO:tasks.workunit.client.0.smithi007.stdout:7/548: mkdir d3/d6/d10/d62/d66/dc2 0 2024-09-21T04:35:29.896 INFO:tasks.workunit.client.0.smithi007.stdout:2/443: creat d0/d11/d31/d46/f79 x:0 0 0 2024-09-21T04:35:29.897 INFO:tasks.workunit.client.0.smithi007.stdout:9/528: dwrite d13/d74/d99/f9e [0,4194304] 0 2024-09-21T04:35:29.897 INFO:tasks.workunit.client.0.smithi007.stdout:1/734: unlink d5/d6/d21/fc7 0 2024-09-21T04:35:29.899 INFO:tasks.workunit.client.0.smithi007.stdout:2/444: write d0/fd [2272052,41006] 0 2024-09-21T04:35:29.900 INFO:tasks.workunit.client.0.smithi007.stdout:6/558: dwrite d3/d3e/d85/d78/fb0 [0,4194304] 0 2024-09-21T04:35:29.900 INFO:tasks.workunit.client.0.smithi007.stdout:4/598: symlink d2/d17/d2e/d45/d5d/d8a/lc4 0 2024-09-21T04:35:29.902 INFO:tasks.workunit.client.0.smithi007.stdout:8/316: truncate d0/d3/d8/f1d 5126254 0 2024-09-21T04:35:29.907 INFO:tasks.workunit.client.0.smithi007.stdout:7/549: creat d3/d6/d40/da1/fc3 x:0 0 0 2024-09-21T04:35:29.907 INFO:tasks.workunit.client.0.smithi007.stdout:0/633: symlink d6/d75/dc9/lee 0 2024-09-21T04:35:29.908 INFO:tasks.workunit.client.0.smithi007.stdout:6/559: chown d3/de/d12/f50 239 1 2024-09-21T04:35:29.911 INFO:tasks.workunit.client.0.smithi007.stdout:7/550: read d3/d6/d10/d4f/f5c [173985,46881] 0 2024-09-21T04:35:29.911 INFO:tasks.workunit.client.0.smithi007.stdout:6/560: dread d3/de/d12/d19/f51 [0,4194304] 0 2024-09-21T04:35:29.912 INFO:tasks.workunit.client.0.smithi007.stdout:0/634: dread d6/d75/dc5/d47/d8a/fc4 [0,4194304] 0 2024-09-21T04:35:29.912 INFO:tasks.workunit.client.0.smithi007.stdout:3/686: symlink dc/d22/d29/d32/d4d/d84/dd0/ldc 0 2024-09-21T04:35:29.916 INFO:tasks.workunit.client.0.smithi007.stdout:8/317: chown d0/d3/d8/de/d26/d35/l4b 3950 1 2024-09-21T04:35:29.918 INFO:tasks.workunit.client.0.smithi007.stdout:8/318: write d0/d3/d8/de/d26/d35/f37 [1721461,25499] 0 2024-09-21T04:35:29.919 INFO:tasks.workunit.client.0.smithi007.stdout:9/529: unlink d13/d74/d99/f56 0 2024-09-21T04:35:29.922 INFO:tasks.workunit.client.0.smithi007.stdout:4/599: mkdir d2/d7/dc5 0 2024-09-21T04:35:29.925 INFO:tasks.workunit.client.0.smithi007.stdout:7/551: dread d3/f88 [0,4194304] 0 2024-09-21T04:35:29.927 INFO:tasks.workunit.client.0.smithi007.stdout:6/561: chown d3/de/d12/f1f 23920531 1 2024-09-21T04:35:29.928 INFO:tasks.workunit.client.0.smithi007.stdout:6/562: chown d3/d3e/d85/d78/f96 58909 1 2024-09-21T04:35:29.929 INFO:tasks.workunit.client.0.smithi007.stdout:2/445: unlink d0/d11/d31/d52/l5c 0 2024-09-21T04:35:29.930 INFO:tasks.workunit.client.0.smithi007.stdout:0/635: fdatasync d6/d75/dc5/d18/d2e/f8b 0 2024-09-21T04:35:29.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/319: mkdir d0/d3/d8/de/d26/d4d 0 2024-09-21T04:35:29.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/320: chown d0/d3/f49 7671 1 2024-09-21T04:35:29.942 INFO:tasks.workunit.client.0.smithi007.stdout:3/687: mknod dc/d18/d41/cdd 0 2024-09-21T04:35:29.943 INFO:tasks.workunit.client.0.smithi007.stdout:3/688: read - dc/d22/d29/d92/db9/fcc zero size 2024-09-21T04:35:29.945 INFO:tasks.workunit.client.0.smithi007.stdout:0/636: chown d6/d75/dc5/c1b 12087 1 2024-09-21T04:35:29.946 INFO:tasks.workunit.client.0.smithi007.stdout:9/530: creat d13/d18/d60/d6d/d85/da1/fa5 x:0 0 0 2024-09-21T04:35:29.947 INFO:tasks.workunit.client.0.smithi007.stdout:1/735: link d5/d6/d21/d2b/d4b/da6/ldb d5/d6/d21/dd4/leb 0 2024-09-21T04:35:29.950 INFO:tasks.workunit.client.0.smithi007.stdout:6/563: mkdir d3/de/dba 0 2024-09-21T04:35:29.953 INFO:tasks.workunit.client.0.smithi007.stdout:6/564: truncate d3/d3e/d46/f68 190468 0 2024-09-21T04:35:29.953 INFO:tasks.workunit.client.0.smithi007.stdout:2/446: dwrite d0/f29 [0,4194304] 0 2024-09-21T04:35:29.956 INFO:tasks.workunit.client.0.smithi007.stdout:6/565: write d3/d3e/d85/d78/fb0 [3139316,81475] 0 2024-09-21T04:35:29.956 INFO:tasks.workunit.client.0.smithi007.stdout:8/321: mkdir d0/d4e 0 2024-09-21T04:35:29.959 INFO:tasks.workunit.client.0.smithi007.stdout:6/566: chown d3/f33 0 1 2024-09-21T04:35:29.961 INFO:tasks.workunit.client.0.smithi007.stdout:7/552: creat d3/d6/d13/fc4 x:0 0 0 2024-09-21T04:35:29.966 INFO:tasks.workunit.client.0.smithi007.stdout:3/689: dread dc/d18/f31 [0,4194304] 0 2024-09-21T04:35:29.967 INFO:tasks.workunit.client.0.smithi007.stdout:2/447: dwrite d0/d11/f4b [0,4194304] 0 2024-09-21T04:35:29.969 INFO:tasks.workunit.client.0.smithi007.stdout:1/736: symlink d5/d6/d21/d2b/d4b/da6/d58/dcf/lec 0 2024-09-21T04:35:29.969 INFO:tasks.workunit.client.0.smithi007.stdout:2/448: dread - d0/d11/d31/d46/f79 zero size 2024-09-21T04:35:29.970 INFO:tasks.workunit.client.0.smithi007.stdout:6/567: rename d3/de/d4e to d3/d3e/d99/dbb 0 2024-09-21T04:35:29.970 INFO:tasks.workunit.client.0.smithi007.stdout:4/600: dwrite d2/d17/d29/f69 [0,4194304] 0 2024-09-21T04:35:29.971 INFO:tasks.workunit.client.0.smithi007.stdout:7/553: truncate d3/d6/d40/da1/fc3 197737 0 2024-09-21T04:35:29.973 INFO:tasks.workunit.client.0.smithi007.stdout:4/601: chown d2/fb6 5117628 1 2024-09-21T04:35:29.974 INFO:tasks.workunit.client.0.smithi007.stdout:7/554: chown d3/d6/d10/d62/d66 3674 1 2024-09-21T04:35:29.974 INFO:tasks.workunit.client.0.smithi007.stdout:3/690: mknod dc/d22/d5a/cde 0 2024-09-21T04:35:29.974 INFO:tasks.workunit.client.0.smithi007.stdout:2/449: dread - d0/d11/d31/d52/f65 zero size 2024-09-21T04:35:29.977 INFO:tasks.workunit.client.0.smithi007.stdout:9/531: write d13/d18/f55 [317683,117050] 0 2024-09-21T04:35:29.978 INFO:tasks.workunit.client.0.smithi007.stdout:0/637: getdents d6 0 2024-09-21T04:35:29.980 INFO:tasks.workunit.client.0.smithi007.stdout:9/532: chown d13/d18/f3f 417638412 1 2024-09-21T04:35:29.980 INFO:tasks.workunit.client.0.smithi007.stdout:6/568: unlink d3/f9e 0 2024-09-21T04:35:29.981 INFO:tasks.workunit.client.0.smithi007.stdout:4/602: write d2/d17/d2e/d45/d8e/fb7 [144670,53349] 0 2024-09-21T04:35:29.982 INFO:tasks.workunit.client.0.smithi007.stdout:8/322: creat d0/f4f x:0 0 0 2024-09-21T04:35:29.985 INFO:tasks.workunit.client.0.smithi007.stdout:0/638: read d6/d75/dc5/d52/f5c [1616279,48014] 0 2024-09-21T04:35:29.987 INFO:tasks.workunit.client.0.smithi007.stdout:4/603: dread d2/d5f/fac [0,4194304] 0 2024-09-21T04:35:29.988 INFO:tasks.workunit.client.0.smithi007.stdout:7/555: chown d3/d6/d10/f34 26881433 1 2024-09-21T04:35:29.995 INFO:tasks.workunit.client.0.smithi007.stdout:3/691: write dc/d18/f67 [1360194,15745] 0 2024-09-21T04:35:29.998 INFO:tasks.workunit.client.0.smithi007.stdout:1/737: dwrite d5/f26 [4194304,4194304] 0 2024-09-21T04:35:30.004 INFO:tasks.workunit.client.0.smithi007.stdout:9/533: dwrite d13/d18/d60/f88 [0,4194304] 0 2024-09-21T04:35:30.005 INFO:tasks.workunit.client.0.smithi007.stdout:1/738: dread d5/d6/f31 [0,4194304] 0 2024-09-21T04:35:30.007 INFO:tasks.workunit.client.0.smithi007.stdout:2/450: write d0/d2/f25 [1681081,124612] 0 2024-09-21T04:35:30.010 INFO:tasks.workunit.client.0.smithi007.stdout:7/556: mkdir d3/d6/dc5 0 2024-09-21T04:35:30.014 INFO:tasks.workunit.client.0.smithi007.stdout:3/692: write dc/d22/d29/d32/fa3 [3498216,130760] 0 2024-09-21T04:35:30.019 INFO:tasks.workunit.client.0.smithi007.stdout:1/739: rmdir d5/d2f 39 2024-09-21T04:35:30.019 INFO:tasks.workunit.client.0.smithi007.stdout:3/693: dread dc/f78 [0,4194304] 0 2024-09-21T04:35:30.020 INFO:tasks.workunit.client.0.smithi007.stdout:3/694: stat dc/d56/faa 0 2024-09-21T04:35:30.022 INFO:tasks.workunit.client.0.smithi007.stdout:1/740: write d5/d6/f40 [934945,111213] 0 2024-09-21T04:35:30.022 INFO:tasks.workunit.client.0.smithi007.stdout:2/451: write d0/d11/d31/d39/f3e [3037511,78641] 0 2024-09-21T04:35:30.026 INFO:tasks.workunit.client.0.smithi007.stdout:0/639: write d6/d75/dc5/d47/d8a/fd6 [1002857,120231] 0 2024-09-21T04:35:30.028 INFO:tasks.workunit.client.0.smithi007.stdout:8/323: rename d0/d3/d16/l29 to d0/l50 0 2024-09-21T04:35:30.032 INFO:tasks.workunit.client.0.smithi007.stdout:9/534: write d13/d74/d99/f3d [937402,9353] 0 2024-09-21T04:35:30.036 INFO:tasks.workunit.client.0.smithi007.stdout:0/640: mknod d6/d75/dc5/d47/d8a/d9c/cef 0 2024-09-21T04:35:30.038 INFO:tasks.workunit.client.0.smithi007.stdout:6/569: rename d3/d3e/d46/f81 to d3/d8e/dae/fbc 0 2024-09-21T04:35:30.040 INFO:tasks.workunit.client.0.smithi007.stdout:4/604: creat d2/d88/fc6 x:0 0 0 2024-09-21T04:35:30.042 INFO:tasks.workunit.client.0.smithi007.stdout:9/535: creat d13/d74/fa6 x:0 0 0 2024-09-21T04:35:30.051 INFO:tasks.workunit.client.0.smithi007.stdout:2/452: rename d0/d11/d31/d39/f5f to d0/d11/d31/d46/f7a 0 2024-09-21T04:35:30.052 INFO:tasks.workunit.client.0.smithi007.stdout:2/453: dread - d0/d11/d31/d52/f65 zero size 2024-09-21T04:35:30.056 INFO:tasks.workunit.client.0.smithi007.stdout:1/741: truncate d5/d6/d21/d2b/d4b/da6/f9c 5645358 0 2024-09-21T04:35:30.057 INFO:tasks.workunit.client.0.smithi007.stdout:1/742: fsync d5/d45/d5e/f60 0 2024-09-21T04:35:30.059 INFO:tasks.workunit.client.0.smithi007.stdout:7/557: write d3/d6/d14/f26 [4216713,71184] 0 2024-09-21T04:35:30.060 INFO:tasks.workunit.client.0.smithi007.stdout:4/605: mkdir d2/d17/d2e/d45/d5d/dc7 0 2024-09-21T04:35:30.061 INFO:tasks.workunit.client.0.smithi007.stdout:0/641: write d6/d75/dc5/d47/d8a/fc4 [455812,25057] 0 2024-09-21T04:35:30.062 INFO:tasks.workunit.client.0.smithi007.stdout:9/536: mknod d13/d18/d60/d6d/d85/da1/ca7 0 2024-09-21T04:35:30.064 INFO:tasks.workunit.client.0.smithi007.stdout:8/324: truncate d0/d3/d8/de/d26/f4a 3851524 0 2024-09-21T04:35:30.070 INFO:tasks.workunit.client.0.smithi007.stdout:2/454: rmdir d0 39 2024-09-21T04:35:30.074 INFO:tasks.workunit.client.0.smithi007.stdout:1/743: rmdir d5/d6/d21/d2b/d86 39 2024-09-21T04:35:30.075 INFO:tasks.workunit.client.0.smithi007.stdout:3/695: link dc/d22/d29/d32/d4d/f8e dc/d22/fdf 0 2024-09-21T04:35:30.080 INFO:tasks.workunit.client.0.smithi007.stdout:4/606: read d2/d97/da5/fad [530239,41165] 0 2024-09-21T04:35:30.083 INFO:tasks.workunit.client.0.smithi007.stdout:9/537: rename fa to d13/d18/d60/d83/fa8 0 2024-09-21T04:35:30.088 INFO:tasks.workunit.client.0.smithi007.stdout:4/607: rmdir d2 39 2024-09-21T04:35:30.089 INFO:tasks.workunit.client.0.smithi007.stdout:0/642: mkdir d6/d75/dc5/d47/df0 0 2024-09-21T04:35:30.090 INFO:tasks.workunit.client.0.smithi007.stdout:0/643: stat d6/d75/dc5/d47/d8a/d9c/cd8 0 2024-09-21T04:35:30.092 INFO:tasks.workunit.client.0.smithi007.stdout:6/570: getdents d3/d3b 0 2024-09-21T04:35:30.094 INFO:tasks.workunit.client.0.smithi007.stdout:7/558: truncate d3/d6/d14/f26 3592674 0 2024-09-21T04:35:30.095 INFO:tasks.workunit.client.0.smithi007.stdout:8/325: rename d0/d3/f49 to d0/d3/d8/de/d26/f51 0 2024-09-21T04:35:30.096 INFO:tasks.workunit.client.0.smithi007.stdout:9/538: fdatasync d13/d18/f4b 0 2024-09-21T04:35:30.097 INFO:tasks.workunit.client.0.smithi007.stdout:9/539: chown d13/l37 117 1 2024-09-21T04:35:30.098 INFO:tasks.workunit.client.0.smithi007.stdout:2/455: unlink d0/d11/d31/d46/d48/l4a 0 2024-09-21T04:35:30.099 INFO:tasks.workunit.client.0.smithi007.stdout:1/744: symlink d5/d6/d21/d2b/d86/dd8/led 0 2024-09-21T04:35:30.102 INFO:tasks.workunit.client.0.smithi007.stdout:3/696: chown dc/d22/d29/d32/d69/d6d 1 1 2024-09-21T04:35:30.103 INFO:tasks.workunit.client.0.smithi007.stdout:3/697: read - dc/d6c/fb1 zero size 2024-09-21T04:35:30.112 INFO:tasks.workunit.client.0.smithi007.stdout:9/540: chown d13/f5e 74352 1 2024-09-21T04:35:30.112 INFO:tasks.workunit.client.0.smithi007.stdout:1/745: mkdir d5/d6/d21/d2b/d4b/d8a/dee 0 2024-09-21T04:35:30.114 INFO:tasks.workunit.client.0.smithi007.stdout:4/608: creat d2/d7/d64/fc8 x:0 0 0 2024-09-21T04:35:30.114 INFO:tasks.workunit.client.0.smithi007.stdout:3/698: rmdir dc/d22/d29/d32/d4d/d84/dbb/dc1 39 2024-09-21T04:35:30.115 INFO:tasks.workunit.client.0.smithi007.stdout:9/541: read fe [267642,25415] 0 2024-09-21T04:35:30.116 INFO:tasks.workunit.client.0.smithi007.stdout:1/746: fdatasync d5/d6/d21/d2b/fd7 0 2024-09-21T04:35:30.118 INFO:tasks.workunit.client.0.smithi007.stdout:5/707: sync 2024-09-21T04:35:30.119 INFO:tasks.workunit.client.0.smithi007.stdout:1/747: chown d5/f48 21856 1 2024-09-21T04:35:30.119 INFO:tasks.workunit.client.0.smithi007.stdout:6/571: dwrite d3/d3e/d46/d60/d77/f5f [0,4194304] 0 2024-09-21T04:35:30.120 INFO:tasks.workunit.client.0.smithi007.stdout:7/559: dwrite d3/d6/d2e/d63/d6a/d81/fa0 [0,4194304] 0 2024-09-21T04:35:30.120 INFO:tasks.workunit.client.0.smithi007.stdout:9/542: write d13/d18/d60/d6d/f9d [783688,85593] 0 2024-09-21T04:35:30.120 INFO:tasks.workunit.client.0.smithi007.stdout:8/326: mkdir d0/d4e/d52 0 2024-09-21T04:35:30.125 INFO:tasks.workunit.client.0.smithi007.stdout:2/456: readlink d0/d32/l4c 0 2024-09-21T04:35:30.128 INFO:tasks.workunit.client.0.smithi007.stdout:3/699: stat dc/c50 0 2024-09-21T04:35:30.131 INFO:tasks.workunit.client.0.smithi007.stdout:0/644: link d6/d75/dc5/d18/d2e/d3f/l98 d6/d91/da3/dbf/dec/lf1 0 2024-09-21T04:35:30.132 INFO:tasks.workunit.client.0.smithi007.stdout:6/572: mkdir d3/d8e/dae/dbd 0 2024-09-21T04:35:30.133 INFO:tasks.workunit.client.0.smithi007.stdout:2/457: mkdir d0/d11/d31/d7b 0 2024-09-21T04:35:30.136 INFO:tasks.workunit.client.0.smithi007.stdout:4/609: rename d2/d17/dc0 to d2/d7/dc9 0 2024-09-21T04:35:30.138 INFO:tasks.workunit.client.0.smithi007.stdout:6/573: rmdir d3/d3e/d46 39 2024-09-21T04:35:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:2/458: creat d0/d11/d31/d46/f7c x:0 0 0 2024-09-21T04:35:30.142 INFO:tasks.workunit.client.0.smithi007.stdout:1/748: dread d5/d45/d5e/f24 [4194304,4194304] 0 2024-09-21T04:35:30.143 INFO:tasks.workunit.client.0.smithi007.stdout:2/459: chown d0/d11/d31/d46/d6c/f78 30563387 1 2024-09-21T04:35:30.146 INFO:tasks.workunit.client.0.smithi007.stdout:6/574: unlink d3/f5c 0 2024-09-21T04:35:30.147 INFO:tasks.workunit.client.0.smithi007.stdout:6/575: chown d3/d3e/c87 48119269 1 2024-09-21T04:35:30.149 INFO:tasks.workunit.client.0.smithi007.stdout:9/543: write d13/f66 [344850,1053] 0 2024-09-21T04:35:30.151 INFO:tasks.workunit.client.0.smithi007.stdout:5/708: rename d2/lb2 to d2/d27/d42/da3/d23/d5d/dc6/le3 0 2024-09-21T04:35:30.153 INFO:tasks.workunit.client.0.smithi007.stdout:1/749: creat d5/d45/d94/da0/d99/fef x:0 0 0 2024-09-21T04:35:30.158 INFO:tasks.workunit.client.0.smithi007.stdout:8/327: write d0/d3/d8/de/d26/f4a [4719437,128035] 0 2024-09-21T04:35:30.159 INFO:tasks.workunit.client.0.smithi007.stdout:8/328: fdatasync d0/f4f 0 2024-09-21T04:35:30.160 INFO:tasks.workunit.client.0.smithi007.stdout:8/329: chown d0/d3/d8/de/d26/c34 7469 1 2024-09-21T04:35:30.162 INFO:tasks.workunit.client.0.smithi007.stdout:9/544: rename d13/d74/d99 to d13/d18/d60/d83/da9 0 2024-09-21T04:35:30.163 INFO:tasks.workunit.client.0.smithi007.stdout:5/709: truncate d2/d27/d42/da3/d28/f97 757100 0 2024-09-21T04:35:30.165 INFO:tasks.workunit.client.0.smithi007.stdout:1/750: symlink d5/d45/d94/da0/lf0 0 2024-09-21T04:35:30.165 INFO:tasks.workunit.client.0.smithi007.stdout:5/710: fsync d2/d27/d42/da3/d23/fc5 0 2024-09-21T04:35:30.169 INFO:tasks.workunit.client.0.smithi007.stdout:7/560: rmdir d3/d6/d10/d62/d66/dc2 0 2024-09-21T04:35:30.170 INFO:tasks.workunit.client.0.smithi007.stdout:7/561: chown d3/d6/d14/d20/d80/f9d 305203528 1 2024-09-21T04:35:30.171 INFO:tasks.workunit.client.0.smithi007.stdout:6/576: creat d3/de/dba/fbe x:0 0 0 2024-09-21T04:35:30.177 INFO:tasks.workunit.client.0.smithi007.stdout:5/711: unlink d2/d27/d42/da3/fb3 0 2024-09-21T04:35:30.178 INFO:tasks.workunit.client.0.smithi007.stdout:5/712: chown d2/d1e/f37 199 1 2024-09-21T04:35:30.179 INFO:tasks.workunit.client.0.smithi007.stdout:2/460: creat d0/f7d x:0 0 0 2024-09-21T04:35:30.179 INFO:tasks.workunit.client.0.smithi007.stdout:4/610: write d2/d88/fa6 [4088511,3337] 0 2024-09-21T04:35:30.180 INFO:tasks.workunit.client.0.smithi007.stdout:1/751: rename d5/d45/d94 to d5/d6/d21/d64/df1 0 2024-09-21T04:35:30.182 INFO:tasks.workunit.client.0.smithi007.stdout:1/752: read d5/f28 [2352167,34265] 0 2024-09-21T04:35:30.186 INFO:tasks.workunit.client.0.smithi007.stdout:7/562: unlink d3/d6/d14/d20/d80/f64 0 2024-09-21T04:35:30.189 INFO:tasks.workunit.client.0.smithi007.stdout:9/545: dwrite d13/f5e [0,4194304] 0 2024-09-21T04:35:30.190 INFO:tasks.workunit.client.0.smithi007.stdout:5/713: chown d2/d27/d42/da3/d23/l29 0 1 2024-09-21T04:35:30.193 INFO:tasks.workunit.client.0.smithi007.stdout:6/577: rename d3/d3e/d85/d78/lb2 to d3/d3e/d85/d88/lbf 0 2024-09-21T04:35:30.197 INFO:tasks.workunit.client.0.smithi007.stdout:8/330: creat d0/d3/f53 x:0 0 0 2024-09-21T04:35:30.201 INFO:tasks.workunit.client.0.smithi007.stdout:7/563: write d3/d6/d2e/f38 [2595726,21499] 0 2024-09-21T04:35:30.201 INFO:tasks.workunit.client.0.smithi007.stdout:9/546: rmdir d13/d18/d60/d6d/d75/d9f 39 2024-09-21T04:35:30.205 INFO:tasks.workunit.client.0.smithi007.stdout:5/714: creat d2/d27/d3d/d56/fe4 x:0 0 0 2024-09-21T04:35:30.206 INFO:tasks.workunit.client.0.smithi007.stdout:1/753: dwrite d5/f5b [0,4194304] 0 2024-09-21T04:35:30.211 INFO:tasks.workunit.client.0.smithi007.stdout:6/578: symlink d3/d3b/lc0 0 2024-09-21T04:35:30.213 INFO:tasks.workunit.client.0.smithi007.stdout:8/331: creat d0/d3/d16/d3b/f54 x:0 0 0 2024-09-21T04:35:30.215 INFO:tasks.workunit.client.0.smithi007.stdout:8/332: write d0/f20 [1773289,96586] 0 2024-09-21T04:35:30.216 INFO:tasks.workunit.client.0.smithi007.stdout:1/754: mkdir d5/d6/d21/d2b/d4b/da6/df2 0 2024-09-21T04:35:30.217 INFO:tasks.workunit.client.0.smithi007.stdout:8/333: fsync d0/d3/f2b 0 2024-09-21T04:35:30.218 INFO:tasks.workunit.client.0.smithi007.stdout:5/715: rename d2/d27/d42/da3/d28/c3b to d2/d27/d3d/d9b/ce5 0 2024-09-21T04:35:30.227 INFO:tasks.workunit.client.0.smithi007.stdout:1/755: chown d5/d6/d21/dd4/leb 12476002 1 2024-09-21T04:35:30.230 INFO:tasks.workunit.client.0.smithi007.stdout:1/756: dread - d5/d6/d21/d64/fc2 zero size 2024-09-21T04:35:30.234 INFO:tasks.workunit.client.0.smithi007.stdout:9/547: dwrite fe [0,4194304] 0 2024-09-21T04:35:30.236 INFO:tasks.workunit.client.0.smithi007.stdout:5/716: dread d2/f5a [0,4194304] 0 2024-09-21T04:35:30.237 INFO:tasks.workunit.client.0.smithi007.stdout:8/334: symlink d0/d3/d8/l55 0 2024-09-21T04:35:30.239 INFO:tasks.workunit.client.0.smithi007.stdout:7/564: getdents d3/d6/d10 0 2024-09-21T04:35:30.241 INFO:tasks.workunit.client.0.smithi007.stdout:1/757: creat d5/d6/d21/d64/df1/da0/ff3 x:0 0 0 2024-09-21T04:35:30.242 INFO:tasks.workunit.client.0.smithi007.stdout:6/579: dread d3/d3e/d99/dbb/f57 [0,4194304] 0 2024-09-21T04:35:30.246 INFO:tasks.workunit.client.0.smithi007.stdout:8/335: rmdir d0/d3/d16 39 2024-09-21T04:35:30.246 INFO:tasks.workunit.client.0.smithi007.stdout:9/548: dwrite d13/f6b [0,4194304] 0 2024-09-21T04:35:30.248 INFO:tasks.workunit.client.0.smithi007.stdout:7/565: mknod d3/d6/d14/d20/d80/cc6 0 2024-09-21T04:35:30.254 INFO:tasks.workunit.client.0.smithi007.stdout:6/580: rename d3/d3e/d85/d88/l91 to d3/d94/lc1 0 2024-09-21T04:35:30.254 INFO:tasks.workunit.client.0.smithi007.stdout:9/549: dread d13/f5e [0,4194304] 0 2024-09-21T04:35:30.255 INFO:tasks.workunit.client.0.smithi007.stdout:7/566: symlink d3/d6/d14/d20/d58/da8/lc7 0 2024-09-21T04:35:30.256 INFO:tasks.workunit.client.0.smithi007.stdout:9/550: chown d13/d18/d60/d83/f8f 16557778 1 2024-09-21T04:35:30.262 INFO:tasks.workunit.client.0.smithi007.stdout:1/758: write d5/d6/d21/d3c/f43 [1989955,119264] 0 2024-09-21T04:35:30.264 INFO:tasks.workunit.client.0.smithi007.stdout:9/551: mkdir d13/d18/d60/d6d/d85/da1/daa 0 2024-09-21T04:35:30.264 INFO:tasks.workunit.client.0.smithi007.stdout:6/581: rmdir d3/d3e/d46/d60/d77 39 2024-09-21T04:35:30.267 INFO:tasks.workunit.client.0.smithi007.stdout:7/567: mknod d3/d6/d2e/d63/d6a/cc8 0 2024-09-21T04:35:30.268 INFO:tasks.workunit.client.0.smithi007.stdout:7/568: chown d3/d6/d10/c52 3441964 1 2024-09-21T04:35:30.268 INFO:tasks.workunit.client.0.smithi007.stdout:1/759: symlink d5/d6/dc5/lf4 0 2024-09-21T04:35:30.268 INFO:tasks.workunit.client.0.smithi007.stdout:8/336: link d0/d3/d8/c2d d0/c56 0 2024-09-21T04:35:30.271 INFO:tasks.workunit.client.0.smithi007.stdout:6/582: truncate d3/d3e/d99/dbb/f57 930783 0 2024-09-21T04:35:30.272 INFO:tasks.workunit.client.0.smithi007.stdout:6/583: write d3/d3e/d46/f68 [256686,58949] 0 2024-09-21T04:35:30.273 INFO:tasks.workunit.client.0.smithi007.stdout:7/569: chown d3/d6/d10/d62/lb0 27244 1 2024-09-21T04:35:30.275 INFO:tasks.workunit.client.0.smithi007.stdout:1/760: creat d5/d6/d21/d2b/d86/dd8/ff5 x:0 0 0 2024-09-21T04:35:30.276 INFO:tasks.workunit.client.0.smithi007.stdout:1/761: write d5/d45/d5e/fc1 [667533,39395] 0 2024-09-21T04:35:30.278 INFO:tasks.workunit.client.0.smithi007.stdout:1/762: write d5/d6/d21/d2b/d4b/da6/d58/fac [32331,53157] 0 2024-09-21T04:35:30.278 INFO:tasks.workunit.client.0.smithi007.stdout:8/337: symlink d0/d3/d8/de/d26/d4d/l57 0 2024-09-21T04:35:30.280 INFO:tasks.workunit.client.0.smithi007.stdout:6/584: creat d3/d94/fc2 x:0 0 0 2024-09-21T04:35:30.281 INFO:tasks.workunit.client.0.smithi007.stdout:8/338: write d0/d3/d8/de/d26/f4a [3072491,8561] 0 2024-09-21T04:35:30.282 INFO:tasks.workunit.client.0.smithi007.stdout:1/763: chown d5/d6/d21/d64/df1/da0/d99/dae/fd1 11994749 1 2024-09-21T04:35:30.283 INFO:tasks.workunit.client.0.smithi007.stdout:8/339: chown d0/d3/d8/de 2 1 2024-09-21T04:35:30.284 INFO:tasks.workunit.client.0.smithi007.stdout:9/552: truncate d13/d18/d60/f88 1691504 0 2024-09-21T04:35:30.284 INFO:tasks.workunit.client.0.smithi007.stdout:8/340: read - d0/f4f zero size 2024-09-21T04:35:30.286 INFO:tasks.workunit.client.0.smithi007.stdout:9/553: stat d13/d18/d60/d6d/d75/f7e 0 2024-09-21T04:35:30.287 INFO:tasks.workunit.client.0.smithi007.stdout:1/764: unlink d5/fe 0 2024-09-21T04:35:30.288 INFO:tasks.workunit.client.0.smithi007.stdout:1/765: write d5/d6/d21/d64/df1/fb1 [1037985,85896] 0 2024-09-21T04:35:30.289 INFO:tasks.workunit.client.0.smithi007.stdout:6/585: truncate d3/d3e/f48 1020437 0 2024-09-21T04:35:30.290 INFO:tasks.workunit.client.0.smithi007.stdout:1/766: write d5/fd0 [2064990,106952] 0 2024-09-21T04:35:30.297 INFO:tasks.workunit.client.0.smithi007.stdout:6/586: dwrite d3/f1e [0,4194304] 0 2024-09-21T04:35:30.305 INFO:tasks.workunit.client.0.smithi007.stdout:7/570: rename d3/d6/d2e/d5e/c76 to d3/d6/d10/d62/cc9 0 2024-09-21T04:35:30.308 INFO:tasks.workunit.client.0.smithi007.stdout:1/767: creat d5/d6/d21/d64/df1/ddd/ff6 x:0 0 0 2024-09-21T04:35:30.310 INFO:tasks.workunit.client.0.smithi007.stdout:6/587: dread d3/d3e/d85/d78/f84 [0,4194304] 0 2024-09-21T04:35:30.315 INFO:tasks.workunit.client.0.smithi007.stdout:1/768: dread d5/d6/d21/d3c/f43 [0,4194304] 0 2024-09-21T04:35:30.337 INFO:tasks.workunit.client.0.smithi007.stdout:9/554: mknod d13/d89/cab 0 2024-09-21T04:35:30.338 INFO:tasks.workunit.client.0.smithi007.stdout:7/571: fdatasync d3/d6/d2e/fab 0 2024-09-21T04:35:30.340 INFO:tasks.workunit.client.0.smithi007.stdout:6/588: fdatasync d3/f97 0 2024-09-21T04:35:30.341 INFO:tasks.workunit.client.0.smithi007.stdout:1/769: fsync d5/f25 0 2024-09-21T04:35:30.344 INFO:tasks.workunit.client.0.smithi007.stdout:1/770: write d5/d6/f40 [894904,97906] 0 2024-09-21T04:35:30.349 INFO:tasks.workunit.client.0.smithi007.stdout:8/341: rename d0/d3/d8/f1d to d0/d3/d8/f58 0 2024-09-21T04:35:30.353 INFO:tasks.workunit.client.0.smithi007.stdout:7/572: fdatasync d3/d6/d13/d4b/f7d 0 2024-09-21T04:35:30.362 INFO:tasks.workunit.client.0.smithi007.stdout:9/555: dread d13/f21 [0,4194304] 0 2024-09-21T04:35:30.423 INFO:tasks.workunit.client.0.smithi007.stdout:7/573: rename d3/d6/d14/d20/d58/da8 to d3/d6/d14/d20/d80/dca 0 2024-09-21T04:35:30.427 INFO:tasks.workunit.client.0.smithi007.stdout:8/342: creat d0/d3/d16/d3b/f59 x:0 0 0 2024-09-21T04:35:30.431 INFO:tasks.workunit.client.0.smithi007.stdout:9/556: unlink d13/d18/l22 0 2024-09-21T04:35:30.431 INFO:tasks.workunit.client.0.smithi007.stdout:6/589: link d3/d3e/d46/d60/f6e d3/de/d12/d19/da3/fc3 0 2024-09-21T04:35:30.435 INFO:tasks.workunit.client.0.smithi007.stdout:7/574: symlink d3/d6/d13/lcb 0 2024-09-21T04:35:30.435 INFO:tasks.workunit.client.0.smithi007.stdout:8/343: mknod d0/d3/d8/c5a 0 2024-09-21T04:35:30.437 INFO:tasks.workunit.client.0.smithi007.stdout:6/590: rename d3/f38 to d3/d94/fc4 0 2024-09-21T04:35:30.438 INFO:tasks.workunit.client.0.smithi007.stdout:9/557: truncate d13/d18/d60/d83/da9/f40 5070378 0 2024-09-21T04:35:30.441 INFO:tasks.workunit.client.0.smithi007.stdout:6/591: dread d3/f54 [0,4194304] 0 2024-09-21T04:35:30.442 INFO:tasks.workunit.client.0.smithi007.stdout:8/344: rename d0/l36 to d0/d3/d8/de/d26/l5b 0 2024-09-21T04:35:30.443 INFO:tasks.workunit.client.0.smithi007.stdout:9/558: truncate f10 225210 0 2024-09-21T04:35:30.444 INFO:tasks.workunit.client.0.smithi007.stdout:7/575: mknod d3/d6/d40/da1/ccc 0 2024-09-21T04:35:30.447 INFO:tasks.workunit.client.0.smithi007.stdout:8/345: symlink d0/d3/d8/de/d26/l5c 0 2024-09-21T04:35:30.448 INFO:tasks.workunit.client.0.smithi007.stdout:9/559: creat d13/d18/d60/d6d/d85/fac x:0 0 0 2024-09-21T04:35:30.451 INFO:tasks.workunit.client.0.smithi007.stdout:8/346: mkdir d0/d3/d8/de/d26/d4d/d5d 0 2024-09-21T04:35:30.451 INFO:tasks.workunit.client.0.smithi007.stdout:7/576: fdatasync d3/d6/f7 0 2024-09-21T04:35:30.453 INFO:tasks.workunit.client.0.smithi007.stdout:6/592: link d3/d3b/f7a d3/d3e/d85/d88/fc5 0 2024-09-21T04:35:30.453 INFO:tasks.workunit.client.0.smithi007.stdout:8/347: truncate d0/d3/d8/de/d40/f45 714597 0 2024-09-21T04:35:30.458 INFO:tasks.workunit.client.0.smithi007.stdout:8/348: mkdir d0/d3/d8/de/d5e 0 2024-09-21T04:35:30.460 INFO:tasks.workunit.client.0.smithi007.stdout:9/560: link d13/d18/f3f d13/d18/d60/d6d/d75/d9f/fad 0 2024-09-21T04:35:30.463 INFO:tasks.workunit.client.0.smithi007.stdout:7/577: link d3/d6/d14/d20/d80/db2/la6 d3/d6/d10/lcd 0 2024-09-21T04:35:30.465 INFO:tasks.workunit.client.0.smithi007.stdout:6/593: dread d3/d3e/d46/d60/d77/f63 [0,4194304] 0 2024-09-21T04:35:30.465 INFO:tasks.workunit.client.0.smithi007.stdout:9/561: rename d13/d18/d60/f7d to d13/d18/d60/d6d/fae 0 2024-09-21T04:35:30.468 INFO:tasks.workunit.client.0.smithi007.stdout:6/594: symlink d3/d94/lc6 0 2024-09-21T04:35:30.469 INFO:tasks.workunit.client.0.smithi007.stdout:8/349: creat d0/d3/d8/f5f x:0 0 0 2024-09-21T04:35:30.471 INFO:tasks.workunit.client.0.smithi007.stdout:9/562: mkdir d13/d18/daf 0 2024-09-21T04:35:30.472 INFO:tasks.workunit.client.0.smithi007.stdout:6/595: mknod d3/d3e/cc7 0 2024-09-21T04:35:30.474 INFO:tasks.workunit.client.0.smithi007.stdout:8/350: rename d0/d3/d16/f2c to d0/d3/d8/d46/f60 0 2024-09-21T04:35:30.476 INFO:tasks.workunit.client.0.smithi007.stdout:9/563: mknod d13/d18/d60/d6d/d85/da1/daa/cb0 0 2024-09-21T04:35:30.477 INFO:tasks.workunit.client.0.smithi007.stdout:8/351: creat d0/d3/d8/de/d5e/f61 x:0 0 0 2024-09-21T04:35:30.480 INFO:tasks.workunit.client.0.smithi007.stdout:8/352: rmdir d0/d3/d8/de/d5e 39 2024-09-21T04:35:30.482 INFO:tasks.workunit.client.0.smithi007.stdout:8/353: symlink d0/d3/d8/de/d26/d4d/d5d/l62 0 2024-09-21T04:35:30.489 INFO:tasks.workunit.client.0.smithi007.stdout:8/354: unlink d0/c7 0 2024-09-21T04:35:30.493 INFO:tasks.workunit.client.0.smithi007.stdout:8/355: link d0/d3/f53 d0/d3/d8/d46/f63 0 2024-09-21T04:35:30.498 INFO:tasks.workunit.client.0.smithi007.stdout:9/564: fsync f10 0 2024-09-21T04:35:30.499 INFO:tasks.workunit.client.0.smithi007.stdout:9/565: read d13/f4f [2232527,102280] 0 2024-09-21T04:35:30.503 INFO:tasks.workunit.client.0.smithi007.stdout:9/566: symlink d13/d18/d60/d83/lb1 0 2024-09-21T04:35:30.508 INFO:tasks.workunit.client.0.smithi007.stdout:9/567: dwrite d13/d18/f3c [0,4194304] 0 2024-09-21T04:35:30.509 INFO:tasks.workunit.client.0.smithi007.stdout:8/356: truncate d0/d3/d8/de/d5e/f61 471702 0 2024-09-21T04:35:30.516 INFO:tasks.workunit.client.0.smithi007.stdout:8/357: dwrite d0/d3/d16/f21 [0,4194304] 0 2024-09-21T04:35:30.519 INFO:tasks.workunit.client.0.smithi007.stdout:7/578: write d3/d6/d13/f47 [513528,98488] 0 2024-09-21T04:35:30.525 INFO:tasks.workunit.client.0.smithi007.stdout:8/358: dwrite d0/d3/d8/f3f [0,4194304] 0 2024-09-21T04:35:30.527 INFO:tasks.workunit.client.0.smithi007.stdout:8/359: truncate d0/d3/d8/de/d26/f51 32069 0 2024-09-21T04:35:30.528 INFO:tasks.workunit.client.0.smithi007.stdout:6/596: write d3/d3e/d85/d78/f73 [3209931,30155] 0 2024-09-21T04:35:30.529 INFO:tasks.workunit.client.0.smithi007.stdout:8/360: stat d0/d3/d16/l1e 0 2024-09-21T04:35:30.529 INFO:tasks.workunit.client.0.smithi007.stdout:6/597: fsync d3/d3e/d85/fb1 0 2024-09-21T04:35:30.538 INFO:tasks.workunit.client.0.smithi007.stdout:8/361: dwrite d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:30.540 INFO:tasks.workunit.client.0.smithi007.stdout:8/362: write d0/d3/d8/de/f18 [3651137,62850] 0 2024-09-21T04:35:30.540 INFO:tasks.workunit.client.0.smithi007.stdout:6/598: link d3/d3e/f55 d3/de/d58/fc8 0 2024-09-21T04:35:30.547 INFO:tasks.workunit.client.0.smithi007.stdout:6/599: truncate d3/de/d12/f17 4213785 0 2024-09-21T04:35:30.549 INFO:tasks.workunit.client.0.smithi007.stdout:6/600: dread - d3/d3e/d85/fb1 zero size 2024-09-21T04:35:30.551 INFO:tasks.workunit.client.0.smithi007.stdout:8/363: getdents d0 0 2024-09-21T04:35:30.554 INFO:tasks.workunit.client.0.smithi007.stdout:9/568: dwrite d13/d74/f78 [0,4194304] 0 2024-09-21T04:35:30.556 INFO:tasks.workunit.client.0.smithi007.stdout:9/569: write d13/d18/d60/d83/da9/f3d [1447889,73256] 0 2024-09-21T04:35:30.557 INFO:tasks.workunit.client.0.smithi007.stdout:8/364: creat d0/d3/d16/d3b/f64 x:0 0 0 2024-09-21T04:35:30.558 INFO:tasks.workunit.client.0.smithi007.stdout:6/601: rename d3/d3e/d99/dbb/d67 to d3/d3e/d99/dbb/d95/dc9 0 2024-09-21T04:35:30.559 INFO:tasks.workunit.client.0.smithi007.stdout:6/602: stat d3/d3e/f55 0 2024-09-21T04:35:30.565 INFO:tasks.workunit.client.0.smithi007.stdout:8/365: mknod d0/d3/d16/d3b/c65 0 2024-09-21T04:35:30.569 INFO:tasks.workunit.client.0.smithi007.stdout:9/570: creat d13/d18/d60/d6d/fb2 x:0 0 0 2024-09-21T04:35:30.573 INFO:tasks.workunit.client.0.smithi007.stdout:8/366: rmdir d0 39 2024-09-21T04:35:30.576 INFO:tasks.workunit.client.0.smithi007.stdout:6/603: dwrite d3/f6 [0,4194304] 0 2024-09-21T04:35:30.578 INFO:tasks.workunit.client.0.smithi007.stdout:6/604: chown d3/de/d12/f50 1876533366 1 2024-09-21T04:35:30.579 INFO:tasks.workunit.client.0.smithi007.stdout:8/367: creat d0/d4e/f66 x:0 0 0 2024-09-21T04:35:30.580 INFO:tasks.workunit.client.0.smithi007.stdout:8/368: stat d0/d3/d16 0 2024-09-21T04:35:30.580 INFO:tasks.workunit.client.0.smithi007.stdout:6/605: mknod d3/d8e/dae/cca 0 2024-09-21T04:35:30.581 INFO:tasks.workunit.client.0.smithi007.stdout:6/606: write d3/de/d12/d19/f20 [3198230,550] 0 2024-09-21T04:35:30.585 INFO:tasks.workunit.client.0.smithi007.stdout:6/607: fsync d3/d94/fc4 0 2024-09-21T04:35:30.586 INFO:tasks.workunit.client.0.smithi007.stdout:9/571: truncate d13/f5b 361476 0 2024-09-21T04:35:30.589 INFO:tasks.workunit.client.0.smithi007.stdout:9/572: dread d13/d18/d60/d6d/d75/d9f/fad [0,4194304] 0 2024-09-21T04:35:30.591 INFO:tasks.workunit.client.0.smithi007.stdout:6/608: fsync d3/de/d12/f2b 0 2024-09-21T04:35:30.593 INFO:tasks.workunit.client.0.smithi007.stdout:8/369: getdents d0/d3/d8/de/d26 0 2024-09-21T04:35:30.594 INFO:tasks.workunit.client.0.smithi007.stdout:8/370: dread - d0/d3/d16/f31 zero size 2024-09-21T04:35:30.594 INFO:tasks.workunit.client.0.smithi007.stdout:9/573: dread d13/d18/f3c [0,4194304] 0 2024-09-21T04:35:30.598 INFO:tasks.workunit.client.0.smithi007.stdout:6/609: rename d3/d3e/d85 to d3/d3e/d46/d60/d77/dcb 0 2024-09-21T04:35:30.600 INFO:tasks.workunit.client.0.smithi007.stdout:9/574: truncate d13/d18/f67 4532250 0 2024-09-21T04:35:30.602 INFO:tasks.workunit.client.0.smithi007.stdout:9/575: chown d13/d18/d60/d83/da9/f9e 20203 1 2024-09-21T04:35:30.603 INFO:tasks.workunit.client.0.smithi007.stdout:9/576: fsync d13/d18/d60/f95 0 2024-09-21T04:35:30.604 INFO:tasks.workunit.client.0.smithi007.stdout:8/371: creat d0/d3/d8/f67 x:0 0 0 2024-09-21T04:35:30.605 INFO:tasks.workunit.client.0.smithi007.stdout:6/610: mkdir d3/d3e/d46/d60/dcc 0 2024-09-21T04:35:30.606 INFO:tasks.workunit.client.0.smithi007.stdout:8/372: chown d0/d3/d16/d3b/c65 109343 1 2024-09-21T04:35:30.606 INFO:tasks.workunit.client.0.smithi007.stdout:9/577: creat d13/d18/d60/d6d/d75/d9f/fb3 x:0 0 0 2024-09-21T04:35:30.609 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:30 smithi007 ceph-mon[27954]: pgmap v36: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 87 MiB/s rd, 224 MiB/s wr, 461 op/s 2024-09-21T04:35:30.617 INFO:tasks.workunit.client.0.smithi007.stdout:8/373: write d0/d3/d8/de/f17 [2078214,45618] 0 2024-09-21T04:35:30.620 INFO:tasks.workunit.client.0.smithi007.stdout:9/578: dread d13/d18/d60/d83/f8f [0,4194304] 0 2024-09-21T04:35:30.621 INFO:tasks.workunit.client.0.smithi007.stdout:8/374: write d0/d3/d8/de/d26/f51 [556236,71100] 0 2024-09-21T04:35:30.625 INFO:tasks.workunit.client.0.smithi007.stdout:8/375: mknod d0/d3/d8/de/c68 0 2024-09-21T04:35:30.626 INFO:tasks.workunit.client.0.smithi007.stdout:8/376: chown d0/d3/d8/de 523547747 1 2024-09-21T04:35:30.627 INFO:tasks.workunit.client.0.smithi007.stdout:8/377: dread - d0/d3/d8/f5f zero size 2024-09-21T04:35:30.631 INFO:tasks.workunit.client.0.smithi007.stdout:8/378: creat d0/d3/d8/de/f69 x:0 0 0 2024-09-21T04:35:30.631 INFO:tasks.workunit.client.0.smithi007.stdout:8/379: readlink d0/l13 0 2024-09-21T04:35:30.634 INFO:tasks.workunit.client.0.smithi007.stdout:9/579: dread d13/d18/f35 [0,4194304] 0 2024-09-21T04:35:30.638 INFO:tasks.workunit.client.0.smithi007.stdout:9/580: creat d13/d18/d60/d83/da9/fb4 x:0 0 0 2024-09-21T04:35:30.644 INFO:tasks.workunit.client.0.smithi007.stdout:8/380: dread d0/d3/d8/de/d26/d35/f37 [0,4194304] 0 2024-09-21T04:35:30.650 INFO:tasks.workunit.client.0.smithi007.stdout:9/581: truncate d13/d18/d60/d83/da9/f3d 700268 0 2024-09-21T04:35:30.650 INFO:tasks.workunit.client.0.smithi007.stdout:8/381: dwrite d0/d3/d8/f3f [0,4194304] 0 2024-09-21T04:35:30.652 INFO:tasks.workunit.client.0.smithi007.stdout:9/582: write d13/d18/f36 [653331,129348] 0 2024-09-21T04:35:30.653 INFO:tasks.workunit.client.0.smithi007.stdout:8/382: dread d0/d3/d8/de/d5e/f61 [0,4194304] 0 2024-09-21T04:35:30.655 INFO:tasks.workunit.client.0.smithi007.stdout:8/383: write d0/d3/d8/de/d26/f4a [1553430,65244] 0 2024-09-21T04:35:30.662 INFO:tasks.workunit.client.0.smithi007.stdout:8/384: rmdir d0/d4e 39 2024-09-21T04:35:30.668 INFO:tasks.workunit.client.0.smithi007.stdout:8/385: link d0/d3/d8/c5a d0/d4e/d52/c6a 0 2024-09-21T04:35:30.669 INFO:tasks.workunit.client.0.smithi007.stdout:8/386: rmdir d0/d4e/d52 39 2024-09-21T04:35:30.672 INFO:tasks.workunit.client.0.smithi007.stdout:8/387: unlink d0/d3/d8/de/d26/d4d/d5d/l62 0 2024-09-21T04:35:30.677 INFO:tasks.workunit.client.0.smithi007.stdout:9/583: dwrite d13/d18/f54 [0,4194304] 0 2024-09-21T04:35:30.679 INFO:tasks.workunit.client.0.smithi007.stdout:8/388: dwrite d0/d3/d16/d3b/f64 [0,4194304] 0 2024-09-21T04:35:30.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:30 smithi053 ceph-mon[35554]: pgmap v36: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 87 MiB/s rd, 224 MiB/s wr, 461 op/s 2024-09-21T04:35:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:9/584: unlink d13/d18/d60/d6d/d85/fac 0 2024-09-21T04:35:30.922 INFO:tasks.workunit.client.0.smithi007.stdout:8/389: readlink d0/d3/d8/lc 0 2024-09-21T04:35:30.925 INFO:tasks.workunit.client.0.smithi007.stdout:9/585: dwrite d13/f66 [0,4194304] 0 2024-09-21T04:35:30.928 INFO:tasks.workunit.client.0.smithi007.stdout:3/700: sync 2024-09-21T04:35:30.937 INFO:tasks.workunit.client.0.smithi007.stdout:8/390: symlink d0/d3/d8/de/d26/d4d/d5d/l6b 0 2024-09-21T04:35:30.938 INFO:tasks.workunit.client.0.smithi007.stdout:9/586: rmdir d13/d18/d80 39 2024-09-21T04:35:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:3/701: unlink dc/d22/d29/d32/d69/d6d/f48 0 2024-09-21T04:35:30.948 INFO:tasks.workunit.client.0.smithi007.stdout:9/587: symlink d13/d18/d60/d6d/d75/d9f/lb5 0 2024-09-21T04:35:30.950 INFO:tasks.workunit.client.0.smithi007.stdout:9/588: read d13/d18/d60/d6d/d75/f7e [2545673,9195] 0 2024-09-21T04:35:30.952 INFO:tasks.workunit.client.0.smithi007.stdout:3/702: fsync dc/d18/f58 0 2024-09-21T04:35:30.962 INFO:tasks.workunit.client.0.smithi007.stdout:3/703: dread dc/d22/d5a/d71/fcf [0,4194304] 0 2024-09-21T04:35:30.964 INFO:tasks.workunit.client.0.smithi007.stdout:3/704: chown dc/d22/d29/d32/d4d/d84/dd0/cc3 845 1 2024-09-21T04:35:31.025 INFO:tasks.workunit.client.0.smithi007.stdout:8/391: truncate d0/d3/d16/d3b/f64 3458133 0 2024-09-21T04:35:31.025 INFO:tasks.workunit.client.0.smithi007.stdout:8/392: chown d0/d3 112452077 1 2024-09-21T04:35:31.027 INFO:tasks.workunit.client.0.smithi007.stdout:0/645: sync 2024-09-21T04:35:31.027 INFO:tasks.workunit.client.0.smithi007.stdout:2/461: sync 2024-09-21T04:35:31.027 INFO:tasks.workunit.client.0.smithi007.stdout:5/717: sync 2024-09-21T04:35:31.027 INFO:tasks.workunit.client.0.smithi007.stdout:6/611: sync 2024-09-21T04:35:31.027 INFO:tasks.workunit.client.0.smithi007.stdout:4/611: sync 2024-09-21T04:35:31.027 INFO:tasks.workunit.client.0.smithi007.stdout:1/771: sync 2024-09-21T04:35:31.027 INFO:tasks.workunit.client.0.smithi007.stdout:7/579: sync 2024-09-21T04:35:31.031 INFO:tasks.workunit.client.0.smithi007.stdout:4/612: write d2/d7/fa [1951983,58064] 0 2024-09-21T04:35:31.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/462: creat d0/d11/d6e/f7e x:0 0 0 2024-09-21T04:35:31.032 INFO:tasks.workunit.client.0.smithi007.stdout:1/772: mknod d5/d6/d21/d3c/dda/cf7 0 2024-09-21T04:35:31.034 INFO:tasks.workunit.client.0.smithi007.stdout:5/718: fdatasync d2/d27/d42/da3/d23/d49/d44/d54/f59 0 2024-09-21T04:35:31.036 INFO:tasks.workunit.client.0.smithi007.stdout:7/580: write d3/d6/d14/d20/d58/f95 [414421,62326] 0 2024-09-21T04:35:31.036 INFO:tasks.workunit.client.0.smithi007.stdout:1/773: readlink d5/d6/dc5/lf4 0 2024-09-21T04:35:31.041 INFO:tasks.workunit.client.0.smithi007.stdout:7/581: chown d3/d6/d14/f1b 792107 1 2024-09-21T04:35:31.042 INFO:tasks.workunit.client.0.smithi007.stdout:8/393: dwrite d0/f20 [4194304,4194304] 0 2024-09-21T04:35:31.043 INFO:tasks.workunit.client.0.smithi007.stdout:4/613: rename d2/d7/fa to d2/d7/fca 0 2024-09-21T04:35:31.044 INFO:tasks.workunit.client.0.smithi007.stdout:1/774: mknod d5/d6/dc5/cf8 0 2024-09-21T04:35:31.046 INFO:tasks.workunit.client.0.smithi007.stdout:8/394: stat d0/d3/d8/de/d40/f45 0 2024-09-21T04:35:31.049 INFO:tasks.workunit.client.0.smithi007.stdout:8/395: chown d0/d3 74 1 2024-09-21T04:35:31.050 INFO:tasks.workunit.client.0.smithi007.stdout:2/463: dwrite d0/d2/fc [0,4194304] 0 2024-09-21T04:35:31.053 INFO:tasks.workunit.client.0.smithi007.stdout:5/719: dwrite d2/d27/d42/da3/d23/d2a/f75 [0,4194304] 0 2024-09-21T04:35:31.061 INFO:tasks.workunit.client.0.smithi007.stdout:3/705: dwrite dc/f13 [0,4194304] 0 2024-09-21T04:35:31.064 INFO:tasks.workunit.client.0.smithi007.stdout:3/706: chown dc 120018 1 2024-09-21T04:35:31.064 INFO:tasks.workunit.client.0.smithi007.stdout:2/464: unlink d0/d11/d31/d39/c61 0 2024-09-21T04:35:31.069 INFO:tasks.workunit.client.0.smithi007.stdout:7/582: dwrite d3/d6/d85/f25 [4194304,4194304] 0 2024-09-21T04:35:31.071 INFO:tasks.workunit.client.0.smithi007.stdout:8/396: dwrite d0/f43 [0,4194304] 0 2024-09-21T04:35:31.072 INFO:tasks.workunit.client.0.smithi007.stdout:3/707: fsync dc/d22/f59 0 2024-09-21T04:35:31.074 INFO:tasks.workunit.client.0.smithi007.stdout:4/614: link d2/d7/d55/l56 d2/d17/d2e/d45/d5e/d90/lcb 0 2024-09-21T04:35:31.078 INFO:tasks.workunit.client.0.smithi007.stdout:7/583: rmdir d3/d6/d40/da1 39 2024-09-21T04:35:31.079 INFO:tasks.workunit.client.0.smithi007.stdout:3/708: readlink dc/d22/lad 0 2024-09-21T04:35:31.081 INFO:tasks.workunit.client.0.smithi007.stdout:8/397: stat d0/d3/d8/de/c3a 0 2024-09-21T04:35:31.081 INFO:tasks.workunit.client.0.smithi007.stdout:2/465: dwrite d0/d2/f6 [4194304,4194304] 0 2024-09-21T04:35:31.089 INFO:tasks.workunit.client.0.smithi007.stdout:8/398: mkdir d0/d3/d8/de/d40/d6c 0 2024-09-21T04:35:31.094 INFO:tasks.workunit.client.0.smithi007.stdout:8/399: symlink d0/d3/d8/de/d26/l6d 0 2024-09-21T04:35:31.094 INFO:tasks.workunit.client.0.smithi007.stdout:2/466: dwrite d0/d11/d31/d46/d48/f62 [0,4194304] 0 2024-09-21T04:35:31.096 INFO:tasks.workunit.client.0.smithi007.stdout:2/467: chown d0/d11/d31/d46/d6d 4898 1 2024-09-21T04:35:31.104 INFO:tasks.workunit.client.0.smithi007.stdout:2/468: dwrite d0/d11/d31/d39/f3e [0,4194304] 0 2024-09-21T04:35:31.110 INFO:tasks.workunit.client.0.smithi007.stdout:2/469: mknod d0/d2/c7f 0 2024-09-21T04:35:31.110 INFO:tasks.workunit.client.0.smithi007.stdout:2/470: chown d0/d11/l69 16347 1 2024-09-21T04:35:31.113 INFO:tasks.workunit.client.0.smithi007.stdout:2/471: dread d0/d11/f4e [0,4194304] 0 2024-09-21T04:35:31.114 INFO:tasks.workunit.client.0.smithi007.stdout:2/472: chown d0/d11/d31/d46/d6d 166891302 1 2024-09-21T04:35:31.115 INFO:tasks.workunit.client.0.smithi007.stdout:2/473: write d0/d11/f28 [2722529,29389] 0 2024-09-21T04:35:31.118 INFO:tasks.workunit.client.0.smithi007.stdout:1/775: write d5/d6/d21/d64/df1/fab [1041791,116751] 0 2024-09-21T04:35:31.122 INFO:tasks.workunit.client.0.smithi007.stdout:5/720: write d2/d27/d42/da3/fb9 [1335438,125513] 0 2024-09-21T04:35:31.123 INFO:tasks.workunit.client.0.smithi007.stdout:1/776: getdents d5/d6/d21/d2b/d86 0 2024-09-21T04:35:31.125 INFO:tasks.workunit.client.0.smithi007.stdout:9/589: rename fe to d13/d18/d60/d6d/fb6 0 2024-09-21T04:35:31.127 INFO:tasks.workunit.client.0.smithi007.stdout:9/590: write d13/d18/d60/d83/da9/fb4 [842857,63108] 0 2024-09-21T04:35:31.131 INFO:tasks.workunit.client.0.smithi007.stdout:6/612: rename d3/c13 to d3/de/d12/ccd 0 2024-09-21T04:35:31.131 INFO:tasks.workunit.client.0.smithi007.stdout:9/591: truncate d13/d18/d60/d6d/d75/d7f/f97 167251 0 2024-09-21T04:35:31.134 INFO:tasks.workunit.client.0.smithi007.stdout:7/584: dwrite d3/d6/d2e/d63/f73 [0,4194304] 0 2024-09-21T04:35:31.135 INFO:tasks.workunit.client.0.smithi007.stdout:0/646: rename d6/d75/dc5/d57/c73 to d6/d75/dc5/d18/d2e/d60/cf2 0 2024-09-21T04:35:31.139 INFO:tasks.workunit.client.0.smithi007.stdout:9/592: creat d13/d18/d60/d83/da9/fb7 x:0 0 0 2024-09-21T04:35:31.140 INFO:tasks.workunit.client.0.smithi007.stdout:1/777: dread d5/fd0 [0,4194304] 0 2024-09-21T04:35:31.144 INFO:tasks.workunit.client.0.smithi007.stdout:4/615: rename d2/d7/c76 to d2/d7/dc9/ccc 0 2024-09-21T04:35:31.146 INFO:tasks.workunit.client.0.smithi007.stdout:9/593: creat d13/d18/d60/d6d/d85/da1/daa/fb8 x:0 0 0 2024-09-21T04:35:31.148 INFO:tasks.workunit.client.0.smithi007.stdout:9/594: chown d13/f5e 982 1 2024-09-21T04:35:31.148 INFO:tasks.workunit.client.0.smithi007.stdout:8/400: dwrite d0/d3/d16/f31 [0,4194304] 0 2024-09-21T04:35:31.148 INFO:tasks.workunit.client.0.smithi007.stdout:9/595: read - d13/d18/d60/f95 zero size 2024-09-21T04:35:31.150 INFO:tasks.workunit.client.0.smithi007.stdout:0/647: creat d6/d75/dc5/d47/d8a/de9/ff3 x:0 0 0 2024-09-21T04:35:31.157 INFO:tasks.workunit.client.0.smithi007.stdout:1/778: mknod d5/d6/d21/d3c/dd5/dea/cf9 0 2024-09-21T04:35:31.159 INFO:tasks.workunit.client.0.smithi007.stdout:1/779: write d5/d6/d21/d2b/d4b/da6/d58/f73 [7027039,128688] 0 2024-09-21T04:35:31.160 INFO:tasks.workunit.client.0.smithi007.stdout:8/401: symlink d0/d3/d16/l6e 0 2024-09-21T04:35:31.163 INFO:tasks.workunit.client.0.smithi007.stdout:8/402: chown d0/d3/d8/f47 9 1 2024-09-21T04:35:31.163 INFO:tasks.workunit.client.0.smithi007.stdout:0/648: dread d6/f13 [0,4194304] 0 2024-09-21T04:35:31.164 INFO:tasks.workunit.client.0.smithi007.stdout:3/709: rename dc/d22/d29/d32/d4d/d84/dd0/da6/d80/l94 to dc/d22/d29/d32/d4d/d84/dbb/le0 0 2024-09-21T04:35:31.166 INFO:tasks.workunit.client.0.smithi007.stdout:0/649: rmdir d6 39 2024-09-21T04:35:31.168 INFO:tasks.workunit.client.0.smithi007.stdout:1/780: unlink d5/d6/d21/d2b/d4b/da6/d58/c91 0 2024-09-21T04:35:31.172 INFO:tasks.workunit.client.0.smithi007.stdout:1/781: fsync d5/f90 0 2024-09-21T04:35:31.174 INFO:tasks.workunit.client.0.smithi007.stdout:2/474: rename d0/d2/l58 to d0/l80 0 2024-09-21T04:35:31.174 INFO:tasks.workunit.client.0.smithi007.stdout:8/403: dread d0/d3/d8/f58 [4194304,4194304] 0 2024-09-21T04:35:31.175 INFO:tasks.workunit.client.0.smithi007.stdout:1/782: fsync d5/d6/d21/d2b/d4b/da6/d58/f67 0 2024-09-21T04:35:31.176 INFO:tasks.workunit.client.0.smithi007.stdout:0/650: mkdir d6/d75/dc5/d47/d8a/df4 0 2024-09-21T04:35:31.178 INFO:tasks.workunit.client.0.smithi007.stdout:3/710: dread dc/d18/f67 [0,4194304] 0 2024-09-21T04:35:31.178 INFO:tasks.workunit.client.0.smithi007.stdout:0/651: chown d6/d75/dc5/d18/d27/f35 105298 1 2024-09-21T04:35:31.179 INFO:tasks.workunit.client.0.smithi007.stdout:3/711: write dc/d18/d41/f4a [2918279,122986] 0 2024-09-21T04:35:31.182 INFO:tasks.workunit.client.0.smithi007.stdout:5/721: rename d2/d27/d42/da3/d23/fc5 to d2/d27/d42/d53/fe6 0 2024-09-21T04:35:31.184 INFO:tasks.workunit.client.0.smithi007.stdout:5/722: fdatasync d2/d27/d42/da3/fd8 0 2024-09-21T04:35:31.185 INFO:tasks.workunit.client.0.smithi007.stdout:1/783: mknod d5/d6/d21/d2b/d4b/da6/cfa 0 2024-09-21T04:35:31.185 INFO:tasks.workunit.client.0.smithi007.stdout:0/652: mkdir d6/df5 0 2024-09-21T04:35:31.187 INFO:tasks.workunit.client.0.smithi007.stdout:1/784: truncate d5/d6/d21/d3c/dda/fe5 201694 0 2024-09-21T04:35:31.188 INFO:tasks.workunit.client.0.smithi007.stdout:1/785: readlink d5/d6/d21/dd4/leb 0 2024-09-21T04:35:31.192 INFO:tasks.workunit.client.0.smithi007.stdout:1/786: stat d5/d6/d21/d2b/d4b/da6/d58/dcf 0 2024-09-21T04:35:31.194 INFO:tasks.workunit.client.0.smithi007.stdout:3/712: dread - dc/d22/dc8/fd2 zero size 2024-09-21T04:35:31.196 INFO:tasks.workunit.client.0.smithi007.stdout:7/585: rename d3/f5 to d3/d6/d85/dad/fce 0 2024-09-21T04:35:31.196 INFO:tasks.workunit.client.0.smithi007.stdout:9/596: dwrite f10 [0,4194304] 0 2024-09-21T04:35:31.197 INFO:tasks.workunit.client.0.smithi007.stdout:1/787: chown d5/d6/d21/f47 349384 1 2024-09-21T04:35:31.200 INFO:tasks.workunit.client.0.smithi007.stdout:4/616: dwrite d2/f43 [0,4194304] 0 2024-09-21T04:35:31.200 INFO:tasks.workunit.client.0.smithi007.stdout:1/788: chown d5/d6/d21/d2b/f5c 2973612 1 2024-09-21T04:35:31.202 INFO:tasks.workunit.client.0.smithi007.stdout:3/713: write dc/d22/d29/d32/d69/d6d/fb3 [1350825,109925] 0 2024-09-21T04:35:31.202 INFO:tasks.workunit.client.0.smithi007.stdout:9/597: write d13/d18/d60/d83/da9/f2d [1854848,58627] 0 2024-09-21T04:35:31.202 INFO:tasks.workunit.client.0.smithi007.stdout:0/653: creat d6/d75/dc9/ff6 x:0 0 0 2024-09-21T04:35:31.208 INFO:tasks.workunit.client.0.smithi007.stdout:8/404: creat d0/f6f x:0 0 0 2024-09-21T04:35:31.214 INFO:tasks.workunit.client.0.smithi007.stdout:2/475: write d0/f1c [1358811,106118] 0 2024-09-21T04:35:31.218 INFO:tasks.workunit.client.0.smithi007.stdout:7/586: dread d3/d6/f7 [4194304,4194304] 0 2024-09-21T04:35:31.219 INFO:tasks.workunit.client.0.smithi007.stdout:4/617: read d2/f6a [411316,23657] 0 2024-09-21T04:35:31.219 INFO:tasks.workunit.client.0.smithi007.stdout:0/654: dwrite d6/d75/dc5/d18/d2e/d60/f5f [0,4194304] 0 2024-09-21T04:35:31.221 INFO:tasks.workunit.client.0.smithi007.stdout:5/723: write d2/d27/d42/da3/d23/d49/d6f/f7f [3846226,3205] 0 2024-09-21T04:35:31.223 INFO:tasks.workunit.client.0.smithi007.stdout:8/405: dwrite d0/d4e/f66 [0,4194304] 0 2024-09-21T04:35:31.224 INFO:tasks.workunit.client.0.smithi007.stdout:1/789: symlink d5/d6/d21/d64/df1/ddd/lfb 0 2024-09-21T04:35:31.232 INFO:tasks.workunit.client.0.smithi007.stdout:5/724: chown d2/d27/d42/da3/d23/d2a/f75 3 1 2024-09-21T04:35:31.234 INFO:tasks.workunit.client.0.smithi007.stdout:9/598: getdents d13/d18/daf 0 2024-09-21T04:35:31.235 INFO:tasks.workunit.client.0.smithi007.stdout:7/587: dwrite d3/d6/d85/f50 [0,4194304] 0 2024-09-21T04:35:31.236 INFO:tasks.workunit.client.0.smithi007.stdout:0/655: rename d6/d75/dc5/d18/d39/f3b to d6/d75/dc5/d18/d27/ff7 0 2024-09-21T04:35:31.238 INFO:tasks.workunit.client.0.smithi007.stdout:1/790: mkdir d5/d6/d21/d3c/dd5/dea/dfc 0 2024-09-21T04:35:31.242 INFO:tasks.workunit.client.0.smithi007.stdout:7/588: dread - d3/d6/d40/f48 zero size 2024-09-21T04:35:31.242 INFO:tasks.workunit.client.0.smithi007.stdout:1/791: chown d5/d6/d21/d2b/f6f 30 1 2024-09-21T04:35:31.242 INFO:tasks.workunit.client.0.smithi007.stdout:9/599: creat d13/d89/fb9 x:0 0 0 2024-09-21T04:35:31.245 INFO:tasks.workunit.client.0.smithi007.stdout:5/725: read d2/d27/d42/da3/d1d/f4b [244284,86854] 0 2024-09-21T04:35:31.247 INFO:tasks.workunit.client.0.smithi007.stdout:4/618: write d2/f81 [348128,7951] 0 2024-09-21T04:35:31.247 INFO:tasks.workunit.client.0.smithi007.stdout:3/714: write dc/d22/d5a/f90 [2591107,92042] 0 2024-09-21T04:35:31.248 INFO:tasks.workunit.client.0.smithi007.stdout:0/656: rename c2 to d6/d75/dc5/d47/d99/cf8 0 2024-09-21T04:35:31.249 INFO:tasks.workunit.client.0.smithi007.stdout:1/792: unlink d5/d6/d21/d64/df1/la1 0 2024-09-21T04:35:31.249 INFO:tasks.workunit.client.0.smithi007.stdout:8/406: dwrite d0/d3/d8/f67 [0,4194304] 0 2024-09-21T04:35:31.251 INFO:tasks.workunit.client.0.smithi007.stdout:9/600: truncate d13/d74/f98 259896 0 2024-09-21T04:35:31.252 INFO:tasks.workunit.client.0.smithi007.stdout:7/589: symlink d3/d6/d14/d20/d80/lcf 0 2024-09-21T04:35:31.255 INFO:tasks.workunit.client.0.smithi007.stdout:4/619: rmdir d2/d17/d2e/d9f 39 2024-09-21T04:35:31.259 INFO:tasks.workunit.client.0.smithi007.stdout:2/476: dread d0/f37 [0,4194304] 0 2024-09-21T04:35:31.259 INFO:tasks.workunit.client.0.smithi007.stdout:1/793: symlink d5/d6/d21/d64/df1/da0/d99/lfd 0 2024-09-21T04:35:31.264 INFO:tasks.workunit.client.0.smithi007.stdout:3/715: dread dc/d18/f65 [0,4194304] 0 2024-09-21T04:35:31.265 INFO:tasks.workunit.client.0.smithi007.stdout:0/657: dwrite d6/d75/dc5/d52/fb8 [0,4194304] 0 2024-09-21T04:35:31.265 INFO:tasks.workunit.client.0.smithi007.stdout:9/601: rename d13/d74/c7b to d13/d18/daf/cba 0 2024-09-21T04:35:31.269 INFO:tasks.workunit.client.0.smithi007.stdout:4/620: rmdir d2/d88/da8 39 2024-09-21T04:35:31.275 INFO:tasks.workunit.client.0.smithi007.stdout:1/794: mknod d5/d6/d21/cfe 0 2024-09-21T04:35:31.277 INFO:tasks.workunit.client.0.smithi007.stdout:1/795: write d5/d6/d21/d2b/d4b/da6/d58/fac [1001817,21694] 0 2024-09-21T04:35:31.278 INFO:tasks.workunit.client.0.smithi007.stdout:1/796: dread - d5/d6/d21/d64/df1/ddd/ff6 zero size 2024-09-21T04:35:31.281 INFO:tasks.workunit.client.0.smithi007.stdout:7/590: dread d3/d6/d14/d20/d80/db2/fb1 [0,4194304] 0 2024-09-21T04:35:31.285 INFO:tasks.workunit.client.0.smithi007.stdout:9/602: symlink d13/d18/d60/lbb 0 2024-09-21T04:35:31.289 INFO:tasks.workunit.client.0.smithi007.stdout:4/621: mkdir d2/d17/d18/dcd 0 2024-09-21T04:35:31.292 INFO:tasks.workunit.client.0.smithi007.stdout:1/797: creat d5/d6/d21/d2b/d4b/d8a/fff x:0 0 0 2024-09-21T04:35:31.294 INFO:tasks.workunit.client.0.smithi007.stdout:0/658: write d6/d75/dc5/d18/d27/fd3 [982234,31038] 0 2024-09-21T04:35:31.296 INFO:tasks.workunit.client.0.smithi007.stdout:2/477: mkdir d0/d11/d31/d46/d6d/d81 0 2024-09-21T04:35:31.297 INFO:tasks.workunit.client.0.smithi007.stdout:7/591: creat d3/d6/d2e/d63/d6a/d81/fd0 x:0 0 0 2024-09-21T04:35:31.298 INFO:tasks.workunit.client.0.smithi007.stdout:3/716: dwrite dc/d18/d26/f68 [0,4194304] 0 2024-09-21T04:35:31.298 INFO:tasks.workunit.client.0.smithi007.stdout:0/659: read d6/d75/dc5/fba [464681,45105] 0 2024-09-21T04:35:31.298 INFO:tasks.workunit.client.0.smithi007.stdout:1/798: read d5/d6/d21/d2b/d4b/da6/d58/f67 [2234248,119083] 0 2024-09-21T04:35:31.300 INFO:tasks.workunit.client.0.smithi007.stdout:9/603: chown d13/d18/d60/d6d/c9b 12 1 2024-09-21T04:35:31.309 INFO:tasks.workunit.client.0.smithi007.stdout:4/622: creat d2/d17/d2e/d45/d5d/d8a/fce x:0 0 0 2024-09-21T04:35:31.313 INFO:tasks.workunit.client.0.smithi007.stdout:2/478: creat d0/d11/d31/d46/d48/f82 x:0 0 0 2024-09-21T04:35:31.315 INFO:tasks.workunit.client.0.smithi007.stdout:7/592: dwrite d3/d6/d2e/d63/f73 [0,4194304] 0 2024-09-21T04:35:31.317 INFO:tasks.workunit.client.0.smithi007.stdout:7/593: write d3/d6/d10/d62/d66/f94 [247200,2280] 0 2024-09-21T04:35:31.318 INFO:tasks.workunit.client.0.smithi007.stdout:7/594: readlink d3/d6/d10/l3a 0 2024-09-21T04:35:31.321 INFO:tasks.workunit.client.0.smithi007.stdout:2/479: dwrite d0/d11/d31/d39/f4f [4194304,4194304] 0 2024-09-21T04:35:31.324 INFO:tasks.workunit.client.0.smithi007.stdout:0/660: creat d6/d75/ff9 x:0 0 0 2024-09-21T04:35:31.325 INFO:tasks.workunit.client.0.smithi007.stdout:9/604: dread - d13/d18/f27 zero size 2024-09-21T04:35:31.328 INFO:tasks.workunit.client.0.smithi007.stdout:0/661: chown d6/d75/dc5/fa4 116672697 1 2024-09-21T04:35:31.332 INFO:tasks.workunit.client.0.smithi007.stdout:1/799: unlink d5/d6/d21/d64/df1/c8c 0 2024-09-21T04:35:31.340 INFO:tasks.workunit.client.0.smithi007.stdout:1/800: dwrite d5/d6/d21/d64/df1/ddd/ff6 [0,4194304] 0 2024-09-21T04:35:31.347 INFO:tasks.workunit.client.0.smithi007.stdout:3/717: mkdir dc/d18/d41/de1 0 2024-09-21T04:35:31.348 INFO:tasks.workunit.client.0.smithi007.stdout:3/718: truncate dc/d22/d5a/fd8 544397 0 2024-09-21T04:35:31.351 INFO:tasks.workunit.client.0.smithi007.stdout:2/480: rename d0/d11/d31/d52/f65 to d0/d11/d31/d46/d6c/f83 0 2024-09-21T04:35:31.354 INFO:tasks.workunit.client.0.smithi007.stdout:9/605: mkdir d13/d18/daf/dbc 0 2024-09-21T04:35:31.357 INFO:tasks.workunit.client.0.smithi007.stdout:1/801: stat d5/d2f/c37 0 2024-09-21T04:35:31.360 INFO:tasks.workunit.client.0.smithi007.stdout:3/719: symlink dc/d22/d29/d32/d4d/d84/dd0/da6/d80/le2 0 2024-09-21T04:35:31.364 INFO:tasks.workunit.client.0.smithi007.stdout:2/481: creat d0/d11/d31/d39/f84 x:0 0 0 2024-09-21T04:35:31.365 INFO:tasks.workunit.client.0.smithi007.stdout:0/662: mknod d6/d75/dc5/d47/cfa 0 2024-09-21T04:35:31.367 INFO:tasks.workunit.client.0.smithi007.stdout:9/606: rename d13/f1d to d13/d18/d60/d83/fbd 0 2024-09-21T04:35:31.368 INFO:tasks.workunit.client.0.smithi007.stdout:9/607: chown d13/f4f 5744426 1 2024-09-21T04:35:31.369 INFO:tasks.workunit.client.0.smithi007.stdout:1/802: rmdir d5/d6/d21/d64/df1/ddd 39 2024-09-21T04:35:31.370 INFO:tasks.workunit.client.0.smithi007.stdout:4/623: creat d2/d17/fcf x:0 0 0 2024-09-21T04:35:31.375 INFO:tasks.workunit.client.0.smithi007.stdout:2/482: unlink d0/d11/d31/d46/d48/f54 0 2024-09-21T04:35:31.428 INFO:tasks.workunit.client.0.smithi007.stdout:4/624: dwrite d2/d7/d55/d61/f9a [0,4194304] 0 2024-09-21T04:35:31.428 INFO:tasks.workunit.client.0.smithi007.stdout:9/608: rmdir d13/d74 39 2024-09-21T04:35:31.429 INFO:tasks.workunit.client.0.smithi007.stdout:0/663: read d6/d75/dc5/d18/d27/f97 [828340,57852] 0 2024-09-21T04:35:31.430 INFO:tasks.workunit.client.0.smithi007.stdout:2/483: creat d0/d2/f85 x:0 0 0 2024-09-21T04:35:31.435 INFO:tasks.workunit.client.0.smithi007.stdout:0/664: fsync d6/d75/dc5/d18/d27/ff7 0 2024-09-21T04:35:31.440 INFO:tasks.workunit.client.0.smithi007.stdout:2/484: dwrite d0/d11/d31/d39/f50 [0,4194304] 0 2024-09-21T04:35:31.440 INFO:tasks.workunit.client.0.smithi007.stdout:9/609: dwrite d13/d18/f5c [0,4194304] 0 2024-09-21T04:35:31.442 INFO:tasks.workunit.client.0.smithi007.stdout:2/485: write d0/d2/f25 [503427,48756] 0 2024-09-21T04:35:31.443 INFO:tasks.workunit.client.0.smithi007.stdout:4/625: symlink d2/d3c/ld0 0 2024-09-21T04:35:31.445 INFO:tasks.workunit.client.0.smithi007.stdout:0/665: read d6/d75/dc5/f28 [1114852,77163] 0 2024-09-21T04:35:31.448 INFO:tasks.workunit.client.0.smithi007.stdout:2/486: rename d0/d11/d31/d46/f7c to d0/d11/d6e/d73/f86 0 2024-09-21T04:35:31.454 INFO:tasks.workunit.client.0.smithi007.stdout:9/610: rename d13/d74/c9a to d13/d18/d80/cbe 0 2024-09-21T04:35:31.460 INFO:tasks.workunit.client.0.smithi007.stdout:4/626: dread d2/d7/d55/f80 [0,4194304] 0 2024-09-21T04:35:31.466 INFO:tasks.workunit.client.0.smithi007.stdout:4/627: creat d2/d17/fd1 x:0 0 0 2024-09-21T04:35:31.469 INFO:tasks.workunit.client.0.smithi007.stdout:4/628: mkdir d2/d17/d2e/d45/d5d/d8a/dd2 0 2024-09-21T04:35:31.471 INFO:tasks.workunit.client.0.smithi007.stdout:4/629: write d2/d17/d2e/d45/d5e/faa [279926,86810] 0 2024-09-21T04:35:31.476 INFO:tasks.workunit.client.0.smithi007.stdout:3/720: rename dc/d22/d29/d32/d4d/d84/dd0/da6/d80 to dc/d22/d29/d32/d4d/d84/dd0/da6/de3 0 2024-09-21T04:35:31.480 INFO:tasks.workunit.client.0.smithi007.stdout:4/630: unlink d2/d17/cbb 0 2024-09-21T04:35:31.483 INFO:tasks.workunit.client.0.smithi007.stdout:7/595: truncate d3/d6/d2e/d63/d6a/d81/f8c 4106974 0 2024-09-21T04:35:31.485 INFO:tasks.workunit.client.0.smithi007.stdout:4/631: creat d2/d17/d2e/d45/d5d/d8a/fd3 x:0 0 0 2024-09-21T04:35:31.490 INFO:tasks.workunit.client.0.smithi007.stdout:3/721: dwrite dc/d22/fc0 [0,4194304] 0 2024-09-21T04:35:31.491 INFO:tasks.workunit.client.0.smithi007.stdout:7/596: fdatasync d3/d6/d13/f5d 0 2024-09-21T04:35:31.495 INFO:tasks.workunit.client.0.smithi007.stdout:3/722: mkdir dc/d22/d29/d32/d69/de4 0 2024-09-21T04:35:31.495 INFO:tasks.workunit.client.0.smithi007.stdout:3/723: readlink dc/l20 0 2024-09-21T04:35:31.498 INFO:tasks.workunit.client.0.smithi007.stdout:7/597: creat d3/d6/d10/d51/fd1 x:0 0 0 2024-09-21T04:35:31.499 INFO:tasks.workunit.client.0.smithi007.stdout:4/632: getdents d2/d7/dc9 0 2024-09-21T04:35:31.500 INFO:tasks.workunit.client.0.smithi007.stdout:4/633: dread - d2/d3c/dbe/fc3 zero size 2024-09-21T04:35:31.502 INFO:tasks.workunit.client.0.smithi007.stdout:7/598: mkdir d3/d6/d85/dd2 0 2024-09-21T04:35:31.503 INFO:tasks.workunit.client.0.smithi007.stdout:4/634: symlink d2/d3c/dbe/ld4 0 2024-09-21T04:35:31.507 INFO:tasks.workunit.client.0.smithi007.stdout:7/599: mknod d3/d6/d10/d51/cd3 0 2024-09-21T04:35:31.508 INFO:tasks.workunit.client.0.smithi007.stdout:4/635: creat d2/d17/d2e/d45/d5d/d8a/fd5 x:0 0 0 2024-09-21T04:35:31.516 INFO:tasks.workunit.client.0.smithi007.stdout:3/724: dread dc/d22/d29/d32/d4d/f79 [0,4194304] 0 2024-09-21T04:35:31.517 INFO:tasks.workunit.client.0.smithi007.stdout:4/636: mkdir d2/d7/dc5/dd6 0 2024-09-21T04:35:31.518 INFO:tasks.workunit.client.0.smithi007.stdout:3/725: chown dc/d22/d29/d32/d4d/d84/dd0/da6/l72 3 1 2024-09-21T04:35:31.519 INFO:tasks.workunit.client.0.smithi007.stdout:3/726: stat dc/d22/d29/d32 0 2024-09-21T04:35:31.520 INFO:tasks.workunit.client.0.smithi007.stdout:3/727: fdatasync dc/d22/d5a/fd8 0 2024-09-21T04:35:31.521 INFO:tasks.workunit.client.0.smithi007.stdout:1/803: write d5/d6/d21/d64/df1/f7b [603815,87534] 0 2024-09-21T04:35:31.529 INFO:tasks.workunit.client.0.smithi007.stdout:4/637: creat d2/d7/dc5/dd6/fd7 x:0 0 0 2024-09-21T04:35:31.535 INFO:tasks.workunit.client.0.smithi007.stdout:4/638: truncate d2/d17/d2e/f71 3324726 0 2024-09-21T04:35:31.538 INFO:tasks.workunit.client.0.smithi007.stdout:1/804: dread d5/d6/d21/d2b/f2d [0,4194304] 0 2024-09-21T04:35:31.538 INFO:tasks.workunit.client.0.smithi007.stdout:4/639: creat d2/d7/dc5/fd8 x:0 0 0 2024-09-21T04:35:31.544 INFO:tasks.workunit.client.0.smithi007.stdout:1/805: creat d5/d6/f100 x:0 0 0 2024-09-21T04:35:31.545 INFO:tasks.workunit.client.0.smithi007.stdout:4/640: dwrite d2/f43 [0,4194304] 0 2024-09-21T04:35:31.553 INFO:tasks.workunit.client.0.smithi007.stdout:4/641: creat d2/d17/d2e/d45/d5d/d8a/fd9 x:0 0 0 2024-09-21T04:35:31.554 INFO:tasks.workunit.client.0.smithi007.stdout:2/487: truncate d0/fd 2142014 0 2024-09-21T04:35:31.559 INFO:tasks.workunit.client.0.smithi007.stdout:0/666: dwrite d6/d75/dc5/d18/d2e/f56 [4194304,4194304] 0 2024-09-21T04:35:31.561 INFO:tasks.workunit.client.0.smithi007.stdout:2/488: truncate d0/d11/d31/d46/d48/f82 277188 0 2024-09-21T04:35:31.562 INFO:tasks.workunit.client.0.smithi007.stdout:0/667: readlink d6/d75/dc5/d18/d27/l36 0 2024-09-21T04:35:31.563 INFO:tasks.workunit.client.0.smithi007.stdout:9/611: dwrite d13/d18/d60/d6d/d75/f7e [0,4194304] 0 2024-09-21T04:35:31.568 INFO:tasks.workunit.client.0.smithi007.stdout:1/806: dread d5/d2f/f95 [0,4194304] 0 2024-09-21T04:35:31.575 INFO:tasks.workunit.client.0.smithi007.stdout:0/668: rename d6/d75/dc5/d18/c7a to d6/d75/dc5/d18/d2e/d3f/cfb 0 2024-09-21T04:35:31.576 INFO:tasks.workunit.client.0.smithi007.stdout:1/807: creat d5/d6/d21/d2b/d4b/da6/d58/f101 x:0 0 0 2024-09-21T04:35:31.579 INFO:tasks.workunit.client.0.smithi007.stdout:1/808: stat d5/d6/d21/d2b/lc6 0 2024-09-21T04:35:31.579 INFO:tasks.workunit.client.0.smithi007.stdout:2/489: dwrite d0/d11/d31/d46/d6c/f83 [0,4194304] 0 2024-09-21T04:35:31.581 INFO:tasks.workunit.client.0.smithi007.stdout:7/600: truncate d3/f88 2797636 0 2024-09-21T04:35:31.585 INFO:tasks.workunit.client.0.smithi007.stdout:0/669: unlink d6/d75/dc5/d18/d39/c79 0 2024-09-21T04:35:31.590 INFO:tasks.workunit.client.0.smithi007.stdout:2/490: creat d0/d11/d31/d52/f87 x:0 0 0 2024-09-21T04:35:31.591 INFO:tasks.workunit.client.0.smithi007.stdout:1/809: truncate d5/d6/d21/d2b/d4b/f66 120550 0 2024-09-21T04:35:31.591 INFO:tasks.workunit.client.0.smithi007.stdout:2/491: chown d0/d11/d6b 20611 1 2024-09-21T04:35:31.591 INFO:tasks.workunit.client.0.smithi007.stdout:3/728: write dc/d22/d29/d32/d69/d6d/f9b [24830,79339] 0 2024-09-21T04:35:31.593 INFO:tasks.workunit.client.0.smithi007.stdout:4/642: getdents d2/d7/dc5 0 2024-09-21T04:35:31.594 INFO:tasks.workunit.client.0.smithi007.stdout:4/643: chown d2/d17/fd1 3745447 1 2024-09-21T04:35:31.595 INFO:tasks.workunit.client.0.smithi007.stdout:4/644: dread - d2/d17/fcf zero size 2024-09-21T04:35:31.596 INFO:tasks.workunit.client.0.smithi007.stdout:0/670: rename d6/d75/dc5/d18/d2e/f56 to d6/d91/da3/dbf/dec/ffc 0 2024-09-21T04:35:31.596 INFO:tasks.workunit.client.0.smithi007.stdout:4/645: chown d2/d7/f24 2516811 1 2024-09-21T04:35:31.597 INFO:tasks.workunit.client.0.smithi007.stdout:7/601: mknod d3/d6/d40/da1/cd4 0 2024-09-21T04:35:31.598 INFO:tasks.workunit.client.0.smithi007.stdout:0/671: chown d6/d75/dc5/d18/d2e/d3f/fe6 121067 1 2024-09-21T04:35:31.598 INFO:tasks.workunit.client.0.smithi007.stdout:7/602: dread - d3/d6/d10/d4f/fbe zero size 2024-09-21T04:35:31.605 INFO:tasks.workunit.client.0.smithi007.stdout:2/492: dread d0/d32/f34 [0,4194304] 0 2024-09-21T04:35:31.606 INFO:tasks.workunit.client.0.smithi007.stdout:2/493: chown d0/d11/d31/d46/d6c/f83 3622 1 2024-09-21T04:35:31.608 INFO:tasks.workunit.client.0.smithi007.stdout:2/494: write d0/d11/d31/d46/f47 [3519919,81778] 0 2024-09-21T04:35:31.613 INFO:tasks.workunit.client.0.smithi007.stdout:1/810: dread d5/d6/fa9 [4194304,4194304] 0 2024-09-21T04:35:31.614 INFO:tasks.workunit.client.0.smithi007.stdout:9/612: write d13/f21 [5626581,75659] 0 2024-09-21T04:35:31.614 INFO:tasks.workunit.client.0.smithi007.stdout:7/603: read d3/d6/d2e/d5e/f93 [108393,111268] 0 2024-09-21T04:35:31.618 INFO:tasks.workunit.client.0.smithi007.stdout:0/672: rename d6/d75/dc5/d18/d2e/fed to d6/d75/ffd 0 2024-09-21T04:35:31.629 INFO:tasks.workunit.client.0.smithi007.stdout:6/613: sync 2024-09-21T04:35:31.630 INFO:tasks.workunit.client.0.smithi007.stdout:0/673: dread d6/d91/da3/dbf/fc3 [0,4194304] 0 2024-09-21T04:35:31.630 INFO:tasks.workunit.client.0.smithi007.stdout:2/495: fsync d0/d32/f34 0 2024-09-21T04:35:31.719 INFO:tasks.workunit.client.0.smithi007.stdout:0/674: rmdir d6/d75/dc5/d52 39 2024-09-21T04:35:31.721 INFO:tasks.workunit.client.0.smithi007.stdout:3/729: truncate dc/d22/f24 2726005 0 2024-09-21T04:35:31.722 INFO:tasks.workunit.client.0.smithi007.stdout:7/604: creat d3/d6/d85/dd2/fd5 x:0 0 0 2024-09-21T04:35:31.724 INFO:tasks.workunit.client.0.smithi007.stdout:7/605: fdatasync d3/d6/d10/d4f/fbc 0 2024-09-21T04:35:31.727 INFO:tasks.workunit.client.0.smithi007.stdout:1/811: truncate d5/d6/d21/d64/df1/f78 3841104 0 2024-09-21T04:35:31.732 INFO:tasks.workunit.client.0.smithi007.stdout:6/614: mknod d3/de/cce 0 2024-09-21T04:35:31.734 INFO:tasks.workunit.client.0.smithi007.stdout:6/615: write d3/d3e/d46/d60/d77/dcb/d78/fb0 [1917673,112166] 0 2024-09-21T04:35:31.738 INFO:tasks.workunit.client.0.smithi007.stdout:0/675: unlink d6/d75/dc5/d57/c68 0 2024-09-21T04:35:31.741 INFO:tasks.workunit.client.0.smithi007.stdout:9/613: rmdir d13/d18/d60/d6d/d75/d9f/da4 0 2024-09-21T04:35:31.742 INFO:tasks.workunit.client.0.smithi007.stdout:1/812: dread d5/f26 [0,4194304] 0 2024-09-21T04:35:31.742 INFO:tasks.workunit.client.0.smithi007.stdout:4/646: truncate d2/d17/d2e/f71 4274001 0 2024-09-21T04:35:31.742 INFO:tasks.workunit.client.0.smithi007.stdout:9/614: chown d13/d18/d60/d6d/fb2 0 1 2024-09-21T04:35:31.743 INFO:tasks.workunit.client.0.smithi007.stdout:1/813: write d5/d6/d21/d2b/f63 [637974,80279] 0 2024-09-21T04:35:31.744 INFO:tasks.workunit.client.0.smithi007.stdout:7/606: creat d3/d6/d10/d62/d66/fd6 x:0 0 0 2024-09-21T04:35:31.749 INFO:tasks.workunit.client.0.smithi007.stdout:6/616: symlink d3/d3e/d46/d60/d77/dcb/d88/lcf 0 2024-09-21T04:35:31.758 INFO:tasks.workunit.client.0.smithi007.stdout:4/647: mknod d2/d17/d2e/d45/d5d/cda 0 2024-09-21T04:35:31.759 INFO:tasks.workunit.client.0.smithi007.stdout:4/648: chown d2/d17/d2e/d45/d5d/d8a/lc4 1587 1 2024-09-21T04:35:31.761 INFO:tasks.workunit.client.0.smithi007.stdout:6/617: fsync d3/de/d58/f90 0 2024-09-21T04:35:31.762 INFO:tasks.workunit.client.0.smithi007.stdout:2/496: link d0/f23 d0/d2/f88 0 2024-09-21T04:35:31.763 INFO:tasks.workunit.client.0.smithi007.stdout:3/730: creat dc/d22/d29/d32/fe5 x:0 0 0 2024-09-21T04:35:31.763 INFO:tasks.workunit.client.0.smithi007.stdout:2/497: chown d0/d11/d6e/d73/f86 124672 1 2024-09-21T04:35:31.763 INFO:tasks.workunit.client.0.smithi007.stdout:0/676: symlink d6/d75/dc5/d47/d8a/df4/lfe 0 2024-09-21T04:35:31.764 INFO:tasks.workunit.client.0.smithi007.stdout:2/498: dread - d0/d11/d6e/f7e zero size 2024-09-21T04:35:31.766 INFO:tasks.workunit.client.0.smithi007.stdout:6/618: symlink d3/d3e/d46/d60/d98/ld0 0 2024-09-21T04:35:31.769 INFO:tasks.workunit.client.0.smithi007.stdout:3/731: dread dc/d18/d41/fa5 [0,4194304] 0 2024-09-21T04:35:31.771 INFO:tasks.workunit.client.0.smithi007.stdout:9/615: creat d13/d18/d60/d6d/d75/fbf x:0 0 0 2024-09-21T04:35:31.772 INFO:tasks.workunit.client.0.smithi007.stdout:3/732: fsync dc/d18/f7f 0 2024-09-21T04:35:31.773 INFO:tasks.workunit.client.0.smithi007.stdout:2/499: dwrite d0/d2/f2b [0,4194304] 0 2024-09-21T04:35:31.773 INFO:tasks.workunit.client.0.smithi007.stdout:7/607: creat d3/d6/d40/fd7 x:0 0 0 2024-09-21T04:35:31.776 INFO:tasks.workunit.client.0.smithi007.stdout:0/677: unlink d6/d75/dc5/d18/d2e/d60/l44 0 2024-09-21T04:35:31.776 INFO:tasks.workunit.client.0.smithi007.stdout:2/500: fdatasync d0/d11/d31/d46/f47 0 2024-09-21T04:35:31.777 INFO:tasks.workunit.client.0.smithi007.stdout:4/649: dwrite d2/f23 [4194304,4194304] 0 2024-09-21T04:35:31.780 INFO:tasks.workunit.client.0.smithi007.stdout:3/733: truncate dc/d22/d5a/f75 643938 0 2024-09-21T04:35:31.785 INFO:tasks.workunit.client.0.smithi007.stdout:6/619: creat d3/d3e/fd1 x:0 0 0 2024-09-21T04:35:31.786 INFO:tasks.workunit.client.0.smithi007.stdout:7/608: mknod d3/d6/d2e/d63/d6a/cd8 0 2024-09-21T04:35:31.792 INFO:tasks.workunit.client.0.smithi007.stdout:2/501: getdents d0/d11/d31 0 2024-09-21T04:35:31.795 INFO:tasks.workunit.client.0.smithi007.stdout:2/502: chown d0/f7d 1636778 1 2024-09-21T04:35:31.796 INFO:tasks.workunit.client.0.smithi007.stdout:3/734: dwrite dc/d22/d5a/f75 [0,4194304] 0 2024-09-21T04:35:31.797 INFO:tasks.workunit.client.0.smithi007.stdout:4/650: dwrite d2/d17/d2e/d45/d5d/d8a/fd9 [0,4194304] 0 2024-09-21T04:35:31.798 INFO:tasks.workunit.client.0.smithi007.stdout:6/620: write d3/de/d12/d19/f35 [3704059,50242] 0 2024-09-21T04:35:31.801 INFO:tasks.workunit.client.0.smithi007.stdout:3/735: readlink dc/d22/d29/d32/d4d/d84/dd0/ldc 0 2024-09-21T04:35:31.801 INFO:tasks.workunit.client.0.smithi007.stdout:9/616: dread d13/d18/f1a [0,4194304] 0 2024-09-21T04:35:31.806 INFO:tasks.workunit.client.0.smithi007.stdout:3/736: rename dc/d18/d41/de1 to dc/d22/d29/de6 0 2024-09-21T04:35:31.814 INFO:tasks.workunit.client.0.smithi007.stdout:9/617: getdents d13/d18/daf/dbc 0 2024-09-21T04:35:31.820 INFO:tasks.workunit.client.0.smithi007.stdout:4/651: getdents d2/d5f 0 2024-09-21T04:35:31.822 INFO:tasks.workunit.client.0.smithi007.stdout:9/618: dwrite d13/d18/f55 [0,4194304] 0 2024-09-21T04:35:31.822 INFO:tasks.workunit.client.0.smithi007.stdout:4/652: fdatasync d2/d17/d2e/d45/d5e/f9c 0 2024-09-21T04:35:31.828 INFO:tasks.workunit.client.0.smithi007.stdout:7/609: truncate d3/d6/d14/d20/d80/db2/fb1 540513 0 2024-09-21T04:35:31.829 INFO:tasks.workunit.client.0.smithi007.stdout:9/619: mkdir d13/d18/daf/dc0 0 2024-09-21T04:35:31.831 INFO:tasks.workunit.client.0.smithi007.stdout:4/653: creat d2/d7/d64/fdb x:0 0 0 2024-09-21T04:35:31.831 INFO:tasks.workunit.client.0.smithi007.stdout:3/737: write dc/d22/d29/d32/d4d/f79 [2360407,39348] 0 2024-09-21T04:35:31.833 INFO:tasks.workunit.client.0.smithi007.stdout:4/654: dread - d2/d7/d55/d4f/fbc zero size 2024-09-21T04:35:31.833 INFO:tasks.workunit.client.0.smithi007.stdout:6/621: getdents d3/d3e/d99/dbb 0 2024-09-21T04:35:31.834 INFO:tasks.workunit.client.0.smithi007.stdout:3/738: write dc/d18/d26/f5f [1404355,51343] 0 2024-09-21T04:35:31.834 INFO:tasks.workunit.client.0.smithi007.stdout:3/739: chown dc/d56 1 1 2024-09-21T04:35:31.836 INFO:tasks.workunit.client.0.smithi007.stdout:7/610: unlink d3/d6/d10/d62/d66/fd6 0 2024-09-21T04:35:31.842 INFO:tasks.workunit.client.0.smithi007.stdout:3/740: rmdir dc/d22/d29/d32/d4f 39 2024-09-21T04:35:31.843 INFO:tasks.workunit.client.0.smithi007.stdout:7/611: symlink d3/d6/d10/ld9 0 2024-09-21T04:35:31.844 INFO:tasks.workunit.client.0.smithi007.stdout:6/622: dwrite d3/d94/f8a [4194304,4194304] 0 2024-09-21T04:35:31.845 INFO:tasks.workunit.client.0.smithi007.stdout:3/741: truncate dc/d22/d29/d32/d69/d6d/f9b 428274 0 2024-09-21T04:35:31.845 INFO:tasks.workunit.client.0.smithi007.stdout:9/620: fsync d13/d18/d60/d83/da9/f3d 0 2024-09-21T04:35:31.848 INFO:tasks.workunit.client.0.smithi007.stdout:3/742: chown dc/d18/f39 598 1 2024-09-21T04:35:31.851 INFO:tasks.workunit.client.0.smithi007.stdout:3/743: chown dc/d18/f7f 906755 1 2024-09-21T04:35:31.854 INFO:tasks.workunit.client.0.smithi007.stdout:9/621: mknod d13/d18/cc1 0 2024-09-21T04:35:31.858 INFO:tasks.workunit.client.0.smithi007.stdout:4/655: dwrite d2/d17/d2e/d45/f4b [0,4194304] 0 2024-09-21T04:35:31.862 INFO:tasks.workunit.client.0.smithi007.stdout:7/612: dwrite d3/d6/f28 [4194304,4194304] 0 2024-09-21T04:35:31.865 INFO:tasks.workunit.client.0.smithi007.stdout:9/622: unlink d13/d18/l6a 0 2024-09-21T04:35:31.869 INFO:tasks.workunit.client.0.smithi007.stdout:4/656: truncate d2/d17/d2e/f7c 976283 0 2024-09-21T04:35:31.871 INFO:tasks.workunit.client.0.smithi007.stdout:9/623: creat d13/d18/d80/fc2 x:0 0 0 2024-09-21T04:35:31.873 INFO:tasks.workunit.client.0.smithi007.stdout:7/613: dwrite d3/d6/d85/dd2/fd5 [0,4194304] 0 2024-09-21T04:35:31.876 INFO:tasks.workunit.client.0.smithi007.stdout:3/744: dread dc/f1c [0,4194304] 0 2024-09-21T04:35:31.876 INFO:tasks.workunit.client.0.smithi007.stdout:4/657: creat d2/d3c/d63/fdc x:0 0 0 2024-09-21T04:35:31.877 INFO:tasks.workunit.client.0.smithi007.stdout:6/623: write d3/de/f49 [4510882,105592] 0 2024-09-21T04:35:31.878 INFO:tasks.workunit.client.0.smithi007.stdout:4/658: chown d2/d7/f24 451995648 1 2024-09-21T04:35:31.882 INFO:tasks.workunit.client.0.smithi007.stdout:9/624: rename d13/d18/c3a to d13/d18/daf/dc0/cc3 0 2024-09-21T04:35:31.883 INFO:tasks.workunit.client.0.smithi007.stdout:3/745: mknod dc/d6c/ce7 0 2024-09-21T04:35:31.891 INFO:tasks.workunit.client.0.smithi007.stdout:7/614: write d3/d6/d10/f7f [777166,119909] 0 2024-09-21T04:35:31.895 INFO:tasks.workunit.client.0.smithi007.stdout:7/615: write d3/d6/d2e/d63/d6a/d81/fd0 [908028,96785] 0 2024-09-21T04:35:31.896 INFO:tasks.workunit.client.0.smithi007.stdout:9/625: symlink d13/d18/daf/dbc/lc4 0 2024-09-21T04:35:31.899 INFO:tasks.workunit.client.0.smithi007.stdout:7/616: chown d3/d6/d13/l6b 739 1 2024-09-21T04:35:31.900 INFO:tasks.workunit.client.0.smithi007.stdout:3/746: creat dc/d22/d8d/fe8 x:0 0 0 2024-09-21T04:35:31.901 INFO:tasks.workunit.client.0.smithi007.stdout:3/747: write dc/d22/d5a/f75 [728438,103264] 0 2024-09-21T04:35:31.902 INFO:tasks.workunit.client.0.smithi007.stdout:3/748: chown dc/d22/fc0 502244 1 2024-09-21T04:35:31.902 INFO:tasks.workunit.client.0.smithi007.stdout:4/659: dread d2/d17/d29/f58 [0,4194304] 0 2024-09-21T04:35:31.903 INFO:tasks.workunit.client.0.smithi007.stdout:9/626: write d13/d18/f1a [5733179,107038] 0 2024-09-21T04:35:31.904 INFO:tasks.workunit.client.0.smithi007.stdout:7/617: creat d3/d6/d40/da1/fda x:0 0 0 2024-09-21T04:35:31.907 INFO:tasks.workunit.client.0.smithi007.stdout:7/618: chown d3/d6/d10/d62/d66/l86 498329132 1 2024-09-21T04:35:31.907 INFO:tasks.workunit.client.0.smithi007.stdout:9/627: mkdir d13/dc5 0 2024-09-21T04:35:31.910 INFO:tasks.workunit.client.0.smithi007.stdout:4/660: link d2/d17/fcf d2/d17/d2e/d45/d5d/d8a/fdd 0 2024-09-21T04:35:31.913 INFO:tasks.workunit.client.0.smithi007.stdout:9/628: dread d13/d18/f6e [0,4194304] 0 2024-09-21T04:35:31.918 INFO:tasks.workunit.client.0.smithi007.stdout:3/749: dread dc/d22/d29/d32/d69/f40 [0,4194304] 0 2024-09-21T04:35:31.919 INFO:tasks.workunit.client.0.smithi007.stdout:3/750: write dc/d22/d5a/f75 [4112522,124158] 0 2024-09-21T04:35:31.927 INFO:tasks.workunit.client.0.smithi007.stdout:4/661: write d2/f6a [1240468,72083] 0 2024-09-21T04:35:31.929 INFO:tasks.workunit.client.0.smithi007.stdout:3/751: rename dc/l62 to dc/d22/d29/d92/db9/le9 0 2024-09-21T04:35:31.929 INFO:tasks.workunit.client.0.smithi007.stdout:7/619: truncate d3/d6/d2e/d63/d6a/d81/f96 2055432 0 2024-09-21T04:35:31.930 INFO:tasks.workunit.client.0.smithi007.stdout:9/629: getdents d13/d74 0 2024-09-21T04:35:31.931 INFO:tasks.workunit.client.0.smithi007.stdout:3/752: write dc/d22/d5a/d71/f9d [3732555,48513] 0 2024-09-21T04:35:31.931 INFO:tasks.workunit.client.0.smithi007.stdout:7/620: write d3/d6/d2e/fbf [1007383,16181] 0 2024-09-21T04:35:31.932 INFO:tasks.workunit.client.0.smithi007.stdout:3/753: chown dc/d6c/da4 40261918 1 2024-09-21T04:35:31.935 INFO:tasks.workunit.client.0.smithi007.stdout:9/630: creat d13/d18/daf/dbc/fc6 x:0 0 0 2024-09-21T04:35:31.935 INFO:tasks.workunit.client.0.smithi007.stdout:3/754: mkdir dc/d18/dea 0 2024-09-21T04:35:31.937 INFO:tasks.workunit.client.0.smithi007.stdout:3/755: write dc/d22/d29/d92/db9/fcc [619651,111462] 0 2024-09-21T04:35:31.938 INFO:tasks.workunit.client.0.smithi007.stdout:4/662: creat d2/d7/d55/fde x:0 0 0 2024-09-21T04:35:31.938 INFO:tasks.workunit.client.0.smithi007.stdout:9/631: rmdir d13 39 2024-09-21T04:35:31.939 INFO:tasks.workunit.client.0.smithi007.stdout:4/663: write d2/d17/d2e/d45/d8e/fb7 [1237210,44718] 0 2024-09-21T04:35:31.942 INFO:tasks.workunit.client.0.smithi007.stdout:9/632: read d13/d18/f45 [4189181,79603] 0 2024-09-21T04:35:31.943 INFO:tasks.workunit.client.0.smithi007.stdout:9/633: fdatasync d13/d18/d60/d83/da9/f9e 0 2024-09-21T04:35:31.946 INFO:tasks.workunit.client.0.smithi007.stdout:3/756: fdatasync dc/d22/d29/d32/fa3 0 2024-09-21T04:35:31.947 INFO:tasks.workunit.client.0.smithi007.stdout:7/621: getdents d3/d6/d13/d4b 0 2024-09-21T04:35:31.948 INFO:tasks.workunit.client.0.smithi007.stdout:3/757: chown dc/d22/d29/d32/d4d/fa0 177418892 1 2024-09-21T04:35:31.955 INFO:tasks.workunit.client.0.smithi007.stdout:7/622: unlink d3/d6/d2e/d5e/f93 0 2024-09-21T04:35:31.955 INFO:tasks.workunit.client.0.smithi007.stdout:9/634: dread d13/d18/f35 [0,4194304] 0 2024-09-21T04:35:31.956 INFO:tasks.workunit.client.0.smithi007.stdout:4/664: mknod d2/d88/da8/db2/cdf 0 2024-09-21T04:35:31.957 INFO:tasks.workunit.client.0.smithi007.stdout:3/758: mknod dc/d22/d29/d92/db9/ceb 0 2024-09-21T04:35:31.965 INFO:tasks.workunit.client.0.smithi007.stdout:9/635: link d13/d18/f35 d13/d18/d80/fc7 0 2024-09-21T04:35:31.967 INFO:tasks.workunit.client.0.smithi007.stdout:4/665: symlink d2/d17/d2e/d45/d5d/dc7/le0 0 2024-09-21T04:35:31.970 INFO:tasks.workunit.client.0.smithi007.stdout:3/759: link dc/d22/f24 dc/d22/d29/d32/d4d/d84/dd0/da6/fec 0 2024-09-21T04:35:31.972 INFO:tasks.workunit.client.0.smithi007.stdout:9/636: mkdir d13/d18/d60/d6d/d75/d9f/dc8 0 2024-09-21T04:35:31.979 INFO:tasks.workunit.client.0.smithi007.stdout:4/666: mknod d2/d7/d55/d4f/ce1 0 2024-09-21T04:35:31.983 INFO:tasks.workunit.client.0.smithi007.stdout:4/667: dread d2/d7/d55/f80 [0,4194304] 0 2024-09-21T04:35:31.987 INFO:tasks.workunit.client.0.smithi007.stdout:7/623: dwrite d3/d6/fa [0,4194304] 0 2024-09-21T04:35:31.989 INFO:tasks.workunit.client.0.smithi007.stdout:9/637: symlink d13/d18/d60/d6d/d75/d9f/dc8/lc9 0 2024-09-21T04:35:31.990 INFO:tasks.workunit.client.0.smithi007.stdout:9/638: write d13/d74/f98 [451876,80727] 0 2024-09-21T04:35:31.992 INFO:tasks.workunit.client.0.smithi007.stdout:9/639: chown d13/d18/d60/d83/da9/l33 57 1 2024-09-21T04:35:31.998 INFO:tasks.workunit.client.0.smithi007.stdout:7/624: unlink d3/d6/d14/d20/d80/lcf 0 2024-09-21T04:35:32.000 INFO:tasks.workunit.client.0.smithi007.stdout:7/625: write d3/d6/d2e/fbf [1127187,46862] 0 2024-09-21T04:35:32.003 INFO:tasks.workunit.client.0.smithi007.stdout:9/640: mknod d13/d18/d60/d6d/d85/cca 0 2024-09-21T04:35:32.004 INFO:tasks.workunit.client.0.smithi007.stdout:3/760: getdents dc/d22/d29/d32/d4d 0 2024-09-21T04:35:32.006 INFO:tasks.workunit.client.0.smithi007.stdout:9/641: dread d13/d18/d60/d6d/f6f [0,4194304] 0 2024-09-21T04:35:32.008 INFO:tasks.workunit.client.0.smithi007.stdout:7/626: creat d3/d6/d10/d4f/fdb x:0 0 0 2024-09-21T04:35:32.014 INFO:tasks.workunit.client.0.smithi007.stdout:9/642: rmdir d13/d18 39 2024-09-21T04:35:32.015 INFO:tasks.workunit.client.0.smithi007.stdout:7/627: creat d3/d6/d10/d51/fdc x:0 0 0 2024-09-21T04:35:32.018 INFO:tasks.workunit.client.0.smithi007.stdout:4/668: link d2/f4c d2/fe2 0 2024-09-21T04:35:32.018 INFO:tasks.workunit.client.0.smithi007.stdout:4/669: write d2/d17/fd1 [139799,11576] 0 2024-09-21T04:35:32.021 INFO:tasks.workunit.client.0.smithi007.stdout:9/643: write d13/d18/f5c [5768905,118989] 0 2024-09-21T04:35:32.024 INFO:tasks.workunit.client.0.smithi007.stdout:3/761: link dc/d6c/l9f dc/d22/d29/d32/d4d/d84/dd0/da6/led 0 2024-09-21T04:35:32.027 INFO:tasks.workunit.client.0.smithi007.stdout:4/670: getdents d2/d5f 0 2024-09-21T04:35:32.030 INFO:tasks.workunit.client.0.smithi007.stdout:3/762: unlink dc/d18/d41/fb2 0 2024-09-21T04:35:32.033 INFO:tasks.workunit.client.0.smithi007.stdout:9/644: creat d13/dc5/fcb x:0 0 0 2024-09-21T04:35:32.033 INFO:tasks.workunit.client.0.smithi007.stdout:9/645: fsync d13/f4f 0 2024-09-21T04:35:32.035 INFO:tasks.workunit.client.0.smithi007.stdout:7/628: write d3/d6/d40/faf [3873604,44303] 0 2024-09-21T04:35:32.036 INFO:tasks.workunit.client.0.smithi007.stdout:7/629: chown d3/d6/d2e/d63/d6a 359303 1 2024-09-21T04:35:32.036 INFO:tasks.workunit.client.0.smithi007.stdout:4/671: symlink d2/d17/d2e/d45/d5d/dae/le3 0 2024-09-21T04:35:32.038 INFO:tasks.workunit.client.0.smithi007.stdout:7/630: chown d3/d6/d14/f1b 4012 1 2024-09-21T04:35:32.039 INFO:tasks.workunit.client.0.smithi007.stdout:3/763: dread dc/d22/f24 [0,4194304] 0 2024-09-21T04:35:32.041 INFO:tasks.workunit.client.0.smithi007.stdout:9/646: chown d13/d18/d60/d83/da9/l57 31902 1 2024-09-21T04:35:32.042 INFO:tasks.workunit.client.0.smithi007.stdout:4/672: creat d2/d17/d18/fe4 x:0 0 0 2024-09-21T04:35:32.046 INFO:tasks.workunit.client.0.smithi007.stdout:3/764: mknod dc/d22/dc8/cee 0 2024-09-21T04:35:32.048 INFO:tasks.workunit.client.0.smithi007.stdout:9/647: mknod d13/d18/daf/dc0/ccc 0 2024-09-21T04:35:32.049 INFO:tasks.workunit.client.0.smithi007.stdout:4/673: dwrite d2/d17/d18/f54 [4194304,4194304] 0 2024-09-21T04:35:32.055 INFO:tasks.workunit.client.0.smithi007.stdout:7/631: creat d3/d6/d40/fdd x:0 0 0 2024-09-21T04:35:32.055 INFO:tasks.workunit.client.0.smithi007.stdout:3/765: dwrite dc/d18/f7f [0,4194304] 0 2024-09-21T04:35:32.060 INFO:tasks.workunit.client.0.smithi007.stdout:3/766: creat dc/d22/d29/d32/d4d/d84/dd0/da6/de3/fef x:0 0 0 2024-09-21T04:35:32.065 INFO:tasks.workunit.client.0.smithi007.stdout:3/767: mkdir dc/d22/d29/d32/d4d/d7d/d9a/df0 0 2024-09-21T04:35:32.065 INFO:tasks.workunit.client.0.smithi007.stdout:4/674: write d2/d17/f51 [691914,23852] 0 2024-09-21T04:35:32.072 INFO:tasks.workunit.client.0.smithi007.stdout:9/648: dread d13/d18/d60/d6d/fb6 [0,4194304] 0 2024-09-21T04:35:32.076 INFO:tasks.workunit.client.0.smithi007.stdout:9/649: mkdir d13/d18/dcd 0 2024-09-21T04:35:32.077 INFO:tasks.workunit.client.0.smithi007.stdout:7/632: dread d3/d6/d13/f5d [0,4194304] 0 2024-09-21T04:35:32.080 INFO:tasks.workunit.client.0.smithi007.stdout:9/650: rename d13/d18/f36 to d13/d18/daf/dbc/fce 0 2024-09-21T04:35:32.081 INFO:tasks.workunit.client.0.smithi007.stdout:4/675: dwrite d2/d17/d2e/f71 [0,4194304] 0 2024-09-21T04:35:32.082 INFO:tasks.workunit.client.0.smithi007.stdout:7/633: chown d3/d6/d2e/l3e 2 1 2024-09-21T04:35:32.082 INFO:tasks.workunit.client.0.smithi007.stdout:9/651: readlink d13/d18/d60/lbb 0 2024-09-21T04:35:32.086 INFO:tasks.workunit.client.0.smithi007.stdout:9/652: rename d13/d18/d60/d6d/d75/d7f/f97 to d13/d18/dcd/fcf 0 2024-09-21T04:35:32.090 INFO:tasks.workunit.client.0.smithi007.stdout:7/634: dwrite d3/d6/d10/d51/fd1 [0,4194304] 0 2024-09-21T04:35:32.092 INFO:tasks.workunit.client.0.smithi007.stdout:7/635: read d3/d6/d10/d51/fd1 [2585846,49036] 0 2024-09-21T04:35:32.093 INFO:tasks.workunit.client.0.smithi007.stdout:9/653: symlink d13/d18/d60/d6d/d75/d9f/dc8/ld0 0 2024-09-21T04:35:32.100 INFO:tasks.workunit.client.0.smithi007.stdout:4/676: write d2/f27 [377941,64951] 0 2024-09-21T04:35:32.107 INFO:tasks.workunit.client.0.smithi007.stdout:7/636: creat d3/d6/d2e/d63/fde x:0 0 0 2024-09-21T04:35:32.107 INFO:tasks.workunit.client.0.smithi007.stdout:9/654: link d13/d18/f4b d13/d89/fd1 0 2024-09-21T04:35:32.112 INFO:tasks.workunit.client.0.smithi007.stdout:7/637: mknod d3/d6/dc5/cdf 0 2024-09-21T04:35:32.115 INFO:tasks.workunit.client.0.smithi007.stdout:9/655: dwrite d13/d18/d60/d6d/d85/da1/fa5 [0,4194304] 0 2024-09-21T04:35:32.117 INFO:tasks.workunit.client.0.smithi007.stdout:4/677: dread d2/d7/d55/d61/f73 [0,4194304] 0 2024-09-21T04:35:32.117 INFO:tasks.workunit.client.0.smithi007.stdout:4/678: readlink d2/d17/d2e/l40 0 2024-09-21T04:35:32.118 INFO:tasks.workunit.client.0.smithi007.stdout:7/638: fdatasync d3/d6/d10/d4f/f65 0 2024-09-21T04:35:32.123 INFO:tasks.workunit.client.0.smithi007.stdout:4/679: chown d2/d17/c2c 2 1 2024-09-21T04:35:32.131 INFO:tasks.workunit.client.0.smithi007.stdout:9/656: dwrite d13/d18/d60/d6d/f6f [0,4194304] 0 2024-09-21T04:35:32.135 INFO:tasks.workunit.client.0.smithi007.stdout:9/657: readlink d13/d74/l81 0 2024-09-21T04:35:32.137 INFO:tasks.workunit.client.0.smithi007.stdout:9/658: write d13/d18/f55 [3041549,55077] 0 2024-09-21T04:35:32.140 INFO:tasks.workunit.client.0.smithi007.stdout:4/680: dread d2/d7/f24 [0,4194304] 0 2024-09-21T04:35:32.143 INFO:tasks.workunit.client.0.smithi007.stdout:9/659: dread d13/d18/d60/d6d/f9d [0,4194304] 0 2024-09-21T04:35:32.145 INFO:tasks.workunit.client.0.smithi007.stdout:9/660: truncate d13/d18/d60/f95 678384 0 2024-09-21T04:35:32.147 INFO:tasks.workunit.client.0.smithi007.stdout:9/661: write d13/d18/f1a [3736320,119644] 0 2024-09-21T04:35:32.147 INFO:tasks.workunit.client.0.smithi007.stdout:4/681: dwrite d2/f6a [0,4194304] 0 2024-09-21T04:35:32.149 INFO:tasks.workunit.client.0.smithi007.stdout:4/682: write d2/d17/f51 [1408543,108413] 0 2024-09-21T04:35:32.150 INFO:tasks.workunit.client.0.smithi007.stdout:4/683: fsync d2/d3c/dbe/fc3 0 2024-09-21T04:35:32.154 INFO:tasks.workunit.client.0.smithi007.stdout:9/662: creat d13/d18/d60/d6d/d75/fd2 x:0 0 0 2024-09-21T04:35:32.157 INFO:tasks.workunit.client.0.smithi007.stdout:9/663: creat d13/d18/d60/d6d/d85/fd3 x:0 0 0 2024-09-21T04:35:32.157 INFO:tasks.workunit.client.0.smithi007.stdout:4/684: link d2/d17/d2e/d45/d5d/d8a/ca7 d2/d7/d64/db8/ce5 0 2024-09-21T04:35:32.158 INFO:tasks.workunit.client.0.smithi007.stdout:9/664: mkdir d13/d18/dcd/dd4 0 2024-09-21T04:35:32.161 INFO:tasks.workunit.client.0.smithi007.stdout:4/685: rename d2/d5f/fac to d2/d7/dc9/fe6 0 2024-09-21T04:35:32.164 INFO:tasks.workunit.client.0.smithi007.stdout:4/686: fsync d2/d88/fa4 0 2024-09-21T04:35:32.171 INFO:tasks.workunit.client.0.smithi007.stdout:4/687: dwrite d2/d17/d2e/d45/d5d/d8a/fd9 [0,4194304] 0 2024-09-21T04:35:32.177 INFO:tasks.workunit.client.0.smithi007.stdout:9/665: dread d13/d89/fd1 [0,4194304] 0 2024-09-21T04:35:32.178 INFO:tasks.workunit.client.0.smithi007.stdout:9/666: read d13/f23 [4102974,35281] 0 2024-09-21T04:35:32.178 INFO:tasks.workunit.client.0.smithi007.stdout:4/688: read d2/d7/f48 [4026754,123830] 0 2024-09-21T04:35:32.182 INFO:tasks.workunit.client.0.smithi007.stdout:4/689: symlink d2/d97/le7 0 2024-09-21T04:35:32.186 INFO:tasks.workunit.client.0.smithi007.stdout:9/667: unlink d13/d18/d60/d6d/d85/da1/daa/cb0 0 2024-09-21T04:35:32.190 INFO:tasks.workunit.client.0.smithi007.stdout:9/668: mknod d13/d18/d60/cd5 0 2024-09-21T04:35:32.206 INFO:tasks.workunit.client.0.smithi007.stdout:4/690: dwrite d2/fe2 [0,4194304] 0 2024-09-21T04:35:32.215 INFO:tasks.workunit.client.0.smithi007.stdout:8/407: sync 2024-09-21T04:35:32.215 INFO:tasks.workunit.client.0.smithi007.stdout:5/726: sync 2024-09-21T04:35:32.217 INFO:tasks.workunit.client.0.smithi007.stdout:8/408: write d0/f6f [867451,95176] 0 2024-09-21T04:35:32.218 INFO:tasks.workunit.client.0.smithi007.stdout:8/409: write d0/d3/d8/f47 [290831,112469] 0 2024-09-21T04:35:32.221 INFO:tasks.workunit.client.0.smithi007.stdout:5/727: creat d2/d27/d42/da3/d23/fe7 x:0 0 0 2024-09-21T04:35:32.223 INFO:tasks.workunit.client.0.smithi007.stdout:8/410: mknod d0/d3/d8/de/d26/c70 0 2024-09-21T04:35:32.223 INFO:tasks.workunit.client.0.smithi007.stdout:4/691: dread d2/d17/d18/f84 [0,4194304] 0 2024-09-21T04:35:32.229 INFO:tasks.workunit.client.0.smithi007.stdout:4/692: getdents d2/d17/d18 0 2024-09-21T04:35:32.230 INFO:tasks.workunit.client.0.smithi007.stdout:8/411: dwrite d0/f4f [0,4194304] 0 2024-09-21T04:35:32.236 INFO:tasks.workunit.client.0.smithi007.stdout:4/693: dread d2/d17/d2e/f71 [0,4194304] 0 2024-09-21T04:35:32.236 INFO:tasks.workunit.client.0.smithi007.stdout:8/412: symlink d0/d3/d8/d46/l71 0 2024-09-21T04:35:32.241 INFO:tasks.workunit.client.0.smithi007.stdout:4/694: mknod d2/d7/dc5/dd6/ce8 0 2024-09-21T04:35:32.243 INFO:tasks.workunit.client.0.smithi007.stdout:4/695: unlink d2/d7/d55/f80 0 2024-09-21T04:35:32.244 INFO:tasks.workunit.client.0.smithi007.stdout:8/413: getdents d0/d3/d16/d3b 0 2024-09-21T04:35:32.245 INFO:tasks.workunit.client.0.smithi007.stdout:4/696: mkdir d2/d17/d29/de9 0 2024-09-21T04:35:32.250 INFO:tasks.workunit.client.0.smithi007.stdout:4/697: creat d2/d7/d64/db8/fea x:0 0 0 2024-09-21T04:35:32.257 INFO:tasks.workunit.client.0.smithi007.stdout:4/698: dwrite d2/d3c/d63/fdc [0,4194304] 0 2024-09-21T04:35:32.258 INFO:tasks.workunit.client.0.smithi007.stdout:4/699: chown d2/d17/d2e/d45/d5e/f82 62 1 2024-09-21T04:35:32.259 INFO:tasks.workunit.client.0.smithi007.stdout:8/414: truncate d0/d3/d8/de/f17 850039 0 2024-09-21T04:35:32.259 INFO:tasks.workunit.client.0.smithi007.stdout:4/700: dread - d2/d7/d55/d4f/fbc zero size 2024-09-21T04:35:32.260 INFO:tasks.workunit.client.0.smithi007.stdout:8/415: fsync d0/d3/d8/de/f69 0 2024-09-21T04:35:32.262 INFO:tasks.workunit.client.0.smithi007.stdout:8/416: write d0/d3/d8/f47 [872018,23773] 0 2024-09-21T04:35:32.264 INFO:tasks.workunit.client.0.smithi007.stdout:4/701: rename d2/d17/d18/cbd to d2/d3c/d63/ceb 0 2024-09-21T04:35:32.265 INFO:tasks.workunit.client.0.smithi007.stdout:4/702: chown d2/d88/fc6 3254 1 2024-09-21T04:35:32.268 INFO:tasks.workunit.client.0.smithi007.stdout:4/703: symlink d2/d7/d55/d4f/lec 0 2024-09-21T04:35:32.273 INFO:tasks.workunit.client.0.smithi007.stdout:4/704: symlink d2/d97/led 0 2024-09-21T04:35:32.279 INFO:tasks.workunit.client.0.smithi007.stdout:4/705: creat d2/d3c/fee x:0 0 0 2024-09-21T04:35:32.282 INFO:tasks.workunit.client.0.smithi007.stdout:4/706: unlink d2/d17/d2e/d45/d5d/d8a/fd3 0 2024-09-21T04:35:32.282 INFO:tasks.workunit.client.0.smithi007.stdout:8/417: dwrite d0/d3/d8/de/d5e/f61 [0,4194304] 0 2024-09-21T04:35:32.294 INFO:tasks.workunit.client.0.smithi007.stdout:8/418: dread d0/d3/d8/f58 [0,4194304] 0 2024-09-21T04:35:32.307 INFO:tasks.workunit.client.0.smithi007.stdout:8/419: truncate d0/d3/d8/de/d26/f4a 4299761 0 2024-09-21T04:35:32.336 INFO:tasks.workunit.client.0.smithi007.stdout:8/420: rename d0/d3/d8/de/d26/f51 to d0/d3/d8/de/d26/d4d/f72 0 2024-09-21T04:35:32.337 INFO:tasks.workunit.client.0.smithi007.stdout:8/421: chown d0 1043640818 1 2024-09-21T04:35:32.337 INFO:tasks.workunit.client.0.smithi007.stdout:8/422: dread - d0/d3/d8/d46/f63 zero size 2024-09-21T04:35:32.359 INFO:tasks.workunit.client.0.smithi007.stdout:8/423: rename d0/d3/d8/de/d26/f4a to d0/d3/d8/de/d26/d35/f73 0 2024-09-21T04:35:32.365 INFO:tasks.workunit.client.0.smithi007.stdout:8/424: rename d0/d4e to d0/d3/d74 0 2024-09-21T04:35:32.407 INFO:tasks.workunit.client.0.smithi007.stdout:8/425: dwrite d0/d3/d8/de/d26/d4d/f72 [0,4194304] 0 2024-09-21T04:35:32.408 INFO:tasks.workunit.client.0.smithi007.stdout:8/426: read d0/d3/d8/de/d40/f45 [306411,37853] 0 2024-09-21T04:35:32.410 INFO:tasks.workunit.client.0.smithi007.stdout:8/427: write d0/d3/d8/f47 [1094717,110474] 0 2024-09-21T04:35:32.416 INFO:tasks.workunit.client.0.smithi007.stdout:8/428: rename d0/c56 to d0/d3/d8/de/d26/d35/c75 0 2024-09-21T04:35:32.420 INFO:tasks.workunit.client.0.smithi007.stdout:8/429: read d0/d3/d8/d46/f60 [2576291,7786] 0 2024-09-21T04:35:32.424 INFO:tasks.workunit.client.0.smithi007.stdout:8/430: creat d0/d3/d8/de/d40/d6c/f76 x:0 0 0 2024-09-21T04:35:32.426 INFO:tasks.workunit.client.0.smithi007.stdout:8/431: rmdir d0/d3/d74/d52 39 2024-09-21T04:35:32.428 INFO:tasks.workunit.client.0.smithi007.stdout:8/432: mknod d0/d3/d8/d46/c77 0 2024-09-21T04:35:32.431 INFO:tasks.workunit.client.0.smithi007.stdout:8/433: mknod d0/d3/d8/de/c78 0 2024-09-21T04:35:32.432 INFO:tasks.workunit.client.0.smithi007.stdout:8/434: chown d0/d3/d16/d3b/f64 15628375 1 2024-09-21T04:35:32.438 INFO:tasks.workunit.client.0.smithi007.stdout:8/435: link d0/d3/d8/d46/c77 d0/c79 0 2024-09-21T04:35:32.450 INFO:tasks.workunit.client.0.smithi007.stdout:8/436: dread d0/d3/d8/f67 [0,4194304] 0 2024-09-21T04:35:32.451 INFO:tasks.workunit.client.0.smithi007.stdout:8/437: fsync d0/d3/d8/de/d26/d4d/f72 0 2024-09-21T04:35:32.463 INFO:tasks.workunit.client.0.smithi007.stdout:8/438: dread d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:32.465 INFO:tasks.workunit.client.0.smithi007.stdout:8/439: unlink d0/d3/d8/de/d26/l5c 0 2024-09-21T04:35:32.468 INFO:tasks.workunit.client.0.smithi007.stdout:8/440: rename d0/d3/d8/de/d26/c34 to d0/d3/d8/de/d26/d4d/d5d/c7a 0 2024-09-21T04:35:32.469 INFO:tasks.workunit.client.0.smithi007.stdout:8/441: write d0/d3/d8/f5f [717098,88000] 0 2024-09-21T04:35:32.470 INFO:tasks.workunit.client.0.smithi007.stdout:8/442: write d0/f6f [304590,24533] 0 2024-09-21T04:35:32.473 INFO:tasks.workunit.client.0.smithi007.stdout:8/443: mkdir d0/d3/d8/de/d5e/d7b 0 2024-09-21T04:35:32.477 INFO:tasks.workunit.client.0.smithi007.stdout:8/444: rename d0/d3/d16/d3b to d0/d3/d7c 0 2024-09-21T04:35:32.478 INFO:tasks.workunit.client.0.smithi007.stdout:8/445: dread - d0/d3/d7c/f59 zero size 2024-09-21T04:35:32.480 INFO:tasks.workunit.client.0.smithi007.stdout:8/446: dread d0/d3/d8/de/d40/f45 [0,4194304] 0 2024-09-21T04:35:32.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:32 smithi053 ceph-mon[35554]: pgmap v37: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 84 MiB/s rd, 221 MiB/s wr, 440 op/s 2024-09-21T04:35:32.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:32 smithi007 ceph-mon[27954]: pgmap v37: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 84 MiB/s rd, 221 MiB/s wr, 440 op/s 2024-09-21T04:35:32.746 INFO:tasks.workunit.client.0.smithi007.stdout:1/814: sync 2024-09-21T04:35:32.750 INFO:tasks.workunit.client.0.smithi007.stdout:1/815: symlink d5/d6/d21/d2b/d4b/da6/d58/dde/l102 0 2024-09-21T04:35:32.753 INFO:tasks.workunit.client.0.smithi007.stdout:1/816: mknod d5/d6/d21/d2b/d4b/d8a/dee/c103 0 2024-09-21T04:35:32.756 INFO:tasks.workunit.client.0.smithi007.stdout:1/817: mkdir d5/d6/d21/d3c/dd5/dea/dfc/d104 0 2024-09-21T04:35:32.757 INFO:tasks.workunit.client.0.smithi007.stdout:1/818: chown d5/d6/d21/d2b/fd7 0 1 2024-09-21T04:35:32.759 INFO:tasks.workunit.client.0.smithi007.stdout:1/819: fdatasync d5/d6/d21/d2b/d4b/da6/f9c 0 2024-09-21T04:35:32.761 INFO:tasks.workunit.client.0.smithi007.stdout:1/820: write d5/d6/f8 [4619286,108273] 0 2024-09-21T04:35:32.770 INFO:tasks.workunit.client.0.smithi007.stdout:1/821: read d5/d6/d21/d2b/d4b/da6/d58/fa5 [18773,51187] 0 2024-09-21T04:35:32.773 INFO:tasks.workunit.client.0.smithi007.stdout:1/822: mkdir d5/d6/d21/d2b/d4b/d8a/d8d/d105 0 2024-09-21T04:35:32.775 INFO:tasks.workunit.client.0.smithi007.stdout:1/823: symlink d5/d6/d21/d2b/l106 0 2024-09-21T04:35:32.781 INFO:tasks.workunit.client.0.smithi007.stdout:1/824: write d5/f90 [806336,102872] 0 2024-09-21T04:35:32.788 INFO:tasks.workunit.client.0.smithi007.stdout:1/825: link d5/d6/d21/d64/df1/da0/d99/dae/fd1 d5/d6/d21/d2b/d4b/d8a/dee/f107 0 2024-09-21T04:35:32.795 INFO:tasks.workunit.client.0.smithi007.stdout:1/826: write d5/d6/d21/d2b/d4b/da6/d58/fa5 [89022,71292] 0 2024-09-21T04:35:32.797 INFO:tasks.workunit.client.0.smithi007.stdout:1/827: write d5/fc3 [1200588,77725] 0 2024-09-21T04:35:32.800 INFO:tasks.workunit.client.0.smithi007.stdout:1/828: mkdir d5/d6/d21/d64/d108 0 2024-09-21T04:35:32.802 INFO:tasks.workunit.client.0.smithi007.stdout:1/829: symlink d5/d6/d21/d2b/d86/dd8/l109 0 2024-09-21T04:35:32.804 INFO:tasks.workunit.client.0.smithi007.stdout:1/830: mkdir d5/d6/d21/d2b/d4b/da6/d58/dde/d10a 0 2024-09-21T04:35:32.805 INFO:tasks.workunit.client.0.smithi007.stdout:1/831: chown d5/d6/d21/d2b 27 1 2024-09-21T04:35:33.165 INFO:tasks.workunit.client.0.smithi007.stdout:0/678: sync 2024-09-21T04:35:33.170 INFO:tasks.workunit.client.0.smithi007.stdout:0/679: rename d6/d75/dc5/d47/f4b to d6/d75/dc5/d18/d2e/fff 0 2024-09-21T04:35:33.174 INFO:tasks.workunit.client.0.smithi007.stdout:0/680: mknod d6/d75/dc5/d18/d2e/c100 0 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:3/768: sync 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:6/624: sync 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:9/669: sync 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:8/447: sync 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:2/503: sync 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:7/639: sync 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:5/728: sync 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:4/707: sync 2024-09-21T04:35:33.324 INFO:tasks.workunit.client.0.smithi007.stdout:1/832: sync 2024-09-21T04:35:33.326 INFO:tasks.workunit.client.0.smithi007.stdout:6/625: fsync d3/d3e/d46/f68 0 2024-09-21T04:35:33.328 INFO:tasks.workunit.client.0.smithi007.stdout:1/833: fsync d5/d6/d21/d2b/d4b/da6/d58/f67 0 2024-09-21T04:35:33.330 INFO:tasks.workunit.client.0.smithi007.stdout:6/626: readlink d3/de/d12/d19/la1 0 2024-09-21T04:35:33.332 INFO:tasks.workunit.client.0.smithi007.stdout:5/729: read d2/d27/d42/da3/d23/d5d/d9a/da6/fba [168588,48450] 0 2024-09-21T04:35:33.334 INFO:tasks.workunit.client.0.smithi007.stdout:4/708: dread d2/fe2 [0,4194304] 0 2024-09-21T04:35:33.334 INFO:tasks.workunit.client.0.smithi007.stdout:7/640: creat d3/d6/d10/fe0 x:0 0 0 2024-09-21T04:35:33.336 INFO:tasks.workunit.client.0.smithi007.stdout:9/670: rename d13/d18/cc1 to d13/d89/cd6 0 2024-09-21T04:35:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:1/834: dwrite d5/d6/d21/d2b/d4b/f74 [0,4194304] 0 2024-09-21T04:35:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:8/448: dread d0/d3/d16/f21 [0,4194304] 0 2024-09-21T04:35:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:5/730: rmdir d2/d27/d42/da3/d1d 39 2024-09-21T04:35:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:8/449: dread d0/d3/d8/de/d40/f45 [0,4194304] 0 2024-09-21T04:35:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:9/671: chown d13/d18/c51 63 1 2024-09-21T04:35:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:3/769: creat dc/d22/d29/d32/d4d/d84/dd0/dbc/dd1/ff1 x:0 0 0 2024-09-21T04:35:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:3/770: write dc/fd4 [210415,67631] 0 2024-09-21T04:35:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:3/771: chown dc/d22/d29/d32/d4d/d84/dd0/da6/de3/fef 21863 1 2024-09-21T04:35:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:6/627: fdatasync d3/de/d12/f8d 0 2024-09-21T04:35:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:3/772: write dc/d22/d29/d32/d4d/d84/dd0/dbc/dd1/ff1 [63042,78169] 0 2024-09-21T04:35:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:5/731: creat d2/d27/d3d/d9b/fe8 x:0 0 0 2024-09-21T04:35:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:8/450: creat d0/d3/d8/d46/f7d x:0 0 0 2024-09-21T04:35:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:5/732: truncate d2/d27/d42/d53/f90 837788 0 2024-09-21T04:35:33.367 INFO:tasks.workunit.client.0.smithi007.stdout:8/451: dread d0/d3/d8/d46/f60 [0,4194304] 0 2024-09-21T04:35:33.370 INFO:tasks.workunit.client.0.smithi007.stdout:1/835: symlink d5/d6/d21/dd4/l10b 0 2024-09-21T04:35:33.374 INFO:tasks.workunit.client.0.smithi007.stdout:3/773: mkdir dc/d22/d29/d32/d8f/df2 0 2024-09-21T04:35:33.374 INFO:tasks.workunit.client.0.smithi007.stdout:2/504: dwrite d0/d11/f4b [4194304,4194304] 0 2024-09-21T04:35:33.378 INFO:tasks.workunit.client.0.smithi007.stdout:8/452: creat d0/d3/d8/de/d5e/d7b/f7e x:0 0 0 2024-09-21T04:35:33.379 INFO:tasks.workunit.client.0.smithi007.stdout:8/453: stat d0/d3/d8/d46/l71 0 2024-09-21T04:35:33.380 INFO:tasks.workunit.client.0.smithi007.stdout:3/774: unlink dc/d22/d29/d32/d4d/d84/dd0/da6/c47 0 2024-09-21T04:35:33.382 INFO:tasks.workunit.client.0.smithi007.stdout:5/733: getdents d2/d27/d42/da3/d23/d49/d6f 0 2024-09-21T04:35:33.383 INFO:tasks.workunit.client.0.smithi007.stdout:2/505: dwrite d0/d2/f6 [8388608,4194304] 0 2024-09-21T04:35:33.386 INFO:tasks.workunit.client.0.smithi007.stdout:8/454: mknod d0/d3/c7f 0 2024-09-21T04:35:33.389 INFO:tasks.workunit.client.0.smithi007.stdout:5/734: mkdir d2/d27/d42/da3/d23/d49/dd4/de9 0 2024-09-21T04:35:33.389 INFO:tasks.workunit.client.0.smithi007.stdout:8/455: fsync d0/d3/d74/f66 0 2024-09-21T04:35:33.391 INFO:tasks.workunit.client.0.smithi007.stdout:3/775: dwrite dc/d18/d41/f4a [0,4194304] 0 2024-09-21T04:35:33.393 INFO:tasks.workunit.client.0.smithi007.stdout:3/776: write dc/d22/d29/d32/d4d/d7d/d9a/fb4 [440997,119280] 0 2024-09-21T04:35:33.394 INFO:tasks.workunit.client.0.smithi007.stdout:8/456: symlink d0/d3/d8/de/d26/d4d/l80 0 2024-09-21T04:35:33.394 INFO:tasks.workunit.client.0.smithi007.stdout:3/777: chown dc/d22/d29/d32/d8f/df2 1673 1 2024-09-21T04:35:33.397 INFO:tasks.workunit.client.0.smithi007.stdout:5/735: creat d2/d27/d42/d53/dcf/fea x:0 0 0 2024-09-21T04:35:33.398 INFO:tasks.workunit.client.0.smithi007.stdout:3/778: creat dc/d22/d29/d32/d4d/d7d/d9a/ff3 x:0 0 0 2024-09-21T04:35:33.401 INFO:tasks.workunit.client.0.smithi007.stdout:5/736: unlink d2/d27/d42/da3/d23/d49/d44/f84 0 2024-09-21T04:35:33.406 INFO:tasks.workunit.client.0.smithi007.stdout:3/779: mknod dc/d18/dea/cf4 0 2024-09-21T04:35:33.412 INFO:tasks.workunit.client.0.smithi007.stdout:5/737: dwrite d2/d1e/f58 [0,4194304] 0 2024-09-21T04:35:33.414 INFO:tasks.workunit.client.0.smithi007.stdout:5/738: chown d2/d27/d42/d53/c7c 1304098830 1 2024-09-21T04:35:33.415 INFO:tasks.workunit.client.0.smithi007.stdout:3/780: unlink dc/f23 0 2024-09-21T04:35:33.416 INFO:tasks.workunit.client.0.smithi007.stdout:3/781: chown dc/d18/f42 0 1 2024-09-21T04:35:33.425 INFO:tasks.workunit.client.0.smithi007.stdout:3/782: rename dc/d18/f28 to dc/d6c/ff5 0 2024-09-21T04:35:33.428 INFO:tasks.workunit.client.0.smithi007.stdout:2/506: dread d0/d2/d57/f5a [0,4194304] 0 2024-09-21T04:35:33.432 INFO:tasks.workunit.client.0.smithi007.stdout:5/739: dwrite d2/d27/d42/da3/d28/f4d [0,4194304] 0 2024-09-21T04:35:33.437 INFO:tasks.workunit.client.0.smithi007.stdout:2/507: creat d0/d11/d31/d7b/f89 x:0 0 0 2024-09-21T04:35:33.440 INFO:tasks.workunit.client.0.smithi007.stdout:3/783: creat dc/d18/d41/ff6 x:0 0 0 2024-09-21T04:35:33.441 INFO:tasks.workunit.client.0.smithi007.stdout:2/508: creat d0/d11/d6e/f8a x:0 0 0 2024-09-21T04:35:33.444 INFO:tasks.workunit.client.0.smithi007.stdout:2/509: symlink d0/d11/d31/d39/l8b 0 2024-09-21T04:35:33.446 INFO:tasks.workunit.client.0.smithi007.stdout:2/510: creat d0/d11/d31/d39/f8c x:0 0 0 2024-09-21T04:35:33.461 INFO:tasks.workunit.client.0.smithi007.stdout:7/641: truncate d3/d6/d2e/d63/d6a/d81/fd0 735111 0 2024-09-21T04:35:33.462 INFO:tasks.workunit.client.0.smithi007.stdout:9/672: write d13/d18/d60/f88 [2413327,46863] 0 2024-09-21T04:35:33.463 INFO:tasks.workunit.client.0.smithi007.stdout:1/836: write d5/d6/d21/d64/df1/da0/d99/dae/fd1 [560767,18953] 0 2024-09-21T04:35:33.467 INFO:tasks.workunit.client.0.smithi007.stdout:4/709: dwrite d2/d7/f8 [0,4194304] 0 2024-09-21T04:35:33.468 INFO:tasks.workunit.client.0.smithi007.stdout:9/673: write d13/d18/d60/f88 [3008432,78151] 0 2024-09-21T04:35:33.471 INFO:tasks.workunit.client.0.smithi007.stdout:4/710: truncate d2/d7/d55/d4f/fbc 803636 0 2024-09-21T04:35:33.474 INFO:tasks.workunit.client.0.smithi007.stdout:4/711: dread d2/d17/d18/f2d [0,4194304] 0 2024-09-21T04:35:33.476 INFO:tasks.workunit.client.0.smithi007.stdout:9/674: write d13/d18/dcd/fcf [507581,94256] 0 2024-09-21T04:35:33.477 INFO:tasks.workunit.client.0.smithi007.stdout:9/675: dread - d13/d18/d60/d83/da9/fb7 zero size 2024-09-21T04:35:33.478 INFO:tasks.workunit.client.0.smithi007.stdout:7/642: symlink d3/d6/d14/d20/d80/db2/db6/le1 0 2024-09-21T04:35:33.480 INFO:tasks.workunit.client.0.smithi007.stdout:4/712: symlink d2/d17/d29/lef 0 2024-09-21T04:35:33.482 INFO:tasks.workunit.client.0.smithi007.stdout:9/676: creat d13/d18/d60/d6d/d85/da1/daa/fd7 x:0 0 0 2024-09-21T04:35:33.484 INFO:tasks.workunit.client.0.smithi007.stdout:7/643: symlink d3/d6/d14/d20/d80/db2/db6/le2 0 2024-09-21T04:35:33.485 INFO:tasks.workunit.client.0.smithi007.stdout:4/713: mkdir d2/d17/d18/df0 0 2024-09-21T04:35:33.487 INFO:tasks.workunit.client.0.smithi007.stdout:7/644: creat d3/d6/d85/dd2/fe3 x:0 0 0 2024-09-21T04:35:33.490 INFO:tasks.workunit.client.0.smithi007.stdout:4/714: creat d2/d5f/ff1 x:0 0 0 2024-09-21T04:35:33.491 INFO:tasks.workunit.client.0.smithi007.stdout:4/715: dread - d2/d5f/ff1 zero size 2024-09-21T04:35:33.492 INFO:tasks.workunit.client.0.smithi007.stdout:4/716: readlink d2/d17/d2e/d45/d5d/d8a/lc4 0 2024-09-21T04:35:33.494 INFO:tasks.workunit.client.0.smithi007.stdout:9/677: creat d13/d18/d60/d6d/d75/fd8 x:0 0 0 2024-09-21T04:35:33.499 INFO:tasks.workunit.client.0.smithi007.stdout:9/678: readlink d13/d5a/l5f 0 2024-09-21T04:35:33.499 INFO:tasks.workunit.client.0.smithi007.stdout:5/740: write d2/d1e/f24 [4871378,105090] 0 2024-09-21T04:35:33.500 INFO:tasks.workunit.client.0.smithi007.stdout:5/741: write d2/f50 [823229,57614] 0 2024-09-21T04:35:33.502 INFO:tasks.workunit.client.0.smithi007.stdout:4/717: mkdir d2/d17/d2e/d45/df2 0 2024-09-21T04:35:33.503 INFO:tasks.workunit.client.0.smithi007.stdout:7/645: getdents d3/d6/d10/d62/d66 0 2024-09-21T04:35:33.508 INFO:tasks.workunit.client.0.smithi007.stdout:4/718: mknod d2/d17/d2e/d45/d5d/cf3 0 2024-09-21T04:35:33.510 INFO:tasks.workunit.client.0.smithi007.stdout:2/511: getdents d0/d11/d31/d39 0 2024-09-21T04:35:33.511 INFO:tasks.workunit.client.0.smithi007.stdout:5/742: dread d2/d1e/f37 [0,4194304] 0 2024-09-21T04:35:33.514 INFO:tasks.workunit.client.0.smithi007.stdout:5/743: write d2/d27/d42/da3/d23/d49/d44/d54/f59 [3456334,96085] 0 2024-09-21T04:35:33.515 INFO:tasks.workunit.client.0.smithi007.stdout:7/646: symlink d3/d6/le4 0 2024-09-21T04:35:33.518 INFO:tasks.workunit.client.0.smithi007.stdout:1/837: write d5/d2f/f3a [3005658,110127] 0 2024-09-21T04:35:33.525 INFO:tasks.workunit.client.0.smithi007.stdout:4/719: creat d2/d17/d2e/d9f/ff4 x:0 0 0 2024-09-21T04:35:33.531 INFO:tasks.workunit.client.0.smithi007.stdout:1/838: write d5/fd0 [1230849,57075] 0 2024-09-21T04:35:33.532 INFO:tasks.workunit.client.0.smithi007.stdout:9/679: getdents d13/d18/d80 0 2024-09-21T04:35:33.532 INFO:tasks.workunit.client.0.smithi007.stdout:1/839: chown d5/f90 1 1 2024-09-21T04:35:33.535 INFO:tasks.workunit.client.0.smithi007.stdout:4/720: creat d2/d17/d2e/d45/d5e/ff5 x:0 0 0 2024-09-21T04:35:33.537 INFO:tasks.workunit.client.0.smithi007.stdout:5/744: creat d2/d27/d42/da3/d23/d49/dd4/de9/feb x:0 0 0 2024-09-21T04:35:33.538 INFO:tasks.workunit.client.0.smithi007.stdout:5/745: dread d2/d1e/f37 [0,4194304] 0 2024-09-21T04:35:33.539 INFO:tasks.workunit.client.0.smithi007.stdout:7/647: creat d3/d6/d40/da1/fe5 x:0 0 0 2024-09-21T04:35:33.541 INFO:tasks.workunit.client.0.smithi007.stdout:9/680: mknod d13/d18/d60/d6d/d85/da1/cd9 0 2024-09-21T04:35:33.541 INFO:tasks.workunit.client.0.smithi007.stdout:2/512: link d0/d2/l33 d0/d11/d31/d46/d6d/d81/l8d 0 2024-09-21T04:35:33.543 INFO:tasks.workunit.client.0.smithi007.stdout:2/513: chown d0/d11/f4b 131778 1 2024-09-21T04:35:33.550 INFO:tasks.workunit.client.0.smithi007.stdout:9/681: dwrite d13/d18/d60/d6d/d85/da1/daa/fb8 [0,4194304] 0 2024-09-21T04:35:33.557 INFO:tasks.workunit.client.0.smithi007.stdout:5/746: dread d2/d27/d42/da3/fb9 [0,4194304] 0 2024-09-21T04:35:33.558 INFO:tasks.workunit.client.0.smithi007.stdout:1/840: dwrite d5/d45/f7d [0,4194304] 0 2024-09-21T04:35:33.561 INFO:tasks.workunit.client.0.smithi007.stdout:2/514: rename d0/d11/d31/d46/f60 to d0/d11/d31/d46/d48/f8e 0 2024-09-21T04:35:33.562 INFO:tasks.workunit.client.0.smithi007.stdout:9/682: mkdir d13/d18/daf/dda 0 2024-09-21T04:35:33.563 INFO:tasks.workunit.client.0.smithi007.stdout:5/747: mkdir d2/d27/d42/da3/d23/d5d/d61/dec 0 2024-09-21T04:35:33.568 INFO:tasks.workunit.client.0.smithi007.stdout:1/841: symlink d5/d6/d21/d2b/d4b/da6/d58/dde/d10a/l10c 0 2024-09-21T04:35:33.571 INFO:tasks.workunit.client.0.smithi007.stdout:4/721: truncate d2/f23 2331280 0 2024-09-21T04:35:33.571 INFO:tasks.workunit.client.0.smithi007.stdout:5/748: dwrite d2/d27/d42/da3/d23/d49/d6f/f7f [0,4194304] 0 2024-09-21T04:35:33.572 INFO:tasks.workunit.client.0.smithi007.stdout:5/749: stat d2/d27/d3d/f69 0 2024-09-21T04:35:33.574 INFO:tasks.workunit.client.0.smithi007.stdout:2/515: rename d0/l80 to d0/d11/d31/d39/l8f 0 2024-09-21T04:35:33.582 INFO:tasks.workunit.client.0.smithi007.stdout:4/722: write d2/d17/d2e/d45/f8f [3974826,87603] 0 2024-09-21T04:35:33.584 INFO:tasks.workunit.client.0.smithi007.stdout:4/723: write d2/d3c/d63/fdc [4527313,69209] 0 2024-09-21T04:35:33.585 INFO:tasks.workunit.client.0.smithi007.stdout:9/683: rename d13/dc5 to d13/d74/ddb 0 2024-09-21T04:35:33.586 INFO:tasks.workunit.client.0.smithi007.stdout:7/648: getdents d3/d6/d10/d62 0 2024-09-21T04:35:33.587 INFO:tasks.workunit.client.0.smithi007.stdout:9/684: chown d13/d18/daf/dbc/fc6 6667085 1 2024-09-21T04:35:33.589 INFO:tasks.workunit.client.0.smithi007.stdout:5/750: link d2/d27/d42/da3/fe1 d2/d27/d3d/dc2/fed 0 2024-09-21T04:35:33.592 INFO:tasks.workunit.client.0.smithi007.stdout:2/516: rename d0/d11/d31/f6f to d0/d11/d6e/f90 0 2024-09-21T04:35:33.592 INFO:tasks.workunit.client.0.smithi007.stdout:9/685: dread d13/d18/d60/d83/da9/f47 [0,4194304] 0 2024-09-21T04:35:33.594 INFO:tasks.workunit.client.0.smithi007.stdout:9/686: read - d13/d18/d60/d6d/d75/fbf zero size 2024-09-21T04:35:33.603 INFO:tasks.workunit.client.0.smithi007.stdout:4/724: symlink d2/d17/d2e/d45/d5d/d8a/dd2/lf6 0 2024-09-21T04:35:33.606 INFO:tasks.workunit.client.0.smithi007.stdout:9/687: creat d13/d18/d60/d83/fdc x:0 0 0 2024-09-21T04:35:33.607 INFO:tasks.workunit.client.0.smithi007.stdout:7/649: symlink d3/d6/d2e/d63/d6a/d81/le6 0 2024-09-21T04:35:33.608 INFO:tasks.workunit.client.0.smithi007.stdout:1/842: getdents d5/d6/d21/d2b/d4b/d8a 0 2024-09-21T04:35:33.609 INFO:tasks.workunit.client.0.smithi007.stdout:5/751: mkdir d2/d27/dee 0 2024-09-21T04:35:33.611 INFO:tasks.workunit.client.0.smithi007.stdout:4/725: readlink d2/d17/l4d 0 2024-09-21T04:35:33.613 INFO:tasks.workunit.client.0.smithi007.stdout:2/517: creat d0/d11/d31/d46/d77/f91 x:0 0 0 2024-09-21T04:35:33.615 INFO:tasks.workunit.client.0.smithi007.stdout:9/688: dwrite d13/d18/d60/d6d/d75/fd8 [0,4194304] 0 2024-09-21T04:35:33.616 INFO:tasks.workunit.client.0.smithi007.stdout:1/843: mkdir d5/d6/d21/d2b/d86/dd8/d10d 0 2024-09-21T04:35:33.621 INFO:tasks.workunit.client.0.smithi007.stdout:4/726: symlink d2/d17/d2e/d45/d5e/lf7 0 2024-09-21T04:35:33.623 INFO:tasks.workunit.client.0.smithi007.stdout:7/650: mkdir d3/d6/d85/de7 0 2024-09-21T04:35:33.624 INFO:tasks.workunit.client.0.smithi007.stdout:2/518: symlink d0/d11/d31/d52/l92 0 2024-09-21T04:35:33.627 INFO:tasks.workunit.client.0.smithi007.stdout:9/689: chown d13/c48 8437 1 2024-09-21T04:35:33.628 INFO:tasks.workunit.client.0.smithi007.stdout:1/844: mknod d5/d6/d21/d3c/dd5/dea/c10e 0 2024-09-21T04:35:33.630 INFO:tasks.workunit.client.0.smithi007.stdout:5/752: mkdir d2/d27/d42/da3/d23/d49/d6f/ddb/def 0 2024-09-21T04:35:33.634 INFO:tasks.workunit.client.0.smithi007.stdout:9/690: creat d13/d18/daf/fdd x:0 0 0 2024-09-21T04:35:33.634 INFO:tasks.workunit.client.0.smithi007.stdout:1/845: mknod d5/d2f/c10f 0 2024-09-21T04:35:33.636 INFO:tasks.workunit.client.0.smithi007.stdout:4/727: mkdir d2/d17/df8 0 2024-09-21T04:35:33.637 INFO:tasks.workunit.client.0.smithi007.stdout:7/651: mknod d3/d6/d2e/ce8 0 2024-09-21T04:35:33.639 INFO:tasks.workunit.client.0.smithi007.stdout:9/691: mknod d13/d18/d60/d83/da9/cde 0 2024-09-21T04:35:33.641 INFO:tasks.workunit.client.0.smithi007.stdout:1/846: unlink d5/d6/d21/d2b/d4b/d8a/lcc 0 2024-09-21T04:35:33.643 INFO:tasks.workunit.client.0.smithi007.stdout:9/692: chown d13/d18/daf/dbc/fce 6795 1 2024-09-21T04:35:33.646 INFO:tasks.workunit.client.0.smithi007.stdout:7/652: dwrite d3/d6/d10/d51/fdc [0,4194304] 0 2024-09-21T04:35:33.649 INFO:tasks.workunit.client.0.smithi007.stdout:2/519: dread d0/f7 [0,4194304] 0 2024-09-21T04:35:33.653 INFO:tasks.workunit.client.0.smithi007.stdout:4/728: creat d2/d3c/ff9 x:0 0 0 2024-09-21T04:35:33.653 INFO:tasks.workunit.client.0.smithi007.stdout:5/753: dread d2/d27/d42/f62 [0,4194304] 0 2024-09-21T04:35:33.661 INFO:tasks.workunit.client.0.smithi007.stdout:1/847: dread d5/f5d [4194304,4194304] 0 2024-09-21T04:35:33.664 INFO:tasks.workunit.client.0.smithi007.stdout:5/754: dread d2/d27/d42/da3/d23/d49/f80 [4194304,4194304] 0 2024-09-21T04:35:33.667 INFO:tasks.workunit.client.0.smithi007.stdout:2/520: truncate d0/fd 1648640 0 2024-09-21T04:35:33.668 INFO:tasks.workunit.client.0.smithi007.stdout:2/521: chown d0/f23 192208 1 2024-09-21T04:35:33.676 INFO:tasks.workunit.client.0.smithi007.stdout:4/729: creat d2/d17/d2e/d45/df2/ffa x:0 0 0 2024-09-21T04:35:33.676 INFO:tasks.workunit.client.0.smithi007.stdout:1/848: read d5/d6/d21/d2b/f51 [584958,70071] 0 2024-09-21T04:35:33.678 INFO:tasks.workunit.client.0.smithi007.stdout:4/730: chown d2/d17/d2e/d45/d5d/d8a/fd9 4531 1 2024-09-21T04:35:33.682 INFO:tasks.workunit.client.0.smithi007.stdout:2/522: creat d0/d11/d31/d7b/f93 x:0 0 0 2024-09-21T04:35:33.684 INFO:tasks.workunit.client.0.smithi007.stdout:9/693: link d13/f5b d13/d18/d80/fdf 0 2024-09-21T04:35:33.687 INFO:tasks.workunit.client.0.smithi007.stdout:1/849: creat d5/d6/dc5/f110 x:0 0 0 2024-09-21T04:35:33.692 INFO:tasks.workunit.client.0.smithi007.stdout:7/653: dwrite d3/d6/d10/d51/f6e [0,4194304] 0 2024-09-21T04:35:33.693 INFO:tasks.workunit.client.0.smithi007.stdout:4/731: dread d2/f25 [0,4194304] 0 2024-09-21T04:35:33.696 INFO:tasks.workunit.client.0.smithi007.stdout:9/694: dwrite d13/d18/f1a [4194304,4194304] 0 2024-09-21T04:35:33.699 INFO:tasks.workunit.client.0.smithi007.stdout:9/695: stat d13/d18/f54 0 2024-09-21T04:35:33.707 INFO:tasks.workunit.client.0.smithi007.stdout:5/755: dwrite d2/d27/d42/da3/d23/d49/f4a [0,4194304] 0 2024-09-21T04:35:33.710 INFO:tasks.workunit.client.0.smithi007.stdout:6/628: sync 2024-09-21T04:35:33.710 INFO:tasks.workunit.client.0.smithi007.stdout:0/681: sync 2024-09-21T04:35:33.715 INFO:tasks.workunit.client.0.smithi007.stdout:2/523: write d0/f2e [1152878,22978] 0 2024-09-21T04:35:33.716 INFO:tasks.workunit.client.0.smithi007.stdout:1/850: rename d5/d6/f8 to d5/d45/f111 0 2024-09-21T04:35:33.722 INFO:tasks.workunit.client.0.smithi007.stdout:4/732: mkdir d2/d7/dfb 0 2024-09-21T04:35:33.725 INFO:tasks.workunit.client.0.smithi007.stdout:1/851: dwrite d5/d6/d21/d64/df1/da0/ff3 [0,4194304] 0 2024-09-21T04:35:33.726 INFO:tasks.workunit.client.0.smithi007.stdout:5/756: fsync d2/d27/d3d/f85 0 2024-09-21T04:35:33.730 INFO:tasks.workunit.client.0.smithi007.stdout:1/852: write d5/d6/d21/d2b/d4b/da6/d58/f67 [5052958,76755] 0 2024-09-21T04:35:33.730 INFO:tasks.workunit.client.0.smithi007.stdout:7/654: dread d3/d6/d14/d20/d80/db2/fb1 [0,4194304] 0 2024-09-21T04:35:33.731 INFO:tasks.workunit.client.0.smithi007.stdout:0/682: write d6/d75/dc5/d52/d6d/de1/fe7 [87753,42171] 0 2024-09-21T04:35:33.738 INFO:tasks.workunit.client.0.smithi007.stdout:5/757: rmdir d2/d1e 39 2024-09-21T04:35:33.738 INFO:tasks.workunit.client.0.smithi007.stdout:5/758: fsync d2/f50 0 2024-09-21T04:35:33.739 INFO:tasks.workunit.client.0.smithi007.stdout:9/696: getdents d13 0 2024-09-21T04:35:33.741 INFO:tasks.workunit.client.0.smithi007.stdout:6/629: mkdir d3/d3e/d46/d60/dd2 0 2024-09-21T04:35:33.742 INFO:tasks.workunit.client.0.smithi007.stdout:6/630: chown d3/d3e/d46/d60/d98 5 1 2024-09-21T04:35:33.742 INFO:tasks.workunit.client.0.smithi007.stdout:7/655: mkdir d3/d6/d85/de9 0 2024-09-21T04:35:33.742 INFO:tasks.workunit.client.0.smithi007.stdout:6/631: chown d3/de/d58/f90 5156 1 2024-09-21T04:35:33.743 INFO:tasks.workunit.client.0.smithi007.stdout:6/632: chown d3/de/d58/fc8 43229 1 2024-09-21T04:35:33.745 INFO:tasks.workunit.client.0.smithi007.stdout:0/683: rename d6/ddb to d6/d75/dc5/d47/d8a/de9/d101 0 2024-09-21T04:35:33.746 INFO:tasks.workunit.client.0.smithi007.stdout:2/524: getdents d0/d2/d57 0 2024-09-21T04:35:33.747 INFO:tasks.workunit.client.0.smithi007.stdout:4/733: symlink d2/d88/lfc 0 2024-09-21T04:35:33.748 INFO:tasks.workunit.client.0.smithi007.stdout:2/525: stat d0/d2/l2c 0 2024-09-21T04:35:33.751 INFO:tasks.workunit.client.0.smithi007.stdout:9/697: unlink d13/d18/d60/d83/da9/l33 0 2024-09-21T04:35:33.752 INFO:tasks.workunit.client.0.smithi007.stdout:1/853: dwrite d5/d6/d21/d64/df1/f62 [4194304,4194304] 0 2024-09-21T04:35:33.759 INFO:tasks.workunit.client.0.smithi007.stdout:6/633: rename d3/d3e/d46 to d3/d3b/dd3 0 2024-09-21T04:35:33.761 INFO:tasks.workunit.client.0.smithi007.stdout:2/526: creat d0/d11/d31/d7b/f94 x:0 0 0 2024-09-21T04:35:33.764 INFO:tasks.workunit.client.0.smithi007.stdout:5/759: dread d2/d1e/f24 [8388608,4194304] 0 2024-09-21T04:35:33.767 INFO:tasks.workunit.client.0.smithi007.stdout:9/698: dread d13/d18/f1a [0,4194304] 0 2024-09-21T04:35:33.775 INFO:tasks.workunit.client.0.smithi007.stdout:0/684: rename d6/d75/dc5/d47/d8a/de9/d101/f8e to d6/d75/dc5/d57/d9d/daf/f102 0 2024-09-21T04:35:33.782 INFO:tasks.workunit.client.0.smithi007.stdout:4/734: rmdir d2/d17/d18/dcd 0 2024-09-21T04:35:33.783 INFO:tasks.workunit.client.0.smithi007.stdout:5/760: rename d2/d1e/f37 to d2/d27/d42/da3/d23/d5d/d9a/ff0 0 2024-09-21T04:35:33.785 INFO:tasks.workunit.client.0.smithi007.stdout:0/685: creat d6/d75/dc5/d52/d6d/de1/f103 x:0 0 0 2024-09-21T04:35:33.786 INFO:tasks.workunit.client.0.smithi007.stdout:0/686: dread - d6/d75/dc5/d57/fe3 zero size 2024-09-21T04:35:33.788 INFO:tasks.workunit.client.0.smithi007.stdout:9/699: symlink d13/d18/daf/dda/le0 0 2024-09-21T04:35:33.789 INFO:tasks.workunit.client.0.smithi007.stdout:1/854: creat d5/d6/f112 x:0 0 0 2024-09-21T04:35:33.790 INFO:tasks.workunit.client.0.smithi007.stdout:7/656: link d3/d6/d10/c52 d3/d6/d2e/d63/d6a/cea 0 2024-09-21T04:35:33.792 INFO:tasks.workunit.client.0.smithi007.stdout:4/735: rmdir d2/d3c/d63 39 2024-09-21T04:35:33.793 INFO:tasks.workunit.client.0.smithi007.stdout:2/527: write d0/d2/fc [1142000,109926] 0 2024-09-21T04:35:33.793 INFO:tasks.workunit.client.0.smithi007.stdout:5/761: creat d2/d27/d42/da3/d23/d5d/d61/ff1 x:0 0 0 2024-09-21T04:35:33.795 INFO:tasks.workunit.client.0.smithi007.stdout:6/634: link d3/d3e/d99/dbb/l5e d3/d3e/d99/ld4 0 2024-09-21T04:35:33.796 INFO:tasks.workunit.client.0.smithi007.stdout:0/687: truncate d6/d91/da3/dbf/fc3 3152074 0 2024-09-21T04:35:33.796 INFO:tasks.workunit.client.0.smithi007.stdout:5/762: chown d2/d27/d42/da3/d23/d2a/c30 0 1 2024-09-21T04:35:33.797 INFO:tasks.workunit.client.0.smithi007.stdout:9/700: mkdir d13/d74/de1 0 2024-09-21T04:35:33.800 INFO:tasks.workunit.client.0.smithi007.stdout:4/736: rmdir d2/d17/d2e/d45/d5d/dc7 39 2024-09-21T04:35:33.807 INFO:tasks.workunit.client.0.smithi007.stdout:1/855: dwrite d5/d2f/f95 [0,4194304] 0 2024-09-21T04:35:33.811 INFO:tasks.workunit.client.0.smithi007.stdout:0/688: read d6/d75/dc5/d18/d27/f3a [2963033,6228] 0 2024-09-21T04:35:33.811 INFO:tasks.workunit.client.0.smithi007.stdout:1/856: readlink d5/d6/d21/d64/df1/da0/lf0 0 2024-09-21T04:35:33.811 INFO:tasks.workunit.client.0.smithi007.stdout:5/763: dread d2/d27/d42/da3/d23/d2a/f63 [0,4194304] 0 2024-09-21T04:35:33.812 INFO:tasks.workunit.client.0.smithi007.stdout:5/764: stat d2/d27/fc8 0 2024-09-21T04:35:33.812 INFO:tasks.workunit.client.0.smithi007.stdout:6/635: dread d3/de/d12/d19/f20 [0,4194304] 0 2024-09-21T04:35:33.815 INFO:tasks.workunit.client.0.smithi007.stdout:6/636: dread - d3/d3b/dd3/d60/d77/dcb/fa6 zero size 2024-09-21T04:35:33.817 INFO:tasks.workunit.client.0.smithi007.stdout:2/528: link d0/d11/d31/d46/d77/f91 d0/d11/f95 0 2024-09-21T04:35:33.820 INFO:tasks.workunit.client.0.smithi007.stdout:1/857: chown d5/d6/d21/d2b/d4b/f66 29853 1 2024-09-21T04:35:33.821 INFO:tasks.workunit.client.0.smithi007.stdout:0/689: truncate d6/d91/da3/f9a 2539357 0 2024-09-21T04:35:33.822 INFO:tasks.workunit.client.0.smithi007.stdout:0/690: stat d6/d75/dc5/d57/d9d/daf/cdf 0 2024-09-21T04:35:33.825 INFO:tasks.workunit.client.0.smithi007.stdout:6/637: symlink d3/d3b/dd3/d60/d77/dcb/d78/ld5 0 2024-09-21T04:35:33.826 INFO:tasks.workunit.client.0.smithi007.stdout:2/529: unlink d0/d11/d31/d52/c70 0 2024-09-21T04:35:33.827 INFO:tasks.workunit.client.0.smithi007.stdout:2/530: dread - d0/f7d zero size 2024-09-21T04:35:33.828 INFO:tasks.workunit.client.0.smithi007.stdout:0/691: dwrite d6/d75/dc5/d52/d6d/fa6 [0,4194304] 0 2024-09-21T04:35:33.829 INFO:tasks.workunit.client.0.smithi007.stdout:5/765: rename d2/d27/d42/l79 to d2/d27/d42/da3/d1d/d20/lf2 0 2024-09-21T04:35:33.833 INFO:tasks.workunit.client.0.smithi007.stdout:6/638: read d3/de/d12/f21 [1013276,25285] 0 2024-09-21T04:35:33.836 INFO:tasks.workunit.client.0.smithi007.stdout:5/766: write d2/d27/d42/da3/d23/fe7 [834876,103127] 0 2024-09-21T04:35:33.837 INFO:tasks.workunit.client.0.smithi007.stdout:9/701: write d13/d18/d60/d83/da9/f40 [4774067,112572] 0 2024-09-21T04:35:33.846 INFO:tasks.workunit.client.0.smithi007.stdout:7/657: dwrite d3/d6/d13/f75 [0,4194304] 0 2024-09-21T04:35:33.847 INFO:tasks.workunit.client.0.smithi007.stdout:9/702: write d13/d18/d60/d6d/f9d [4662485,32090] 0 2024-09-21T04:35:33.852 INFO:tasks.workunit.client.0.smithi007.stdout:7/658: creat d3/d6/d14/d20/d58/feb x:0 0 0 2024-09-21T04:35:33.852 INFO:tasks.workunit.client.0.smithi007.stdout:2/531: dread d0/d11/d6e/f90 [0,4194304] 0 2024-09-21T04:35:33.853 INFO:tasks.workunit.client.0.smithi007.stdout:2/532: chown d0/d2 1 1 2024-09-21T04:35:33.854 INFO:tasks.workunit.client.0.smithi007.stdout:9/703: symlink d13/d18/d60/d6d/d85/le2 0 2024-09-21T04:35:33.854 INFO:tasks.workunit.client.0.smithi007.stdout:2/533: chown d0/d11/d31/d46/d77 0 1 2024-09-21T04:35:33.854 INFO:tasks.workunit.client.0.smithi007.stdout:6/639: dread d3/de/d12/f7c [0,4194304] 0 2024-09-21T04:35:33.855 INFO:tasks.workunit.client.0.smithi007.stdout:5/767: mkdir d2/d27/d42/da3/d23/d49/d6f/ddb/def/df3 0 2024-09-21T04:35:33.856 INFO:tasks.workunit.client.0.smithi007.stdout:6/640: dread - d3/d3e/d99/dbb/f9a zero size 2024-09-21T04:35:33.856 INFO:tasks.workunit.client.0.smithi007.stdout:5/768: readlink d2/d27/d3d/d9e/la8 0 2024-09-21T04:35:33.857 INFO:tasks.workunit.client.0.smithi007.stdout:2/534: chown d0/d11/d31/d52/c5b 66 1 2024-09-21T04:35:33.859 INFO:tasks.workunit.client.0.smithi007.stdout:2/535: write d0/d11/d31/d39/f4f [7057652,12003] 0 2024-09-21T04:35:33.861 INFO:tasks.workunit.client.0.smithi007.stdout:9/704: symlink d13/d89/le3 0 2024-09-21T04:35:33.865 INFO:tasks.workunit.client.0.smithi007.stdout:7/659: fdatasync d3/d6/d2e/f6d 0 2024-09-21T04:35:33.866 INFO:tasks.workunit.client.0.smithi007.stdout:0/692: dwrite d6/d91/da3/f8c [0,4194304] 0 2024-09-21T04:35:33.868 INFO:tasks.workunit.client.0.smithi007.stdout:6/641: readlink d3/de/d12/l2e 0 2024-09-21T04:35:33.870 INFO:tasks.workunit.client.0.smithi007.stdout:9/705: mknod d13/d5a/ce4 0 2024-09-21T04:35:33.882 INFO:tasks.workunit.client.0.smithi007.stdout:9/706: creat d13/fe5 x:0 0 0 2024-09-21T04:35:33.887 INFO:tasks.workunit.client.0.smithi007.stdout:6/642: mkdir d3/d3b/dd3/dd6 0 2024-09-21T04:35:33.890 INFO:tasks.workunit.client.0.smithi007.stdout:6/643: unlink d3/d3e/c56 0 2024-09-21T04:35:33.892 INFO:tasks.workunit.client.0.smithi007.stdout:7/660: write d3/f88 [277292,5000] 0 2024-09-21T04:35:33.893 INFO:tasks.workunit.client.0.smithi007.stdout:0/693: dwrite d6/d75/dc5/d18/d39/fe0 [0,4194304] 0 2024-09-21T04:35:33.896 INFO:tasks.workunit.client.0.smithi007.stdout:7/661: read d3/d6/d14/f18 [527587,16256] 0 2024-09-21T04:35:33.898 INFO:tasks.workunit.client.0.smithi007.stdout:6/644: getdents d3/d94 0 2024-09-21T04:35:33.903 INFO:tasks.workunit.client.0.smithi007.stdout:6/645: mkdir d3/d3b/dd3/d60/dd7 0 2024-09-21T04:35:33.906 INFO:tasks.workunit.client.0.smithi007.stdout:0/694: rename d6/d75/dc5/d18/c42 to d6/d75/dc5/d18/d2e/d3f/c104 0 2024-09-21T04:35:33.907 INFO:tasks.workunit.client.0.smithi007.stdout:6/646: mkdir d3/de/d12/d19/dd8 0 2024-09-21T04:35:33.908 INFO:tasks.workunit.client.0.smithi007.stdout:6/647: truncate d3/d94/fc2 444227 0 2024-09-21T04:35:33.909 INFO:tasks.workunit.client.0.smithi007.stdout:0/695: read d6/d75/dc5/f86 [3896221,64152] 0 2024-09-21T04:35:33.911 INFO:tasks.workunit.client.0.smithi007.stdout:7/662: dread d3/d6/d2e/d63/d6a/f70 [0,4194304] 0 2024-09-21T04:35:33.914 INFO:tasks.workunit.client.0.smithi007.stdout:3/784: sync 2024-09-21T04:35:33.914 INFO:tasks.workunit.client.0.smithi007.stdout:8/457: sync 2024-09-21T04:35:33.915 INFO:tasks.workunit.client.0.smithi007.stdout:7/663: creat d3/d6/d14/d20/d80/fec x:0 0 0 2024-09-21T04:35:33.917 INFO:tasks.workunit.client.0.smithi007.stdout:6/648: dwrite d3/d3b/dd3/d60/d77/f63 [0,4194304] 0 2024-09-21T04:35:33.924 INFO:tasks.workunit.client.0.smithi007.stdout:7/664: symlink d3/d6/d2e/d63/d6a/d81/led 0 2024-09-21T04:35:33.926 INFO:tasks.workunit.client.0.smithi007.stdout:6/649: mkdir d3/d3e/d99/dbb/d95/dc9/dd9 0 2024-09-21T04:35:33.928 INFO:tasks.workunit.client.0.smithi007.stdout:3/785: link dc/f1c dc/d6c/ff7 0 2024-09-21T04:35:33.930 INFO:tasks.workunit.client.0.smithi007.stdout:7/665: mknod d3/d6/d10/d62/cee 0 2024-09-21T04:35:33.931 INFO:tasks.workunit.client.0.smithi007.stdout:6/650: creat d3/de/dba/fda x:0 0 0 2024-09-21T04:35:33.934 INFO:tasks.workunit.client.0.smithi007.stdout:7/666: creat d3/d6/d2e/d5e/fef x:0 0 0 2024-09-21T04:35:33.935 INFO:tasks.workunit.client.0.smithi007.stdout:3/786: dwrite dc/d22/d5a/fd8 [0,4194304] 0 2024-09-21T04:35:33.937 INFO:tasks.workunit.client.0.smithi007.stdout:0/696: write d6/d75/dc5/d47/fa0 [748658,56300] 0 2024-09-21T04:35:33.937 INFO:tasks.workunit.client.0.smithi007.stdout:7/667: chown d3/d6/d40/da1/cb4 0 1 2024-09-21T04:35:33.940 INFO:tasks.workunit.client.0.smithi007.stdout:8/458: write d0/d3/d8/de/d40/f45 [1653844,79215] 0 2024-09-21T04:35:33.941 INFO:tasks.workunit.client.0.smithi007.stdout:7/668: dread - d3/d6/d2e/d63/fde zero size 2024-09-21T04:35:33.943 INFO:tasks.workunit.client.0.smithi007.stdout:7/669: readlink d3/d6/d14/d20/l98 0 2024-09-21T04:35:33.944 INFO:tasks.workunit.client.0.smithi007.stdout:6/651: dwrite d3/de/d58/faa [0,4194304] 0 2024-09-21T04:35:33.946 INFO:tasks.workunit.client.0.smithi007.stdout:0/697: symlink d6/d75/dc5/d47/d8a/de9/l105 0 2024-09-21T04:35:33.948 INFO:tasks.workunit.client.0.smithi007.stdout:8/459: creat d0/d3/d8/de/d5e/f81 x:0 0 0 2024-09-21T04:35:33.948 INFO:tasks.workunit.client.0.smithi007.stdout:0/698: chown d6/d75/dc5/d18/d2e/d60/lad 226 1 2024-09-21T04:35:33.952 INFO:tasks.workunit.client.0.smithi007.stdout:6/652: mknod d3/d3e/d99/dbb/d95/cdb 0 2024-09-21T04:35:33.952 INFO:tasks.workunit.client.0.smithi007.stdout:6/653: readlink d3/d94/la8 0 2024-09-21T04:35:33.959 INFO:tasks.workunit.client.0.smithi007.stdout:8/460: mkdir d0/d3/d82 0 2024-09-21T04:35:33.960 INFO:tasks.workunit.client.0.smithi007.stdout:0/699: mknod d6/d75/dc5/d18/d2e/c106 0 2024-09-21T04:35:33.964 INFO:tasks.workunit.client.0.smithi007.stdout:7/670: dwrite d3/d6/d14/f18 [0,4194304] 0 2024-09-21T04:35:33.964 INFO:tasks.workunit.client.0.smithi007.stdout:8/461: dread d0/d3/d8/f5f [0,4194304] 0 2024-09-21T04:35:33.964 INFO:tasks.workunit.client.0.smithi007.stdout:8/462: chown d0/d3/d8/f67 121532020 1 2024-09-21T04:35:34.068 INFO:tasks.workunit.client.0.smithi007.stdout:8/463: readlink d0/d3/d16/l30 0 2024-09-21T04:35:34.070 INFO:tasks.workunit.client.0.smithi007.stdout:7/671: symlink d3/d6/d13/d4b/lf0 0 2024-09-21T04:35:34.145 INFO:tasks.workunit.client.0.smithi007.stdout:6/654: write d3/de/f10 [3434252,92488] 0 2024-09-21T04:35:34.145 INFO:tasks.workunit.client.0.smithi007.stdout:0/700: symlink d6/d75/dc5/d18/d2e/l107 0 2024-09-21T04:35:34.147 INFO:tasks.workunit.client.0.smithi007.stdout:8/464: mknod d0/d3/d82/c83 0 2024-09-21T04:35:34.147 INFO:tasks.workunit.client.0.smithi007.stdout:7/672: read d3/f4c [3552011,129937] 0 2024-09-21T04:35:34.149 INFO:tasks.workunit.client.0.smithi007.stdout:8/465: chown d0/d3/d8/de/d26/d4d/l57 182 1 2024-09-21T04:35:34.149 INFO:tasks.workunit.client.0.smithi007.stdout:0/701: creat d6/d75/dc5/d18/d27/f108 x:0 0 0 2024-09-21T04:35:34.150 INFO:tasks.workunit.client.0.smithi007.stdout:8/466: write d0/d3/d7c/f54 [852324,207] 0 2024-09-21T04:35:34.152 INFO:tasks.workunit.client.0.smithi007.stdout:0/702: rmdir d6/d75/dc5/d52/d6d/de1 39 2024-09-21T04:35:34.156 INFO:tasks.workunit.client.0.smithi007.stdout:8/467: readlink d0/d3/d8/l14 0 2024-09-21T04:35:34.158 INFO:tasks.workunit.client.0.smithi007.stdout:7/673: readlink d3/d6/d10/d62/d66/l7a 0 2024-09-21T04:35:34.164 INFO:tasks.workunit.client.0.smithi007.stdout:6/655: dwrite d3/d3b/dd3/d60/f93 [0,4194304] 0 2024-09-21T04:35:34.168 INFO:tasks.workunit.client.0.smithi007.stdout:6/656: write d3/d3b/dd3/d60/d77/dcb/fa6 [754355,105832] 0 2024-09-21T04:35:34.170 INFO:tasks.workunit.client.0.smithi007.stdout:7/674: getdents d3/d6/d13 0 2024-09-21T04:35:34.170 INFO:tasks.workunit.client.0.smithi007.stdout:8/468: dwrite d0/d3/d8/f67 [0,4194304] 0 2024-09-21T04:35:34.174 INFO:tasks.workunit.client.0.smithi007.stdout:7/675: write d3/d6/d14/d20/d80/f9d [675336,96477] 0 2024-09-21T04:35:34.175 INFO:tasks.workunit.client.0.smithi007.stdout:8/469: mknod d0/d3/d8/de/d26/d4d/d5d/c84 0 2024-09-21T04:35:34.176 INFO:tasks.workunit.client.0.smithi007.stdout:7/676: readlink d3/d6/d10/ld9 0 2024-09-21T04:35:34.177 INFO:tasks.workunit.client.0.smithi007.stdout:0/703: dread d6/d75/dc5/d47/d8a/de9/d101/fb0 [0,4194304] 0 2024-09-21T04:35:34.180 INFO:tasks.workunit.client.0.smithi007.stdout:7/677: rmdir d3/d6/d10/d62 39 2024-09-21T04:35:34.181 INFO:tasks.workunit.client.0.smithi007.stdout:6/657: dwrite d3/de/d12/f21 [0,4194304] 0 2024-09-21T04:35:34.182 INFO:tasks.workunit.client.0.smithi007.stdout:6/658: chown d3/d3b/dd3/d60/d98/faf 1391448 1 2024-09-21T04:35:34.183 INFO:tasks.workunit.client.0.smithi007.stdout:7/678: chown d3/d6/d10/d51/cd3 15563389 1 2024-09-21T04:35:34.184 INFO:tasks.workunit.client.0.smithi007.stdout:6/659: truncate d3/d3e/fd1 562893 0 2024-09-21T04:35:34.186 INFO:tasks.workunit.client.0.smithi007.stdout:8/470: link d0/f4f d0/d3/d8/de/d26/f85 0 2024-09-21T04:35:34.192 INFO:tasks.workunit.client.0.smithi007.stdout:8/471: creat d0/d3/d8/de/d26/d4d/d5d/f86 x:0 0 0 2024-09-21T04:35:34.193 INFO:tasks.workunit.client.0.smithi007.stdout:6/660: dwrite d3/de/d12/f21 [0,4194304] 0 2024-09-21T04:35:34.193 INFO:tasks.workunit.client.0.smithi007.stdout:8/472: write d0/f6f [1647013,51177] 0 2024-09-21T04:35:34.197 INFO:tasks.workunit.client.0.smithi007.stdout:6/661: symlink d3/d3b/dd3/d60/d77/dcb/d78/ldc 0 2024-09-21T04:35:34.198 INFO:tasks.workunit.client.0.smithi007.stdout:6/662: chown d3/d3b/dd3/d60/d77 0 1 2024-09-21T04:35:34.198 INFO:tasks.workunit.client.0.smithi007.stdout:6/663: chown d3/f1e 30581726 1 2024-09-21T04:35:34.199 INFO:tasks.workunit.client.0.smithi007.stdout:8/473: creat d0/d3/d8/de/d26/d35/f87 x:0 0 0 2024-09-21T04:35:34.201 INFO:tasks.workunit.client.0.smithi007.stdout:0/704: write d6/d75/dc5/d57/d9d/daf/f102 [1958870,34473] 0 2024-09-21T04:35:34.202 INFO:tasks.workunit.client.0.smithi007.stdout:8/474: rmdir d0/d3/d74 39 2024-09-21T04:35:34.205 INFO:tasks.workunit.client.0.smithi007.stdout:0/705: fdatasync d6/d75/dc5/d57/d9d/fc2 0 2024-09-21T04:35:34.207 INFO:tasks.workunit.client.0.smithi007.stdout:8/475: mkdir d0/d3/d8/de/d26/d4d/d5d/d88 0 2024-09-21T04:35:34.215 INFO:tasks.workunit.client.0.smithi007.stdout:8/476: getdents d0 0 2024-09-21T04:35:34.217 INFO:tasks.workunit.client.0.smithi007.stdout:6/664: dwrite d3/f15 [0,4194304] 0 2024-09-21T04:35:34.220 INFO:tasks.workunit.client.0.smithi007.stdout:0/706: truncate d6/d75/dc5/d52/fb8 1515231 0 2024-09-21T04:35:34.222 INFO:tasks.workunit.client.0.smithi007.stdout:0/707: chown d6/d75/dc5/d18/d39/dd5/fda 626 1 2024-09-21T04:35:34.223 INFO:tasks.workunit.client.0.smithi007.stdout:6/665: unlink d3/d3e/d99/dbb/f57 0 2024-09-21T04:35:34.224 INFO:tasks.workunit.client.0.smithi007.stdout:8/477: dwrite d0/d3/d8/de/d5e/f81 [0,4194304] 0 2024-09-21T04:35:34.225 INFO:tasks.workunit.client.0.smithi007.stdout:6/666: write d3/fc [1359117,5481] 0 2024-09-21T04:35:34.228 INFO:tasks.workunit.client.0.smithi007.stdout:6/667: chown d3/d3e/d99/dbb/d95/dc9/c8c 3125 1 2024-09-21T04:35:34.231 INFO:tasks.workunit.client.0.smithi007.stdout:8/478: symlink d0/d3/d8/de/d26/d4d/d5d/l89 0 2024-09-21T04:35:34.233 INFO:tasks.workunit.client.0.smithi007.stdout:6/668: unlink d3/de/dba/fbe 0 2024-09-21T04:35:34.234 INFO:tasks.workunit.client.0.smithi007.stdout:8/479: truncate d0/d3/d8/de/f69 381666 0 2024-09-21T04:35:34.234 INFO:tasks.workunit.client.0.smithi007.stdout:6/669: chown d3/d8e/dae 209821106 1 2024-09-21T04:35:34.236 INFO:tasks.workunit.client.0.smithi007.stdout:8/480: read d0/d3/f2b [2741072,62017] 0 2024-09-21T04:35:34.240 INFO:tasks.workunit.client.0.smithi007.stdout:8/481: mknod d0/d3/d8/de/d26/d35/c8a 0 2024-09-21T04:35:34.241 INFO:tasks.workunit.client.0.smithi007.stdout:6/670: symlink d3/d3e/ldd 0 2024-09-21T04:35:34.246 INFO:tasks.workunit.client.0.smithi007.stdout:6/671: creat d3/d3b/dd3/d60/d77/dcb/d88/fde x:0 0 0 2024-09-21T04:35:34.247 INFO:tasks.workunit.client.0.smithi007.stdout:0/708: dwrite d6/d75/dc5/f22 [0,4194304] 0 2024-09-21T04:35:34.254 INFO:tasks.workunit.client.0.smithi007.stdout:8/482: write d0/d3/d8/f5f [1587817,66463] 0 2024-09-21T04:35:34.257 INFO:tasks.workunit.client.0.smithi007.stdout:8/483: read d0/f6f [1399449,62792] 0 2024-09-21T04:35:34.263 INFO:tasks.workunit.client.0.smithi007.stdout:6/672: dwrite d3/de/d12/f8d [0,4194304] 0 2024-09-21T04:35:34.265 INFO:tasks.workunit.client.0.smithi007.stdout:0/709: truncate d6/d75/dc5/d52/fb8 2283956 0 2024-09-21T04:35:34.267 INFO:tasks.workunit.client.0.smithi007.stdout:6/673: mknod d3/de/dba/cdf 0 2024-09-21T04:35:34.267 INFO:tasks.workunit.client.0.smithi007.stdout:6/674: chown d3/de/d58/l89 16 1 2024-09-21T04:35:34.269 INFO:tasks.workunit.client.0.smithi007.stdout:8/484: getdents d0/d3/d8/de/d5e 0 2024-09-21T04:35:34.270 INFO:tasks.workunit.client.0.smithi007.stdout:8/485: readlink d0/d3/d8/de/d26/l5b 0 2024-09-21T04:35:34.273 INFO:tasks.workunit.client.0.smithi007.stdout:8/486: unlink d0/d3/d8/de/d26/d4d/d5d/l6b 0 2024-09-21T04:35:34.274 INFO:tasks.workunit.client.0.smithi007.stdout:6/675: getdents d3/d3b/dd3/d60/dd7 0 2024-09-21T04:35:34.276 INFO:tasks.workunit.client.0.smithi007.stdout:0/710: write d6/d75/dc5/d57/f67 [1714491,115262] 0 2024-09-21T04:35:34.277 INFO:tasks.workunit.client.0.smithi007.stdout:0/711: chown d6/d75/dc5/d47 37761654 1 2024-09-21T04:35:34.278 INFO:tasks.workunit.client.0.smithi007.stdout:8/487: symlink d0/d3/l8b 0 2024-09-21T04:35:34.280 INFO:tasks.workunit.client.0.smithi007.stdout:6/676: mkdir d3/de/de0 0 2024-09-21T04:35:34.285 INFO:tasks.workunit.client.0.smithi007.stdout:6/677: dread d3/d3e/d99/dbb/f9a [0,4194304] 0 2024-09-21T04:35:34.285 INFO:tasks.workunit.client.0.smithi007.stdout:8/488: dread d0/d3/d8/f47 [0,4194304] 0 2024-09-21T04:35:34.287 INFO:tasks.workunit.client.0.smithi007.stdout:8/489: symlink d0/d3/d7c/l8c 0 2024-09-21T04:35:34.290 INFO:tasks.workunit.client.0.smithi007.stdout:6/678: getdents d3/d3b/dd3/d60/dd2 0 2024-09-21T04:35:34.294 INFO:tasks.workunit.client.0.smithi007.stdout:0/712: dwrite d6/d75/dc5/d52/f81 [0,4194304] 0 2024-09-21T04:35:34.297 INFO:tasks.workunit.client.0.smithi007.stdout:6/679: creat d3/d3b/dd3/d60/dd2/fe1 x:0 0 0 2024-09-21T04:35:34.299 INFO:tasks.workunit.client.0.smithi007.stdout:8/490: dwrite d0/d3/d8/de/d26/d4d/f72 [4194304,4194304] 0 2024-09-21T04:35:34.300 INFO:tasks.workunit.client.0.smithi007.stdout:0/713: mkdir d6/d75/dc5/d47/d8a/de9/d101/d109 0 2024-09-21T04:35:34.307 INFO:tasks.workunit.client.0.smithi007.stdout:0/714: fsync d6/d75/dc5/d18/d39/f6a 0 2024-09-21T04:35:34.313 INFO:tasks.workunit.client.0.smithi007.stdout:6/680: creat d3/d3b/dd3/fe2 x:0 0 0 2024-09-21T04:35:34.323 INFO:tasks.workunit.client.0.smithi007.stdout:0/715: dread d6/d75/dc5/d47/d8a/fd6 [0,4194304] 0 2024-09-21T04:35:34.327 INFO:tasks.workunit.client.0.smithi007.stdout:8/491: truncate d0/d3/d8/de/d5e/f61 3684052 0 2024-09-21T04:35:34.331 INFO:tasks.workunit.client.0.smithi007.stdout:1/858: sync 2024-09-21T04:35:34.331 INFO:tasks.workunit.client.0.smithi007.stdout:4/737: sync 2024-09-21T04:35:34.331 INFO:tasks.workunit.client.0.smithi007.stdout:5/769: sync 2024-09-21T04:35:34.356 INFO:tasks.workunit.client.0.smithi007.stdout:8/492: creat d0/d3/d16/f8d x:0 0 0 2024-09-21T04:35:34.358 INFO:tasks.workunit.client.0.smithi007.stdout:8/493: write d0/d3/d8/d46/f7d [900145,59114] 0 2024-09-21T04:35:34.359 INFO:tasks.workunit.client.0.smithi007.stdout:1/859: creat d5/d6/d21/d2b/d4b/d8a/dee/f113 x:0 0 0 2024-09-21T04:35:34.363 INFO:tasks.workunit.client.0.smithi007.stdout:5/770: dread d2/d27/d42/da3/d23/d2a/f7a [0,4194304] 0 2024-09-21T04:35:34.364 INFO:tasks.workunit.client.0.smithi007.stdout:5/771: chown d2/d1e 9939 1 2024-09-21T04:35:34.366 INFO:tasks.workunit.client.0.smithi007.stdout:5/772: readlink d2/d27/d42/da3/d23/d5d/d61/l92 0 2024-09-21T04:35:34.367 INFO:tasks.workunit.client.0.smithi007.stdout:8/494: dwrite d0/d3/d8/de/f69 [0,4194304] 0 2024-09-21T04:35:34.368 INFO:tasks.workunit.client.0.smithi007.stdout:1/860: creat d5/d6/d21/d3c/dd5/f114 x:0 0 0 2024-09-21T04:35:34.369 INFO:tasks.workunit.client.0.smithi007.stdout:1/861: write d5/d2f/f95 [1023506,53513] 0 2024-09-21T04:35:34.370 INFO:tasks.workunit.client.0.smithi007.stdout:7/679: sync 2024-09-21T04:35:34.370 INFO:tasks.workunit.client.0.smithi007.stdout:9/707: sync 2024-09-21T04:35:34.370 INFO:tasks.workunit.client.0.smithi007.stdout:2/536: sync 2024-09-21T04:35:34.370 INFO:tasks.workunit.client.0.smithi007.stdout:3/787: sync 2024-09-21T04:35:34.372 INFO:tasks.workunit.client.0.smithi007.stdout:6/681: getdents d3/d3b/dd3/d60/d77/dcb/d88 0 2024-09-21T04:35:34.373 INFO:tasks.workunit.client.0.smithi007.stdout:3/788: write dc/d22/d29/d32/fe5 [164281,16762] 0 2024-09-21T04:35:34.381 INFO:tasks.workunit.client.0.smithi007.stdout:1/862: creat d5/d6/d21/d3c/db6/f115 x:0 0 0 2024-09-21T04:35:34.382 INFO:tasks.workunit.client.0.smithi007.stdout:4/738: getdents d2/d88/da8/db2 0 2024-09-21T04:35:34.384 INFO:tasks.workunit.client.0.smithi007.stdout:3/789: creat dc/d6c/da4/ff8 x:0 0 0 2024-09-21T04:35:34.385 INFO:tasks.workunit.client.0.smithi007.stdout:3/790: stat dc/d18/c1f 0 2024-09-21T04:35:34.386 INFO:tasks.workunit.client.0.smithi007.stdout:6/682: mkdir d3/de/d12/d19/da3/de3 0 2024-09-21T04:35:34.387 INFO:tasks.workunit.client.0.smithi007.stdout:2/537: dwrite d0/f37 [4194304,4194304] 0 2024-09-21T04:35:34.389 INFO:tasks.workunit.client.0.smithi007.stdout:7/680: symlink d3/lf1 0 2024-09-21T04:35:34.393 INFO:tasks.workunit.client.0.smithi007.stdout:3/791: creat dc/d22/d5a/d71/ff9 x:0 0 0 2024-09-21T04:35:34.397 INFO:tasks.workunit.client.0.smithi007.stdout:8/495: getdents d0/d3/d8/de/d40/d6c 0 2024-09-21T04:35:34.398 INFO:tasks.workunit.client.0.smithi007.stdout:2/538: unlink d0/d32/l63 0 2024-09-21T04:35:34.404 INFO:tasks.workunit.client.0.smithi007.stdout:2/539: symlink d0/d11/d6b/l96 0 2024-09-21T04:35:34.404 INFO:tasks.workunit.client.0.smithi007.stdout:9/708: link d13/d18/f27 d13/d18/d60/d6d/fe6 0 2024-09-21T04:35:34.405 INFO:tasks.workunit.client.0.smithi007.stdout:8/496: dwrite d0/d3/d8/de/d26/d4d/f72 [4194304,4194304] 0 2024-09-21T04:35:34.407 INFO:tasks.workunit.client.0.smithi007.stdout:7/681: getdents d3/d6/d2e/d63/d6a 0 2024-09-21T04:35:34.408 INFO:tasks.workunit.client.0.smithi007.stdout:9/709: creat d13/d18/dcd/fe7 x:0 0 0 2024-09-21T04:35:34.410 INFO:tasks.workunit.client.0.smithi007.stdout:2/540: symlink d0/d11/d31/d46/d6c/l97 0 2024-09-21T04:35:34.414 INFO:tasks.workunit.client.0.smithi007.stdout:9/710: unlink d13/d18/d60/d83/fa8 0 2024-09-21T04:35:34.415 INFO:tasks.workunit.client.0.smithi007.stdout:2/541: mknod d0/d11/d31/d52/c98 0 2024-09-21T04:35:34.415 INFO:tasks.workunit.client.0.smithi007.stdout:3/792: dread dc/d18/f45 [4194304,4194304] 0 2024-09-21T04:35:34.415 INFO:tasks.workunit.client.0.smithi007.stdout:8/497: dread d0/d3/d8/d46/f7d [0,4194304] 0 2024-09-21T04:35:34.419 INFO:tasks.workunit.client.0.smithi007.stdout:9/711: symlink d13/d18/d60/d6d/d75/d9f/le8 0 2024-09-21T04:35:34.419 INFO:tasks.workunit.client.0.smithi007.stdout:7/682: dwrite d3/d6/d10/d51/fd1 [4194304,4194304] 0 2024-09-21T04:35:34.421 INFO:tasks.workunit.client.0.smithi007.stdout:8/498: creat d0/d3/d8/de/d26/d35/f8e x:0 0 0 2024-09-21T04:35:34.425 INFO:tasks.workunit.client.0.smithi007.stdout:2/542: creat d0/d11/f99 x:0 0 0 2024-09-21T04:35:34.426 INFO:tasks.workunit.client.0.smithi007.stdout:9/712: mknod d13/d18/daf/dc0/ce9 0 2024-09-21T04:35:34.428 INFO:tasks.workunit.client.0.smithi007.stdout:8/499: rename d0/l24 to d0/d3/d8/d46/l8f 0 2024-09-21T04:35:34.432 INFO:tasks.workunit.client.0.smithi007.stdout:9/713: mkdir d13/d18/d80/dea 0 2024-09-21T04:35:34.434 INFO:tasks.workunit.client.0.smithi007.stdout:3/793: rmdir dc/d22/d29/d32/d4d/d84/dd0/da6/de3/da8 0 2024-09-21T04:35:34.440 INFO:tasks.workunit.client.0.smithi007.stdout:9/714: dwrite d13/d18/d60/d6d/d75/fd8 [0,4194304] 0 2024-09-21T04:35:34.440 INFO:tasks.workunit.client.0.smithi007.stdout:7/683: getdents d3/d6/d2e 0 2024-09-21T04:35:34.445 INFO:tasks.workunit.client.0.smithi007.stdout:9/715: creat d13/d18/d60/d6d/d75/d9f/feb x:0 0 0 2024-09-21T04:35:34.446 INFO:tasks.workunit.client.0.smithi007.stdout:7/684: creat d3/d6/d85/de9/ff2 x:0 0 0 2024-09-21T04:35:34.449 INFO:tasks.workunit.client.0.smithi007.stdout:9/716: mkdir d13/d74/de1/dec 0 2024-09-21T04:35:34.454 INFO:tasks.workunit.client.0.smithi007.stdout:9/717: creat d13/d89/fed x:0 0 0 2024-09-21T04:35:34.458 INFO:tasks.workunit.client.0.smithi007.stdout:9/718: unlink d13/d18/d60/d6d/c9b 0 2024-09-21T04:35:34.459 INFO:tasks.workunit.client.0.smithi007.stdout:9/719: readlink d13/d18/d60/d6d/l8b 0 2024-09-21T04:35:34.466 INFO:tasks.workunit.client.0.smithi007.stdout:9/720: creat d13/d18/fee x:0 0 0 2024-09-21T04:35:34.469 INFO:tasks.workunit.client.0.smithi007.stdout:0/716: write d6/fd4 [1533868,80489] 0 2024-09-21T04:35:34.477 INFO:tasks.workunit.client.0.smithi007.stdout:9/721: link d13/d18/d60/d6d/d75/f7e d13/d18/d60/d83/da9/fef 0 2024-09-21T04:35:34.479 INFO:tasks.workunit.client.0.smithi007.stdout:0/717: dwrite d6/d75/dc5/d47/d8a/fc4 [0,4194304] 0 2024-09-21T04:35:34.491 INFO:tasks.workunit.client.0.smithi007.stdout:9/722: rename f10 to d13/d18/d60/d6d/d75/d9f/ff0 0 2024-09-21T04:35:34.491 INFO:tasks.workunit.client.0.smithi007.stdout:5/773: dwrite d2/f70 [0,4194304] 0 2024-09-21T04:35:34.492 INFO:tasks.workunit.client.0.smithi007.stdout:9/723: dread - d13/d18/d60/d6d/d75/fd2 zero size 2024-09-21T04:35:34.493 INFO:tasks.workunit.client.0.smithi007.stdout:5/774: chown d2/l7e 57882646 1 2024-09-21T04:35:34.495 INFO:tasks.workunit.client.0.smithi007.stdout:5/775: dread - d2/d27/d42/da3/d23/d49/dd4/de9/feb zero size 2024-09-21T04:35:34.497 INFO:tasks.workunit.client.0.smithi007.stdout:5/776: chown d2/d27/d42/d53/daf/dcd 21664919 1 2024-09-21T04:35:34.497 INFO:tasks.workunit.client.0.smithi007.stdout:5/777: chown d2/d27/dae 160569950 1 2024-09-21T04:35:34.499 INFO:tasks.workunit.client.0.smithi007.stdout:0/718: mkdir d6/d75/dc5/d52/d10a 0 2024-09-21T04:35:34.500 INFO:tasks.workunit.client.0.smithi007.stdout:0/719: chown d6/d75/dc5/d57/fa5 10750845 1 2024-09-21T04:35:34.501 INFO:tasks.workunit.client.0.smithi007.stdout:0/720: chown d6 12389 1 2024-09-21T04:35:34.504 INFO:tasks.workunit.client.0.smithi007.stdout:5/778: read d2/d27/d42/da3/f16 [10483591,80422] 0 2024-09-21T04:35:34.505 INFO:tasks.workunit.client.0.smithi007.stdout:4/739: write d2/f43 [480494,122271] 0 2024-09-21T04:35:34.515 INFO:tasks.workunit.client.0.smithi007.stdout:5/779: creat d2/d27/d42/da3/d23/d49/ff4 x:0 0 0 2024-09-21T04:35:34.520 INFO:tasks.workunit.client.0.smithi007.stdout:5/780: rmdir d2/d27/d42/da3/d23/d5d/d61 39 2024-09-21T04:35:34.528 INFO:tasks.workunit.client.0.smithi007.stdout:5/781: mknod d2/d27/d42/da3/d23/d49/d6f/ddb/def/df3/cf5 0 2024-09-21T04:35:34.536 INFO:tasks.workunit.client.0.smithi007.stdout:5/782: link d2/d27/d42/da3/fb9 d2/d27/d42/da3/d23/d49/dd4/ff6 0 2024-09-21T04:35:34.537 INFO:tasks.workunit.client.0.smithi007.stdout:8/500: symlink d0/d3/d8/de/d40/l90 0 2024-09-21T04:35:34.540 INFO:tasks.workunit.client.0.smithi007.stdout:5/783: fsync d2/d27/d42/da3/d23/d2a/f7a 0 2024-09-21T04:35:34.543 INFO:tasks.workunit.client.0.smithi007.stdout:8/501: creat d0/d3/d8/de/d26/d4d/f91 x:0 0 0 2024-09-21T04:35:34.547 INFO:tasks.workunit.client.0.smithi007.stdout:5/784: mkdir d2/d27/d42/da3/d23/d5d/dc6/df7 0 2024-09-21T04:35:34.549 INFO:tasks.workunit.client.0.smithi007.stdout:8/502: mkdir d0/d3/d8/de/d40/d92 0 2024-09-21T04:35:34.551 INFO:tasks.workunit.client.0.smithi007.stdout:2/543: dwrite d0/f8 [4194304,4194304] 0 2024-09-21T04:35:34.558 INFO:tasks.workunit.client.0.smithi007.stdout:3/794: dwrite dc/d22/d29/d32/fa3 [0,4194304] 0 2024-09-21T04:35:34.560 INFO:tasks.workunit.client.0.smithi007.stdout:7/685: truncate d3/d6/d14/d20/f35 2617718 0 2024-09-21T04:35:34.565 INFO:tasks.workunit.client.0.smithi007.stdout:7/686: mknod d3/d6/d10/d4f/cf3 0 2024-09-21T04:35:34.567 INFO:tasks.workunit.client.0.smithi007.stdout:7/687: read d3/d6/d85/dad/fb5 [302608,6291] 0 2024-09-21T04:35:34.571 INFO:tasks.workunit.client.0.smithi007.stdout:2/544: link d0/d11/l1e d0/d11/l9a 0 2024-09-21T04:35:34.575 INFO:tasks.workunit.client.0.smithi007.stdout:7/688: link d3/d6/d10/d51/l7c d3/d6/d14/d20/d80/dca/lf4 0 2024-09-21T04:35:34.577 INFO:tasks.workunit.client.0.smithi007.stdout:9/724: truncate d13/d18/d60/d6d/f9d 3529848 0 2024-09-21T04:35:34.578 INFO:tasks.workunit.client.0.smithi007.stdout:9/725: write d13/d18/dcd/fcf [1196897,69673] 0 2024-09-21T04:35:34.583 INFO:tasks.workunit.client.0.smithi007.stdout:7/689: creat d3/d6/d10/ff5 x:0 0 0 2024-09-21T04:35:34.587 INFO:tasks.workunit.client.0.smithi007.stdout:9/726: chown d13/l3e 39232 1 2024-09-21T04:35:34.587 INFO:tasks.workunit.client.0.smithi007.stdout:4/740: dwrite d2/d17/f26 [0,4194304] 0 2024-09-21T04:35:34.588 INFO:tasks.workunit.client.0.smithi007.stdout:9/727: read - d13/d18/daf/fdd zero size 2024-09-21T04:35:34.591 INFO:tasks.workunit.client.0.smithi007.stdout:9/728: truncate d13/d18/d60/d6d/fb2 340370 0 2024-09-21T04:35:34.594 INFO:tasks.workunit.client.0.smithi007.stdout:4/741: symlink d2/d5f/lfd 0 2024-09-21T04:35:34.596 INFO:tasks.workunit.client.0.smithi007.stdout:9/729: rmdir d13 39 2024-09-21T04:35:34.598 INFO:tasks.workunit.client.0.smithi007.stdout:4/742: fdatasync d2/f4c 0 2024-09-21T04:35:34.599 INFO:tasks.workunit.client.0.smithi007.stdout:8/503: rename d0/d3/d8/de to d0/d3/d16/d93 0 2024-09-21T04:35:34.601 INFO:tasks.workunit.client.0.smithi007.stdout:8/504: chown d0/d3/d16/d93/d26/l5b 235424975 1 2024-09-21T04:35:34.602 INFO:tasks.workunit.client.0.smithi007.stdout:9/730: chown d13/d18/d60/d83/da9/l4a 22 1 2024-09-21T04:35:34.602 INFO:tasks.workunit.client.0.smithi007.stdout:8/505: readlink d0/l13 0 2024-09-21T04:35:34.606 INFO:tasks.workunit.client.0.smithi007.stdout:7/690: rename d3/d6/d10/c56 to d3/d6/dc5/cf6 0 2024-09-21T04:35:34.608 INFO:tasks.workunit.client.0.smithi007.stdout:3/795: dwrite dc/d22/d29/d32/d4d/f8b [0,4194304] 0 2024-09-21T04:35:34.611 INFO:tasks.workunit.client.0.smithi007.stdout:8/506: write d0/d3/d8/fa [7315384,77328] 0 2024-09-21T04:35:34.612 INFO:tasks.workunit.client.0.smithi007.stdout:9/731: creat d13/d18/d60/d6d/d85/ff1 x:0 0 0 2024-09-21T04:35:34.613 INFO:tasks.workunit.client.0.smithi007.stdout:3/796: truncate dc/d22/d5a/d71/ff9 1004522 0 2024-09-21T04:35:34.615 INFO:tasks.workunit.client.0.smithi007.stdout:4/743: symlink d2/d3c/lfe 0 2024-09-21T04:35:34.615 INFO:tasks.workunit.client.0.smithi007.stdout:3/797: read dc/d22/d29/d92/db9/fcc [668201,46858] 0 2024-09-21T04:35:34.615 INFO:tasks.workunit.client.0.smithi007.stdout:9/732: chown d13/d18/d60/d6d/d85/ff1 58 1 2024-09-21T04:35:34.617 INFO:tasks.workunit.client.0.smithi007.stdout:3/798: readlink dc/d22/d5a/lb5 0 2024-09-21T04:35:34.618 INFO:tasks.workunit.client.0.smithi007.stdout:9/733: write d13/d18/d60/d83/da9/f40 [1392268,80224] 0 2024-09-21T04:35:34.619 INFO:tasks.workunit.client.0.smithi007.stdout:7/691: rename d3/d6/d14/d20/d80/l9a to d3/d6/d14/d20/d80/db2/lf7 0 2024-09-21T04:35:34.620 INFO:tasks.workunit.client.0.smithi007.stdout:7/692: chown d3/d6/d14/f1b 368802734 1 2024-09-21T04:35:34.621 INFO:tasks.workunit.client.0.smithi007.stdout:9/734: chown d13/d18/d60/d6d/d85/le2 5 1 2024-09-21T04:35:34.622 INFO:tasks.workunit.client.0.smithi007.stdout:7/693: readlink d3/d6/d2e/d63/d6a/d81/led 0 2024-09-21T04:35:34.623 INFO:tasks.workunit.client.0.smithi007.stdout:8/507: write d0/d3/d16/f21 [4882317,95916] 0 2024-09-21T04:35:34.623 INFO:tasks.workunit.client.0.smithi007.stdout:7/694: dread - d3/d6/d10/ff5 zero size 2024-09-21T04:35:34.625 INFO:tasks.workunit.client.0.smithi007.stdout:4/744: mknod d2/d7/dc5/dd6/cff 0 2024-09-21T04:35:34.628 INFO:tasks.workunit.client.0.smithi007.stdout:3/799: mkdir dc/d22/d29/d92/db9/dfa 0 2024-09-21T04:35:34.629 INFO:tasks.workunit.client.0.smithi007.stdout:4/745: dread d2/d17/fd1 [0,4194304] 0 2024-09-21T04:35:34.629 INFO:tasks.workunit.client.0.smithi007.stdout:9/735: mknod d13/d18/d80/cf2 0 2024-09-21T04:35:34.631 INFO:tasks.workunit.client.0.smithi007.stdout:4/746: chown d2/d17/d2e/d45/d5d/cf3 1234 1 2024-09-21T04:35:34.632 INFO:tasks.workunit.client.0.smithi007.stdout:8/508: dwrite d0/d3/d16/d93/d26/d35/f87 [0,4194304] 0 2024-09-21T04:35:34.635 INFO:tasks.workunit.client.0.smithi007.stdout:7/695: truncate d3/d6/d10/d51/fb3 523401 0 2024-09-21T04:35:34.643 INFO:tasks.workunit.client.0.smithi007.stdout:4/747: creat d2/d17/d2e/d45/d5e/f100 x:0 0 0 2024-09-21T04:35:34.648 INFO:tasks.workunit.client.0.smithi007.stdout:9/736: symlink d13/d5a/lf3 0 2024-09-21T04:35:34.650 INFO:tasks.workunit.client.0.smithi007.stdout:8/509: mknod d0/d3/d8/c94 0 2024-09-21T04:35:34.653 INFO:tasks.workunit.client.0.smithi007.stdout:3/800: dread dc/d22/d29/f55 [0,4194304] 0 2024-09-21T04:35:34.653 INFO:tasks.workunit.client.0.smithi007.stdout:7/696: mkdir d3/d6/d2e/d63/df8 0 2024-09-21T04:35:34.654 INFO:tasks.workunit.client.0.smithi007.stdout:7/697: fdatasync d3/d6/d10/fe0 0 2024-09-21T04:35:34.655 INFO:tasks.workunit.client.0.smithi007.stdout:8/510: creat d0/d3/d16/d93/d40/f95 x:0 0 0 2024-09-21T04:35:34.656 INFO:tasks.workunit.client.0.smithi007.stdout:4/748: mkdir d2/d17/d2e/d45/d5d/d101 0 2024-09-21T04:35:34.657 INFO:tasks.workunit.client.0.smithi007.stdout:8/511: write d0/d3/d8/f3f [775266,87058] 0 2024-09-21T04:35:34.659 INFO:tasks.workunit.client.0.smithi007.stdout:4/749: stat d2/d17/d2e/d45/d5d/dae/lb3 0 2024-09-21T04:35:34.661 INFO:tasks.workunit.client.0.smithi007.stdout:9/737: dread d13/d18/f4b [0,4194304] 0 2024-09-21T04:35:34.662 INFO:tasks.workunit.client.0.smithi007.stdout:3/801: mkdir dc/d22/d29/d32/d4d/d84/dd0/dfb 0 2024-09-21T04:35:34.663 INFO:tasks.workunit.client.0.smithi007.stdout:7/698: mknod d3/d6/d2e/d63/cf9 0 2024-09-21T04:35:34.666 INFO:tasks.workunit.client.0.smithi007.stdout:4/750: dread - d2/d7/d64/fa9 zero size 2024-09-21T04:35:34.669 INFO:tasks.workunit.client.0.smithi007.stdout:9/738: dwrite d13/d18/d60/d83/f8f [4194304,4194304] 0 2024-09-21T04:35:34.670 INFO:tasks.workunit.client.0.smithi007.stdout:3/802: rename dc/d18/f58 to dc/d22/d29/d92/db9/ffc 0 2024-09-21T04:35:34.671 INFO:tasks.workunit.client.0.smithi007.stdout:7/699: creat d3/d6/d14/d20/d80/dca/ffa x:0 0 0 2024-09-21T04:35:34.675 INFO:tasks.workunit.client.0.smithi007.stdout:3/803: truncate dc/d22/d29/d32/d69/d6d/f9b 1268007 0 2024-09-21T04:35:34.677 INFO:tasks.workunit.client.0.smithi007.stdout:9/739: fdatasync d13/d18/d60/d6d/d75/d9f/ff0 0 2024-09-21T04:35:34.682 INFO:tasks.workunit.client.0.smithi007.stdout:9/740: dread - d13/d74/ddb/fcb zero size 2024-09-21T04:35:34.682 INFO:tasks.workunit.client.0.smithi007.stdout:3/804: mkdir dc/d6c/da7/dfd 0 2024-09-21T04:35:34.684 INFO:tasks.workunit.client.0.smithi007.stdout:7/700: dwrite d3/d6/d40/da1/fe5 [0,4194304] 0 2024-09-21T04:35:34.687 INFO:tasks.workunit.client.0.smithi007.stdout:4/751: dread d2/f12 [4194304,4194304] 0 2024-09-21T04:35:34.689 INFO:tasks.workunit.client.0.smithi007.stdout:4/752: truncate d2/d3c/fee 37159 0 2024-09-21T04:35:34.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:34 smithi053 ceph-mon[35554]: pgmap v38: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 142 MiB/s wr, 333 op/s 2024-09-21T04:35:34.693 INFO:tasks.workunit.client.0.smithi007.stdout:8/512: truncate d0/d3/d16/d93/d26/d35/f37 1536718 0 2024-09-21T04:35:34.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:34 smithi007 ceph-mon[27954]: pgmap v38: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 142 MiB/s wr, 333 op/s 2024-09-21T04:35:34.694 INFO:tasks.workunit.client.0.smithi007.stdout:8/513: dread - d0/d3/d8/d46/f63 zero size 2024-09-21T04:35:34.697 INFO:tasks.workunit.client.0.smithi007.stdout:7/701: unlink d3/d6/d10/d4f/fbe 0 2024-09-21T04:35:34.698 INFO:tasks.workunit.client.0.smithi007.stdout:7/702: dread - d3/d6/d13/fc4 zero size 2024-09-21T04:35:34.698 INFO:tasks.workunit.client.0.smithi007.stdout:4/753: rmdir d2/d17/d2e 39 2024-09-21T04:35:34.703 INFO:tasks.workunit.client.0.smithi007.stdout:8/514: truncate d0/d3/d8/d46/f63 1013132 0 2024-09-21T04:35:34.707 INFO:tasks.workunit.client.0.smithi007.stdout:7/703: symlink d3/d6/d14/d20/d80/lfb 0 2024-09-21T04:35:34.708 INFO:tasks.workunit.client.0.smithi007.stdout:8/515: symlink d0/d3/d16/d93/d40/l96 0 2024-09-21T04:35:34.711 INFO:tasks.workunit.client.0.smithi007.stdout:3/805: dwrite dc/d22/d29/d32/d4d/d84/dd0/da6/f5c [0,4194304] 0 2024-09-21T04:35:34.713 INFO:tasks.workunit.client.0.smithi007.stdout:4/754: creat d2/d17/d2e/d45/d5d/d101/f102 x:0 0 0 2024-09-21T04:35:34.714 INFO:tasks.workunit.client.0.smithi007.stdout:9/741: dwrite d13/d18/d60/d6d/d75/f9c [0,4194304] 0 2024-09-21T04:35:34.714 INFO:tasks.workunit.client.0.smithi007.stdout:8/516: dread - d0/d3/d16/d93/d5e/d7b/f7e zero size 2024-09-21T04:35:34.724 INFO:tasks.workunit.client.0.smithi007.stdout:7/704: write d3/d6/d14/d20/d80/f9c [6027560,49221] 0 2024-09-21T04:35:34.728 INFO:tasks.workunit.client.0.smithi007.stdout:8/517: dwrite d0/d3/d16/f31 [0,4194304] 0 2024-09-21T04:35:34.736 INFO:tasks.workunit.client.0.smithi007.stdout:3/806: creat dc/d22/d29/d32/d4d/d84/dd0/ffe x:0 0 0 2024-09-21T04:35:34.738 INFO:tasks.workunit.client.0.smithi007.stdout:7/705: mkdir d3/d6/d2e/d63/df8/dfc 0 2024-09-21T04:35:34.740 INFO:tasks.workunit.client.0.smithi007.stdout:9/742: creat d13/d18/ff4 x:0 0 0 2024-09-21T04:35:34.740 INFO:tasks.workunit.client.0.smithi007.stdout:8/518: dwrite d0/d3/d16/f31 [0,4194304] 0 2024-09-21T04:35:34.743 INFO:tasks.workunit.client.0.smithi007.stdout:4/755: dread d2/d17/d2e/d45/f8f [0,4194304] 0 2024-09-21T04:35:34.744 INFO:tasks.workunit.client.0.smithi007.stdout:7/706: mknod d3/d6/d10/d62/cfd 0 2024-09-21T04:35:34.744 INFO:tasks.workunit.client.0.smithi007.stdout:3/807: write dc/d22/d29/d32/d4d/d84/dd0/da6/f5c [318356,92407] 0 2024-09-21T04:35:34.746 INFO:tasks.workunit.client.0.smithi007.stdout:9/743: write d13/d18/d60/d6d/d75/d9f/ff0 [4324844,13501] 0 2024-09-21T04:35:34.749 INFO:tasks.workunit.client.0.smithi007.stdout:8/519: chown d0/d3/d16/d93/d26/d4d/d5d/c84 608 1 2024-09-21T04:35:34.750 INFO:tasks.workunit.client.0.smithi007.stdout:3/808: fdatasync dc/d22/d29/f6a 0 2024-09-21T04:35:34.751 INFO:tasks.workunit.client.0.smithi007.stdout:9/744: truncate d13/d18/d60/d83/da9/f2d 4366995 0 2024-09-21T04:35:34.752 INFO:tasks.workunit.client.0.smithi007.stdout:3/809: chown dc/d18/f45 33 1 2024-09-21T04:35:34.752 INFO:tasks.workunit.client.0.smithi007.stdout:7/707: symlink d3/d6/d2e/d63/df8/lfe 0 2024-09-21T04:35:34.753 INFO:tasks.workunit.client.0.smithi007.stdout:4/756: truncate d2/d7/d55/d4f/f9d 1208782 0 2024-09-21T04:35:34.753 INFO:tasks.workunit.client.0.smithi007.stdout:7/708: fsync d3/d6/d10/d4f/fdb 0 2024-09-21T04:35:34.756 INFO:tasks.workunit.client.0.smithi007.stdout:7/709: read d3/d6/d14/f18 [3950723,33083] 0 2024-09-21T04:35:34.759 INFO:tasks.workunit.client.0.smithi007.stdout:9/745: dread d13/d5a/f61 [0,4194304] 0 2024-09-21T04:35:34.762 INFO:tasks.workunit.client.0.smithi007.stdout:9/746: chown d13/d18/d60/d6d 228544 1 2024-09-21T04:35:34.763 INFO:tasks.workunit.client.0.smithi007.stdout:3/810: dwrite dc/d22/d29/d32/d4d/d84/dd0/ffe [0,4194304] 0 2024-09-21T04:35:34.763 INFO:tasks.workunit.client.0.smithi007.stdout:4/757: creat d2/d17/d2e/d45/d5e/f103 x:0 0 0 2024-09-21T04:35:34.764 INFO:tasks.workunit.client.0.smithi007.stdout:4/758: chown d2/d17/c2c 5 1 2024-09-21T04:35:34.767 INFO:tasks.workunit.client.0.smithi007.stdout:8/520: dread d0/d3/d16/d93/d26/d35/f37 [0,4194304] 0 2024-09-21T04:35:34.771 INFO:tasks.workunit.client.0.smithi007.stdout:4/759: dwrite d2/f43 [0,4194304] 0 2024-09-21T04:35:34.773 INFO:tasks.workunit.client.0.smithi007.stdout:9/747: dread - d13/d18/f27 zero size 2024-09-21T04:35:34.779 INFO:tasks.workunit.client.0.smithi007.stdout:8/521: creat d0/d3/d16/d93/d40/d6c/f97 x:0 0 0 2024-09-21T04:35:34.782 INFO:tasks.workunit.client.0.smithi007.stdout:4/760: read d2/d17/d2e/f7c [86678,69217] 0 2024-09-21T04:35:34.783 INFO:tasks.workunit.client.0.smithi007.stdout:9/748: creat d13/d18/d60/d83/da9/ff5 x:0 0 0 2024-09-21T04:35:34.787 INFO:tasks.workunit.client.0.smithi007.stdout:7/710: write d3/d6/d2e/d63/d6a/f70 [4255015,75490] 0 2024-09-21T04:35:34.787 INFO:tasks.workunit.client.0.smithi007.stdout:4/761: chown d2/d7/d64/c72 1 1 2024-09-21T04:35:34.789 INFO:tasks.workunit.client.0.smithi007.stdout:9/749: readlink d13/l1b 0 2024-09-21T04:35:34.791 INFO:tasks.workunit.client.0.smithi007.stdout:7/711: truncate d3/d6/d10/d62/d66/f94 1558344 0 2024-09-21T04:35:34.794 INFO:tasks.workunit.client.0.smithi007.stdout:9/750: mknod d13/d18/dcd/dd4/cf6 0 2024-09-21T04:35:34.798 INFO:tasks.workunit.client.0.smithi007.stdout:8/522: dread d0/d3/d16/d93/d26/d35/f73 [0,4194304] 0 2024-09-21T04:35:34.799 INFO:tasks.workunit.client.0.smithi007.stdout:7/712: rename d3/d6/d10/c44 to d3/d6/d14/d20/d80/db2/cff 0 2024-09-21T04:35:34.802 INFO:tasks.workunit.client.0.smithi007.stdout:9/751: creat d13/ff7 x:0 0 0 2024-09-21T04:35:34.803 INFO:tasks.workunit.client.0.smithi007.stdout:7/713: mknod d3/d6/d14/d20/d80/dca/c100 0 2024-09-21T04:35:34.804 INFO:tasks.workunit.client.0.smithi007.stdout:4/762: dwrite d2/d17/d2e/d45/d5e/d90/fa2 [0,4194304] 0 2024-09-21T04:35:34.810 INFO:tasks.workunit.client.0.smithi007.stdout:9/752: dwrite d13/d18/daf/dbc/fc6 [0,4194304] 0 2024-09-21T04:35:34.810 INFO:tasks.workunit.client.0.smithi007.stdout:7/714: rmdir d3/d6/d2e/d5e 39 2024-09-21T04:35:34.812 INFO:tasks.workunit.client.0.smithi007.stdout:8/523: truncate d0/f20 7442193 0 2024-09-21T04:35:34.814 INFO:tasks.workunit.client.0.smithi007.stdout:4/763: mknod d2/d7/d55/c104 0 2024-09-21T04:35:34.816 INFO:tasks.workunit.client.0.smithi007.stdout:9/753: stat d13/d18/daf/dc0/cc3 0 2024-09-21T04:35:34.819 INFO:tasks.workunit.client.0.smithi007.stdout:7/715: dwrite d3/d6/d10/fe0 [0,4194304] 0 2024-09-21T04:35:34.820 INFO:tasks.workunit.client.0.smithi007.stdout:4/764: write d2/d7/d55/d61/f9a [1427598,80620] 0 2024-09-21T04:35:34.823 INFO:tasks.workunit.client.0.smithi007.stdout:8/524: dread d0/d3/d8/f67 [0,4194304] 0 2024-09-21T04:35:34.824 INFO:tasks.workunit.client.0.smithi007.stdout:9/754: rmdir d13/d18/d60/d6d/d75/d7f 39 2024-09-21T04:35:34.826 INFO:tasks.workunit.client.0.smithi007.stdout:4/765: stat d2/d17/l62 0 2024-09-21T04:35:34.828 INFO:tasks.workunit.client.0.smithi007.stdout:9/755: creat d13/d18/dcd/ff8 x:0 0 0 2024-09-21T04:35:34.829 INFO:tasks.workunit.client.0.smithi007.stdout:7/716: mkdir d3/d6/d10/d62/d101 0 2024-09-21T04:35:34.830 INFO:tasks.workunit.client.0.smithi007.stdout:8/525: creat d0/d3/d16/d93/d26/d4d/d5d/d88/f98 x:0 0 0 2024-09-21T04:35:34.830 INFO:tasks.workunit.client.0.smithi007.stdout:7/717: readlink d3/d6/d13/d4b/lf0 0 2024-09-21T04:35:34.831 INFO:tasks.workunit.client.0.smithi007.stdout:8/526: read - d0/d3/d16/d93/d26/d4d/f91 zero size 2024-09-21T04:35:34.833 INFO:tasks.workunit.client.0.smithi007.stdout:4/766: dread d2/d17/fd1 [0,4194304] 0 2024-09-21T04:35:34.834 INFO:tasks.workunit.client.0.smithi007.stdout:9/756: creat d13/d18/d60/d6d/d85/ff9 x:0 0 0 2024-09-21T04:35:34.836 INFO:tasks.workunit.client.0.smithi007.stdout:8/527: rename d0/d3/d16/d93/d5e/f81 to d0/d3/d16/f99 0 2024-09-21T04:35:34.837 INFO:tasks.workunit.client.0.smithi007.stdout:4/767: mknod d2/d7/dc5/dd6/c105 0 2024-09-21T04:35:34.839 INFO:tasks.workunit.client.0.smithi007.stdout:9/757: stat d13/d18/f67 0 2024-09-21T04:35:34.841 INFO:tasks.workunit.client.0.smithi007.stdout:9/758: write d13/d18/d60/d6d/d75/fd2 [351113,34900] 0 2024-09-21T04:35:34.844 INFO:tasks.workunit.client.0.smithi007.stdout:8/528: dwrite d0/d3/d16/f21 [0,4194304] 0 2024-09-21T04:35:34.848 INFO:tasks.workunit.client.0.smithi007.stdout:8/529: truncate d0/d3/d16/d93/d26/f85 4876792 0 2024-09-21T04:35:34.849 INFO:tasks.workunit.client.0.smithi007.stdout:4/768: truncate d2/d3c/d63/fa3 589129 0 2024-09-21T04:35:34.850 INFO:tasks.workunit.client.0.smithi007.stdout:9/759: dwrite d13/d18/d60/d6d/d85/da1/fa5 [0,4194304] 0 2024-09-21T04:35:34.859 INFO:tasks.workunit.client.0.smithi007.stdout:4/769: dwrite d2/f4c [0,4194304] 0 2024-09-21T04:35:34.863 INFO:tasks.workunit.client.0.smithi007.stdout:9/760: dwrite d13/d18/d80/fc2 [0,4194304] 0 2024-09-21T04:35:34.864 INFO:tasks.workunit.client.0.smithi007.stdout:4/770: dread - d2/d17/d2e/d45/d5d/fb1 zero size 2024-09-21T04:35:34.866 INFO:tasks.workunit.client.0.smithi007.stdout:4/771: write d2/d3c/dbe/fc3 [496218,16950] 0 2024-09-21T04:35:34.868 INFO:tasks.workunit.client.0.smithi007.stdout:9/761: mknod d13/d18/d80/cfa 0 2024-09-21T04:35:34.868 INFO:tasks.workunit.client.0.smithi007.stdout:4/772: read - d2/d7/d55/fde zero size 2024-09-21T04:35:34.874 INFO:tasks.workunit.client.0.smithi007.stdout:4/773: readlink d2/d7/d55/l56 0 2024-09-21T04:35:34.877 INFO:tasks.workunit.client.0.smithi007.stdout:8/530: dwrite d0/d3/d16/d93/d26/d35/f73 [4194304,4194304] 0 2024-09-21T04:35:34.879 INFO:tasks.workunit.client.0.smithi007.stdout:8/531: write d0/d3/d16/d93/d40/f45 [1553756,65433] 0 2024-09-21T04:35:34.881 INFO:tasks.workunit.client.0.smithi007.stdout:9/762: unlink d13/d18/l41 0 2024-09-21T04:35:34.883 INFO:tasks.workunit.client.0.smithi007.stdout:4/774: truncate d2/d17/fd1 91431 0 2024-09-21T04:35:34.886 INFO:tasks.workunit.client.0.smithi007.stdout:8/532: creat d0/d3/d16/d93/d40/d92/f9a x:0 0 0 2024-09-21T04:35:34.888 INFO:tasks.workunit.client.0.smithi007.stdout:9/763: creat d13/d18/d80/dea/ffb x:0 0 0 2024-09-21T04:35:34.889 INFO:tasks.workunit.client.0.smithi007.stdout:9/764: stat d13/d5a/l5f 0 2024-09-21T04:35:34.889 INFO:tasks.workunit.client.0.smithi007.stdout:9/765: dread - d13/fe5 zero size 2024-09-21T04:35:34.889 INFO:tasks.workunit.client.0.smithi007.stdout:8/533: creat d0/d3/d16/d93/d26/d4d/f9b x:0 0 0 2024-09-21T04:35:34.894 INFO:tasks.workunit.client.0.smithi007.stdout:4/775: getdents d2/d7/d64/db8 0 2024-09-21T04:35:34.895 INFO:tasks.workunit.client.0.smithi007.stdout:8/534: mknod d0/d3/d16/d93/c9c 0 2024-09-21T04:35:34.895 INFO:tasks.workunit.client.0.smithi007.stdout:8/535: chown d0/d3/d8 110498723 1 2024-09-21T04:35:34.898 INFO:tasks.workunit.client.0.smithi007.stdout:8/536: mkdir d0/d3/d16/d93/d40/d6c/d9d 0 2024-09-21T04:35:34.900 INFO:tasks.workunit.client.0.smithi007.stdout:8/537: write d0/d3/d16/d93/f18 [3219574,63608] 0 2024-09-21T04:35:34.900 INFO:tasks.workunit.client.0.smithi007.stdout:4/776: rename d2/d7/d55/d61/f9a to d2/d3c/f106 0 2024-09-21T04:35:34.904 INFO:tasks.workunit.client.0.smithi007.stdout:4/777: dread d2/f43 [0,4194304] 0 2024-09-21T04:35:34.907 INFO:tasks.workunit.client.0.smithi007.stdout:9/766: write d13/d18/f4b [4352196,128052] 0 2024-09-21T04:35:34.909 INFO:tasks.workunit.client.0.smithi007.stdout:9/767: stat d13/d18/d60/d6d/d85/da1/daa/fb8 0 2024-09-21T04:35:34.912 INFO:tasks.workunit.client.0.smithi007.stdout:4/778: dwrite d2/d17/d18/f54 [0,4194304] 0 2024-09-21T04:35:34.916 INFO:tasks.workunit.client.0.smithi007.stdout:4/779: creat d2/d3c/dbe/f107 x:0 0 0 2024-09-21T04:35:34.918 INFO:tasks.workunit.client.0.smithi007.stdout:4/780: mknod d2/d7/c108 0 2024-09-21T04:35:34.919 INFO:tasks.workunit.client.0.smithi007.stdout:4/781: stat d2/d17/f51 0 2024-09-21T04:35:34.920 INFO:tasks.workunit.client.0.smithi007.stdout:8/538: write d0/d3/d16/d93/f17 [477975,73421] 0 2024-09-21T04:35:34.923 INFO:tasks.workunit.client.0.smithi007.stdout:8/539: mkdir d0/d3/d16/d93/d40/d9e 0 2024-09-21T04:35:34.928 INFO:tasks.workunit.client.0.smithi007.stdout:8/540: unlink d0/f6f 0 2024-09-21T04:35:34.930 INFO:tasks.workunit.client.0.smithi007.stdout:9/768: dread d13/d18/f35 [0,4194304] 0 2024-09-21T04:35:34.931 INFO:tasks.workunit.client.0.smithi007.stdout:8/541: write d0/d3/d16/d93/d26/d35/f87 [4856874,15733] 0 2024-09-21T04:35:35.010 INFO:tasks.workunit.client.0.smithi007.stdout:4/782: truncate d2/d7/f59 2932157 0 2024-09-21T04:35:35.015 INFO:tasks.workunit.client.0.smithi007.stdout:9/769: dwrite d13/d18/f5c [4194304,4194304] 0 2024-09-21T04:35:35.017 INFO:tasks.workunit.client.0.smithi007.stdout:9/770: readlink d13/l17 0 2024-09-21T04:35:35.025 INFO:tasks.workunit.client.0.smithi007.stdout:9/771: dwrite d13/d18/d60/d6d/d75/f9c [0,4194304] 0 2024-09-21T04:35:35.032 INFO:tasks.workunit.client.0.smithi007.stdout:8/542: truncate d0/d3/d16/d93/d26/d4d/f72 6833283 0 2024-09-21T04:35:35.034 INFO:tasks.workunit.client.0.smithi007.stdout:8/543: chown d0/f20 283926679 1 2024-09-21T04:35:35.034 INFO:tasks.workunit.client.0.smithi007.stdout:4/783: dwrite d2/f6 [0,4194304] 0 2024-09-21T04:35:35.039 INFO:tasks.workunit.client.0.smithi007.stdout:8/544: unlink d0/d3/d16/d93/d26/d35/l4b 0 2024-09-21T04:35:35.040 INFO:tasks.workunit.client.0.smithi007.stdout:8/545: fdatasync d0/d3/d8/f5f 0 2024-09-21T04:35:35.040 INFO:tasks.workunit.client.0.smithi007.stdout:8/546: readlink d0/l13 0 2024-09-21T04:35:35.042 INFO:tasks.workunit.client.0.smithi007.stdout:9/772: write d13/d18/f71 [383699,18853] 0 2024-09-21T04:35:35.046 INFO:tasks.workunit.client.0.smithi007.stdout:8/547: rmdir d0/d3/d16/d93 39 2024-09-21T04:35:35.047 INFO:tasks.workunit.client.0.smithi007.stdout:9/773: creat d13/d5a/ffc x:0 0 0 2024-09-21T04:35:35.049 INFO:tasks.workunit.client.0.smithi007.stdout:9/774: write d13/d18/d60/d6d/d85/fd3 [857881,13993] 0 2024-09-21T04:35:35.049 INFO:tasks.workunit.client.0.smithi007.stdout:9/775: stat d13/d18/daf/dc0/ce9 0 2024-09-21T04:35:35.053 INFO:tasks.workunit.client.0.smithi007.stdout:8/548: rmdir d0/d3/d16/d93/d5e 39 2024-09-21T04:35:35.059 INFO:tasks.workunit.client.0.smithi007.stdout:4/784: truncate d2/d17/d2e/d45/d5d/d8a/fd9 686635 0 2024-09-21T04:35:35.060 INFO:tasks.workunit.client.0.smithi007.stdout:8/549: dwrite d0/d3/d8/f3f [0,4194304] 0 2024-09-21T04:35:35.060 INFO:tasks.workunit.client.0.smithi007.stdout:4/785: stat d2/d7/d55/l56 0 2024-09-21T04:35:35.062 INFO:tasks.workunit.client.0.smithi007.stdout:4/786: fsync d2/d17/d2e/d45/d5d/d8a/fd5 0 2024-09-21T04:35:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:9/776: dread d13/d18/d60/d6d/fb6 [0,4194304] 0 2024-09-21T04:35:35.064 INFO:tasks.workunit.client.0.smithi007.stdout:8/550: symlink d0/d3/d16/d93/d26/l9f 0 2024-09-21T04:35:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:9/777: write d13/d74/fa6 [85442,122522] 0 2024-09-21T04:35:35.065 INFO:tasks.workunit.client.0.smithi007.stdout:8/551: stat d0/d3/d16/f8d 0 2024-09-21T04:35:35.067 INFO:tasks.workunit.client.0.smithi007.stdout:8/552: write d0/d3/d16/d93/f17 [290491,85329] 0 2024-09-21T04:35:35.081 INFO:tasks.workunit.client.0.smithi007.stdout:4/787: rename d2/d17/l3b to d2/d5f/l109 0 2024-09-21T04:35:35.081 INFO:tasks.workunit.client.0.smithi007.stdout:4/788: chown d2/f81 7757206 1 2024-09-21T04:35:35.086 INFO:tasks.workunit.client.0.smithi007.stdout:9/778: dwrite d13/d18/d60/d6d/fae [0,4194304] 0 2024-09-21T04:35:35.089 INFO:tasks.workunit.client.0.smithi007.stdout:9/779: readlink d13/d18/l73 0 2024-09-21T04:35:35.090 INFO:tasks.workunit.client.0.smithi007.stdout:9/780: write d13/d18/d60/d83/da9/fb4 [269508,24287] 0 2024-09-21T04:35:35.095 INFO:tasks.workunit.client.0.smithi007.stdout:9/781: mkdir d13/d89/dfd 0 2024-09-21T04:35:35.100 INFO:tasks.workunit.client.0.smithi007.stdout:9/782: rename d13/d5a/la2 to d13/lfe 0 2024-09-21T04:35:35.104 INFO:tasks.workunit.client.0.smithi007.stdout:4/789: write d2/d17/f79 [361740,24976] 0 2024-09-21T04:35:35.110 INFO:tasks.workunit.client.0.smithi007.stdout:4/790: dwrite d2/d7/d64/fc8 [0,4194304] 0 2024-09-21T04:35:35.111 INFO:tasks.workunit.client.0.smithi007.stdout:9/783: getdents d13 0 2024-09-21T04:35:35.116 INFO:tasks.workunit.client.0.smithi007.stdout:4/791: write d2/d3c/f106 [3433301,24816] 0 2024-09-21T04:35:35.117 INFO:tasks.workunit.client.0.smithi007.stdout:4/792: write d2/f27 [1141861,12328] 0 2024-09-21T04:35:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:9/784: getdents d13/d18/daf/dda 0 2024-09-21T04:35:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:9/785: link d13/d18/f1a d13/d18/dcd/dd4/fff 0 2024-09-21T04:35:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:9/786: write d13/d89/fd1 [1769545,18759] 0 2024-09-21T04:35:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:9/787: readlink d13/d18/d60/d6d/d75/d9f/dc8/lc9 0 2024-09-21T04:35:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:4/793: dread d2/d7/f8 [0,4194304] 0 2024-09-21T04:35:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:9/788: chown d13/d18/d60/f88 26695810 1 2024-09-21T04:35:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:9/789: stat d13/d74/ddb 0 2024-09-21T04:35:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:9/790: chown d13/d18/d60/d83/fdc 12 1 2024-09-21T04:35:35.132 INFO:tasks.workunit.client.0.smithi007.stdout:4/794: mkdir d2/d17/d2e/d45/d5d/d10a 0 2024-09-21T04:35:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:9/791: symlink d13/d18/daf/dc0/l100 0 2024-09-21T04:35:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:9/792: mkdir d13/d18/d60/d6d/d85/da1/d101 0 2024-09-21T04:35:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:4/795: link d2/d17/d2e/d45/d5d/dc7/le0 d2/d17/d2e/d45/d8e/l10b 0 2024-09-21T04:35:35.147 INFO:tasks.workunit.client.0.smithi007.stdout:4/796: rename d2/d17/df8 to d2/d17/d2e/d45/d5e/d10c 0 2024-09-21T04:35:35.151 INFO:tasks.workunit.client.0.smithi007.stdout:4/797: dread d2/d17/d18/f54 [0,4194304] 0 2024-09-21T04:35:35.152 INFO:tasks.workunit.client.0.smithi007.stdout:4/798: write d2/d7/dc5/fd8 [309915,107763] 0 2024-09-21T04:35:35.155 INFO:tasks.workunit.client.0.smithi007.stdout:9/793: write d13/d18/f68 [958777,4181] 0 2024-09-21T04:35:35.156 INFO:tasks.workunit.client.0.smithi007.stdout:4/799: truncate d2/d7/f96 18349 0 2024-09-21T04:35:35.158 INFO:tasks.workunit.client.0.smithi007.stdout:9/794: mkdir d13/d74/ddb/d102 0 2024-09-21T04:35:35.160 INFO:tasks.workunit.client.0.smithi007.stdout:4/800: creat d2/d7/d55/f10d x:0 0 0 2024-09-21T04:35:35.161 INFO:tasks.workunit.client.0.smithi007.stdout:4/801: chown d2/d7/d55/d4f 632580101 1 2024-09-21T04:35:35.162 INFO:tasks.workunit.client.0.smithi007.stdout:9/795: unlink d13/d18/d60/d83/da9/f32 0 2024-09-21T04:35:35.163 INFO:tasks.workunit.client.0.smithi007.stdout:4/802: creat d2/d97/da5/f10e x:0 0 0 2024-09-21T04:35:35.166 INFO:tasks.workunit.client.0.smithi007.stdout:4/803: mknod d2/d17/d2e/d45/d5d/dc7/c10f 0 2024-09-21T04:35:35.167 INFO:tasks.workunit.client.0.smithi007.stdout:9/796: truncate d13/d18/d60/d6d/f9d 3297537 0 2024-09-21T04:35:35.167 INFO:tasks.workunit.client.0.smithi007.stdout:9/797: fdatasync d13/f6b 0 2024-09-21T04:35:35.168 INFO:tasks.workunit.client.0.smithi007.stdout:4/804: creat d2/d3c/dbe/f110 x:0 0 0 2024-09-21T04:35:35.170 INFO:tasks.workunit.client.0.smithi007.stdout:9/798: creat d13/d18/d80/dea/f103 x:0 0 0 2024-09-21T04:35:35.171 INFO:tasks.workunit.client.0.smithi007.stdout:9/799: dread - d13/d18/d60/d83/fdc zero size 2024-09-21T04:35:35.178 INFO:tasks.workunit.client.0.smithi007.stdout:4/805: dread d2/d17/d29/f69 [0,4194304] 0 2024-09-21T04:35:35.184 INFO:tasks.workunit.client.0.smithi007.stdout:4/806: dwrite d2/d17/d2e/d45/d5e/faa [0,4194304] 0 2024-09-21T04:35:35.185 INFO:tasks.workunit.client.0.smithi007.stdout:4/807: fsync d2/d17/d2e/d45/d5e/f100 0 2024-09-21T04:35:35.219 INFO:tasks.workunit.client.0.smithi007.stdout:1/863: sync 2024-09-21T04:35:35.221 INFO:tasks.workunit.client.0.smithi007.stdout:9/800: mkdir d13/d18/dcd/d104 0 2024-09-21T04:35:35.222 INFO:tasks.workunit.client.0.smithi007.stdout:9/801: truncate d13/d74/fa6 338531 0 2024-09-21T04:35:35.231 INFO:tasks.workunit.client.0.smithi007.stdout:9/802: fdatasync d13/d18/dcd/dd4/fff 0 2024-09-21T04:35:35.232 INFO:tasks.workunit.client.0.smithi007.stdout:9/803: chown d13/d18/f4b 173152 1 2024-09-21T04:35:35.233 INFO:tasks.workunit.client.0.smithi007.stdout:9/804: readlink d13/d18/d60/d83/da9/l7a 0 2024-09-21T04:35:35.235 INFO:tasks.workunit.client.0.smithi007.stdout:9/805: mkdir d13/d74/de1/d105 0 2024-09-21T04:35:35.240 INFO:tasks.workunit.client.0.smithi007.stdout:1/864: write d5/d6/d21/d2b/d86/dd8/f6e [1236906,91482] 0 2024-09-21T04:35:35.241 INFO:tasks.workunit.client.0.smithi007.stdout:4/808: write d2/d17/f2a [1270973,92214] 0 2024-09-21T04:35:35.247 INFO:tasks.workunit.client.0.smithi007.stdout:4/809: symlink d2/d17/l111 0 2024-09-21T04:35:35.248 INFO:tasks.workunit.client.0.smithi007.stdout:1/865: dread d5/d6/d21/d2b/d4b/da6/f65 [0,4194304] 0 2024-09-21T04:35:35.249 INFO:tasks.workunit.client.0.smithi007.stdout:9/806: dread d13/d74/f78 [0,4194304] 0 2024-09-21T04:35:35.251 INFO:tasks.workunit.client.0.smithi007.stdout:4/810: mkdir d2/d17/d2e/d45/d5d/d10a/d112 0 2024-09-21T04:35:35.254 INFO:tasks.workunit.client.0.smithi007.stdout:1/866: rename d5/d6/lb2 to d5/d6/d21/d2b/d4b/da6/d58/dcf/l116 0 2024-09-21T04:35:35.255 INFO:tasks.workunit.client.0.smithi007.stdout:9/807: dwrite d13/f5e [0,4194304] 0 2024-09-21T04:35:35.256 INFO:tasks.workunit.client.0.smithi007.stdout:1/867: truncate d5/fc3 4224727 0 2024-09-21T04:35:35.256 INFO:tasks.workunit.client.0.smithi007.stdout:4/811: dread d2/d17/d2e/d45/d5e/f82 [0,4194304] 0 2024-09-21T04:35:35.258 INFO:tasks.workunit.client.0.smithi007.stdout:4/812: truncate d2/d7/dc5/dd6/fd7 300795 0 2024-09-21T04:35:35.259 INFO:tasks.workunit.client.0.smithi007.stdout:1/868: fdatasync d5/f48 0 2024-09-21T04:35:35.264 INFO:tasks.workunit.client.0.smithi007.stdout:1/869: mkdir d5/d6/d21/d2b/d4b/da6/d58/dcf/d117 0 2024-09-21T04:35:35.271 INFO:tasks.workunit.client.0.smithi007.stdout:1/870: link d5/d6/d21/d2b/d4b/da6/f9c d5/d6/d21/d2b/d4b/d8a/dee/f118 0 2024-09-21T04:35:35.272 INFO:tasks.workunit.client.0.smithi007.stdout:1/871: fsync d5/f48 0 2024-09-21T04:35:35.274 INFO:tasks.workunit.client.0.smithi007.stdout:1/872: chown d5/d6/d21/d2b/d4b 1172405 1 2024-09-21T04:35:35.279 INFO:tasks.workunit.client.0.smithi007.stdout:9/808: dwrite d13/d18/d60/d6d/fe6 [0,4194304] 0 2024-09-21T04:35:35.280 INFO:tasks.workunit.client.0.smithi007.stdout:9/809: chown d13/d5a/f61 27094 1 2024-09-21T04:35:35.283 INFO:tasks.workunit.client.0.smithi007.stdout:1/873: creat d5/d6/f119 x:0 0 0 2024-09-21T04:35:35.283 INFO:tasks.workunit.client.0.smithi007.stdout:4/813: dwrite d2/d17/d2e/d9f/fab [0,4194304] 0 2024-09-21T04:35:35.286 INFO:tasks.workunit.client.0.smithi007.stdout:4/814: chown d2/d17/d2e/d45/d5e/l65 1887857 1 2024-09-21T04:35:35.288 INFO:tasks.workunit.client.0.smithi007.stdout:4/815: write d2/d3c/fee [763898,43105] 0 2024-09-21T04:35:35.292 INFO:tasks.workunit.client.0.smithi007.stdout:9/810: dwrite d13/d18/f27 [0,4194304] 0 2024-09-21T04:35:35.294 INFO:tasks.workunit.client.0.smithi007.stdout:1/874: dread d5/d6/d21/d64/df1/f59 [4194304,4194304] 0 2024-09-21T04:35:35.298 INFO:tasks.workunit.client.0.smithi007.stdout:9/811: creat d13/d74/f106 x:0 0 0 2024-09-21T04:35:35.299 INFO:tasks.workunit.client.0.smithi007.stdout:1/875: rmdir d5/d6/d21/d2b/d4b 39 2024-09-21T04:35:35.304 INFO:tasks.workunit.client.0.smithi007.stdout:4/816: link d2/d17/d2e/d45/d5d/fb1 d2/d7/dfb/f113 0 2024-09-21T04:35:35.309 INFO:tasks.workunit.client.0.smithi007.stdout:1/876: dwrite d5/d6/d21/d2b/f2d [4194304,4194304] 0 2024-09-21T04:35:35.310 INFO:tasks.workunit.client.0.smithi007.stdout:9/812: rmdir d13/d89/dfd 0 2024-09-21T04:35:35.315 INFO:tasks.workunit.client.0.smithi007.stdout:9/813: symlink d13/d18/dcd/dd4/l107 0 2024-09-21T04:35:35.319 INFO:tasks.workunit.client.0.smithi007.stdout:9/814: rmdir d13/d18/d60/d6d/d75/d9f/dc8 39 2024-09-21T04:35:35.331 INFO:tasks.workunit.client.0.smithi007.stdout:9/815: dread d13/d18/d80/fdf [0,4194304] 0 2024-09-21T04:35:35.332 INFO:tasks.workunit.client.0.smithi007.stdout:9/816: fdatasync d13/d89/fb9 0 2024-09-21T04:35:35.335 INFO:tasks.workunit.client.0.smithi007.stdout:4/817: write d2/d7/d55/d4f/f5c [193499,109341] 0 2024-09-21T04:35:35.341 INFO:tasks.workunit.client.0.smithi007.stdout:1/877: dwrite d5/d6/d21/d64/df1/f78 [0,4194304] 0 2024-09-21T04:35:35.369 INFO:tasks.workunit.client.0.smithi007.stdout:9/817: mknod d13/d18/dcd/dd4/c108 0 2024-09-21T04:35:35.369 INFO:tasks.workunit.client.0.smithi007.stdout:9/818: chown d13/d18/f1a 2 1 2024-09-21T04:35:35.370 INFO:tasks.workunit.client.0.smithi007.stdout:9/819: chown d13/d18/d60/c90 623768 1 2024-09-21T04:35:35.370 INFO:tasks.workunit.client.0.smithi007.stdout:4/818: mkdir d2/d88/da8/d114 0 2024-09-21T04:35:35.372 INFO:tasks.workunit.client.0.smithi007.stdout:1/878: read d5/d6/d21/d2b/d4b/da6/f65 [421043,50570] 0 2024-09-21T04:35:35.373 INFO:tasks.workunit.client.0.smithi007.stdout:1/879: stat d5/d6/d21/d64/df1/da0/d99/dcd 0 2024-09-21T04:35:35.374 INFO:tasks.workunit.client.0.smithi007.stdout:9/820: rename d13/d18/d80/fc2 to d13/d74/de1/dec/f109 0 2024-09-21T04:35:35.377 INFO:tasks.workunit.client.0.smithi007.stdout:1/880: symlink d5/d6/d21/d2b/d4b/da6/l11a 0 2024-09-21T04:35:35.378 INFO:tasks.workunit.client.0.smithi007.stdout:9/821: creat d13/d18/d60/f10a x:0 0 0 2024-09-21T04:35:35.379 INFO:tasks.workunit.client.0.smithi007.stdout:4/819: link d2/d17/d2e/d45/d5d/d8a/fd5 d2/d17/d2e/d45/d5d/d101/f115 0 2024-09-21T04:35:35.380 INFO:tasks.workunit.client.0.smithi007.stdout:1/881: symlink d5/d6/d21/d2b/d4b/d8a/dee/l11b 0 2024-09-21T04:35:35.382 INFO:tasks.workunit.client.0.smithi007.stdout:1/882: fdatasync d5/d6/d21/d64/df1/da0/ff3 0 2024-09-21T04:35:35.383 INFO:tasks.workunit.client.0.smithi007.stdout:9/822: dread d13/d18/d80/fc7 [0,4194304] 0 2024-09-21T04:35:35.388 INFO:tasks.workunit.client.0.smithi007.stdout:4/820: dwrite d2/d17/d2e/d45/d5e/ff5 [0,4194304] 0 2024-09-21T04:35:35.392 INFO:tasks.workunit.client.0.smithi007.stdout:4/821: symlink d2/d88/da8/db2/l116 0 2024-09-21T04:35:35.395 INFO:tasks.workunit.client.0.smithi007.stdout:9/823: dwrite d13/d18/d60/d83/fbd [0,4194304] 0 2024-09-21T04:35:35.397 INFO:tasks.workunit.client.0.smithi007.stdout:4/822: dread d2/f81 [0,4194304] 0 2024-09-21T04:35:35.398 INFO:tasks.workunit.client.0.smithi007.stdout:1/883: dread d5/d6/d21/d2b/d4b/da6/f80 [0,4194304] 0 2024-09-21T04:35:35.403 INFO:tasks.workunit.client.0.smithi007.stdout:9/824: rename d13/d18/f4b to d13/d18/d60/d6d/d75/d9f/dc8/f10b 0 2024-09-21T04:35:35.408 INFO:tasks.workunit.client.0.smithi007.stdout:9/825: dwrite d13/d18/d60/f95 [0,4194304] 0 2024-09-21T04:35:35.414 INFO:tasks.workunit.client.0.smithi007.stdout:4/823: getdents d2/d7/d55 0 2024-09-21T04:35:35.415 INFO:tasks.workunit.client.0.smithi007.stdout:9/826: symlink d13/d18/d80/l10c 0 2024-09-21T04:35:35.419 INFO:tasks.workunit.client.0.smithi007.stdout:9/827: creat d13/d18/daf/f10d x:0 0 0 2024-09-21T04:35:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:4/824: dread d2/d17/f2a [0,4194304] 0 2024-09-21T04:35:35.431 INFO:tasks.workunit.client.0.smithi007.stdout:9/828: rename d13/d5a/c7c to d13/d18/d60/d6d/d75/c10e 0 2024-09-21T04:35:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:9/829: chown d13/d18/d60/d6d/d75 653800386 1 2024-09-21T04:35:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:9/830: read - d13/d89/fed zero size 2024-09-21T04:35:35.435 INFO:tasks.workunit.client.0.smithi007.stdout:9/831: read d13/d18/d60/d6d/d85/da1/daa/fb8 [4181572,98892] 0 2024-09-21T04:35:35.435 INFO:tasks.workunit.client.0.smithi007.stdout:4/825: creat d2/f117 x:0 0 0 2024-09-21T04:35:35.438 INFO:tasks.workunit.client.0.smithi007.stdout:9/832: mknod d13/d74/ddb/c10f 0 2024-09-21T04:35:35.438 INFO:tasks.workunit.client.0.smithi007.stdout:9/833: readlink d13/d5a/lf3 0 2024-09-21T04:35:35.440 INFO:tasks.workunit.client.0.smithi007.stdout:4/826: rename d2/d17/d2e/l40 to d2/d7/dfb/l118 0 2024-09-21T04:35:35.443 INFO:tasks.workunit.client.0.smithi007.stdout:9/834: mknod d13/d18/d60/d6d/d85/da1/c110 0 2024-09-21T04:35:35.444 INFO:tasks.workunit.client.0.smithi007.stdout:4/827: mknod d2/d3c/dbe/c119 0 2024-09-21T04:35:35.449 INFO:tasks.workunit.client.0.smithi007.stdout:9/835: dread d13/d18/f68 [0,4194304] 0 2024-09-21T04:35:35.457 INFO:tasks.workunit.client.0.smithi007.stdout:9/836: dwrite d13/d18/d60/d6d/fae [0,4194304] 0 2024-09-21T04:35:35.458 INFO:tasks.workunit.client.0.smithi007.stdout:9/837: read d13/d89/fd1 [2678484,109377] 0 2024-09-21T04:35:35.462 INFO:tasks.workunit.client.0.smithi007.stdout:4/828: getdents d2/d17/d2e/d45/d8e 0 2024-09-21T04:35:35.469 INFO:tasks.workunit.client.0.smithi007.stdout:9/838: dwrite d13/d18/d60/d6d/f6f [0,4194304] 0 2024-09-21T04:35:35.470 INFO:tasks.workunit.client.0.smithi007.stdout:4/829: mknod d2/d7/c11a 0 2024-09-21T04:35:35.471 INFO:tasks.workunit.client.0.smithi007.stdout:9/839: stat d13/d18/d60/d6d/d85/da1/d101 0 2024-09-21T04:35:35.473 INFO:tasks.workunit.client.0.smithi007.stdout:9/840: truncate d13/d18/d60/d83/da9/ff5 981413 0 2024-09-21T04:35:35.478 INFO:tasks.workunit.client.0.smithi007.stdout:4/830: chown d2/d17/d2e/d45/d5d/d8a/fd9 0 1 2024-09-21T04:35:35.480 INFO:tasks.workunit.client.0.smithi007.stdout:1/884: write d5/d2f/fc9 [1006985,110979] 0 2024-09-21T04:35:35.484 INFO:tasks.workunit.client.0.smithi007.stdout:4/831: dread d2/d7/f8 [0,4194304] 0 2024-09-21T04:35:35.492 INFO:tasks.workunit.client.0.smithi007.stdout:4/832: dwrite d2/f117 [0,4194304] 0 2024-09-21T04:35:35.492 INFO:tasks.workunit.client.0.smithi007.stdout:9/841: fsync d13/d18/d60/d6d/f6f 0 2024-09-21T04:35:35.499 INFO:tasks.workunit.client.0.smithi007.stdout:1/885: dread d5/d6/fa9 [4194304,4194304] 0 2024-09-21T04:35:35.500 INFO:tasks.workunit.client.0.smithi007.stdout:9/842: unlink d13/d18/d60/l76 0 2024-09-21T04:35:35.501 INFO:tasks.workunit.client.0.smithi007.stdout:9/843: dread - d13/d18/d60/d6d/d85/ff9 zero size 2024-09-21T04:35:35.501 INFO:tasks.workunit.client.0.smithi007.stdout:4/833: rename d2/d17/l4d to d2/d7/d64/db8/l11b 0 2024-09-21T04:35:35.502 INFO:tasks.workunit.client.0.smithi007.stdout:9/844: chown d13/d74/de1/d105 1954913 1 2024-09-21T04:35:35.503 INFO:tasks.workunit.client.0.smithi007.stdout:4/834: fdatasync d2/d17/d2e/f33 0 2024-09-21T04:35:35.503 INFO:tasks.workunit.client.0.smithi007.stdout:1/886: symlink d5/d6/d21/d64/l11c 0 2024-09-21T04:35:35.506 INFO:tasks.workunit.client.0.smithi007.stdout:9/845: truncate d13/d18/d80/dea/f103 617438 0 2024-09-21T04:35:35.509 INFO:tasks.workunit.client.0.smithi007.stdout:1/887: dread d5/d6/fa [0,4194304] 0 2024-09-21T04:35:35.516 INFO:tasks.workunit.client.0.smithi007.stdout:1/888: mknod d5/d6/d21/d2b/d4b/da6/d58/dcf/c11d 0 2024-09-21T04:35:35.517 INFO:tasks.workunit.client.0.smithi007.stdout:9/846: dread d13/f21 [4194304,4194304] 0 2024-09-21T04:35:35.522 INFO:tasks.workunit.client.0.smithi007.stdout:4/835: rmdir d2/d17/d2e/d45/d5d/d10a/d112 0 2024-09-21T04:35:35.523 INFO:tasks.workunit.client.0.smithi007.stdout:1/889: stat d5/d6/d21/d2b/c6d 0 2024-09-21T04:35:35.527 INFO:tasks.workunit.client.0.smithi007.stdout:9/847: dwrite d13/d18/d60/d6d/d85/ff9 [0,4194304] 0 2024-09-21T04:35:35.528 INFO:tasks.workunit.client.0.smithi007.stdout:9/848: dread - d13/fe5 zero size 2024-09-21T04:35:35.534 INFO:tasks.workunit.client.0.smithi007.stdout:1/890: write d5/d6/d21/d2b/d4b/da6/d58/f67 [5971528,40823] 0 2024-09-21T04:35:35.535 INFO:tasks.workunit.client.0.smithi007.stdout:4/836: truncate d2/d7/f96 306683 0 2024-09-21T04:35:35.537 INFO:tasks.workunit.client.0.smithi007.stdout:9/849: dwrite d13/f4f [0,4194304] 0 2024-09-21T04:35:35.539 INFO:tasks.workunit.client.0.smithi007.stdout:1/891: write d5/d6/d21/d2b/d4b/d8a/dee/f107 [1001999,10923] 0 2024-09-21T04:35:35.546 INFO:tasks.workunit.client.0.smithi007.stdout:4/837: rename d2/d17/d2e/d45/d5d/d101/f102 to d2/d7/dc5/dd6/f11c 0 2024-09-21T04:35:35.547 INFO:tasks.workunit.client.0.smithi007.stdout:4/838: stat d2/d17/d2e/f3e 0 2024-09-21T04:35:35.548 INFO:tasks.workunit.client.0.smithi007.stdout:4/839: write d2/d3c/fee [1008100,124006] 0 2024-09-21T04:35:35.550 INFO:tasks.workunit.client.0.smithi007.stdout:1/892: symlink d5/d6/d21/d3c/dd5/dea/dfc/l11e 0 2024-09-21T04:35:35.552 INFO:tasks.workunit.client.0.smithi007.stdout:9/850: mkdir d13/d18/d60/d83/d111 0 2024-09-21T04:35:35.553 INFO:tasks.workunit.client.0.smithi007.stdout:9/851: truncate d13/f21 6002941 0 2024-09-21T04:35:35.554 INFO:tasks.workunit.client.0.smithi007.stdout:9/852: write d13/d18/d60/d6d/d75/fd2 [351460,35994] 0 2024-09-21T04:35:35.558 INFO:tasks.workunit.client.0.smithi007.stdout:1/893: mknod d5/d6/d21/d64/df1/da0/d99/dae/c11f 0 2024-09-21T04:35:35.558 INFO:tasks.workunit.client.0.smithi007.stdout:1/894: chown d5/f90 464457 1 2024-09-21T04:35:35.560 INFO:tasks.workunit.client.0.smithi007.stdout:9/853: mknod d13/d18/d60/d6d/c112 0 2024-09-21T04:35:35.562 INFO:tasks.workunit.client.0.smithi007.stdout:1/895: creat d5/d6/d21/d2b/d4b/da6/d58/dde/f120 x:0 0 0 2024-09-21T04:35:35.563 INFO:tasks.workunit.client.0.smithi007.stdout:9/854: mkdir d13/d18/d60/d6d/d75/d113 0 2024-09-21T04:35:35.564 INFO:tasks.workunit.client.0.smithi007.stdout:9/855: dread - d13/d74/ddb/fcb zero size 2024-09-21T04:35:35.565 INFO:tasks.workunit.client.0.smithi007.stdout:9/856: read - d13/d89/fb9 zero size 2024-09-21T04:35:35.568 INFO:tasks.workunit.client.0.smithi007.stdout:9/857: unlink d13/d18/d60/d6d/fb2 0 2024-09-21T04:35:35.572 INFO:tasks.workunit.client.0.smithi007.stdout:9/858: mknod d13/d18/d60/d6d/d85/da1/daa/c114 0 2024-09-21T04:35:35.578 INFO:tasks.workunit.client.0.smithi007.stdout:9/859: getdents d13/d18/daf/dda 0 2024-09-21T04:35:35.578 INFO:tasks.workunit.client.0.smithi007.stdout:4/840: dwrite d2/d17/d29/fbf [0,4194304] 0 2024-09-21T04:35:35.579 INFO:tasks.workunit.client.0.smithi007.stdout:9/860: readlink d13/d18/l73 0 2024-09-21T04:35:35.583 INFO:tasks.workunit.client.0.smithi007.stdout:4/841: creat d2/d17/d2e/d45/d8e/f11d x:0 0 0 2024-09-21T04:35:35.583 INFO:tasks.workunit.client.0.smithi007.stdout:1/896: dwrite d5/f28 [0,4194304] 0 2024-09-21T04:35:35.583 INFO:tasks.workunit.client.0.smithi007.stdout:9/861: creat d13/d5a/f115 x:0 0 0 2024-09-21T04:35:35.584 INFO:tasks.workunit.client.0.smithi007.stdout:4/842: chown d2/d7/dc9 1075883 1 2024-09-21T04:35:35.585 INFO:tasks.workunit.client.0.smithi007.stdout:1/897: chown d5/d6/cbb 31 1 2024-09-21T04:35:35.586 INFO:tasks.workunit.client.0.smithi007.stdout:9/862: unlink d13/d18/f67 0 2024-09-21T04:35:35.587 INFO:tasks.workunit.client.0.smithi007.stdout:4/843: mkdir d2/d17/d2e/d45/d5d/dc7/d11e 0 2024-09-21T04:35:35.588 INFO:tasks.workunit.client.0.smithi007.stdout:9/863: write d13/d18/d60/d6d/d75/d9f/feb [108854,90827] 0 2024-09-21T04:35:35.589 INFO:tasks.workunit.client.0.smithi007.stdout:9/864: chown d13/d18/d80/cbe 177 1 2024-09-21T04:35:35.589 INFO:tasks.workunit.client.0.smithi007.stdout:1/898: rename d5/d6/d21/d2b/d4b/da6/d58/f67 to d5/d6/dc5/de6/f121 0 2024-09-21T04:35:35.593 INFO:tasks.workunit.client.0.smithi007.stdout:4/844: truncate d2/d7/d55/d4f/f9d 2145038 0 2024-09-21T04:35:35.594 INFO:tasks.workunit.client.0.smithi007.stdout:1/899: symlink d5/d6/d21/d64/df1/da0/d99/de2/l122 0 2024-09-21T04:35:35.595 INFO:tasks.workunit.client.0.smithi007.stdout:4/845: mknod d2/d97/da5/c11f 0 2024-09-21T04:35:35.597 INFO:tasks.workunit.client.0.smithi007.stdout:1/900: rmdir d5/d45 39 2024-09-21T04:35:35.598 INFO:tasks.workunit.client.0.smithi007.stdout:9/865: getdents d13 0 2024-09-21T04:35:35.599 INFO:tasks.workunit.client.0.smithi007.stdout:4/846: dread d2/d7/f8 [0,4194304] 0 2024-09-21T04:35:35.600 INFO:tasks.workunit.client.0.smithi007.stdout:4/847: chown d2/d17/f2a 507 1 2024-09-21T04:35:35.601 INFO:tasks.workunit.client.0.smithi007.stdout:1/901: mknod d5/d6/d21/d64/df1/da0/d99/dcd/c123 0 2024-09-21T04:35:35.603 INFO:tasks.workunit.client.0.smithi007.stdout:9/866: dread d13/d18/d60/d6d/d85/fd3 [0,4194304] 0 2024-09-21T04:35:35.606 INFO:tasks.workunit.client.0.smithi007.stdout:1/902: dread d5/d6/d21/d64/df1/f7b [0,4194304] 0 2024-09-21T04:35:35.609 INFO:tasks.workunit.client.0.smithi007.stdout:9/867: dread d13/d18/f3f [0,4194304] 0 2024-09-21T04:35:35.612 INFO:tasks.workunit.client.0.smithi007.stdout:1/903: fsync d5/d6/d21/d2b/d4b/da6/f9c 0 2024-09-21T04:35:35.615 INFO:tasks.workunit.client.0.smithi007.stdout:1/904: chown d5/d6/d21/d2b/d86/dd8/cb3 97673 1 2024-09-21T04:35:35.616 INFO:tasks.workunit.client.0.smithi007.stdout:4/848: dread d2/d7/f48 [0,4194304] 0 2024-09-21T04:35:35.617 INFO:tasks.workunit.client.0.smithi007.stdout:9/868: dwrite d13/d18/daf/fdd [0,4194304] 0 2024-09-21T04:35:35.621 INFO:tasks.workunit.client.0.smithi007.stdout:1/905: creat d5/d6/d21/d2b/d86/f124 x:0 0 0 2024-09-21T04:35:35.622 INFO:tasks.workunit.client.0.smithi007.stdout:4/849: mkdir d2/d17/d2e/d45/df2/d120 0 2024-09-21T04:35:35.622 INFO:tasks.workunit.client.0.smithi007.stdout:1/906: write d5/d6/d21/d2b/d86/dd8/f6e [1437149,21058] 0 2024-09-21T04:35:35.627 INFO:tasks.workunit.client.0.smithi007.stdout:4/850: mknod d2/d88/da8/c121 0 2024-09-21T04:35:35.629 INFO:tasks.workunit.client.0.smithi007.stdout:1/907: creat d5/d6/d21/d2b/d4b/da6/d58/dde/d10a/f125 x:0 0 0 2024-09-21T04:35:35.635 INFO:tasks.workunit.client.0.smithi007.stdout:9/869: link d13/c69 d13/d18/d60/d83/da9/c116 0 2024-09-21T04:35:35.635 INFO:tasks.workunit.client.0.smithi007.stdout:1/908: dwrite d5/d6/f119 [0,4194304] 0 2024-09-21T04:35:35.642 INFO:tasks.workunit.client.0.smithi007.stdout:9/870: creat d13/d18/d60/d83/f117 x:0 0 0 2024-09-21T04:35:35.643 INFO:tasks.workunit.client.0.smithi007.stdout:1/909: creat d5/d6/d21/d64/df1/da0/d99/dcd/f126 x:0 0 0 2024-09-21T04:35:35.644 INFO:tasks.workunit.client.0.smithi007.stdout:1/910: chown d5/d6/f119 315355220 1 2024-09-21T04:35:35.648 INFO:tasks.workunit.client.0.smithi007.stdout:1/911: creat d5/d6/d21/d2b/d86/f127 x:0 0 0 2024-09-21T04:35:35.651 INFO:tasks.workunit.client.0.smithi007.stdout:9/871: symlink d13/d18/d60/d6d/d75/l118 0 2024-09-21T04:35:35.651 INFO:tasks.workunit.client.0.smithi007.stdout:9/872: dread - d13/d18/dcd/ff8 zero size 2024-09-21T04:35:35.655 INFO:tasks.workunit.client.0.smithi007.stdout:4/851: truncate d2/d7/d55/d4f/fbc 355481 0 2024-09-21T04:35:35.656 INFO:tasks.workunit.client.0.smithi007.stdout:1/912: rename d5/d6/d21/d2b/d4b/da6/f65 to d5/d6/d21/d64/df1/f128 0 2024-09-21T04:35:35.659 INFO:tasks.workunit.client.0.smithi007.stdout:1/913: symlink d5/d6/d21/d2b/d4b/l129 0 2024-09-21T04:35:35.662 INFO:tasks.workunit.client.0.smithi007.stdout:1/914: write d5/d6/f40 [611300,48550] 0 2024-09-21T04:35:35.662 INFO:tasks.workunit.client.0.smithi007.stdout:4/852: link d2/d17/d18/c28 d2/d7/d64/c122 0 2024-09-21T04:35:35.670 INFO:tasks.workunit.client.0.smithi007.stdout:9/873: dwrite d13/d18/d60/d6d/d75/fa0 [0,4194304] 0 2024-09-21T04:35:35.676 INFO:tasks.workunit.client.0.smithi007.stdout:4/853: dwrite d2/d3c/d63/fdc [4194304,4194304] 0 2024-09-21T04:35:35.678 INFO:tasks.workunit.client.0.smithi007.stdout:9/874: rmdir d13/d18/d60/d6d/d75/d9f/dc8 39 2024-09-21T04:35:35.679 INFO:tasks.workunit.client.0.smithi007.stdout:1/915: dwrite d5/d6/d21/d2b/d4b/d8a/d8d/fca [4194304,4194304] 0 2024-09-21T04:35:35.686 INFO:tasks.workunit.client.0.smithi007.stdout:9/875: mknod d13/d18/d60/d6d/c119 0 2024-09-21T04:35:35.688 INFO:tasks.workunit.client.0.smithi007.stdout:9/876: symlink d13/d18/d60/d83/l11a 0 2024-09-21T04:35:35.691 INFO:tasks.workunit.client.0.smithi007.stdout:4/854: getdents d2 0 2024-09-21T04:35:35.692 INFO:tasks.workunit.client.0.smithi007.stdout:1/916: rename d5/d6/d21/d2b/d4b/da6/d58/dcf/lec to d5/l12a 0 2024-09-21T04:35:35.693 INFO:tasks.workunit.client.0.smithi007.stdout:9/877: unlink d13/d18/d60/d83/da9/c3b 0 2024-09-21T04:35:35.695 INFO:tasks.workunit.client.0.smithi007.stdout:9/878: symlink d13/d5a/l11b 0 2024-09-21T04:35:35.696 INFO:tasks.workunit.client.0.smithi007.stdout:9/879: read d13/f5b [70842,59963] 0 2024-09-21T04:35:35.698 INFO:tasks.workunit.client.0.smithi007.stdout:9/880: chown d13/d18/d60/d6d/d85/ff9 223 1 2024-09-21T04:35:35.699 INFO:tasks.workunit.client.0.smithi007.stdout:1/917: link d5/d6/d21/d2b/d86/f124 d5/d6/d21/d64/df1/da0/d99/f12b 0 2024-09-21T04:35:35.700 INFO:tasks.workunit.client.0.smithi007.stdout:4/855: dwrite d2/f12 [4194304,4194304] 0 2024-09-21T04:35:35.703 INFO:tasks.workunit.client.0.smithi007.stdout:4/856: fsync d2/d17/d2e/f71 0 2024-09-21T04:35:35.704 INFO:tasks.workunit.client.0.smithi007.stdout:4/857: chown d2/d17/d2e/d45/d5d/d8a/fb5 58 1 2024-09-21T04:35:35.705 INFO:tasks.workunit.client.0.smithi007.stdout:1/918: symlink d5/d6/l12c 0 2024-09-21T04:35:35.705 INFO:tasks.workunit.client.0.smithi007.stdout:9/881: symlink d13/d18/d60/d6d/d75/d113/l11c 0 2024-09-21T04:35:35.708 INFO:tasks.workunit.client.0.smithi007.stdout:1/919: mkdir d5/d6/d21/d3c/db6/d12d 0 2024-09-21T04:35:35.711 INFO:tasks.workunit.client.0.smithi007.stdout:9/882: chown d13/c69 40 1 2024-09-21T04:35:35.714 INFO:tasks.workunit.client.0.smithi007.stdout:9/883: chown d13/d18/d80/l10c 8 1 2024-09-21T04:35:35.716 INFO:tasks.workunit.client.0.smithi007.stdout:1/920: readlink d5/d6/d21/d3c/db6/ld3 0 2024-09-21T04:35:35.720 INFO:tasks.workunit.client.0.smithi007.stdout:1/921: dread d5/d6/d21/d3c/fa3 [0,4194304] 0 2024-09-21T04:35:35.725 INFO:tasks.workunit.client.0.smithi007.stdout:1/922: dread - d5/d6/d21/d64/df1/ddd/fdf zero size 2024-09-21T04:35:35.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/923: creat d5/d6/d21/d2b/d86/dd8/d10d/f12e x:0 0 0 2024-09-21T04:35:35.731 INFO:tasks.workunit.client.0.smithi007.stdout:1/924: fsync d5/d6/d21/d64/df1/ddd/fdf 0 2024-09-21T04:35:35.732 INFO:tasks.workunit.client.0.smithi007.stdout:4/858: write d2/d7/d55/d4f/f9d [176378,100362] 0 2024-09-21T04:35:35.732 INFO:tasks.workunit.client.0.smithi007.stdout:1/925: chown d5/d6/dc5 136151 1 2024-09-21T04:35:35.737 INFO:tasks.workunit.client.0.smithi007.stdout:1/926: unlink d5/d6/d21/d2b/f2d 0 2024-09-21T04:35:35.738 INFO:tasks.workunit.client.0.smithi007.stdout:1/927: read d5/d6/d21/d2b/d86/dd8/f6e [3887049,92361] 0 2024-09-21T04:35:35.741 INFO:tasks.workunit.client.0.smithi007.stdout:1/928: mknod d5/d6/d21/d64/df1/da0/c12f 0 2024-09-21T04:35:35.745 INFO:tasks.workunit.client.0.smithi007.stdout:4/859: write d2/d7/f59 [2733941,114296] 0 2024-09-21T04:35:35.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/860: write d2/d17/d2e/d45/d8e/f11d [430552,18354] 0 2024-09-21T04:35:35.753 INFO:tasks.workunit.client.0.smithi007.stdout:4/861: dwrite d2/f27 [0,4194304] 0 2024-09-21T04:35:35.757 INFO:tasks.workunit.client.0.smithi007.stdout:4/862: symlink d2/d17/d2e/d45/l123 0 2024-09-21T04:35:35.760 INFO:tasks.workunit.client.0.smithi007.stdout:4/863: unlink d2/d17/d29/fbf 0 2024-09-21T04:35:35.761 INFO:tasks.workunit.client.0.smithi007.stdout:4/864: dread - d2/d7/dc5/dd6/f11c zero size 2024-09-21T04:35:35.762 INFO:tasks.workunit.client.0.smithi007.stdout:4/865: chown d2/d7/f24 292507 1 2024-09-21T04:35:35.765 INFO:tasks.workunit.client.0.smithi007.stdout:4/866: unlink d2/d17/d2e/f95 0 2024-09-21T04:35:35.785 INFO:tasks.workunit.client.0.smithi007.stdout:4/867: dwrite d2/d7/f96 [0,4194304] 0 2024-09-21T04:35:35.790 INFO:tasks.workunit.client.0.smithi007.stdout:4/868: creat d2/d7/d64/db8/f124 x:0 0 0 2024-09-21T04:35:35.796 INFO:tasks.workunit.client.0.smithi007.stdout:1/929: dread d5/d6/d21/d2b/f6f [0,4194304] 0 2024-09-21T04:35:35.802 INFO:tasks.workunit.client.0.smithi007.stdout:4/869: truncate d2/d7/f96 1121876 0 2024-09-21T04:35:35.805 INFO:tasks.workunit.client.0.smithi007.stdout:1/930: rename d5/d6/d21/d3c to d5/d6/d21/d2b/d130 0 2024-09-21T04:35:35.806 INFO:tasks.workunit.client.0.smithi007.stdout:9/884: dread d13/f21 [0,4194304] 0 2024-09-21T04:35:35.837 INFO:tasks.workunit.client.0.smithi007.stdout:4/870: truncate d2/d17/d2e/d45/d5d/d8a/fd9 473901 0 2024-09-21T04:35:35.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/871: fdatasync d2/d3c/f53 0 2024-09-21T04:35:35.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/872: stat d2/d17/d18/df0 0 2024-09-21T04:35:35.847 INFO:tasks.workunit.client.0.smithi007.stdout:1/931: dread - d5/d45/fb0 zero size 2024-09-21T04:35:35.852 INFO:tasks.workunit.client.0.smithi007.stdout:1/932: dread d5/d2f/fc9 [0,4194304] 0 2024-09-21T04:35:35.854 INFO:tasks.workunit.client.0.smithi007.stdout:4/873: truncate d2/d3c/d63/fa3 1540232 0 2024-09-21T04:35:35.857 INFO:tasks.workunit.client.0.smithi007.stdout:1/933: symlink d5/d6/da2/l131 0 2024-09-21T04:35:35.859 INFO:tasks.workunit.client.0.smithi007.stdout:4/874: creat d2/d3c/dbe/f125 x:0 0 0 2024-09-21T04:35:35.861 INFO:tasks.workunit.client.0.smithi007.stdout:9/885: dread d13/d18/d60/d6d/d85/da1/daa/fb8 [0,4194304] 0 2024-09-21T04:35:35.867 INFO:tasks.workunit.client.0.smithi007.stdout:4/875: dwrite d2/f27 [0,4194304] 0 2024-09-21T04:35:35.867 INFO:tasks.workunit.client.0.smithi007.stdout:9/886: rename d13/l5d to d13/d74/de1/l11d 0 2024-09-21T04:35:35.870 INFO:tasks.workunit.client.0.smithi007.stdout:4/876: mknod d2/d7/d55/d61/c126 0 2024-09-21T04:35:35.877 INFO:tasks.workunit.client.0.smithi007.stdout:4/877: dwrite d2/d5f/ff1 [0,4194304] 0 2024-09-21T04:35:35.885 INFO:tasks.workunit.client.0.smithi007.stdout:1/934: link d5/d6/dc5/de6/f121 d5/d45/f132 0 2024-09-21T04:35:35.921 INFO:tasks.workunit.client.0.smithi007.stdout:4/878: write d2/d17/d2e/d45/d5d/d101/f115 [777662,47290] 0 2024-09-21T04:35:35.923 INFO:tasks.workunit.client.0.smithi007.stdout:1/935: write d5/d6/d21/f2c [2214056,12695] 0 2024-09-21T04:35:35.925 INFO:tasks.workunit.client.0.smithi007.stdout:1/936: write d5/d6/f100 [33767,113115] 0 2024-09-21T04:35:35.927 INFO:tasks.workunit.client.0.smithi007.stdout:4/879: dread d2/d7/d55/d4f/f5c [0,4194304] 0 2024-09-21T04:35:35.928 INFO:tasks.workunit.client.0.smithi007.stdout:1/937: symlink d5/d6/d21/d64/df1/ddd/l133 0 2024-09-21T04:35:35.929 INFO:tasks.workunit.client.0.smithi007.stdout:1/938: dread - d5/d45/fb0 zero size 2024-09-21T04:35:35.930 INFO:tasks.workunit.client.0.smithi007.stdout:4/880: creat d2/d97/da5/f127 x:0 0 0 2024-09-21T04:35:35.932 INFO:tasks.workunit.client.0.smithi007.stdout:4/881: symlink d2/d17/d2e/d45/d8e/l128 0 2024-09-21T04:35:35.934 INFO:tasks.workunit.client.0.smithi007.stdout:4/882: mkdir d2/d17/d2e/d45/d5d/d101/d129 0 2024-09-21T04:35:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:4/883: read - d2/d17/d2e/d45/d5d/d8a/fb5 zero size 2024-09-21T04:35:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:9/887: write d13/d18/d60/d6d/d75/d9f/fad [1360773,72586] 0 2024-09-21T04:35:35.940 INFO:tasks.workunit.client.0.smithi007.stdout:9/888: chown d13/d18/c58 1047628678 1 2024-09-21T04:35:35.942 INFO:tasks.workunit.client.0.smithi007.stdout:4/884: rename d2/d17/d2e/d45/c67 to d2/d17/d29/c12a 0 2024-09-21T04:35:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:9/889: mknod d13/d89/c11e 0 2024-09-21T04:35:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/890: chown d13/d18/d60/d6d/d85/da1 236988590 1 2024-09-21T04:35:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:4/885: truncate d2/d17/d18/f84 1534679 0 2024-09-21T04:35:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:1/939: write d5/d6/d21/d64/df1/da0/d99/fef [410811,47050] 0 2024-09-21T04:35:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:9/891: creat d13/d18/d60/d6d/d85/da1/daa/f11f x:0 0 0 2024-09-21T04:35:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:6/683: sync 2024-09-21T04:35:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:4/886: symlink d2/d3c/l12b 0 2024-09-21T04:35:35.950 INFO:tasks.workunit.client.0.smithi007.stdout:1/940: mknod d5/d6/d21/d2b/d4b/da6/d58/c134 0 2024-09-21T04:35:35.951 INFO:tasks.workunit.client.0.smithi007.stdout:1/941: write d5/d6/d21/d2b/d4b/da6/d58/f73 [3331855,74985] 0 2024-09-21T04:35:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:6/684: mkdir d3/d3b/dd3/d60/d98/de4 0 2024-09-21T04:35:35.954 INFO:tasks.workunit.client.0.smithi007.stdout:9/892: mkdir d13/d74/de1/dec/d120 0 2024-09-21T04:35:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:1/942: symlink d5/d6/d21/d2b/d86/l135 0 2024-09-21T04:35:35.961 INFO:tasks.workunit.client.0.smithi007.stdout:1/943: creat d5/d6/d21/d2b/d130/db6/f136 x:0 0 0 2024-09-21T04:35:35.962 INFO:tasks.workunit.client.0.smithi007.stdout:9/893: dwrite d13/d18/d60/d6d/d85/da1/fa5 [4194304,4194304] 0 2024-09-21T04:35:35.962 INFO:tasks.workunit.client.0.smithi007.stdout:6/685: dread d3/d3e/d99/dbb/f5d [4194304,4194304] 0 2024-09-21T04:35:35.964 INFO:tasks.workunit.client.0.smithi007.stdout:1/944: read - d5/d6/d21/d64/df1/da0/d99/f12b zero size 2024-09-21T04:35:35.970 INFO:tasks.workunit.client.0.smithi007.stdout:1/945: creat d5/d6/d21/f137 x:0 0 0 2024-09-21T04:35:35.971 INFO:tasks.workunit.client.0.smithi007.stdout:4/887: write d2/d7/dfb/f113 [1033672,115209] 0 2024-09-21T04:35:35.973 INFO:tasks.workunit.client.0.smithi007.stdout:1/946: write d5/d6/d21/d2b/d86/dd8/d10d/f12e [731892,3171] 0 2024-09-21T04:35:35.975 INFO:tasks.workunit.client.0.smithi007.stdout:4/888: truncate d2/d7/dc5/dd6/fd7 336969 0 2024-09-21T04:35:35.976 INFO:tasks.workunit.client.0.smithi007.stdout:6/686: creat d3/de/d12/fe5 x:0 0 0 2024-09-21T04:35:35.979 INFO:tasks.workunit.client.0.smithi007.stdout:4/889: mknod d2/d17/d2e/d45/d5d/d8a/dd2/c12c 0 2024-09-21T04:35:35.981 INFO:tasks.workunit.client.0.smithi007.stdout:6/687: rename d3/de/f10 to d3/de/d12/d19/da3/fe6 0 2024-09-21T04:35:35.983 INFO:tasks.workunit.client.0.smithi007.stdout:4/890: chown d2/d17/d29/c12a 11474 1 2024-09-21T04:35:35.987 INFO:tasks.workunit.client.0.smithi007.stdout:1/947: getdents d5/d6/d21/d64/df1 0 2024-09-21T04:35:35.990 INFO:tasks.workunit.client.0.smithi007.stdout:4/891: dwrite d2/d3c/f106 [4194304,4194304] 0 2024-09-21T04:35:35.994 INFO:tasks.workunit.client.0.smithi007.stdout:1/948: rename d5/d6/d21/d64/df1/ddd/ff6 to d5/d6/d21/d2b/d4b/d8a/f138 0 2024-09-21T04:35:35.996 INFO:tasks.workunit.client.0.smithi007.stdout:1/949: read d5/d6/d21/d2b/d86/dd8/d10d/f12e [567350,73758] 0 2024-09-21T04:35:35.997 INFO:tasks.workunit.client.0.smithi007.stdout:6/688: truncate d3/f15 945983 0 2024-09-21T04:35:35.998 INFO:tasks.workunit.client.0.smithi007.stdout:4/892: link d2/d88/da8/db2/l116 d2/d7/dfb/l12d 0 2024-09-21T04:35:36.000 INFO:tasks.workunit.client.0.smithi007.stdout:1/950: unlink d5/d6/d21/d2b/d4b/l129 0 2024-09-21T04:35:36.003 INFO:tasks.workunit.client.0.smithi007.stdout:6/689: rename d3/d3b/dd3/d60/d77/dcb/d78/l71 to d3/d3e/d99/dbb/le7 0 2024-09-21T04:35:36.004 INFO:tasks.workunit.client.0.smithi007.stdout:6/690: truncate d3/de/d12/f8d 5128344 0 2024-09-21T04:35:36.005 INFO:tasks.workunit.client.0.smithi007.stdout:6/691: readlink d3/d94/lc6 0 2024-09-21T04:35:36.006 INFO:tasks.workunit.client.0.smithi007.stdout:4/893: symlink d2/d3c/l12e 0 2024-09-21T04:35:36.008 INFO:tasks.workunit.client.0.smithi007.stdout:6/692: rename d3/d3b/dd3/f68 to d3/d3e/d99/dbb/d95/fe8 0 2024-09-21T04:35:36.010 INFO:tasks.workunit.client.0.smithi007.stdout:4/894: readlink d2/d17/d2e/d45/l5b 0 2024-09-21T04:35:36.018 INFO:tasks.workunit.client.0.smithi007.stdout:4/895: dread d2/d97/da5/fad [0,4194304] 0 2024-09-21T04:35:36.020 INFO:tasks.workunit.client.0.smithi007.stdout:1/951: dwrite d5/d6/d21/d2b/f51 [0,4194304] 0 2024-09-21T04:35:36.021 INFO:tasks.workunit.client.0.smithi007.stdout:4/896: write d2/d17/d2e/d45/d5d/d8a/fd5 [705433,22040] 0 2024-09-21T04:35:36.023 INFO:tasks.workunit.client.0.smithi007.stdout:6/693: dread d3/de/f49 [4194304,4194304] 0 2024-09-21T04:35:36.025 INFO:tasks.workunit.client.0.smithi007.stdout:4/897: rmdir d2/d7 39 2024-09-21T04:35:36.027 INFO:tasks.workunit.client.0.smithi007.stdout:1/952: symlink d5/d6/d21/d2b/d4b/l139 0 2024-09-21T04:35:36.030 INFO:tasks.workunit.client.0.smithi007.stdout:1/953: mknod d5/d6/dc5/de6/c13a 0 2024-09-21T04:35:36.031 INFO:tasks.workunit.client.0.smithi007.stdout:1/954: dread - d5/f25 zero size 2024-09-21T04:35:36.033 INFO:tasks.workunit.client.0.smithi007.stdout:6/694: getdents d3/de/d12/d19 0 2024-09-21T04:35:36.034 INFO:tasks.workunit.client.0.smithi007.stdout:6/695: dread - d3/de/d12/fe5 zero size 2024-09-21T04:35:36.036 INFO:tasks.workunit.client.0.smithi007.stdout:1/955: mknod d5/d6/d21/d2b/d130/dd5/dea/dfc/c13b 0 2024-09-21T04:35:36.038 INFO:tasks.workunit.client.0.smithi007.stdout:6/696: read d3/de/f1a [738897,107817] 0 2024-09-21T04:35:36.042 INFO:tasks.workunit.client.0.smithi007.stdout:4/898: dread d2/f25 [0,4194304] 0 2024-09-21T04:35:36.043 INFO:tasks.workunit.client.0.smithi007.stdout:6/697: symlink d3/de/le9 0 2024-09-21T04:35:36.043 INFO:tasks.workunit.client.0.smithi007.stdout:1/956: dwrite d5/d6/f119 [0,4194304] 0 2024-09-21T04:35:36.050 INFO:tasks.workunit.client.0.smithi007.stdout:1/957: creat d5/d6/d21/d2b/d4b/d8a/f13c x:0 0 0 2024-09-21T04:35:36.053 INFO:tasks.workunit.client.0.smithi007.stdout:4/899: fdatasync d2/d97/da5/fb0 0 2024-09-21T04:35:36.059 INFO:tasks.workunit.client.0.smithi007.stdout:4/900: truncate d2/d7/dc5/dd6/fd7 417939 0 2024-09-21T04:35:36.060 INFO:tasks.workunit.client.0.smithi007.stdout:6/698: rmdir d3/d8e/dae/dbd 0 2024-09-21T04:35:36.061 INFO:tasks.workunit.client.0.smithi007.stdout:1/958: unlink d5/d6/d21/dd4/le0 0 2024-09-21T04:35:36.062 INFO:tasks.workunit.client.0.smithi007.stdout:1/959: stat d5/d6/d21/d64/df1/da0/d99/de2 0 2024-09-21T04:35:36.065 INFO:tasks.workunit.client.0.smithi007.stdout:4/901: mknod d2/d88/da8/d114/c12f 0 2024-09-21T04:35:36.066 INFO:tasks.workunit.client.0.smithi007.stdout:1/960: mkdir d5/d45/d13d 0 2024-09-21T04:35:36.067 INFO:tasks.workunit.client.0.smithi007.stdout:6/699: getdents d3/d3b/dd3/d60/d98 0 2024-09-21T04:35:36.072 INFO:tasks.workunit.client.0.smithi007.stdout:1/961: rename d5/d6/d21/d2b/d4b/f74 to d5/d6/d21/d64/df1/da0/d99/dae/f13e 0 2024-09-21T04:35:36.076 INFO:tasks.workunit.client.0.smithi007.stdout:1/962: fsync d5/d45/d5e/f24 0 2024-09-21T04:35:36.077 INFO:tasks.workunit.client.0.smithi007.stdout:6/700: dwrite d3/d94/fc2 [0,4194304] 0 2024-09-21T04:35:36.078 INFO:tasks.workunit.client.0.smithi007.stdout:4/902: dwrite d2/d3c/dbe/f110 [0,4194304] 0 2024-09-21T04:35:36.088 INFO:tasks.workunit.client.0.smithi007.stdout:4/903: dwrite d2/d7/dc5/dd6/f11c [0,4194304] 0 2024-09-21T04:35:36.093 INFO:tasks.workunit.client.0.smithi007.stdout:6/701: write d3/d3e/d99/dbb/f5d [6240440,86876] 0 2024-09-21T04:35:36.094 INFO:tasks.workunit.client.0.smithi007.stdout:4/904: rename d2/d17/d2e/d45/d5d/d8a/dd2/c12c to d2/d17/d29/c130 0 2024-09-21T04:35:36.095 INFO:tasks.workunit.client.0.smithi007.stdout:1/963: write d5/d6/d21/d2b/d4b/da6/d58/fb7 [233992,42028] 0 2024-09-21T04:35:36.097 INFO:tasks.workunit.client.0.smithi007.stdout:1/964: chown d5/d6/d21/d2b/d4b/d8a/d8d/fd6 17 1 2024-09-21T04:35:36.098 INFO:tasks.workunit.client.0.smithi007.stdout:4/905: fsync d2/fb6 0 2024-09-21T04:35:36.100 INFO:tasks.workunit.client.0.smithi007.stdout:1/965: mknod d5/d6/d21/d2b/d4b/da6/d58/dde/d10a/c13f 0 2024-09-21T04:35:36.101 INFO:tasks.workunit.client.0.smithi007.stdout:4/906: symlink d2/d17/d2e/d45/l131 0 2024-09-21T04:35:36.102 INFO:tasks.workunit.client.0.smithi007.stdout:1/966: symlink d5/d6/d21/d2b/d130/dd5/dea/dfc/l140 0 2024-09-21T04:35:36.102 INFO:tasks.workunit.client.0.smithi007.stdout:4/907: stat d2/d3c/l6f 0 2024-09-21T04:35:36.104 INFO:tasks.workunit.client.0.smithi007.stdout:1/967: unlink d5/d6/d21/d2b/d86/f127 0 2024-09-21T04:35:36.106 INFO:tasks.workunit.client.0.smithi007.stdout:4/908: mkdir d2/d17/d2e/d45/d5d/d10a/d132 0 2024-09-21T04:35:36.108 INFO:tasks.workunit.client.0.smithi007.stdout:1/968: stat d5/d6/d21/ce7 0 2024-09-21T04:35:36.110 INFO:tasks.workunit.client.0.smithi007.stdout:6/702: truncate d3/de/d12/d19/f36 4847738 0 2024-09-21T04:35:36.113 INFO:tasks.workunit.client.0.smithi007.stdout:1/969: rename d5/d6/d21/d64/df1/da0/d99/dae/fd1 to d5/d6/d21/d2b/d130/dd5/dea/f141 0 2024-09-21T04:35:36.114 INFO:tasks.workunit.client.0.smithi007.stdout:6/703: rmdir d3/d3e/d99/dbb/d95/dc9 39 2024-09-21T04:35:36.116 INFO:tasks.workunit.client.0.smithi007.stdout:4/909: link d2/f12 d2/d17/d29/de9/f133 0 2024-09-21T04:35:36.117 INFO:tasks.workunit.client.0.smithi007.stdout:4/910: chown d2/d7/d64/db8/fea 1 1 2024-09-21T04:35:36.118 INFO:tasks.workunit.client.0.smithi007.stdout:4/911: chown d2/d17/d2e/d45/df2/d120 234 1 2024-09-21T04:35:36.119 INFO:tasks.workunit.client.0.smithi007.stdout:4/912: fsync d2/d17/d2e/d45/d5d/d101/f115 0 2024-09-21T04:35:36.121 INFO:tasks.workunit.client.0.smithi007.stdout:1/970: creat d5/d6/d21/d2b/d4b/da6/df2/f142 x:0 0 0 2024-09-21T04:35:36.122 INFO:tasks.workunit.client.0.smithi007.stdout:1/971: readlink d5/d6/d21/d64/df1/ddd/l133 0 2024-09-21T04:35:36.125 INFO:tasks.workunit.client.0.smithi007.stdout:4/913: mkdir d2/d17/d2e/d45/d134 0 2024-09-21T04:35:36.127 INFO:tasks.workunit.client.0.smithi007.stdout:1/972: rename d5/d6/d21/d2b/d4b/l69 to d5/d6/d21/d2b/d4b/da6/d58/l143 0 2024-09-21T04:35:36.131 INFO:tasks.workunit.client.0.smithi007.stdout:4/914: mkdir d2/d17/d2e/d45/df2/d120/d135 0 2024-09-21T04:35:36.132 INFO:tasks.workunit.client.0.smithi007.stdout:4/915: fsync d2/d3c/dbe/f125 0 2024-09-21T04:35:36.135 INFO:tasks.workunit.client.0.smithi007.stdout:4/916: creat d2/d88/da8/db2/f136 x:0 0 0 2024-09-21T04:35:36.138 INFO:tasks.workunit.client.0.smithi007.stdout:1/973: dwrite d5/d6/d21/d2b/d4b/d8a/dee/f107 [0,4194304] 0 2024-09-21T04:35:36.141 INFO:tasks.workunit.client.0.smithi007.stdout:4/917: read d2/d5f/f93 [2162442,29216] 0 2024-09-21T04:35:36.142 INFO:tasks.workunit.client.0.smithi007.stdout:6/704: dwrite d3/f28 [0,4194304] 0 2024-09-21T04:35:36.144 INFO:tasks.workunit.client.0.smithi007.stdout:6/705: stat d3/de/d12/d19/da3/fe6 0 2024-09-21T04:35:36.144 INFO:tasks.workunit.client.0.smithi007.stdout:1/974: read d5/fc [3296477,121005] 0 2024-09-21T04:35:36.146 INFO:tasks.workunit.client.0.smithi007.stdout:4/918: dread d2/d3c/f106 [4194304,4194304] 0 2024-09-21T04:35:36.147 INFO:tasks.workunit.client.0.smithi007.stdout:1/975: chown d5/d6/d21/d64/df1/da0/d99/dae/c11f 0 1 2024-09-21T04:35:36.148 INFO:tasks.workunit.client.0.smithi007.stdout:1/976: readlink d5/d6/d21/d2b/d130/dd5/dea/dfc/l11e 0 2024-09-21T04:35:36.255 INFO:tasks.workunit.client.0.smithi007.stdout:3/811: sync 2024-09-21T04:35:36.255 INFO:tasks.workunit.client.0.smithi007.stdout:7/718: sync 2024-09-21T04:35:36.255 INFO:tasks.workunit.client.0.smithi007.stdout:5/785: sync 2024-09-21T04:35:36.255 INFO:tasks.workunit.client.0.smithi007.stdout:0/721: sync 2024-09-21T04:35:36.255 INFO:tasks.workunit.client.0.smithi007.stdout:8/553: sync 2024-09-21T04:35:36.255 INFO:tasks.workunit.client.0.smithi007.stdout:2/545: sync 2024-09-21T04:35:36.258 INFO:tasks.workunit.client.0.smithi007.stdout:8/554: rename d0/d3/d16/d93/d26/d4d/d5d to d0/d3/d16/d93/d26/d4d/d5d/da0 22 2024-09-21T04:35:36.258 INFO:tasks.workunit.client.0.smithi007.stdout:6/706: mknod d3/d3b/dd3/d60/d77/dcb/d78/cea 0 2024-09-21T04:35:36.284 INFO:tasks.workunit.client.0.smithi007.stdout:1/977: symlink d5/d6/dc5/de6/l144 0 2024-09-21T04:35:36.286 INFO:tasks.workunit.client.0.smithi007.stdout:1/978: dread d5/d2f/fc9 [0,4194304] 0 2024-09-21T04:35:36.289 INFO:tasks.workunit.client.0.smithi007.stdout:3/812: symlink dc/d22/d29/de6/lff 0 2024-09-21T04:35:36.292 INFO:tasks.workunit.client.0.smithi007.stdout:5/786: creat d2/d27/d3d/d9b/ff8 x:0 0 0 2024-09-21T04:35:36.293 INFO:tasks.workunit.client.0.smithi007.stdout:8/555: mknod d0/d3/d82/ca1 0 2024-09-21T04:35:36.294 INFO:tasks.workunit.client.0.smithi007.stdout:8/556: chown d0/d3/l3c 12291 1 2024-09-21T04:35:36.295 INFO:tasks.workunit.client.0.smithi007.stdout:6/707: symlink d3/d3b/dd3/d60/d77/dcb/d78/leb 0 2024-09-21T04:35:36.296 INFO:tasks.workunit.client.0.smithi007.stdout:2/546: mknod d0/c9b 0 2024-09-21T04:35:36.296 INFO:tasks.workunit.client.0.smithi007.stdout:2/547: stat d0/f23 0 2024-09-21T04:35:36.297 INFO:tasks.workunit.client.0.smithi007.stdout:2/548: chown d0/d11/d31/d46/d6d/d81 63692 1 2024-09-21T04:35:36.298 INFO:tasks.workunit.client.0.smithi007.stdout:2/549: write d0/d11/d31/d7b/f89 [572979,90542] 0 2024-09-21T04:35:36.300 INFO:tasks.workunit.client.0.smithi007.stdout:2/550: write d0/d11/d31/d39/f4f [4376709,82866] 0 2024-09-21T04:35:36.301 INFO:tasks.workunit.client.0.smithi007.stdout:3/813: rename dc/d22/d29/d32/d69/d6d/fb3 to dc/d22/d29/f100 0 2024-09-21T04:35:36.301 INFO:tasks.workunit.client.0.smithi007.stdout:3/814: stat dc/f1c 0 2024-09-21T04:35:36.302 INFO:tasks.workunit.client.0.smithi007.stdout:3/815: chown dc/d18/l1b 0 1 2024-09-21T04:35:36.303 INFO:tasks.workunit.client.0.smithi007.stdout:5/787: truncate d2/d27/d3d/d9e/fd9 2339870 0 2024-09-21T04:35:36.304 INFO:tasks.workunit.client.0.smithi007.stdout:4/919: creat d2/d17/d2e/f137 x:0 0 0 2024-09-21T04:35:36.305 INFO:tasks.workunit.client.0.smithi007.stdout:3/816: dread dc/d22/d5a/d71/ff9 [0,4194304] 0 2024-09-21T04:35:36.306 INFO:tasks.workunit.client.0.smithi007.stdout:6/708: rmdir d3/d3b/dd3/d60/d77 39 2024-09-21T04:35:36.307 INFO:tasks.workunit.client.0.smithi007.stdout:4/920: truncate d2/d17/d2e/d45/d5e/f9c 87938 0 2024-09-21T04:35:36.313 INFO:tasks.workunit.client.0.smithi007.stdout:2/551: write d0/d11/d6e/f90 [4676515,3537] 0 2024-09-21T04:35:36.322 INFO:tasks.workunit.client.0.smithi007.stdout:4/921: read d2/f6a [1486920,49363] 0 2024-09-21T04:35:36.325 INFO:tasks.workunit.client.0.smithi007.stdout:6/709: write d3/d3b/dd3/d60/d77/dcb/fb1 [914236,129132] 0 2024-09-21T04:35:36.325 INFO:tasks.workunit.client.0.smithi007.stdout:7/719: dwrite d3/d6/d2e/d63/d6a/d81/f96 [0,4194304] 0 2024-09-21T04:35:36.328 INFO:tasks.workunit.client.0.smithi007.stdout:2/552: creat d0/d11/d31/d46/d6c/f9c x:0 0 0 2024-09-21T04:35:36.336 INFO:tasks.workunit.client.0.smithi007.stdout:5/788: creat d2/d27/d3d/dc2/dda/ff9 x:0 0 0 2024-09-21T04:35:36.337 INFO:tasks.workunit.client.0.smithi007.stdout:8/557: symlink d0/d3/d74/la2 0 2024-09-21T04:35:36.338 INFO:tasks.workunit.client.0.smithi007.stdout:5/789: stat d2/l8b 0 2024-09-21T04:35:36.338 INFO:tasks.workunit.client.0.smithi007.stdout:4/922: mknod d2/d7/dc9/c138 0 2024-09-21T04:35:36.340 INFO:tasks.workunit.client.0.smithi007.stdout:1/979: getdents d5/d6/d21/d64/df1/da0/d99/dcd 0 2024-09-21T04:35:36.341 INFO:tasks.workunit.client.0.smithi007.stdout:4/923: stat d2/d17/d18/f2d 0 2024-09-21T04:35:36.343 INFO:tasks.workunit.client.0.smithi007.stdout:3/817: dwrite dc/f21 [0,4194304] 0 2024-09-21T04:35:36.345 INFO:tasks.workunit.client.0.smithi007.stdout:7/720: dwrite d3/d6/d13/d4b/f7d [0,4194304] 0 2024-09-21T04:35:36.346 INFO:tasks.workunit.client.0.smithi007.stdout:0/722: getdents d6/d75/dc5/d52/d6d/de1 0 2024-09-21T04:35:36.347 INFO:tasks.workunit.client.0.smithi007.stdout:7/721: chown d3/l22 6 1 2024-09-21T04:35:36.349 INFO:tasks.workunit.client.0.smithi007.stdout:0/723: chown d6/d91/lbe 40453882 1 2024-09-21T04:35:36.356 INFO:tasks.workunit.client.0.smithi007.stdout:9/894: sync 2024-09-21T04:35:36.366 INFO:tasks.workunit.client.0.smithi007.stdout:6/710: dwrite d3/f2f [0,4194304] 0 2024-09-21T04:35:36.389 INFO:tasks.workunit.client.0.smithi007.stdout:5/790: fdatasync d2/f8d 0 2024-09-21T04:35:36.394 INFO:tasks.workunit.client.0.smithi007.stdout:1/980: fdatasync d5/d6/d21/d2b/fd7 0 2024-09-21T04:35:36.396 INFO:tasks.workunit.client.0.smithi007.stdout:3/818: creat dc/d22/d29/d32/d4d/d7d/d9a/f101 x:0 0 0 2024-09-21T04:35:36.401 INFO:tasks.workunit.client.0.smithi007.stdout:6/711: creat d3/d3b/dd3/d60/d77/fec x:0 0 0 2024-09-21T04:35:36.403 INFO:tasks.workunit.client.0.smithi007.stdout:6/712: write d3/d3e/fd1 [1532034,91066] 0 2024-09-21T04:35:36.407 INFO:tasks.workunit.client.0.smithi007.stdout:3/819: read - dc/d18/d41/d96/fd9 zero size 2024-09-21T04:35:36.410 INFO:tasks.workunit.client.0.smithi007.stdout:1/981: rename d5/d6/d21/d2b/d4b/da6/df2 to d5/d6/d21/d2b/d130/dda/d145 0 2024-09-21T04:35:36.410 INFO:tasks.workunit.client.0.smithi007.stdout:3/820: rename dc/d22/d29/d32/d4d to dc/d22/d29/d32/d4d/d84/dd0/dbc/d102 22 2024-09-21T04:35:36.413 INFO:tasks.workunit.client.0.smithi007.stdout:9/895: mknod d13/d18/c121 0 2024-09-21T04:35:36.418 INFO:tasks.workunit.client.0.smithi007.stdout:6/713: fsync d3/d3b/dd3/d60/d77/dcb/d78/f96 0 2024-09-21T04:35:36.421 INFO:tasks.workunit.client.0.smithi007.stdout:3/821: dread dc/d22/fdf [0,4194304] 0 2024-09-21T04:35:36.422 INFO:tasks.workunit.client.0.smithi007.stdout:0/724: symlink d6/d75/dc5/d18/d2e/d3f/l10b 0 2024-09-21T04:35:36.427 INFO:tasks.workunit.client.0.smithi007.stdout:1/982: rename d5/d6/d21/d2b/d4b/da6/l82 to d5/d6/d21/d64/df1/l146 0 2024-09-21T04:35:36.428 INFO:tasks.workunit.client.0.smithi007.stdout:1/983: read d5/d6/f119 [873278,119002] 0 2024-09-21T04:35:36.429 INFO:tasks.workunit.client.0.smithi007.stdout:1/984: write d5/fd0 [4631750,88786] 0 2024-09-21T04:35:36.431 INFO:tasks.workunit.client.0.smithi007.stdout:2/553: getdents d0/d11 0 2024-09-21T04:35:36.432 INFO:tasks.workunit.client.0.smithi007.stdout:8/558: getdents d0/d3/d16/d93/d26 0 2024-09-21T04:35:36.439 INFO:tasks.workunit.client.0.smithi007.stdout:2/554: dwrite d0/fd [0,4194304] 0 2024-09-21T04:35:36.440 INFO:tasks.workunit.client.0.smithi007.stdout:7/722: getdents d3/d6/d14/d20/d58 0 2024-09-21T04:35:36.442 INFO:tasks.workunit.client.0.smithi007.stdout:6/714: dread d3/de/f6a [0,4194304] 0 2024-09-21T04:35:36.443 INFO:tasks.workunit.client.0.smithi007.stdout:7/723: write d3/d6/d10/f7f [842911,72822] 0 2024-09-21T04:35:36.450 INFO:tasks.workunit.client.0.smithi007.stdout:3/822: mknod dc/d22/d29/d32/d69/de4/c103 0 2024-09-21T04:35:36.452 INFO:tasks.workunit.client.0.smithi007.stdout:2/555: mkdir d0/d11/d6b/d9d 0 2024-09-21T04:35:36.454 INFO:tasks.workunit.client.0.smithi007.stdout:6/715: fdatasync d3/d3b/dd3/d60/d98/faf 0 2024-09-21T04:35:36.456 INFO:tasks.workunit.client.0.smithi007.stdout:9/896: creat d13/d18/d60/d6d/d75/d9f/f122 x:0 0 0 2024-09-21T04:35:36.458 INFO:tasks.workunit.client.0.smithi007.stdout:0/725: rename d6/d75/dc5/d57/fb1 to d6/d75/dc5/f10c 0 2024-09-21T04:35:36.460 INFO:tasks.workunit.client.0.smithi007.stdout:7/724: mkdir d3/d6/d14/d20/d102 0 2024-09-21T04:35:36.462 INFO:tasks.workunit.client.0.smithi007.stdout:7/725: chown d3/d6/d14/l3d 3 1 2024-09-21T04:35:36.491 INFO:tasks.workunit.client.0.smithi007.stdout:7/726: dread - d3/d6/d85/de9/ff2 zero size 2024-09-21T04:35:36.494 INFO:tasks.workunit.client.0.smithi007.stdout:8/559: write d0/d3/d16/d93/d26/d35/f37 [147399,116011] 0 2024-09-21T04:35:36.636 INFO:tasks.workunit.client.0.smithi007.stdout:6/716: link d3/de/d12/d19/f20 d3/de/fed 0 2024-09-21T04:35:36.637 INFO:tasks.workunit.client.0.smithi007.stdout:9/897: rename d13/c53 to d13/d18/d60/d6d/d75/d113/c123 0 2024-09-21T04:35:36.640 INFO:tasks.workunit.client.0.smithi007.stdout:8/560: read d0/d3/d8/f67 [1903279,110956] 0 2024-09-21T04:35:36.640 INFO:tasks.workunit.client.0.smithi007.stdout:7/727: read - d3/d6/d40/faa zero size 2024-09-21T04:35:36.643 INFO:tasks.workunit.client.0.smithi007.stdout:4/924: dwrite d2/fb6 [0,4194304] 0 2024-09-21T04:35:36.646 INFO:tasks.workunit.client.0.smithi007.stdout:8/561: fsync d0/d3/d16/f8d 0 2024-09-21T04:35:36.647 INFO:tasks.workunit.client.0.smithi007.stdout:6/717: creat d3/d94/fee x:0 0 0 2024-09-21T04:35:36.648 INFO:tasks.workunit.client.0.smithi007.stdout:1/985: readlink d5/d6/d21/d64/df1/da0/lf0 0 2024-09-21T04:35:36.648 INFO:tasks.workunit.client.0.smithi007.stdout:5/791: dwrite d2/d1e/f5b [0,4194304] 0 2024-09-21T04:35:36.652 INFO:tasks.workunit.client.0.smithi007.stdout:9/898: fsync d13/d18/f5c 0 2024-09-21T04:35:36.655 INFO:tasks.workunit.client.0.smithi007.stdout:2/556: dwrite d0/d11/d31/d46/d77/f91 [0,4194304] 0 2024-09-21T04:35:36.655 INFO:tasks.workunit.client.0.smithi007.stdout:8/562: fdatasync d0/d3/d16/d93/f17 0 2024-09-21T04:35:36.656 INFO:tasks.workunit.client.0.smithi007.stdout:0/726: dwrite d6/d75/dc5/d18/d39/f6a [0,4194304] 0 2024-09-21T04:35:36.660 INFO:tasks.workunit.client.0.smithi007.stdout:9/899: write d13/d18/d60/f10a [197865,124737] 0 2024-09-21T04:35:36.662 INFO:tasks.workunit.client.0.smithi007.stdout:2/557: chown d0/d2/cb 43097 1 2024-09-21T04:35:36.663 INFO:tasks.workunit.client.0.smithi007.stdout:8/563: stat d0/d3/d16/d93/d26/d35/f73 0 2024-09-21T04:35:36.665 INFO:tasks.workunit.client.0.smithi007.stdout:6/718: symlink d3/de/d12/d19/da3/lef 0 2024-09-21T04:35:36.666 INFO:tasks.workunit.client.0.smithi007.stdout:5/792: mkdir d2/d27/d42/da3/d23/d49/dd4/dfa 0 2024-09-21T04:35:36.668 INFO:tasks.workunit.client.0.smithi007.stdout:1/986: creat d5/d6/d21/d2b/d4b/d8a/dee/f147 x:0 0 0 2024-09-21T04:35:36.670 INFO:tasks.workunit.client.0.smithi007.stdout:2/558: symlink d0/d11/d31/d39/l9e 0 2024-09-21T04:35:36.672 INFO:tasks.workunit.client.0.smithi007.stdout:0/727: truncate d6/d75/dc5/d18/d27/f9b 1198661 0 2024-09-21T04:35:36.674 INFO:tasks.workunit.client.0.smithi007.stdout:7/728: getdents d3/d6/d85 0 2024-09-21T04:35:36.674 INFO:tasks.workunit.client.0.smithi007.stdout:2/559: dread d0/d11/f4e [0,4194304] 0 2024-09-21T04:35:36.675 INFO:tasks.workunit.client.0.smithi007.stdout:8/564: symlink d0/d3/d16/d93/d26/d4d/la3 0 2024-09-21T04:35:36.675 INFO:tasks.workunit.client.0.smithi007.stdout:2/560: write d0/f7d [1042306,117486] 0 2024-09-21T04:35:36.676 INFO:tasks.workunit.client.0.smithi007.stdout:2/561: dread - d0/d11/d31/d7b/f93 zero size 2024-09-21T04:35:36.677 INFO:tasks.workunit.client.0.smithi007.stdout:5/793: symlink d2/d27/d42/da3/d23/d49/d6f/lfb 0 2024-09-21T04:35:36.679 INFO:tasks.workunit.client.0.smithi007.stdout:1/987: readlink d5/d6/d21/d64/df1/lbf 0 2024-09-21T04:35:36.680 INFO:tasks.workunit.client.0.smithi007.stdout:5/794: dread - d2/d27/d3d/dc2/fed zero size 2024-09-21T04:35:36.682 INFO:tasks.workunit.client.0.smithi007.stdout:8/565: truncate d0/d3/d16/d93/d26/d4d/d5d/f86 925576 0 2024-09-21T04:35:36.683 INFO:tasks.workunit.client.0.smithi007.stdout:4/925: link d2/d17/d2e/d45/d8e/la0 d2/d17/d18/df0/l139 0 2024-09-21T04:35:36.684 INFO:tasks.workunit.client.0.smithi007.stdout:5/795: chown d2/d27/d42/da3/d23/d2a/f2f 6 1 2024-09-21T04:35:36.686 INFO:tasks.workunit.client.0.smithi007.stdout:2/562: fsync d0/d2/d57/f5a 0 2024-09-21T04:35:36.686 INFO:tasks.workunit.client.0.smithi007.stdout:6/719: dwrite d3/de/d58/faa [0,4194304] 0 2024-09-21T04:35:36.690 INFO:tasks.workunit.client.0.smithi007.stdout:1/988: dread d5/d6/d21/d2b/d4b/da6/f80 [0,4194304] 0 2024-09-21T04:35:36.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:36 smithi053 ceph-mon[35554]: pgmap v39: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 76 MiB/s rd, 184 MiB/s wr, 466 op/s 2024-09-21T04:35:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:36 smithi007 ceph-mon[27954]: pgmap v39: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 76 MiB/s rd, 184 MiB/s wr, 466 op/s 2024-09-21T04:35:36.694 INFO:tasks.workunit.client.0.smithi007.stdout:8/566: dwrite d0/d3/d16/d93/d26/d4d/f91 [0,4194304] 0 2024-09-21T04:35:36.697 INFO:tasks.workunit.client.0.smithi007.stdout:2/563: mkdir d0/d11/d31/d9f 0 2024-09-21T04:35:36.701 INFO:tasks.workunit.client.0.smithi007.stdout:9/900: getdents d13/d18/d60/d6d 0 2024-09-21T04:35:36.702 INFO:tasks.workunit.client.0.smithi007.stdout:9/901: chown d13/d18/d60/f63 53308 1 2024-09-21T04:35:36.706 INFO:tasks.workunit.client.0.smithi007.stdout:0/728: creat d6/d75/dc5/d47/f10d x:0 0 0 2024-09-21T04:35:36.708 INFO:tasks.workunit.client.0.smithi007.stdout:7/729: dwrite d3/d6/d13/f29 [0,4194304] 0 2024-09-21T04:35:36.712 INFO:tasks.workunit.client.0.smithi007.stdout:2/564: rename d0/d2 to d0/d11/d31/d46/d6d/d81/da0 0 2024-09-21T04:35:36.715 INFO:tasks.workunit.client.0.smithi007.stdout:9/902: symlink d13/d18/daf/dda/l124 0 2024-09-21T04:35:36.719 INFO:tasks.workunit.client.0.smithi007.stdout:8/567: mknod d0/d3/d74/ca4 0 2024-09-21T04:35:36.721 INFO:tasks.workunit.client.0.smithi007.stdout:2/565: rmdir d0 39 2024-09-21T04:35:36.722 INFO:tasks.workunit.client.0.smithi007.stdout:6/720: truncate d3/de/f49 1628800 0 2024-09-21T04:35:36.727 INFO:tasks.workunit.client.0.smithi007.stdout:1/989: truncate d5/d2f/f95 1885526 0 2024-09-21T04:35:36.728 INFO:tasks.workunit.client.0.smithi007.stdout:7/730: write d3/d6/d10/f34 [960341,105849] 0 2024-09-21T04:35:36.728 INFO:tasks.workunit.client.0.smithi007.stdout:1/990: write d5/f39 [1696337,62295] 0 2024-09-21T04:35:36.730 INFO:tasks.workunit.client.0.smithi007.stdout:6/721: rename d3/de/d12/f50 to d3/d3b/dd3/d60/dd2/ff0 0 2024-09-21T04:35:36.731 INFO:tasks.workunit.client.0.smithi007.stdout:6/722: chown d3/d3b/dd3/d60/f93 286 1 2024-09-21T04:35:36.732 INFO:tasks.workunit.client.0.smithi007.stdout:8/568: write d0/d3/d8/f58 [3140786,18484] 0 2024-09-21T04:35:36.733 INFO:tasks.workunit.client.0.smithi007.stdout:8/569: readlink d0/d3/l8b 0 2024-09-21T04:35:36.734 INFO:tasks.workunit.client.0.smithi007.stdout:0/729: getdents d6/d75/dc5/d47/d8a/de9/d101 0 2024-09-21T04:35:36.734 INFO:tasks.workunit.client.0.smithi007.stdout:8/570: chown d0/d3/d7c/f59 4851 1 2024-09-21T04:35:36.736 INFO:tasks.workunit.client.0.smithi007.stdout:7/731: mkdir d3/d6/d85/dad/d103 0 2024-09-21T04:35:36.738 INFO:tasks.workunit.client.0.smithi007.stdout:7/732: chown d3/d6/d14/d20/d80/dca/c100 1 1 2024-09-21T04:35:36.740 INFO:tasks.workunit.client.0.smithi007.stdout:7/733: write d3/d6/d85/dd2/fe3 [766113,27096] 0 2024-09-21T04:35:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:2/566: link d0/d11/d31/d46/d6d/d81/da0/d57/f5a d0/d11/d6e/fa1 0 2024-09-21T04:35:36.746 INFO:tasks.workunit.client.0.smithi007.stdout:7/734: rmdir d3/d6 39 2024-09-21T04:35:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:6/723: dwrite d3/de/d58/faa [0,4194304] 0 2024-09-21T04:35:36.750 INFO:tasks.workunit.client.0.smithi007.stdout:6/724: read d3/d3b/dd3/d60/d77/f63 [2799077,112936] 0 2024-09-21T04:35:36.751 INFO:tasks.workunit.client.0.smithi007.stdout:8/571: dread d0/d3/f2b [0,4194304] 0 2024-09-21T04:35:36.752 INFO:tasks.workunit.client.0.smithi007.stdout:7/735: write d3/d6/d13/f2c [4511369,130226] 0 2024-09-21T04:35:36.753 INFO:tasks.workunit.client.0.smithi007.stdout:8/572: fdatasync d0/d3/d16/d93/d26/d4d/f91 0 2024-09-21T04:35:36.755 INFO:tasks.workunit.client.0.smithi007.stdout:6/725: mkdir d3/d3e/d99/dbb/df1 0 2024-09-21T04:35:36.757 INFO:tasks.workunit.client.0.smithi007.stdout:8/573: truncate d0/d3/d8/f67 5053098 0 2024-09-21T04:35:36.758 INFO:tasks.workunit.client.0.smithi007.stdout:7/736: write d3/d6/d10/d4f/fdb [803742,33923] 0 2024-09-21T04:35:36.760 INFO:tasks.workunit.client.0.smithi007.stdout:6/726: stat d3/de/d12/d19/da3 0 2024-09-21T04:35:36.762 INFO:tasks.workunit.client.0.smithi007.stdout:0/730: dwrite d6/d75/dc5/d47/d8a/de9/ff3 [0,4194304] 0 2024-09-21T04:35:36.764 INFO:tasks.workunit.client.0.smithi007.stdout:8/574: chown d0/d3/d16/d93/f69 26794412 1 2024-09-21T04:35:36.767 INFO:tasks.workunit.client.0.smithi007.stdout:8/575: chown d0/d3/d16/f1b 1 1 2024-09-21T04:35:36.768 INFO:tasks.workunit.client.0.smithi007.stdout:7/737: mknod d3/d6/d85/dad/c104 0 2024-09-21T04:35:36.770 INFO:tasks.workunit.client.0.smithi007.stdout:6/727: read - d3/de/d12/d19/da3/fac zero size 2024-09-21T04:35:36.772 INFO:tasks.workunit.client.0.smithi007.stdout:1/991: dwrite d5/d6/d21/d2b/f5c [0,4194304] 0 2024-09-21T04:35:36.773 INFO:tasks.workunit.client.0.smithi007.stdout:7/738: mknod d3/d6/d10/d4f/c105 0 2024-09-21T04:35:36.775 INFO:tasks.workunit.client.0.smithi007.stdout:2/567: dread d0/d32/f34 [0,4194304] 0 2024-09-21T04:35:36.775 INFO:tasks.workunit.client.0.smithi007.stdout:0/731: rename d6/d75/dc5/d18/d27/c48 to d6/d75/dc9/c10e 0 2024-09-21T04:35:36.777 INFO:tasks.workunit.client.0.smithi007.stdout:7/739: write d3/d6/d14/d20/d80/dca/ffa [294304,78864] 0 2024-09-21T04:35:36.779 INFO:tasks.workunit.client.0.smithi007.stdout:8/576: symlink d0/d3/d74/d52/la5 0 2024-09-21T04:35:36.779 INFO:tasks.workunit.client.0.smithi007.stdout:2/568: write d0/d11/d31/d46/d6d/d81/da0/f25 [2589280,75843] 0 2024-09-21T04:35:36.781 INFO:tasks.workunit.client.0.smithi007.stdout:2/569: stat d0/d11/d31/d52 0 2024-09-21T04:35:36.787 INFO:tasks.workunit.client.0.smithi007.stdout:7/740: truncate d3/d6/d2e/d63/f9e 174675 0 2024-09-21T04:35:36.788 INFO:tasks.workunit.client.0.smithi007.stdout:2/570: mkdir d0/d11/d31/d7b/da2 0 2024-09-21T04:35:36.789 INFO:tasks.workunit.client.0.smithi007.stdout:2/571: fsync d0/d11/d31/d7b/f94 0 2024-09-21T04:35:36.789 INFO:tasks.workunit.client.0.smithi007.stdout:6/728: creat d3/d3b/dd3/d60/ff2 x:0 0 0 2024-09-21T04:35:36.790 INFO:tasks.workunit.client.0.smithi007.stdout:2/572: chown d0/d11/d31/d46/f6a 155137 1 2024-09-21T04:35:36.790 INFO:tasks.workunit.client.0.smithi007.stdout:6/729: chown d3/d3e/d99/dbb/f9f 18532551 1 2024-09-21T04:35:36.796 INFO:tasks.workunit.client.0.smithi007.stdout:7/741: creat d3/d6/d14/d20/d58/f106 x:0 0 0 2024-09-21T04:35:36.800 INFO:tasks.workunit.client.0.smithi007.stdout:6/730: unlink d3/c8b 0 2024-09-21T04:35:36.802 INFO:tasks.workunit.client.0.smithi007.stdout:7/742: dwrite d3/d6/d40/fd7 [0,4194304] 0 2024-09-21T04:35:36.803 INFO:tasks.workunit.client.0.smithi007.stdout:0/732: rename d6/d75/dc5/d57/d9d/daf/f102 to d6/d91/da3/f10f 0 2024-09-21T04:35:36.804 INFO:tasks.workunit.client.0.smithi007.stdout:0/733: chown d6/d75/dc5/d18/d39/dd5 0 1 2024-09-21T04:35:36.809 INFO:tasks.workunit.client.0.smithi007.stdout:1/992: getdents d5/d6 0 2024-09-21T04:35:36.812 INFO:tasks.workunit.client.0.smithi007.stdout:8/577: write d0/f4f [2551343,112905] 0 2024-09-21T04:35:36.813 INFO:tasks.workunit.client.0.smithi007.stdout:6/731: fsync d3/f54 0 2024-09-21T04:35:36.817 INFO:tasks.workunit.client.0.smithi007.stdout:0/734: mkdir d6/d75/dc5/d47/d99/d110 0 2024-09-21T04:35:36.820 INFO:tasks.workunit.client.0.smithi007.stdout:6/732: dwrite d3/d94/f8a [4194304,4194304] 0 2024-09-21T04:35:36.820 INFO:tasks.workunit.client.0.smithi007.stdout:1/993: mkdir d5/d6/da2/d148 0 2024-09-21T04:35:36.823 INFO:tasks.workunit.client.0.smithi007.stdout:2/573: creat d0/fa3 x:0 0 0 2024-09-21T04:35:36.825 INFO:tasks.workunit.client.0.smithi007.stdout:8/578: fsync d0/d3/d8/f9 0 2024-09-21T04:35:36.830 INFO:tasks.workunit.client.0.smithi007.stdout:2/574: mkdir d0/d11/d31/da4 0 2024-09-21T04:35:36.834 INFO:tasks.workunit.client.0.smithi007.stdout:7/743: creat d3/d6/d2e/f107 x:0 0 0 2024-09-21T04:35:36.835 INFO:tasks.workunit.client.0.smithi007.stdout:0/735: symlink d6/d75/dc5/d18/l111 0 2024-09-21T04:35:36.838 INFO:tasks.workunit.client.0.smithi007.stdout:6/733: write d3/d94/fb3 [30559,14359] 0 2024-09-21T04:35:36.841 INFO:tasks.workunit.client.0.smithi007.stdout:6/734: write d3/de/dba/fda [660444,26349] 0 2024-09-21T04:35:36.841 INFO:tasks.workunit.client.0.smithi007.stdout:1/994: dread d5/d6/d21/d64/df1/f62 [0,4194304] 0 2024-09-21T04:35:36.843 INFO:tasks.workunit.client.0.smithi007.stdout:8/579: creat d0/fa6 x:0 0 0 2024-09-21T04:35:36.844 INFO:tasks.workunit.client.0.smithi007.stdout:2/575: mkdir d0/d11/d31/d7b/da2/da5 0 2024-09-21T04:35:36.847 INFO:tasks.workunit.client.0.smithi007.stdout:6/735: unlink d3/d3e/c87 0 2024-09-21T04:35:36.849 INFO:tasks.workunit.client.0.smithi007.stdout:2/576: dread - d0/d11/d31/d46/f6a zero size 2024-09-21T04:35:36.853 INFO:tasks.workunit.client.0.smithi007.stdout:0/736: dread d6/d75/dc5/d52/f5c [0,4194304] 0 2024-09-21T04:35:36.853 INFO:tasks.workunit.client.0.smithi007.stdout:1/995: mkdir d5/d6/d21/d64/df1/d149 0 2024-09-21T04:35:36.854 INFO:tasks.workunit.client.0.smithi007.stdout:1/996: dread d5/d2f/fc9 [0,4194304] 0 2024-09-21T04:35:36.858 INFO:tasks.workunit.client.0.smithi007.stdout:7/744: write d3/d6/f2d [689837,48627] 0 2024-09-21T04:35:36.859 INFO:tasks.workunit.client.0.smithi007.stdout:2/577: fsync d0/d11/d31/d39/f3a 0 2024-09-21T04:35:36.860 INFO:tasks.workunit.client.0.smithi007.stdout:7/745: write d3/d6/d85/dd2/fd5 [125326,52956] 0 2024-09-21T04:35:36.860 INFO:tasks.workunit.client.0.smithi007.stdout:2/578: chown d0/f37 234427 1 2024-09-21T04:35:36.863 INFO:tasks.workunit.client.0.smithi007.stdout:6/736: creat d3/d3b/dd3/d60/dd7/ff3 x:0 0 0 2024-09-21T04:35:36.867 INFO:tasks.workunit.client.0.smithi007.stdout:0/737: getdents d6/df5 0 2024-09-21T04:35:36.867 INFO:tasks.workunit.client.0.smithi007.stdout:2/579: rmdir d0/d11/d6b 39 2024-09-21T04:35:36.869 INFO:tasks.workunit.client.0.smithi007.stdout:7/746: creat d3/d6/d14/d20/d80/db2/f108 x:0 0 0 2024-09-21T04:35:36.871 INFO:tasks.workunit.client.0.smithi007.stdout:1/997: mknod d5/d6/d21/d64/d108/c14a 0 2024-09-21T04:35:36.874 INFO:tasks.workunit.client.0.smithi007.stdout:6/737: mkdir d3/d3b/dd3/d60/d77/dcb/df4 0 2024-09-21T04:35:36.875 INFO:tasks.workunit.client.0.smithi007.stdout:7/747: dwrite d3/d6/d10/fe0 [0,4194304] 0 2024-09-21T04:35:36.876 INFO:tasks.workunit.client.0.smithi007.stdout:7/748: stat d3/d6/d10/d4f/f9b 0 2024-09-21T04:35:36.878 INFO:tasks.workunit.client.0.smithi007.stdout:1/998: dread d5/d6/f119 [0,4194304] 0 2024-09-21T04:35:36.882 INFO:tasks.workunit.client.0.smithi007.stdout:8/580: link d0/l50 d0/d3/d16/d93/d5e/d7b/la7 0 2024-09-21T04:35:36.884 INFO:tasks.workunit.client.0.smithi007.stdout:2/580: readlink d0/d11/d31/d39/l3b 0 2024-09-21T04:35:36.887 INFO:tasks.workunit.client.0.smithi007.stdout:7/749: mknod d3/d6/d10/d62/d101/c109 0 2024-09-21T04:35:36.890 INFO:tasks.workunit.client.0.smithi007.stdout:8/581: dwrite d0/d3/d16/f31 [4194304,4194304] 0 2024-09-21T04:35:36.892 INFO:tasks.workunit.client.0.smithi007.stdout:8/582: chown d0/d3/d16/f8d 12 1 2024-09-21T04:35:36.893 INFO:tasks.workunit.client.0.smithi007.stdout:8/583: readlink d0/d3/d16/d93/l48 0 2024-09-21T04:35:36.894 INFO:tasks.workunit.client.0.smithi007.stdout:8/584: fdatasync d0/d3/d8/f9 0 2024-09-21T04:35:36.896 INFO:tasks.workunit.client.0.smithi007.stdout:7/750: read d3/d6/d85/f50 [842147,244] 0 2024-09-21T04:35:36.897 INFO:tasks.workunit.client.0.smithi007.stdout:8/585: readlink d0/d3/d16/d93/l48 0 2024-09-21T04:35:36.902 INFO:tasks.workunit.client.0.smithi007.stdout:0/738: dwrite d6/d75/dc5/d47/d99/fbc [0,4194304] 0 2024-09-21T04:35:36.907 INFO:tasks.workunit.client.0.smithi007.stdout:7/751: mkdir d3/d6/d2e/d63/df8/d10a 0 2024-09-21T04:35:36.915 INFO:tasks.workunit.client.0.smithi007.stdout:2/581: dwrite d0/f1c [0,4194304] 0 2024-09-21T04:35:36.915 INFO:tasks.workunit.client.0.smithi007.stdout:7/752: unlink d3/d6/d14/d20/d80/cae 0 2024-09-21T04:35:36.922 INFO:tasks.workunit.client.0.smithi007.stdout:0/739: write d6/d75/dc5/d18/d2e/d60/f5f [3201870,113535] 0 2024-09-21T04:35:36.926 INFO:tasks.workunit.client.0.smithi007.stdout:2/582: link d0/d11/d31/d39/l9e d0/d11/d31/d7b/la6 0 2024-09-21T04:35:36.928 INFO:tasks.workunit.client.0.smithi007.stdout:0/740: symlink d6/d75/dc5/d47/d8a/d9c/l112 0 2024-09-21T04:35:36.930 INFO:tasks.workunit.client.0.smithi007.stdout:7/753: getdents d3/d6 0 2024-09-21T04:35:36.933 INFO:tasks.workunit.client.0.smithi007.stdout:7/754: dread d3/d6/d10/d4f/f65 [0,4194304] 0 2024-09-21T04:35:36.936 INFO:tasks.workunit.client.0.smithi007.stdout:2/583: rename d0/d11/d31/l75 to d0/d11/d6b/d9d/la7 0 2024-09-21T04:35:36.937 INFO:tasks.workunit.client.0.smithi007.stdout:8/586: dread d0/d3/d8/f67 [0,4194304] 0 2024-09-21T04:35:36.938 INFO:tasks.workunit.client.0.smithi007.stdout:7/755: mkdir d3/d6/d14/d20/d80/db2/d10b 0 2024-09-21T04:35:36.939 INFO:tasks.workunit.client.0.smithi007.stdout:8/587: write d0/d3/d16/d93/d26/d4d/f9b [1004830,87888] 0 2024-09-21T04:35:36.945 INFO:tasks.workunit.client.0.smithi007.stdout:7/756: creat d3/d6/d10/d62/d101/f10c x:0 0 0 2024-09-21T04:35:36.950 INFO:tasks.workunit.client.0.smithi007.stdout:8/588: mknod d0/ca8 0 2024-09-21T04:35:36.951 INFO:tasks.workunit.client.0.smithi007.stdout:0/741: write d6/d75/dc5/d18/d2e/f87 [5046976,35181] 0 2024-09-21T04:35:36.952 INFO:tasks.workunit.client.0.smithi007.stdout:8/589: stat d0/d3/d16/d93/d26/d4d/d5d 0 2024-09-21T04:35:36.953 INFO:tasks.workunit.client.0.smithi007.stdout:7/757: symlink d3/d6/d40/da1/l10d 0 2024-09-21T04:35:36.954 INFO:tasks.workunit.client.0.smithi007.stdout:8/590: chown d0/d3/d16/f99 251897 1 2024-09-21T04:35:36.959 INFO:tasks.workunit.client.0.smithi007.stdout:2/584: dwrite d0/d11/d31/d39/f3e [0,4194304] 0 2024-09-21T04:35:36.960 INFO:tasks.workunit.client.0.smithi007.stdout:8/591: rmdir d0/d3/d16/d93/d26/d35 39 2024-09-21T04:35:36.965 INFO:tasks.workunit.client.0.smithi007.stdout:8/592: symlink d0/d3/d16/d93/d26/d4d/la9 0 2024-09-21T04:35:36.969 INFO:tasks.workunit.client.0.smithi007.stdout:2/585: creat d0/fa8 x:0 0 0 2024-09-21T04:35:36.970 INFO:tasks.workunit.client.0.smithi007.stdout:7/758: link d3/d6/d14/d20/d80/db2/la6 d3/d6/d14/d20/d102/l10e 0 2024-09-21T04:35:36.975 INFO:tasks.workunit.client.0.smithi007.stdout:0/742: dread d6/d75/dc5/d52/d6d/fa6 [0,4194304] 0 2024-09-21T04:35:36.978 INFO:tasks.workunit.client.0.smithi007.stdout:0/743: creat d6/d75/dc9/f113 x:0 0 0 2024-09-21T04:35:36.980 INFO:tasks.workunit.client.0.smithi007.stdout:7/759: getdents d3/d6/d85/de9 0 2024-09-21T04:35:36.983 INFO:tasks.workunit.client.0.smithi007.stdout:0/744: rename d6/d75/dc5/d47/d8a/fd6 to d6/d75/dc5/d47/df0/f114 0 2024-09-21T04:35:37.053 INFO:tasks.workunit.client.0.smithi007.stdout:7/760: creat d3/d6/d40/da1/f10f x:0 0 0 2024-09-21T04:35:37.056 INFO:tasks.workunit.client.0.smithi007.stdout:7/761: unlink d3/d6/d85/dd2/fe3 0 2024-09-21T04:35:37.059 INFO:tasks.workunit.client.0.smithi007.stdout:7/762: fdatasync d3/d6/d85/dd2/fd5 0 2024-09-21T04:35:37.060 INFO:tasks.workunit.client.0.smithi007.stdout:8/593: dwrite d0/f20 [4194304,4194304] 0 2024-09-21T04:35:37.061 INFO:tasks.workunit.client.0.smithi007.stdout:2/586: dwrite d0/d11/d31/d46/d48/f62 [0,4194304] 0 2024-09-21T04:35:37.061 INFO:tasks.workunit.client.0.smithi007.stdout:7/763: chown d3/d6/d85/f49 490185710 1 2024-09-21T04:35:37.064 INFO:tasks.workunit.client.0.smithi007.stdout:8/594: chown d0/d3/d16/d93/d26/d35 2 1 2024-09-21T04:35:37.071 INFO:tasks.workunit.client.0.smithi007.stdout:2/587: mkdir d0/d11/d31/d9f/da9 0 2024-09-21T04:35:37.072 INFO:tasks.workunit.client.0.smithi007.stdout:2/588: readlink d0/l76 0 2024-09-21T04:35:37.074 INFO:tasks.workunit.client.0.smithi007.stdout:7/764: getdents d3/d6/d40/da1 0 2024-09-21T04:35:37.076 INFO:tasks.workunit.client.0.smithi007.stdout:2/589: mkdir d0/d11/d31/d46/d6d/daa 0 2024-09-21T04:35:37.080 INFO:tasks.workunit.client.0.smithi007.stdout:2/590: mknod d0/d11/d31/cab 0 2024-09-21T04:35:37.081 INFO:tasks.workunit.client.0.smithi007.stdout:7/765: dwrite d3/d6/d10/d51/fd1 [4194304,4194304] 0 2024-09-21T04:35:37.081 INFO:tasks.workunit.client.0.smithi007.stdout:8/595: dread d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:37.083 INFO:tasks.workunit.client.0.smithi007.stdout:8/596: fdatasync d0/d3/d7c/f59 0 2024-09-21T04:35:37.084 INFO:tasks.workunit.client.0.smithi007.stdout:2/591: getdents d0/d11/d31/d46/d6d/daa 0 2024-09-21T04:35:37.085 INFO:tasks.workunit.client.0.smithi007.stdout:8/597: truncate d0/d3/d8/f3f 4979431 0 2024-09-21T04:35:37.090 INFO:tasks.workunit.client.0.smithi007.stdout:2/592: mknod d0/d11/cac 0 2024-09-21T04:35:37.091 INFO:tasks.workunit.client.0.smithi007.stdout:3/823: sync 2024-09-21T04:35:37.091 INFO:tasks.workunit.client.0.smithi007.stdout:7/766: link d3/d6/d85/dad/c104 d3/d6/d2e/d63/c110 0 2024-09-21T04:35:37.093 INFO:tasks.workunit.client.0.smithi007.stdout:3/824: truncate dc/fd4 411752 0 2024-09-21T04:35:37.094 INFO:tasks.workunit.client.0.smithi007.stdout:2/593: write d0/f2e [916456,62039] 0 2024-09-21T04:35:37.095 INFO:tasks.workunit.client.0.smithi007.stdout:3/825: stat dc/d22/d5a/d71/d74 0 2024-09-21T04:35:37.096 INFO:tasks.workunit.client.0.smithi007.stdout:8/598: dwrite d0/d3/d16/d93/d40/d6c/f97 [0,4194304] 0 2024-09-21T04:35:37.097 INFO:tasks.workunit.client.0.smithi007.stdout:8/599: chown d0/d3/d74/d52 56 1 2024-09-21T04:35:37.097 INFO:tasks.workunit.client.0.smithi007.stdout:7/767: fsync d3/d6/d10/d4f/f65 0 2024-09-21T04:35:37.107 INFO:tasks.workunit.client.0.smithi007.stdout:2/594: dwrite d0/d11/d6e/f7e [0,4194304] 0 2024-09-21T04:35:37.111 INFO:tasks.workunit.client.0.smithi007.stdout:7/768: creat d3/d6/d2e/d63/d6a/d81/f111 x:0 0 0 2024-09-21T04:35:37.121 INFO:tasks.workunit.client.0.smithi007.stdout:3/826: dread dc/d18/d26/f68 [0,4194304] 0 2024-09-21T04:35:37.122 INFO:tasks.workunit.client.0.smithi007.stdout:3/827: dread - dc/d18/d41/ff6 zero size 2024-09-21T04:35:37.128 INFO:tasks.workunit.client.0.smithi007.stdout:8/600: rename d0/d3/d16/d93/d40/d6c to d0/d3/daa 0 2024-09-21T04:35:37.129 INFO:tasks.workunit.client.0.smithi007.stdout:8/601: write d0/d3/d16/f8d [681286,95325] 0 2024-09-21T04:35:37.134 INFO:tasks.workunit.client.0.smithi007.stdout:8/602: creat d0/d3/d16/d93/d26/d4d/d5d/fab x:0 0 0 2024-09-21T04:35:37.134 INFO:tasks.workunit.client.0.smithi007.stdout:2/595: write d0/d11/f28 [3860638,8356] 0 2024-09-21T04:35:37.136 INFO:tasks.workunit.client.0.smithi007.stdout:2/596: chown d0/f23 0 1 2024-09-21T04:35:37.136 INFO:tasks.workunit.client.0.smithi007.stdout:8/603: write d0/d3/daa/f76 [299580,2804] 0 2024-09-21T04:35:37.138 INFO:tasks.workunit.client.0.smithi007.stdout:7/769: dwrite d3/d6/d40/faa [0,4194304] 0 2024-09-21T04:35:37.144 INFO:tasks.workunit.client.0.smithi007.stdout:7/770: mkdir d3/d6/d40/d112 0 2024-09-21T04:35:37.147 INFO:tasks.workunit.client.0.smithi007.stdout:3/828: dwrite dc/d22/fdf [0,4194304] 0 2024-09-21T04:35:37.147 INFO:tasks.workunit.client.0.smithi007.stdout:8/604: creat d0/d3/d74/fac x:0 0 0 2024-09-21T04:35:37.148 INFO:tasks.workunit.client.0.smithi007.stdout:7/771: rmdir d3/d6/d10/d62 39 2024-09-21T04:35:37.150 INFO:tasks.workunit.client.0.smithi007.stdout:2/597: dwrite d0/d11/d31/d46/f47 [0,4194304] 0 2024-09-21T04:35:37.154 INFO:tasks.workunit.client.0.smithi007.stdout:3/829: fsync dc/d18/d26/f68 0 2024-09-21T04:35:37.154 INFO:tasks.workunit.client.0.smithi007.stdout:8/605: write d0/d3/d16/f31 [3730392,74889] 0 2024-09-21T04:35:37.154 INFO:tasks.workunit.client.0.smithi007.stdout:2/598: truncate d0/d11/d6e/f8a 157245 0 2024-09-21T04:35:37.154 INFO:tasks.workunit.client.0.smithi007.stdout:7/772: rmdir d3/d6/d14/d20/d80/dca 39 2024-09-21T04:35:37.155 INFO:tasks.workunit.client.0.smithi007.stdout:8/606: chown d0/d3/d16/d93/d26/d4d/la9 119 1 2024-09-21T04:35:37.156 INFO:tasks.workunit.client.0.smithi007.stdout:2/599: write d0/fa3 [830772,74461] 0 2024-09-21T04:35:37.157 INFO:tasks.workunit.client.0.smithi007.stdout:3/830: creat dc/d22/dc8/f104 x:0 0 0 2024-09-21T04:35:37.157 INFO:tasks.workunit.client.0.smithi007.stdout:2/600: readlink d0/l76 0 2024-09-21T04:35:37.158 INFO:tasks.workunit.client.0.smithi007.stdout:8/607: readlink d0/d3/d8/l2f 0 2024-09-21T04:35:37.158 INFO:tasks.workunit.client.0.smithi007.stdout:3/831: write dc/fd4 [423669,126602] 0 2024-09-21T04:35:37.162 INFO:tasks.workunit.client.0.smithi007.stdout:2/601: mknod d0/d11/d31/d52/cad 0 2024-09-21T04:35:37.166 INFO:tasks.workunit.client.0.smithi007.stdout:8/608: dread d0/d3/d16/d93/d26/d4d/f9b [0,4194304] 0 2024-09-21T04:35:37.171 INFO:tasks.workunit.client.0.smithi007.stdout:2/602: mknod d0/d11/d31/d9f/da9/cae 0 2024-09-21T04:35:37.172 INFO:tasks.workunit.client.0.smithi007.stdout:8/609: mknod d0/d3/d16/d93/d26/d4d/d5d/cad 0 2024-09-21T04:35:37.177 INFO:tasks.workunit.client.0.smithi007.stdout:3/832: dread dc/d22/d29/d32/d4d/d84/dd0/f83 [0,4194304] 0 2024-09-21T04:35:37.177 INFO:tasks.workunit.client.0.smithi007.stdout:8/610: getdents d0/d3/d74/d52 0 2024-09-21T04:35:37.178 INFO:tasks.workunit.client.0.smithi007.stdout:7/773: dwrite d3/d6/d2e/d63/f9e [0,4194304] 0 2024-09-21T04:35:37.179 INFO:tasks.workunit.client.0.smithi007.stdout:8/611: stat d0/d3/d16/d93/d26/d4d 0 2024-09-21T04:35:37.181 INFO:tasks.workunit.client.0.smithi007.stdout:7/774: chown d3/d6/d10/d62 166046235 1 2024-09-21T04:35:37.182 INFO:tasks.workunit.client.0.smithi007.stdout:2/603: write d0/d11/d31/d46/f79 [20545,38209] 0 2024-09-21T04:35:37.182 INFO:tasks.workunit.client.0.smithi007.stdout:3/833: creat dc/d22/d29/d32/d4d/f105 x:0 0 0 2024-09-21T04:35:37.184 INFO:tasks.workunit.client.0.smithi007.stdout:7/775: chown d3/d6/d14/d20/l5a 52 1 2024-09-21T04:35:37.184 INFO:tasks.workunit.client.0.smithi007.stdout:8/612: stat d0/d3/d8/d46/c77 0 2024-09-21T04:35:37.186 INFO:tasks.workunit.client.0.smithi007.stdout:2/604: creat d0/d11/d31/d39/faf x:0 0 0 2024-09-21T04:35:37.188 INFO:tasks.workunit.client.0.smithi007.stdout:8/613: chown d0/l50 254268 1 2024-09-21T04:35:37.192 INFO:tasks.workunit.client.0.smithi007.stdout:7/776: mkdir d3/d6/d14/d20/d80/dca/d113 0 2024-09-21T04:35:37.192 INFO:tasks.workunit.client.0.smithi007.stdout:8/614: dread d0/d3/d16/f31 [4194304,4194304] 0 2024-09-21T04:35:37.193 INFO:tasks.workunit.client.0.smithi007.stdout:7/777: stat d3/d6/d13/l53 0 2024-09-21T04:35:37.193 INFO:tasks.workunit.client.0.smithi007.stdout:3/834: write dc/d22/d29/d32/d4d/fc2 [338321,69381] 0 2024-09-21T04:35:37.194 INFO:tasks.workunit.client.0.smithi007.stdout:8/615: dread - d0/fa6 zero size 2024-09-21T04:35:37.195 INFO:tasks.workunit.client.0.smithi007.stdout:7/778: chown d3/d6/d10/c1d 18 1 2024-09-21T04:35:37.195 INFO:tasks.workunit.client.0.smithi007.stdout:7/779: stat d3/d6/d14/d20/d58 0 2024-09-21T04:35:37.197 INFO:tasks.workunit.client.0.smithi007.stdout:3/835: readlink dc/d22/d5a/lda 0 2024-09-21T04:35:37.198 INFO:tasks.workunit.client.0.smithi007.stdout:2/605: truncate d0/d32/f34 3756214 0 2024-09-21T04:35:37.201 INFO:tasks.workunit.client.0.smithi007.stdout:8/616: symlink d0/d3/d16/d93/d40/d9e/lae 0 2024-09-21T04:35:37.207 INFO:tasks.workunit.client.0.smithi007.stdout:7/780: unlink d3/d6/d14/d20/d80/dca/cb7 0 2024-09-21T04:35:37.208 INFO:tasks.workunit.client.0.smithi007.stdout:8/617: mknod d0/d3/d16/d93/d5e/caf 0 2024-09-21T04:35:37.208 INFO:tasks.workunit.client.0.smithi007.stdout:7/781: chown d3/d6/d10/d4f/f5c 284 1 2024-09-21T04:35:37.209 INFO:tasks.workunit.client.0.smithi007.stdout:2/606: dwrite d0/d11/d31/d46/f7a [4194304,4194304] 0 2024-09-21T04:35:37.219 INFO:tasks.workunit.client.0.smithi007.stdout:8/618: dread d0/d3/f53 [0,4194304] 0 2024-09-21T04:35:37.224 INFO:tasks.workunit.client.0.smithi007.stdout:3/836: dwrite dc/d18/d41/d96/fd9 [0,4194304] 0 2024-09-21T04:35:37.224 INFO:tasks.workunit.client.0.smithi007.stdout:7/782: creat d3/d6/d14/d20/d80/f114 x:0 0 0 2024-09-21T04:35:37.226 INFO:tasks.workunit.client.0.smithi007.stdout:8/619: symlink d0/d3/d16/d93/d40/d92/lb0 0 2024-09-21T04:35:37.229 INFO:tasks.workunit.client.0.smithi007.stdout:2/607: dwrite d0/d11/f99 [0,4194304] 0 2024-09-21T04:35:37.234 INFO:tasks.workunit.client.0.smithi007.stdout:2/608: mknod d0/d11/d31/cb0 0 2024-09-21T04:35:37.238 INFO:tasks.workunit.client.0.smithi007.stdout:8/620: rename d0/d3/d74/ca4 to d0/d3/d16/d93/d26/d4d/cb1 0 2024-09-21T04:35:37.239 INFO:tasks.workunit.client.0.smithi007.stdout:8/621: fsync d0/d3/f2b 0 2024-09-21T04:35:37.240 INFO:tasks.workunit.client.0.smithi007.stdout:2/609: mkdir d0/d11/db1 0 2024-09-21T04:35:37.243 INFO:tasks.workunit.client.0.smithi007.stdout:8/622: rmdir d0/d3/d16/d93/d26/d4d/d5d 39 2024-09-21T04:35:37.247 INFO:tasks.workunit.client.0.smithi007.stdout:3/837: dwrite dc/d18/d41/fa5 [0,4194304] 0 2024-09-21T04:35:37.249 INFO:tasks.workunit.client.0.smithi007.stdout:7/783: dwrite d3/d6/d14/d20/d58/f95 [0,4194304] 0 2024-09-21T04:35:37.251 INFO:tasks.workunit.client.0.smithi007.stdout:8/623: write d0/d3/d16/d93/d26/d4d/d5d/f86 [819579,129462] 0 2024-09-21T04:35:37.252 INFO:tasks.workunit.client.0.smithi007.stdout:7/784: truncate d3/d6/d14/d20/d80/db2/f108 880844 0 2024-09-21T04:35:37.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/785: write d3/d6/d2e/d63/f9e [2555091,81517] 0 2024-09-21T04:35:37.255 INFO:tasks.workunit.client.0.smithi007.stdout:7/786: chown d3/d6/d13/c23 727157132 1 2024-09-21T04:35:37.265 INFO:tasks.workunit.client.0.smithi007.stdout:7/787: fsync d3/d6/d14/f26 0 2024-09-21T04:35:37.265 INFO:tasks.workunit.client.0.smithi007.stdout:8/624: creat d0/d3/fb2 x:0 0 0 2024-09-21T04:35:37.266 INFO:tasks.workunit.client.0.smithi007.stdout:2/610: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:37.273 INFO:tasks.workunit.client.0.smithi007.stdout:8/625: truncate d0/d3/d16/d93/d40/f95 445817 0 2024-09-21T04:35:37.273 INFO:tasks.workunit.client.0.smithi007.stdout:3/838: write dc/d22/d29/d32/d69/f49 [2435172,97711] 0 2024-09-21T04:35:37.273 INFO:tasks.workunit.client.0.smithi007.stdout:7/788: creat d3/d6/d10/d62/f115 x:0 0 0 2024-09-21T04:35:37.274 INFO:tasks.workunit.client.0.smithi007.stdout:2/611: rmdir d0/d11/d31/d7b 39 2024-09-21T04:35:37.279 INFO:tasks.workunit.client.0.smithi007.stdout:3/839: getdents dc/d22/d29/d32/d4d/d84/dd0/ddb 0 2024-09-21T04:35:37.281 INFO:tasks.workunit.client.0.smithi007.stdout:3/840: write dc/d22/d8d/fe8 [147455,72581] 0 2024-09-21T04:35:37.281 INFO:tasks.workunit.client.0.smithi007.stdout:7/789: creat d3/d6/d40/f116 x:0 0 0 2024-09-21T04:35:37.282 INFO:tasks.workunit.client.0.smithi007.stdout:2/612: creat d0/d11/d31/d7b/fb2 x:0 0 0 2024-09-21T04:35:37.285 INFO:tasks.workunit.client.0.smithi007.stdout:8/626: dread d0/d3/d8/f58 [4194304,4194304] 0 2024-09-21T04:35:37.289 INFO:tasks.workunit.client.0.smithi007.stdout:2/613: creat d0/d11/db1/fb3 x:0 0 0 2024-09-21T04:35:37.289 INFO:tasks.workunit.client.0.smithi007.stdout:7/790: dread d3/d6/d2e/d63/f9e [0,4194304] 0 2024-09-21T04:35:37.291 INFO:tasks.workunit.client.0.smithi007.stdout:7/791: chown d3/d6/d13/fba 437 1 2024-09-21T04:35:37.291 INFO:tasks.workunit.client.0.smithi007.stdout:3/841: dwrite dc/d22/d29/d32/d4d/f8b [4194304,4194304] 0 2024-09-21T04:35:37.292 INFO:tasks.workunit.client.0.smithi007.stdout:8/627: rename d0/d3/d16/d93/d26/d4d/la3 to d0/d3/d16/d93/d5e/lb3 0 2024-09-21T04:35:37.293 INFO:tasks.workunit.client.0.smithi007.stdout:7/792: write d3/d6/d14/d20/d80/db2/f108 [108013,11206] 0 2024-09-21T04:35:37.296 INFO:tasks.workunit.client.0.smithi007.stdout:2/614: unlink d0/d11/d31/d39/l8b 0 2024-09-21T04:35:37.297 INFO:tasks.workunit.client.0.smithi007.stdout:3/842: rmdir dc/d18/d26 39 2024-09-21T04:35:37.300 INFO:tasks.workunit.client.0.smithi007.stdout:8/628: symlink d0/d3/d16/d93/d26/lb4 0 2024-09-21T04:35:37.303 INFO:tasks.workunit.client.0.smithi007.stdout:7/793: mknod d3/d6/d85/dad/d103/c117 0 2024-09-21T04:35:37.305 INFO:tasks.workunit.client.0.smithi007.stdout:7/794: write d3/d6/d2e/d63/d6a/d81/f111 [68261,117359] 0 2024-09-21T04:35:37.305 INFO:tasks.workunit.client.0.smithi007.stdout:2/615: link d0/d11/d31/d39/f8c d0/d11/d31/d46/d6c/fb4 0 2024-09-21T04:35:37.307 INFO:tasks.workunit.client.0.smithi007.stdout:3/843: mknod dc/d22/d29/d32/d4d/d84/dbb/dc1/c106 0 2024-09-21T04:35:37.308 INFO:tasks.workunit.client.0.smithi007.stdout:3/844: chown dc/d18/d41/f4a 397438 1 2024-09-21T04:35:37.311 INFO:tasks.workunit.client.0.smithi007.stdout:8/629: creat d0/d3/d16/d93/d26/fb5 x:0 0 0 2024-09-21T04:35:37.314 INFO:tasks.workunit.client.0.smithi007.stdout:7/795: symlink d3/d6/d2e/l118 0 2024-09-21T04:35:37.316 INFO:tasks.workunit.client.0.smithi007.stdout:2/616: creat d0/d11/d31/d7b/da2/da5/fb5 x:0 0 0 2024-09-21T04:35:37.317 INFO:tasks.workunit.client.0.smithi007.stdout:3/845: truncate dc/f1c 1016052 0 2024-09-21T04:35:37.319 INFO:tasks.workunit.client.0.smithi007.stdout:8/630: mknod d0/d3/d16/d93/d26/d4d/d5d/d88/cb6 0 2024-09-21T04:35:37.321 INFO:tasks.workunit.client.0.smithi007.stdout:7/796: unlink d3/d6/f2d 0 2024-09-21T04:35:37.324 INFO:tasks.workunit.client.0.smithi007.stdout:3/846: symlink dc/d22/d5a/d71/l107 0 2024-09-21T04:35:37.327 INFO:tasks.workunit.client.0.smithi007.stdout:8/631: mkdir d0/d3/d8/d46/db7 0 2024-09-21T04:35:37.327 INFO:tasks.workunit.client.0.smithi007.stdout:8/632: fdatasync d0/d3/d16/f8d 0 2024-09-21T04:35:37.329 INFO:tasks.workunit.client.0.smithi007.stdout:3/847: truncate dc/d22/d29/d32/d4d/d84/dd0/fbf 146223 0 2024-09-21T04:35:37.330 INFO:tasks.workunit.client.0.smithi007.stdout:3/848: chown dc/d22/d29/d32/d4d/d7d 889005 1 2024-09-21T04:35:37.341 INFO:tasks.workunit.client.0.smithi007.stdout:2/617: getdents d0/d11/d31/d9f/da9 0 2024-09-21T04:35:37.346 INFO:tasks.workunit.client.0.smithi007.stdout:3/849: dwrite dc/d22/d29/f100 [0,4194304] 0 2024-09-21T04:35:37.348 INFO:tasks.workunit.client.0.smithi007.stdout:8/633: link d0/d3/d16/d93/d40/d92/f9a d0/fb8 0 2024-09-21T04:35:37.349 INFO:tasks.workunit.client.0.smithi007.stdout:8/634: read d0/d3/d8/d46/f60 [1384399,32594] 0 2024-09-21T04:35:37.383 INFO:tasks.workunit.client.0.smithi007.stdout:2/618: mknod d0/d11/d31/d7b/cb6 0 2024-09-21T04:35:37.386 INFO:tasks.workunit.client.0.smithi007.stdout:8/635: rename d0/d3/d16/d93/d26/d4d/d5d/d88/f98 to d0/d3/d82/fb9 0 2024-09-21T04:35:37.389 INFO:tasks.workunit.client.0.smithi007.stdout:7/797: write d3/d6/d10/d4f/fdb [677743,84984] 0 2024-09-21T04:35:37.390 INFO:tasks.workunit.client.0.smithi007.stdout:2/619: unlink d0/f7 0 2024-09-21T04:35:37.392 INFO:tasks.workunit.client.0.smithi007.stdout:8/636: mkdir d0/d3/d16/d93/d26/d4d/d5d/dba 0 2024-09-21T04:35:37.392 INFO:tasks.workunit.client.0.smithi007.stdout:2/620: write d0/d11/d31/d7b/f89 [1082395,131069] 0 2024-09-21T04:35:37.393 INFO:tasks.workunit.client.0.smithi007.stdout:7/798: readlink d3/d6/d10/d51/l7c 0 2024-09-21T04:35:37.394 INFO:tasks.workunit.client.0.smithi007.stdout:2/621: readlink d0/d11/l59 0 2024-09-21T04:35:37.395 INFO:tasks.workunit.client.0.smithi007.stdout:8/637: mknod d0/d3/d8/d46/cbb 0 2024-09-21T04:35:37.397 INFO:tasks.workunit.client.0.smithi007.stdout:8/638: stat d0/c79 0 2024-09-21T04:35:37.398 INFO:tasks.workunit.client.0.smithi007.stdout:7/799: creat d3/d6/d14/d20/d80/db2/d10b/f119 x:0 0 0 2024-09-21T04:35:37.402 INFO:tasks.workunit.client.0.smithi007.stdout:7/800: creat d3/d6/d2e/d63/df8/dfc/f11a x:0 0 0 2024-09-21T04:35:37.404 INFO:tasks.workunit.client.0.smithi007.stdout:8/639: dwrite d0/d3/d16/f99 [0,4194304] 0 2024-09-21T04:35:37.408 INFO:tasks.workunit.client.0.smithi007.stdout:7/801: rename d3/d6/d10/d62 to d3/d6/d85/de7/d11b 0 2024-09-21T04:35:37.409 INFO:tasks.workunit.client.0.smithi007.stdout:8/640: creat d0/d3/d8/d46/db7/fbc x:0 0 0 2024-09-21T04:35:37.414 INFO:tasks.workunit.client.0.smithi007.stdout:8/641: getdents d0/d3/d16/d93/d26 0 2024-09-21T04:35:37.418 INFO:tasks.workunit.client.0.smithi007.stdout:7/802: dwrite d3/d6/d13/fc4 [0,4194304] 0 2024-09-21T04:35:37.421 INFO:tasks.workunit.client.0.smithi007.stdout:8/642: link d0/d3/d8/f47 d0/d3/d16/d93/d26/d4d/fbd 0 2024-09-21T04:35:37.422 INFO:tasks.workunit.client.0.smithi007.stdout:7/803: mkdir d3/d6/d10/d51/d11c 0 2024-09-21T04:35:37.423 INFO:tasks.workunit.client.0.smithi007.stdout:8/643: truncate d0/d3/d8/d46/db7/fbc 822192 0 2024-09-21T04:35:37.424 INFO:tasks.workunit.client.0.smithi007.stdout:8/644: chown d0/d3/d16/d93/d26/d35 43603855 1 2024-09-21T04:35:37.424 INFO:tasks.workunit.client.0.smithi007.stdout:7/804: creat d3/d6/d2e/d63/d6a/d81/f11d x:0 0 0 2024-09-21T04:35:37.427 INFO:tasks.workunit.client.0.smithi007.stdout:4/926: sync 2024-09-21T04:35:37.434 INFO:tasks.workunit.client.0.smithi007.stdout:4/927: truncate d2/d17/d2e/d45/d5d/d8a/fd5 100475 0 2024-09-21T04:35:37.442 INFO:tasks.workunit.client.0.smithi007.stdout:8/645: dread d0/d3/d8/f58 [0,4194304] 0 2024-09-21T04:35:37.446 INFO:tasks.workunit.client.0.smithi007.stdout:4/928: fdatasync d2/d7/d55/f92 0 2024-09-21T04:35:37.449 INFO:tasks.workunit.client.0.smithi007.stdout:8/646: rename d0/d3/d8/d46/c77 to d0/d3/d7c/cbe 0 2024-09-21T04:35:37.458 INFO:tasks.workunit.client.0.smithi007.stdout:8/647: dwrite d0/d3/d16/f21 [4194304,4194304] 0 2024-09-21T04:35:37.460 INFO:tasks.workunit.client.0.smithi007.stdout:8/648: write d0/d3/d7c/f54 [523688,6327] 0 2024-09-21T04:35:37.474 INFO:tasks.workunit.client.0.smithi007.stdout:3/850: write dc/d22/dc8/fd2 [855109,117649] 0 2024-09-21T04:35:37.478 INFO:tasks.workunit.client.0.smithi007.stdout:3/851: mknod dc/d22/d5a/d71/d89/c108 0 2024-09-21T04:35:37.480 INFO:tasks.workunit.client.0.smithi007.stdout:3/852: dread - dc/d6c/fb1 zero size 2024-09-21T04:35:37.485 INFO:tasks.workunit.client.0.smithi007.stdout:3/853: mkdir dc/d18/d41/d109 0 2024-09-21T04:35:37.494 INFO:tasks.workunit.client.0.smithi007.stdout:2/622: write d0/d11/d31/d46/d6c/fb4 [481510,118494] 0 2024-09-21T04:35:37.500 INFO:tasks.workunit.client.0.smithi007.stdout:2/623: dwrite d0/d11/d31/d7b/f89 [0,4194304] 0 2024-09-21T04:35:37.502 INFO:tasks.workunit.client.0.smithi007.stdout:3/854: dread dc/d18/f7f [0,4194304] 0 2024-09-21T04:35:37.505 INFO:tasks.workunit.client.0.smithi007.stdout:3/855: truncate dc/d22/d29/f55 1742619 0 2024-09-21T04:35:37.506 INFO:tasks.workunit.client.0.smithi007.stdout:2/624: dwrite d0/fd [0,4194304] 0 2024-09-21T04:35:37.509 INFO:tasks.workunit.client.0.smithi007.stdout:3/856: unlink dc/d22/d5a/d71/ff9 0 2024-09-21T04:35:37.512 INFO:tasks.workunit.client.0.smithi007.stdout:2/625: mkdir d0/d11/d31/d46/d6c/db7 0 2024-09-21T04:35:37.516 INFO:tasks.workunit.client.0.smithi007.stdout:3/857: rename dc/d22/d29/d32/f9e to dc/d22/d29/d92/db9/f10a 0 2024-09-21T04:35:37.520 INFO:tasks.workunit.client.0.smithi007.stdout:2/626: getdents d0/d11/d6e 0 2024-09-21T04:35:37.522 INFO:tasks.workunit.client.0.smithi007.stdout:2/627: write d0/d11/d31/d7b/f93 [1000319,25706] 0 2024-09-21T04:35:37.529 INFO:tasks.workunit.client.0.smithi007.stdout:7/805: write d3/d6/d14/f26 [3696285,112772] 0 2024-09-21T04:35:37.529 INFO:tasks.workunit.client.0.smithi007.stdout:2/628: dwrite d0/d11/d31/d39/f3e [0,4194304] 0 2024-09-21T04:35:37.532 INFO:tasks.workunit.client.0.smithi007.stdout:3/858: dread dc/d22/d5a/f93 [0,4194304] 0 2024-09-21T04:35:37.533 INFO:tasks.workunit.client.0.smithi007.stdout:8/649: write d0/d3/d16/d93/d40/d92/f9a [774423,41826] 0 2024-09-21T04:35:37.535 INFO:tasks.workunit.client.0.smithi007.stdout:8/650: fsync d0/d3/d74/f66 0 2024-09-21T04:35:37.537 INFO:tasks.workunit.client.0.smithi007.stdout:2/629: readlink d0/d11/d31/d39/l9e 0 2024-09-21T04:35:37.538 INFO:tasks.workunit.client.0.smithi007.stdout:3/859: write dc/d22/f24 [2352163,50980] 0 2024-09-21T04:35:37.538 INFO:tasks.workunit.client.0.smithi007.stdout:8/651: fsync d0/d3/d7c/f54 0 2024-09-21T04:35:37.541 INFO:tasks.workunit.client.0.smithi007.stdout:8/652: chown d0/d3/d74/f66 3447 1 2024-09-21T04:35:37.542 INFO:tasks.workunit.client.0.smithi007.stdout:4/929: dwrite d2/d7/d64/f68 [0,4194304] 0 2024-09-21T04:35:37.542 INFO:tasks.workunit.client.0.smithi007.stdout:2/630: write d0/d11/d31/d7b/f89 [947760,123478] 0 2024-09-21T04:35:37.548 INFO:tasks.workunit.client.0.smithi007.stdout:8/653: mkdir d0/d3/d7c/dbf 0 2024-09-21T04:35:37.548 INFO:tasks.workunit.client.0.smithi007.stdout:8/654: write d0/d3/d74/f66 [3066128,61823] 0 2024-09-21T04:35:37.551 INFO:tasks.workunit.client.0.smithi007.stdout:8/655: write d0/d3/d16/d93/d26/d35/f73 [6304720,40375] 0 2024-09-21T04:35:37.553 INFO:tasks.workunit.client.0.smithi007.stdout:2/631: mkdir d0/d11/db8 0 2024-09-21T04:35:37.559 INFO:tasks.workunit.client.0.smithi007.stdout:2/632: mkdir d0/d11/d31/d46/d6d/db9 0 2024-09-21T04:35:37.562 INFO:tasks.workunit.client.0.smithi007.stdout:3/860: link dc/d22/d29/d32/d69/cb7 dc/d18/d41/d96/c10b 0 2024-09-21T04:35:37.562 INFO:tasks.workunit.client.0.smithi007.stdout:8/656: symlink d0/d3/d16/lc0 0 2024-09-21T04:35:37.564 INFO:tasks.workunit.client.0.smithi007.stdout:2/633: chown d0/l67 189339 1 2024-09-21T04:35:37.567 INFO:tasks.workunit.client.0.smithi007.stdout:2/634: write d0/d11/d31/d39/faf [217189,11170] 0 2024-09-21T04:35:37.568 INFO:tasks.workunit.client.0.smithi007.stdout:7/806: dwrite d3/d6/d85/f25 [8388608,4194304] 0 2024-09-21T04:35:37.572 INFO:tasks.workunit.client.0.smithi007.stdout:8/657: dread d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:37.575 INFO:tasks.workunit.client.0.smithi007.stdout:4/930: dwrite d2/d17/d2e/d45/d8e/fb7 [0,4194304] 0 2024-09-21T04:35:37.575 INFO:tasks.workunit.client.0.smithi007.stdout:5/796: sync 2024-09-21T04:35:37.576 INFO:tasks.workunit.client.0.smithi007.stdout:2/635: rename d0/f29 to d0/d11/d31/d9f/da9/fba 0 2024-09-21T04:35:37.581 INFO:tasks.workunit.client.0.smithi007.stdout:8/658: fsync d0/d3/d16/d93/d5e/f61 0 2024-09-21T04:35:37.584 INFO:tasks.workunit.client.0.smithi007.stdout:7/807: rename d3/d6/d10/f34 to d3/d6/d85/de9/f11e 0 2024-09-21T04:35:37.590 INFO:tasks.workunit.client.0.smithi007.stdout:5/797: dwrite d2/f5a [0,4194304] 0 2024-09-21T04:35:37.591 INFO:tasks.workunit.client.0.smithi007.stdout:4/931: write d2/d7/d55/d61/f73 [5020372,54424] 0 2024-09-21T04:35:37.593 INFO:tasks.workunit.client.0.smithi007.stdout:3/861: dwrite dc/d18/f7b [0,4194304] 0 2024-09-21T04:35:37.593 INFO:tasks.workunit.client.0.smithi007.stdout:7/808: dread - d3/d6/d40/f48 zero size 2024-09-21T04:35:37.594 INFO:tasks.workunit.client.0.smithi007.stdout:4/932: chown d2/d17/d29/c8d 3423400 1 2024-09-21T04:35:37.595 INFO:tasks.workunit.client.0.smithi007.stdout:2/636: creat d0/d11/d31/da4/fbb x:0 0 0 2024-09-21T04:35:37.597 INFO:tasks.workunit.client.0.smithi007.stdout:8/659: creat d0/d3/d7c/fc1 x:0 0 0 2024-09-21T04:35:37.598 INFO:tasks.workunit.client.0.smithi007.stdout:8/660: read d0/d3/daa/f97 [2069847,89604] 0 2024-09-21T04:35:37.600 INFO:tasks.workunit.client.0.smithi007.stdout:5/798: mknod d2/d27/d42/d53/daf/cfc 0 2024-09-21T04:35:37.602 INFO:tasks.workunit.client.0.smithi007.stdout:5/799: read d2/d1e/f58 [686524,65886] 0 2024-09-21T04:35:37.602 INFO:tasks.workunit.client.0.smithi007.stdout:4/933: creat d2/d17/d2e/d45/df2/f13a x:0 0 0 2024-09-21T04:35:37.602 INFO:tasks.workunit.client.0.smithi007.stdout:2/637: rmdir d0/d11/d6b/d9d 39 2024-09-21T04:35:37.603 INFO:tasks.workunit.client.0.smithi007.stdout:4/934: chown d2/d97/le7 17043749 1 2024-09-21T04:35:37.604 INFO:tasks.workunit.client.0.smithi007.stdout:5/800: chown d2/d27/d42/da3/d23/d2a/f63 53 1 2024-09-21T04:35:37.604 INFO:tasks.workunit.client.0.smithi007.stdout:7/809: unlink d3/d6/d2e/d63/df8/dfc/f11a 0 2024-09-21T04:35:37.607 INFO:tasks.workunit.client.0.smithi007.stdout:8/661: mknod d0/d3/daa/cc2 0 2024-09-21T04:35:37.610 INFO:tasks.workunit.client.0.smithi007.stdout:8/662: write d0/d3/d8/d46/db7/fbc [1737054,36915] 0 2024-09-21T04:35:37.611 INFO:tasks.workunit.client.0.smithi007.stdout:2/638: dread - d0/d11/d31/d52/f87 zero size 2024-09-21T04:35:37.613 INFO:tasks.workunit.client.0.smithi007.stdout:5/801: dwrite d2/d27/d3d/d9b/ff8 [0,4194304] 0 2024-09-21T04:35:37.615 INFO:tasks.workunit.client.0.smithi007.stdout:5/802: chown d2/c57 697 1 2024-09-21T04:35:37.617 INFO:tasks.workunit.client.0.smithi007.stdout:9/903: sync 2024-09-21T04:35:37.617 INFO:tasks.workunit.client.0.smithi007.stdout:1/999: sync 2024-09-21T04:35:37.617 INFO:tasks.workunit.client.0.smithi007.stdout:0/745: sync 2024-09-21T04:35:37.617 INFO:tasks.workunit.client.0.smithi007.stdout:6/738: sync 2024-09-21T04:35:37.626 INFO:tasks.workunit.client.0.smithi007.stdout:2/639: mkdir d0/d11/d31/d46/d6c/dbc 0 2024-09-21T04:35:37.628 INFO:tasks.workunit.client.0.smithi007.stdout:3/862: dwrite dc/d18/d26/f4b [0,4194304] 0 2024-09-21T04:35:37.631 INFO:tasks.workunit.client.0.smithi007.stdout:5/803: symlink d2/d27/d42/da3/d23/d5d/d9a/lfd 0 2024-09-21T04:35:37.632 INFO:tasks.workunit.client.0.smithi007.stdout:7/810: fsync d3/d6/f2f 0 2024-09-21T04:35:37.635 INFO:tasks.workunit.client.0.smithi007.stdout:5/804: stat d2/d27/d42/da3/d23/d5d/lca 0 2024-09-21T04:35:37.637 INFO:tasks.workunit.client.0.smithi007.stdout:2/640: dwrite d0/f18 [0,4194304] 0 2024-09-21T04:35:37.639 INFO:tasks.workunit.client.0.smithi007.stdout:0/746: rename d6/d75/dc5/d18/f69 to d6/f115 0 2024-09-21T04:35:37.639 INFO:tasks.workunit.client.0.smithi007.stdout:4/935: dread d2/f6 [0,4194304] 0 2024-09-21T04:35:37.643 INFO:tasks.workunit.client.0.smithi007.stdout:8/663: write d0/d3/d16/d93/d5e/f61 [3861646,12093] 0 2024-09-21T04:35:37.650 INFO:tasks.workunit.client.0.smithi007.stdout:9/904: write d13/d74/f98 [837353,78222] 0 2024-09-21T04:35:37.653 INFO:tasks.workunit.client.0.smithi007.stdout:9/905: write d13/d74/f98 [795709,22744] 0 2024-09-21T04:35:37.653 INFO:tasks.workunit.client.0.smithi007.stdout:8/664: dwrite d0/d3/d8/f9 [4194304,4194304] 0 2024-09-21T04:35:37.655 INFO:tasks.workunit.client.0.smithi007.stdout:5/805: mknod d2/d27/d3d/cfe 0 2024-09-21T04:35:37.656 INFO:tasks.workunit.client.0.smithi007.stdout:0/747: stat d6/d75/dc5/l1a 0 2024-09-21T04:35:37.660 INFO:tasks.workunit.client.0.smithi007.stdout:7/811: dread d3/d6/d13/f29 [0,4194304] 0 2024-09-21T04:35:37.660 INFO:tasks.workunit.client.0.smithi007.stdout:6/739: truncate d3/de/d12/d19/da3/fc3 848506 0 2024-09-21T04:35:37.661 INFO:tasks.workunit.client.0.smithi007.stdout:7/812: dread - d3/d6/d10/ff5 zero size 2024-09-21T04:35:37.667 INFO:tasks.workunit.client.0.smithi007.stdout:4/936: dwrite d2/f27 [0,4194304] 0 2024-09-21T04:35:37.672 INFO:tasks.workunit.client.0.smithi007.stdout:5/806: creat d2/d27/d42/da3/d23/d49/dd4/de9/fff x:0 0 0 2024-09-21T04:35:37.678 INFO:tasks.workunit.client.0.smithi007.stdout:2/641: getdents d0/d11/d31/d46/d6c/db7 0 2024-09-21T04:35:37.681 INFO:tasks.workunit.client.0.smithi007.stdout:7/813: symlink d3/d6/d2e/d63/df8/l11f 0 2024-09-21T04:35:37.682 INFO:tasks.workunit.client.0.smithi007.stdout:3/863: truncate dc/d22/d29/d92/db9/ffc 452491 0 2024-09-21T04:35:37.685 INFO:tasks.workunit.client.0.smithi007.stdout:9/906: rename d13/lfe to d13/d74/de1/d105/l125 0 2024-09-21T04:35:37.689 INFO:tasks.workunit.client.0.smithi007.stdout:7/814: read d3/d6/f2f [4268614,46460] 0 2024-09-21T04:35:37.689 INFO:tasks.workunit.client.0.smithi007.stdout:3/864: mknod dc/d22/d29/d32/d4d/d84/dd0/da6/c10c 0 2024-09-21T04:35:37.692 INFO:tasks.workunit.client.0.smithi007.stdout:0/748: dwrite d6/d75/dc5/f86 [0,4194304] 0 2024-09-21T04:35:37.692 INFO:tasks.workunit.client.0.smithi007.stdout:8/665: rename d0/d3/d16/lc0 to d0/d3/d7c/dbf/lc3 0 2024-09-21T04:35:37.693 INFO:tasks.workunit.client.0.smithi007.stdout:8/666: readlink d0/d3/d16/d93/d26/l6d 0 2024-09-21T04:35:37.694 INFO:tasks.workunit.client.0.smithi007.stdout:8/667: stat d0/d3/d16/d93/d5e/f61 0 2024-09-21T04:35:37.698 INFO:tasks.workunit.client.0.smithi007.stdout:5/807: unlink d2/d27/d42/da3/d28/f97 0 2024-09-21T04:35:37.702 INFO:tasks.workunit.client.0.smithi007.stdout:8/668: dwrite d0/d3/d16/d93/d5e/d7b/f7e [0,4194304] 0 2024-09-21T04:35:37.706 INFO:tasks.workunit.client.0.smithi007.stdout:4/937: truncate d2/d17/d2e/d45/f8f 684541 0 2024-09-21T04:35:37.715 INFO:tasks.workunit.client.0.smithi007.stdout:6/740: link d3/de/d12/d19/f1b d3/d3b/ff5 0 2024-09-21T04:35:37.717 INFO:tasks.workunit.client.0.smithi007.stdout:6/741: write d3/d94/fb3 [279167,72129] 0 2024-09-21T04:35:37.720 INFO:tasks.workunit.client.0.smithi007.stdout:7/815: dwrite d3/d6/d14/d20/f35 [0,4194304] 0 2024-09-21T04:35:37.722 INFO:tasks.workunit.client.0.smithi007.stdout:7/816: chown d3/d6/d2e/d63/d6a/c74 44 1 2024-09-21T04:35:37.724 INFO:tasks.workunit.client.0.smithi007.stdout:0/749: mkdir d6/d75/dc5/d18/d2e/d116 0 2024-09-21T04:35:37.725 INFO:tasks.workunit.client.0.smithi007.stdout:7/817: truncate d3/d6/d14/d20/d58/f106 541518 0 2024-09-21T04:35:37.726 INFO:tasks.workunit.client.0.smithi007.stdout:7/818: readlink d3/l6c 0 2024-09-21T04:35:37.726 INFO:tasks.workunit.client.0.smithi007.stdout:9/907: creat d13/d18/d60/d6d/d75/d9f/f126 x:0 0 0 2024-09-21T04:35:37.726 INFO:tasks.workunit.client.0.smithi007.stdout:5/808: mknod d2/d27/dee/c100 0 2024-09-21T04:35:37.728 INFO:tasks.workunit.client.0.smithi007.stdout:8/669: mknod d0/d3/d16/cc4 0 2024-09-21T04:35:37.729 INFO:tasks.workunit.client.0.smithi007.stdout:9/908: write d13/d18/d60/d6d/d75/d9f/f122 [865533,22396] 0 2024-09-21T04:35:37.730 INFO:tasks.workunit.client.0.smithi007.stdout:3/865: link dc/d18/d26/f5f dc/d6c/da4/f10d 0 2024-09-21T04:35:37.735 INFO:tasks.workunit.client.0.smithi007.stdout:0/750: fdatasync d6/d75/dc5/d18/d27/fb7 0 2024-09-21T04:35:37.736 INFO:tasks.workunit.client.0.smithi007.stdout:0/751: chown d6/d75/dc5/d18/d2e/l107 847880 1 2024-09-21T04:35:37.739 INFO:tasks.workunit.client.0.smithi007.stdout:8/670: creat d0/d3/daa/fc5 x:0 0 0 2024-09-21T04:35:37.743 INFO:tasks.workunit.client.0.smithi007.stdout:5/809: truncate d2/d27/d42/da3/d28/f8c 2072861 0 2024-09-21T04:35:37.743 INFO:tasks.workunit.client.0.smithi007.stdout:9/909: dread d13/d74/de1/dec/f109 [0,4194304] 0 2024-09-21T04:35:37.745 INFO:tasks.workunit.client.0.smithi007.stdout:5/810: dread - d2/d27/d3d/f69 zero size 2024-09-21T04:35:37.747 INFO:tasks.workunit.client.0.smithi007.stdout:0/752: mkdir d6/d75/dc5/d57/d117 0 2024-09-21T04:35:37.748 INFO:tasks.workunit.client.0.smithi007.stdout:8/671: rename d0/d3/d16/d93/d5e/f61 to d0/d3/d7c/fc6 0 2024-09-21T04:35:37.749 INFO:tasks.workunit.client.0.smithi007.stdout:6/742: getdents d3/d3b/dd3/d60/d98 0 2024-09-21T04:35:37.750 INFO:tasks.workunit.client.0.smithi007.stdout:4/938: dwrite d2/d7/d55/d4f/fbc [0,4194304] 0 2024-09-21T04:35:37.750 INFO:tasks.workunit.client.0.smithi007.stdout:3/866: mkdir dc/d22/d29/d32/d4d/d84/dbb/dc1/dca/d10e 0 2024-09-21T04:35:37.752 INFO:tasks.workunit.client.0.smithi007.stdout:7/819: dread d3/d6/d2e/d63/f73 [0,4194304] 0 2024-09-21T04:35:37.752 INFO:tasks.workunit.client.0.smithi007.stdout:9/910: creat d13/d74/de1/dec/f127 x:0 0 0 2024-09-21T04:35:37.753 INFO:tasks.workunit.client.0.smithi007.stdout:8/672: chown d0/d3/d16/d93/d40/d92/lb0 1212 1 2024-09-21T04:35:37.755 INFO:tasks.workunit.client.0.smithi007.stdout:5/811: rename d2/d27/d42/d53/f90 to d2/d27/d42/da3/d23/d49/dd4/f101 0 2024-09-21T04:35:37.761 INFO:tasks.workunit.client.0.smithi007.stdout:3/867: fdatasync dc/d22/d29/d32/d4d/fa0 0 2024-09-21T04:35:37.761 INFO:tasks.workunit.client.0.smithi007.stdout:9/911: dwrite d13/fe5 [0,4194304] 0 2024-09-21T04:35:37.771 INFO:tasks.workunit.client.0.smithi007.stdout:8/673: readlink d0/d3/d7c/dbf/lc3 0 2024-09-21T04:35:37.773 INFO:tasks.workunit.client.0.smithi007.stdout:6/743: dread d3/de/d12/d19/f20 [0,4194304] 0 2024-09-21T04:35:37.774 INFO:tasks.workunit.client.0.smithi007.stdout:7/820: unlink d3/d6/d14/d20/d80/db2/d10b/f119 0 2024-09-21T04:35:37.778 INFO:tasks.workunit.client.0.smithi007.stdout:5/812: rename d2/d27/d42/da3/d23/fdc to d2/d27/d42/da3/d23/d49/dd4/dd5/f102 0 2024-09-21T04:35:37.781 INFO:tasks.workunit.client.0.smithi007.stdout:9/912: chown d13/f5e 1644 1 2024-09-21T04:35:37.781 INFO:tasks.workunit.client.0.smithi007.stdout:3/868: symlink dc/d22/d29/d32/d4d/l10f 0 2024-09-21T04:35:37.782 INFO:tasks.workunit.client.0.smithi007.stdout:4/939: write d2/d17/d2e/d45/d5d/d8a/fdd [837343,118800] 0 2024-09-21T04:35:37.785 INFO:tasks.workunit.client.0.smithi007.stdout:6/744: creat d3/d3b/dd3/d60/dd7/ff6 x:0 0 0 2024-09-21T04:35:37.786 INFO:tasks.workunit.client.0.smithi007.stdout:7/821: unlink d3/d6/d40/da1/cd4 0 2024-09-21T04:35:37.788 INFO:tasks.workunit.client.0.smithi007.stdout:5/813: rename d2 to d2/d103 22 2024-09-21T04:35:37.794 INFO:tasks.workunit.client.0.smithi007.stdout:4/940: creat d2/d7/dfb/f13b x:0 0 0 2024-09-21T04:35:37.794 INFO:tasks.workunit.client.0.smithi007.stdout:6/745: rmdir d3/d3e/d99/dbb/d95 39 2024-09-21T04:35:37.796 INFO:tasks.workunit.client.0.smithi007.stdout:5/814: dwrite d2/d27/d42/da3/d23/d49/f4a [0,4194304] 0 2024-09-21T04:35:37.798 INFO:tasks.workunit.client.0.smithi007.stdout:8/674: creat d0/d3/fc7 x:0 0 0 2024-09-21T04:35:37.798 INFO:tasks.workunit.client.0.smithi007.stdout:9/913: rmdir d13/d89 39 2024-09-21T04:35:37.798 INFO:tasks.workunit.client.0.smithi007.stdout:3/869: getdents dc/d18/d41/d109 0 2024-09-21T04:35:37.799 INFO:tasks.workunit.client.0.smithi007.stdout:4/941: chown d2/l75 692 1 2024-09-21T04:35:37.800 INFO:tasks.workunit.client.0.smithi007.stdout:8/675: truncate d0/fa6 560346 0 2024-09-21T04:35:37.801 INFO:tasks.workunit.client.0.smithi007.stdout:9/914: truncate d13/d18/d60/f88 3355386 0 2024-09-21T04:35:37.801 INFO:tasks.workunit.client.0.smithi007.stdout:6/746: unlink d3/de/d12/d19/da3/cb9 0 2024-09-21T04:35:37.802 INFO:tasks.workunit.client.0.smithi007.stdout:7/822: dread d3/d6/d85/de7/d11b/d66/fa3 [0,4194304] 0 2024-09-21T04:35:37.803 INFO:tasks.workunit.client.0.smithi007.stdout:5/815: mkdir d2/d27/d42/da3/d23/d49/d6f/d104 0 2024-09-21T04:35:37.803 INFO:tasks.workunit.client.0.smithi007.stdout:9/915: dread - d13/d74/ddb/fcb zero size 2024-09-21T04:35:37.805 INFO:tasks.workunit.client.0.smithi007.stdout:3/870: unlink dc/d56/faa 0 2024-09-21T04:35:37.808 INFO:tasks.workunit.client.0.smithi007.stdout:4/942: symlink d2/d17/d2e/d45/d5e/d10c/l13c 0 2024-09-21T04:35:37.810 INFO:tasks.workunit.client.0.smithi007.stdout:4/943: readlink d2/d17/d2e/d45/d5d/dae/lb3 0 2024-09-21T04:35:37.811 INFO:tasks.workunit.client.0.smithi007.stdout:4/944: chown d2/l75 245 1 2024-09-21T04:35:37.812 INFO:tasks.workunit.client.0.smithi007.stdout:9/916: dwrite d13/d18/f3f [0,4194304] 0 2024-09-21T04:35:37.813 INFO:tasks.workunit.client.0.smithi007.stdout:5/816: mknod d2/d27/d42/da3/d23/d49/d6f/ddb/def/c105 0 2024-09-21T04:35:37.815 INFO:tasks.workunit.client.0.smithi007.stdout:3/871: mknod dc/d22/d29/d32/d69/de4/c110 0 2024-09-21T04:35:37.816 INFO:tasks.workunit.client.0.smithi007.stdout:6/747: mknod d3/d3e/d99/dbb/df1/cf7 0 2024-09-21T04:35:37.818 INFO:tasks.workunit.client.0.smithi007.stdout:5/817: mknod d2/d27/d42/da3/d23/d49/dd4/de9/c106 0 2024-09-21T04:35:37.820 INFO:tasks.workunit.client.0.smithi007.stdout:8/676: rename d0/d3/d16/d93/d26/d4d/d5d/c84 to d0/d3/cc8 0 2024-09-21T04:35:37.820 INFO:tasks.workunit.client.0.smithi007.stdout:3/872: symlink dc/d6c/l111 0 2024-09-21T04:35:37.822 INFO:tasks.workunit.client.0.smithi007.stdout:9/917: write d13/f21 [6452930,124600] 0 2024-09-21T04:35:37.823 INFO:tasks.workunit.client.0.smithi007.stdout:7/823: link d3/d6/d40/ca4 d3/d6/d40/d112/c120 0 2024-09-21T04:35:37.823 INFO:tasks.workunit.client.0.smithi007.stdout:4/945: dread d2/d17/d29/f85 [0,4194304] 0 2024-09-21T04:35:37.824 INFO:tasks.workunit.client.0.smithi007.stdout:5/818: symlink d2/d27/d42/da3/d1d/d20/l107 0 2024-09-21T04:35:37.826 INFO:tasks.workunit.client.0.smithi007.stdout:8/677: chown d0/d3/d16/d93/d26/d4d/d5d/c7a 1649 1 2024-09-21T04:35:37.826 INFO:tasks.workunit.client.0.smithi007.stdout:6/748: creat d3/d3b/dd3/dd6/ff8 x:0 0 0 2024-09-21T04:35:37.828 INFO:tasks.workunit.client.0.smithi007.stdout:8/678: truncate d0/d3/d8/d46/f7d 1982457 0 2024-09-21T04:35:37.829 INFO:tasks.workunit.client.0.smithi007.stdout:7/824: fdatasync d3/d6/d40/da1/fc3 0 2024-09-21T04:35:37.830 INFO:tasks.workunit.client.0.smithi007.stdout:5/819: chown d2/d27/d42/da3/f14 4408793 1 2024-09-21T04:35:37.830 INFO:tasks.workunit.client.0.smithi007.stdout:4/946: mkdir d2/d7/dc9/d13d 0 2024-09-21T04:35:37.833 INFO:tasks.workunit.client.0.smithi007.stdout:3/873: mkdir dc/d18/d41/d109/d112 0 2024-09-21T04:35:37.834 INFO:tasks.workunit.client.0.smithi007.stdout:3/874: chown dc/d22/d29/dac 60 1 2024-09-21T04:35:37.834 INFO:tasks.workunit.client.0.smithi007.stdout:6/749: fsync d3/d94/fc4 0 2024-09-21T04:35:37.837 INFO:tasks.workunit.client.0.smithi007.stdout:6/750: write d3/de/d58/faa [2258156,95868] 0 2024-09-21T04:35:37.838 INFO:tasks.workunit.client.0.smithi007.stdout:3/875: read dc/d22/d29/d32/d4d/fc2 [75475,68630] 0 2024-09-21T04:35:37.839 INFO:tasks.workunit.client.0.smithi007.stdout:4/947: dwrite d2/d17/d2e/d45/d5d/fb1 [0,4194304] 0 2024-09-21T04:35:37.840 INFO:tasks.workunit.client.0.smithi007.stdout:4/948: stat d2/d17/d29 0 2024-09-21T04:35:37.849 INFO:tasks.workunit.client.0.smithi007.stdout:9/918: write d13/d18/d80/fdf [563215,99368] 0 2024-09-21T04:35:37.852 INFO:tasks.workunit.client.0.smithi007.stdout:6/751: mkdir d3/d94/df9 0 2024-09-21T04:35:37.853 INFO:tasks.workunit.client.0.smithi007.stdout:6/752: fdatasync d3/d3b/dd3/d60/ff2 0 2024-09-21T04:35:37.854 INFO:tasks.workunit.client.0.smithi007.stdout:3/876: creat dc/d18/dea/f113 x:0 0 0 2024-09-21T04:35:37.856 INFO:tasks.workunit.client.0.smithi007.stdout:4/949: mknod d2/d17/d2e/d9f/c13e 0 2024-09-21T04:35:37.857 INFO:tasks.workunit.client.0.smithi007.stdout:4/950: chown d2/d88/da8/d114/c12f 1007513047 1 2024-09-21T04:35:37.859 INFO:tasks.workunit.client.0.smithi007.stdout:9/919: fsync d13/d18/d60/d83/da9/f9e 0 2024-09-21T04:35:37.860 INFO:tasks.workunit.client.0.smithi007.stdout:6/753: mknod d3/d3b/dd3/dd6/cfa 0 2024-09-21T04:35:37.862 INFO:tasks.workunit.client.0.smithi007.stdout:7/825: dwrite d3/d6/d2e/d63/f73 [0,4194304] 0 2024-09-21T04:35:37.864 INFO:tasks.workunit.client.0.smithi007.stdout:8/679: getdents d0/d3/d82 0 2024-09-21T04:35:37.865 INFO:tasks.workunit.client.0.smithi007.stdout:4/951: unlink d2/d17/d2e/d45/d5d/d8a/fdd 0 2024-09-21T04:35:37.865 INFO:tasks.workunit.client.0.smithi007.stdout:8/680: chown d0/d3/d16/d93/d26/d4d/d5d/l89 6241425 1 2024-09-21T04:35:37.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/877: link dc/d22/d29/d32/d69/f49 dc/d22/d29/d32/d69/d6d/f114 0 2024-09-21T04:35:37.871 INFO:tasks.workunit.client.0.smithi007.stdout:5/820: getdents d2/d27/d3d/d56 0 2024-09-21T04:35:37.876 INFO:tasks.workunit.client.0.smithi007.stdout:9/920: rename d13/d18/d60/d6d/d85/ff1 to d13/d18/dcd/d104/f128 0 2024-09-21T04:35:37.876 INFO:tasks.workunit.client.0.smithi007.stdout:5/821: rename d2 to d2/d27/d42/da3/d28/de2/d108 22 2024-09-21T04:35:37.879 INFO:tasks.workunit.client.0.smithi007.stdout:4/952: stat d2/d17/d18/f84 0 2024-09-21T04:35:37.882 INFO:tasks.workunit.client.0.smithi007.stdout:7/826: symlink d3/d6/d85/de7/d11b/l121 0 2024-09-21T04:35:37.887 INFO:tasks.workunit.client.0.smithi007.stdout:8/681: truncate d0/d3/d16/d93/d26/d4d/f91 187140 0 2024-09-21T04:35:37.888 INFO:tasks.workunit.client.0.smithi007.stdout:8/682: chown d0/d3/fc7 2618657 1 2024-09-21T04:35:37.891 INFO:tasks.workunit.client.0.smithi007.stdout:5/822: dwrite d2/d27/d42/da3/d23/d49/dd4/f101 [0,4194304] 0 2024-09-21T04:35:37.891 INFO:tasks.workunit.client.0.smithi007.stdout:6/754: creat d3/d3e/d99/dbb/d95/dc9/ffb x:0 0 0 2024-09-21T04:35:37.892 INFO:tasks.workunit.client.0.smithi007.stdout:5/823: chown d2/fdd 528941 1 2024-09-21T04:35:37.894 INFO:tasks.workunit.client.0.smithi007.stdout:5/824: chown d2/d27/d42/da3/d23/d2a/f9f 153 1 2024-09-21T04:35:37.898 INFO:tasks.workunit.client.0.smithi007.stdout:4/953: mknod d2/d7/dc9/d13d/c13f 0 2024-09-21T04:35:37.902 INFO:tasks.workunit.client.0.smithi007.stdout:3/878: dwrite dc/d18/f7f [0,4194304] 0 2024-09-21T04:35:37.905 INFO:tasks.workunit.client.0.smithi007.stdout:3/879: chown dc/d22/d29/d32/d4d/d84/dd0/dfb 9432223 1 2024-09-21T04:35:37.907 INFO:tasks.workunit.client.0.smithi007.stdout:5/825: mkdir d2/d27/d42/da3/d23/d49/d6f/ddb/def/d109 0 2024-09-21T04:35:37.908 INFO:tasks.workunit.client.0.smithi007.stdout:4/954: dwrite d2/d3c/dbe/f107 [0,4194304] 0 2024-09-21T04:35:37.911 INFO:tasks.workunit.client.0.smithi007.stdout:6/755: unlink d3/d3e/d99/dbb/le7 0 2024-09-21T04:35:37.912 INFO:tasks.workunit.client.0.smithi007.stdout:5/826: stat d2/d27/d42/da3/fd8 0 2024-09-21T04:35:37.912 INFO:tasks.workunit.client.0.smithi007.stdout:9/921: dwrite d13/d18/d60/d6d/d75/f7e [0,4194304] 0 2024-09-21T04:35:37.913 INFO:tasks.workunit.client.0.smithi007.stdout:4/955: write d2/d5f/f93 [3827955,64413] 0 2024-09-21T04:35:37.915 INFO:tasks.workunit.client.0.smithi007.stdout:4/956: readlink d2/d7/d55/d4f/l7f 0 2024-09-21T04:35:37.915 INFO:tasks.workunit.client.0.smithi007.stdout:7/827: link d3/d6/d2e/d63/d6a/cc8 d3/d6/d85/dad/c122 0 2024-09-21T04:35:37.918 INFO:tasks.workunit.client.0.smithi007.stdout:5/827: write d2/d27/d42/da3/d23/d2a/f63 [4016454,125859] 0 2024-09-21T04:35:37.918 INFO:tasks.workunit.client.0.smithi007.stdout:3/880: creat dc/d22/d29/d92/db9/f115 x:0 0 0 2024-09-21T04:35:37.918 INFO:tasks.workunit.client.0.smithi007.stdout:4/957: rmdir d2/d97 39 2024-09-21T04:35:37.920 INFO:tasks.workunit.client.0.smithi007.stdout:5/828: chown d2/d27/d42/da3/d23/d2a/f63 123 1 2024-09-21T04:35:37.921 INFO:tasks.workunit.client.0.smithi007.stdout:9/922: dread d13/d18/d60/d6d/d75/fd2 [0,4194304] 0 2024-09-21T04:35:37.925 INFO:tasks.workunit.client.0.smithi007.stdout:7/828: mknod d3/d6/d40/da1/c123 0 2024-09-21T04:35:37.929 INFO:tasks.workunit.client.0.smithi007.stdout:3/881: read dc/d18/fbd [147105,68786] 0 2024-09-21T04:35:37.930 INFO:tasks.workunit.client.0.smithi007.stdout:4/958: mknod d2/d17/d29/c140 0 2024-09-21T04:35:37.932 INFO:tasks.workunit.client.0.smithi007.stdout:5/829: fdatasync d2/d27/d42/da3/d1d/f4b 0 2024-09-21T04:35:37.936 INFO:tasks.workunit.client.0.smithi007.stdout:7/829: creat d3/d6/d85/dad/d103/f124 x:0 0 0 2024-09-21T04:35:37.940 INFO:tasks.workunit.client.0.smithi007.stdout:9/923: write d13/d18/d60/d83/da9/f9e [1274335,111932] 0 2024-09-21T04:35:37.941 INFO:tasks.workunit.client.0.smithi007.stdout:3/882: rename dc/d22/d5a to dc/d22/d29/d32/d4d/d7d/d9a/df0/d116 0 2024-09-21T04:35:37.944 INFO:tasks.workunit.client.0.smithi007.stdout:5/830: creat d2/d27/d42/da3/d23/d49/d6f/ddb/f10a x:0 0 0 2024-09-21T04:35:37.945 INFO:tasks.workunit.client.0.smithi007.stdout:7/830: symlink d3/d6/d14/d20/d80/db2/d10b/l125 0 2024-09-21T04:35:37.945 INFO:tasks.workunit.client.0.smithi007.stdout:5/831: write d2/d27/d42/d53/fe6 [2340719,127599] 0 2024-09-21T04:35:37.947 INFO:tasks.workunit.client.0.smithi007.stdout:9/924: creat d13/d18/d60/d6d/d75/d113/f129 x:0 0 0 2024-09-21T04:35:37.948 INFO:tasks.workunit.client.0.smithi007.stdout:3/883: truncate dc/d22/d29/d32/d4d/d84/dd0/f83 2945447 0 2024-09-21T04:35:37.950 INFO:tasks.workunit.client.0.smithi007.stdout:7/831: unlink d3/d6/d10/c52 0 2024-09-21T04:35:37.953 INFO:tasks.workunit.client.0.smithi007.stdout:4/959: getdents d2/d3c/d63 0 2024-09-21T04:35:37.954 INFO:tasks.workunit.client.0.smithi007.stdout:9/925: mknod d13/d74/ddb/d102/c12a 0 2024-09-21T04:35:37.955 INFO:tasks.workunit.client.0.smithi007.stdout:9/926: readlink d13/d18/daf/dda/le0 0 2024-09-21T04:35:37.958 INFO:tasks.workunit.client.0.smithi007.stdout:4/960: creat d2/d17/d2e/d45/d5d/d8a/f141 x:0 0 0 2024-09-21T04:35:37.959 INFO:tasks.workunit.client.0.smithi007.stdout:5/832: creat d2/d27/d42/da3/d23/d5d/d61/dec/f10b x:0 0 0 2024-09-21T04:35:37.963 INFO:tasks.workunit.client.0.smithi007.stdout:4/961: creat d2/d17/d2e/d45/d5d/dc7/f142 x:0 0 0 2024-09-21T04:35:37.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/833: mknod d2/d27/dee/c10c 0 2024-09-21T04:35:37.969 INFO:tasks.workunit.client.0.smithi007.stdout:3/884: write dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/d71/d89/f8a [691651,101386] 0 2024-09-21T04:35:37.972 INFO:tasks.workunit.client.0.smithi007.stdout:9/927: dread d13/d18/d60/d6d/f9d [0,4194304] 0 2024-09-21T04:35:37.973 INFO:tasks.workunit.client.0.smithi007.stdout:3/885: chown dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/c87 14 1 2024-09-21T04:35:37.976 INFO:tasks.workunit.client.0.smithi007.stdout:4/962: rename d2/c30 to d2/d17/d2e/d45/d5e/c143 0 2024-09-21T04:35:37.976 INFO:tasks.workunit.client.0.smithi007.stdout:5/834: dread d2/d27/d42/da3/fd8 [0,4194304] 0 2024-09-21T04:35:37.985 INFO:tasks.workunit.client.0.smithi007.stdout:5/835: fsync d2/d27/d3d/d56/f96 0 2024-09-21T04:35:37.987 INFO:tasks.workunit.client.0.smithi007.stdout:4/963: dread d2/d3c/dbe/f110 [0,4194304] 0 2024-09-21T04:35:37.989 INFO:tasks.workunit.client.0.smithi007.stdout:9/928: rename d13/d18/d60/d6d/d85/da1/ca7 to d13/d5a/c12b 0 2024-09-21T04:35:38.001 INFO:tasks.workunit.client.0.smithi007.stdout:9/929: rename d13/d18/d60/lbb to d13/d5a/l12c 0 2024-09-21T04:35:38.003 INFO:tasks.workunit.client.0.smithi007.stdout:5/836: mknod d2/c10d 0 2024-09-21T04:35:38.005 INFO:tasks.workunit.client.0.smithi007.stdout:9/930: creat d13/d18/d60/d6d/d85/da1/f12d x:0 0 0 2024-09-21T04:35:38.006 INFO:tasks.workunit.client.0.smithi007.stdout:4/964: dwrite d2/d88/fc6 [0,4194304] 0 2024-09-21T04:35:38.007 INFO:tasks.workunit.client.0.smithi007.stdout:4/965: fdatasync d2/d17/d2e/d9f/ff4 0 2024-09-21T04:35:38.010 INFO:tasks.workunit.client.0.smithi007.stdout:9/931: chown d13/d89/fed 289 1 2024-09-21T04:35:38.015 INFO:tasks.workunit.client.0.smithi007.stdout:9/932: creat d13/d74/f12e x:0 0 0 2024-09-21T04:35:38.019 INFO:tasks.workunit.client.0.smithi007.stdout:5/837: creat d2/d27/d42/da3/d23/d49/f10e x:0 0 0 2024-09-21T04:35:38.020 INFO:tasks.workunit.client.0.smithi007.stdout:5/838: stat d2/d27/d42/da3/d23/d49/d6f 0 2024-09-21T04:35:38.021 INFO:tasks.workunit.client.0.smithi007.stdout:5/839: readlink d2/d27/d42/da3/l17 0 2024-09-21T04:35:38.021 INFO:tasks.workunit.client.0.smithi007.stdout:9/933: rename d13/d18/d60/d6d/f6f to d13/d74/de1/d105/f12f 0 2024-09-21T04:35:38.024 INFO:tasks.workunit.client.0.smithi007.stdout:4/966: getdents d2/d3c 0 2024-09-21T04:35:38.025 INFO:tasks.workunit.client.0.smithi007.stdout:9/934: mkdir d13/d18/d80/d130 0 2024-09-21T04:35:38.027 INFO:tasks.workunit.client.0.smithi007.stdout:4/967: dread - d2/d17/d2e/d45/d5d/d8a/fce zero size 2024-09-21T04:35:38.029 INFO:tasks.workunit.client.0.smithi007.stdout:9/935: mkdir d13/d18/d60/d83/da9/d131 0 2024-09-21T04:35:38.031 INFO:tasks.workunit.client.0.smithi007.stdout:4/968: fdatasync d2/d17/d2e/d45/d5e/fb4 0 2024-09-21T04:35:38.037 INFO:tasks.workunit.client.0.smithi007.stdout:5/840: dread d2/d27/d3d/d56/f81 [0,4194304] 0 2024-09-21T04:35:38.042 INFO:tasks.workunit.client.0.smithi007.stdout:9/936: creat d13/d18/d60/d6d/d75/d9f/dc8/f132 x:0 0 0 2024-09-21T04:35:38.045 INFO:tasks.workunit.client.0.smithi007.stdout:5/841: creat d2/d27/d42/da3/d23/d49/d6f/ddb/def/f10f x:0 0 0 2024-09-21T04:35:38.046 INFO:tasks.workunit.client.0.smithi007.stdout:4/969: dread d2/d17/f51 [0,4194304] 0 2024-09-21T04:35:38.047 INFO:tasks.workunit.client.0.smithi007.stdout:9/937: mknod d13/d18/dcd/d104/c133 0 2024-09-21T04:35:38.048 INFO:tasks.workunit.client.0.smithi007.stdout:5/842: write d2/d27/d42/da3/d23/d49/f4a [726865,3864] 0 2024-09-21T04:35:38.051 INFO:tasks.workunit.client.0.smithi007.stdout:9/938: rmdir d13/d18/dcd/d104 39 2024-09-21T04:35:38.053 INFO:tasks.workunit.client.0.smithi007.stdout:9/939: write d13/f5e [688323,107697] 0 2024-09-21T04:35:38.055 INFO:tasks.workunit.client.0.smithi007.stdout:4/970: dwrite d2/d17/d2e/f71 [4194304,4194304] 0 2024-09-21T04:35:38.063 INFO:tasks.workunit.client.0.smithi007.stdout:9/940: mkdir d13/d18/d60/d6d/d75/d113/d134 0 2024-09-21T04:35:38.065 INFO:tasks.workunit.client.0.smithi007.stdout:4/971: link d2/d7/d64/fc8 d2/d7/dc5/dd6/f144 0 2024-09-21T04:35:38.070 INFO:tasks.workunit.client.0.smithi007.stdout:4/972: rmdir d2/d7/dc9 39 2024-09-21T04:35:38.071 INFO:tasks.workunit.client.0.smithi007.stdout:4/973: chown d2/d17/d2e/d45/d5e/ff5 20 1 2024-09-21T04:35:38.078 INFO:tasks.workunit.client.0.smithi007.stdout:9/941: rename d13/f4f to d13/d74/de1/dec/d120/f135 0 2024-09-21T04:35:38.082 INFO:tasks.workunit.client.0.smithi007.stdout:9/942: stat d13/d18/dcd/dd4/cf6 0 2024-09-21T04:35:38.084 INFO:tasks.workunit.client.0.smithi007.stdout:4/974: rename d2/d97/da5 to d2/d17/d18/d145 0 2024-09-21T04:35:38.084 INFO:tasks.workunit.client.0.smithi007.stdout:9/943: truncate d13/d18/daf/f10d 132933 0 2024-09-21T04:35:38.086 INFO:tasks.workunit.client.0.smithi007.stdout:4/975: creat d2/d17/d29/de9/f146 x:0 0 0 2024-09-21T04:35:38.087 INFO:tasks.workunit.client.0.smithi007.stdout:9/944: symlink d13/d18/l136 0 2024-09-21T04:35:38.089 INFO:tasks.workunit.client.0.smithi007.stdout:4/976: creat d2/d17/d2e/d45/d8e/f147 x:0 0 0 2024-09-21T04:35:38.091 INFO:tasks.workunit.client.0.smithi007.stdout:4/977: creat d2/d7/d64/f148 x:0 0 0 2024-09-21T04:35:38.094 INFO:tasks.workunit.client.0.smithi007.stdout:9/945: dwrite d13/d5a/f115 [0,4194304] 0 2024-09-21T04:35:38.095 INFO:tasks.workunit.client.0.smithi007.stdout:9/946: chown d13/d18/d60 57 1 2024-09-21T04:35:38.097 INFO:tasks.workunit.client.0.smithi007.stdout:4/978: dread d2/d7/dfb/f113 [0,4194304] 0 2024-09-21T04:35:38.099 INFO:tasks.workunit.client.0.smithi007.stdout:9/947: write d13/d74/de1/dec/d120/f135 [1058983,45782] 0 2024-09-21T04:35:38.100 INFO:tasks.workunit.client.0.smithi007.stdout:4/979: rmdir d2/d17/d2e/d45/d5d/d101 39 2024-09-21T04:35:38.100 INFO:tasks.workunit.client.0.smithi007.stdout:9/948: fdatasync d13/d18/fee 0 2024-09-21T04:35:38.102 INFO:tasks.workunit.client.0.smithi007.stdout:9/949: write d13/d74/f98 [418614,70181] 0 2024-09-21T04:35:38.102 INFO:tasks.workunit.client.0.smithi007.stdout:4/980: stat d2/d7/dc9/c138 0 2024-09-21T04:35:38.103 INFO:tasks.workunit.client.0.smithi007.stdout:4/981: chown d2/d17/d2e/f3e 22 1 2024-09-21T04:35:38.104 INFO:tasks.workunit.client.0.smithi007.stdout:4/982: chown d2/d7/d55/d61/c70 0 1 2024-09-21T04:35:38.105 INFO:tasks.workunit.client.0.smithi007.stdout:4/983: chown d2 56325002 1 2024-09-21T04:35:38.106 INFO:tasks.workunit.client.0.smithi007.stdout:9/950: mkdir d13/d74/de1/dec/d137 0 2024-09-21T04:35:38.106 INFO:tasks.workunit.client.0.smithi007.stdout:9/951: stat d13/d5a/f115 0 2024-09-21T04:35:38.107 INFO:tasks.workunit.client.0.smithi007.stdout:4/984: fdatasync d2/fe2 0 2024-09-21T04:35:38.111 INFO:tasks.workunit.client.0.smithi007.stdout:4/985: mknod d2/d5f/c149 0 2024-09-21T04:35:38.114 INFO:tasks.workunit.client.0.smithi007.stdout:9/952: rename d13/l1b to d13/d18/d60/d6d/d75/d9f/l138 0 2024-09-21T04:35:38.116 INFO:tasks.workunit.client.0.smithi007.stdout:4/986: rename d2/d7/d55/d61/c126 to d2/d17/d2e/d45/d5e/d90/c14a 0 2024-09-21T04:35:38.117 INFO:tasks.workunit.client.0.smithi007.stdout:4/987: chown d2/d5f/ff1 3 1 2024-09-21T04:35:38.118 INFO:tasks.workunit.client.0.smithi007.stdout:9/953: creat d13/d18/d60/d6d/d85/da1/daa/f139 x:0 0 0 2024-09-21T04:35:38.119 INFO:tasks.workunit.client.0.smithi007.stdout:4/988: readlink d2/d88/da8/db2/l116 0 2024-09-21T04:35:38.120 INFO:tasks.workunit.client.0.smithi007.stdout:4/989: chown d2/d7/dc9 15509519 1 2024-09-21T04:35:38.121 INFO:tasks.workunit.client.0.smithi007.stdout:9/954: dread d13/d18/f55 [0,4194304] 0 2024-09-21T04:35:38.122 INFO:tasks.workunit.client.0.smithi007.stdout:4/990: getdents d2/d3c 0 2024-09-21T04:35:38.123 INFO:tasks.workunit.client.0.smithi007.stdout:9/955: write d13/d18/d60/d83/da9/fb4 [1117517,61354] 0 2024-09-21T04:35:38.125 INFO:tasks.workunit.client.0.smithi007.stdout:4/991: mkdir d2/d17/d2e/d45/d5d/d8a/d14b 0 2024-09-21T04:35:38.126 INFO:tasks.workunit.client.0.smithi007.stdout:4/992: chown d2/d7/dfb 10636784 1 2024-09-21T04:35:38.127 INFO:tasks.workunit.client.0.smithi007.stdout:9/956: readlink d13/d18/d60/d6d/d75/d7f/l8c 0 2024-09-21T04:35:38.130 INFO:tasks.workunit.client.0.smithi007.stdout:9/957: creat d13/d18/dcd/f13a x:0 0 0 2024-09-21T04:35:38.131 INFO:tasks.workunit.client.0.smithi007.stdout:9/958: stat d13/l3e 0 2024-09-21T04:35:38.135 INFO:tasks.workunit.client.0.smithi007.stdout:4/993: write d2/d3c/f106 [8150171,79901] 0 2024-09-21T04:35:38.137 INFO:tasks.workunit.client.0.smithi007.stdout:9/959: dwrite d13/d74/f106 [0,4194304] 0 2024-09-21T04:35:38.142 INFO:tasks.workunit.client.0.smithi007.stdout:4/994: truncate d2/d7/d64/fc8 2785854 0 2024-09-21T04:35:38.145 INFO:tasks.workunit.client.0.smithi007.stdout:9/960: dwrite d13/d18/d60/d6d/d75/d9f/f126 [0,4194304] 0 2024-09-21T04:35:38.147 INFO:tasks.workunit.client.0.smithi007.stdout:9/961: read d13/f21 [6092932,99801] 0 2024-09-21T04:35:38.147 INFO:tasks.workunit.client.0.smithi007.stdout:4/995: unlink d2/d7/d64/db8/ce5 0 2024-09-21T04:35:38.162 INFO:tasks.workunit.client.0.smithi007.stdout:4/996: dread d2/d17/d2e/f33 [0,4194304] 0 2024-09-21T04:35:38.163 INFO:tasks.workunit.client.0.smithi007.stdout:9/962: truncate d13/d74/f98 402683 0 2024-09-21T04:35:38.167 INFO:tasks.workunit.client.0.smithi007.stdout:9/963: getdents d13/d18/d60/d83/da9/d131 0 2024-09-21T04:35:38.172 INFO:tasks.workunit.client.0.smithi007.stdout:9/964: dwrite d13/d18/f27 [0,4194304] 0 2024-09-21T04:35:38.177 INFO:tasks.workunit.client.0.smithi007.stdout:4/997: dread d2/d17/d29/f86 [0,4194304] 0 2024-09-21T04:35:38.186 INFO:tasks.workunit.client.0.smithi007.stdout:9/965: dread d13/d18/d60/f95 [0,4194304] 0 2024-09-21T04:35:38.187 INFO:tasks.workunit.client.0.smithi007.stdout:9/966: stat d13/d18/d60/d83/da9/c46 0 2024-09-21T04:35:38.190 INFO:tasks.workunit.client.0.smithi007.stdout:9/967: mkdir d13/d18/d60/d6d/d85/da1/daa/d13b 0 2024-09-21T04:35:38.191 INFO:tasks.workunit.client.0.smithi007.stdout:4/998: write d2/d17/d2e/d45/f8f [253155,116281] 0 2024-09-21T04:35:38.191 INFO:tasks.workunit.client.0.smithi007.stdout:9/968: chown d13/d18/daf 57888 1 2024-09-21T04:35:38.193 INFO:tasks.workunit.client.0.smithi007.stdout:9/969: read d13/d18/d60/d6d/fae [4009834,63852] 0 2024-09-21T04:35:38.194 INFO:tasks.workunit.client.0.smithi007.stdout:4/999: creat d2/d7/d64/db8/f14c x:0 0 0 2024-09-21T04:35:38.194 INFO:tasks.workunit.client.0.smithi007.stdout:9/970: chown d13/d18/daf/dc0 37013 1 2024-09-21T04:35:38.205 INFO:tasks.workunit.client.0.smithi007.stdout:9/971: dread d13/d18/f54 [0,4194304] 0 2024-09-21T04:35:38.218 INFO:tasks.workunit.client.0.smithi007.stdout:9/972: dread d13/d18/d60/d6d/d85/ff9 [0,4194304] 0 2024-09-21T04:35:38.311 INFO:tasks.workunit.client.0.smithi007.stdout:9/973: getdents d13/d18/d60 0 2024-09-21T04:35:38.316 INFO:tasks.workunit.client.0.smithi007.stdout:9/974: dwrite d13/d18/d60/d83/da9/f2d [0,4194304] 0 2024-09-21T04:35:38.317 INFO:tasks.workunit.client.0.smithi007.stdout:9/975: readlink d13/d18/l136 0 2024-09-21T04:35:38.320 INFO:tasks.workunit.client.0.smithi007.stdout:9/976: creat d13/d74/de1/d105/f13c x:0 0 0 2024-09-21T04:35:38.323 INFO:tasks.workunit.client.0.smithi007.stdout:9/977: creat d13/d74/ddb/d102/f13d x:0 0 0 2024-09-21T04:35:38.367 INFO:tasks.workunit.client.0.smithi007.stdout:9/978: rename d13/d18/d60/d83/f117 to d13/d18/dcd/d104/f13e 0 2024-09-21T04:35:38.368 INFO:tasks.workunit.client.0.smithi007.stdout:9/979: write d13/d74/f12e [699834,15073] 0 2024-09-21T04:35:38.371 INFO:tasks.workunit.client.0.smithi007.stdout:9/980: symlink d13/d18/d60/d83/da9/l13f 0 2024-09-21T04:35:38.372 INFO:tasks.workunit.client.0.smithi007.stdout:9/981: write d13/d18/d80/dea/f103 [1200514,70695] 0 2024-09-21T04:35:38.376 INFO:tasks.workunit.client.0.smithi007.stdout:9/982: dread d13/d18/d60/d83/da9/fef [0,4194304] 0 2024-09-21T04:35:38.383 INFO:tasks.workunit.client.0.smithi007.stdout:9/983: rmdir d13/d18/d60/d83/da9/d131 0 2024-09-21T04:35:38.389 INFO:tasks.workunit.client.0.smithi007.stdout:9/984: rename d13/d18/d60/d6d/d85/fd3 to d13/d18/d60/d6d/d75/f140 0 2024-09-21T04:35:38.462 INFO:tasks.workunit.client.0.smithi007.stdout:2/642: sync 2024-09-21T04:35:38.634 INFO:tasks.workunit.client.0.smithi007.stdout:0/753: sync 2024-09-21T04:35:38.642 INFO:tasks.workunit.client.0.smithi007.stdout:0/754: write d6/f38 [1490530,24143] 0 2024-09-21T04:35:38.645 INFO:tasks.workunit.client.0.smithi007.stdout:0/755: rmdir d6/d75/dc5/d47/d99 39 2024-09-21T04:35:38.647 INFO:tasks.workunit.client.0.smithi007.stdout:0/756: truncate d6/d75/dc5/d18/d27/fa1 929028 0 2024-09-21T04:35:38.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:38 smithi053 ceph-mon[35554]: pgmap v40: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 40 MiB/s rd, 90 MiB/s wr, 268 op/s 2024-09-21T04:35:38.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:38 smithi007 ceph-mon[27954]: pgmap v40: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 40 MiB/s rd, 90 MiB/s wr, 268 op/s 2024-09-21T04:35:38.772 INFO:tasks.workunit.client.0.smithi007.stdout:8/683: sync 2024-09-21T04:35:38.775 INFO:tasks.workunit.client.0.smithi007.stdout:8/684: fdatasync d0/f43 0 2024-09-21T04:35:38.776 INFO:tasks.workunit.client.0.smithi007.stdout:8/685: write d0/d3/d16/d93/d26/d4d/d5d/f86 [879684,18282] 0 2024-09-21T04:35:38.788 INFO:tasks.workunit.client.0.smithi007.stdout:8/686: dread d0/d3/d16/f99 [0,4194304] 0 2024-09-21T04:35:38.790 INFO:tasks.workunit.client.0.smithi007.stdout:8/687: unlink d0/d3/d16/d93/d26/d35/c8a 0 2024-09-21T04:35:38.793 INFO:tasks.workunit.client.0.smithi007.stdout:8/688: write d0/d3/d7c/fc6 [4556036,63154] 0 2024-09-21T04:35:38.795 INFO:tasks.workunit.client.0.smithi007.stdout:8/689: symlink d0/d3/d74/lc9 0 2024-09-21T04:35:38.798 INFO:tasks.workunit.client.0.smithi007.stdout:8/690: getdents d0/d3/daa 0 2024-09-21T04:35:38.803 INFO:tasks.workunit.client.0.smithi007.stdout:8/691: rename d0/d3/d16/l1e to d0/d3/d8/lca 0 2024-09-21T04:35:38.805 INFO:tasks.workunit.client.0.smithi007.stdout:8/692: mkdir d0/d3/d16/d93/d40/d92/dcb 0 2024-09-21T04:35:38.808 INFO:tasks.workunit.client.0.smithi007.stdout:8/693: creat d0/d3/d16/d93/d40/d9e/fcc x:0 0 0 2024-09-21T04:35:38.811 INFO:tasks.workunit.client.0.smithi007.stdout:8/694: creat d0/d3/d16/d93/d26/d4d/d5d/dba/fcd x:0 0 0 2024-09-21T04:35:38.821 INFO:tasks.workunit.client.0.smithi007.stdout:8/695: write d0/d3/d7c/f64 [4306615,47206] 0 2024-09-21T04:35:38.827 INFO:tasks.workunit.client.0.smithi007.stdout:8/696: dwrite d0/d3/d16/d93/f69 [0,4194304] 0 2024-09-21T04:35:38.829 INFO:tasks.workunit.client.0.smithi007.stdout:8/697: rmdir d0/d3/d16/d93/d26/d4d/d5d/dba 39 2024-09-21T04:35:38.840 INFO:tasks.workunit.client.0.smithi007.stdout:8/698: truncate d0/d3/d16/d93/d26/d4d/d5d/f86 302636 0 2024-09-21T04:35:38.846 INFO:tasks.workunit.client.0.smithi007.stdout:8/699: dwrite d0/d3/fb2 [0,4194304] 0 2024-09-21T04:35:38.853 INFO:tasks.workunit.client.0.smithi007.stdout:6/756: sync 2024-09-21T04:35:38.853 INFO:tasks.workunit.client.0.smithi007.stdout:7/832: sync 2024-09-21T04:35:38.858 INFO:tasks.workunit.client.0.smithi007.stdout:6/757: rmdir d3/d3e/d99/dbb/d95/dc9 39 2024-09-21T04:35:38.861 INFO:tasks.workunit.client.0.smithi007.stdout:7/833: fsync d3/d6/d10/d51/fb3 0 2024-09-21T04:35:38.863 INFO:tasks.workunit.client.0.smithi007.stdout:7/834: readlink d3/d6/d14/d20/d80/dca/lc0 0 2024-09-21T04:35:38.866 INFO:tasks.workunit.client.0.smithi007.stdout:7/835: write d3/d6/d85/dad/fce [4810050,3970] 0 2024-09-21T04:35:38.866 INFO:tasks.workunit.client.0.smithi007.stdout:8/700: dwrite d0/d3/d16/d93/d26/d4d/f91 [0,4194304] 0 2024-09-21T04:35:38.867 INFO:tasks.workunit.client.0.smithi007.stdout:6/758: write d3/d3b/f45 [884073,118967] 0 2024-09-21T04:35:38.877 INFO:tasks.workunit.client.0.smithi007.stdout:8/701: dwrite d0/d3/d16/d93/d26/d35/f87 [4194304,4194304] 0 2024-09-21T04:35:38.880 INFO:tasks.workunit.client.0.smithi007.stdout:8/702: read d0/d3/f53 [315836,71534] 0 2024-09-21T04:35:38.881 INFO:tasks.workunit.client.0.smithi007.stdout:8/703: readlink d0/d3/d16/d93/l1c 0 2024-09-21T04:35:38.883 INFO:tasks.workunit.client.0.smithi007.stdout:7/836: dwrite d3/d6/d40/faf [0,4194304] 0 2024-09-21T04:35:38.885 INFO:tasks.workunit.client.0.smithi007.stdout:8/704: write d0/d3/d16/f99 [2169665,73723] 0 2024-09-21T04:35:38.887 INFO:tasks.workunit.client.0.smithi007.stdout:6/759: dread d3/de/d12/f1f [0,4194304] 0 2024-09-21T04:35:38.888 INFO:tasks.workunit.client.0.smithi007.stdout:8/705: dread d0/fa6 [0,4194304] 0 2024-09-21T04:35:38.891 INFO:tasks.workunit.client.0.smithi007.stdout:7/837: creat d3/d6/d14/d20/d80/db2/db6/f126 x:0 0 0 2024-09-21T04:35:38.893 INFO:tasks.workunit.client.0.smithi007.stdout:7/838: write d3/d6/d85/dad/d103/f124 [573407,108695] 0 2024-09-21T04:35:38.896 INFO:tasks.workunit.client.0.smithi007.stdout:6/760: symlink d3/de/d12/d19/dd8/lfc 0 2024-09-21T04:35:38.897 INFO:tasks.workunit.client.0.smithi007.stdout:7/839: symlink d3/d6/d13/d4b/l127 0 2024-09-21T04:35:38.898 INFO:tasks.workunit.client.0.smithi007.stdout:7/840: dread - d3/d6/d40/f116 zero size 2024-09-21T04:35:38.900 INFO:tasks.workunit.client.0.smithi007.stdout:7/841: write d3/d6/d85/f50 [1607593,112336] 0 2024-09-21T04:35:38.901 INFO:tasks.workunit.client.0.smithi007.stdout:6/761: symlink d3/d3b/dd3/d60/d77/dcb/d78/lfd 0 2024-09-21T04:35:38.903 INFO:tasks.workunit.client.0.smithi007.stdout:2/643: sync 2024-09-21T04:35:38.903 INFO:tasks.workunit.client.0.smithi007.stdout:3/886: sync 2024-09-21T04:35:38.903 INFO:tasks.workunit.client.0.smithi007.stdout:5/843: sync 2024-09-21T04:35:38.903 INFO:tasks.workunit.client.0.smithi007.stdout:9/985: sync 2024-09-21T04:35:38.903 INFO:tasks.workunit.client.0.smithi007.stdout:0/757: sync 2024-09-21T04:35:38.905 INFO:tasks.workunit.client.0.smithi007.stdout:0/758: stat d6/d75/dc5 0 2024-09-21T04:35:38.905 INFO:tasks.workunit.client.0.smithi007.stdout:2/644: dread - d0/d11/d31/d7b/f94 zero size 2024-09-21T04:35:38.906 INFO:tasks.workunit.client.0.smithi007.stdout:7/842: creat d3/d6/d14/d20/d80/db2/d10b/f128 x:0 0 0 2024-09-21T04:35:38.907 INFO:tasks.workunit.client.0.smithi007.stdout:8/706: dread d0/d3/d74/f66 [0,4194304] 0 2024-09-21T04:35:38.908 INFO:tasks.workunit.client.0.smithi007.stdout:8/707: fdatasync d0/d3/d74/fac 0 2024-09-21T04:35:38.909 INFO:tasks.workunit.client.0.smithi007.stdout:5/844: mkdir d2/d27/d42/d53/dcf/d110 0 2024-09-21T04:35:38.909 INFO:tasks.workunit.client.0.smithi007.stdout:8/708: chown d0/d3/daa 28514666 1 2024-09-21T04:35:38.910 INFO:tasks.workunit.client.0.smithi007.stdout:5/845: chown d2/d27/dee/c10c 14109 1 2024-09-21T04:35:38.912 INFO:tasks.workunit.client.0.smithi007.stdout:8/709: dread d0/d3/daa/f76 [0,4194304] 0 2024-09-21T04:35:38.914 INFO:tasks.workunit.client.0.smithi007.stdout:0/759: truncate d6/d75/dc5/d18/f5d 2009022 0 2024-09-21T04:35:38.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/887: creat dc/d22/d29/d92/db9/dfa/f117 x:0 0 0 2024-09-21T04:35:38.917 INFO:tasks.workunit.client.0.smithi007.stdout:7/843: mkdir d3/d6/d14/d20/d80/d129 0 2024-09-21T04:35:38.919 INFO:tasks.workunit.client.0.smithi007.stdout:7/844: truncate d3/d6/d10/d4f/fdb 1586830 0 2024-09-21T04:35:38.920 INFO:tasks.workunit.client.0.smithi007.stdout:5/846: rename d2/d27/d42/da3/d23/d49/d44 to d2/d27/d42/da3/d23/d49/dd4/dd5/d111 0 2024-09-21T04:35:38.920 INFO:tasks.workunit.client.0.smithi007.stdout:7/845: read - d3/d6/d14/d20/d80/f114 zero size 2024-09-21T04:35:38.922 INFO:tasks.workunit.client.0.smithi007.stdout:8/710: creat d0/d3/d16/d93/d5e/fce x:0 0 0 2024-09-21T04:35:38.924 INFO:tasks.workunit.client.0.smithi007.stdout:8/711: read - d0/d3/d7c/f59 zero size 2024-09-21T04:35:38.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/760: creat d6/d75/dc5/d57/d9d/daf/f118 x:0 0 0 2024-09-21T04:35:38.926 INFO:tasks.workunit.client.0.smithi007.stdout:5/847: dread d2/d27/d42/da3/d23/d49/dd4/f101 [0,4194304] 0 2024-09-21T04:35:38.927 INFO:tasks.workunit.client.0.smithi007.stdout:3/888: dread dc/f21 [0,4194304] 0 2024-09-21T04:35:38.927 INFO:tasks.workunit.client.0.smithi007.stdout:5/848: chown d2/d27/d3d/cfe 142819 1 2024-09-21T04:35:38.928 INFO:tasks.workunit.client.0.smithi007.stdout:6/762: write d3/de/f4f [2433917,61154] 0 2024-09-21T04:35:38.930 INFO:tasks.workunit.client.0.smithi007.stdout:9/986: write d13/d18/f6e [1197624,110066] 0 2024-09-21T04:35:38.937 INFO:tasks.workunit.client.0.smithi007.stdout:3/889: symlink dc/d18/d41/d96/l118 0 2024-09-21T04:35:38.941 INFO:tasks.workunit.client.0.smithi007.stdout:6/763: dread d3/de/d12/f21 [0,4194304] 0 2024-09-21T04:35:38.946 INFO:tasks.workunit.client.0.smithi007.stdout:7/846: unlink d3/d6/d14/d20/d80/db2/lf7 0 2024-09-21T04:35:38.948 INFO:tasks.workunit.client.0.smithi007.stdout:0/761: chown d6/d75/dc5/d18/d2e/d3f/fbb 517689 1 2024-09-21T04:35:38.951 INFO:tasks.workunit.client.0.smithi007.stdout:3/890: rename dc/d22/d29/d32/d4d/d84/dd0/da6/fc4 to dc/d22/d29/d92/f119 0 2024-09-21T04:35:38.952 INFO:tasks.workunit.client.0.smithi007.stdout:7/847: dwrite d3/d6/d13/f2c [0,4194304] 0 2024-09-21T04:35:38.953 INFO:tasks.workunit.client.0.smithi007.stdout:3/891: chown dc/d22/d29/d92/db9/dfa/f117 46997958 1 2024-09-21T04:35:38.954 INFO:tasks.workunit.client.0.smithi007.stdout:3/892: chown dc/d22/d29/d92/db9 87517021 1 2024-09-21T04:35:38.961 INFO:tasks.workunit.client.0.smithi007.stdout:5/849: dwrite d2/d27/d42/da3/d23/d5d/fb6 [0,4194304] 0 2024-09-21T04:35:38.971 INFO:tasks.workunit.client.0.smithi007.stdout:6/764: dwrite d3/d3b/dd3/d60/d77/dcb/d78/f84 [0,4194304] 0 2024-09-21T04:35:38.973 INFO:tasks.workunit.client.0.smithi007.stdout:7/848: mkdir d3/d6/d14/d20/d80/dca/d113/d12a 0 2024-09-21T04:35:38.978 INFO:tasks.workunit.client.0.smithi007.stdout:9/987: getdents d13/d18/d80 0 2024-09-21T04:35:38.980 INFO:tasks.workunit.client.0.smithi007.stdout:3/893: dread dc/d22/d29/d32/d4d/f8b [0,4194304] 0 2024-09-21T04:35:38.981 INFO:tasks.workunit.client.0.smithi007.stdout:6/765: dwrite d3/d3b/dd3/d60/dd2/fe1 [0,4194304] 0 2024-09-21T04:35:38.985 INFO:tasks.workunit.client.0.smithi007.stdout:5/850: unlink d2/d27/d42/da3/d23/d5d/dc6/le3 0 2024-09-21T04:35:38.987 INFO:tasks.workunit.client.0.smithi007.stdout:7/849: creat d3/d6/d85/dad/f12b x:0 0 0 2024-09-21T04:35:38.987 INFO:tasks.workunit.client.0.smithi007.stdout:9/988: rmdir d13/d18 39 2024-09-21T04:35:38.987 INFO:tasks.workunit.client.0.smithi007.stdout:0/762: getdents d6/d75/dc5/d47/d8a/df4 0 2024-09-21T04:35:38.989 INFO:tasks.workunit.client.0.smithi007.stdout:6/766: chown d3/d3b/dd3/c47 57 1 2024-09-21T04:35:38.992 INFO:tasks.workunit.client.0.smithi007.stdout:5/851: rename d2/d27/f87 to d2/d27/d3d/d9e/f112 0 2024-09-21T04:35:38.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/894: dwrite dc/d22/d29/d32/d4d/d84/dd0/da6/fec [0,4194304] 0 2024-09-21T04:35:38.994 INFO:tasks.workunit.client.0.smithi007.stdout:9/989: chown d13/d18/l136 1011 1 2024-09-21T04:35:38.998 INFO:tasks.workunit.client.0.smithi007.stdout:6/767: chown d3/d3e/d99/dbb/cad 1601927 1 2024-09-21T04:35:38.998 INFO:tasks.workunit.client.0.smithi007.stdout:3/895: stat dc/d22/l3a 0 2024-09-21T04:35:38.999 INFO:tasks.workunit.client.0.smithi007.stdout:5/852: mknod d2/d27/d42/da3/d23/d49/d6f/ddb/def/c113 0 2024-09-21T04:35:39.001 INFO:tasks.workunit.client.0.smithi007.stdout:7/850: stat d3/d6/d2e/d63/d6a/d81/fa0 0 2024-09-21T04:35:39.002 INFO:tasks.workunit.client.0.smithi007.stdout:0/763: mknod d6/d75/dc5/d47/d99/c119 0 2024-09-21T04:35:39.004 INFO:tasks.workunit.client.0.smithi007.stdout:7/851: read d3/d6/d14/d20/d58/f87 [1988251,80514] 0 2024-09-21T04:35:39.008 INFO:tasks.workunit.client.0.smithi007.stdout:9/990: rename d13/d74/f98 to d13/d74/de1/f141 0 2024-09-21T04:35:39.010 INFO:tasks.workunit.client.0.smithi007.stdout:0/764: dread - d6/d75/dc5/d47/d99/fae zero size 2024-09-21T04:35:39.012 INFO:tasks.workunit.client.0.smithi007.stdout:7/852: dwrite d3/d6/d85/de7/d11b/d66/f94 [0,4194304] 0 2024-09-21T04:35:39.023 INFO:tasks.workunit.client.0.smithi007.stdout:5/853: rename d2/d27/d42/da3/d1d/d20/l107 to d2/d27/d42/d53/dcf/l114 0 2024-09-21T04:35:39.024 INFO:tasks.workunit.client.0.smithi007.stdout:9/991: creat d13/d18/dcd/d104/f142 x:0 0 0 2024-09-21T04:35:39.026 INFO:tasks.workunit.client.0.smithi007.stdout:9/992: readlink d13/d18/d60/d83/l96 0 2024-09-21T04:35:39.027 INFO:tasks.workunit.client.0.smithi007.stdout:3/896: dwrite dc/d22/d29/d32/d4d/d84/dd0/da6/de3/fef [0,4194304] 0 2024-09-21T04:35:39.028 INFO:tasks.workunit.client.0.smithi007.stdout:6/768: creat d3/de/ffe x:0 0 0 2024-09-21T04:35:39.031 INFO:tasks.workunit.client.0.smithi007.stdout:0/765: fdatasync d6/d75/dc5/d18/d27/f9b 0 2024-09-21T04:35:39.033 INFO:tasks.workunit.client.0.smithi007.stdout:5/854: rename d2/d27/d3d/d56/f81 to d2/d27/dee/f115 0 2024-09-21T04:35:39.038 INFO:tasks.workunit.client.0.smithi007.stdout:9/993: rmdir d13/d18/d60/d6d/d75/d113 39 2024-09-21T04:35:39.038 INFO:tasks.workunit.client.0.smithi007.stdout:9/994: fdatasync d13/d18/f27 0 2024-09-21T04:35:39.039 INFO:tasks.workunit.client.0.smithi007.stdout:7/853: dread d3/d6/d2e/f38 [0,4194304] 0 2024-09-21T04:35:39.040 INFO:tasks.workunit.client.0.smithi007.stdout:9/995: write d13/d18/dcd/fe7 [705237,28061] 0 2024-09-21T04:35:39.040 INFO:tasks.workunit.client.0.smithi007.stdout:6/769: mknod d3/de/d12/d19/da3/cff 0 2024-09-21T04:35:39.042 INFO:tasks.workunit.client.0.smithi007.stdout:6/770: read - d3/de/ffe zero size 2024-09-21T04:35:39.042 INFO:tasks.workunit.client.0.smithi007.stdout:7/854: chown d3/d6/d2e/d63/d6a/d81 33166309 1 2024-09-21T04:35:39.045 INFO:tasks.workunit.client.0.smithi007.stdout:5/855: creat d2/d27/d42/da3/d23/d49/dd4/de9/f116 x:0 0 0 2024-09-21T04:35:39.047 INFO:tasks.workunit.client.0.smithi007.stdout:9/996: creat d13/d18/daf/dc0/f143 x:0 0 0 2024-09-21T04:35:39.048 INFO:tasks.workunit.client.0.smithi007.stdout:7/855: chown d3/d6/d2e/f6d 0 1 2024-09-21T04:35:39.050 INFO:tasks.workunit.client.0.smithi007.stdout:6/771: truncate d3/de/d12/d19/f1c 2474337 0 2024-09-21T04:35:39.055 INFO:tasks.workunit.client.0.smithi007.stdout:3/897: dwrite dc/d18/fab [0,4194304] 0 2024-09-21T04:35:39.055 INFO:tasks.workunit.client.0.smithi007.stdout:0/766: rmdir d6/d75/dc5/d18/d2e/d116 0 2024-09-21T04:35:39.059 INFO:tasks.workunit.client.0.smithi007.stdout:6/772: symlink d3/d94/l100 0 2024-09-21T04:35:39.063 INFO:tasks.workunit.client.0.smithi007.stdout:5/856: write d2/d27/d42/da3/f16 [5454356,248] 0 2024-09-21T04:35:39.064 INFO:tasks.workunit.client.0.smithi007.stdout:7/856: write d3/d6/d13/f47 [989421,73326] 0 2024-09-21T04:35:39.066 INFO:tasks.workunit.client.0.smithi007.stdout:9/997: link d13/d18/d60/d6d/d85/da1/daa/f11f d13/d18/d60/d6d/d75/d9f/dc8/f144 0 2024-09-21T04:35:39.068 INFO:tasks.workunit.client.0.smithi007.stdout:0/767: fsync d6/d75/dc5/d18/d2e/f8b 0 2024-09-21T04:35:39.069 INFO:tasks.workunit.client.0.smithi007.stdout:7/857: truncate d3/d6/d14/d20/d80/dca/ffa 591740 0 2024-09-21T04:35:39.071 INFO:tasks.workunit.client.0.smithi007.stdout:3/898: mkdir dc/d22/d29/d32/d4d/d7d/d9a/d11a 0 2024-09-21T04:35:39.071 INFO:tasks.workunit.client.0.smithi007.stdout:6/773: truncate d3/de/d12/d19/f64 407649 0 2024-09-21T04:35:39.075 INFO:tasks.workunit.client.0.smithi007.stdout:0/768: fdatasync d6/d75/dc9/ff6 0 2024-09-21T04:35:39.084 INFO:tasks.workunit.client.0.smithi007.stdout:7/858: dread d3/d6/d2e/d63/d6a/d81/fd0 [0,4194304] 0 2024-09-21T04:35:39.086 INFO:tasks.workunit.client.0.smithi007.stdout:7/859: fdatasync d3/d6/d85/de7/d11b/d101/f10c 0 2024-09-21T04:35:39.087 INFO:tasks.workunit.client.0.smithi007.stdout:6/774: write d3/de/d12/f1f [1956080,125377] 0 2024-09-21T04:35:39.087 INFO:tasks.workunit.client.0.smithi007.stdout:7/860: stat d3/d6/d10/d51/fdc 0 2024-09-21T04:35:39.088 INFO:tasks.workunit.client.0.smithi007.stdout:0/769: creat d6/d75/dc5/d47/d99/f11a x:0 0 0 2024-09-21T04:35:39.089 INFO:tasks.workunit.client.0.smithi007.stdout:5/857: truncate d2/d27/d42/da3/d23/d49/dd4/dd5/d111/d54/f59 65971 0 2024-09-21T04:35:39.095 INFO:tasks.workunit.client.0.smithi007.stdout:3/899: dread dc/f78 [0,4194304] 0 2024-09-21T04:35:39.095 INFO:tasks.workunit.client.0.smithi007.stdout:7/861: read d3/d6/d2e/d63/d6a/f70 [3149607,109639] 0 2024-09-21T04:35:39.097 INFO:tasks.workunit.client.0.smithi007.stdout:6/775: getdents d3/d94/df9 0 2024-09-21T04:35:39.105 INFO:tasks.workunit.client.0.smithi007.stdout:0/770: rename d6/d75/dc5/d18/d39/fe0 to d6/d91/da3/f11b 0 2024-09-21T04:35:39.107 INFO:tasks.workunit.client.0.smithi007.stdout:3/900: dread dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/f90 [0,4194304] 0 2024-09-21T04:35:39.108 INFO:tasks.workunit.client.0.smithi007.stdout:5/858: dread d2/d27/d42/da3/d23/d49/dd4/dd5/d111/d54/f59 [0,4194304] 0 2024-09-21T04:35:39.109 INFO:tasks.workunit.client.0.smithi007.stdout:3/901: chown dc/d22/d29/d32/d4d/d84/dd0/c95 305 1 2024-09-21T04:35:39.115 INFO:tasks.workunit.client.0.smithi007.stdout:0/771: creat d6/d91/da3/dbf/dec/f11c x:0 0 0 2024-09-21T04:35:39.116 INFO:tasks.workunit.client.0.smithi007.stdout:5/859: dwrite d2/d1e/f5b [0,4194304] 0 2024-09-21T04:35:39.118 INFO:tasks.workunit.client.0.smithi007.stdout:7/862: creat d3/d6/d40/f12c x:0 0 0 2024-09-21T04:35:39.131 INFO:tasks.workunit.client.0.smithi007.stdout:5/860: creat d2/d27/d42/da3/d28/de2/f117 x:0 0 0 2024-09-21T04:35:39.132 INFO:tasks.workunit.client.0.smithi007.stdout:6/776: dwrite d3/d3b/dd3/d60/d77/dcb/d78/f73 [0,4194304] 0 2024-09-21T04:35:39.136 INFO:tasks.workunit.client.0.smithi007.stdout:7/863: rename d3/d6/d2e/d63/f73 to d3/d6/d2e/d63/f12d 0 2024-09-21T04:35:39.137 INFO:tasks.workunit.client.0.smithi007.stdout:3/902: getdents dc/d22/d29/d32/d4d/d7d 0 2024-09-21T04:35:39.138 INFO:tasks.workunit.client.0.smithi007.stdout:0/772: dread d6/d75/dc5/d52/fb8 [0,4194304] 0 2024-09-21T04:35:39.140 INFO:tasks.workunit.client.0.smithi007.stdout:5/861: mknod d2/d27/d42/da3/d1d/c118 0 2024-09-21T04:35:39.142 INFO:tasks.workunit.client.0.smithi007.stdout:3/903: unlink dc/d22/d29/d32/fe5 0 2024-09-21T04:35:39.143 INFO:tasks.workunit.client.0.smithi007.stdout:3/904: fsync dc/d22/d29/d32/fa3 0 2024-09-21T04:35:39.144 INFO:tasks.workunit.client.0.smithi007.stdout:3/905: chown dc/d18/f45 153615 1 2024-09-21T04:35:39.146 INFO:tasks.workunit.client.0.smithi007.stdout:5/862: symlink d2/d1e/l119 0 2024-09-21T04:35:39.149 INFO:tasks.workunit.client.0.smithi007.stdout:0/773: dread d6/d75/dc5/d52/f78 [0,4194304] 0 2024-09-21T04:35:39.149 INFO:tasks.workunit.client.0.smithi007.stdout:3/906: fdatasync dc/d22/d29/d32/d4d/d84/dd0/da6/de3/f97 0 2024-09-21T04:35:39.151 INFO:tasks.workunit.client.0.smithi007.stdout:0/774: write d6/fd4 [1631987,77883] 0 2024-09-21T04:35:39.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/863: creat d2/d27/f11a x:0 0 0 2024-09-21T04:35:39.154 INFO:tasks.workunit.client.0.smithi007.stdout:6/777: dwrite d3/d3b/dd3/d60/d77/f63 [4194304,4194304] 0 2024-09-21T04:35:39.158 INFO:tasks.workunit.client.0.smithi007.stdout:0/775: write d6/d75/ffd [1026194,31976] 0 2024-09-21T04:35:39.161 INFO:tasks.workunit.client.0.smithi007.stdout:6/778: creat d3/d94/f101 x:0 0 0 2024-09-21T04:35:39.162 INFO:tasks.workunit.client.0.smithi007.stdout:5/864: rename d2/d27/d42/da3/fe1 to d2/d27/d42/da3/d23/d49/dd4/de9/f11b 0 2024-09-21T04:35:39.164 INFO:tasks.workunit.client.0.smithi007.stdout:3/907: truncate dc/d18/d26/f5f 2181963 0 2024-09-21T04:35:39.165 INFO:tasks.workunit.client.0.smithi007.stdout:0/776: creat d6/d75/dc5/d47/df0/f11d x:0 0 0 2024-09-21T04:35:39.167 INFO:tasks.workunit.client.0.smithi007.stdout:0/777: readlink d6/d75/dc5/d18/d2e/d3f/l10b 0 2024-09-21T04:35:39.168 INFO:tasks.workunit.client.0.smithi007.stdout:3/908: dread dc/f78 [0,4194304] 0 2024-09-21T04:35:39.168 INFO:tasks.workunit.client.0.smithi007.stdout:5/865: rename d2/d27/d42/d53/daf/dcd to d2/d27/d42/da3/d1d/d20/d11c 0 2024-09-21T04:35:39.170 INFO:tasks.workunit.client.0.smithi007.stdout:3/909: write dc/d22/d29/d32/d4d/f8e [377396,19507] 0 2024-09-21T04:35:39.171 INFO:tasks.workunit.client.0.smithi007.stdout:6/779: mkdir d3/d3e/d99/dbb/d102 0 2024-09-21T04:35:39.173 INFO:tasks.workunit.client.0.smithi007.stdout:6/780: read d3/d3e/f55 [985318,41866] 0 2024-09-21T04:35:39.174 INFO:tasks.workunit.client.0.smithi007.stdout:0/778: truncate d6/d75/dc5/d18/d27/f97 2533804 0 2024-09-21T04:35:39.183 INFO:tasks.workunit.client.0.smithi007.stdout:6/781: creat d3/de/d12/d19/da3/f103 x:0 0 0 2024-09-21T04:35:39.186 INFO:tasks.workunit.client.0.smithi007.stdout:3/910: unlink dc/d22/d29/d32/f77 0 2024-09-21T04:35:39.191 INFO:tasks.workunit.client.0.smithi007.stdout:3/911: mkdir dc/d22/d29/d92/db9/d11b 0 2024-09-21T04:35:39.193 INFO:tasks.workunit.client.0.smithi007.stdout:0/779: truncate d6/d75/dc5/d47/d8a/fc4 3426003 0 2024-09-21T04:35:39.193 INFO:tasks.workunit.client.0.smithi007.stdout:6/782: mkdir d3/de/d12/d19/da3/de3/d104 0 2024-09-21T04:35:39.194 INFO:tasks.workunit.client.0.smithi007.stdout:5/866: getdents d2/d27/d42/d53/daf 0 2024-09-21T04:35:39.199 INFO:tasks.workunit.client.0.smithi007.stdout:3/912: fdatasync dc/d22/d29/f55 0 2024-09-21T04:35:39.201 INFO:tasks.workunit.client.0.smithi007.stdout:3/913: truncate dc/d22/d29/d92/db9/dfa/f117 137308 0 2024-09-21T04:35:39.206 INFO:tasks.workunit.client.0.smithi007.stdout:6/783: write d3/de/f1a [616167,45448] 0 2024-09-21T04:35:39.206 INFO:tasks.workunit.client.0.smithi007.stdout:6/784: stat d3/de/d12/f21 0 2024-09-21T04:35:39.209 INFO:tasks.workunit.client.0.smithi007.stdout:0/780: write d6/d91/da3/dbf/fc3 [469026,127651] 0 2024-09-21T04:35:39.210 INFO:tasks.workunit.client.0.smithi007.stdout:0/781: readlink d6/d75/dc5/d47/d8a/d9c/l112 0 2024-09-21T04:35:39.212 INFO:tasks.workunit.client.0.smithi007.stdout:5/867: write d2/d27/d42/da3/d23/d49/dd4/de9/f11b [192777,111984] 0 2024-09-21T04:35:39.216 INFO:tasks.workunit.client.0.smithi007.stdout:0/782: chown d6/d75/dc5/d18/d2e/d60/l4f 49982383 1 2024-09-21T04:35:39.218 INFO:tasks.workunit.client.0.smithi007.stdout:3/914: creat dc/d22/d29/d32/f11c x:0 0 0 2024-09-21T04:35:39.218 INFO:tasks.workunit.client.0.smithi007.stdout:6/785: write d3/de/d58/fc8 [3125276,111325] 0 2024-09-21T04:35:39.220 INFO:tasks.workunit.client.0.smithi007.stdout:3/915: write dc/d18/f7b [436879,16042] 0 2024-09-21T04:35:39.226 INFO:tasks.workunit.client.0.smithi007.stdout:0/783: symlink d6/d75/dc5/d47/l11e 0 2024-09-21T04:35:39.226 INFO:tasks.workunit.client.0.smithi007.stdout:6/786: write d3/d94/fc4 [1679776,11961] 0 2024-09-21T04:35:39.229 INFO:tasks.workunit.client.0.smithi007.stdout:3/916: truncate dc/d22/d29/d32/d4d/fa0 1783572 0 2024-09-21T04:35:39.230 INFO:tasks.workunit.client.0.smithi007.stdout:5/868: dwrite d2/d27/d3d/d9b/fe8 [0,4194304] 0 2024-09-21T04:35:39.235 INFO:tasks.workunit.client.0.smithi007.stdout:3/917: creat dc/d22/d29/d92/db9/dfa/f11d x:0 0 0 2024-09-21T04:35:39.236 INFO:tasks.workunit.client.0.smithi007.stdout:5/869: dread d2/d27/d42/da3/d23/d5d/fb6 [0,4194304] 0 2024-09-21T04:35:39.239 INFO:tasks.workunit.client.0.smithi007.stdout:0/784: rename d6/d75/dc5/d18/d2e/d3f/l46 to d6/d75/l11f 0 2024-09-21T04:35:39.240 INFO:tasks.workunit.client.0.smithi007.stdout:0/785: chown d6/d75/dc5/d18/fab 235013609 1 2024-09-21T04:35:39.243 INFO:tasks.workunit.client.0.smithi007.stdout:6/787: write d3/d3b/dd3/d60/d98/faf [217199,34746] 0 2024-09-21T04:35:39.248 INFO:tasks.workunit.client.0.smithi007.stdout:6/788: mknod d3/d3e/d99/dbb/df1/c105 0 2024-09-21T04:35:39.250 INFO:tasks.workunit.client.0.smithi007.stdout:6/789: dread - d3/de/ffe zero size 2024-09-21T04:35:39.250 INFO:tasks.workunit.client.0.smithi007.stdout:0/786: fdatasync d6/d75/dc5/d18/d27/f97 0 2024-09-21T04:35:39.251 INFO:tasks.workunit.client.0.smithi007.stdout:3/918: rmdir dc/d22/d29/d32/d4d/d7d/d9a/d11a 0 2024-09-21T04:35:39.252 INFO:tasks.workunit.client.0.smithi007.stdout:6/790: write d3/de/f4f [2173384,30] 0 2024-09-21T04:35:39.253 INFO:tasks.workunit.client.0.smithi007.stdout:5/870: dwrite d2/d27/d3d/d9e/fd2 [0,4194304] 0 2024-09-21T04:35:39.258 INFO:tasks.workunit.client.0.smithi007.stdout:0/787: mknod d6/d75/dc5/d18/d39/c120 0 2024-09-21T04:35:39.262 INFO:tasks.workunit.client.0.smithi007.stdout:5/871: creat d2/d27/d42/da3/d23/d49/dd4/dd5/d111/f11d x:0 0 0 2024-09-21T04:35:39.263 INFO:tasks.workunit.client.0.smithi007.stdout:0/788: creat d6/d75/dc5/d57/d9d/f121 x:0 0 0 2024-09-21T04:35:39.264 INFO:tasks.workunit.client.0.smithi007.stdout:3/919: dread dc/d22/d29/d32/d4d/d84/dd0/da6/f60 [0,4194304] 0 2024-09-21T04:35:39.265 INFO:tasks.workunit.client.0.smithi007.stdout:6/791: dread d3/de/f6a [0,4194304] 0 2024-09-21T04:35:39.265 INFO:tasks.workunit.client.0.smithi007.stdout:0/789: stat d6/d75/dc5/d47/d8a/d9c/cd8 0 2024-09-21T04:35:39.266 INFO:tasks.workunit.client.0.smithi007.stdout:6/792: stat d3/de/fed 0 2024-09-21T04:35:39.269 INFO:tasks.workunit.client.0.smithi007.stdout:6/793: chown d3/de/d12/l52 10 1 2024-09-21T04:35:39.269 INFO:tasks.workunit.client.0.smithi007.stdout:3/920: mknod dc/d22/d29/dac/c11e 0 2024-09-21T04:35:39.271 INFO:tasks.workunit.client.0.smithi007.stdout:6/794: chown d3/d3b/dd3/d60/d77/dcb/d78/fb0 614 1 2024-09-21T04:35:39.271 INFO:tasks.workunit.client.0.smithi007.stdout:6/795: stat d3/de/d12/d19/f1c 0 2024-09-21T04:35:39.272 INFO:tasks.workunit.client.0.smithi007.stdout:3/921: mknod dc/d18/d41/d96/c11f 0 2024-09-21T04:35:39.274 INFO:tasks.workunit.client.0.smithi007.stdout:0/790: rename d6/d75/dc5/d18/d27/f97 to d6/d75/dc5/f122 0 2024-09-21T04:35:39.275 INFO:tasks.workunit.client.0.smithi007.stdout:6/796: creat d3/de/d12/d19/dd8/f106 x:0 0 0 2024-09-21T04:35:39.276 INFO:tasks.workunit.client.0.smithi007.stdout:6/797: chown d3/d3b/f45 127 1 2024-09-21T04:35:39.277 INFO:tasks.workunit.client.0.smithi007.stdout:0/791: mknod d6/d75/dc5/d57/c123 0 2024-09-21T04:35:39.279 INFO:tasks.workunit.client.0.smithi007.stdout:5/872: truncate d2/d1e/f24 5253794 0 2024-09-21T04:35:39.280 INFO:tasks.workunit.client.0.smithi007.stdout:6/798: mkdir d3/de/dba/d107 0 2024-09-21T04:35:39.281 INFO:tasks.workunit.client.0.smithi007.stdout:6/799: write d3/d94/fc2 [461411,82709] 0 2024-09-21T04:35:39.285 INFO:tasks.workunit.client.0.smithi007.stdout:5/873: dread - d2/d27/d42/da3/d28/fa5 zero size 2024-09-21T04:35:39.292 INFO:tasks.workunit.client.0.smithi007.stdout:6/800: write d3/d3b/dd3/d60/f93 [3851771,37090] 0 2024-09-21T04:35:39.292 INFO:tasks.workunit.client.0.smithi007.stdout:3/922: dwrite dc/d22/d29/d92/db9/f10a [0,4194304] 0 2024-09-21T04:35:39.294 INFO:tasks.workunit.client.0.smithi007.stdout:5/874: creat d2/d27/d42/da3/d23/d5d/dc6/f11e x:0 0 0 2024-09-21T04:35:39.295 INFO:tasks.workunit.client.0.smithi007.stdout:0/792: dread d6/d75/dc5/d47/d8a/fc4 [0,4194304] 0 2024-09-21T04:35:39.306 INFO:tasks.workunit.client.0.smithi007.stdout:0/793: creat d6/d91/da3/dbf/f124 x:0 0 0 2024-09-21T04:35:39.309 INFO:tasks.workunit.client.0.smithi007.stdout:3/923: write dc/d22/d29/d32/d4d/d7d/d9a/fc5 [1018959,89713] 0 2024-09-21T04:35:39.316 INFO:tasks.workunit.client.0.smithi007.stdout:6/801: getdents d3/d8e 0 2024-09-21T04:35:39.318 INFO:tasks.workunit.client.0.smithi007.stdout:3/924: mkdir dc/d22/d29/d32/d4d/d84/dbb/d120 0 2024-09-21T04:35:39.319 INFO:tasks.workunit.client.0.smithi007.stdout:5/875: dwrite d2/d27/d3d/d9e/f112 [0,4194304] 0 2024-09-21T04:35:39.324 INFO:tasks.workunit.client.0.smithi007.stdout:0/794: dwrite d6/d75/dc5/d52/f78 [0,4194304] 0 2024-09-21T04:35:39.329 INFO:tasks.workunit.client.0.smithi007.stdout:3/925: rename dc/d22/d29/d32/d69/d6d/f30 to dc/d22/d29/d32/d69/d6d/f121 0 2024-09-21T04:35:39.329 INFO:tasks.workunit.client.0.smithi007.stdout:6/802: write d3/d3e/f48 [1177797,110159] 0 2024-09-21T04:35:39.330 INFO:tasks.workunit.client.0.smithi007.stdout:3/926: read - dc/d18/dea/f113 zero size 2024-09-21T04:35:39.331 INFO:tasks.workunit.client.0.smithi007.stdout:5/876: mknod d2/d27/d42/da3/d23/d49/c11f 0 2024-09-21T04:35:39.333 INFO:tasks.workunit.client.0.smithi007.stdout:3/927: rmdir dc/d18/d41/d96 39 2024-09-21T04:35:39.335 INFO:tasks.workunit.client.0.smithi007.stdout:3/928: readlink dc/d22/d29/d32/d4d/d84/dd0/dbc/lbe 0 2024-09-21T04:35:39.335 INFO:tasks.workunit.client.0.smithi007.stdout:0/795: link d6/d75/dc5/d18/d2e/f87 d6/d75/dc5/d18/d27/f125 0 2024-09-21T04:35:39.338 INFO:tasks.workunit.client.0.smithi007.stdout:5/877: rename d2/d27/d42/da3/d23/d49/dd4/dd5/d111/d54/f59 to d2/d27/d3d/dc2/dda/f120 0 2024-09-21T04:35:39.342 INFO:tasks.workunit.client.0.smithi007.stdout:3/929: creat dc/d22/d29/d32/d4d/d84/dd0/dbc/dd1/f122 x:0 0 0 2024-09-21T04:35:39.345 INFO:tasks.workunit.client.0.smithi007.stdout:5/878: read d2/d27/d42/da3/d1d/f4b [940849,6206] 0 2024-09-21T04:35:39.346 INFO:tasks.workunit.client.0.smithi007.stdout:6/803: dwrite d3/d3e/f40 [0,4194304] 0 2024-09-21T04:35:39.355 INFO:tasks.workunit.client.0.smithi007.stdout:0/796: dread d6/d75/dc5/d18/d2e/d60/f63 [0,4194304] 0 2024-09-21T04:35:39.355 INFO:tasks.workunit.client.0.smithi007.stdout:5/879: fsync d2/d27/d42/da3/d23/d49/f6a 0 2024-09-21T04:35:39.356 INFO:tasks.workunit.client.0.smithi007.stdout:0/797: stat d6/d75/dc5/d18/d2e/fff 0 2024-09-21T04:35:39.357 INFO:tasks.workunit.client.0.smithi007.stdout:5/880: chown d2/d27/d42/da3/d23/d2a/f2d 224 1 2024-09-21T04:35:39.362 INFO:tasks.workunit.client.0.smithi007.stdout:5/881: mknod d2/d27/dee/c121 0 2024-09-21T04:35:39.364 INFO:tasks.workunit.client.0.smithi007.stdout:0/798: mkdir d6/d75/dc5/d47/d8a/de9/d101/d109/d126 0 2024-09-21T04:35:39.366 INFO:tasks.workunit.client.0.smithi007.stdout:5/882: rmdir d2/d27/dee 39 2024-09-21T04:35:39.372 INFO:tasks.workunit.client.0.smithi007.stdout:6/804: write d3/de/d12/d19/da3/fac [823725,61721] 0 2024-09-21T04:35:39.372 INFO:tasks.workunit.client.0.smithi007.stdout:0/799: write d6/d91/da3/dbf/dec/ffc [1760596,104791] 0 2024-09-21T04:35:39.375 INFO:tasks.workunit.client.0.smithi007.stdout:6/805: stat d3/d3b/dd3/d60/dd7/ff6 0 2024-09-21T04:35:39.411 INFO:tasks.workunit.client.0.smithi007.stdout:5/883: creat d2/d27/d3d/d9e/f122 x:0 0 0 2024-09-21T04:35:39.421 INFO:tasks.workunit.client.0.smithi007.stdout:0/800: getdents d6/d75/dc5/d47/d8a/de9/d101/d109 0 2024-09-21T04:35:39.422 INFO:tasks.workunit.client.0.smithi007.stdout:5/884: dwrite d2/d27/d42/da3/d23/d2a/f63 [0,4194304] 0 2024-09-21T04:35:39.426 INFO:tasks.workunit.client.0.smithi007.stdout:0/801: write d6/d75/dc5/d52/f78 [3834774,26219] 0 2024-09-21T04:35:39.434 INFO:tasks.workunit.client.0.smithi007.stdout:5/885: dwrite d2/d27/d42/da3/d23/d49/dd4/de9/f116 [0,4194304] 0 2024-09-21T04:35:39.435 INFO:tasks.workunit.client.0.smithi007.stdout:0/802: truncate d6/d75/dc5/d18/d27/ff7 2342386 0 2024-09-21T04:35:39.437 INFO:tasks.workunit.client.0.smithi007.stdout:0/803: write d6/d75/dc5/d47/d99/f11a [877061,74400] 0 2024-09-21T04:35:39.444 INFO:tasks.workunit.client.0.smithi007.stdout:5/886: unlink d2/d27/d42/da3/d23/d49/dd4/dd5/d111/d54/cb8 0 2024-09-21T04:35:39.446 INFO:tasks.workunit.client.0.smithi007.stdout:5/887: dread - d2/d27/d42/da3/d23/d49/f10e zero size 2024-09-21T04:35:39.448 INFO:tasks.workunit.client.0.smithi007.stdout:5/888: truncate d2/d27/f11a 1040273 0 2024-09-21T04:35:39.455 INFO:tasks.workunit.client.0.smithi007.stdout:5/889: dwrite d2/d27/d3d/d9b/ff8 [0,4194304] 0 2024-09-21T04:35:39.461 INFO:tasks.workunit.client.0.smithi007.stdout:5/890: rmdir d2/d27/d42/d53 39 2024-09-21T04:35:39.464 INFO:tasks.workunit.client.0.smithi007.stdout:5/891: fdatasync d2/d27/d42/f62 0 2024-09-21T04:35:39.469 INFO:tasks.workunit.client.0.smithi007.stdout:5/892: mknod d2/d27/d42/da3/d28/c123 0 2024-09-21T04:35:39.479 INFO:tasks.workunit.client.0.smithi007.stdout:6/806: write d3/de/d12/d19/f36 [5652363,76381] 0 2024-09-21T04:35:39.483 INFO:tasks.workunit.client.0.smithi007.stdout:6/807: creat d3/de/d12/d19/da3/de3/f108 x:0 0 0 2024-09-21T04:35:39.485 INFO:tasks.workunit.client.0.smithi007.stdout:6/808: mknod d3/d3b/dd3/d60/d77/dcb/d78/c109 0 2024-09-21T04:35:39.487 INFO:tasks.workunit.client.0.smithi007.stdout:5/893: write d2/d27/d42/da3/d28/fa5 [345753,72294] 0 2024-09-21T04:35:39.487 INFO:tasks.workunit.client.0.smithi007.stdout:6/809: dread - d3/d3e/d99/dbb/d95/dc9/ffb zero size 2024-09-21T04:35:39.488 INFO:tasks.workunit.client.0.smithi007.stdout:0/804: write d6/d75/dc5/d18/d27/ff7 [1456970,29409] 0 2024-09-21T04:35:39.491 INFO:tasks.workunit.client.0.smithi007.stdout:0/805: dread - d6/d75/dc9/ff6 zero size 2024-09-21T04:35:39.495 INFO:tasks.workunit.client.0.smithi007.stdout:5/894: write d2/d27/d42/da3/d1d/d20/f6d [1752261,78084] 0 2024-09-21T04:35:39.495 INFO:tasks.workunit.client.0.smithi007.stdout:0/806: mknod d6/df5/c127 0 2024-09-21T04:35:39.498 INFO:tasks.workunit.client.0.smithi007.stdout:5/895: symlink d2/d27/d42/da3/d23/d5d/d9a/l124 0 2024-09-21T04:35:39.498 INFO:tasks.workunit.client.0.smithi007.stdout:0/807: read d6/d75/dc5/d47/fa0 [387042,114463] 0 2024-09-21T04:35:39.499 INFO:tasks.workunit.client.0.smithi007.stdout:0/808: truncate d6/d75/dc5/d47/f10d 725437 0 2024-09-21T04:35:39.501 INFO:tasks.workunit.client.0.smithi007.stdout:5/896: chown d2/d27/d42/da3/d28/f64 5757 1 2024-09-21T04:35:39.502 INFO:tasks.workunit.client.0.smithi007.stdout:0/809: mknod d6/d75/dc5/d47/d8a/de9/c128 0 2024-09-21T04:35:39.503 INFO:tasks.workunit.client.0.smithi007.stdout:0/810: chown d6/d75/dc5/d47/d99/fbc 13750645 1 2024-09-21T04:35:39.506 INFO:tasks.workunit.client.0.smithi007.stdout:0/811: mknod d6/d75/dc5/d52/d6d/de1/c129 0 2024-09-21T04:35:39.511 INFO:tasks.workunit.client.0.smithi007.stdout:0/812: rename d6/d75/dc5/d18/d39/f6a to d6/d75/dc5/d47/d8a/d9c/f12a 0 2024-09-21T04:35:39.516 INFO:tasks.workunit.client.0.smithi007.stdout:0/813: symlink d6/d75/dc9/l12b 0 2024-09-21T04:35:39.520 INFO:tasks.workunit.client.0.smithi007.stdout:0/814: creat d6/d91/da3/dbf/f12c x:0 0 0 2024-09-21T04:35:39.520 INFO:tasks.workunit.client.0.smithi007.stdout:5/897: write d2/fc [1139782,111274] 0 2024-09-21T04:35:39.522 INFO:tasks.workunit.client.0.smithi007.stdout:5/898: write d2/d27/d42/da3/d28/de2/f117 [659682,88166] 0 2024-09-21T04:35:39.523 INFO:tasks.workunit.client.0.smithi007.stdout:0/815: creat d6/d75/dc5/d18/d39/f12d x:0 0 0 2024-09-21T04:35:39.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/816: symlink d6/d75/dc5/d47/d8a/l12e 0 2024-09-21T04:35:39.528 INFO:tasks.workunit.client.0.smithi007.stdout:5/899: mknod d2/d27/d42/da3/d23/d49/d6f/d104/c125 0 2024-09-21T04:35:39.528 INFO:tasks.workunit.client.0.smithi007.stdout:0/817: dread - d6/d75/dc5/d18/fde zero size 2024-09-21T04:35:39.532 INFO:tasks.workunit.client.0.smithi007.stdout:0/818: dread d6/d75/dc5/d52/f78 [0,4194304] 0 2024-09-21T04:35:39.532 INFO:tasks.workunit.client.0.smithi007.stdout:5/900: mkdir d2/d27/d42/da3/d23/d49/d6f/ddb/def/d126 0 2024-09-21T04:35:39.535 INFO:tasks.workunit.client.0.smithi007.stdout:5/901: unlink d2/d27/d3d/d9b/cac 0 2024-09-21T04:35:39.548 INFO:tasks.workunit.client.0.smithi007.stdout:0/819: write d6/d75/dc5/d47/d8a/de9/d101/fc8 [579879,15283] 0 2024-09-21T04:35:39.554 INFO:tasks.workunit.client.0.smithi007.stdout:5/902: dwrite d2/d1e/f6c [0,4194304] 0 2024-09-21T04:35:39.554 INFO:tasks.workunit.client.0.smithi007.stdout:0/820: fsync d6/d75/dc5/f10c 0 2024-09-21T04:35:39.554 INFO:tasks.workunit.client.0.smithi007.stdout:0/821: readlink d6/d75/dc5/d18/d39/l50 0 2024-09-21T04:35:39.558 INFO:tasks.workunit.client.0.smithi007.stdout:5/903: fdatasync d2/d27/d42/da3/d23/fe7 0 2024-09-21T04:35:39.560 INFO:tasks.workunit.client.0.smithi007.stdout:0/822: unlink d6/d75/dc5/d47/d8a/de9/ff3 0 2024-09-21T04:35:39.566 INFO:tasks.workunit.client.0.smithi007.stdout:5/904: creat d2/d27/d3d/dc2/dda/f127 x:0 0 0 2024-09-21T04:35:39.567 INFO:tasks.workunit.client.0.smithi007.stdout:5/905: stat d2/d27/d3d/dc2/dda 0 2024-09-21T04:35:39.571 INFO:tasks.workunit.client.0.smithi007.stdout:5/906: unlink d2/d27/d42/d53/c7c 0 2024-09-21T04:35:39.577 INFO:tasks.workunit.client.0.smithi007.stdout:5/907: rename d2/d27/d3d/d9b to d2/d27/d42/da3/d23/d49/d6f/ddb/def/d128 0 2024-09-21T04:35:39.577 INFO:tasks.workunit.client.0.smithi007.stdout:0/823: truncate d6/d75/dc5/f86 630668 0 2024-09-21T04:35:39.577 INFO:tasks.workunit.client.0.smithi007.stdout:5/908: readlink d2/d1e/l86 0 2024-09-21T04:35:39.578 INFO:tasks.workunit.client.0.smithi007.stdout:0/824: stat d6/d75/dc5/d18/d39/fac 0 2024-09-21T04:35:39.579 INFO:tasks.workunit.client.0.smithi007.stdout:5/909: dread - d2/d27/d42/da3/d23/d49/dd4/dd5/d111/f11d zero size 2024-09-21T04:35:39.581 INFO:tasks.workunit.client.0.smithi007.stdout:0/825: fdatasync d6/d75/dc5/d18/d27/fd3 0 2024-09-21T04:35:39.582 INFO:tasks.workunit.client.0.smithi007.stdout:0/826: chown d6/d75/dc5/d18/d27/f3a 10342759 1 2024-09-21T04:35:39.582 INFO:tasks.workunit.client.0.smithi007.stdout:5/910: creat d2/d27/d42/da3/d1d/f129 x:0 0 0 2024-09-21T04:35:39.584 INFO:tasks.workunit.client.0.smithi007.stdout:5/911: fdatasync d2/d27/d42/da3/d23/d49/d6f/ddb/def/d128/fe8 0 2024-09-21T04:35:39.590 INFO:tasks.workunit.client.0.smithi007.stdout:5/912: mkdir d2/d27/d42/da3/d28/d12a 0 2024-09-21T04:35:39.592 INFO:tasks.workunit.client.0.smithi007.stdout:0/827: mknod d6/d75/dc5/c12f 0 2024-09-21T04:35:39.593 INFO:tasks.workunit.client.0.smithi007.stdout:5/913: fdatasync d2/d27/d3d/f69 0 2024-09-21T04:35:39.594 INFO:tasks.workunit.client.0.smithi007.stdout:5/914: fsync d2/d27/d3d/d9e/fd2 0 2024-09-21T04:35:39.595 INFO:tasks.workunit.client.0.smithi007.stdout:5/915: write d2/d1e/f6c [3687943,53908] 0 2024-09-21T04:35:39.598 INFO:tasks.workunit.client.0.smithi007.stdout:5/916: truncate d2/d27/d42/da3/d23/d49/dd4/de9/feb 666102 0 2024-09-21T04:35:39.598 INFO:tasks.workunit.client.0.smithi007.stdout:5/917: fdatasync d2/d27/d42/da3/fb9 0 2024-09-21T04:35:39.599 INFO:tasks.workunit.client.0.smithi007.stdout:5/918: read d2/d27/d42/da3/f3e [1487875,36996] 0 2024-09-21T04:35:39.602 INFO:tasks.workunit.client.0.smithi007.stdout:5/919: mknod d2/d27/d42/da3/d1d/c12b 0 2024-09-21T04:35:39.608 INFO:tasks.workunit.client.0.smithi007.stdout:5/920: dwrite d2/d27/d42/da3/d23/d49/d6f/ddb/def/d128/fe8 [0,4194304] 0 2024-09-21T04:35:39.614 INFO:tasks.workunit.client.0.smithi007.stdout:5/921: rename d2/d27/d42/da3/d23/d5d/d61 to d2/d27/d42/da3/d12c 0 2024-09-21T04:35:39.620 INFO:tasks.workunit.client.0.smithi007.stdout:5/922: dwrite d2/d27/d42/da3/f16 [8388608,4194304] 0 2024-09-21T04:35:39.643 INFO:tasks.workunit.client.0.smithi007.stdout:5/923: dwrite d2/d1e/f24 [4194304,4194304] 0 2024-09-21T04:35:39.644 INFO:tasks.workunit.client.0.smithi007.stdout:5/924: chown d2/f50 1160 1 2024-09-21T04:35:39.658 INFO:tasks.workunit.client.0.smithi007.stdout:5/925: dwrite d2/d1e/f58 [0,4194304] 0 2024-09-21T04:35:39.659 INFO:tasks.workunit.client.0.smithi007.stdout:5/926: chown d2/d1e/f6c 3 1 2024-09-21T04:35:39.671 INFO:tasks.workunit.client.0.smithi007.stdout:5/927: write d2/d27/dee/f115 [562665,68169] 0 2024-09-21T04:35:39.674 INFO:tasks.workunit.client.0.smithi007.stdout:5/928: mknod d2/d27/d42/d53/dcf/c12d 0 2024-09-21T04:35:39.676 INFO:tasks.workunit.client.0.smithi007.stdout:5/929: mknod d2/d27/d42/da3/d23/d49/d6f/ddb/c12e 0 2024-09-21T04:35:39.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:39 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:39.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:39 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:39.725 INFO:tasks.workunit.client.0.smithi007.stdout:2/645: sync 2024-09-21T04:35:39.738 INFO:tasks.workunit.client.0.smithi007.stdout:2/646: dread - d0/d11/d31/d7b/f94 zero size 2024-09-21T04:35:39.738 INFO:tasks.workunit.client.0.smithi007.stdout:5/930: write d2/d27/d42/da3/d23/d49/dd4/de9/fff [693257,5110] 0 2024-09-21T04:35:39.739 INFO:tasks.workunit.client.0.smithi007.stdout:5/931: dread - d2/d27/d3d/d56/fe4 zero size 2024-09-21T04:35:39.741 INFO:tasks.workunit.client.0.smithi007.stdout:2/647: mkdir d0/d11/d31/d46/d6d/daa/dbd 0 2024-09-21T04:35:39.742 INFO:tasks.workunit.client.0.smithi007.stdout:5/932: creat d2/d27/d3d/f12f x:0 0 0 2024-09-21T04:35:39.853 INFO:tasks.workunit.client.0.smithi007.stdout:2/648: creat d0/d11/d31/d46/d77/fbe x:0 0 0 2024-09-21T04:35:39.855 INFO:tasks.workunit.client.0.smithi007.stdout:2/649: mkdir d0/d11/db1/dbf 0 2024-09-21T04:35:39.856 INFO:tasks.workunit.client.0.smithi007.stdout:5/933: link d2/d27/d42/da3/d28/c123 d2/d27/d42/da3/d1d/c130 0 2024-09-21T04:35:39.857 INFO:tasks.workunit.client.0.smithi007.stdout:2/650: symlink d0/d11/d31/d9f/lc0 0 2024-09-21T04:35:39.877 INFO:tasks.workunit.client.0.smithi007.stdout:5/934: write d2/d27/d42/da3/d12c/ff1 [630181,9307] 0 2024-09-21T04:35:39.878 INFO:tasks.workunit.client.0.smithi007.stdout:2/651: write d0/d11/f20 [75152,27770] 0 2024-09-21T04:35:39.879 INFO:tasks.workunit.client.0.smithi007.stdout:5/935: creat d2/d27/dee/f131 x:0 0 0 2024-09-21T04:35:39.883 INFO:tasks.workunit.client.0.smithi007.stdout:2/652: creat d0/d11/d6b/fc1 x:0 0 0 2024-09-21T04:35:39.894 INFO:tasks.workunit.client.0.smithi007.stdout:8/712: sync 2024-09-21T04:35:39.895 INFO:tasks.workunit.client.0.smithi007.stdout:8/713: chown d0/d3/l8b 964313 1 2024-09-21T04:35:39.896 INFO:tasks.workunit.client.0.smithi007.stdout:2/653: dread d0/f1c [0,4194304] 0 2024-09-21T04:35:39.898 INFO:tasks.workunit.client.0.smithi007.stdout:8/714: getdents d0/d3/d82 0 2024-09-21T04:35:39.899 INFO:tasks.workunit.client.0.smithi007.stdout:2/654: creat d0/d11/d31/d46/d48/fc2 x:0 0 0 2024-09-21T04:35:39.900 INFO:tasks.workunit.client.0.smithi007.stdout:2/655: write d0/d11/d6b/fc1 [882190,79501] 0 2024-09-21T04:35:39.901 INFO:tasks.workunit.client.0.smithi007.stdout:2/656: write d0/d11/d31/d39/f3e [603390,9583] 0 2024-09-21T04:35:39.901 INFO:tasks.workunit.client.0.smithi007.stdout:8/715: mknod d0/d3/d74/d52/ccf 0 2024-09-21T04:35:39.904 INFO:tasks.workunit.client.0.smithi007.stdout:8/716: chown d0/d3/d8/c2d 5295933 1 2024-09-21T04:35:39.906 INFO:tasks.workunit.client.0.smithi007.stdout:8/717: symlink d0/d3/d16/d93/d5e/ld0 0 2024-09-21T04:35:39.907 INFO:tasks.workunit.client.0.smithi007.stdout:2/657: getdents d0/d11/d31/d7b/da2 0 2024-09-21T04:35:39.907 INFO:tasks.workunit.client.0.smithi007.stdout:8/718: readlink d0/d3/d8/l14 0 2024-09-21T04:35:39.907 INFO:tasks.workunit.client.0.smithi007.stdout:2/658: chown d0/d11/cac 0 1 2024-09-21T04:35:39.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/659: mknod d0/d11/d6e/d73/cc3 0 2024-09-21T04:35:39.911 INFO:tasks.workunit.client.0.smithi007.stdout:8/719: getdents d0/d3/d8/d46/db7 0 2024-09-21T04:35:39.912 INFO:tasks.workunit.client.0.smithi007.stdout:8/720: write d0/d3/d7c/f59 [527581,63250] 0 2024-09-21T04:35:39.913 INFO:tasks.workunit.client.0.smithi007.stdout:2/660: link d0/d11/d31/da4/fbb d0/d11/d31/d46/d6d/daa/dbd/fc4 0 2024-09-21T04:35:39.916 INFO:tasks.workunit.client.0.smithi007.stdout:2/661: truncate d0/d11/d31/d46/d6d/d81/da0/f88 240200 0 2024-09-21T04:35:39.918 INFO:tasks.workunit.client.0.smithi007.stdout:8/721: creat d0/d3/d16/d93/d26/d4d/d5d/fd1 x:0 0 0 2024-09-21T04:35:39.922 INFO:tasks.workunit.client.0.smithi007.stdout:2/662: creat d0/d11/d6b/d9d/fc5 x:0 0 0 2024-09-21T04:35:39.930 INFO:tasks.workunit.client.0.smithi007.stdout:8/722: dread d0/d3/d16/d93/d26/d4d/f72 [0,4194304] 0 2024-09-21T04:35:39.935 INFO:tasks.workunit.client.0.smithi007.stdout:8/723: mkdir d0/d3/d8/d46/dd2 0 2024-09-21T04:35:39.939 INFO:tasks.workunit.client.0.smithi007.stdout:8/724: symlink d0/d3/d16/d93/d26/d4d/d5d/dba/ld3 0 2024-09-21T04:35:39.942 INFO:tasks.workunit.client.0.smithi007.stdout:8/725: mknod d0/d3/d16/d93/d40/d92/cd4 0 2024-09-21T04:35:39.952 INFO:tasks.workunit.client.0.smithi007.stdout:2/663: dwrite d0/d11/d31/d46/f6a [0,4194304] 0 2024-09-21T04:35:39.957 INFO:tasks.workunit.client.0.smithi007.stdout:2/664: creat d0/d11/d31/d46/d6c/db7/fc6 x:0 0 0 2024-09-21T04:35:39.958 INFO:tasks.workunit.client.0.smithi007.stdout:2/665: chown d0/d11 426206668 1 2024-09-21T04:35:39.960 INFO:tasks.workunit.client.0.smithi007.stdout:8/726: dwrite d0/f43 [0,4194304] 0 2024-09-21T04:35:39.962 INFO:tasks.workunit.client.0.smithi007.stdout:2/666: creat d0/d11/d6e/fc7 x:0 0 0 2024-09-21T04:35:39.964 INFO:tasks.workunit.client.0.smithi007.stdout:8/727: truncate d0/d3/d16/d93/d26/d35/f8e 263070 0 2024-09-21T04:35:39.965 INFO:tasks.workunit.client.0.smithi007.stdout:8/728: read - d0/d3/d74/fac zero size 2024-09-21T04:35:39.966 INFO:tasks.workunit.client.0.smithi007.stdout:8/729: stat d0/d3/l22 0 2024-09-21T04:35:39.966 INFO:tasks.workunit.client.0.smithi007.stdout:2/667: getdents d0/d11/d31/d7b/da2 0 2024-09-21T04:35:39.967 INFO:tasks.workunit.client.0.smithi007.stdout:8/730: readlink d0/d3/d16/d93/d26/d4d/l57 0 2024-09-21T04:35:39.967 INFO:tasks.workunit.client.0.smithi007.stdout:8/731: fsync d0/d3/d7c/f59 0 2024-09-21T04:35:39.970 INFO:tasks.workunit.client.0.smithi007.stdout:8/732: rename d0/d3/d16/d93/d40/f95 to d0/d3/d7c/fd5 0 2024-09-21T04:35:39.974 INFO:tasks.workunit.client.0.smithi007.stdout:8/733: creat d0/d3/d74/d52/fd6 x:0 0 0 2024-09-21T04:35:39.981 INFO:tasks.workunit.client.0.smithi007.stdout:8/734: dwrite d0/d3/d8/d46/f60 [0,4194304] 0 2024-09-21T04:35:39.983 INFO:tasks.workunit.client.0.smithi007.stdout:8/735: write d0/d3/d74/fac [83302,17923] 0 2024-09-21T04:35:39.987 INFO:tasks.workunit.client.0.smithi007.stdout:8/736: dread d0/d3/d16/d93/d26/d4d/fbd [0,4194304] 0 2024-09-21T04:35:39.994 INFO:tasks.workunit.client.0.smithi007.stdout:8/737: dwrite d0/d3/d7c/f64 [0,4194304] 0 2024-09-21T04:35:40.006 INFO:tasks.workunit.client.0.smithi007.stdout:8/738: getdents d0/d3/d16/d93/d5e 0 2024-09-21T04:35:40.019 INFO:tasks.workunit.client.0.smithi007.stdout:8/739: dread d0/d3/d8/fa [0,4194304] 0 2024-09-21T04:35:40.021 INFO:tasks.workunit.client.0.smithi007.stdout:8/740: rmdir d0/d3/d16/d93/d26/d35 39 2024-09-21T04:35:40.025 INFO:tasks.workunit.client.0.smithi007.stdout:8/741: rename d0/d3/d16/d93/d40/l96 to d0/d3/daa/d9d/ld7 0 2024-09-21T04:35:40.026 INFO:tasks.workunit.client.0.smithi007.stdout:8/742: dread d0/d3/daa/f76 [0,4194304] 0 2024-09-21T04:35:40.032 INFO:tasks.workunit.client.0.smithi007.stdout:8/743: link d0/d3/d16/d93/d26/d35/f8e d0/d3/daa/fd8 0 2024-09-21T04:35:40.043 INFO:tasks.workunit.client.0.smithi007.stdout:8/744: dread d0/d3/d16/d93/f69 [0,4194304] 0 2024-09-21T04:35:40.045 INFO:tasks.workunit.client.0.smithi007.stdout:8/745: mknod d0/d3/d16/d93/d26/d35/cd9 0 2024-09-21T04:35:40.048 INFO:tasks.workunit.client.0.smithi007.stdout:8/746: symlink d0/d3/d16/d93/d40/d92/lda 0 2024-09-21T04:35:40.063 INFO:tasks.workunit.client.0.smithi007.stdout:8/747: dwrite d0/d3/d8/f47 [0,4194304] 0 2024-09-21T04:35:40.071 INFO:tasks.workunit.client.0.smithi007.stdout:8/748: dwrite d0/d3/f2b [0,4194304] 0 2024-09-21T04:35:40.071 INFO:tasks.workunit.client.0.smithi007.stdout:8/749: chown d0/d3/d7c/fc6 3 1 2024-09-21T04:35:40.078 INFO:tasks.workunit.client.0.smithi007.stdout:8/750: write d0/fa6 [1084446,68745] 0 2024-09-21T04:35:40.082 INFO:tasks.workunit.client.0.smithi007.stdout:8/751: creat d0/fdb x:0 0 0 2024-09-21T04:35:40.083 INFO:tasks.workunit.client.0.smithi007.stdout:8/752: mknod d0/d3/daa/cdc 0 2024-09-21T04:35:40.084 INFO:tasks.workunit.client.0.smithi007.stdout:8/753: truncate d0/d3/fc7 946997 0 2024-09-21T04:35:40.086 INFO:tasks.workunit.client.0.smithi007.stdout:8/754: dread - d0/d3/d16/d93/d26/fb5 zero size 2024-09-21T04:35:40.091 INFO:tasks.workunit.client.0.smithi007.stdout:8/755: creat d0/d3/d16/d93/d40/d92/dcb/fdd x:0 0 0 2024-09-21T04:35:40.092 INFO:tasks.workunit.client.0.smithi007.stdout:8/756: readlink d0/l13 0 2024-09-21T04:35:40.093 INFO:tasks.workunit.client.0.smithi007.stdout:8/757: write d0/d3/fc7 [1401355,80592] 0 2024-09-21T04:35:40.094 INFO:tasks.workunit.client.0.smithi007.stdout:8/758: stat d0/d3/d16/d93/d26/d35/f73 0 2024-09-21T04:35:40.097 INFO:tasks.workunit.client.0.smithi007.stdout:8/759: creat d0/d3/d8/fde x:0 0 0 2024-09-21T04:35:40.100 INFO:tasks.workunit.client.0.smithi007.stdout:8/760: creat d0/d3/d16/d93/d26/d4d/d5d/fdf x:0 0 0 2024-09-21T04:35:40.101 INFO:tasks.workunit.client.0.smithi007.stdout:8/761: stat d0/d3/d16/d93 0 2024-09-21T04:35:40.107 INFO:tasks.workunit.client.0.smithi007.stdout:8/762: write d0/d3/d74/f66 [1739710,82902] 0 2024-09-21T04:35:40.109 INFO:tasks.workunit.client.0.smithi007.stdout:8/763: write d0/d3/d8/d46/f60 [5011102,41605] 0 2024-09-21T04:35:40.109 INFO:tasks.workunit.client.0.smithi007.stdout:9/998: sync 2024-09-21T04:35:40.112 INFO:tasks.workunit.client.0.smithi007.stdout:8/764: creat d0/d3/d8/d46/db7/fe0 x:0 0 0 2024-09-21T04:35:40.117 INFO:tasks.workunit.client.0.smithi007.stdout:9/999: creat d13/d18/d60/f145 x:0 0 0 2024-09-21T04:35:40.119 INFO:tasks.workunit.client.0.smithi007.stdout:8/765: symlink d0/d3/d8/d46/dd2/le1 0 2024-09-21T04:35:40.123 INFO:tasks.workunit.client.0.smithi007.stdout:8/766: rename d0/d3/d16/d93/f69 to d0/d3/d8/d46/db7/fe2 0 2024-09-21T04:35:40.126 INFO:tasks.workunit.client.0.smithi007.stdout:8/767: rename d0/d3/d16/d93/d26 to d0/d3/d16/d93/d5e/de3 0 2024-09-21T04:35:40.129 INFO:tasks.workunit.client.0.smithi007.stdout:8/768: rename d0/d3/d7c/fd5 to d0/d3/d7c/dbf/fe4 0 2024-09-21T04:35:40.131 INFO:tasks.workunit.client.0.smithi007.stdout:8/769: rmdir d0/d3/d16/d93/d40/d9e 39 2024-09-21T04:35:40.135 INFO:tasks.workunit.client.0.smithi007.stdout:8/770: rename d0/d3/d8/d46/f60 to d0/d3/d16/d93/d5e/fe5 0 2024-09-21T04:35:40.137 INFO:tasks.workunit.client.0.smithi007.stdout:8/771: creat d0/d3/d82/fe6 x:0 0 0 2024-09-21T04:35:40.138 INFO:tasks.workunit.client.0.smithi007.stdout:8/772: write d0/d3/d16/d93/d5e/fce [695321,3328] 0 2024-09-21T04:35:40.140 INFO:tasks.workunit.client.0.smithi007.stdout:8/773: write d0/d3/d16/d93/d5e/de3/d35/f87 [291918,29286] 0 2024-09-21T04:35:40.142 INFO:tasks.workunit.client.0.smithi007.stdout:8/774: creat d0/d3/d16/d93/d5e/fe7 x:0 0 0 2024-09-21T04:35:40.149 INFO:tasks.workunit.client.0.smithi007.stdout:8/775: dwrite d0/d3/d16/d93/d5e/de3/d4d/d5d/fd1 [0,4194304] 0 2024-09-21T04:35:40.153 INFO:tasks.workunit.client.0.smithi007.stdout:8/776: dread d0/d3/d16/d93/d5e/fe5 [0,4194304] 0 2024-09-21T04:35:40.155 INFO:tasks.workunit.client.0.smithi007.stdout:8/777: dread - d0/d3/d16/d93/d5e/de3/d4d/d5d/dba/fcd zero size 2024-09-21T04:35:40.156 INFO:tasks.workunit.client.0.smithi007.stdout:8/778: fdatasync d0/fb8 0 2024-09-21T04:35:40.162 INFO:tasks.workunit.client.0.smithi007.stdout:8/779: dwrite d0/d3/d16/d93/d5e/de3/d4d/d5d/fdf [0,4194304] 0 2024-09-21T04:35:40.169 INFO:tasks.workunit.client.0.smithi007.stdout:8/780: symlink d0/d3/d74/d52/le8 0 2024-09-21T04:35:40.179 INFO:tasks.workunit.client.0.smithi007.stdout:8/781: getdents d0/d3/d74/d52 0 2024-09-21T04:35:40.256 INFO:tasks.workunit.client.0.smithi007.stdout:7/864: sync 2024-09-21T04:35:40.288 INFO:tasks.workunit.client.0.smithi007.stdout:7/865: creat d3/d6/d40/f12e x:0 0 0 2024-09-21T04:35:40.293 INFO:tasks.workunit.client.0.smithi007.stdout:7/866: link d3/d6/d2e/d63/d6a/cc8 d3/d6/d2e/d63/c12f 0 2024-09-21T04:35:40.295 INFO:tasks.workunit.client.0.smithi007.stdout:7/867: read - d3/d6/d14/d20/d80/db2/db6/f126 zero size 2024-09-21T04:35:40.297 INFO:tasks.workunit.client.0.smithi007.stdout:7/868: mknod d3/d6/d85/de9/c130 0 2024-09-21T04:35:40.309 INFO:tasks.workunit.client.0.smithi007.stdout:7/869: dread d3/d6/d13/d4b/f57 [0,4194304] 0 2024-09-21T04:35:40.312 INFO:tasks.workunit.client.0.smithi007.stdout:7/870: truncate d3/d6/d10/d4f/f5c 338129 0 2024-09-21T04:35:40.317 INFO:tasks.workunit.client.0.smithi007.stdout:7/871: dwrite d3/d6/d14/d20/f35 [0,4194304] 0 2024-09-21T04:35:40.324 INFO:tasks.workunit.client.0.smithi007.stdout:7/872: write d3/d6/d85/dad/fce [2886989,84524] 0 2024-09-21T04:35:40.326 INFO:tasks.workunit.client.0.smithi007.stdout:7/873: readlink d3/d6/d10/d51/lbd 0 2024-09-21T04:35:40.329 INFO:tasks.workunit.client.0.smithi007.stdout:7/874: symlink d3/d6/d13/d4b/l131 0 2024-09-21T04:35:40.338 INFO:tasks.workunit.client.0.smithi007.stdout:7/875: write d3/d6/d10/d4f/f9b [331887,88403] 0 2024-09-21T04:35:40.341 INFO:tasks.workunit.client.0.smithi007.stdout:7/876: mknod d3/d6/d14/d20/d102/c132 0 2024-09-21T04:35:40.350 INFO:tasks.workunit.client.0.smithi007.stdout:7/877: dwrite d3/d6/d85/dad/fb5 [0,4194304] 0 2024-09-21T04:35:40.363 INFO:tasks.workunit.client.0.smithi007.stdout:7/878: read d3/d6/d2e/d63/d6a/d81/f8c [4064291,70825] 0 2024-09-21T04:35:40.406 INFO:tasks.workunit.client.0.smithi007.stdout:7/879: fdatasync d3/d6/d13/d4b/f57 0 2024-09-21T04:35:40.408 INFO:tasks.workunit.client.0.smithi007.stdout:7/880: write d3/d6/d40/da1/f10f [170364,79206] 0 2024-09-21T04:35:40.519 INFO:tasks.workunit.client.0.smithi007.stdout:7/881: write d3/d6/d2e/d63/fde [793037,130228] 0 2024-09-21T04:35:40.525 INFO:tasks.workunit.client.0.smithi007.stdout:7/882: write d3/d6/d10/d4f/f65 [368823,102582] 0 2024-09-21T04:35:40.528 INFO:tasks.workunit.client.0.smithi007.stdout:7/883: truncate d3/d6/d40/da1/fe5 1930785 0 2024-09-21T04:35:40.528 INFO:tasks.workunit.client.0.smithi007.stdout:7/884: chown d3/d6/d85/f49 125736 1 2024-09-21T04:35:40.536 INFO:tasks.workunit.client.0.smithi007.stdout:7/885: write d3/d6/d40/fdd [516325,101915] 0 2024-09-21T04:35:40.539 INFO:tasks.workunit.client.0.smithi007.stdout:7/886: mknod d3/d6/d14/d20/d58/c133 0 2024-09-21T04:35:40.542 INFO:tasks.workunit.client.0.smithi007.stdout:7/887: rename d3/f88 to d3/d6/dc5/f134 0 2024-09-21T04:35:40.543 INFO:tasks.workunit.client.0.smithi007.stdout:7/888: chown d3/d6/d13/fc4 437 1 2024-09-21T04:35:40.544 INFO:tasks.workunit.client.0.smithi007.stdout:7/889: read d3/d6/f2f [3699279,103678] 0 2024-09-21T04:35:40.546 INFO:tasks.workunit.client.0.smithi007.stdout:7/890: creat d3/d6/d85/f135 x:0 0 0 2024-09-21T04:35:40.549 INFO:tasks.workunit.client.0.smithi007.stdout:7/891: creat d3/d6/d40/da1/f136 x:0 0 0 2024-09-21T04:35:40.563 INFO:tasks.workunit.client.0.smithi007.stdout:7/892: dwrite d3/d6/d14/d20/d80/f9d [0,4194304] 0 2024-09-21T04:35:40.574 INFO:tasks.workunit.client.0.smithi007.stdout:7/893: write d3/d6/d2e/d5e/fef [123502,16488] 0 2024-09-21T04:35:40.577 INFO:tasks.workunit.client.0.smithi007.stdout:7/894: write d3/d6/d85/de9/f11e [289144,77818] 0 2024-09-21T04:35:40.579 INFO:tasks.workunit.client.0.smithi007.stdout:7/895: symlink d3/d6/d40/l137 0 2024-09-21T04:35:40.581 INFO:tasks.workunit.client.0.smithi007.stdout:7/896: mkdir d3/d6/d14/d20/d138 0 2024-09-21T04:35:40.584 INFO:tasks.workunit.client.0.smithi007.stdout:7/897: unlink d3/d6/d40/d112/c120 0 2024-09-21T04:35:40.586 INFO:tasks.workunit.client.0.smithi007.stdout:7/898: creat d3/d6/d14/d20/d80/dca/d113/d12a/f139 x:0 0 0 2024-09-21T04:35:40.600 INFO:tasks.workunit.client.0.smithi007.stdout:7/899: truncate d3/d6/d13/fc4 3700592 0 2024-09-21T04:35:40.606 INFO:tasks.workunit.client.0.smithi007.stdout:7/900: dwrite d3/d6/d2e/d63/d6a/d81/f111 [0,4194304] 0 2024-09-21T04:35:40.612 INFO:tasks.workunit.client.0.smithi007.stdout:7/901: rename d3/d6/d2e/d63/d6a/cd8 to d3/d6/d85/c13a 0 2024-09-21T04:35:40.618 INFO:tasks.workunit.client.0.smithi007.stdout:7/902: dwrite d3/d6/d10/fe0 [0,4194304] 0 2024-09-21T04:35:40.625 INFO:tasks.workunit.client.0.smithi007.stdout:7/903: dwrite d3/d6/d85/dad/f12b [0,4194304] 0 2024-09-21T04:35:40.631 INFO:tasks.workunit.client.0.smithi007.stdout:7/904: dread d3/d6/d13/d4b/f57 [0,4194304] 0 2024-09-21T04:35:40.633 INFO:tasks.workunit.client.0.smithi007.stdout:7/905: mkdir d3/d6/d2e/d63/df8/d13b 0 2024-09-21T04:35:40.645 INFO:tasks.workunit.client.0.smithi007.stdout:7/906: dread d3/d6/d2e/d63/d6a/d81/fa0 [0,4194304] 0 2024-09-21T04:35:40.647 INFO:tasks.workunit.client.0.smithi007.stdout:7/907: rmdir d3/d6/d85/de7/d11b/d101 39 2024-09-21T04:35:40.655 INFO:tasks.workunit.client.0.smithi007.stdout:7/908: dwrite d3/d6/d85/de7/d11b/f97 [0,4194304] 0 2024-09-21T04:35:40.657 INFO:tasks.workunit.client.0.smithi007.stdout:7/909: fdatasync d3/d6/d14/d20/d80/fec 0 2024-09-21T04:35:40.666 INFO:tasks.workunit.client.0.smithi007.stdout:7/910: dwrite d3/d6/d40/f116 [0,4194304] 0 2024-09-21T04:35:40.667 INFO:tasks.workunit.client.0.smithi007.stdout:7/911: fsync d3/d6/d40/f12e 0 2024-09-21T04:35:40.670 INFO:tasks.workunit.client.0.smithi007.stdout:7/912: creat d3/d6/d14/d20/d80/db2/f13c x:0 0 0 2024-09-21T04:35:40.672 INFO:tasks.workunit.client.0.smithi007.stdout:7/913: unlink d3/f4c 0 2024-09-21T04:35:40.675 INFO:tasks.workunit.client.0.smithi007.stdout:7/914: symlink d3/d6/d14/d20/d80/dca/d113/d12a/l13d 0 2024-09-21T04:35:40.680 INFO:tasks.workunit.client.0.smithi007.stdout:7/915: creat d3/d6/d2e/f13e x:0 0 0 2024-09-21T04:35:40.683 INFO:tasks.workunit.client.0.smithi007.stdout:7/916: stat d3/d6/d40/ca4 0 2024-09-21T04:35:40.685 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:40 smithi007 ceph-mon[27954]: pgmap v41: 65 pgs: 65 active+clean; 2.8 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 50 MiB/s rd, 118 MiB/s wr, 354 op/s 2024-09-21T04:35:40.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.682+0000 7f65258a2640 1 -- 172.21.15.7:0/716636232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65201023f0 msgr2=0x7f65201047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:40.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.682+0000 7f65258a2640 1 --2- 172.21.15.7:0/716636232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65201023f0 0x7f65201047e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f65100098b0 tx=0x7f651002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:35:40.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.682+0000 7f65258a2640 1 -- 172.21.15.7:0/716636232 shutdown_connections 2024-09-21T04:35:40.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.682+0000 7f65258a2640 1 --2- 172.21.15.7:0/716636232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6520104d20 0x7f6520107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.682+0000 7f65258a2640 1 --2- 172.21.15.7:0/716636232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65201023f0 0x7f65201047e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.682+0000 7f65258a2640 1 -- 172.21.15.7:0/716636232 >> 172.21.15.7:0/716636232 conn(0x7f65200fc010 msgr2=0x7f65200fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.682+0000 7f65258a2640 1 -- 172.21.15.7:0/716636232 shutdown_connections 2024-09-21T04:35:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.682+0000 7f65258a2640 1 -- 172.21.15.7:0/716636232 wait complete. 2024-09-21T04:35:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f65258a2640 1 Processor -- start 2024-09-21T04:35:40.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f65258a2640 1 -- start start 2024-09-21T04:35:40.687 INFO:tasks.workunit.client.0.smithi007.stdout:7/917: symlink d3/d6/d85/de9/l13f 0 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f65258a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65201023f0 0x7f65201996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f65258a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6520104d20 0x7f6520199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f65258a2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f652010a7c0 con 0x7f6520104d20 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f65258a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f652010a640 con 0x7f65201023f0 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f651effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65201023f0 0x7f65201996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f651effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65201023f0 0x7f65201996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:42904/0 (socket says 172.21.15.7:42904) 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f651effd640 1 -- 172.21.15.7:0/903964695 learned_addr learned my addr 172.21.15.7:0/903964695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f6517fff640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6520104d20 0x7f6520199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f651effd640 1 -- 172.21.15.7:0/903964695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6520104d20 msgr2=0x7f6520199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:40.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.683+0000 7f651effd640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6520104d20 0x7f6520199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.684+0000 7f651effd640 1 -- 172.21.15.7:0/903964695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652019a2a0 con 0x7f65201023f0 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.684+0000 7f651effd640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65201023f0 0x7f65201996e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f65100099d0 tx=0x7f65100045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.684+0000 7f65248a0640 1 -- 172.21.15.7:0/903964695 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f651003f070 con 0x7f65201023f0 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.684+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652019ec80 con 0x7f65201023f0 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.684+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652019f1c0 con 0x7f65201023f0 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.684+0000 7f65248a0640 1 -- 172.21.15.7:0/903964695 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65100049d0 con 0x7f65201023f0 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.684+0000 7f65248a0640 1 -- 172.21.15.7:0/903964695 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6510035030 con 0x7f65201023f0 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.686+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65000052c0 con 0x7f65201023f0 2024-09-21T04:35:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.686+0000 7f65248a0640 1 -- 172.21.15.7:0/903964695 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f6510042070 con 0x7f65201023f0 2024-09-21T04:35:40.690 INFO:tasks.workunit.client.0.smithi007.stdout:7/918: mknod d3/d6/d13/c140 0 2024-09-21T04:35:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.687+0000 7f65248a0640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f650c074a80 0x7f650c076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.687+0000 7f65248a0640 1 -- 172.21.15.7:0/903964695 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f65100b6e00 con 0x7f65201023f0 2024-09-21T04:35:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.687+0000 7f6517fff640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f650c074a80 0x7f650c076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.688+0000 7f6517fff640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f650c074a80 0x7f650c076f40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f652019ab90 tx=0x7f6508005f90 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:40.691 INFO:tasks.workunit.client.0.smithi007.stdout:7/919: readlink d3/d6/d14/d20/d80/db2/la6 0 2024-09-21T04:35:40.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:40 smithi053 ceph-mon[35554]: pgmap v41: 65 pgs: 65 active+clean; 2.8 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 50 MiB/s rd, 118 MiB/s wr, 354 op/s 2024-09-21T04:35:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:40.692+0000 7f65248a0640 1 -- 172.21.15.7:0/903964695 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f6510082350 con 0x7f65201023f0 2024-09-21T04:35:40.701 INFO:tasks.workunit.client.0.smithi007.stdout:7/920: dread d3/d6/f7 [0,4194304] 0 2024-09-21T04:35:40.703 INFO:tasks.workunit.client.0.smithi007.stdout:7/921: symlink d3/d6/d85/dad/l141 0 2024-09-21T04:35:40.705 INFO:tasks.workunit.client.0.smithi007.stdout:7/922: read d3/d6/d2e/d63/f12d [982938,54799] 0 2024-09-21T04:35:40.709 INFO:tasks.workunit.client.0.smithi007.stdout:3/930: sync 2024-09-21T04:35:40.711 INFO:tasks.workunit.client.0.smithi007.stdout:7/923: dwrite d3/d6/d40/faa [0,4194304] 0 2024-09-21T04:35:40.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/924: dwrite d3/d6/d14/d20/d58/f106 [0,4194304] 0 2024-09-21T04:35:40.728 INFO:tasks.workunit.client.0.smithi007.stdout:3/931: dwrite dc/d22/d29/d32/d4d/d7d/d9a/ff3 [0,4194304] 0 2024-09-21T04:35:40.735 INFO:tasks.workunit.client.0.smithi007.stdout:3/932: symlink dc/d22/d29/d32/d4d/d84/l123 0 2024-09-21T04:35:40.739 INFO:tasks.workunit.client.0.smithi007.stdout:7/925: dwrite d3/d6/d13/fba [0,4194304] 0 2024-09-21T04:35:40.743 INFO:tasks.workunit.client.0.smithi007.stdout:3/933: creat dc/d18/d41/d109/f124 x:0 0 0 2024-09-21T04:35:40.748 INFO:tasks.workunit.client.0.smithi007.stdout:7/926: unlink d3/d6/d85/dad/c122 0 2024-09-21T04:35:40.750 INFO:tasks.workunit.client.0.smithi007.stdout:3/934: creat dc/d22/d29/d32/d4d/d84/dbb/dc1/f125 x:0 0 0 2024-09-21T04:35:40.754 INFO:tasks.workunit.client.0.smithi007.stdout:7/927: dwrite d3/d6/d85/dad/fb5 [0,4194304] 0 2024-09-21T04:35:40.757 INFO:tasks.workunit.client.0.smithi007.stdout:7/928: stat d3/d6/d13/fc4 0 2024-09-21T04:35:40.758 INFO:tasks.workunit.client.0.smithi007.stdout:8/782: sync 2024-09-21T04:35:40.758 INFO:tasks.workunit.client.0.smithi007.stdout:6/810: sync 2024-09-21T04:35:40.758 INFO:tasks.workunit.client.0.smithi007.stdout:2/668: sync 2024-09-21T04:35:40.758 INFO:tasks.workunit.client.0.smithi007.stdout:5/936: sync 2024-09-21T04:35:40.758 INFO:tasks.workunit.client.0.smithi007.stdout:0/828: sync 2024-09-21T04:35:40.759 INFO:tasks.workunit.client.0.smithi007.stdout:8/783: fsync d0/d3/d74/d52/fd6 0 2024-09-21T04:35:40.762 INFO:tasks.workunit.client.0.smithi007.stdout:7/929: symlink d3/d6/d14/d20/d80/dca/d113/d12a/l142 0 2024-09-21T04:35:40.763 INFO:tasks.workunit.client.0.smithi007.stdout:6/811: dread - d3/de/d58/f90 zero size 2024-09-21T04:35:40.763 INFO:tasks.workunit.client.0.smithi007.stdout:8/784: dread - d0/d3/d16/d93/d5e/de3/fb5 zero size 2024-09-21T04:35:40.767 INFO:tasks.workunit.client.0.smithi007.stdout:8/785: read - d0/d3/d8/d46/db7/fe0 zero size 2024-09-21T04:35:40.769 INFO:tasks.workunit.client.0.smithi007.stdout:3/935: dwrite dc/d18/f39 [0,4194304] 0 2024-09-21T04:35:40.772 INFO:tasks.workunit.client.0.smithi007.stdout:0/829: write d6/d75/dc5/d18/f9e [2773819,52394] 0 2024-09-21T04:35:40.773 INFO:tasks.workunit.client.0.smithi007.stdout:2/669: symlink d0/d11/db8/lc8 0 2024-09-21T04:35:40.776 INFO:tasks.workunit.client.0.smithi007.stdout:8/786: symlink d0/d3/d8/d46/db7/le9 0 2024-09-21T04:35:40.779 INFO:tasks.workunit.client.0.smithi007.stdout:6/812: unlink d3/d3b/f7a 0 2024-09-21T04:35:40.779 INFO:tasks.workunit.client.0.smithi007.stdout:6/813: chown d3/d3e/cc7 12932302 1 2024-09-21T04:35:40.781 INFO:tasks.workunit.client.0.smithi007.stdout:3/936: creat dc/d18/dea/f126 x:0 0 0 2024-09-21T04:35:40.783 INFO:tasks.workunit.client.0.smithi007.stdout:7/930: mkdir d3/d6/d14/d20/d80/d143 0 2024-09-21T04:35:40.789 INFO:tasks.workunit.client.0.smithi007.stdout:8/787: stat d0/c1a 0 2024-09-21T04:35:40.790 INFO:tasks.workunit.client.0.smithi007.stdout:5/937: write d2/d27/d42/da3/d23/d5d/d9a/ff0 [1612254,102780] 0 2024-09-21T04:35:40.796 INFO:tasks.workunit.client.0.smithi007.stdout:3/937: dread dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/f93 [0,4194304] 0 2024-09-21T04:35:40.802 INFO:tasks.workunit.client.0.smithi007.stdout:5/938: creat d2/d27/d42/da3/d23/d49/dd4/dd5/d111/d54/f132 x:0 0 0 2024-09-21T04:35:40.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/814: symlink d3/de/de0/l10a 0 2024-09-21T04:35:40.804 INFO:tasks.workunit.client.0.smithi007.stdout:2/670: dwrite d0/d11/d31/d46/d6d/daa/dbd/fc4 [0,4194304] 0 2024-09-21T04:35:40.805 INFO:tasks.workunit.client.0.smithi007.stdout:8/788: dread d0/d3/d16/d93/d5e/d7b/f7e [0,4194304] 0 2024-09-21T04:35:40.818 INFO:tasks.workunit.client.0.smithi007.stdout:2/671: dread d0/d11/d31/d46/d48/f62 [0,4194304] 0 2024-09-21T04:35:40.862 INFO:tasks.workunit.client.0.smithi007.stdout:3/938: dread dc/d22/d29/d32/d4d/d84/dd0/dbc/dd1/ff1 [0,4194304] 0 2024-09-21T04:35:40.871 INFO:tasks.workunit.client.0.smithi007.stdout:8/789: creat d0/d3/d16/d93/d40/d92/dcb/fea x:0 0 0 2024-09-21T04:35:40.891 INFO:tasks.workunit.client.0.smithi007.stdout:2/672: link d0/d11/d31/d39/f44 d0/d11/d31/d7b/da2/da5/fc9 0 2024-09-21T04:35:40.896 INFO:tasks.workunit.client.0.smithi007.stdout:2/673: mknod d0/d11/d31/d46/d6d/daa/dbd/cca 0 2024-09-21T04:35:40.897 INFO:tasks.workunit.client.0.smithi007.stdout:2/674: chown d0/d11/d31/d39/f4f 1304 1 2024-09-21T04:35:40.899 INFO:tasks.workunit.client.0.smithi007.stdout:2/675: dread d0/d11/d31/d39/f3e [4194304,4194304] 0 2024-09-21T04:35:40.923 INFO:tasks.workunit.client.0.smithi007.stdout:8/790: rmdir d0/d3/d16/d93/d5e/de3/d4d/d5d/d88 39 2024-09-21T04:35:40.926 INFO:tasks.workunit.client.0.smithi007.stdout:8/791: unlink d0/d3/d16/d93/d5e/de3/d4d/l80 0 2024-09-21T04:35:40.943 INFO:tasks.workunit.client.0.smithi007.stdout:5/939: dwrite d2/d27/d42/da3/d28/f64 [0,4194304] 0 2024-09-21T04:35:40.945 INFO:tasks.workunit.client.0.smithi007.stdout:5/940: stat d2/d27/d42/da3/d23/d49/dd4/dd5/d111/d54/c76 0 2024-09-21T04:35:40.947 INFO:tasks.workunit.client.0.smithi007.stdout:3/939: write dc/d18/f42 [511253,8929] 0 2024-09-21T04:35:40.947 INFO:tasks.workunit.client.0.smithi007.stdout:5/941: chown d2/d27/d42/da3/d23/d49/ff4 16237 1 2024-09-21T04:35:40.948 INFO:tasks.workunit.client.0.smithi007.stdout:0/830: link d6/d75/dc5/d18/d2e/d3f/c104 d6/d75/dc5/c130 0 2024-09-21T04:35:40.950 INFO:tasks.workunit.client.0.smithi007.stdout:5/942: unlink d2/d27/d42/da3/d23/d49/d6f/d104/c125 0 2024-09-21T04:35:40.952 INFO:tasks.workunit.client.0.smithi007.stdout:5/943: chown d2/d27/d42/da3/d23/d49/dd4/de9/f11b 15338168 1 2024-09-21T04:35:40.954 INFO:tasks.workunit.client.0.smithi007.stdout:5/944: creat d2/d27/d42/d53/f133 x:0 0 0 2024-09-21T04:35:40.957 INFO:tasks.workunit.client.0.smithi007.stdout:5/945: creat d2/d27/d42/da3/d23/d2a/f134 x:0 0 0 2024-09-21T04:35:40.977 INFO:tasks.workunit.client.0.smithi007.stdout:0/831: rmdir d6/d75/dc5/d47/d99 39 2024-09-21T04:35:40.977 INFO:tasks.workunit.client.0.smithi007.stdout:2/676: truncate d0/d11/d31/d7b/f89 3057411 0 2024-09-21T04:35:40.978 INFO:tasks.workunit.client.0.smithi007.stdout:6/815: rename d3/d3b/dd3/dd6 to d3/d3e/d99/dbb/d95/dc9/d10b 0 2024-09-21T04:35:40.979 INFO:tasks.workunit.client.0.smithi007.stdout:2/677: stat d0/d11/d31/d39/f50 0 2024-09-21T04:35:40.979 INFO:tasks.workunit.client.0.smithi007.stdout:5/946: truncate d2/d27/d42/da3/d23/d49/f80 2979422 0 2024-09-21T04:35:40.980 INFO:tasks.workunit.client.0.smithi007.stdout:5/947: readlink d2/l7e 0 2024-09-21T04:35:40.981 INFO:tasks.workunit.client.0.smithi007.stdout:2/678: write d0/d11/d31/d7b/fb2 [1005276,72651] 0 2024-09-21T04:35:40.984 INFO:tasks.workunit.client.0.smithi007.stdout:2/679: dread - d0/d11/d6e/d73/f86 zero size 2024-09-21T04:35:40.984 INFO:tasks.workunit.client.0.smithi007.stdout:0/832: creat d6/d75/dc5/d47/d8a/df4/f131 x:0 0 0 2024-09-21T04:35:40.986 INFO:tasks.workunit.client.0.smithi007.stdout:8/792: rename d0/d3/f53 to d0/d3/d16/d93/d5e/de3/d35/feb 0 2024-09-21T04:35:40.986 INFO:tasks.workunit.client.0.smithi007.stdout:0/833: truncate d6/d75/dc5/d47/df0/f11d 775326 0 2024-09-21T04:35:40.994 INFO:tasks.workunit.client.0.smithi007.stdout:8/793: dwrite d0/d3/d74/fac [0,4194304] 0 2024-09-21T04:35:40.998 INFO:tasks.workunit.client.0.smithi007.stdout:2/680: dread d0/f2e [0,4194304] 0 2024-09-21T04:35:41.003 INFO:tasks.workunit.client.0.smithi007.stdout:0/834: rename d6/d75/dc5/d18/d39/c7b to d6/d75/dc5/d52/d10a/c132 0 2024-09-21T04:35:41.008 INFO:tasks.workunit.client.0.smithi007.stdout:8/794: creat d0/d3/d7c/fec x:0 0 0 2024-09-21T04:35:41.009 INFO:tasks.workunit.client.0.smithi007.stdout:8/795: stat d0/d3/c7f 0 2024-09-21T04:35:41.012 INFO:tasks.workunit.client.0.smithi007.stdout:0/835: creat d6/d75/dc5/d52/d10a/f133 x:0 0 0 2024-09-21T04:35:41.014 INFO:tasks.workunit.client.0.smithi007.stdout:8/796: fsync d0/d3/d16/d93/d5e/de3/d4d/fbd 0 2024-09-21T04:35:41.014 INFO:tasks.workunit.client.0.smithi007.stdout:6/816: dwrite d3/d3e/d99/fa5 [0,4194304] 0 2024-09-21T04:35:41.020 INFO:tasks.workunit.client.0.smithi007.stdout:5/948: truncate d2/d27/d42/da3/d23/d49/d6f/ddb/def/d128/fe8 2151341 0 2024-09-21T04:35:41.023 INFO:tasks.workunit.client.0.smithi007.stdout:0/836: creat d6/d91/da3/dbf/dec/f134 x:0 0 0 2024-09-21T04:35:41.027 INFO:tasks.workunit.client.0.smithi007.stdout:8/797: dread d0/d3/fc7 [0,4194304] 0 2024-09-21T04:35:41.029 INFO:tasks.workunit.client.0.smithi007.stdout:6/817: mkdir d3/d3b/dd3/d60/d10c 0 2024-09-21T04:35:41.031 INFO:tasks.workunit.client.0.smithi007.stdout:2/681: dread d0/d11/f4b [4194304,4194304] 0 2024-09-21T04:35:41.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/682: readlink d0/l67 0 2024-09-21T04:35:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.056+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6500000bf0 con 0x7f650c074a80 2024-09-21T04:35:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.058+0000 7f65248a0640 1 -- 172.21.15.7:0/903964695 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f6500000bf0 con 0x7f650c074a80 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f650c074a80 msgr2=0x7f650c076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f650c074a80 0x7f650c076f40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f652019ab90 tx=0x7f6508005f90 comp rx=0 tx=0).stop 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65201023f0 msgr2=0x7f65201996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65201023f0 0x7f65201996e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f65100099d0 tx=0x7f65100045e0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 shutdown_connections 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f650c074a80 0x7f650c076f40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6520104d20 0x7f6520199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 --2- 172.21.15.7:0/903964695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65201023f0 0x7f65201996e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 >> 172.21.15.7:0/903964695 conn(0x7f65200fc010 msgr2=0x7f6520101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 shutdown_connections 2024-09-21T04:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.060+0000 7f65258a2640 1 -- 172.21.15.7:0/903964695 wait complete. 2024-09-21T04:35:41.071 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:35:41.109 INFO:tasks.workunit.client.0.smithi007.stdout:6/818: symlink d3/d3e/l10d 0 2024-09-21T04:35:41.110 INFO:tasks.workunit.client.0.smithi007.stdout:6/819: read d3/de/d12/f8d [3657063,27892] 0 2024-09-21T04:35:41.114 INFO:tasks.workunit.client.0.smithi007.stdout:5/949: getdents d2/d1e 0 2024-09-21T04:35:41.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.146+0000 7f74f960d640 1 -- 172.21.15.7:0/2251105092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4101a60 msgr2=0x7f74f4101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.146+0000 7f74f960d640 1 --2- 172.21.15.7:0/2251105092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4101a60 0x7f74f4101ee0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009880 tx=0x7f74e802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.146+0000 7f74f960d640 1 -- 172.21.15.7:0/2251105092 shutdown_connections 2024-09-21T04:35:41.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.146+0000 7f74f960d640 1 --2- 172.21.15.7:0/2251105092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4101a60 0x7f74f4101ee0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.146+0000 7f74f960d640 1 --2- 172.21.15.7:0/2251105092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f4100860 0x7f74f4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.146+0000 7f74f960d640 1 -- 172.21.15.7:0/2251105092 >> 172.21.15.7:0/2251105092 conn(0x7f74f40fc010 msgr2=0x7f74f40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:41.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.146+0000 7f74f960d640 1 -- 172.21.15.7:0/2251105092 shutdown_connections 2024-09-21T04:35:41.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.146+0000 7f74f960d640 1 -- 172.21.15.7:0/2251105092 wait complete. 2024-09-21T04:35:41.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f960d640 1 Processor -- start 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f960d640 1 -- start start 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f960d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4100860 0x7f74f4104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f960d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f4101a60 0x7f74f4102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f960d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74f41096a0 con 0x7f74f4100860 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f960d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74f4109520 con 0x7f74f4101a60 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4100860 0x7f74f4104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4100860 0x7f74f4104780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53168/0 (socket says 172.21.15.7:53168) 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.147+0000 7f74f2ffd640 1 -- 172.21.15.7:0/3773269229 learned_addr learned my addr 172.21.15.7:0/3773269229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74f27fc640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f4101a60 0x7f74f4102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74f27fc640 1 -- 172.21.15.7:0/3773269229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4100860 msgr2=0x7f74f4104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74f27fc640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4100860 0x7f74f4104780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74f27fc640 1 -- 172.21.15.7:0/3773269229 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f4103380 con 0x7f74f4101a60 2024-09-21T04:35:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74f27fc640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f4101a60 0x7f74f4102e40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f74e8006010 tx=0x7f74e80045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74dbfff640 1 -- 172.21.15.7:0/3773269229 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74e803f070 con 0x7f74f4101a60 2024-09-21T04:35:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74dbfff640 1 -- 172.21.15.7:0/3773269229 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f74e80049d0 con 0x7f74f4101a60 2024-09-21T04:35:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f4103610 con 0x7f74f4101a60 2024-09-21T04:35:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74dbfff640 1 -- 172.21.15.7:0/3773269229 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74e8035030 con 0x7f74f4101a60 2024-09-21T04:35:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.148+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f4103b40 con 0x7f74f4101a60 2024-09-21T04:35:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.149+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f4100c60 con 0x7f74f4101a60 2024-09-21T04:35:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.150+0000 7f74dbfff640 1 -- 172.21.15.7:0/3773269229 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f74e8042070 con 0x7f74f4101a60 2024-09-21T04:35:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.150+0000 7f74dbfff640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f74dc074a30 0x7f74dc076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.151+0000 7f74dbfff640 1 -- 172.21.15.7:0/3773269229 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f74e80b6dc0 con 0x7f74f4101a60 2024-09-21T04:35:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.151+0000 7f74f2ffd640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f74dc074a30 0x7f74dc076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.151+0000 7f74f2ffd640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f74dc074a30 0x7f74dc076ef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f74e40058c0 tx=0x7f74e4005f90 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:41.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.155+0000 7f74dbfff640 1 -- 172.21.15.7:0/3773269229 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f74e8082250 con 0x7f74f4101a60 2024-09-21T04:35:41.206 INFO:tasks.workunit.client.0.smithi007.stdout:6/820: symlink d3/d3e/d99/dbb/d95/dc9/d10b/l10e 0 2024-09-21T04:35:41.206 INFO:tasks.workunit.client.0.smithi007.stdout:0/837: truncate d6/d75/dc5/d18/d27/ff7 470449 0 2024-09-21T04:35:41.210 INFO:tasks.workunit.client.0.smithi007.stdout:5/950: creat d2/d27/d42/da3/d28/f135 x:0 0 0 2024-09-21T04:35:41.216 INFO:tasks.workunit.client.0.smithi007.stdout:5/951: creat d2/d27/d42/da3/d1d/d20/d11c/f136 x:0 0 0 2024-09-21T04:35:41.217 INFO:tasks.workunit.client.0.smithi007.stdout:0/838: dread d6/d75/dc5/d52/d6d/f6f [0,4194304] 0 2024-09-21T04:35:41.220 INFO:tasks.workunit.client.0.smithi007.stdout:6/821: dread d3/d3b/dd3/d60/dd2/fe1 [0,4194304] 0 2024-09-21T04:35:41.221 INFO:tasks.workunit.client.0.smithi007.stdout:6/822: fdatasync d3/d3e/f55 0 2024-09-21T04:35:41.222 INFO:tasks.workunit.client.0.smithi007.stdout:2/683: getdents d0/d11/d31/d46/d6d/d81 0 2024-09-21T04:35:41.226 INFO:tasks.workunit.client.0.smithi007.stdout:0/839: unlink d6/d75/dc5/d47/d8a/d9c/cd8 0 2024-09-21T04:35:41.230 INFO:tasks.workunit.client.0.smithi007.stdout:2/684: mknod d0/d11/db1/ccb 0 2024-09-21T04:35:41.236 INFO:tasks.workunit.client.0.smithi007.stdout:5/952: dread d2/d27/d42/da3/d12c/f74 [0,4194304] 0 2024-09-21T04:35:41.236 INFO:tasks.workunit.client.0.smithi007.stdout:0/840: rename d6/d75/dc5/d57/l74 to d6/d75/dc5/d47/df0/l135 0 2024-09-21T04:35:41.239 INFO:tasks.workunit.client.0.smithi007.stdout:6/823: write d3/de/d12/f7c [321223,61459] 0 2024-09-21T04:35:41.242 INFO:tasks.workunit.client.0.smithi007.stdout:5/953: creat d2/d27/d42/da3/d28/de2/f137 x:0 0 0 2024-09-21T04:35:41.245 INFO:tasks.workunit.client.0.smithi007.stdout:5/954: truncate d2/d27/d42/da3/d28/fa5 933703 0 2024-09-21T04:35:41.247 INFO:tasks.workunit.client.0.smithi007.stdout:6/824: creat d3/d3e/d99/dbb/d95/dc9/d10b/f10f x:0 0 0 2024-09-21T04:35:41.250 INFO:tasks.workunit.client.0.smithi007.stdout:5/955: write d2/d27/d42/da3/d1d/d20/fce [5013913,129741] 0 2024-09-21T04:35:41.252 INFO:tasks.workunit.client.0.smithi007.stdout:2/685: dread d0/d11/d31/d46/d6d/d81/da0/f6 [0,4194304] 0 2024-09-21T04:35:41.259 INFO:tasks.workunit.client.0.smithi007.stdout:2/686: link d0/d11/d31/d39/f8c d0/d11/d31/d39/fcc 0 2024-09-21T04:35:41.260 INFO:tasks.workunit.client.0.smithi007.stdout:0/841: dread d6/d75/dc5/d18/d39/fa7 [0,4194304] 0 2024-09-21T04:35:41.265 INFO:tasks.workunit.client.0.smithi007.stdout:6/825: truncate d3/d3b/dd3/d60/d77/dcb/d78/f84 3126680 0 2024-09-21T04:35:41.267 INFO:tasks.workunit.client.0.smithi007.stdout:2/687: creat d0/d11/d31/d46/d6d/db9/fcd x:0 0 0 2024-09-21T04:35:41.268 INFO:tasks.workunit.client.0.smithi007.stdout:6/826: read d3/d3b/dd3/d60/d77/dcb/d78/f73 [428490,50669] 0 2024-09-21T04:35:41.272 INFO:tasks.workunit.client.0.smithi007.stdout:0/842: write d6/d75/dc5/d18/d27/f9b [1459775,22409] 0 2024-09-21T04:35:41.274 INFO:tasks.workunit.client.0.smithi007.stdout:0/843: write d6/d75/ffd [2067545,6746] 0 2024-09-21T04:35:41.276 INFO:tasks.workunit.client.0.smithi007.stdout:0/844: chown d6/d75/dc5/f28 84 1 2024-09-21T04:35:41.277 INFO:tasks.workunit.client.0.smithi007.stdout:6/827: dwrite d3/de/d12/fe5 [0,4194304] 0 2024-09-21T04:35:41.280 INFO:tasks.workunit.client.0.smithi007.stdout:2/688: getdents d0/d11/d31/d9f 0 2024-09-21T04:35:41.280 INFO:tasks.workunit.client.0.smithi007.stdout:6/828: dread d3/de/d12/d19/f51 [0,4194304] 0 2024-09-21T04:35:41.283 INFO:tasks.workunit.client.0.smithi007.stdout:6/829: creat d3/d94/f110 x:0 0 0 2024-09-21T04:35:41.285 INFO:tasks.workunit.client.0.smithi007.stdout:0/845: rename d6/d75/dc5/d18/d2e/d3f/f7c to d6/d91/da3/dbf/dec/f136 0 2024-09-21T04:35:41.286 INFO:tasks.workunit.client.0.smithi007.stdout:2/689: mkdir d0/d11/db1/dbf/dce 0 2024-09-21T04:35:41.287 INFO:tasks.workunit.client.0.smithi007.stdout:6/830: mknod d3/de/d58/c111 0 2024-09-21T04:35:41.289 INFO:tasks.workunit.client.0.smithi007.stdout:0/846: unlink d6/d75/dc5/d18/d2e/c106 0 2024-09-21T04:35:41.289 INFO:tasks.workunit.client.0.smithi007.stdout:6/831: rmdir d3/d3e/d99 39 2024-09-21T04:35:41.292 INFO:tasks.workunit.client.0.smithi007.stdout:0/847: creat d6/d91/da3/dbf/f137 x:0 0 0 2024-09-21T04:35:41.295 INFO:tasks.workunit.client.0.smithi007.stdout:2/690: link d0/f23 d0/d11/d31/d52/fcf 0 2024-09-21T04:35:41.297 INFO:tasks.workunit.client.0.smithi007.stdout:0/848: readlink d6/d75/dc5/d18/d2e/d3f/l98 0 2024-09-21T04:35:41.298 INFO:tasks.workunit.client.0.smithi007.stdout:6/832: dwrite d3/d3b/dd3/d60/d98/faf [0,4194304] 0 2024-09-21T04:35:41.299 INFO:tasks.workunit.client.0.smithi007.stdout:2/691: rename d0/d11/d31/d46/d6d/d81 to d0/d11/d31/d46/d6d/d81/da0/d57/dd0 22 2024-09-21T04:35:41.303 INFO:tasks.workunit.client.0.smithi007.stdout:2/692: write d0/d11/d31/d39/faf [556369,60927] 0 2024-09-21T04:35:41.307 INFO:tasks.workunit.client.0.smithi007.stdout:0/849: getdents d6/d75/dc5/d57/d117 0 2024-09-21T04:35:41.309 INFO:tasks.workunit.client.0.smithi007.stdout:2/693: mkdir d0/d11/d31/d46/d48/dd1 0 2024-09-21T04:35:41.311 INFO:tasks.workunit.client.0.smithi007.stdout:6/833: mkdir d3/de/d12/d19/da3/de3/d104/d112 0 2024-09-21T04:35:41.314 INFO:tasks.workunit.client.0.smithi007.stdout:0/850: chown d6/d75/dc5/d18/d2e/d3f/l61 1 1 2024-09-21T04:35:41.315 INFO:tasks.workunit.client.0.smithi007.stdout:2/694: dread d0/d11/d31/d46/d48/f82 [0,4194304] 0 2024-09-21T04:35:41.317 INFO:tasks.workunit.client.0.smithi007.stdout:2/695: chown d0/d11/d6e/f90 845 1 2024-09-21T04:35:41.319 INFO:tasks.workunit.client.0.smithi007.stdout:2/696: stat d0/d11/d31/d46/f47 0 2024-09-21T04:35:41.320 INFO:tasks.workunit.client.0.smithi007.stdout:6/834: dwrite d3/d3b/dd3/d60/d98/faf [0,4194304] 0 2024-09-21T04:35:41.323 INFO:tasks.workunit.client.0.smithi007.stdout:2/697: creat d0/d11/d6b/d9d/fd2 x:0 0 0 2024-09-21T04:35:41.326 INFO:tasks.workunit.client.0.smithi007.stdout:6/835: dread - d3/d3e/d99/dbb/d95/dc9/d10b/ff8 zero size 2024-09-21T04:35:41.327 INFO:tasks.workunit.client.0.smithi007.stdout:2/698: mkdir d0/d11/d31/d46/d6d/d81/da0/dd3 0 2024-09-21T04:35:41.332 INFO:tasks.workunit.client.0.smithi007.stdout:2/699: symlink d0/d11/d31/d39/ld4 0 2024-09-21T04:35:41.333 INFO:tasks.workunit.client.0.smithi007.stdout:2/700: dread - d0/d11/d31/d7b/da2/da5/fb5 zero size 2024-09-21T04:35:41.343 INFO:tasks.workunit.client.0.smithi007.stdout:0/851: dwrite d6/d75/dc5/d47/df0/f114 [0,4194304] 0 2024-09-21T04:35:41.353 INFO:tasks.workunit.client.0.smithi007.stdout:2/701: dwrite d0/d11/f4b [0,4194304] 0 2024-09-21T04:35:41.354 INFO:tasks.workunit.client.0.smithi007.stdout:2/702: fsync d0/d11/f28 0 2024-09-21T04:35:41.360 INFO:tasks.workunit.client.0.smithi007.stdout:2/703: symlink d0/d11/d31/ld5 0 2024-09-21T04:35:41.361 INFO:tasks.workunit.client.0.smithi007.stdout:2/704: readlink d0/d11/d31/d46/d6d/d81/da0/l26 0 2024-09-21T04:35:41.366 INFO:tasks.workunit.client.0.smithi007.stdout:0/852: dread d6/d75/dc5/d18/d2e/d60/f63 [4194304,4194304] 0 2024-09-21T04:35:41.373 INFO:tasks.workunit.client.0.smithi007.stdout:2/705: creat d0/d11/d31/d46/fd6 x:0 0 0 2024-09-21T04:35:41.374 INFO:tasks.workunit.client.0.smithi007.stdout:2/706: chown d0/d11/d31/d7b/la6 280 1 2024-09-21T04:35:41.374 INFO:tasks.workunit.client.0.smithi007.stdout:0/853: chown d6/d75/dc5/d47/lcb 1 1 2024-09-21T04:35:41.376 INFO:tasks.workunit.client.0.smithi007.stdout:2/707: dread - d0/d11/d31/d39/f84 zero size 2024-09-21T04:35:41.378 INFO:tasks.workunit.client.0.smithi007.stdout:2/708: write d0/d11/d6e/d73/f86 [25827,99807] 0 2024-09-21T04:35:41.378 INFO:tasks.workunit.client.0.smithi007.stdout:2/709: chown d0/d11/d6b/fc1 2164 1 2024-09-21T04:35:41.379 INFO:tasks.workunit.client.0.smithi007.stdout:2/710: chown d0/d11/d31/d46/d6c/l97 24244460 1 2024-09-21T04:35:41.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.446+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f74f406a930 con 0x7f74dc074a30 2024-09-21T04:35:41.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.448+0000 7f74dbfff640 1 -- 172.21.15.7:0/3773269229 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f74f406a930 con 0x7f74dc074a30 2024-09-21T04:35:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f74dc074a30 msgr2=0x7f74dc076ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f74dc074a30 0x7f74dc076ef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f74e40058c0 tx=0x7f74e4005f90 comp rx=0 tx=0).stop 2024-09-21T04:35:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f4101a60 msgr2=0x7f74f4102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f4101a60 0x7f74f4102e40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f74e8006010 tx=0x7f74e80045e0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 shutdown_connections 2024-09-21T04:35:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f74dc074a30 0x7f74dc076ef0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f4101a60 0x7f74f4102e40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 --2- 172.21.15.7:0/3773269229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f4100860 0x7f74f4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 >> 172.21.15.7:0/3773269229 conn(0x7f74f40fc010 msgr2=0x7f74f406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:41.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 shutdown_connections 2024-09-21T04:35:41.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.450+0000 7f74f960d640 1 -- 172.21.15.7:0/3773269229 wait complete. 2024-09-21T04:35:41.469 INFO:tasks.workunit.client.0.smithi007.stdout:7/931: sync 2024-09-21T04:35:41.533 INFO:tasks.workunit.client.0.smithi007.stdout:7/932: rename d3/l6c to d3/d6/d85/dad/l144 0 2024-09-21T04:35:41.534 INFO:tasks.workunit.client.0.smithi007.stdout:2/711: mknod d0/d11/db1/dbf/dce/cd7 0 2024-09-21T04:35:41.535 INFO:tasks.workunit.client.0.smithi007.stdout:7/933: symlink d3/d6/d10/d4f/l145 0 2024-09-21T04:35:41.537 INFO:tasks.workunit.client.0.smithi007.stdout:2/712: rename d0/d11/db8/lc8 to d0/d11/d31/d46/d77/ld8 0 2024-09-21T04:35:41.540 INFO:tasks.workunit.client.0.smithi007.stdout:2/713: creat d0/d11/d31/d46/d6d/fd9 x:0 0 0 2024-09-21T04:35:41.542 INFO:tasks.workunit.client.0.smithi007.stdout:2/714: rmdir d0/d11/d31/d46/d6d/daa/dbd 39 2024-09-21T04:35:41.545 INFO:tasks.workunit.client.0.smithi007.stdout:7/934: rmdir d3/d6/d14/d20/d80/d143 0 2024-09-21T04:35:41.545 INFO:tasks.workunit.client.0.smithi007.stdout:2/715: truncate d0/d11/d31/d46/d6d/db9/fcd 608295 0 2024-09-21T04:35:41.548 INFO:tasks.workunit.client.0.smithi007.stdout:2/716: creat d0/d32/fda x:0 0 0 2024-09-21T04:35:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.565+0000 7fe82b4f3640 1 -- 172.21.15.7:0/1415412603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 msgr2=0x7fe8241071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.565+0000 7fe82b4f3640 1 --2- 172.21.15.7:0/1415412603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 0x7fe8241071d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe814009880 tx=0x7fe81402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.565+0000 7fe82b4f3640 1 -- 172.21.15.7:0/1415412603 shutdown_connections 2024-09-21T04:35:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.565+0000 7fe82b4f3640 1 --2- 172.21.15.7:0/1415412603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 0x7fe8241071d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.565+0000 7fe82b4f3640 1 --2- 172.21.15.7:0/1415412603 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8241024b0 0x7fe8241048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.565+0000 7fe82b4f3640 1 -- 172.21.15.7:0/1415412603 >> 172.21.15.7:0/1415412603 conn(0x7fe8240fc010 msgr2=0x7fe8240fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.565+0000 7fe82b4f3640 1 -- 172.21.15.7:0/1415412603 shutdown_connections 2024-09-21T04:35:41.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.565+0000 7fe82b4f3640 1 -- 172.21.15.7:0/1415412603 wait complete. 2024-09-21T04:35:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe82b4f3640 1 Processor -- start 2024-09-21T04:35:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe82b4f3640 1 -- start start 2024-09-21T04:35:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe82b4f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8241024b0 0x7fe824197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe82b4f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 0x7fe824197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe82b4f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410a780 con 0x7fe824104de0 2024-09-21T04:35:41.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe82b4f3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410a600 con 0x7fe8241024b0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe828a8e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 0x7fe824197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe828a8e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 0x7fe824197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53188/0 (socket says 172.21.15.7:53188) 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe828a8e640 1 -- 172.21.15.7:0/2995940957 learned_addr learned my addr 172.21.15.7:0/2995940957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.566+0000 7fe82928f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8241024b0 0x7fe824197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe82928f640 1 -- 172.21.15.7:0/2995940957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 msgr2=0x7fe824197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe82928f640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 0x7fe824197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe82928f640 1 -- 172.21.15.7:0/2995940957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8241980d0 con 0x7fe8241024b0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe828a8e640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 0x7fe824197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe82928f640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8241024b0 0x7fe824197510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe82000e970 tx=0x7fe82000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe81a7fc640 1 -- 172.21.15.7:0/2995940957 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe82000cac0 con 0x7fe8241024b0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82419cb10 con 0x7fe8241024b0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe81a7fc640 1 -- 172.21.15.7:0/2995940957 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe82000cc60 con 0x7fe8241024b0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe81a7fc640 1 -- 172.21.15.7:0/2995940957 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe820018720 con 0x7fe8241024b0 2024-09-21T04:35:41.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.567+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82419d0a0 con 0x7fe8241024b0 2024-09-21T04:35:41.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.568+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe824102890 con 0x7fe8241024b0 2024-09-21T04:35:41.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.569+0000 7fe81a7fc640 1 -- 172.21.15.7:0/2995940957 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7fe820010430 con 0x7fe8241024b0 2024-09-21T04:35:41.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.569+0000 7fe81a7fc640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fe810074a30 0x7fe810076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.570+0000 7fe81a7fc640 1 -- 172.21.15.7:0/2995940957 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7fe8200968b0 con 0x7fe8241024b0 2024-09-21T04:35:41.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.570+0000 7fe828a8e640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fe810074a30 0x7fe810076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.570+0000 7fe828a8e640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fe810074a30 0x7fe810076ef0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe8140096c0 tx=0x7fe8140058d0 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:41.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.574+0000 7fe81a7fc640 1 -- 172.21.15.7:0/2995940957 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fe820061d40 con 0x7fe8241024b0 2024-09-21T04:35:41.636 INFO:tasks.workunit.client.0.smithi007.stdout:8/798: sync 2024-09-21T04:35:41.636 INFO:tasks.workunit.client.0.smithi007.stdout:3/940: sync 2024-09-21T04:35:41.637 INFO:tasks.workunit.client.0.smithi007.stdout:8/799: chown d0/d3/d82/ca1 135819990 1 2024-09-21T04:35:41.644 INFO:tasks.workunit.client.0.smithi007.stdout:3/941: getdents dc/d22/d29/d32/d4d/d84/dbb/dc1/dca/d10e 0 2024-09-21T04:35:41.645 INFO:tasks.workunit.client.0.smithi007.stdout:8/800: write d0/d3/d16/d93/d5e/de3/d4d/f72 [665636,108605] 0 2024-09-21T04:35:41.648 INFO:tasks.workunit.client.0.smithi007.stdout:8/801: mknod d0/d3/d7c/dbf/ced 0 2024-09-21T04:35:41.649 INFO:tasks.workunit.client.0.smithi007.stdout:3/942: write dc/d18/d26/f51 [817193,66217] 0 2024-09-21T04:35:41.654 INFO:tasks.workunit.client.0.smithi007.stdout:8/802: creat d0/d3/d16/d93/d5e/de3/d4d/d5d/fee x:0 0 0 2024-09-21T04:35:41.655 INFO:tasks.workunit.client.0.smithi007.stdout:8/803: write d0/f43 [1319056,76580] 0 2024-09-21T04:35:41.669 INFO:tasks.workunit.client.0.smithi007.stdout:8/804: write d0/d3/d8/d46/f63 [475606,71871] 0 2024-09-21T04:35:41.672 INFO:tasks.workunit.client.0.smithi007.stdout:8/805: creat d0/d3/d16/d93/d40/d9e/fef x:0 0 0 2024-09-21T04:35:41.673 INFO:tasks.workunit.client.0.smithi007.stdout:8/806: chown d0/d3/d16/d93/d5e/fe7 2455649 1 2024-09-21T04:35:41.682 INFO:tasks.workunit.client.0.smithi007.stdout:8/807: dwrite d0/d3/d16/d93/d5e/de3/f85 [4194304,4194304] 0 2024-09-21T04:35:41.689 INFO:tasks.workunit.client.0.smithi007.stdout:8/808: getdents d0/d3/d82 0 2024-09-21T04:35:41.696 INFO:tasks.workunit.client.0.smithi007.stdout:8/809: dwrite d0/d3/d16/d93/d40/d92/dcb/fdd [0,4194304] 0 2024-09-21T04:35:41.699 INFO:tasks.workunit.client.0.smithi007.stdout:8/810: fdatasync d0/d3/d16/d93/d5e/de3/d4d/f9b 0 2024-09-21T04:35:41.701 INFO:tasks.workunit.client.0.smithi007.stdout:8/811: write d0/d3/d16/d93/d5e/de3/d4d/d5d/fab [269672,15006] 0 2024-09-21T04:35:41.703 INFO:tasks.workunit.client.0.smithi007.stdout:8/812: write d0/d3/d7c/f54 [1239208,1043] 0 2024-09-21T04:35:41.711 INFO:tasks.workunit.client.0.smithi007.stdout:8/813: truncate d0/d3/d16/f1b 1900360 0 2024-09-21T04:35:41.712 INFO:tasks.workunit.client.0.smithi007.stdout:8/814: fdatasync d0/d3/d7c/dbf/fe4 0 2024-09-21T04:35:41.715 INFO:tasks.workunit.client.0.smithi007.stdout:8/815: rename d0/l13 to d0/d3/d16/d93/d40/d9e/lf0 0 2024-09-21T04:35:41.716 INFO:tasks.workunit.client.0.smithi007.stdout:8/816: symlink d0/d3/d82/lf1 0 2024-09-21T04:35:41.717 INFO:tasks.workunit.client.0.smithi007.stdout:8/817: write d0/d3/d16/d93/d40/d92/f9a [624461,2515] 0 2024-09-21T04:35:41.721 INFO:tasks.workunit.client.0.smithi007.stdout:8/818: unlink d0/d3/d16/d93/d40/d92/f9a 0 2024-09-21T04:35:41.728 INFO:tasks.workunit.client.0.smithi007.stdout:8/819: dwrite d0/d3/d74/d52/fd6 [0,4194304] 0 2024-09-21T04:35:41.731 INFO:tasks.workunit.client.0.smithi007.stdout:8/820: unlink d0/d3/d16/d93/d5e/de3/d4d/d5d/c7a 0 2024-09-21T04:35:41.737 INFO:tasks.workunit.client.0.smithi007.stdout:8/821: mkdir d0/d3/d16/d93/d40/d92/dcb/df2 0 2024-09-21T04:35:41.739 INFO:tasks.workunit.client.0.smithi007.stdout:8/822: write d0/d3/d16/d93/d5e/de3/d35/feb [2016322,120385] 0 2024-09-21T04:35:41.754 INFO:tasks.workunit.client.0.smithi007.stdout:8/823: unlink d0/d3/d16/d93/d5e/de3/d4d/fbd 0 2024-09-21T04:35:41.756 INFO:tasks.workunit.client.0.smithi007.stdout:8/824: symlink d0/d3/daa/d9d/lf3 0 2024-09-21T04:35:41.769 INFO:tasks.workunit.client.0.smithi007.stdout:8/825: dread d0/d3/d8/f3f [0,4194304] 0 2024-09-21T04:35:41.778 INFO:tasks.workunit.client.0.smithi007.stdout:8/826: fsync d0/d3/d8/f47 0 2024-09-21T04:35:41.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/827: write d0/d3/d16/d93/d5e/de3/d4d/d5d/fdf [4075205,10742] 0 2024-09-21T04:35:41.784 INFO:tasks.workunit.client.0.smithi007.stdout:8/828: unlink d0/d3/d8/f3f 0 2024-09-21T04:35:41.792 INFO:tasks.workunit.client.0.smithi007.stdout:8/829: dwrite d0/fdb [0,4194304] 0 2024-09-21T04:35:41.795 INFO:tasks.workunit.client.0.smithi007.stdout:8/830: fdatasync d0/d3/d8/fa 0 2024-09-21T04:35:41.796 INFO:tasks.workunit.client.0.smithi007.stdout:8/831: chown d0/d3/d16/c33 3 1 2024-09-21T04:35:41.798 INFO:tasks.workunit.client.0.smithi007.stdout:8/832: chown d0/d3/d16/d93/d5e/ld0 1031037 1 2024-09-21T04:35:41.801 INFO:tasks.workunit.client.0.smithi007.stdout:8/833: rename d0/d3/d16/d93/d5e/caf to d0/d3/d16/d93/d40/d92/dcb/cf4 0 2024-09-21T04:35:41.802 INFO:tasks.workunit.client.0.smithi007.stdout:8/834: chown d0/fdb 64 1 2024-09-21T04:35:41.804 INFO:tasks.workunit.client.0.smithi007.stdout:8/835: creat d0/d3/d16/d93/d5e/de3/d4d/d5d/dba/ff5 x:0 0 0 2024-09-21T04:35:41.807 INFO:tasks.workunit.client.0.smithi007.stdout:8/836: mkdir d0/d3/d16/d93/d5e/de3/d35/df6 0 2024-09-21T04:35:41.807 INFO:tasks.workunit.client.0.smithi007.stdout:8/837: chown d0/d3/d7c/c3d 330499 1 2024-09-21T04:35:41.813 INFO:tasks.workunit.client.0.smithi007.stdout:8/838: dwrite d0/d3/d8/f9 [0,4194304] 0 2024-09-21T04:35:41.814 INFO:tasks.workunit.client.0.smithi007.stdout:8/839: stat d0/d3/d16/d93/d5e/de3/d35/f8e 0 2024-09-21T04:35:41.821 INFO:tasks.workunit.client.0.smithi007.stdout:8/840: getdents d0/d3/d16/d93/d5e/de3/d4d/d5d 0 2024-09-21T04:35:41.825 INFO:tasks.workunit.client.0.smithi007.stdout:8/841: dread d0/d3/d8/d46/db7/fe2 [0,4194304] 0 2024-09-21T04:35:41.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.874+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe8241055c0 con 0x7fe810074a30 2024-09-21T04:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.884+0000 7fe81a7fc640 1 -- 172.21.15.7:0/2995940957 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fe8241055c0 con 0x7fe810074a30 2024-09-21T04:35:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:35:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (7m) 69s ago 8m 24.6M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:35:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 starting - - - - 2024-09-21T04:35:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 starting - - - - 2024-09-21T04:35:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (8m) 69s ago 8m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:35:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (7m) 66s ago 7m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (7m) 69s ago 8m 77.5M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (3m) 69s ago 3m 122M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (3m) 69s ago 3m 103M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (3m) 66s ago 3m 19.7M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (3m) 66s ago 3m 25.7M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:9283,8765,8443 running (10m) 69s ago 10m 250M - 19.1.1-373-g4e45e6b4 164465d2334a c324ee791e08 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (90s) 66s ago 7m 499M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7d35aae8ab52 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (10m) 69s ago 10m 52.9M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (7m) 66s ago 7m 43.9M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (8m) 69s ago 8m 15.8M - 1.5.0 0da6a335fe13 b4a8255a594a 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (7m) 66s ago 7m 16.8M - 1.5.0 0da6a335fe13 88246b15ca5d 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (6m) 69s ago 6m 70.3M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (6m) 69s ago 6m 78.0M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (5m) 69s ago 5m 73.5M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (5m) 66s ago 5m 76.8M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (4m) 66s ago 4m 80.3M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:35:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (4m) 66s ago 4m 64.5M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:35:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 starting - - - - 2024-09-21T04:35:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fe810074a30 msgr2=0x7fe810076ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fe810074a30 0x7fe810076ef0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe8140096c0 tx=0x7fe8140058d0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8241024b0 msgr2=0x7fe824197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8241024b0 0x7fe824197510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe82000e970 tx=0x7fe82000ee30 comp rx=0 tx=0).stop 2024-09-21T04:35:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 shutdown_connections 2024-09-21T04:35:41.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fe810074a30 0x7fe810076ef0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824104de0 0x7fe824197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 --2- 172.21.15.7:0/2995940957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8241024b0 0x7fe824197510 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 >> 172.21.15.7:0/2995940957 conn(0x7fe8240fc010 msgr2=0x7fe8240fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:41.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 shutdown_connections 2024-09-21T04:35:41.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.887+0000 7fe82b4f3640 1 -- 172.21.15.7:0/2995940957 wait complete. 2024-09-21T04:35:41.922 INFO:tasks.workunit.client.0.smithi007.stdout:5/956: sync 2024-09-21T04:35:41.928 INFO:tasks.workunit.client.0.smithi007.stdout:5/957: dwrite d2/d27/d42/da3/d28/f4d [0,4194304] 0 2024-09-21T04:35:41.930 INFO:tasks.workunit.client.0.smithi007.stdout:5/958: chown d2/d27/d42/da3/d1d/d20/f6d 19443 1 2024-09-21T04:35:41.931 INFO:tasks.workunit.client.0.smithi007.stdout:5/959: stat d2/d27/d42/da3/d23/d49/dd4/dfa 0 2024-09-21T04:35:41.931 INFO:tasks.workunit.client.0.smithi007.stdout:5/960: write d2/d27/d42/d53/f133 [781787,11192] 0 2024-09-21T04:35:41.934 INFO:tasks.workunit.client.0.smithi007.stdout:5/961: mkdir d2/d27/d42/da3/d28/de2/d138 0 2024-09-21T04:35:41.935 INFO:tasks.workunit.client.0.smithi007.stdout:5/962: readlink d2/d1e/ld3 0 2024-09-21T04:35:41.943 INFO:tasks.workunit.client.0.smithi007.stdout:5/963: dwrite d2/d27/d42/da3/d23/d49/d6f/ddb/f10a [0,4194304] 0 2024-09-21T04:35:41.948 INFO:tasks.workunit.client.0.smithi007.stdout:5/964: mkdir d2/d27/d42/da3/d23/d49/dd4/dd5/d139 0 2024-09-21T04:35:41.949 INFO:tasks.workunit.client.0.smithi007.stdout:5/965: chown d2/f55 34764 1 2024-09-21T04:35:41.952 INFO:tasks.workunit.client.0.smithi007.stdout:5/966: readlink d2/d27/d42/da3/d23/l26 0 2024-09-21T04:35:41.952 INFO:tasks.workunit.client.0.smithi007.stdout:5/967: dread - d2/f8d zero size 2024-09-21T04:35:41.954 INFO:tasks.workunit.client.0.smithi007.stdout:5/968: write d2/d27/d42/da3/f16 [3000168,48919] 0 2024-09-21T04:35:41.956 INFO:tasks.workunit.client.0.smithi007.stdout:5/969: read d2/d27/d42/da3/d23/d49/f6a [129932,46232] 0 2024-09-21T04:35:41.965 INFO:tasks.workunit.client.0.smithi007.stdout:5/970: creat d2/d27/d42/da3/d23/d49/dd4/f13a x:0 0 0 2024-09-21T04:35:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.988+0000 7fdc8029d640 1 -- 172.21.15.7:0/1630066650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78102b60 msgr2=0x7fdc78102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.988+0000 7fdc8029d640 1 --2- 172.21.15.7:0/1630066650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78102b60 0x7fdc78102fe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009880 tx=0x7fdc6802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.988+0000 7fdc8029d640 1 -- 172.21.15.7:0/1630066650 shutdown_connections 2024-09-21T04:35:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.988+0000 7fdc8029d640 1 --2- 172.21.15.7:0/1630066650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78102b60 0x7fdc78102fe0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.988+0000 7fdc8029d640 1 --2- 172.21.15.7:0/1630066650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78101960 0x7fdc78101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.988+0000 7fdc8029d640 1 -- 172.21.15.7:0/1630066650 >> 172.21.15.7:0/1630066650 conn(0x7fdc780fd0d0 msgr2=0x7fdc780ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.988+0000 7fdc8029d640 1 -- 172.21.15.7:0/1630066650 shutdown_connections 2024-09-21T04:35:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.988+0000 7fdc8029d640 1 -- 172.21.15.7:0/1630066650 wait complete. 2024-09-21T04:35:41.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc8029d640 1 Processor -- start 2024-09-21T04:35:41.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc8029d640 1 -- start start 2024-09-21T04:35:41.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc8029d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78101960 0x7fdc78197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc8029d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102b60 0x7fdc781979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc8029d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7810a780 con 0x7fdc78101960 2024-09-21T04:35:41.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc8029d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7810a600 con 0x7fdc78102b60 2024-09-21T04:35:41.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc7e039640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78101960 0x7fdc78197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc7e039640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78101960 0x7fdc78197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53206/0 (socket says 172.21.15.7:53206) 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.989+0000 7fdc7e039640 1 -- 172.21.15.7:0/986700087 learned_addr learned my addr 172.21.15.7:0/986700087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc7d838640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102b60 0x7fdc781979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc7e039640 1 -- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102b60 msgr2=0x7fdc781979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc7e039640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102b60 0x7fdc781979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc7e039640 1 -- 172.21.15.7:0/986700087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc78198030 con 0x7fdc78101960 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc7d838640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102b60 0x7fdc781979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc7e039640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78101960 0x7fdc78197470 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400e940 tx=0x7fdc7400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc6f7fe640 1 -- 172.21.15.7:0/986700087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdc7400cac0 con 0x7fdc78101960 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc6f7fe640 1 -- 172.21.15.7:0/986700087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc7400cc60 con 0x7fdc78101960 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7819ca70 con 0x7fdc78101960 2024-09-21T04:35:41.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.990+0000 7fdc6f7fe640 1 -- 172.21.15.7:0/986700087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdc74003de0 con 0x7fdc78101960 2024-09-21T04:35:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.991+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7819cfd0 con 0x7fdc78101960 2024-09-21T04:35:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.992+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc78101d60 con 0x7fdc78101960 2024-09-21T04:35:41.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.992+0000 7fdc6f7fe640 1 -- 172.21.15.7:0/986700087 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7fdc74003f80 con 0x7fdc78101960 2024-09-21T04:35:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.993+0000 7fdc6f7fe640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fdc64074a80 0x7fdc64076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.993+0000 7fdc6f7fe640 1 -- 172.21.15.7:0/986700087 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7fdc74096960 con 0x7fdc78101960 2024-09-21T04:35:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.993+0000 7fdc7d838640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fdc64074a80 0x7fdc64076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:41.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:41.994+0000 7fdc7d838640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fdc64074a80 0x7fdc64076f40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdc68000c00 tx=0x7fdc680058d0 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.000+0000 7fdc6f7fe640 1 -- 172.21.15.7:0/986700087 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fdc74061f10 con 0x7fdc78101960 2024-09-21T04:35:42.104 INFO:tasks.workunit.client.0.smithi007.stdout:6/836: sync 2024-09-21T04:35:42.105 INFO:tasks.workunit.client.0.smithi007.stdout:6/837: dread - d3/d3e/d99/dbb/d95/dc9/ffb zero size 2024-09-21T04:35:42.119 INFO:tasks.workunit.client.0.smithi007.stdout:6/838: dread d3/d3b/f79 [0,4194304] 0 2024-09-21T04:35:42.122 INFO:tasks.workunit.client.0.smithi007.stdout:6/839: chown d3/d3b/dd3/d60/d77/dcb/d88/fc5 19 1 2024-09-21T04:35:42.126 INFO:tasks.workunit.client.0.smithi007.stdout:6/840: symlink d3/de/d12/l113 0 2024-09-21T04:35:42.130 INFO:tasks.workunit.client.0.smithi007.stdout:6/841: dread d3/de/d12/f7c [0,4194304] 0 2024-09-21T04:35:42.141 INFO:tasks.workunit.client.0.smithi007.stdout:6/842: dread d3/d3b/dd3/d60/d77/dcb/d78/fb0 [0,4194304] 0 2024-09-21T04:35:42.154 INFO:tasks.workunit.client.0.smithi007.stdout:6/843: dread d3/de/d58/faa [0,4194304] 0 2024-09-21T04:35:42.156 INFO:tasks.workunit.client.0.smithi007.stdout:6/844: mknod d3/d3b/dd3/d60/dd2/c114 0 2024-09-21T04:35:42.157 INFO:tasks.workunit.client.0.smithi007.stdout:6/845: fsync d3/d3e/d99/dbb/f9f 0 2024-09-21T04:35:42.159 INFO:tasks.workunit.client.0.smithi007.stdout:6/846: symlink d3/d3b/dd3/d60/d98/l115 0 2024-09-21T04:35:42.163 INFO:tasks.workunit.client.0.smithi007.stdout:6/847: mkdir d3/de/d12/d19/da3/de3/d104/d112/d116 0 2024-09-21T04:35:42.169 INFO:tasks.workunit.client.0.smithi007.stdout:6/848: dread d3/d3e/fd1 [0,4194304] 0 2024-09-21T04:35:42.171 INFO:tasks.workunit.client.0.smithi007.stdout:6/849: fsync d3/d3b/f83 0 2024-09-21T04:35:42.184 INFO:tasks.workunit.client.0.smithi007.stdout:6/850: dread d3/de/d12/d19/f24 [0,4194304] 0 2024-09-21T04:35:42.186 INFO:tasks.workunit.client.0.smithi007.stdout:6/851: symlink d3/d3b/dd3/d60/d77/l117 0 2024-09-21T04:35:42.187 INFO:tasks.workunit.client.0.smithi007.stdout:6/852: chown d3/de/d58/l89 40 1 2024-09-21T04:35:42.267 INFO:tasks.workunit.client.0.smithi007.stdout:0/854: sync 2024-09-21T04:35:42.267 INFO:tasks.workunit.client.0.smithi007.stdout:7/935: sync 2024-09-21T04:35:42.270 INFO:tasks.workunit.client.0.smithi007.stdout:7/936: fdatasync d3/d6/d2e/d63/d6a/d81/f96 0 2024-09-21T04:35:42.274 INFO:tasks.workunit.client.0.smithi007.stdout:0/855: getdents d6/d75 0 2024-09-21T04:35:42.276 INFO:tasks.workunit.client.0.smithi007.stdout:7/937: rename d3/d6/d2e/d63/d6a/d81/fa0 to d3/d6/d85/dad/f146 0 2024-09-21T04:35:42.281 INFO:tasks.workunit.client.0.smithi007.stdout:0/856: rename d6/d91/da3/dbf/dec/f134 to d6/d75/dc5/d47/d8a/de9/d101/d109/d126/f138 0 2024-09-21T04:35:42.285 INFO:tasks.workunit.client.0.smithi007.stdout:7/938: mkdir d3/d6/d40/da1/d147 0 2024-09-21T04:35:42.295 INFO:tasks.workunit.client.0.smithi007.stdout:0/857: symlink d6/d75/dc5/d47/d8a/de9/d101/d109/l139 0 2024-09-21T04:35:42.295 INFO:tasks.workunit.client.0.smithi007.stdout:0/858: readlink d6/d75/dc5/d18/d39/l80 0 2024-09-21T04:35:42.299 INFO:tasks.workunit.client.0.smithi007.stdout:7/939: creat d3/d6/d14/d20/d102/f148 x:0 0 0 2024-09-21T04:35:42.299 INFO:tasks.workunit.client.0.smithi007.stdout:7/940: chown d3/d6/d40/da1/f10f 11 1 2024-09-21T04:35:42.316 INFO:tasks.workunit.client.0.smithi007.stdout:7/941: dwrite d3/d6/fa [0,4194304] 0 2024-09-21T04:35:42.318 INFO:tasks.workunit.client.0.smithi007.stdout:7/942: mknod d3/d6/d14/d20/d138/c149 0 2024-09-21T04:35:42.321 INFO:tasks.workunit.client.0.smithi007.stdout:7/943: rmdir d3/d6/d14/d20/d80/dca/d113 39 2024-09-21T04:35:42.323 INFO:tasks.workunit.client.0.smithi007.stdout:7/944: read d3/d6/d40/da1/fe5 [660982,88886] 0 2024-09-21T04:35:42.325 INFO:tasks.workunit.client.0.smithi007.stdout:7/945: fdatasync d3/d6/d14/d20/d80/f9d 0 2024-09-21T04:35:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.327+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fdc78062df0 con 0x7fdc78101960 2024-09-21T04:35:42.330 INFO:tasks.workunit.client.0.smithi007.stdout:7/946: creat d3/d6/d14/d20/d80/d129/f14a x:0 0 0 2024-09-21T04:35:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.328+0000 7fdc6f7fe640 1 -- 172.21.15.7:0/986700087 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+841 (secure 0 0 0) 0x7fdc74066dc0 con 0x7fdc78101960 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:35:42.331 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:35:42.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:35:42.332 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:35:42.332 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 13, 2024-09-21T04:35:42.332 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:35:42.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:35:42.332 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:35:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fdc64074a80 msgr2=0x7fdc64076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fdc64074a80 0x7fdc64076f40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdc68000c00 tx=0x7fdc680058d0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78101960 msgr2=0x7fdc78197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78101960 0x7fdc78197470 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400e940 tx=0x7fdc7400ee00 comp rx=0 tx=0).stop 2024-09-21T04:35:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 shutdown_connections 2024-09-21T04:35:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7fdc64074a80 0x7fdc64076f40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc78102b60 0x7fdc781979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 --2- 172.21.15.7:0/986700087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc78101960 0x7fdc78197470 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 >> 172.21.15.7:0/986700087 conn(0x7fdc780fd0d0 msgr2=0x7fdc78105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 shutdown_connections 2024-09-21T04:35:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.331+0000 7fdc8029d640 1 -- 172.21.15.7:0/986700087 wait complete. 2024-09-21T04:35:42.335 INFO:tasks.workunit.client.0.smithi007.stdout:7/947: dread d3/d6/d14/d20/d80/dca/ffa [0,4194304] 0 2024-09-21T04:35:42.337 INFO:tasks.workunit.client.0.smithi007.stdout:7/948: creat d3/d6/d13/f14b x:0 0 0 2024-09-21T04:35:42.354 INFO:tasks.workunit.client.0.smithi007.stdout:7/949: dwrite d3/d6/d85/de7/d11b/d66/fa3 [0,4194304] 0 2024-09-21T04:35:42.360 INFO:tasks.workunit.client.0.smithi007.stdout:7/950: link d3/d6/cb d3/d6/d2e/d5e/c14c 0 2024-09-21T04:35:42.362 INFO:tasks.workunit.client.0.smithi007.stdout:7/951: rmdir d3/d6/d14/d20/d58 39 2024-09-21T04:35:42.364 INFO:tasks.workunit.client.0.smithi007.stdout:7/952: symlink d3/d6/d2e/l14d 0 2024-09-21T04:35:42.368 INFO:tasks.workunit.client.0.smithi007.stdout:7/953: getdents d3/d6/d14/d20/d138 0 2024-09-21T04:35:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.444+0000 7f1c610cd640 1 -- 172.21.15.7:0/718981227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c102b60 msgr2=0x7f1c5c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.444+0000 7f1c610cd640 1 --2- 172.21.15.7:0/718981227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c102b60 0x7f1c5c102fe0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009880 tx=0x7f1c5002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.444+0000 7f1c610cd640 1 -- 172.21.15.7:0/718981227 shutdown_connections 2024-09-21T04:35:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.444+0000 7f1c610cd640 1 --2- 172.21.15.7:0/718981227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c102b60 0x7f1c5c102fe0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.444+0000 7f1c610cd640 1 --2- 172.21.15.7:0/718981227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c101960 0x7f1c5c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.444+0000 7f1c610cd640 1 -- 172.21.15.7:0/718981227 >> 172.21.15.7:0/718981227 conn(0x7f1c5c0fd0f0 msgr2=0x7f1c5c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.444+0000 7f1c610cd640 1 -- 172.21.15.7:0/718981227 shutdown_connections 2024-09-21T04:35:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.444+0000 7f1c610cd640 1 -- 172.21.15.7:0/718981227 wait complete. 2024-09-21T04:35:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c610cd640 1 Processor -- start 2024-09-21T04:35:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c610cd640 1 -- start start 2024-09-21T04:35:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c610cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c101960 0x7f1c5c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c610cd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102b60 0x7f1c5c199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c610cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c10a820 con 0x7f1c5c101960 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c610cd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c10a6a0 con 0x7f1c5c102b60 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c5ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c101960 0x7f1c5c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c5ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c101960 0x7f1c5c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53236/0 (socket says 172.21.15.7:53236) 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.445+0000 7f1c5ad9d640 1 -- 172.21.15.7:0/4001617406 learned_addr learned my addr 172.21.15.7:0/4001617406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c5a59c640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102b60 0x7f1c5c199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c5ad9d640 1 -- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102b60 msgr2=0x7f1c5c199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c5ad9d640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102b60 0x7f1c5c199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c5ad9d640 1 -- 172.21.15.7:0/4001617406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c19a2d0 con 0x7f1c5c101960 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c5ad9d640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c101960 0x7f1c5c199710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00b3d0 tx=0x7f1c4c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c43fff640 1 -- 172.21.15.7:0/4001617406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1c4c015020 con 0x7f1c5c101960 2024-09-21T04:35:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c43fff640 1 -- 172.21.15.7:0/4001617406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c4c011af0 con 0x7f1c5c101960 2024-09-21T04:35:42.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c19ed10 con 0x7f1c5c101960 2024-09-21T04:35:42.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c43fff640 1 -- 172.21.15.7:0/4001617406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1c4c010da0 con 0x7f1c5c101960 2024-09-21T04:35:42.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.446+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c19f260 con 0x7f1c5c101960 2024-09-21T04:35:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.447+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c101d60 con 0x7f1c5c101960 2024-09-21T04:35:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.448+0000 7f1c43fff640 1 -- 172.21.15.7:0/4001617406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 98034+0+0 (secure 0 0 0) 0x7f1c4c018020 con 0x7f1c5c101960 2024-09-21T04:35:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.448+0000 7f1c43fff640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f1c44074a30 0x7f1c44076ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:35:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.448+0000 7f1c43fff640 1 -- 172.21.15.7:0/4001617406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5812+0+0 (secure 0 0 0) 0x7f1c4c095dc0 con 0x7f1c5c101960 2024-09-21T04:35:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.449+0000 7f1c5a59c640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f1c44074a30 0x7f1c44076ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:35:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.449+0000 7f1c5a59c640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f1c44074a30 0x7f1c44076ef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c19abc0 tx=0x7f1c50005a20 comp rx=0 tx=0).ready entity=mgr.24429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:35:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.453+0000 7f1c43fff640 1 -- 172.21.15.7:0/4001617406 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f1c4c01f020 con 0x7f1c5c101960 2024-09-21T04:35:42.472 INFO:tasks.workunit.client.0.smithi007.stdout:3/943: sync 2024-09-21T04:35:42.472 INFO:tasks.workunit.client.0.smithi007.stdout:5/971: sync 2024-09-21T04:35:42.472 INFO:tasks.workunit.client.0.smithi007.stdout:2/717: sync 2024-09-21T04:35:42.472 INFO:tasks.workunit.client.0.smithi007.stdout:8/842: sync 2024-09-21T04:35:42.535 INFO:tasks.workunit.client.0.smithi007.stdout:3/944: creat dc/d22/d29/f127 x:0 0 0 2024-09-21T04:35:42.538 INFO:tasks.workunit.client.0.smithi007.stdout:2/718: rename d0/d11/d31/d46/d6d/d81/da0/f85 to d0/d32/fdb 0 2024-09-21T04:35:42.538 INFO:tasks.workunit.client.0.smithi007.stdout:5/972: getdents d2/d1e 0 2024-09-21T04:35:42.539 INFO:tasks.workunit.client.0.smithi007.stdout:3/945: mknod dc/d22/d29/d32/d69/de4/c128 0 2024-09-21T04:35:42.542 INFO:tasks.workunit.client.0.smithi007.stdout:8/843: link d0/d3/d8/d46/db7/fbc d0/d3/d16/d93/d40/d92/dcb/ff7 0 2024-09-21T04:35:42.547 INFO:tasks.workunit.client.0.smithi007.stdout:5/973: symlink d2/d27/d42/d53/daf/db0/l13b 0 2024-09-21T04:35:42.548 INFO:tasks.workunit.client.0.smithi007.stdout:5/974: read d2/d27/dee/f115 [1965434,23253] 0 2024-09-21T04:35:42.549 INFO:tasks.workunit.client.0.smithi007.stdout:2/719: fdatasync d0/d11/d31/d9f/da9/fba 0 2024-09-21T04:35:42.552 INFO:tasks.workunit.client.0.smithi007.stdout:3/946: mknod dc/d18/d41/c129 0 2024-09-21T04:35:42.555 INFO:tasks.workunit.client.0.smithi007.stdout:5/975: unlink d2/d27/d42/da3/d23/d49/f10e 0 2024-09-21T04:35:42.558 INFO:tasks.workunit.client.0.smithi007.stdout:8/844: mkdir d0/d3/d16/d93/d5e/de3/d4d/df8 0 2024-09-21T04:35:42.562 INFO:tasks.workunit.client.0.smithi007.stdout:2/720: dread d0/d11/d31/d46/d6d/d81/da0/f88 [0,4194304] 0 2024-09-21T04:35:42.567 INFO:tasks.workunit.client.0.smithi007.stdout:5/976: mkdir d2/d27/d42/da3/d23/d49/dd4/de9/d13c 0 2024-09-21T04:35:42.568 INFO:tasks.workunit.client.0.smithi007.stdout:5/977: read - d2/d27/d42/da3/d28/de2/f137 zero size 2024-09-21T04:35:42.575 INFO:tasks.workunit.client.0.smithi007.stdout:5/978: mkdir d2/d27/d42/da3/d23/d49/dd4/d13d 0 2024-09-21T04:35:42.581 INFO:tasks.workunit.client.0.smithi007.stdout:2/721: rename d0/d11/f99 to d0/d11/d31/fdc 0 2024-09-21T04:35:42.582 INFO:tasks.workunit.client.0.smithi007.stdout:2/722: chown d0/d11/d31/d52/cad 26228 1 2024-09-21T04:35:42.585 INFO:tasks.workunit.client.0.smithi007.stdout:2/723: mkdir d0/d11/d31/d46/d6d/db9/ddd 0 2024-09-21T04:35:42.587 INFO:tasks.workunit.client.0.smithi007.stdout:5/979: link d2/d27/d42/da3/d23/d49/l5f d2/d27/d42/da3/l13e 0 2024-09-21T04:35:42.593 INFO:tasks.workunit.client.0.smithi007.stdout:5/980: dwrite d2/d27/d42/da3/fb9 [0,4194304] 0 2024-09-21T04:35:42.597 INFO:tasks.workunit.client.0.smithi007.stdout:5/981: creat d2/d27/d42/d53/daf/f13f x:0 0 0 2024-09-21T04:35:42.599 INFO:tasks.workunit.client.0.smithi007.stdout:6/853: sync 2024-09-21T04:35:42.609 INFO:tasks.workunit.client.0.smithi007.stdout:6/854: creat d3/d3e/d99/dbb/d95/dc9/dd9/f118 x:0 0 0 2024-09-21T04:35:42.610 INFO:tasks.workunit.client.0.smithi007.stdout:5/982: fdatasync d2/d27/d42/da3/d1d/f1f 0 2024-09-21T04:35:42.615 INFO:tasks.workunit.client.0.smithi007.stdout:5/983: creat d2/d27/d42/da3/d28/de2/d138/f140 x:0 0 0 2024-09-21T04:35:42.616 INFO:tasks.workunit.client.0.smithi007.stdout:2/724: truncate d0/fa3 1649728 0 2024-09-21T04:35:42.618 INFO:tasks.workunit.client.0.smithi007.stdout:5/984: fsync d2/f50 0 2024-09-21T04:35:42.628 INFO:tasks.workunit.client.0.smithi007.stdout:5/985: creat d2/d27/d42/da3/d28/f141 x:0 0 0 2024-09-21T04:35:42.628 INFO:tasks.workunit.client.0.smithi007.stdout:6/855: dread d3/de/d12/d19/f39 [0,4194304] 0 2024-09-21T04:35:42.631 INFO:tasks.workunit.client.0.smithi007.stdout:2/725: getdents d0/d11/db8 0 2024-09-21T04:35:42.632 INFO:tasks.workunit.client.0.smithi007.stdout:2/726: read - d0/d11/d6b/d9d/fc5 zero size 2024-09-21T04:35:42.634 INFO:tasks.workunit.client.0.smithi007.stdout:2/727: write d0/d11/d31/d46/d6d/d81/da0/fc [3281173,87218] 0 2024-09-21T04:35:42.635 INFO:tasks.workunit.client.0.smithi007.stdout:6/856: symlink d3/d3e/l119 0 2024-09-21T04:35:42.638 INFO:tasks.workunit.client.0.smithi007.stdout:5/986: getdents d2/d27/d42/da3/d23/d49/d6f/ddb/def/d109 0 2024-09-21T04:35:42.644 INFO:tasks.workunit.client.0.smithi007.stdout:2/728: dwrite d0/d11/d31/d7b/f94 [0,4194304] 0 2024-09-21T04:35:42.647 INFO:tasks.workunit.client.0.smithi007.stdout:5/987: truncate d2/d27/d42/da3/d12c/f74 2740029 0 2024-09-21T04:35:42.648 INFO:tasks.workunit.client.0.smithi007.stdout:5/988: stat d2/d27/d3d/cfe 0 2024-09-21T04:35:42.653 INFO:tasks.workunit.client.0.smithi007.stdout:5/989: fdatasync d2/f19 0 2024-09-21T04:35:42.655 INFO:tasks.workunit.client.0.smithi007.stdout:5/990: dread d2/d27/f11a [0,4194304] 0 2024-09-21T04:35:42.673 INFO:tasks.workunit.client.0.smithi007.stdout:0/859: sync 2024-09-21T04:35:42.673 INFO:tasks.workunit.client.0.smithi007.stdout:7/954: sync 2024-09-21T04:35:42.675 INFO:tasks.workunit.client.0.smithi007.stdout:5/991: dread d2/d27/d42/da3/d23/d5d/fb6 [0,4194304] 0 2024-09-21T04:35:42.684 INFO:tasks.workunit.client.0.smithi007.stdout:5/992: mknod d2/d27/d3d/dc2/dda/c142 0 2024-09-21T04:35:42.684 INFO:tasks.workunit.client.0.smithi007.stdout:3/947: write dc/d22/d29/d92/f119 [711352,12586] 0 2024-09-21T04:35:42.690 INFO:tasks.workunit.client.0.smithi007.stdout:8/845: truncate d0/d3/d16/d93/d5e/de3/d35/f73 3380228 0 2024-09-21T04:35:42.691 INFO:tasks.workunit.client.0.smithi007.stdout:8/846: read - d0/d3/d16/d93/d5e/de3/fb5 zero size 2024-09-21T04:35:42.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:42 smithi053 ceph-mon[35554]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:42.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:42 smithi053 ceph-mon[35554]: pgmap v42: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 55 MiB/s rd, 126 MiB/s wr, 372 op/s 2024-09-21T04:35:42.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:42 smithi053 ceph-mon[35554]: from='client.24497 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:42.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:42 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/986700087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:35:42.693 INFO:tasks.workunit.client.0.smithi007.stdout:3/948: mknod dc/d22/d29/d32/d4d/d84/dbb/dc1/dca/d10e/c12a 0 2024-09-21T04:35:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:42 smithi007 ceph-mon[27954]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:42 smithi007 ceph-mon[27954]: pgmap v42: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 55 MiB/s rd, 126 MiB/s wr, 372 op/s 2024-09-21T04:35:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:42 smithi007 ceph-mon[27954]: from='client.24497 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:42 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/986700087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:35:42.694 INFO:tasks.workunit.client.0.smithi007.stdout:5/993: truncate d2/d27/d42/da3/d23/d49/dd4/fde 554789 0 2024-09-21T04:35:42.695 INFO:tasks.workunit.client.0.smithi007.stdout:8/847: dread - d0/d3/d16/d93/d5e/de3/d4d/d5d/dba/ff5 zero size 2024-09-21T04:35:42.701 INFO:tasks.workunit.client.0.smithi007.stdout:5/994: dread d2/d27/d42/da3/d23/d49/d6f/ddb/f10a [0,4194304] 0 2024-09-21T04:35:42.702 INFO:tasks.workunit.client.0.smithi007.stdout:8/848: rmdir d0/d3/d16/d93/d5e/de3/d4d/d5d 39 2024-09-21T04:35:42.704 INFO:tasks.workunit.client.0.smithi007.stdout:8/849: write d0/d3/d7c/fc1 [703885,44080] 0 2024-09-21T04:35:42.704 INFO:tasks.workunit.client.0.smithi007.stdout:3/949: getdents dc/d22/d29/d32/d4d/d7d 0 2024-09-21T04:35:42.707 INFO:tasks.workunit.client.0.smithi007.stdout:3/950: fsync dc/d18/d41/f4a 0 2024-09-21T04:35:42.714 INFO:tasks.workunit.client.0.smithi007.stdout:5/995: dread d2/d27/d3d/d9e/fd9 [0,4194304] 0 2024-09-21T04:35:42.717 INFO:tasks.workunit.client.0.smithi007.stdout:5/996: truncate d2/f70 4960146 0 2024-09-21T04:35:42.718 INFO:tasks.workunit.client.0.smithi007.stdout:8/850: getdents d0/d3/d16/d93/d5e/d7b 0 2024-09-21T04:35:42.722 INFO:tasks.workunit.client.0.smithi007.stdout:8/851: mkdir d0/d3/d7c/dbf/df9 0 2024-09-21T04:35:42.722 INFO:tasks.workunit.client.0.smithi007.stdout:6/857: write d3/d3e/d99/dbb/f66 [5739260,26173] 0 2024-09-21T04:35:42.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/729: write d0/d11/d31/d39/f8c [991463,60364] 0 2024-09-21T04:35:42.728 INFO:tasks.workunit.client.0.smithi007.stdout:2/730: chown d0/d11/d31/d46/d77/f91 31870693 1 2024-09-21T04:35:42.729 INFO:tasks.workunit.client.0.smithi007.stdout:6/858: dread - d3/d3b/dd3/d60/dd7/ff3 zero size 2024-09-21T04:35:42.731 INFO:tasks.workunit.client.0.smithi007.stdout:5/997: getdents d2/d27/d42/da3/d23/d49/d6f/ddb/def/d126 0 2024-09-21T04:35:42.736 INFO:tasks.workunit.client.0.smithi007.stdout:0/860: write d6/d75/dc5/d47/d99/fae [1038129,41562] 0 2024-09-21T04:35:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:7/955: dwrite d3/d6/d14/d20/d80/dca/ffa [0,4194304] 0 2024-09-21T04:35:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:7/956: fsync d3/d6/d2e/d63/d6a/f70 0 2024-09-21T04:35:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:7/957: chown d3/d6/d85/de7/d11b/d66/l86 92073 1 2024-09-21T04:35:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:5/998: creat d2/d27/d42/da3/d23/d49/d6f/ddb/def/d109/f143 x:0 0 0 2024-09-21T04:35:42.750 INFO:tasks.workunit.client.0.smithi007.stdout:3/951: write dc/d22/d29/d32/d4d/fc2 [678129,34742] 0 2024-09-21T04:35:42.750 INFO:tasks.workunit.client.0.smithi007.stdout:8/852: link d0/d3/d7c/cbe d0/d3/daa/d9d/cfa 0 2024-09-21T04:35:42.753 INFO:tasks.workunit.client.0.smithi007.stdout:0/861: truncate d6/d75/dc5/d18/d2e/d3f/fbb 536213 0 2024-09-21T04:35:42.757 INFO:tasks.workunit.client.0.smithi007.stdout:7/958: unlink d3/d6/d13/l6b 0 2024-09-21T04:35:42.758 INFO:tasks.workunit.client.0.smithi007.stdout:5/999: unlink d2/d27/d42/da3/c2c 0 2024-09-21T04:35:42.761 INFO:tasks.workunit.client.0.smithi007.stdout:3/952: creat dc/d22/d29/d32/d4d/d84/dd0/dbc/dd1/f12b x:0 0 0 2024-09-21T04:35:42.762 INFO:tasks.workunit.client.0.smithi007.stdout:2/731: truncate d0/d11/d31/da4/fbb 402677 0 2024-09-21T04:35:42.764 INFO:tasks.workunit.client.0.smithi007.stdout:0/862: mknod d6/d75/dc5/d57/d9d/c13a 0 2024-09-21T04:35:42.769 INFO:tasks.workunit.client.0.smithi007.stdout:3/953: creat dc/d22/d29/dac/f12c x:0 0 0 2024-09-21T04:35:42.770 INFO:tasks.workunit.client.0.smithi007.stdout:2/732: rename d0/d11/d31/d46/d6c/f83 to d0/d11/db1/dbf/fde 0 2024-09-21T04:35:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.769+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 --> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1c5c0626f0 con 0x7f1c44074a30 2024-09-21T04:35:42.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.771+0000 7f1c43fff640 1 -- 172.21.15.7:0/4001617406 <== mgr.24429 v2:172.21.15.53:6828/2371363507 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f1c5c0626f0 con 0x7f1c44074a30 2024-09-21T04:35:42.773 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:35:42.773 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:35:42.773 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:35:42.774 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:35:42.774 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-09-21T04:35:42.774 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "1/2 daemons upgraded", 2024-09-21T04:35:42.774 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mgr daemons", 2024-09-21T04:35:42.774 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:35:42.774 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f1c44074a30 msgr2=0x7f1c44076ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f1c44074a30 0x7f1c44076ef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c19abc0 tx=0x7f1c50005a20 comp rx=0 tx=0).stop 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c101960 msgr2=0x7f1c5c199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c101960 0x7f1c5c199710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00b3d0 tx=0x7f1c4c00b890 comp rx=0 tx=0).stop 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 shutdown_connections 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:6828/2371363507,v1:172.21.15.53:6829/2371363507] conn(0x7f1c44074a30 0x7f1c44076ef0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102b60 0x7f1c5c199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 --2- 172.21.15.7:0/4001617406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c101960 0x7f1c5c199710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.773+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 >> 172.21.15.7:0/4001617406 conn(0x7f1c5c0fd0f0 msgr2=0x7f1c5c103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.774+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 shutdown_connections 2024-09-21T04:35:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:35:42.774+0000 7f1c610cd640 1 -- 172.21.15.7:0/4001617406 wait complete. 2024-09-21T04:35:42.779 INFO:tasks.workunit.client.0.smithi007.stdout:8/853: getdents d0/d3/d16/d93/d5e/de3 0 2024-09-21T04:35:42.784 INFO:tasks.workunit.client.0.smithi007.stdout:3/954: rename dc/d18/f7b to dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/d71/d89/f12d 0 2024-09-21T04:35:42.793 INFO:tasks.workunit.client.0.smithi007.stdout:7/959: dwrite d3/d6/d85/de9/ff2 [0,4194304] 0 2024-09-21T04:35:42.795 INFO:tasks.workunit.client.0.smithi007.stdout:3/955: dread dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/d71/f85 [0,4194304] 0 2024-09-21T04:35:42.798 INFO:tasks.workunit.client.0.smithi007.stdout:8/854: rmdir d0/d3/daa 39 2024-09-21T04:35:42.802 INFO:tasks.workunit.client.0.smithi007.stdout:2/733: write d0/d11/f4e [60661,43996] 0 2024-09-21T04:35:42.807 INFO:tasks.workunit.client.0.smithi007.stdout:3/956: dwrite dc/d18/d26/f51 [0,4194304] 0 2024-09-21T04:35:42.807 INFO:tasks.workunit.client.0.smithi007.stdout:0/863: dwrite d6/d75/dc5/f86 [0,4194304] 0 2024-09-21T04:35:42.810 INFO:tasks.workunit.client.0.smithi007.stdout:0/864: write d6/d75/dc5/d57/d9d/daf/f118 [734917,60367] 0 2024-09-21T04:35:42.815 INFO:tasks.workunit.client.0.smithi007.stdout:7/960: symlink d3/d6/d10/d51/d11c/l14e 0 2024-09-21T04:35:42.818 INFO:tasks.workunit.client.0.smithi007.stdout:2/734: creat d0/d11/d31/d46/d6c/fdf x:0 0 0 2024-09-21T04:35:42.819 INFO:tasks.workunit.client.0.smithi007.stdout:7/961: dread d3/d6/d2e/d5e/fef [0,4194304] 0 2024-09-21T04:35:42.819 INFO:tasks.workunit.client.0.smithi007.stdout:3/957: mkdir dc/d6c/d12e 0 2024-09-21T04:35:42.823 INFO:tasks.workunit.client.0.smithi007.stdout:0/865: dread d6/d75/dc5/d18/d2e/d3f/fbb [0,4194304] 0 2024-09-21T04:35:42.823 INFO:tasks.workunit.client.0.smithi007.stdout:8/855: mknod d0/d3/d16/d93/d40/d92/dcb/df2/cfb 0 2024-09-21T04:35:42.828 INFO:tasks.workunit.client.0.smithi007.stdout:8/856: rmdir d0/d3/d8/d46 39 2024-09-21T04:35:42.831 INFO:tasks.workunit.client.0.smithi007.stdout:2/735: unlink d0/c13 0 2024-09-21T04:35:42.834 INFO:tasks.workunit.client.0.smithi007.stdout:7/962: truncate d3/d6/d85/de7/d11b/d66/f94 5022483 0 2024-09-21T04:35:42.835 INFO:tasks.workunit.client.0.smithi007.stdout:3/958: write dc/f10 [3033184,37180] 0 2024-09-21T04:35:42.836 INFO:tasks.workunit.client.0.smithi007.stdout:8/857: rmdir d0/d3/d16/d93/d5e/de3 39 2024-09-21T04:35:42.839 INFO:tasks.workunit.client.0.smithi007.stdout:0/866: write d6/d91/da3/f9a [617515,59478] 0 2024-09-21T04:35:42.842 INFO:tasks.workunit.client.0.smithi007.stdout:2/736: creat d0/d11/d31/d46/d6d/db9/ddd/fe0 x:0 0 0 2024-09-21T04:35:42.845 INFO:tasks.workunit.client.0.smithi007.stdout:0/867: mkdir d6/d75/d13b 0 2024-09-21T04:35:42.847 INFO:tasks.workunit.client.0.smithi007.stdout:8/858: unlink d0/d3/d7c/cbe 0 2024-09-21T04:35:42.848 INFO:tasks.workunit.client.0.smithi007.stdout:2/737: creat d0/d11/d6e/d73/fe1 x:0 0 0 2024-09-21T04:35:42.849 INFO:tasks.workunit.client.0.smithi007.stdout:2/738: chown d0/d11/d31/d7b/da2/da5 2 1 2024-09-21T04:35:42.850 INFO:tasks.workunit.client.0.smithi007.stdout:3/959: mknod dc/d6c/da7/dfd/c12f 0 2024-09-21T04:35:42.851 INFO:tasks.workunit.client.0.smithi007.stdout:7/963: link d3/d6/d14/d20/d80/db2/la6 d3/d6/d10/d4f/l14f 0 2024-09-21T04:35:42.851 INFO:tasks.workunit.client.0.smithi007.stdout:3/960: stat dc/d22/d29/d92 0 2024-09-21T04:35:42.853 INFO:tasks.workunit.client.0.smithi007.stdout:8/859: creat d0/d3/d82/ffc x:0 0 0 2024-09-21T04:35:42.854 INFO:tasks.workunit.client.0.smithi007.stdout:2/739: rename d0/d11/d31/d46/d48/fc2 to d0/d11/d31/d7b/da2/da5/fe2 0 2024-09-21T04:35:42.855 INFO:tasks.workunit.client.0.smithi007.stdout:7/964: symlink d3/d6/d85/dad/l150 0 2024-09-21T04:35:42.858 INFO:tasks.workunit.client.0.smithi007.stdout:3/961: unlink dc/d22/d29/d32/d69/d6d/l36 0 2024-09-21T04:35:42.859 INFO:tasks.workunit.client.0.smithi007.stdout:3/962: chown dc/d22/d29/d32/d4d/d84/dd0/cc3 635 1 2024-09-21T04:35:42.860 INFO:tasks.workunit.client.0.smithi007.stdout:8/860: chown d0/d3/d16/d93/d5e/de3/l44 4 1 2024-09-21T04:35:42.867 INFO:tasks.workunit.client.0.smithi007.stdout:0/868: write d6/d75/dc5/d18/d2e/f87 [3107305,116095] 0 2024-09-21T04:35:42.871 INFO:tasks.workunit.client.0.smithi007.stdout:2/740: fsync d0/d11/d31/d46/d48/f8e 0 2024-09-21T04:35:42.874 INFO:tasks.workunit.client.0.smithi007.stdout:7/965: write d3/d6/d13/f5d [4909693,81830] 0 2024-09-21T04:35:42.876 INFO:tasks.workunit.client.0.smithi007.stdout:3/963: dread dc/d18/f45 [0,4194304] 0 2024-09-21T04:35:42.880 INFO:tasks.workunit.client.0.smithi007.stdout:0/869: fsync d6/d75/dc5/d52/d6d/fa6 0 2024-09-21T04:35:42.883 INFO:tasks.workunit.client.0.smithi007.stdout:2/741: mkdir d0/d11/d31/d9f/da9/de3 0 2024-09-21T04:35:42.884 INFO:tasks.workunit.client.0.smithi007.stdout:2/742: dread - d0/d11/d6e/fc7 zero size 2024-09-21T04:35:42.887 INFO:tasks.workunit.client.0.smithi007.stdout:8/861: creat d0/d3/d16/d93/d5e/ffd x:0 0 0 2024-09-21T04:35:42.889 INFO:tasks.workunit.client.0.smithi007.stdout:0/870: mknod d6/d75/dc5/d57/c13c 0 2024-09-21T04:35:42.890 INFO:tasks.workunit.client.0.smithi007.stdout:3/964: dread dc/d22/f24 [0,4194304] 0 2024-09-21T04:35:42.895 INFO:tasks.workunit.client.0.smithi007.stdout:7/966: dread d3/d6/d10/d51/fd1 [0,4194304] 0 2024-09-21T04:35:42.896 INFO:tasks.workunit.client.0.smithi007.stdout:0/871: mknod d6/d75/dc5/d47/d8a/df4/c13d 0 2024-09-21T04:35:42.897 INFO:tasks.workunit.client.0.smithi007.stdout:2/743: symlink d0/d11/d31/d46/d6d/d81/da0/dd3/le4 0 2024-09-21T04:35:42.898 INFO:tasks.workunit.client.0.smithi007.stdout:2/744: fsync d0/d11/d6e/d73/fe1 0 2024-09-21T04:35:42.903 INFO:tasks.workunit.client.0.smithi007.stdout:8/862: chown d0/d3/d16/d93/c78 185279 1 2024-09-21T04:35:42.903 INFO:tasks.workunit.client.0.smithi007.stdout:7/967: dwrite d3/d6/d40/fdd [0,4194304] 0 2024-09-21T04:35:42.904 INFO:tasks.workunit.client.0.smithi007.stdout:0/872: mkdir d6/d13e 0 2024-09-21T04:35:42.909 INFO:tasks.workunit.client.0.smithi007.stdout:0/873: mknod d6/d91/c13f 0 2024-09-21T04:35:42.914 INFO:tasks.workunit.client.0.smithi007.stdout:2/745: rename d0/d11/db1/dbf/fde to d0/d11/d31/d46/d77/fe5 0 2024-09-21T04:35:42.914 INFO:tasks.workunit.client.0.smithi007.stdout:2/746: dread - d0/d32/fda zero size 2024-09-21T04:35:42.916 INFO:tasks.workunit.client.0.smithi007.stdout:0/874: creat d6/d75/dc5/d47/d8a/f140 x:0 0 0 2024-09-21T04:35:42.919 INFO:tasks.workunit.client.0.smithi007.stdout:2/747: mknod d0/d11/d31/d46/d77/ce6 0 2024-09-21T04:35:42.920 INFO:tasks.workunit.client.0.smithi007.stdout:8/863: dread d0/d3/d16/f21 [4194304,4194304] 0 2024-09-21T04:35:42.920 INFO:tasks.workunit.client.0.smithi007.stdout:0/875: readlink d6/d75/dc5/d18/d2e/lb4 0 2024-09-21T04:35:42.921 INFO:tasks.workunit.client.0.smithi007.stdout:3/965: write dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/d71/f9d [1731009,33957] 0 2024-09-21T04:35:42.928 INFO:tasks.workunit.client.0.smithi007.stdout:3/966: rmdir dc/d22/d29/d32/d69/de4 39 2024-09-21T04:35:42.931 INFO:tasks.workunit.client.0.smithi007.stdout:8/864: dread d0/d3/d8/d46/f7d [0,4194304] 0 2024-09-21T04:35:42.932 INFO:tasks.workunit.client.0.smithi007.stdout:7/968: dwrite d3/d6/d85/dd2/fd5 [0,4194304] 0 2024-09-21T04:35:42.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/748: write d0/d11/d31/d46/d48/f62 [2004183,102565] 0 2024-09-21T04:35:42.936 INFO:tasks.workunit.client.0.smithi007.stdout:2/749: readlink d0/d11/d31/d46/d6d/d81/da0/l1f 0 2024-09-21T04:35:42.937 INFO:tasks.workunit.client.0.smithi007.stdout:7/969: chown d3/d6/d85/de7/d11b/d66/l7a 0 1 2024-09-21T04:35:42.938 INFO:tasks.workunit.client.0.smithi007.stdout:0/876: creat d6/d75/dc5/f141 x:0 0 0 2024-09-21T04:35:42.942 INFO:tasks.workunit.client.0.smithi007.stdout:3/967: creat dc/d22/d29/d32/d4d/d84/dbb/f130 x:0 0 0 2024-09-21T04:35:42.943 INFO:tasks.workunit.client.0.smithi007.stdout:7/970: truncate d3/d6/d40/da1/fe5 2325047 0 2024-09-21T04:35:42.946 INFO:tasks.workunit.client.0.smithi007.stdout:2/750: dwrite d0/d11/d6b/fc1 [0,4194304] 0 2024-09-21T04:35:42.947 INFO:tasks.workunit.client.0.smithi007.stdout:2/751: stat d0/f1c 0 2024-09-21T04:35:42.949 INFO:tasks.workunit.client.0.smithi007.stdout:3/968: read dc/d22/d29/d32/d4d/f8b [6568353,74495] 0 2024-09-21T04:35:42.949 INFO:tasks.workunit.client.0.smithi007.stdout:7/971: write d3/d6/d85/de9/f11e [1237778,34895] 0 2024-09-21T04:35:42.949 INFO:tasks.workunit.client.0.smithi007.stdout:0/877: fdatasync d6/d75/dc5/d18/d27/fd3 0 2024-09-21T04:35:42.952 INFO:tasks.workunit.client.0.smithi007.stdout:2/752: symlink d0/d11/d31/d9f/da9/le7 0 2024-09-21T04:35:42.953 INFO:tasks.workunit.client.0.smithi007.stdout:8/865: truncate d0/d3/d8/f58 4529291 0 2024-09-21T04:35:42.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/753: chown d0/d11/d31/d46/d6d/db9 1462 1 2024-09-21T04:35:42.956 INFO:tasks.workunit.client.0.smithi007.stdout:0/878: creat d6/d75/dc5/d18/d2e/d60/f142 x:0 0 0 2024-09-21T04:35:42.957 INFO:tasks.workunit.client.0.smithi007.stdout:0/879: readlink d6/d75/dc5/d18/d2e/d3f/l61 0 2024-09-21T04:35:42.958 INFO:tasks.workunit.client.0.smithi007.stdout:8/866: fsync d0/d3/d16/d93/d5e/d7b/f7e 0 2024-09-21T04:35:42.959 INFO:tasks.workunit.client.0.smithi007.stdout:2/754: dread - d0/d11/d31/d46/d48/f8e zero size 2024-09-21T04:35:42.961 INFO:tasks.workunit.client.0.smithi007.stdout:2/755: readlink d0/d11/d31/d46/d6d/d81/da0/l2c 0 2024-09-21T04:35:42.961 INFO:tasks.workunit.client.0.smithi007.stdout:7/972: unlink d3/d6/d85/de7/d11b/d101/c109 0 2024-09-21T04:35:42.962 INFO:tasks.workunit.client.0.smithi007.stdout:0/880: fsync d6/d75/dc5/d18/d27/fb7 0 2024-09-21T04:35:42.964 INFO:tasks.workunit.client.0.smithi007.stdout:2/756: dread d0/d11/d31/d52/fcf [0,4194304] 0 2024-09-21T04:35:42.964 INFO:tasks.workunit.client.0.smithi007.stdout:2/757: stat d0/d11/d31/d9f/da9/le7 0 2024-09-21T04:35:42.977 INFO:tasks.workunit.client.0.smithi007.stdout:3/969: dwrite dc/d22/d29/d32/d4d/d84/dd0/dbc/dd1/ff1 [0,4194304] 0 2024-09-21T04:35:42.979 INFO:tasks.workunit.client.0.smithi007.stdout:8/867: dread d0/f20 [0,4194304] 0 2024-09-21T04:35:42.981 INFO:tasks.workunit.client.0.smithi007.stdout:7/973: write d3/d6/d10/d51/fdc [671350,91880] 0 2024-09-21T04:35:42.983 INFO:tasks.workunit.client.0.smithi007.stdout:0/881: truncate d6/d75/dc5/d18/d27/fa1 1584571 0 2024-09-21T04:35:42.992 INFO:tasks.workunit.client.0.smithi007.stdout:8/868: dread d0/d3/d8/d46/db7/fbc [0,4194304] 0 2024-09-21T04:35:43.053 INFO:tasks.workunit.client.0.smithi007.stdout:3/970: mknod dc/d22/d29/d32/d8f/c131 0 2024-09-21T04:35:43.055 INFO:tasks.workunit.client.0.smithi007.stdout:2/758: write d0/d11/d31/d46/d6d/d81/da0/d57/f5a [1350345,26750] 0 2024-09-21T04:35:43.065 INFO:tasks.workunit.client.0.smithi007.stdout:2/759: rename d0/d11/d31/d46/d6d to d0/d11/d31/d7b/de8 0 2024-09-21T04:35:43.065 INFO:tasks.workunit.client.0.smithi007.stdout:0/882: creat d6/d13e/f143 x:0 0 0 2024-09-21T04:35:43.066 INFO:tasks.workunit.client.0.smithi007.stdout:8/869: write d0/d3/d16/d93/f18 [827050,27275] 0 2024-09-21T04:35:43.069 INFO:tasks.workunit.client.0.smithi007.stdout:2/760: write d0/d11/d31/d7b/de8/d81/da0/d57/f5a [2189548,78497] 0 2024-09-21T04:35:43.070 INFO:tasks.workunit.client.0.smithi007.stdout:7/974: rename d3/d6/d40 to d3/d6/d14/d20/d80/dca/d113/d151 0 2024-09-21T04:35:43.072 INFO:tasks.workunit.client.0.smithi007.stdout:0/883: creat d6/d75/dc5/d47/d8a/f144 x:0 0 0 2024-09-21T04:35:43.074 INFO:tasks.workunit.client.0.smithi007.stdout:3/971: dwrite dc/d18/f45 [0,4194304] 0 2024-09-21T04:35:43.077 INFO:tasks.workunit.client.0.smithi007.stdout:3/972: stat dc/d22/d29/d32/d4d/d84/dd0/da6 0 2024-09-21T04:35:43.079 INFO:tasks.workunit.client.0.smithi007.stdout:0/884: dread d6/d75/dc5/d47/df0/f114 [0,4194304] 0 2024-09-21T04:35:43.081 INFO:tasks.workunit.client.0.smithi007.stdout:0/885: chown d6/d75/dc5/d52/ldd 14050401 1 2024-09-21T04:35:43.083 INFO:tasks.workunit.client.0.smithi007.stdout:0/886: dread d6/d75/dc5/d18/d2e/d3f/fbb [0,4194304] 0 2024-09-21T04:35:43.084 INFO:tasks.workunit.client.0.smithi007.stdout:2/761: creat d0/d11/fe9 x:0 0 0 2024-09-21T04:35:43.088 INFO:tasks.workunit.client.0.smithi007.stdout:0/887: fdatasync d6/d91/da3/f8c 0 2024-09-21T04:35:43.090 INFO:tasks.workunit.client.0.smithi007.stdout:2/762: creat d0/d11/d31/d7b/de8/d81/da0/d57/fea x:0 0 0 2024-09-21T04:35:43.091 INFO:tasks.workunit.client.0.smithi007.stdout:2/763: read d0/d11/f4e [426348,41825] 0 2024-09-21T04:35:43.097 INFO:tasks.workunit.client.0.smithi007.stdout:8/870: dwrite d0/d3/d16/f1b [0,4194304] 0 2024-09-21T04:35:43.098 INFO:tasks.workunit.client.0.smithi007.stdout:8/871: fdatasync d0/d3/d8/d46/db7/fe0 0 2024-09-21T04:35:43.099 INFO:tasks.workunit.client.0.smithi007.stdout:2/764: mkdir d0/d11/d6e/deb 0 2024-09-21T04:35:43.107 INFO:tasks.workunit.client.0.smithi007.stdout:8/872: creat d0/d3/d16/d93/d5e/de3/ffe x:0 0 0 2024-09-21T04:35:43.109 INFO:tasks.workunit.client.0.smithi007.stdout:8/873: readlink d0/d3/d7c/dbf/lc3 0 2024-09-21T04:35:43.109 INFO:tasks.workunit.client.0.smithi007.stdout:0/888: write d6/d75/dc5/d18/d27/ff7 [455859,28711] 0 2024-09-21T04:35:43.114 INFO:tasks.workunit.client.0.smithi007.stdout:2/765: dwrite d0/f2e [4194304,4194304] 0 2024-09-21T04:35:43.115 INFO:tasks.workunit.client.0.smithi007.stdout:7/975: dwrite d3/d6/d2e/d63/d6a/d81/f8c [0,4194304] 0 2024-09-21T04:35:43.116 INFO:tasks.workunit.client.0.smithi007.stdout:8/874: creat d0/d3/d7c/dbf/fff x:0 0 0 2024-09-21T04:35:43.117 INFO:tasks.workunit.client.0.smithi007.stdout:8/875: write d0/d3/d7c/fec [882311,111105] 0 2024-09-21T04:35:43.120 INFO:tasks.workunit.client.0.smithi007.stdout:0/889: write d6/d75/dc5/d18/d2e/d3f/fcd [749249,20254] 0 2024-09-21T04:35:43.124 INFO:tasks.workunit.client.0.smithi007.stdout:2/766: creat d0/d11/fec x:0 0 0 2024-09-21T04:35:43.128 INFO:tasks.workunit.client.0.smithi007.stdout:7/976: dwrite d3/d6/d2e/f13e [0,4194304] 0 2024-09-21T04:35:43.128 INFO:tasks.workunit.client.0.smithi007.stdout:0/890: dread d6/d75/dc5/d47/f10d [0,4194304] 0 2024-09-21T04:35:43.129 INFO:tasks.workunit.client.0.smithi007.stdout:0/891: chown d6/d75/dc5/d18/d39/c65 46397 1 2024-09-21T04:35:43.135 INFO:tasks.workunit.client.0.smithi007.stdout:2/767: unlink d0/d11/d31/d39/f3e 0 2024-09-21T04:35:43.136 INFO:tasks.workunit.client.0.smithi007.stdout:2/768: chown d0/f8 0 1 2024-09-21T04:35:43.136 INFO:tasks.workunit.client.0.smithi007.stdout:8/876: creat d0/d3/d8/f100 x:0 0 0 2024-09-21T04:35:43.142 INFO:tasks.workunit.client.0.smithi007.stdout:2/769: mknod d0/d11/ced 0 2024-09-21T04:35:43.146 INFO:tasks.workunit.client.0.smithi007.stdout:0/892: creat d6/d91/da3/f145 x:0 0 0 2024-09-21T04:35:43.149 INFO:tasks.workunit.client.0.smithi007.stdout:7/977: dwrite d3/d6/d14/d20/d80/dca/ffa [0,4194304] 0 2024-09-21T04:35:43.151 INFO:tasks.workunit.client.0.smithi007.stdout:0/893: readlink d6/d75/dc5/d18/d39/l50 0 2024-09-21T04:35:43.155 INFO:tasks.workunit.client.0.smithi007.stdout:2/770: dwrite d0/d11/d6e/fc7 [0,4194304] 0 2024-09-21T04:35:43.157 INFO:tasks.workunit.client.0.smithi007.stdout:2/771: chown d0/d11/d31/d46/d77 867018254 1 2024-09-21T04:35:43.158 INFO:tasks.workunit.client.0.smithi007.stdout:8/877: dread d0/f4f [0,4194304] 0 2024-09-21T04:35:43.163 INFO:tasks.workunit.client.0.smithi007.stdout:0/894: chown d6/d75/dc5/c130 41 1 2024-09-21T04:35:43.164 INFO:tasks.workunit.client.0.smithi007.stdout:2/772: fsync d0/d11/d31/d46/d6c/f78 0 2024-09-21T04:35:43.170 INFO:tasks.workunit.client.0.smithi007.stdout:7/978: mknod d3/d6/d13/c152 0 2024-09-21T04:35:43.176 INFO:tasks.workunit.client.0.smithi007.stdout:2/773: creat d0/d11/d31/d7b/da2/fee x:0 0 0 2024-09-21T04:35:43.180 INFO:tasks.workunit.client.0.smithi007.stdout:0/895: write d6/d75/dc5/d18/d39/dd5/fda [431068,39313] 0 2024-09-21T04:35:43.182 INFO:tasks.workunit.client.0.smithi007.stdout:0/896: write d6/d75/dc5/d18/d39/dd5/fda [616685,97019] 0 2024-09-21T04:35:43.190 INFO:tasks.workunit.client.0.smithi007.stdout:8/878: dread d0/d3/d16/d93/d5e/de3/d35/f87 [4194304,4194304] 0 2024-09-21T04:35:43.190 INFO:tasks.workunit.client.0.smithi007.stdout:7/979: rename d3/d6/d85/dad/fb5 to d3/d6/d14/d20/d80/dca/f153 0 2024-09-21T04:35:43.192 INFO:tasks.workunit.client.0.smithi007.stdout:7/980: read d3/d6/d85/de9/ff2 [1560726,49322] 0 2024-09-21T04:35:43.195 INFO:tasks.workunit.client.0.smithi007.stdout:8/879: rename d0/d3/d8/f47 to d0/d3/d16/d93/d40/d92/dcb/f101 0 2024-09-21T04:35:43.199 INFO:tasks.workunit.client.0.smithi007.stdout:0/897: getdents d6/d75/dc5/d47/d8a/df4 0 2024-09-21T04:35:43.203 INFO:tasks.workunit.client.0.smithi007.stdout:7/981: dwrite d3/d6/d10/d4f/f9b [0,4194304] 0 2024-09-21T04:35:43.204 INFO:tasks.workunit.client.0.smithi007.stdout:0/898: symlink d6/l146 0 2024-09-21T04:35:43.206 INFO:tasks.workunit.client.0.smithi007.stdout:8/880: chown d0/d3/d16/d93/d5e/de3/d4d/d5d/dba/fcd 10 1 2024-09-21T04:35:43.206 INFO:tasks.workunit.client.0.smithi007.stdout:7/982: write d3/d6/fa [1181373,25019] 0 2024-09-21T04:35:43.213 INFO:tasks.workunit.client.0.smithi007.stdout:7/983: mkdir d3/d6/d2e/d63/d154 0 2024-09-21T04:35:43.213 INFO:tasks.workunit.client.0.smithi007.stdout:8/881: dwrite d0/d3/d74/f66 [0,4194304] 0 2024-09-21T04:35:43.216 INFO:tasks.workunit.client.0.smithi007.stdout:8/882: dread - d0/d3/d16/d93/d40/d92/dcb/fea zero size 2024-09-21T04:35:43.219 INFO:tasks.workunit.client.0.smithi007.stdout:8/883: write d0/d3/d16/d93/d5e/fce [638952,35247] 0 2024-09-21T04:35:43.229 INFO:tasks.workunit.client.0.smithi007.stdout:0/899: dwrite d6/d75/dc5/d18/d27/fa1 [0,4194304] 0 2024-09-21T04:35:43.231 INFO:tasks.workunit.client.0.smithi007.stdout:7/984: dread d3/d6/d85/f41 [0,4194304] 0 2024-09-21T04:35:43.240 INFO:tasks.workunit.client.0.smithi007.stdout:0/900: creat d6/d75/dc5/d52/d6d/f147 x:0 0 0 2024-09-21T04:35:43.240 INFO:tasks.workunit.client.0.smithi007.stdout:8/884: dwrite d0/d3/d8/f58 [0,4194304] 0 2024-09-21T04:35:43.242 INFO:tasks.workunit.client.0.smithi007.stdout:8/885: write d0/d3/d82/ffc [435272,51358] 0 2024-09-21T04:35:43.243 INFO:tasks.workunit.client.0.smithi007.stdout:7/985: write d3/d6/f28 [11013145,61592] 0 2024-09-21T04:35:43.244 INFO:tasks.workunit.client.0.smithi007.stdout:8/886: dread - d0/d3/d7c/dbf/fff zero size 2024-09-21T04:35:43.245 INFO:tasks.workunit.client.0.smithi007.stdout:7/986: dread - d3/d6/d85/de7/d11b/d101/f10c zero size 2024-09-21T04:35:43.251 INFO:tasks.workunit.client.0.smithi007.stdout:7/987: chown d3/d6/d14/d20/d102/l10e 103697 1 2024-09-21T04:35:43.251 INFO:tasks.workunit.client.0.smithi007.stdout:7/988: stat d3/d6/d13/d4b/f7d 0 2024-09-21T04:35:43.256 INFO:tasks.workunit.client.0.smithi007.stdout:8/887: rename d0/d3/d16/d93/d5e/de3/c70 to d0/d3/d16/d93/c102 0 2024-09-21T04:35:43.259 INFO:tasks.workunit.client.0.smithi007.stdout:8/888: fsync d0/d3/d16/d93/d40/f45 0 2024-09-21T04:35:43.263 INFO:tasks.workunit.client.0.smithi007.stdout:8/889: creat d0/d3/daa/f103 x:0 0 0 2024-09-21T04:35:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:7/989: truncate d3/d6/d10/d4f/f65 219172 0 2024-09-21T04:35:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:7/990: mkdir d3/d155 0 2024-09-21T04:35:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:8/890: creat d0/d3/d16/d93/f104 x:0 0 0 2024-09-21T04:35:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:7/991: getdents d3/d6/d2e/d63/df8/d13b 0 2024-09-21T04:35:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:8/891: rmdir d0/d3/d7c/dbf/df9 0 2024-09-21T04:35:43.280 INFO:tasks.workunit.client.0.smithi007.stdout:7/992: mkdir d3/d6/d14/d20/d80/dca/d113/d151/d156 0 2024-09-21T04:35:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:7/993: mkdir d3/d6/d2e/d63/df8/d13b/d157 0 2024-09-21T04:35:43.285 INFO:tasks.workunit.client.0.smithi007.stdout:7/994: dread - d3/d6/d85/f135 zero size 2024-09-21T04:35:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:8/892: dwrite d0/d3/d16/d93/d5e/de3/ffe [0,4194304] 0 2024-09-21T04:35:43.288 INFO:tasks.workunit.client.0.smithi007.stdout:8/893: truncate d0/d3/d82/ffc 1184548 0 2024-09-21T04:35:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:8/894: write d0/f4f [3872312,48001] 0 2024-09-21T04:35:43.294 INFO:tasks.workunit.client.0.smithi007.stdout:8/895: creat d0/d3/d8/f105 x:0 0 0 2024-09-21T04:35:43.303 INFO:tasks.workunit.client.0.smithi007.stdout:8/896: dwrite d0/f43 [0,4194304] 0 2024-09-21T04:35:43.309 INFO:tasks.workunit.client.0.smithi007.stdout:8/897: getdents d0/d3/d16/d93/d5e/de3/d4d/d5d/dba 0 2024-09-21T04:35:43.311 INFO:tasks.workunit.client.0.smithi007.stdout:8/898: truncate d0/d3/d7c/f59 1221609 0 2024-09-21T04:35:43.313 INFO:tasks.workunit.client.0.smithi007.stdout:8/899: creat d0/d3/d82/f106 x:0 0 0 2024-09-21T04:35:43.315 INFO:tasks.workunit.client.0.smithi007.stdout:8/900: truncate d0/d3/d16/d93/d5e/de3/d4d/d5d/fab 1259111 0 2024-09-21T04:35:43.317 INFO:tasks.workunit.client.0.smithi007.stdout:8/901: mkdir d0/d3/d7c/dbf/d107 0 2024-09-21T04:35:43.321 INFO:tasks.workunit.client.0.smithi007.stdout:8/902: symlink d0/l108 0 2024-09-21T04:35:43.322 INFO:tasks.workunit.client.0.smithi007.stdout:8/903: write d0/d3/d8/fde [404169,78349] 0 2024-09-21T04:35:43.325 INFO:tasks.workunit.client.0.smithi007.stdout:8/904: mknod d0/d3/d8/c109 0 2024-09-21T04:35:43.331 INFO:tasks.workunit.client.0.smithi007.stdout:8/905: dwrite d0/d3/d74/fac [0,4194304] 0 2024-09-21T04:35:43.343 INFO:tasks.workunit.client.0.smithi007.stdout:8/906: truncate d0/fb8 337355 0 2024-09-21T04:35:43.347 INFO:tasks.workunit.client.0.smithi007.stdout:8/907: creat d0/d3/d16/d93/d5e/de3/d35/df6/f10a x:0 0 0 2024-09-21T04:35:43.351 INFO:tasks.workunit.client.0.smithi007.stdout:8/908: rename d0/d3/d74/d52 to d0/d3/d16/d93/d5e/de3/d10b 0 2024-09-21T04:35:43.469 INFO:tasks.workunit.client.0.smithi007.stdout:6/859: sync 2024-09-21T04:35:43.498 INFO:tasks.workunit.client.0.smithi007.stdout:6/860: truncate d3/d3e/d99/dbb/f9a 55509 0 2024-09-21T04:35:43.502 INFO:tasks.workunit.client.0.smithi007.stdout:6/861: truncate d3/de/d12/d19/f39 3046011 0 2024-09-21T04:35:43.508 INFO:tasks.workunit.client.0.smithi007.stdout:6/862: getdents d3/d3b/dd3/d60 0 2024-09-21T04:35:43.517 INFO:tasks.workunit.client.0.smithi007.stdout:6/863: rmdir d3/d94/df9 0 2024-09-21T04:35:43.535 INFO:tasks.workunit.client.0.smithi007.stdout:6/864: dread d3/de/d12/d19/f39 [0,4194304] 0 2024-09-21T04:35:43.536 INFO:tasks.workunit.client.0.smithi007.stdout:6/865: dread - d3/d94/f101 zero size 2024-09-21T04:35:43.538 INFO:tasks.workunit.client.0.smithi007.stdout:6/866: truncate d3/d3b/dd3/d60/d77/fec 199658 0 2024-09-21T04:35:43.541 INFO:tasks.workunit.client.0.smithi007.stdout:6/867: truncate d3/de/d12/d19/f64 869607 0 2024-09-21T04:35:43.542 INFO:tasks.workunit.client.0.smithi007.stdout:6/868: write d3/de/d12/f7c [1205073,24519] 0 2024-09-21T04:35:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:43 smithi053 ceph-mon[35554]: from='client.24501 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:43 smithi007 ceph-mon[27954]: from='client.24501 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:43.830 INFO:tasks.workunit.client.0.smithi007.stdout:3/973: sync 2024-09-21T04:35:43.850 INFO:tasks.workunit.client.0.smithi007.stdout:3/974: write dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/f93 [4038547,28867] 0 2024-09-21T04:35:43.852 INFO:tasks.workunit.client.0.smithi007.stdout:3/975: read dc/d22/d29/d32/d4d/d84/dd0/da6/f60 [1266135,107995] 0 2024-09-21T04:35:43.854 INFO:tasks.workunit.client.0.smithi007.stdout:3/976: creat dc/d22/d29/d92/db9/f132 x:0 0 0 2024-09-21T04:35:43.855 INFO:tasks.workunit.client.0.smithi007.stdout:3/977: chown dc/d18/c63 180438 1 2024-09-21T04:35:43.860 INFO:tasks.workunit.client.0.smithi007.stdout:3/978: write dc/d22/d29/d32/d69/f40 [4031023,94613] 0 2024-09-21T04:35:43.866 INFO:tasks.workunit.client.0.smithi007.stdout:3/979: dwrite dc/d18/f31 [4194304,4194304] 0 2024-09-21T04:35:43.871 INFO:tasks.workunit.client.0.smithi007.stdout:3/980: getdents dc/d22/d29/d32/d4d/d84/dd0/dbc/dd1 0 2024-09-21T04:35:43.873 INFO:tasks.workunit.client.0.smithi007.stdout:3/981: mkdir dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/d133 0 2024-09-21T04:35:43.896 INFO:tasks.workunit.client.0.smithi007.stdout:3/982: truncate dc/f21 2429914 0 2024-09-21T04:35:43.905 INFO:tasks.workunit.client.0.smithi007.stdout:3/983: link dc/d22/d29/d32/d4d/d84/dd0/da6/de3/le2 dc/d22/d29/d32/d4d/d84/dd0/da6/de3/l134 0 2024-09-21T04:35:43.909 INFO:tasks.workunit.client.0.smithi007.stdout:3/984: mknod dc/d22/d29/d32/d4d/d84/dbb/dc1/dca/d10e/c135 0 2024-09-21T04:35:43.914 INFO:tasks.workunit.client.0.smithi007.stdout:3/985: fdatasync dc/d22/d29/d32/d4d/d84/dd0/da6/f60 0 2024-09-21T04:35:43.917 INFO:tasks.workunit.client.0.smithi007.stdout:3/986: rmdir dc/d6c/da4 39 2024-09-21T04:35:43.920 INFO:tasks.workunit.client.0.smithi007.stdout:3/987: getdents dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/d133 0 2024-09-21T04:35:43.923 INFO:tasks.workunit.client.0.smithi007.stdout:3/988: mknod dc/d22/d29/d32/d4d/d84/dd0/dfb/c136 0 2024-09-21T04:35:43.929 INFO:tasks.workunit.client.0.smithi007.stdout:3/989: dwrite dc/d18/f31 [4194304,4194304] 0 2024-09-21T04:35:43.931 INFO:tasks.workunit.client.0.smithi007.stdout:3/990: truncate dc/d22/d29/d32/d4d/fc2 1002768 0 2024-09-21T04:35:43.946 INFO:tasks.workunit.client.0.smithi007.stdout:3/991: write dc/d22/d29/d92/db9/ffc [1322166,21988] 0 2024-09-21T04:35:43.949 INFO:tasks.workunit.client.0.smithi007.stdout:3/992: unlink dc/d22/d29/dac/f12c 0 2024-09-21T04:35:43.962 INFO:tasks.workunit.client.0.smithi007.stdout:3/993: dread dc/d18/f39 [0,4194304] 0 2024-09-21T04:35:43.973 INFO:tasks.workunit.client.0.smithi007.stdout:3/994: dwrite dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/d71/fcf [0,4194304] 0 2024-09-21T04:35:43.975 INFO:tasks.workunit.client.0.smithi007.stdout:3/995: fsync dc/d22/f59 0 2024-09-21T04:35:43.976 INFO:tasks.workunit.client.0.smithi007.stdout:3/996: chown dc/d22/d29/d32/d4d/d84/dd0/ddb 174 1 2024-09-21T04:35:43.980 INFO:tasks.workunit.client.0.smithi007.stdout:3/997: symlink dc/d22/d29/d32/d4d/d7d/d9a/df0/d116/d71/l137 0 2024-09-21T04:35:43.983 INFO:tasks.workunit.client.0.smithi007.stdout:3/998: mknod dc/d22/d29/d92/c138 0 2024-09-21T04:35:43.983 INFO:tasks.workunit.client.0.smithi007.stdout:3/999: chown dc/d18/f45 2 1 2024-09-21T04:35:44.166 INFO:tasks.workunit.client.0.smithi007.stdout:2/774: sync 2024-09-21T04:35:44.168 INFO:tasks.workunit.client.0.smithi007.stdout:2/775: mknod d0/d11/d31/da4/cef 0 2024-09-21T04:35:44.171 INFO:tasks.workunit.client.0.smithi007.stdout:2/776: creat d0/d11/d31/d7b/de8/d81/ff0 x:0 0 0 2024-09-21T04:35:44.174 INFO:tasks.workunit.client.0.smithi007.stdout:2/777: creat d0/d11/d31/d7b/de8/db9/ff1 x:0 0 0 2024-09-21T04:35:44.175 INFO:tasks.workunit.client.0.smithi007.stdout:2/778: chown d0/d11/d31/d7b/de8/d81/da0/f6 0 1 2024-09-21T04:35:44.175 INFO:tasks.workunit.client.0.smithi007.stdout:2/779: fdatasync d0/f2e 0 2024-09-21T04:35:44.177 INFO:tasks.workunit.client.0.smithi007.stdout:2/780: write d0/d11/d31/d7b/de8/db9/fcd [10943,31850] 0 2024-09-21T04:35:44.325 INFO:tasks.workunit.client.0.smithi007.stdout:8/909: sync 2024-09-21T04:35:44.326 INFO:tasks.workunit.client.0.smithi007.stdout:6/869: sync 2024-09-21T04:35:44.326 INFO:tasks.workunit.client.0.smithi007.stdout:7/995: sync 2024-09-21T04:35:44.326 INFO:tasks.workunit.client.0.smithi007.stdout:0/901: sync 2024-09-21T04:35:44.328 INFO:tasks.workunit.client.0.smithi007.stdout:8/910: symlink d0/d3/d82/l10c 0 2024-09-21T04:35:44.329 INFO:tasks.workunit.client.0.smithi007.stdout:8/911: chown d0/d3/d16/d93/c78 22 1 2024-09-21T04:35:44.334 INFO:tasks.workunit.client.0.smithi007.stdout:7/996: truncate d3/d6/d14/d20/d58/f87 3653868 0 2024-09-21T04:35:44.335 INFO:tasks.workunit.client.0.smithi007.stdout:7/997: fsync d3/d6/d14/d20/d80/db2/db6/f126 0 2024-09-21T04:35:44.336 INFO:tasks.workunit.client.0.smithi007.stdout:7/998: chown d3/d6/d14/d20/d80/dca/d113/d151/faf 3138 1 2024-09-21T04:35:44.341 INFO:tasks.workunit.client.0.smithi007.stdout:8/912: getdents d0/d3/d16/d93/d5e/de3/d10b 0 2024-09-21T04:35:44.341 INFO:tasks.workunit.client.0.smithi007.stdout:0/902: write d6/d75/dc5/d18/d2e/feb [189546,115149] 0 2024-09-21T04:35:44.343 INFO:tasks.workunit.client.0.smithi007.stdout:8/913: chown d0/d3/d7c/f54 0 1 2024-09-21T04:35:44.343 INFO:tasks.workunit.client.0.smithi007.stdout:7/999: dwrite d3/d6/d14/d20/d80/dca/d113/d151/faf [4194304,4194304] 0 2024-09-21T04:35:44.347 INFO:tasks.workunit.client.0.smithi007.stdout:6/870: truncate d3/de/d12/d19/f20 835415 0 2024-09-21T04:35:44.349 INFO:tasks.workunit.client.0.smithi007.stdout:8/914: read d0/d3/d8/d46/db7/fe2 [2206273,57397] 0 2024-09-21T04:35:44.351 INFO:tasks.workunit.client.0.smithi007.stdout:2/781: sync 2024-09-21T04:35:44.351 INFO:tasks.workunit.client.0.smithi007.stdout:8/915: write d0/d3/d8/f105 [335339,49942] 0 2024-09-21T04:35:44.356 INFO:tasks.workunit.client.0.smithi007.stdout:6/871: dread d3/de/d12/d19/f64 [0,4194304] 0 2024-09-21T04:35:44.356 INFO:tasks.workunit.client.0.smithi007.stdout:2/782: truncate d0/d11/d31/d7b/de8/fd9 234809 0 2024-09-21T04:35:44.358 INFO:tasks.workunit.client.0.smithi007.stdout:8/916: creat d0/d3/d16/d93/d40/d92/dcb/f10d x:0 0 0 2024-09-21T04:35:44.358 INFO:tasks.workunit.client.0.smithi007.stdout:0/903: dwrite d6/d75/dc5/d52/f5c [0,4194304] 0 2024-09-21T04:35:44.359 INFO:tasks.workunit.client.0.smithi007.stdout:8/917: write d0/d3/d82/ffc [765853,33237] 0 2024-09-21T04:35:44.361 INFO:tasks.workunit.client.0.smithi007.stdout:8/918: chown d0/d3/d16/d93/d40/d92/lda 102 1 2024-09-21T04:35:44.366 INFO:tasks.workunit.client.0.smithi007.stdout:0/904: symlink d6/d75/dc5/d47/d99/l148 0 2024-09-21T04:35:44.367 INFO:tasks.workunit.client.0.smithi007.stdout:6/872: getdents d3/d3b/dd3/d60/d77/dcb/df4 0 2024-09-21T04:35:44.372 INFO:tasks.workunit.client.0.smithi007.stdout:8/919: creat d0/d3/d82/f10e x:0 0 0 2024-09-21T04:35:44.374 INFO:tasks.workunit.client.0.smithi007.stdout:0/905: chown d6/d75/dc5/f5e 7454 1 2024-09-21T04:35:44.376 INFO:tasks.workunit.client.0.smithi007.stdout:0/906: write d6/d75/dc5/d18/d2e/d60/f142 [754712,7377] 0 2024-09-21T04:35:44.378 INFO:tasks.workunit.client.0.smithi007.stdout:2/783: write d0/d11/d31/d46/d48/f82 [565618,15982] 0 2024-09-21T04:35:44.391 INFO:tasks.workunit.client.0.smithi007.stdout:2/784: creat d0/d11/d31/d39/ff2 x:0 0 0 2024-09-21T04:35:44.395 INFO:tasks.workunit.client.0.smithi007.stdout:0/907: mknod d6/d75/dc5/d47/c149 0 2024-09-21T04:35:44.399 INFO:tasks.workunit.client.0.smithi007.stdout:6/873: creat d3/d3e/f11a x:0 0 0 2024-09-21T04:35:44.400 INFO:tasks.workunit.client.0.smithi007.stdout:2/785: dwrite d0/d11/d31/d7b/de8/db9/ddd/fe0 [0,4194304] 0 2024-09-21T04:35:44.405 INFO:tasks.workunit.client.0.smithi007.stdout:8/920: dwrite d0/d3/d8/f5f [0,4194304] 0 2024-09-21T04:35:44.406 INFO:tasks.workunit.client.0.smithi007.stdout:8/921: fdatasync d0/f43 0 2024-09-21T04:35:44.416 INFO:tasks.workunit.client.0.smithi007.stdout:2/786: link d0/d11/ced d0/d11/d6e/cf3 0 2024-09-21T04:35:44.424 INFO:tasks.workunit.client.0.smithi007.stdout:6/874: getdents d3/de/d12/d19/da3 0 2024-09-21T04:35:44.428 INFO:tasks.workunit.client.0.smithi007.stdout:0/908: write d6/d75/dc5/f5e [918047,102256] 0 2024-09-21T04:35:44.430 INFO:tasks.workunit.client.0.smithi007.stdout:0/909: write d6/d75/dc5/d47/d99/f11a [914399,14143] 0 2024-09-21T04:35:44.462 INFO:tasks.workunit.client.0.smithi007.stdout:8/922: getdents d0/d3/d8/d46/db7 0 2024-09-21T04:35:44.478 INFO:tasks.workunit.client.0.smithi007.stdout:8/923: dwrite d0/fa6 [0,4194304] 0 2024-09-21T04:35:44.478 INFO:tasks.workunit.client.0.smithi007.stdout:0/910: symlink d6/d75/dc5/d52/d6d/l14a 0 2024-09-21T04:35:44.482 INFO:tasks.workunit.client.0.smithi007.stdout:6/875: creat d3/f11b x:0 0 0 2024-09-21T04:35:44.486 INFO:tasks.workunit.client.0.smithi007.stdout:8/924: creat d0/d3/d7c/f10f x:0 0 0 2024-09-21T04:35:44.490 INFO:tasks.workunit.client.0.smithi007.stdout:0/911: creat d6/d75/dc5/d18/d27/f14b x:0 0 0 2024-09-21T04:35:44.492 INFO:tasks.workunit.client.0.smithi007.stdout:0/912: readlink d6/d75/dc5/d47/d8a/l12e 0 2024-09-21T04:35:44.493 INFO:tasks.workunit.client.0.smithi007.stdout:0/913: chown d6/d75/dc5/d57/c123 199 1 2024-09-21T04:35:44.500 INFO:tasks.workunit.client.0.smithi007.stdout:6/876: getdents d3/d8e/dae 0 2024-09-21T04:35:44.504 INFO:tasks.workunit.client.0.smithi007.stdout:6/877: mkdir d3/de/d12/d19/da3/d11c 0 2024-09-21T04:35:44.505 INFO:tasks.workunit.client.0.smithi007.stdout:0/914: getdents d6/d75/dc5/d47/d8a/de9/d101/d109/d126 0 2024-09-21T04:35:44.514 INFO:tasks.workunit.client.0.smithi007.stdout:6/878: dwrite d3/de/f4f [0,4194304] 0 2024-09-21T04:35:44.516 INFO:tasks.workunit.client.0.smithi007.stdout:6/879: chown d3/d3e/d99/c9c 1691 1 2024-09-21T04:35:44.529 INFO:tasks.workunit.client.0.smithi007.stdout:6/880: creat d3/d3e/d99/dbb/d95/f11d x:0 0 0 2024-09-21T04:35:44.534 INFO:tasks.workunit.client.0.smithi007.stdout:6/881: truncate d3/d3b/dd3/d60/f6e 1320103 0 2024-09-21T04:35:44.537 INFO:tasks.workunit.client.0.smithi007.stdout:2/787: truncate d0/d11/d31/d46/d48/f62 373402 0 2024-09-21T04:35:44.539 INFO:tasks.workunit.client.0.smithi007.stdout:6/882: mkdir d3/de/d12/d19/da3/de3/d104/d112/d116/d11e 0 2024-09-21T04:35:44.546 INFO:tasks.workunit.client.0.smithi007.stdout:6/883: rmdir d3/d3b/dd3/d60/dcc 0 2024-09-21T04:35:44.547 INFO:tasks.workunit.client.0.smithi007.stdout:2/788: link d0/d11/d31/d7b/de8/d81/da0/f88 d0/d11/d31/d7b/de8/daa/ff4 0 2024-09-21T04:35:44.549 INFO:tasks.workunit.client.0.smithi007.stdout:6/884: mkdir d3/de/d12/d19/da3/de3/d104/d112/d11f 0 2024-09-21T04:35:44.549 INFO:tasks.workunit.client.0.smithi007.stdout:0/915: truncate d6/d75/dc5/d18/d2e/f87 716517 0 2024-09-21T04:35:44.550 INFO:tasks.workunit.client.0.smithi007.stdout:6/885: dread - d3/de/d58/f90 zero size 2024-09-21T04:35:44.553 INFO:tasks.workunit.client.0.smithi007.stdout:2/789: getdents d0/d11/d31/da4 0 2024-09-21T04:35:44.554 INFO:tasks.workunit.client.0.smithi007.stdout:2/790: chown d0/fd 1 1 2024-09-21T04:35:44.555 INFO:tasks.workunit.client.0.smithi007.stdout:0/916: dread d6/d75/dc5/d18/d2e/d60/f142 [0,4194304] 0 2024-09-21T04:35:44.556 INFO:tasks.workunit.client.0.smithi007.stdout:2/791: readlink d0/d11/d31/d9f/da9/le7 0 2024-09-21T04:35:44.556 INFO:tasks.workunit.client.0.smithi007.stdout:6/886: mknod d3/d3b/dd3/d60/d98/de4/c120 0 2024-09-21T04:35:44.559 INFO:tasks.workunit.client.0.smithi007.stdout:2/792: write d0/d11/d31/d46/d77/fbe [326726,80602] 0 2024-09-21T04:35:44.559 INFO:tasks.workunit.client.0.smithi007.stdout:0/917: chown d6/d75/dc5/d18/f5d 11 1 2024-09-21T04:35:44.559 INFO:tasks.workunit.client.0.smithi007.stdout:6/887: dread - d3/d3b/dd3/d60/dd7/ff6 zero size 2024-09-21T04:35:44.581 INFO:tasks.workunit.client.0.smithi007.stdout:6/888: write d3/de/f1a [1563838,1406] 0 2024-09-21T04:35:44.582 INFO:tasks.workunit.client.0.smithi007.stdout:2/793: write d0/fa3 [571336,69417] 0 2024-09-21T04:35:44.587 INFO:tasks.workunit.client.0.smithi007.stdout:0/918: dwrite d6/d75/ff9 [0,4194304] 0 2024-09-21T04:35:44.593 INFO:tasks.workunit.client.0.smithi007.stdout:2/794: symlink d0/d11/d6e/deb/lf5 0 2024-09-21T04:35:44.593 INFO:tasks.workunit.client.0.smithi007.stdout:6/889: write d3/d3e/f40 [1301656,26547] 0 2024-09-21T04:35:44.595 INFO:tasks.workunit.client.0.smithi007.stdout:2/795: dread - d0/d11/d31/d39/f84 zero size 2024-09-21T04:35:44.595 INFO:tasks.workunit.client.0.smithi007.stdout:6/890: chown d3/d3e/f40 77586845 1 2024-09-21T04:35:44.596 INFO:tasks.workunit.client.0.smithi007.stdout:0/919: mkdir d6/d75/dc5/d47/d99/d110/d14c 0 2024-09-21T04:35:44.599 INFO:tasks.workunit.client.0.smithi007.stdout:6/891: fdatasync d3/d8e/dae/fbc 0 2024-09-21T04:35:44.607 INFO:tasks.workunit.client.0.smithi007.stdout:0/920: truncate d6/d75/dc5/f5e 3044149 0 2024-09-21T04:35:44.610 INFO:tasks.workunit.client.0.smithi007.stdout:0/921: creat d6/d75/dc9/f14d x:0 0 0 2024-09-21T04:35:44.612 INFO:tasks.workunit.client.0.smithi007.stdout:2/796: dwrite d0/d11/db1/fb3 [0,4194304] 0 2024-09-21T04:35:44.617 INFO:tasks.workunit.client.0.smithi007.stdout:6/892: dwrite d3/de/d12/d19/dd8/f106 [0,4194304] 0 2024-09-21T04:35:44.619 INFO:tasks.workunit.client.0.smithi007.stdout:0/922: creat d6/d75/dc5/d47/d99/f14e x:0 0 0 2024-09-21T04:35:44.620 INFO:tasks.workunit.client.0.smithi007.stdout:2/797: symlink d0/d11/d31/d7b/de8/d81/da0/dd3/lf6 0 2024-09-21T04:35:44.632 INFO:tasks.workunit.client.0.smithi007.stdout:0/923: read d6/d75/dc5/f5e [1800551,88522] 0 2024-09-21T04:35:44.635 INFO:tasks.workunit.client.0.smithi007.stdout:6/893: dwrite d3/d3b/dd3/d60/dd7/ff3 [0,4194304] 0 2024-09-21T04:35:44.635 INFO:tasks.workunit.client.0.smithi007.stdout:0/924: write d6/d75/dc5/d18/d27/ff7 [815314,14267] 0 2024-09-21T04:35:44.640 INFO:tasks.workunit.client.0.smithi007.stdout:0/925: rmdir d6/d75/dc5/d47/d8a/df4 39 2024-09-21T04:35:44.641 INFO:tasks.workunit.client.0.smithi007.stdout:2/798: dread d0/d11/d31/d46/f6a [0,4194304] 0 2024-09-21T04:35:44.642 INFO:tasks.workunit.client.0.smithi007.stdout:6/894: rmdir d3/de/d12/d19/da3/de3/d104 39 2024-09-21T04:35:44.644 INFO:tasks.workunit.client.0.smithi007.stdout:0/926: truncate d6/d75/dc5/d47/d99/f11a 974463 0 2024-09-21T04:35:44.644 INFO:tasks.workunit.client.0.smithi007.stdout:2/799: creat d0/d11/d31/d39/ff7 x:0 0 0 2024-09-21T04:35:44.649 INFO:tasks.workunit.client.0.smithi007.stdout:6/895: dwrite d3/d3e/d99/dbb/f66 [0,4194304] 0 2024-09-21T04:35:44.649 INFO:tasks.workunit.client.0.smithi007.stdout:0/927: creat d6/d91/da3/f14f x:0 0 0 2024-09-21T04:35:44.651 INFO:tasks.workunit.client.0.smithi007.stdout:6/896: write d3/de/f1a [2607105,107395] 0 2024-09-21T04:35:44.653 INFO:tasks.workunit.client.0.smithi007.stdout:0/928: readlink d6/d75/dc5/d47/d8a/df4/lfe 0 2024-09-21T04:35:44.656 INFO:tasks.workunit.client.0.smithi007.stdout:6/897: rmdir d3/d3b/dd3/d60/dd7 39 2024-09-21T04:35:44.658 INFO:tasks.workunit.client.0.smithi007.stdout:0/929: truncate d6/d75/dc5/f22 3269983 0 2024-09-21T04:35:44.659 INFO:tasks.workunit.client.0.smithi007.stdout:6/898: write d3/de/d12/f7c [1197346,129072] 0 2024-09-21T04:35:44.662 INFO:tasks.workunit.client.0.smithi007.stdout:2/800: dwrite d0/d11/f4e [0,4194304] 0 2024-09-21T04:35:44.664 INFO:tasks.workunit.client.0.smithi007.stdout:0/930: fsync d6/d91/da3/dbf/dec/ffc 0 2024-09-21T04:35:44.669 INFO:tasks.workunit.client.0.smithi007.stdout:2/801: dwrite d0/d11/d31/d7b/fb2 [0,4194304] 0 2024-09-21T04:35:44.671 INFO:tasks.workunit.client.0.smithi007.stdout:2/802: fdatasync d0/d11/d31/d7b/de8/d81/da0/f68 0 2024-09-21T04:35:44.676 INFO:tasks.workunit.client.0.smithi007.stdout:0/931: truncate d6/d75/dc5/d57/d9d/f121 21124 0 2024-09-21T04:35:44.680 INFO:tasks.workunit.client.0.smithi007.stdout:0/932: unlink d6/fd4 0 2024-09-21T04:35:44.682 INFO:tasks.workunit.client.0.smithi007.stdout:2/803: symlink d0/d11/d31/d7b/de8/daa/lf8 0 2024-09-21T04:35:44.683 INFO:tasks.workunit.client.0.smithi007.stdout:6/899: write d3/f33 [1618282,62640] 0 2024-09-21T04:35:44.683 INFO:tasks.workunit.client.0.smithi007.stdout:2/804: write d0/fa3 [1633202,41648] 0 2024-09-21T04:35:44.689 INFO:tasks.workunit.client.0.smithi007.stdout:0/933: rename d6/d91/da3/dbf/f124 to d6/d75/dc5/d47/d8a/f150 0 2024-09-21T04:35:44.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:44 smithi053 ceph-mon[35554]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:44.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:44 smithi053 ceph-mon[35554]: pgmap v43: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 99 MiB/s wr, 281 op/s 2024-09-21T04:35:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:44 smithi007 ceph-mon[27954]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:35:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:44 smithi007 ceph-mon[27954]: pgmap v43: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 99 MiB/s wr, 281 op/s 2024-09-21T04:35:44.695 INFO:tasks.workunit.client.0.smithi007.stdout:0/934: dwrite d6/d75/dc5/d47/d99/fae [0,4194304] 0 2024-09-21T04:35:44.697 INFO:tasks.workunit.client.0.smithi007.stdout:2/805: creat d0/d11/d31/d7b/de8/daa/ff9 x:0 0 0 2024-09-21T04:35:44.700 INFO:tasks.workunit.client.0.smithi007.stdout:6/900: dwrite d3/d94/fc2 [4194304,4194304] 0 2024-09-21T04:35:44.704 INFO:tasks.workunit.client.0.smithi007.stdout:2/806: mkdir d0/d11/d31/d7b/de8/daa/dfa 0 2024-09-21T04:35:44.705 INFO:tasks.workunit.client.0.smithi007.stdout:6/901: symlink d3/de/d12/l121 0 2024-09-21T04:35:44.713 INFO:tasks.workunit.client.0.smithi007.stdout:6/902: getdents d3/d3e/d99/dbb/d95/dc9/dd9 0 2024-09-21T04:35:44.716 INFO:tasks.workunit.client.0.smithi007.stdout:0/935: dread d6/d75/dc5/f5e [0,4194304] 0 2024-09-21T04:35:44.717 INFO:tasks.workunit.client.0.smithi007.stdout:6/903: creat d3/d3e/f122 x:0 0 0 2024-09-21T04:35:44.717 INFO:tasks.workunit.client.0.smithi007.stdout:0/936: stat d6/d75/dc5/d18/l111 0 2024-09-21T04:35:44.718 INFO:tasks.workunit.client.0.smithi007.stdout:2/807: write d0/f23 [865248,33209] 0 2024-09-21T04:35:44.721 INFO:tasks.workunit.client.0.smithi007.stdout:6/904: creat d3/d3e/d99/dbb/d95/dc9/dd9/f123 x:0 0 0 2024-09-21T04:35:44.724 INFO:tasks.workunit.client.0.smithi007.stdout:0/937: creat d6/d75/dc5/f151 x:0 0 0 2024-09-21T04:35:44.724 INFO:tasks.workunit.client.0.smithi007.stdout:2/808: read d0/d11/d31/d46/f47 [440531,123623] 0 2024-09-21T04:35:44.727 INFO:tasks.workunit.client.0.smithi007.stdout:0/938: write d6/d13e/f143 [63303,2255] 0 2024-09-21T04:35:44.730 INFO:tasks.workunit.client.0.smithi007.stdout:6/905: dwrite d3/d3e/d99/dbb/d95/dc9/d10b/f10f [0,4194304] 0 2024-09-21T04:35:44.733 INFO:tasks.workunit.client.0.smithi007.stdout:2/809: dwrite d0/d11/f28 [0,4194304] 0 2024-09-21T04:35:44.735 INFO:tasks.workunit.client.0.smithi007.stdout:2/810: write d0/d11/d6e/fa1 [3629726,16886] 0 2024-09-21T04:35:44.735 INFO:tasks.workunit.client.0.smithi007.stdout:0/939: truncate d6/d75/f8f 2851810 0 2024-09-21T04:35:44.738 INFO:tasks.workunit.client.0.smithi007.stdout:2/811: write d0/d11/d31/d7b/de8/daa/ff4 [1360506,90692] 0 2024-09-21T04:35:44.752 INFO:tasks.workunit.client.0.smithi007.stdout:6/906: write d3/d3b/dd3/d60/d77/dcb/d88/fc5 [303686,57272] 0 2024-09-21T04:35:44.756 INFO:tasks.workunit.client.0.smithi007.stdout:6/907: mknod d3/d3e/d99/dbb/df1/c124 0 2024-09-21T04:35:44.757 INFO:tasks.workunit.client.0.smithi007.stdout:0/940: dread d6/d91/da3/dbf/fc3 [0,4194304] 0 2024-09-21T04:35:44.760 INFO:tasks.workunit.client.0.smithi007.stdout:2/812: dwrite d0/d11/d31/fdc [0,4194304] 0 2024-09-21T04:35:44.769 INFO:tasks.workunit.client.0.smithi007.stdout:2/813: dwrite d0/d11/d31/d7b/de8/d81/ff0 [0,4194304] 0 2024-09-21T04:35:44.770 INFO:tasks.workunit.client.0.smithi007.stdout:0/941: link d6/d75/dc5/f22 d6/d75/dc5/d52/d6d/de1/f152 0 2024-09-21T04:35:44.774 INFO:tasks.workunit.client.0.smithi007.stdout:6/908: dwrite d3/d3b/dd3/d60/d77/dcb/fa6 [0,4194304] 0 2024-09-21T04:35:44.776 INFO:tasks.workunit.client.0.smithi007.stdout:2/814: dread d0/d11/d31/d7b/fb2 [0,4194304] 0 2024-09-21T04:35:44.784 INFO:tasks.workunit.client.0.smithi007.stdout:2/815: dread d0/d11/d31/d46/f6a [0,4194304] 0 2024-09-21T04:35:44.786 INFO:tasks.workunit.client.0.smithi007.stdout:0/942: symlink d6/d75/dc5/d18/l153 0 2024-09-21T04:35:44.787 INFO:tasks.workunit.client.0.smithi007.stdout:6/909: dwrite d3/d3b/dd3/d60/dd7/ff3 [0,4194304] 0 2024-09-21T04:35:44.795 INFO:tasks.workunit.client.0.smithi007.stdout:6/910: symlink d3/d8e/dae/l125 0 2024-09-21T04:35:44.796 INFO:tasks.workunit.client.0.smithi007.stdout:6/911: dread - d3/d94/f110 zero size 2024-09-21T04:35:44.801 INFO:tasks.workunit.client.0.smithi007.stdout:2/816: dread d0/d11/f20 [0,4194304] 0 2024-09-21T04:35:44.803 INFO:tasks.workunit.client.0.smithi007.stdout:0/943: dwrite d6/d75/dc5/d52/d6d/fa6 [0,4194304] 0 2024-09-21T04:35:44.804 INFO:tasks.workunit.client.0.smithi007.stdout:2/817: truncate d0/d11/d6b/d9d/fc5 398161 0 2024-09-21T04:35:44.805 INFO:tasks.workunit.client.0.smithi007.stdout:8/925: sync 2024-09-21T04:35:44.811 INFO:tasks.workunit.client.0.smithi007.stdout:0/944: rename d6/d75/dc5/d18/d39/fac to d6/d75/dc5/d47/d8a/de9/d101/f154 0 2024-09-21T04:35:44.812 INFO:tasks.workunit.client.0.smithi007.stdout:8/926: read d0/d3/d16/d93/d5e/d7b/f7e [3520707,85338] 0 2024-09-21T04:35:44.812 INFO:tasks.workunit.client.0.smithi007.stdout:6/912: dwrite d3/f1e [0,4194304] 0 2024-09-21T04:35:44.818 INFO:tasks.workunit.client.0.smithi007.stdout:2/818: link d0/d11/d31/d39/l3b d0/d11/d31/d46/d6c/lfb 0 2024-09-21T04:35:44.819 INFO:tasks.workunit.client.0.smithi007.stdout:8/927: symlink d0/d3/d16/d93/d40/d92/dcb/l110 0 2024-09-21T04:35:44.821 INFO:tasks.workunit.client.0.smithi007.stdout:6/913: creat d3/d3e/d99/dbb/f126 x:0 0 0 2024-09-21T04:35:44.823 INFO:tasks.workunit.client.0.smithi007.stdout:6/914: write d3/de/d12/fe5 [2858445,61652] 0 2024-09-21T04:35:44.824 INFO:tasks.workunit.client.0.smithi007.stdout:2/819: mkdir d0/d11/d31/d7b/de8/d81/dfc 0 2024-09-21T04:35:44.826 INFO:tasks.workunit.client.0.smithi007.stdout:8/928: unlink d0/d3/d16/d93/d5e/de3/d4d/d5d/f86 0 2024-09-21T04:35:44.829 INFO:tasks.workunit.client.0.smithi007.stdout:0/945: dwrite d6/d75/dc5/d18/fab [4194304,4194304] 0 2024-09-21T04:35:44.829 INFO:tasks.workunit.client.0.smithi007.stdout:6/915: fdatasync d3/d3b/dd3/d60/d77/dcb/f92 0 2024-09-21T04:35:44.831 INFO:tasks.workunit.client.0.smithi007.stdout:0/946: chown d6/d75/dc5/d47/d8a/de9/d101 19015833 1 2024-09-21T04:35:44.833 INFO:tasks.workunit.client.0.smithi007.stdout:2/820: rename d0/d11/d31/d9f/da9/de3 to d0/d11/db1/dbf/dce/dfd 0 2024-09-21T04:35:44.835 INFO:tasks.workunit.client.0.smithi007.stdout:0/947: fsync d6/d75/dc5/d52/f5c 0 2024-09-21T04:35:44.843 INFO:tasks.workunit.client.0.smithi007.stdout:8/929: dread d0/d3/d16/d93/d40/d92/dcb/fdd [0,4194304] 0 2024-09-21T04:35:44.848 INFO:tasks.workunit.client.0.smithi007.stdout:6/916: write d3/de/d12/d19/da3/fac [1448758,25877] 0 2024-09-21T04:35:44.850 INFO:tasks.workunit.client.0.smithi007.stdout:8/930: unlink d0/d3/d8/l14 0 2024-09-21T04:35:44.851 INFO:tasks.workunit.client.0.smithi007.stdout:8/931: truncate d0/d3/d8/f100 274461 0 2024-09-21T04:35:44.855 INFO:tasks.workunit.client.0.smithi007.stdout:2/821: rename d0/d11/d31/d7b/de8/daa to d0/d11/d31/dfe 0 2024-09-21T04:35:44.856 INFO:tasks.workunit.client.0.smithi007.stdout:8/932: fdatasync d0/fdb 0 2024-09-21T04:35:44.859 INFO:tasks.workunit.client.0.smithi007.stdout:0/948: write d6/d75/dc5/d18/d2e/f87 [1168268,97234] 0 2024-09-21T04:35:44.870 INFO:tasks.workunit.client.0.smithi007.stdout:6/917: dwrite d3/d3b/dd3/d60/d77/dcb/d88/fde [0,4194304] 0 2024-09-21T04:35:44.870 INFO:tasks.workunit.client.0.smithi007.stdout:0/949: write d6/d75/dc5/d18/d2e/f8b [578187,129390] 0 2024-09-21T04:35:44.876 INFO:tasks.workunit.client.0.smithi007.stdout:8/933: dwrite d0/d3/d16/d93/d40/f45 [0,4194304] 0 2024-09-21T04:35:44.878 INFO:tasks.workunit.client.0.smithi007.stdout:2/822: getdents d0/d11/d31/d7b 0 2024-09-21T04:35:44.886 INFO:tasks.workunit.client.0.smithi007.stdout:2/823: mknod d0/d11/cff 0 2024-09-21T04:35:44.886 INFO:tasks.workunit.client.0.smithi007.stdout:6/918: dwrite d3/de/f6a [0,4194304] 0 2024-09-21T04:35:44.890 INFO:tasks.workunit.client.0.smithi007.stdout:2/824: symlink d0/d11/d31/d7b/de8/d81/da0/dd3/l100 0 2024-09-21T04:35:44.892 INFO:tasks.workunit.client.0.smithi007.stdout:2/825: write d0/d11/d31/d46/d6c/db7/fc6 [716136,32326] 0 2024-09-21T04:35:44.893 INFO:tasks.workunit.client.0.smithi007.stdout:2/826: dread - d0/d11/d31/d52/f87 zero size 2024-09-21T04:35:44.895 INFO:tasks.workunit.client.0.smithi007.stdout:2/827: write d0/d11/f20 [4121742,46085] 0 2024-09-21T04:35:44.900 INFO:tasks.workunit.client.0.smithi007.stdout:8/934: write d0/d3/d16/d93/f17 [1225514,70142] 0 2024-09-21T04:35:44.901 INFO:tasks.workunit.client.0.smithi007.stdout:0/950: dread d6/d75/dc5/f22 [0,4194304] 0 2024-09-21T04:35:44.903 INFO:tasks.workunit.client.0.smithi007.stdout:0/951: dread d6/d75/dc5/d47/df0/f11d [0,4194304] 0 2024-09-21T04:35:44.906 INFO:tasks.workunit.client.0.smithi007.stdout:6/919: dread d3/de/d12/fe5 [0,4194304] 0 2024-09-21T04:35:44.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/828: truncate d0/d11/d31/d7b/de8/db9/ddd/fe0 2574895 0 2024-09-21T04:35:44.910 INFO:tasks.workunit.client.0.smithi007.stdout:2/829: chown d0/d11/d31/d39/fcc 8 1 2024-09-21T04:35:44.911 INFO:tasks.workunit.client.0.smithi007.stdout:6/920: truncate d3/d3b/dd3/d60/d77/f69 1176435 0 2024-09-21T04:35:44.912 INFO:tasks.workunit.client.0.smithi007.stdout:8/935: dread d0/f20 [0,4194304] 0 2024-09-21T04:35:44.914 INFO:tasks.workunit.client.0.smithi007.stdout:2/830: creat d0/d11/d6e/f101 x:0 0 0 2024-09-21T04:35:44.915 INFO:tasks.workunit.client.0.smithi007.stdout:6/921: mkdir d3/d8e/d127 0 2024-09-21T04:35:44.916 INFO:tasks.workunit.client.0.smithi007.stdout:6/922: stat d3/de/d58/c7e 0 2024-09-21T04:35:44.916 INFO:tasks.workunit.client.0.smithi007.stdout:2/831: write d0/d11/d31/d46/d6c/fb4 [2086907,25352] 0 2024-09-21T04:35:44.919 INFO:tasks.workunit.client.0.smithi007.stdout:2/832: dread d0/d11/d31/d7b/de8/fd9 [0,4194304] 0 2024-09-21T04:35:44.920 INFO:tasks.workunit.client.0.smithi007.stdout:8/936: symlink d0/d3/d16/l111 0 2024-09-21T04:35:44.922 INFO:tasks.workunit.client.0.smithi007.stdout:0/952: link d6/d75/dc5/d47/l11e d6/d75/dc5/d18/l155 0 2024-09-21T04:35:44.924 INFO:tasks.workunit.client.0.smithi007.stdout:6/923: fsync d3/de/d12/d19/da3/fe6 0 2024-09-21T04:35:44.927 INFO:tasks.workunit.client.0.smithi007.stdout:8/937: symlink d0/d3/d16/d93/d40/d92/dcb/l112 0 2024-09-21T04:35:44.931 INFO:tasks.workunit.client.0.smithi007.stdout:2/833: mkdir d0/d11/d31/d7b/de8/d81/dfc/d102 0 2024-09-21T04:35:44.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/938: dwrite d0/d3/d8/d46/f63 [0,4194304] 0 2024-09-21T04:35:44.935 INFO:tasks.workunit.client.0.smithi007.stdout:0/953: creat d6/d75/d13b/f156 x:0 0 0 2024-09-21T04:35:44.939 INFO:tasks.workunit.client.0.smithi007.stdout:2/834: dread d0/d11/d31/d7b/de8/db9/fcd [0,4194304] 0 2024-09-21T04:35:44.941 INFO:tasks.workunit.client.0.smithi007.stdout:8/939: rmdir d0/d3/d7c/dbf 39 2024-09-21T04:35:44.943 INFO:tasks.workunit.client.0.smithi007.stdout:0/954: mkdir d6/d75/dc5/d47/d8a/de9/d101/d157 0 2024-09-21T04:35:44.943 INFO:tasks.workunit.client.0.smithi007.stdout:0/955: stat d6/d75/dc5/d57/l66 0 2024-09-21T04:35:44.944 INFO:tasks.workunit.client.0.smithi007.stdout:0/956: chown d6/d75/dc5/d18/d39/f12d 500719 1 2024-09-21T04:35:44.947 INFO:tasks.workunit.client.0.smithi007.stdout:2/835: creat d0/d11/d31/d7b/de8/db9/ddd/f103 x:0 0 0 2024-09-21T04:35:44.949 INFO:tasks.workunit.client.0.smithi007.stdout:8/940: symlink d0/d3/d16/d93/d40/d9e/l113 0 2024-09-21T04:35:44.952 INFO:tasks.workunit.client.0.smithi007.stdout:8/941: symlink d0/d3/d8/d46/db7/l114 0 2024-09-21T04:35:44.953 INFO:tasks.workunit.client.0.smithi007.stdout:2/836: mkdir d0/d11/d31/d7b/de8/d81/dfc/d102/d104 0 2024-09-21T04:35:44.955 INFO:tasks.workunit.client.0.smithi007.stdout:8/942: fdatasync d0/d3/d82/fb9 0 2024-09-21T04:35:44.955 INFO:tasks.workunit.client.0.smithi007.stdout:2/837: chown d0/d11/d6b/l74 112 1 2024-09-21T04:35:44.956 INFO:tasks.workunit.client.0.smithi007.stdout:0/957: dread d6/d75/dc5/d47/d8a/fc4 [0,4194304] 0 2024-09-21T04:35:44.957 INFO:tasks.workunit.client.0.smithi007.stdout:2/838: chown d0/d11/d31/d46 31 1 2024-09-21T04:35:44.958 INFO:tasks.workunit.client.0.smithi007.stdout:6/924: dwrite d3/d3e/d99/dbb/f9a [0,4194304] 0 2024-09-21T04:35:44.958 INFO:tasks.workunit.client.0.smithi007.stdout:8/943: write d0/d3/d8/f5f [3406333,91878] 0 2024-09-21T04:35:44.960 INFO:tasks.workunit.client.0.smithi007.stdout:8/944: stat d0/d3/d16/d93/d40/d92/cd4 0 2024-09-21T04:35:44.960 INFO:tasks.workunit.client.0.smithi007.stdout:6/925: chown d3/d94/fc2 277315 1 2024-09-21T04:35:44.968 INFO:tasks.workunit.client.0.smithi007.stdout:6/926: mkdir d3/d8e/d127/d128 0 2024-09-21T04:35:44.970 INFO:tasks.workunit.client.0.smithi007.stdout:2/839: creat d0/d11/d31/dfe/f105 x:0 0 0 2024-09-21T04:35:44.973 INFO:tasks.workunit.client.0.smithi007.stdout:2/840: write d0/d11/d31/d7b/da2/da5/fb5 [125463,120555] 0 2024-09-21T04:35:44.974 INFO:tasks.workunit.client.0.smithi007.stdout:0/958: write d6/d75/dc5/f122 [1121221,16768] 0 2024-09-21T04:35:44.977 INFO:tasks.workunit.client.0.smithi007.stdout:8/945: truncate d0/d3/d16/d93/d5e/de3/d35/f87 2107530 0 2024-09-21T04:35:44.978 INFO:tasks.workunit.client.0.smithi007.stdout:6/927: dwrite d3/d3e/d99/dbb/f126 [0,4194304] 0 2024-09-21T04:35:44.981 INFO:tasks.workunit.client.0.smithi007.stdout:8/946: stat d0/d3/d16/d93/d5e/de3/d4d/d5d/d88 0 2024-09-21T04:35:44.983 INFO:tasks.workunit.client.0.smithi007.stdout:0/959: symlink d6/d75/dc5/d47/d99/d110/d14c/l158 0 2024-09-21T04:35:44.985 INFO:tasks.workunit.client.0.smithi007.stdout:0/960: chown d6/f13 3194 1 2024-09-21T04:35:44.985 INFO:tasks.workunit.client.0.smithi007.stdout:2/841: dwrite d0/d11/d6b/d9d/fd2 [0,4194304] 0 2024-09-21T04:35:44.988 INFO:tasks.workunit.client.0.smithi007.stdout:2/842: dread - d0/d11/d31/d39/f84 zero size 2024-09-21T04:35:44.989 INFO:tasks.workunit.client.0.smithi007.stdout:0/961: mkdir d6/d75/d13b/d159 0 2024-09-21T04:35:44.993 INFO:tasks.workunit.client.0.smithi007.stdout:8/947: dwrite d0/d3/d16/d93/d5e/de3/d4d/d5d/fd1 [4194304,4194304] 0 2024-09-21T04:35:44.996 INFO:tasks.workunit.client.0.smithi007.stdout:2/843: read d0/d11/d31/d7b/de8/d81/da0/fc [2722658,57586] 0 2024-09-21T04:35:44.996 INFO:tasks.workunit.client.0.smithi007.stdout:8/948: chown d0/d3/d16/d93/d5e/de3/d4d/d5d/d88 14 1 2024-09-21T04:35:44.997 INFO:tasks.workunit.client.0.smithi007.stdout:0/962: creat d6/d75/dc5/d52/d10a/f15a x:0 0 0 2024-09-21T04:35:45.000 INFO:tasks.workunit.client.0.smithi007.stdout:2/844: creat d0/d11/db1/dbf/dce/dfd/f106 x:0 0 0 2024-09-21T04:35:45.001 INFO:tasks.workunit.client.0.smithi007.stdout:2/845: dread - d0/d11/d31/d46/d6c/f78 zero size 2024-09-21T04:35:45.004 INFO:tasks.workunit.client.0.smithi007.stdout:0/963: creat d6/d91/da3/dbf/dec/f15b x:0 0 0 2024-09-21T04:35:45.005 INFO:tasks.workunit.client.0.smithi007.stdout:8/949: symlink d0/d3/d16/d93/d40/d92/dcb/df2/l115 0 2024-09-21T04:35:45.007 INFO:tasks.workunit.client.0.smithi007.stdout:2/846: mkdir d0/d11/d31/d46/d77/d107 0 2024-09-21T04:35:45.008 INFO:tasks.workunit.client.0.smithi007.stdout:0/964: dread d6/d75/dc5/d18/d27/f35 [0,4194304] 0 2024-09-21T04:35:45.009 INFO:tasks.workunit.client.0.smithi007.stdout:2/847: chown d0/d11/f4e 3 1 2024-09-21T04:35:45.014 INFO:tasks.workunit.client.0.smithi007.stdout:8/950: creat d0/d3/d7c/f116 x:0 0 0 2024-09-21T04:35:45.016 INFO:tasks.workunit.client.0.smithi007.stdout:8/951: write d0/d3/d16/d93/f104 [138610,42929] 0 2024-09-21T04:35:45.019 INFO:tasks.workunit.client.0.smithi007.stdout:2/848: rename d0/d11/db1 to d0/d11/d31/d7b/de8/db9/d108 0 2024-09-21T04:35:45.021 INFO:tasks.workunit.client.0.smithi007.stdout:8/952: unlink d0/d3/d16/d93/d5e/de3/d4d/l57 0 2024-09-21T04:35:45.023 INFO:tasks.workunit.client.0.smithi007.stdout:2/849: mknod d0/d11/d31/d7b/de8/d81/dfc/d102/c109 0 2024-09-21T04:35:45.024 INFO:tasks.workunit.client.0.smithi007.stdout:0/965: dread d6/d75/dc5/d18/d27/f29 [0,4194304] 0 2024-09-21T04:35:45.025 INFO:tasks.workunit.client.0.smithi007.stdout:8/953: dread - d0/d3/d16/d93/d5e/de3/d35/df6/f10a zero size 2024-09-21T04:35:45.025 INFO:tasks.workunit.client.0.smithi007.stdout:2/850: readlink d0/d11/d31/d7b/de8/d81/da0/l2c 0 2024-09-21T04:35:45.026 INFO:tasks.workunit.client.0.smithi007.stdout:0/966: readlink d6/d75/dc5/d18/d39/ld2 0 2024-09-21T04:35:45.027 INFO:tasks.workunit.client.0.smithi007.stdout:2/851: rmdir d0/d11/d31/d7b/de8/db9 39 2024-09-21T04:35:45.029 INFO:tasks.workunit.client.0.smithi007.stdout:2/852: chown d0/d11/d31/cab 5871 1 2024-09-21T04:35:45.029 INFO:tasks.workunit.client.0.smithi007.stdout:8/954: symlink d0/d3/d8/d46/l117 0 2024-09-21T04:35:45.030 INFO:tasks.workunit.client.0.smithi007.stdout:2/853: stat d0/d11/f4e 0 2024-09-21T04:35:45.031 INFO:tasks.workunit.client.0.smithi007.stdout:2/854: readlink d0/d11/l69 0 2024-09-21T04:35:45.036 INFO:tasks.workunit.client.0.smithi007.stdout:0/967: getdents d6/d75/dc5/d18 0 2024-09-21T04:35:45.038 INFO:tasks.workunit.client.0.smithi007.stdout:0/968: write d6/d75/dc5/d47/d8a/f140 [716611,96042] 0 2024-09-21T04:35:45.040 INFO:tasks.workunit.client.0.smithi007.stdout:2/855: dwrite d0/d11/d31/dfe/ff4 [0,4194304] 0 2024-09-21T04:35:45.042 INFO:tasks.workunit.client.0.smithi007.stdout:8/955: write d0/d3/d16/d93/d5e/de3/d35/f73 [673104,68765] 0 2024-09-21T04:35:45.043 INFO:tasks.workunit.client.0.smithi007.stdout:0/969: symlink d6/d75/dc5/d47/d8a/d9c/l15c 0 2024-09-21T04:35:45.047 INFO:tasks.workunit.client.0.smithi007.stdout:8/956: unlink d0/d3/f2b 0 2024-09-21T04:35:45.050 INFO:tasks.workunit.client.0.smithi007.stdout:2/856: creat d0/d11/d31/d7b/de8/d81/dfc/d102/d104/f10a x:0 0 0 2024-09-21T04:35:45.052 INFO:tasks.workunit.client.0.smithi007.stdout:0/970: dwrite d6/d75/d13b/f156 [0,4194304] 0 2024-09-21T04:35:45.053 INFO:tasks.workunit.client.0.smithi007.stdout:8/957: creat d0/d3/d82/f118 x:0 0 0 2024-09-21T04:35:45.058 INFO:tasks.workunit.client.0.smithi007.stdout:0/971: fsync d6/d75/dc5/d57/d9d/fdc 0 2024-09-21T04:35:45.059 INFO:tasks.workunit.client.0.smithi007.stdout:0/972: fsync d6/d75/dc5/d18/d2e/f8b 0 2024-09-21T04:35:45.064 INFO:tasks.workunit.client.0.smithi007.stdout:0/973: dread d6/d75/dc5/d47/d99/fae [0,4194304] 0 2024-09-21T04:35:45.065 INFO:tasks.workunit.client.0.smithi007.stdout:0/974: stat d6/d75/dc5/d52/d6d 0 2024-09-21T04:35:45.067 INFO:tasks.workunit.client.0.smithi007.stdout:2/857: write d0/d11/d31/d7b/da2/da5/fc9 [771466,46893] 0 2024-09-21T04:35:45.077 INFO:tasks.workunit.client.0.smithi007.stdout:0/975: write d6/d75/dc5/fba [338611,11927] 0 2024-09-21T04:35:45.079 INFO:tasks.workunit.client.0.smithi007.stdout:8/958: link d0/d3/d16/d93/d5e/de3/d4d/d5d/fee d0/d3/d16/f119 0 2024-09-21T04:35:45.081 INFO:tasks.workunit.client.0.smithi007.stdout:2/858: mkdir d0/d11/d31/dfe/d10b 0 2024-09-21T04:35:45.082 INFO:tasks.workunit.client.0.smithi007.stdout:0/976: creat d6/d75/f15d x:0 0 0 2024-09-21T04:35:45.084 INFO:tasks.workunit.client.0.smithi007.stdout:8/959: unlink d0/d3/d82/fb9 0 2024-09-21T04:35:45.084 INFO:tasks.workunit.client.0.smithi007.stdout:2/859: creat d0/d11/d6e/f10c x:0 0 0 2024-09-21T04:35:45.088 INFO:tasks.workunit.client.0.smithi007.stdout:0/977: mkdir d6/d75/dc5/d47/d15e 0 2024-09-21T04:35:45.089 INFO:tasks.workunit.client.0.smithi007.stdout:0/978: fsync d6/d75/ffd 0 2024-09-21T04:35:45.094 INFO:tasks.workunit.client.0.smithi007.stdout:2/860: dread d0/d11/d31/dfe/dbd/fc4 [0,4194304] 0 2024-09-21T04:35:45.096 INFO:tasks.workunit.client.0.smithi007.stdout:2/861: rmdir d0/d11/d31/d9f/da9 39 2024-09-21T04:35:45.203 INFO:tasks.workunit.client.0.smithi007.stdout:2/862: unlink d0/d11/d31/d39/f50 0 2024-09-21T04:35:45.211 INFO:tasks.workunit.client.0.smithi007.stdout:2/863: truncate d0/d11/d31/d7b/de8/db9/fcd 1590361 0 2024-09-21T04:35:45.213 INFO:tasks.workunit.client.0.smithi007.stdout:8/960: dwrite d0/fa6 [4194304,4194304] 0 2024-09-21T04:35:45.216 INFO:tasks.workunit.client.0.smithi007.stdout:2/864: chown d0/d11/d31/d46/d48/l71 59 1 2024-09-21T04:35:45.222 INFO:tasks.workunit.client.0.smithi007.stdout:0/979: dwrite d6/d75/dc5/d18/d2e/d3f/fe6 [0,4194304] 0 2024-09-21T04:35:45.226 INFO:tasks.workunit.client.0.smithi007.stdout:0/980: write d6/d91/da3/dbf/f12c [929465,1912] 0 2024-09-21T04:35:45.229 INFO:tasks.workunit.client.0.smithi007.stdout:2/865: write d0/f18 [2363014,22928] 0 2024-09-21T04:35:45.230 INFO:tasks.workunit.client.0.smithi007.stdout:2/866: chown d0/d11/d31/d39/f8c 0 1 2024-09-21T04:35:45.232 INFO:tasks.workunit.client.0.smithi007.stdout:0/981: unlink d6/d75/dc5/d47/d99/f11a 0 2024-09-21T04:35:45.236 INFO:tasks.workunit.client.0.smithi007.stdout:0/982: creat d6/d91/da3/dbf/dec/f15f x:0 0 0 2024-09-21T04:35:45.237 INFO:tasks.workunit.client.0.smithi007.stdout:2/867: mknod d0/d11/d31/dfe/d10b/c10d 0 2024-09-21T04:35:45.238 INFO:tasks.workunit.client.0.smithi007.stdout:8/961: dwrite d0/d3/d8/d46/db7/fbc [0,4194304] 0 2024-09-21T04:35:45.238 INFO:tasks.workunit.client.0.smithi007.stdout:2/868: fsync d0/f2e 0 2024-09-21T04:35:45.240 INFO:tasks.workunit.client.0.smithi007.stdout:2/869: write d0/d11/d31/d46/d6c/db7/fc6 [1084332,70472] 0 2024-09-21T04:35:45.243 INFO:tasks.workunit.client.0.smithi007.stdout:0/983: write d6/d75/dc5/d47/d99/fae [3452885,124282] 0 2024-09-21T04:35:45.244 INFO:tasks.workunit.client.0.smithi007.stdout:2/870: write d0/d11/d31/d7b/de8/d81/da0/f68 [1293786,73781] 0 2024-09-21T04:35:45.246 INFO:tasks.workunit.client.0.smithi007.stdout:2/871: write d0/d11/d31/d39/f44 [1211921,127512] 0 2024-09-21T04:35:45.246 INFO:tasks.workunit.client.0.smithi007.stdout:8/962: symlink d0/d3/d7c/l11a 0 2024-09-21T04:35:45.248 INFO:tasks.workunit.client.0.smithi007.stdout:2/872: write d0/d11/d31/d39/f44 [1289479,57457] 0 2024-09-21T04:35:45.251 INFO:tasks.workunit.client.0.smithi007.stdout:2/873: fdatasync d0/d11/f4e 0 2024-09-21T04:35:45.255 INFO:tasks.workunit.client.0.smithi007.stdout:2/874: chown d0/d11/d31/d46/d6c/fb4 1 1 2024-09-21T04:35:45.256 INFO:tasks.workunit.client.0.smithi007.stdout:8/963: dwrite d0/d3/d8/f105 [0,4194304] 0 2024-09-21T04:35:45.258 INFO:tasks.workunit.client.0.smithi007.stdout:0/984: dwrite d6/d75/dc5/d52/f78 [0,4194304] 0 2024-09-21T04:35:45.264 INFO:tasks.workunit.client.0.smithi007.stdout:0/985: chown d6/d75/dc5/d18/d27/ce2 7478 1 2024-09-21T04:35:45.272 INFO:tasks.workunit.client.0.smithi007.stdout:2/875: truncate d0/d11/d31/d7b/de8/d81/da0/f68 3892817 0 2024-09-21T04:35:45.272 INFO:tasks.workunit.client.0.smithi007.stdout:0/986: write d6/d75/dc5/d47/fa0 [388595,90965] 0 2024-09-21T04:35:45.275 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T04:35:45.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-21T04:35:45.276 INFO:tasks.workunit.client.0.smithi007.stdout:2/876: creat d0/d11/d31/d46/d77/f10e x:0 0 0 2024-09-21T04:35:45.276 INFO:tasks.workunit.client.0.smithi007.stdout:0/987: symlink d6/d75/dc5/d18/d39/dd5/l160 0 2024-09-21T04:35:45.279 INFO:tasks.workunit.client.0.smithi007.stdout:0/988: getdents d6/d75/dc5/d47/d8a/d9c 0 2024-09-21T04:35:45.280 INFO:tasks.workunit.client.0.smithi007.stdout:0/989: truncate d6/d75/dc5/d52/fb8 2293500 0 2024-09-21T04:35:45.282 INFO:tasks.workunit.client.0.smithi007.stdout:0/990: mknod d6/d75/dc5/d47/d8a/df4/c161 0 2024-09-21T04:35:45.291 INFO:tasks.workunit.client.0.smithi007.stdout:0/991: dwrite d6/d75/dc5/f28 [0,4194304] 0 2024-09-21T04:35:45.294 INFO:tasks.workunit.client.0.smithi007.stdout:0/992: dread d6/d75/dc5/d47/df0/f11d [0,4194304] 0 2024-09-21T04:35:45.297 INFO:tasks.workunit.client.0.smithi007.stdout:0/993: dread d6/d75/ff9 [0,4194304] 0 2024-09-21T04:35:45.299 INFO:tasks.workunit.client.0.smithi007.stdout:0/994: readlink d6/d75/dc5/d18/d39/l95 0 2024-09-21T04:35:45.300 INFO:tasks.workunit.client.0.smithi007.stdout:0/995: mknod d6/d75/dc5/d18/c162 0 2024-09-21T04:35:45.301 INFO:tasks.workunit.client.0.smithi007.stdout:0/996: readlink d6/d75/dc5/d52/d6d/l85 0 2024-09-21T04:35:45.303 INFO:tasks.workunit.client.0.smithi007.stdout:0/997: truncate d6/d75/dc9/ff6 125973 0 2024-09-21T04:35:45.305 INFO:tasks.workunit.client.0.smithi007.stdout:0/998: creat d6/d75/dc5/d47/d8a/de9/d101/d109/f163 x:0 0 0 2024-09-21T04:35:45.306 INFO:tasks.workunit.client.0.smithi007.stdout:0/999: write d6/d75/dc5/f141 [252319,107290] 0 2024-09-21T04:35:45.324 INFO:tasks.workunit.client.0.smithi007.stdout:2/877: read d0/d11/d31/d7b/de8/d81/da0/f25 [1566984,13396] 0 2024-09-21T04:35:45.326 INFO:tasks.workunit.client.0.smithi007.stdout:2/878: chown d0/d11/d31/d7b/de8/d81/da0/f6 13 1 2024-09-21T04:35:45.327 INFO:tasks.workunit.client.0.smithi007.stdout:8/964: dread d0/d3/d16/d93/d5e/de3/d4d/d5d/fab [0,4194304] 0 2024-09-21T04:35:45.331 INFO:tasks.workunit.client.0.smithi007.stdout:2/879: truncate d0/d11/d31/d7b/de8/d81/da0/f6 2160796 0 2024-09-21T04:35:45.332 INFO:tasks.workunit.client.0.smithi007.stdout:8/965: dread d0/fb8 [0,4194304] 0 2024-09-21T04:35:45.334 INFO:tasks.workunit.client.0.smithi007.stdout:8/966: symlink d0/d3/l11b 0 2024-09-21T04:35:45.336 INFO:tasks.workunit.client.0.smithi007.stdout:8/967: chown d0/d3/d7c/dbf/ced 60102 1 2024-09-21T04:35:45.338 INFO:tasks.workunit.client.0.smithi007.stdout:2/880: write d0/d11/d6e/f8a [902135,50481] 0 2024-09-21T04:35:45.339 INFO:tasks.workunit.client.0.smithi007.stdout:8/968: truncate d0/d3/d16/d93/d40/d92/dcb/f10d 171368 0 2024-09-21T04:35:45.341 INFO:tasks.workunit.client.0.smithi007.stdout:8/969: fsync d0/d3/daa/fd8 0 2024-09-21T04:35:45.341 INFO:tasks.workunit.client.0.smithi007.stdout:8/970: stat d0/d3/d16/d93 0 2024-09-21T04:35:45.342 INFO:tasks.workunit.client.0.smithi007.stdout:8/971: chown d0/d3/d16/d93/d40/d92/cd4 44224653 1 2024-09-21T04:35:45.346 INFO:tasks.workunit.client.0.smithi007.stdout:6/928: sync 2024-09-21T04:35:45.347 INFO:tasks.workunit.client.0.smithi007.stdout:6/929: readlink d3/de/l65 0 2024-09-21T04:35:45.349 INFO:tasks.workunit.client.0.smithi007.stdout:8/972: dwrite d0/d3/daa/fd8 [0,4194304] 0 2024-09-21T04:35:45.351 INFO:tasks.workunit.client.0.smithi007.stdout:6/930: truncate d3/d3b/dd3/d60/d77/f5f 4278035 0 2024-09-21T04:35:45.355 INFO:tasks.workunit.client.0.smithi007.stdout:6/931: creat d3/de/d58/f129 x:0 0 0 2024-09-21T04:35:45.357 INFO:tasks.workunit.client.0.smithi007.stdout:8/973: link d0/d3/d16/d93/d5e/de3/d4d/d5d/dba/fcd d0/d3/d8/f11c 0 2024-09-21T04:35:45.362 INFO:tasks.workunit.client.0.smithi007.stdout:6/932: dwrite d3/f1e [0,4194304] 0 2024-09-21T04:35:45.363 INFO:tasks.workunit.client.0.smithi007.stdout:6/933: dread - d3/d3b/dd3/d60/ff2 zero size 2024-09-21T04:35:45.364 INFO:tasks.workunit.client.0.smithi007.stdout:8/974: truncate d0/d3/d16/d93/d5e/de3/d35/f87 602473 0 2024-09-21T04:35:45.369 INFO:tasks.workunit.client.0.smithi007.stdout:6/934: mknod d3/d3b/dd3/d60/d10c/c12a 0 2024-09-21T04:35:45.371 INFO:tasks.workunit.client.0.smithi007.stdout:8/975: rename d0/d3/d16/d93/d40/d92/dcb/f10d to d0/d3/d16/d93/d5e/de3/d4d/d5d/f11d 0 2024-09-21T04:35:45.373 INFO:tasks.workunit.client.0.smithi007.stdout:8/976: rmdir d0/d3/d16/d93/d5e/de3/d35/df6 39 2024-09-21T04:35:45.376 INFO:tasks.workunit.client.0.smithi007.stdout:8/977: mknod d0/d3/d16/d93/d40/d9e/c11e 0 2024-09-21T04:35:45.382 INFO:tasks.workunit.client.0.smithi007.stdout:8/978: symlink d0/d3/l11f 0 2024-09-21T04:35:45.388 INFO:tasks.workunit.client.0.smithi007.stdout:6/935: dwrite d3/de/d12/d19/f1b [0,4194304] 0 2024-09-21T04:35:45.398 INFO:tasks.workunit.client.0.smithi007.stdout:8/979: link d0/d3/d16/d93/d40/d92/dcb/l112 d0/d3/d16/d93/d5e/de3/d4d/l120 0 2024-09-21T04:35:45.404 INFO:tasks.workunit.client.0.smithi007.stdout:6/936: dread d3/d3b/dd3/d60/dd2/fe1 [0,4194304] 0 2024-09-21T04:35:45.404 INFO:tasks.workunit.client.0.smithi007.stdout:6/937: read - d3/d3e/f11a zero size 2024-09-21T04:35:45.409 INFO:tasks.workunit.client.0.smithi007.stdout:6/938: dread d3/d3b/ff5 [0,4194304] 0 2024-09-21T04:35:45.411 INFO:tasks.workunit.client.0.smithi007.stdout:6/939: creat d3/d94/f12b x:0 0 0 2024-09-21T04:35:45.413 INFO:tasks.workunit.client.0.smithi007.stdout:6/940: chown d3/de/d12/d19/da3/de3/d104/d112 5533576 1 2024-09-21T04:35:45.415 INFO:tasks.workunit.client.0.smithi007.stdout:6/941: mkdir d3/de/d12c 0 2024-09-21T04:35:45.416 INFO:tasks.workunit.client.0.smithi007.stdout:8/980: write d0/d3/d16/d93/d40/d92/dcb/f101 [5142327,76059] 0 2024-09-21T04:35:45.422 INFO:tasks.workunit.client.0.smithi007.stdout:6/942: link d3/d3b/dd3/c47 d3/de/d12/d19/dd8/c12d 0 2024-09-21T04:35:45.430 INFO:tasks.workunit.client.0.smithi007.stdout:8/981: dread d0/d3/d16/d93/d5e/de3/d35/f37 [0,4194304] 0 2024-09-21T04:35:45.434 INFO:tasks.workunit.client.0.smithi007.stdout:8/982: dread d0/d3/d7c/fc1 [0,4194304] 0 2024-09-21T04:35:45.445 INFO:tasks.workunit.client.0.smithi007.stdout:8/983: dread d0/d3/d8/d46/db7/fe2 [0,4194304] 0 2024-09-21T04:35:45.461 INFO:tasks.workunit.client.0.smithi007.stdout:6/943: rename d3/d3e/d99/dbb/df1/cf7 to d3/d3b/dd3/c12e 0 2024-09-21T04:35:45.464 INFO:tasks.workunit.client.0.smithi007.stdout:6/944: rename d3/d3b/c41 to d3/d3b/dd3/d60/d98/c12f 0 2024-09-21T04:35:45.507 INFO:tasks.workunit.client.0.smithi007.stdout:8/984: dwrite d0/d3/d7c/fc6 [0,4194304] 0 2024-09-21T04:35:45.509 INFO:tasks.workunit.client.0.smithi007.stdout:8/985: write d0/fa6 [2253395,12148] 0 2024-09-21T04:35:45.513 INFO:tasks.workunit.client.0.smithi007.stdout:6/945: write d3/de/d12/f1f [851679,107619] 0 2024-09-21T04:35:45.516 INFO:tasks.workunit.client.0.smithi007.stdout:6/946: write d3/d3e/d99/dbb/d95/dc9/d10b/ff8 [990721,118857] 0 2024-09-21T04:35:45.517 INFO:tasks.workunit.client.0.smithi007.stdout:8/986: truncate d0/d3/d16/d93/d5e/de3/d35/df6/f10a 156300 0 2024-09-21T04:35:45.518 INFO:tasks.workunit.client.0.smithi007.stdout:8/987: stat d0/d3/d7c/dbf/fff 0 2024-09-21T04:35:45.521 INFO:tasks.workunit.client.0.smithi007.stdout:6/947: symlink d3/l130 0 2024-09-21T04:35:45.522 INFO:tasks.workunit.client.0.smithi007.stdout:6/948: readlink d3/de/l65 0 2024-09-21T04:35:45.532 INFO:tasks.workunit.client.0.smithi007.stdout:6/949: dwrite d3/de/d12/d19/da3/fe6 [0,4194304] 0 2024-09-21T04:35:45.540 INFO:tasks.workunit.client.0.smithi007.stdout:6/950: creat d3/de/d12/d19/dd8/f131 x:0 0 0 2024-09-21T04:35:45.541 INFO:tasks.workunit.client.0.smithi007.stdout:8/988: dread d0/f4f [4194304,4194304] 0 2024-09-21T04:35:45.546 INFO:tasks.workunit.client.0.smithi007.stdout:6/951: unlink d3/ld 0 2024-09-21T04:35:45.548 INFO:tasks.workunit.client.0.smithi007.stdout:6/952: read - d3/de/d58/f129 zero size 2024-09-21T04:35:45.552 INFO:tasks.workunit.client.0.smithi007.stdout:6/953: mkdir d3/d8e/d127/d128/d132 0 2024-09-21T04:35:45.556 INFO:tasks.workunit.client.0.smithi007.stdout:6/954: unlink d3/de/d12/d19/c37 0 2024-09-21T04:35:45.559 INFO:tasks.workunit.client.0.smithi007.stdout:8/989: truncate d0/d3/fc7 1328831 0 2024-09-21T04:35:45.560 INFO:tasks.workunit.client.0.smithi007.stdout:6/955: creat d3/d8e/dae/f133 x:0 0 0 2024-09-21T04:35:45.561 INFO:tasks.workunit.client.0.smithi007.stdout:6/956: read d3/d94/fc2 [6085288,112243] 0 2024-09-21T04:35:45.562 INFO:tasks.workunit.client.0.smithi007.stdout:8/990: write d0/d3/d16/d93/d5e/fe5 [2151034,47111] 0 2024-09-21T04:35:45.571 INFO:tasks.workunit.client.0.smithi007.stdout:6/957: mkdir d3/de/d12/d19/d134 0 2024-09-21T04:35:45.578 INFO:tasks.workunit.client.0.smithi007.stdout:6/958: creat d3/d3b/dd3/f135 x:0 0 0 2024-09-21T04:35:45.578 INFO:tasks.workunit.client.0.smithi007.stdout:8/991: write d0/d3/d8/f11c [36607,66046] 0 2024-09-21T04:35:45.582 INFO:tasks.workunit.client.0.smithi007.stdout:6/959: symlink d3/d8e/d127/d128/d132/l136 0 2024-09-21T04:35:45.585 INFO:tasks.workunit.client.0.smithi007.stdout:8/992: dwrite d0/d3/d7c/f10f [0,4194304] 0 2024-09-21T04:35:45.589 INFO:tasks.workunit.client.0.smithi007.stdout:8/993: rmdir d0/d3/d82 39 2024-09-21T04:35:45.592 INFO:tasks.workunit.client.0.smithi007.stdout:8/994: mknod d0/d3/d16/d93/d5e/de3/d4d/d5d/c121 0 2024-09-21T04:35:45.597 INFO:tasks.workunit.client.0.smithi007.stdout:6/960: dwrite d3/de/d12/d19/f51 [0,4194304] 0 2024-09-21T04:35:45.598 INFO:tasks.workunit.client.0.smithi007.stdout:6/961: write d3/de/d12/f1f [2196966,111127] 0 2024-09-21T04:35:45.603 INFO:tasks.workunit.client.0.smithi007.stdout:8/995: creat d0/d3/d16/d93/d5e/de3/d4d/df8/f122 x:0 0 0 2024-09-21T04:35:45.605 INFO:tasks.workunit.client.0.smithi007.stdout:6/962: write d3/de/d12/d19/f1b [3194430,96670] 0 2024-09-21T04:35:45.605 INFO:tasks.workunit.client.0.smithi007.stdout:8/996: readlink d0/d3/d16/d93/d5e/de3/d10b/la5 0 2024-09-21T04:35:45.606 INFO:tasks.workunit.client.0.smithi007.stdout:2/881: sync 2024-09-21T04:35:45.609 INFO:tasks.workunit.client.0.smithi007.stdout:6/963: mknod d3/de/d12/d19/da3/c137 0 2024-09-21T04:35:45.612 INFO:tasks.workunit.client.0.smithi007.stdout:2/882: dread d0/d11/d31/dfe/ff4 [0,4194304] 0 2024-09-21T04:35:45.613 INFO:tasks.workunit.client.0.smithi007.stdout:8/997: creat d0/f123 x:0 0 0 2024-09-21T04:35:45.616 INFO:tasks.workunit.client.0.smithi007.stdout:8/998: creat d0/d3/d8/d46/f124 x:0 0 0 2024-09-21T04:35:45.620 INFO:tasks.workunit.client.0.smithi007.stdout:2/883: chown d0/d11/d31/d9f/da9/cae 126474952 1 2024-09-21T04:35:45.621 INFO:tasks.workunit.client.0.smithi007.stdout:8/999: chown d0/d3/d16/l30 2982 1 2024-09-21T04:35:45.626 INFO:tasks.workunit.client.0.smithi007.stdout:6/964: dread d3/f28 [0,4194304] 0 2024-09-21T04:35:45.626 INFO:tasks.workunit.client.0.smithi007.stdout:2/884: unlink d0/c66 0 2024-09-21T04:35:45.627 INFO:tasks.workunit.client.0.smithi007.stdout:6/965: fsync d3/d3e/f40 0 2024-09-21T04:35:45.629 INFO:tasks.workunit.client.0.smithi007.stdout:2/885: read d0/d11/d31/d39/f44 [275194,110154] 0 2024-09-21T04:35:45.642 INFO:tasks.workunit.client.0.smithi007.stdout:6/966: write d3/d3b/dd3/d60/ff2 [501598,93974] 0 2024-09-21T04:35:45.647 INFO:tasks.workunit.client.0.smithi007.stdout:2/886: dwrite d0/d11/d31/d7b/f93 [0,4194304] 0 2024-09-21T04:35:45.649 INFO:tasks.workunit.client.0.smithi007.stdout:6/967: rmdir d3/d3e/d99/dbb/d95 39 2024-09-21T04:35:45.656 INFO:tasks.workunit.client.0.smithi007.stdout:2/887: creat d0/d11/d31/d7b/de8/d81/dfc/f10f x:0 0 0 2024-09-21T04:35:45.657 INFO:tasks.workunit.client.0.smithi007.stdout:6/968: dread - d3/de/d12/d19/da3/de3/f108 zero size 2024-09-21T04:35:45.664 INFO:tasks.workunit.client.0.smithi007.stdout:2/888: unlink d0/d11/d31/d7b/de8/d81/da0/c2a 0 2024-09-21T04:35:45.666 INFO:tasks.workunit.client.0.smithi007.stdout:2/889: rmdir d0/d11/d31/d7b/de8/d81/dfc/d102/d104 39 2024-09-21T04:35:45.668 INFO:tasks.workunit.client.0.smithi007.stdout:2/890: stat d0/d11/d6b/fc1 0 2024-09-21T04:35:45.671 INFO:tasks.workunit.client.0.smithi007.stdout:6/969: dwrite d3/d3b/f5b [4194304,4194304] 0 2024-09-21T04:35:45.675 INFO:tasks.workunit.client.0.smithi007.stdout:6/970: chown d3/de/d58/f129 64 1 2024-09-21T04:35:45.678 INFO:tasks.workunit.client.0.smithi007.stdout:2/891: dwrite d0/d11/d31/d7b/de8/db9/ff1 [0,4194304] 0 2024-09-21T04:35:45.683 INFO:tasks.workunit.client.0.smithi007.stdout:6/971: dread d3/de/d12/d19/f1b [0,4194304] 0 2024-09-21T04:35:45.686 INFO:tasks.workunit.client.0.smithi007.stdout:6/972: write d3/de/d12/d19/dd8/f106 [3259180,72560] 0 2024-09-21T04:35:45.688 INFO:tasks.workunit.client.0.smithi007.stdout:2/892: rename d0/d11/f20 to d0/d11/d31/d7b/de8/db9/d108/dbf/f110 0 2024-09-21T04:35:45.690 INFO:tasks.workunit.client.0.smithi007.stdout:2/893: stat d0/d11/d31/d7b/de8/d81/da0/d57/fea 0 2024-09-21T04:35:45.693 INFO:tasks.workunit.client.0.smithi007.stdout:6/973: rmdir d3/d8e/d127 39 2024-09-21T04:35:45.693 INFO:tasks.workunit.client.0.smithi007.stdout:2/894: rmdir d0/d11/d31/d52 39 2024-09-21T04:35:45.693 INFO:tasks.workunit.client.0.smithi007.stdout:6/974: chown d3/d3e/cb7 20 1 2024-09-21T04:35:45.696 INFO:tasks.workunit.client.0.smithi007.stdout:6/975: rename d3/de/de0 to d3/de/d58/d138 0 2024-09-21T04:35:45.698 INFO:tasks.workunit.client.0.smithi007.stdout:2/895: rmdir d0/d11/d31/d7b/de8/db9/d108/dbf/dce/dfd 39 2024-09-21T04:35:45.699 INFO:tasks.workunit.client.0.smithi007.stdout:6/976: truncate d3/de/fb5 1001995 0 2024-09-21T04:35:45.701 INFO:tasks.workunit.client.0.smithi007.stdout:2/896: creat d0/d11/d31/d52/f111 x:0 0 0 2024-09-21T04:35:45.702 INFO:tasks.workunit.client.0.smithi007.stdout:6/977: mkdir d3/d3e/d139 0 2024-09-21T04:35:45.704 INFO:tasks.workunit.client.0.smithi007.stdout:2/897: fsync d0/d11/d31/d52/f87 0 2024-09-21T04:35:45.710 INFO:tasks.workunit.client.0.smithi007.stdout:6/978: symlink d3/d8e/d127/l13a 0 2024-09-21T04:35:45.711 INFO:tasks.workunit.client.0.smithi007.stdout:6/979: write d3/d8e/dae/fbc [382888,45957] 0 2024-09-21T04:35:45.714 INFO:tasks.workunit.client.0.smithi007.stdout:6/980: fdatasync d3/d3b/dd3/d60/d77/dcb/d88/fde 0 2024-09-21T04:35:45.716 INFO:tasks.workunit.client.0.smithi007.stdout:6/981: fdatasync d3/d3b/dd3/d60/d77/dcb/fa6 0 2024-09-21T04:35:45.716 INFO:tasks.workunit.client.0.smithi007.stdout:2/898: dwrite d0/d11/d31/d46/f47 [0,4194304] 0 2024-09-21T04:35:45.722 INFO:tasks.workunit.client.0.smithi007.stdout:2/899: dread d0/d11/d31/d46/d6c/db7/fc6 [0,4194304] 0 2024-09-21T04:35:45.725 INFO:tasks.workunit.client.0.smithi007.stdout:6/982: mkdir d3/de/d12/d13b 0 2024-09-21T04:35:45.734 INFO:tasks.workunit.client.0.smithi007.stdout:6/983: truncate d3/de/d12/d19/f1b 1075411 0 2024-09-21T04:35:45.740 INFO:tasks.workunit.client.0.smithi007.stdout:6/984: dwrite d3/d3e/d99/dbb/f9a [0,4194304] 0 2024-09-21T04:35:45.745 INFO:tasks.workunit.client.0.smithi007.stdout:6/985: symlink d3/d3b/dd3/d60/d77/dcb/d88/l13c 0 2024-09-21T04:35:45.756 INFO:tasks.workunit.client.0.smithi007.stdout:6/986: dwrite d3/d3b/dd3/d60/dd7/ff6 [0,4194304] 0 2024-09-21T04:35:45.762 INFO:tasks.workunit.client.0.smithi007.stdout:6/987: rename d3/d3e/d99/dbb/f9a to d3/d8e/f13d 0 2024-09-21T04:35:45.762 INFO:tasks.workunit.client.0.smithi007.stdout:6/988: fdatasync d3/f11b 0 2024-09-21T04:35:45.778 INFO:tasks.workunit.client.0.smithi007.stdout:6/989: dread d3/d3e/d99/fa5 [0,4194304] 0 2024-09-21T04:35:45.782 INFO:tasks.workunit.client.0.smithi007.stdout:6/990: write d3/d3b/dd3/d60/d77/dcb/d78/f96 [316139,33860] 0 2024-09-21T04:35:45.784 INFO:tasks.workunit.client.0.smithi007.stdout:6/991: chown d3/d3b/dd3/d60/dd7/ff6 59 1 2024-09-21T04:35:45.789 INFO:tasks.workunit.client.0.smithi007.stdout:6/992: rename d3/d3e/d99/dbb/d95/f11d to d3/de/d12c/f13e 0 2024-09-21T04:35:45.799 INFO:tasks.workunit.client.0.smithi007.stdout:6/993: dread d3/de/d12/f21 [0,4194304] 0 2024-09-21T04:35:45.800 INFO:tasks.workunit.client.0.smithi007.stdout:6/994: readlink d3/d3e/ldd 0 2024-09-21T04:35:45.803 INFO:tasks.workunit.client.0.smithi007.stdout:6/995: creat d3/d8e/f13f x:0 0 0 2024-09-21T04:35:45.804 INFO:tasks.workunit.client.0.smithi007.stdout:6/996: write d3/d3b/dd3/d60/d77/dcb/d88/fde [2239401,110051] 0 2024-09-21T04:35:45.829 INFO:tasks.workunit.client.0.smithi007.stdout:6/997: creat d3/f140 x:0 0 0 2024-09-21T04:35:45.837 INFO:tasks.workunit.client.0.smithi007.stdout:6/998: link d3/de/d12/d19/f36 d3/de/f141 0 2024-09-21T04:35:45.841 INFO:tasks.workunit.client.0.smithi007.stdout:6/999: dread d3/de/fed [0,4194304] 0 2024-09-21T04:35:46.146 INFO:tasks.workunit.client.0.smithi007.stdout:2/900: sync 2024-09-21T04:35:46.148 INFO:tasks.workunit.client.0.smithi007.stdout:2/901: rmdir d0/d11/d6b/d9d 39 2024-09-21T04:35:46.160 INFO:tasks.workunit.client.0.smithi007.stdout:2/902: dread d0/f1c [0,4194304] 0 2024-09-21T04:35:46.166 INFO:tasks.workunit.client.0.smithi007.stdout:2/903: link d0/d11/d31/d7b/de8/db9/ddd/fe0 d0/d11/d31/d7b/de8/db9/d108/dbf/dce/dfd/f112 0 2024-09-21T04:35:46.167 INFO:tasks.workunit.client.0.smithi007.stdout:2/904: write d0/d11/d31/d46/d6c/fb4 [1169907,35189] 0 2024-09-21T04:35:46.169 INFO:tasks.workunit.client.0.smithi007.stdout:2/905: mknod d0/d11/d31/d39/c113 0 2024-09-21T04:35:46.174 INFO:tasks.workunit.client.0.smithi007.stdout:2/906: link d0/d11/f4b d0/d11/d31/da4/f114 0 2024-09-21T04:35:46.178 INFO:tasks.workunit.client.0.smithi007.stdout:2/907: dread d0/d11/d31/d7b/de8/d81/da0/f88 [0,4194304] 0 2024-09-21T04:35:46.189 INFO:tasks.workunit.client.0.smithi007.stdout:2/908: dread d0/d11/d31/d7b/de8/d81/da0/fc [0,4194304] 0 2024-09-21T04:35:46.191 INFO:tasks.workunit.client.0.smithi007.stdout:2/909: unlink d0/d11/d6b/l96 0 2024-09-21T04:35:46.193 INFO:tasks.workunit.client.0.smithi007.stdout:2/910: write d0/d11/d31/d46/d77/f10e [550057,113352] 0 2024-09-21T04:35:46.194 INFO:tasks.workunit.client.0.smithi007.stdout:2/911: rmdir d0/d11/d31/d9f 39 2024-09-21T04:35:46.196 INFO:tasks.workunit.client.0.smithi007.stdout:2/912: stat d0/d11/d31/d7b/de8/d81/dfc/d102/d104 0 2024-09-21T04:35:46.197 INFO:tasks.workunit.client.0.smithi007.stdout:2/913: read d0/d11/d31/d46/f6a [3389529,98128] 0 2024-09-21T04:35:46.198 INFO:tasks.workunit.client.0.smithi007.stdout:2/914: readlink d0/d32/l4c 0 2024-09-21T04:35:46.199 INFO:tasks.workunit.client.0.smithi007.stdout:2/915: write d0/d11/d31/d7b/da2/da5/fe2 [864402,98420] 0 2024-09-21T04:35:46.200 INFO:tasks.workunit.client.0.smithi007.stdout:2/916: dread - d0/d11/d31/d46/fd6 zero size 2024-09-21T04:35:46.202 INFO:tasks.workunit.client.0.smithi007.stdout:2/917: readlink d0/d11/d31/d46/d77/ld8 0 2024-09-21T04:35:46.207 INFO:tasks.workunit.client.0.smithi007.stdout:2/918: write d0/d11/d31/d7b/de8/d81/da0/fc [3709401,68166] 0 2024-09-21T04:35:46.215 INFO:tasks.workunit.client.0.smithi007.stdout:2/919: dwrite d0/d11/d31/d46/d6c/f78 [0,4194304] 0 2024-09-21T04:35:46.224 INFO:tasks.workunit.client.0.smithi007.stdout:2/920: write d0/d32/f34 [1226387,39419] 0 2024-09-21T04:35:46.228 INFO:tasks.workunit.client.0.smithi007.stdout:2/921: dread d0/d11/d31/d39/f44 [0,4194304] 0 2024-09-21T04:35:46.230 INFO:tasks.workunit.client.0.smithi007.stdout:2/922: mknod d0/d11/d31/d46/d6c/c115 0 2024-09-21T04:35:46.232 INFO:tasks.workunit.client.0.smithi007.stdout:2/923: write d0/d11/d6b/d9d/fc5 [95550,38771] 0 2024-09-21T04:35:46.233 INFO:tasks.workunit.client.0.smithi007.stdout:2/924: chown d0/d11/d31/c45 317204811 1 2024-09-21T04:35:46.233 INFO:tasks.workunit.client.0.smithi007.stdout:2/925: stat d0/d11/db8 0 2024-09-21T04:35:46.236 INFO:tasks.workunit.client.0.smithi007.stdout:2/926: unlink d0/c9b 0 2024-09-21T04:35:46.237 INFO:tasks.workunit.client.0.smithi007.stdout:2/927: fdatasync d0/d11/d31/d52/f111 0 2024-09-21T04:35:46.247 INFO:tasks.workunit.client.0.smithi007.stdout:2/928: dwrite d0/fa8 [0,4194304] 0 2024-09-21T04:35:46.248 INFO:tasks.workunit.client.0.smithi007.stdout:2/929: chown d0/d11/d31/d52/f87 27416 1 2024-09-21T04:35:46.258 INFO:tasks.workunit.client.0.smithi007.stdout:2/930: dwrite d0/d11/d31/d7b/de8/d81/da0/f68 [4194304,4194304] 0 2024-09-21T04:35:46.260 INFO:tasks.workunit.client.0.smithi007.stdout:2/931: write d0/d11/d6e/fa1 [1269603,126763] 0 2024-09-21T04:35:46.264 INFO:tasks.workunit.client.0.smithi007.stdout:2/932: dread d0/d11/d31/d39/f44 [0,4194304] 0 2024-09-21T04:35:46.267 INFO:tasks.workunit.client.0.smithi007.stdout:2/933: creat d0/d11/d31/d7b/da2/da5/f116 x:0 0 0 2024-09-21T04:35:46.269 INFO:tasks.workunit.client.0.smithi007.stdout:2/934: creat d0/d11/d31/dfe/dbd/f117 x:0 0 0 2024-09-21T04:35:46.271 INFO:tasks.workunit.client.0.smithi007.stdout:2/935: fdatasync d0/d11/d31/d7b/da2/da5/fc9 0 2024-09-21T04:35:46.273 INFO:tasks.workunit.client.0.smithi007.stdout:2/936: write d0/d11/d31/d7b/de8/d81/ff0 [5062077,95826] 0 2024-09-21T04:35:46.280 INFO:tasks.workunit.client.0.smithi007.stdout:2/937: dwrite d0/d11/fec [0,4194304] 0 2024-09-21T04:35:46.281 INFO:tasks.workunit.client.0.smithi007.stdout:2/938: fdatasync d0/d11/d6e/f101 0 2024-09-21T04:35:46.290 INFO:tasks.workunit.client.0.smithi007.stdout:2/939: dwrite d0/d11/d31/d52/fcf [0,4194304] 0 2024-09-21T04:35:46.292 INFO:tasks.workunit.client.0.smithi007.stdout:2/940: chown d0/d11/d31/d39/f84 237 1 2024-09-21T04:35:46.298 INFO:tasks.workunit.client.0.smithi007.stdout:2/941: dwrite d0/d11/f28 [0,4194304] 0 2024-09-21T04:35:46.300 INFO:tasks.workunit.client.0.smithi007.stdout:2/942: truncate d0/d11/d31/d52/f111 708053 0 2024-09-21T04:35:46.311 INFO:tasks.workunit.client.0.smithi007.stdout:2/943: truncate d0/d11/d6e/fc7 1713947 0 2024-09-21T04:35:46.322 INFO:tasks.workunit.client.0.smithi007.stdout:2/944: write d0/d11/d31/d52/f87 [737070,57985] 0 2024-09-21T04:35:46.323 INFO:tasks.workunit.client.0.smithi007.stdout:2/945: chown d0/d11 106280043 1 2024-09-21T04:35:46.323 INFO:tasks.workunit.client.0.smithi007.stdout:2/946: chown d0/d11/d31/d46/d77 104365 1 2024-09-21T04:35:46.333 INFO:tasks.workunit.client.0.smithi007.stdout:2/947: write d0/d11/d31/d7b/de8/db9/fcd [1761718,87899] 0 2024-09-21T04:35:46.337 INFO:tasks.workunit.client.0.smithi007.stdout:2/948: creat d0/d11/d31/d7b/de8/f118 x:0 0 0 2024-09-21T04:35:46.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:46 smithi053 ceph-mon[35554]: pgmap v44: 65 pgs: 65 active+clean; 2.4 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 59 MiB/s rd, 126 MiB/s wr, 384 op/s 2024-09-21T04:35:46.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:46 smithi007 ceph-mon[27954]: pgmap v44: 65 pgs: 65 active+clean; 2.4 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 59 MiB/s rd, 126 MiB/s wr, 384 op/s 2024-09-21T04:35:46.848 INFO:tasks.workunit.client.0.smithi007.stdout:2/949: sync 2024-09-21T04:35:46.852 INFO:tasks.workunit.client.0.smithi007.stdout:2/950: dread d0/d11/d31/d39/f3a [4194304,4194304] 0 2024-09-21T04:35:46.854 INFO:tasks.workunit.client.0.smithi007.stdout:2/951: symlink d0/d11/d6e/deb/l119 0 2024-09-21T04:35:46.856 INFO:tasks.workunit.client.0.smithi007.stdout:2/952: mkdir d0/d11/d6e/d73/d11a 0 2024-09-21T04:35:46.857 INFO:tasks.workunit.client.0.smithi007.stdout:2/953: fdatasync d0/d11/d31/d7b/de8/d81/da0/fc 0 2024-09-21T04:35:46.858 INFO:tasks.workunit.client.0.smithi007.stdout:2/954: write d0/d11/d6e/f10c [226008,73627] 0 2024-09-21T04:35:46.860 INFO:tasks.workunit.client.0.smithi007.stdout:2/955: read d0/d11/d31/d46/d77/fe5 [3200857,25257] 0 2024-09-21T04:35:46.863 INFO:tasks.workunit.client.0.smithi007.stdout:2/956: truncate d0/d11/d31/d39/faf 153562 0 2024-09-21T04:35:46.875 INFO:tasks.workunit.client.0.smithi007.stdout:2/957: write d0/d11/d31/d7b/de8/db9/d108/dbf/dce/dfd/f112 [3543985,58567] 0 2024-09-21T04:35:46.880 INFO:tasks.workunit.client.0.smithi007.stdout:2/958: creat d0/d11/d31/f11b x:0 0 0 2024-09-21T04:35:46.882 INFO:tasks.workunit.client.0.smithi007.stdout:2/959: rmdir d0/d11/d31/d7b/de8/d81/dfc/d102 39 2024-09-21T04:35:46.883 INFO:tasks.workunit.client.0.smithi007.stdout:2/960: fdatasync d0/d11/d31/d7b/de8/db9/ddd/fe0 0 2024-09-21T04:35:46.885 INFO:tasks.workunit.client.0.smithi007.stdout:2/961: mkdir d0/d11/d31/d46/d11c 0 2024-09-21T04:35:46.895 INFO:tasks.workunit.client.0.smithi007.stdout:2/962: rename d0/d11/d31/d46 to d0/d11/d31/d7b/da2/d11d 0 2024-09-21T04:35:46.899 INFO:tasks.workunit.client.0.smithi007.stdout:2/963: dread d0/d11/d31/d39/f3a [0,4194304] 0 2024-09-21T04:35:46.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/964: dread d0/f8 [4194304,4194304] 0 2024-09-21T04:35:46.913 INFO:tasks.workunit.client.0.smithi007.stdout:2/965: creat d0/d11/f11e x:0 0 0 2024-09-21T04:35:46.915 INFO:tasks.workunit.client.0.smithi007.stdout:2/966: unlink d0/l2f 0 2024-09-21T04:35:46.917 INFO:tasks.workunit.client.0.smithi007.stdout:2/967: creat d0/d11/d31/d7b/da2/d11d/f11f x:0 0 0 2024-09-21T04:35:46.918 INFO:tasks.workunit.client.0.smithi007.stdout:2/968: symlink d0/d11/d31/d7b/da2/l120 0 2024-09-21T04:35:46.920 INFO:tasks.workunit.client.0.smithi007.stdout:2/969: truncate d0/f1c 5052799 0 2024-09-21T04:35:46.924 INFO:tasks.workunit.client.0.smithi007.stdout:2/970: rmdir d0/d11/db8 0 2024-09-21T04:35:46.925 INFO:tasks.workunit.client.0.smithi007.stdout:2/971: fsync d0/d11/d31/d7b/de8/d81/da0/f6 0 2024-09-21T04:35:46.926 INFO:tasks.workunit.client.0.smithi007.stdout:2/972: read - d0/d11/d6e/d73/fe1 zero size 2024-09-21T04:35:46.928 INFO:tasks.workunit.client.0.smithi007.stdout:2/973: chown d0/d11/d31/d7b/da2/d11d/d48/f62 62 1 2024-09-21T04:35:46.930 INFO:tasks.workunit.client.0.smithi007.stdout:2/974: creat d0/d11/d31/d7b/de8/f121 x:0 0 0 2024-09-21T04:35:46.933 INFO:tasks.workunit.client.0.smithi007.stdout:2/975: creat d0/d11/d31/d7b/de8/db9/d108/dbf/dce/dfd/f122 x:0 0 0 2024-09-21T04:35:46.945 INFO:tasks.workunit.client.0.smithi007.stdout:2/976: dread d0/d11/d6b/fc1 [0,4194304] 0 2024-09-21T04:35:46.946 INFO:tasks.workunit.client.0.smithi007.stdout:2/977: read - d0/d11/d31/d7b/da2/d11d/f11f zero size 2024-09-21T04:35:46.953 INFO:tasks.workunit.client.0.smithi007.stdout:2/978: dwrite d0/d11/d31/d7b/de8/db9/d108/dbf/dce/dfd/f112 [4194304,4194304] 0 2024-09-21T04:35:46.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/979: fdatasync d0/d11/d31/d7b/da2/d11d/d77/f10e 0 2024-09-21T04:35:46.964 INFO:tasks.workunit.client.0.smithi007.stdout:2/980: symlink d0/d11/d31/d7b/de8/db9/d108/dbf/dce/l123 0 2024-09-21T04:35:46.967 INFO:tasks.workunit.client.0.smithi007.stdout:2/981: dread d0/d11/d6b/d9d/fc5 [0,4194304] 0 2024-09-21T04:35:46.970 INFO:tasks.workunit.client.0.smithi007.stdout:2/982: creat d0/d11/f124 x:0 0 0 2024-09-21T04:35:46.972 INFO:tasks.workunit.client.0.smithi007.stdout:2/983: chown d0/d11/f95 36 1 2024-09-21T04:35:46.973 INFO:tasks.workunit.client.0.smithi007.stdout:2/984: mknod d0/d11/d31/dfe/d10b/c125 0 2024-09-21T04:35:46.980 INFO:tasks.workunit.client.0.smithi007.stdout:2/985: dwrite d0/d11/d31/dfe/ff4 [0,4194304] 0 2024-09-21T04:35:46.986 INFO:tasks.workunit.client.0.smithi007.stdout:2/986: creat d0/d11/d31/d7b/de8/d81/dfc/f126 x:0 0 0 2024-09-21T04:35:46.988 INFO:tasks.workunit.client.0.smithi007.stdout:2/987: creat d0/d11/d31/dfe/dbd/f127 x:0 0 0 2024-09-21T04:35:46.995 INFO:tasks.workunit.client.0.smithi007.stdout:2/988: dwrite d0/d11/d6e/f8a [0,4194304] 0 2024-09-21T04:35:46.996 INFO:tasks.workunit.client.0.smithi007.stdout:2/989: chown d0/d11/d31/d7b/de8/db9/ddd 190 1 2024-09-21T04:35:46.999 INFO:tasks.workunit.client.0.smithi007.stdout:2/990: truncate d0/d11/d31/d7b/f89 3771846 0 2024-09-21T04:35:47.000 INFO:tasks.workunit.client.0.smithi007.stdout:2/991: fdatasync d0/d11/d31/d7b/da2/d11d/d6c/f78 0 2024-09-21T04:35:47.005 INFO:tasks.workunit.client.0.smithi007.stdout:2/992: creat d0/d11/d31/d7b/de8/db9/ddd/f128 x:0 0 0 2024-09-21T04:35:47.006 INFO:tasks.workunit.client.0.smithi007.stdout:2/993: chown d0/d11/d31/d7b/de8/db9/d108/dbf/dce/dfd/f106 2785 1 2024-09-21T04:35:47.011 INFO:tasks.workunit.client.0.smithi007.stdout:2/994: fdatasync d0/d11/d31/d39/faf 0 2024-09-21T04:35:47.013 INFO:tasks.workunit.client.0.smithi007.stdout:2/995: symlink d0/d11/d31/d39/l129 0 2024-09-21T04:35:47.016 INFO:tasks.workunit.client.0.smithi007.stdout:2/996: write d0/d11/d31/d7b/da2/d11d/f6a [2596108,125605] 0 2024-09-21T04:35:47.017 INFO:tasks.workunit.client.0.smithi007.stdout:2/997: chown d0/d11/d31/d7b/de8/db9/ddd/f103 2695 1 2024-09-21T04:35:47.019 INFO:tasks.workunit.client.0.smithi007.stdout:2/998: mkdir d0/d11/d31/da4/d12a 0 2024-09-21T04:35:47.021 INFO:tasks.workunit.client.0.smithi007.stdout:2/999: mknod d0/d11/d6e/c12b 0 2024-09-21T04:35:47.026 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm -rf -- ./tmp.FBBCV7FZnT 2024-09-21T04:35:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:48 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:48 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:48.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:48 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:35:48.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:48 smithi007 ceph-mon[27954]: pgmap v45: 65 pgs: 65 active+clean; 2.4 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 38 MiB/s rd, 84 MiB/s wr, 251 op/s 2024-09-21T04:35:48.614 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:48 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:48.615 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:48 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:48.615 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:48 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:35:48.615 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:48 smithi053 ceph-mon[35554]: pgmap v45: 65 pgs: 65 active+clean; 2.4 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 38 MiB/s rd, 84 MiB/s wr, 251 op/s 2024-09-21T04:35:50.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:50 smithi053 ceph-mon[35554]: pgmap v46: 65 pgs: 65 active+clean; 2.2 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 44 MiB/s rd, 100 MiB/s wr, 326 op/s 2024-09-21T04:35:50.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:50 smithi007 ceph-mon[27954]: pgmap v46: 65 pgs: 65 active+clean; 2.2 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 44 MiB/s rd, 100 MiB/s wr, 326 op/s 2024-09-21T04:35:52.163 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:51 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:52.163 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:51 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:52.163 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:51 smithi007 ceph-mon[27954]: pgmap v47: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 77 MiB/s wr, 253 op/s 2024-09-21T04:35:52.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:51 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:52.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:51 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:52.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:51 smithi053 ceph-mon[35554]: pgmap v47: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 77 MiB/s wr, 253 op/s 2024-09-21T04:35:54.329 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:54 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:54.329 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:54 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:54.329 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:54 smithi007 ceph-mon[27954]: pgmap v48: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 23 MiB/s rd, 49 MiB/s wr, 192 op/s 2024-09-21T04:35:54.330 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:54 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:54.330 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:54 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:54.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:54 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:54.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:54 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:54.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:54 smithi053 ceph-mon[35554]: pgmap v48: 65 pgs: 65 active+clean; 2.1 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 23 MiB/s rd, 49 MiB/s wr, 192 op/s 2024-09-21T04:35:54.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:54 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:54.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:54 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:55.375 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:55 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:55.376 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:55 smithi007 ceph-mon[27954]: Standby manager daemon smithi007.huohfl restarted 2024-09-21T04:35:55.376 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:55 smithi007 ceph-mon[27954]: Standby manager daemon smithi007.huohfl started 2024-09-21T04:35:55.376 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:55 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.huohfl/crt"}]: dispatch 2024-09-21T04:35:55.376 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:55 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:35:55.376 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:55 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.huohfl/key"}]: dispatch 2024-09-21T04:35:55.376 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:55 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:35:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:55 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:55 smithi053 ceph-mon[35554]: Standby manager daemon smithi007.huohfl restarted 2024-09-21T04:35:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:55 smithi053 ceph-mon[35554]: Standby manager daemon smithi007.huohfl started 2024-09-21T04:35:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:55 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.huohfl/crt"}]: dispatch 2024-09-21T04:35:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:55 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:35:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:55 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.huohfl/key"}]: dispatch 2024-09-21T04:35:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:55 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:35:56.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:56 smithi053 ceph-mon[35554]: mgrmap e26: smithi053.lnadao(active, since 91s), standbys: smithi007.huohfl 2024-09-21T04:35:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:56 smithi053 ceph-mon[35554]: pgmap v49: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 23 MiB/s rd, 49 MiB/s wr, 255 op/s 2024-09-21T04:35:56.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:56 smithi007 ceph-mon[27954]: mgrmap e26: smithi053.lnadao(active, since 91s), standbys: smithi007.huohfl 2024-09-21T04:35:56.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:56 smithi007 ceph-mon[27954]: pgmap v49: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 23 MiB/s rd, 49 MiB/s wr, 255 op/s 2024-09-21T04:35:57.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:57 smithi053 ceph-mon[35554]: mgrmap e27: smithi053.lnadao(active, since 92s), standbys: smithi007.huohfl 2024-09-21T04:35:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:57 smithi007 ceph-mon[27954]: mgrmap e27: smithi053.lnadao(active, since 92s), standbys: smithi007.huohfl 2024-09-21T04:35:58.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: pgmap v50: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 8.5 MiB/s rd, 22 MiB/s wr, 152 op/s 2024-09-21T04:35:58.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:58.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:58.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr fail", "who": "smithi053.lnadao"}]: dispatch 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr fail", "who": "smithi053.lnadao"}]: dispatch 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: Activating manager daemon smithi007.huohfl 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd='[{"prefix": "mgr fail", "who": "smithi053.lnadao"}]': finished 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: osdmap e46: 6 total, 6 up, 6 in 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: mgrmap e28: smithi007.huohfl(active, starting, since 0.0102876s) 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"}]: dispatch 2024-09-21T04:35:58.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi007.huohfl", "id": "smithi007.huohfl"}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:35:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:58 smithi053 ceph-mon[35554]: Manager daemon smithi007.huohfl is now available 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: pgmap v50: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 8.5 MiB/s rd, 22 MiB/s wr, 152 op/s 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 172.21.15.53:0/2277712985' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr fail", "who": "smithi053.lnadao"}]: dispatch 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "mgr fail", "who": "smithi053.lnadao"}]: dispatch 2024-09-21T04:35:58.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: Activating manager daemon smithi007.huohfl 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.24429 ' entity='mgr.smithi053.lnadao' cmd='[{"prefix": "mgr fail", "who": "smithi053.lnadao"}]': finished 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: osdmap e46: 6 total, 6 up, 6 in 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: mgrmap e28: smithi007.huohfl(active, starting, since 0.0102876s) 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata", "id": "smithi053"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi007.huohfl", "id": "smithi007.huohfl"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-21T04:35:58.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-21T04:35:58.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-21T04:35:58.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-21T04:35:58.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-21T04:35:58.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-21T04:35:58.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-21T04:35:58.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-21T04:35:58.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:58 smithi007 ceph-mon[27954]: Manager daemon smithi007.huohfl is now available 2024-09-21T04:35:59.318 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/trash_purge_schedule"}]: dispatch 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/trash_purge_schedule"}]: dispatch 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: mgrmap e29: smithi007.huohfl(active, since 1.01415s) 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:35:59.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:35:59 smithi007 ceph-mon[27954]: pgmap v3: 65 pgs: 65 active+clean; 1.2 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-09-21T04:35:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:35:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:35:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:35:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:35:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/mirror_snapshot_schedule"}]: dispatch 2024-09-21T04:35:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/trash_purge_schedule"}]: dispatch 2024-09-21T04:35:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.huohfl/trash_purge_schedule"}]: dispatch 2024-09-21T04:35:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: mgrmap e29: smithi007.huohfl(active, since 1.01415s) 2024-09-21T04:35:59.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:35:59.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:35:59 smithi053 ceph-mon[35554]: pgmap v3: 65 pgs: 65 active+clean; 1.2 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-09-21T04:36:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:00 smithi053 ceph-mon[35554]: pgmap v4: 65 pgs: 65 active+clean; 1.2 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-09-21T04:36:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:00 smithi007 ceph-mon[27954]: pgmap v4: 65 pgs: 65 active+clean; 1.2 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-09-21T04:36:01.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:01 smithi053 ceph-mon[35554]: mgrmap e30: smithi007.huohfl(active, since 3s) 2024-09-21T04:36:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:01 smithi053 ceph-mon[35554]: [21/Sep/2024:04:36:00] ENGINE Bus STARTING 2024-09-21T04:36:01.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:01 smithi007 ceph-mon[27954]: mgrmap e30: smithi007.huohfl(active, since 3s) 2024-09-21T04:36:01.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:01 smithi007 ceph-mon[27954]: [21/Sep/2024:04:36:00] ENGINE Bus STARTING 2024-09-21T04:36:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:02 smithi007 ceph-mon[27954]: [21/Sep/2024:04:36:01] ENGINE Serving on https://172.21.15.7:7150 2024-09-21T04:36:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:02 smithi007 ceph-mon[27954]: [21/Sep/2024:04:36:01] ENGINE Error in HTTPServer.serve 2024-09-21T04:36:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Traceback (most recent call last): 2024-09-21T04:36:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:36:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:36:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._run(expiration_interval) 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: s = self.context.wrap_socket( 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: return self.sslsocket_class._create( 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self.do_handshake() 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: self._sslobj.do_handshake() 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:36:02.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:02 smithi007 ceph-mon[27954]: [21/Sep/2024:04:36:01] ENGINE Serving on http://172.21.15.7:8765 2024-09-21T04:36:02.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:02 smithi007 ceph-mon[27954]: [21/Sep/2024:04:36:01] ENGINE Bus STARTED 2024-09-21T04:36:02.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:02 smithi007 ceph-mon[27954]: pgmap v5: 65 pgs: 65 active+clean; 1.2 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-09-21T04:36:02.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:02 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:02.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:02 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:03.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:02 smithi053 ceph-mon[35554]: [21/Sep/2024:04:36:01] ENGINE Serving on https://172.21.15.7:7150 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:02 smithi053 ceph-mon[35554]: [21/Sep/2024:04:36:01] ENGINE Error in HTTPServer.serve 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Traceback (most recent call last): 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._connections.run(self.expiration_interval) 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._run(expiration_interval) 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: s = self.context.wrap_socket( 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T04:36:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: return self.sslsocket_class._create( 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self.do_handshake() 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: self._sslobj.do_handshake() 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:02 smithi053 ceph-mon[35554]: [21/Sep/2024:04:36:01] ENGINE Serving on http://172.21.15.7:8765 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:02 smithi053 ceph-mon[35554]: [21/Sep/2024:04:36:01] ENGINE Bus STARTED 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:02 smithi053 ceph-mon[35554]: pgmap v5: 65 pgs: 65 active+clean; 1.2 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:02 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:03.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:02 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:04.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:04 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:04 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:04 smithi053 ceph-mon[35554]: pgmap v6: 65 pgs: 65 active+clean; 1.1 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 748 KiB/s rd, 844 KiB/s wr, 48 op/s 2024-09-21T04:36:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:04 smithi053 ceph-mon[35554]: Standby manager daemon smithi053.lnadao started 2024-09-21T04:36:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:04 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/582304463' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/crt"}]: dispatch 2024-09-21T04:36:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:04 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/582304463' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:36:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:04 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/582304463' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/key"}]: dispatch 2024-09-21T04:36:04.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:04 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/582304463' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:36:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:04 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:04 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:04 smithi007 ceph-mon[27954]: pgmap v6: 65 pgs: 65 active+clean; 1.1 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 748 KiB/s rd, 844 KiB/s wr, 48 op/s 2024-09-21T04:36:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:04 smithi007 ceph-mon[27954]: Standby manager daemon smithi053.lnadao started 2024-09-21T04:36:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:04 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/582304463' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/crt"}]: dispatch 2024-09-21T04:36:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:04 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/582304463' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:36:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:04 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/582304463' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/key"}]: dispatch 2024-09-21T04:36:04.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:04 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/582304463' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:36:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:05 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi053.lnadao", "id": "smithi053.lnadao"}]: dispatch 2024-09-21T04:36:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:05 smithi053 ceph-mon[35554]: mgrmap e31: smithi007.huohfl(active, since 6s), standbys: smithi053.lnadao 2024-09-21T04:36:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:05 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:05 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:05 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr metadata", "who": "smithi053.lnadao", "id": "smithi053.lnadao"}]: dispatch 2024-09-21T04:36:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:05 smithi007 ceph-mon[27954]: mgrmap e31: smithi007.huohfl(active, since 6s), standbys: smithi053.lnadao 2024-09-21T04:36:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:05 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:05 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:06.239 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T04:36:06.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T04:36:06.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:06 smithi007 ceph-mon[27954]: pgmap v7: 65 pgs: 65 active+clean; 751 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-09-21T04:36:06.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:06 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:06.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:06 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:07.029 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:06 smithi053 ceph-mon[35554]: pgmap v7: 65 pgs: 65 active+clean; 751 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-09-21T04:36:07.029 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:06 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:07.029 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:06 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:09.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:08 smithi053 ceph-mon[35554]: pgmap v8: 65 pgs: 65 active+clean; 751 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 894 KiB/s rd, 986 KiB/s wr, 76 op/s 2024-09-21T04:36:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:08 smithi007 ceph-mon[27954]: pgmap v8: 65 pgs: 65 active+clean; 751 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 894 KiB/s rd, 986 KiB/s wr, 76 op/s 2024-09-21T04:36:11.093 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:10 smithi007 ceph-mon[27954]: pgmap v9: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-21T04:36:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:10 smithi053 ceph-mon[35554]: pgmap v9: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-21T04:36:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.869+0000 7f312f886640 1 -- 172.21.15.7:0/4240060928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 msgr2=0x7f3128074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.869+0000 7f312f886640 1 --2- 172.21.15.7:0/4240060928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 0x7f3128074620 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3124009880 tx=0x7f312402a530 comp rx=0 tx=0).stop 2024-09-21T04:36:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.869+0000 7f312f886640 1 -- 172.21.15.7:0/4240060928 shutdown_connections 2024-09-21T04:36:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.869+0000 7f312f886640 1 --2- 172.21.15.7:0/4240060928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3128074bf0 0x7f3128075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.869+0000 7f312f886640 1 --2- 172.21.15.7:0/4240060928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 0x7f3128074620 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.870+0000 7f312f886640 1 -- 172.21.15.7:0/4240060928 >> 172.21.15.7:0/4240060928 conn(0x7f31280fd140 msgr2=0x7f31280ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.870+0000 7f312f886640 1 -- 172.21.15.7:0/4240060928 shutdown_connections 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.870+0000 7f312f886640 1 -- 172.21.15.7:0/4240060928 wait complete. 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.870+0000 7f312f886640 1 Processor -- start 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.870+0000 7f312f886640 1 -- start start 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312f886640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3128074bf0 0x7f31281996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312f886640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 0x7f3128199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312f886640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f312810c9b0 con 0x7f31280761c0 2024-09-21T04:36:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312f886640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f312810c830 con 0x7f3128074bf0 2024-09-21T04:36:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312ce21640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 0x7f3128199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312ce21640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 0x7f3128199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50822/0 (socket says 172.21.15.7:50822) 2024-09-21T04:36:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312d622640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3128074bf0 0x7f31281996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312ce21640 1 -- 172.21.15.7:0/2878632870 learned_addr learned my addr 172.21.15.7:0/2878632870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312d622640 1 -- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 msgr2=0x7f3128199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312d622640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 0x7f3128199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.871+0000 7f312d622640 1 -- 172.21.15.7:0/2878632870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312819a290 con 0x7f3128074bf0 2024-09-21T04:36:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.872+0000 7f312d622640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3128074bf0 0x7f31281996d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3124006010 tx=0x7f31240045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.872+0000 7f311e7fc640 1 -- 172.21.15.7:0/2878632870 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f312403f070 con 0x7f3128074bf0 2024-09-21T04:36:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.872+0000 7f311e7fc640 1 -- 172.21.15.7:0/2878632870 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f31240049d0 con 0x7f3128074bf0 2024-09-21T04:36:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.872+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312819ec70 con 0x7f3128074bf0 2024-09-21T04:36:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.872+0000 7f311e7fc640 1 -- 172.21.15.7:0/2878632870 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3124035030 con 0x7f3128074bf0 2024-09-21T04:36:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.873+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312819f1b0 con 0x7f3128074bf0 2024-09-21T04:36:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.874+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3128075050 con 0x7f3128074bf0 2024-09-21T04:36:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.874+0000 7f311e7fc640 1 -- 172.21.15.7:0/2878632870 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7f3124042070 con 0x7f3128074bf0 2024-09-21T04:36:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.875+0000 7f311e7fc640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f311407b050 0x7f311407d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.875+0000 7f311e7fc640 1 -- 172.21.15.7:0/2878632870 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f31240be810 con 0x7f3128074bf0 2024-09-21T04:36:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.875+0000 7f312ce21640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f311407b050 0x7f311407d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.876+0000 7f312ce21640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f311407b050 0x7f311407d510 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f312819ab80 tx=0x7f3118005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:12.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:12.880+0000 7f311e7fc640 1 -- 172.21.15.7:0/2878632870 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f3124089c40 con 0x7f3128074bf0 2024-09-21T04:36:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.169+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f31280626f0 con 0x7f311407b050 2024-09-21T04:36:13.172 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:12 smithi007 ceph-mon[27954]: pgmap v10: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-09-21T04:36:13.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.171+0000 7f311e7fc640 1 -- 172.21.15.7:0/2878632870 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+329 (secure 0 0 0) 0x7f31280626f0 con 0x7f311407b050 2024-09-21T04:36:13.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f311407b050 msgr2=0x7f311407d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:13.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f311407b050 0x7f311407d510 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f312819ab80 tx=0x7f3118005f90 comp rx=0 tx=0).stop 2024-09-21T04:36:13.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3128074bf0 msgr2=0x7f31281996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:13.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3128074bf0 0x7f31281996d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3124006010 tx=0x7f31240045e0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 shutdown_connections 2024-09-21T04:36:13.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f311407b050 0x7f311407d510 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31280761c0 0x7f3128199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 --2- 172.21.15.7:0/2878632870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3128074bf0 0x7f31281996d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 >> 172.21.15.7:0/2878632870 conn(0x7f31280fd140 msgr2=0x7f3128107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:13.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 shutdown_connections 2024-09-21T04:36:13.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.174+0000 7f312f886640 1 -- 172.21.15.7:0/2878632870 wait complete. 2024-09-21T04:36:13.187 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:36:13.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:12 smithi053 ceph-mon[35554]: pgmap v10: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-09-21T04:36:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.297+0000 7f93a56a5640 1 -- 172.21.15.7:0/142741221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93a0073b60 msgr2=0x7f93a0109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.297+0000 7f93a56a5640 1 --2- 172.21.15.7:0/142741221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93a0073b60 0x7f93a0109570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f939800ad30 tx=0x7f939802b860 comp rx=0 tx=0).stop 2024-09-21T04:36:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.298+0000 7f93a56a5640 1 -- 172.21.15.7:0/142741221 shutdown_connections 2024-09-21T04:36:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.298+0000 7f93a56a5640 1 --2- 172.21.15.7:0/142741221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93a0109ab0 0x7f93a0109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.298+0000 7f93a56a5640 1 --2- 172.21.15.7:0/142741221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93a0073b60 0x7f93a0109570 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.298+0000 7f93a56a5640 1 -- 172.21.15.7:0/142741221 >> 172.21.15.7:0/142741221 conn(0x7f93a006ed60 msgr2=0x7f93a00711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.299+0000 7f93a56a5640 1 -- 172.21.15.7:0/142741221 shutdown_connections 2024-09-21T04:36:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.299+0000 7f93a56a5640 1 -- 172.21.15.7:0/142741221 wait complete. 2024-09-21T04:36:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.299+0000 7f93a56a5640 1 Processor -- start 2024-09-21T04:36:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.300+0000 7f93a56a5640 1 -- start start 2024-09-21T04:36:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.300+0000 7f93a56a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93a0073b60 0x7f93a0118eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.300+0000 7f93a56a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93a0109ab0 0x7f93a01193f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.300+0000 7f93a56a5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0110240 con 0x7f93a0109ab0 2024-09-21T04:36:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.300+0000 7f93a56a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01100c0 con 0x7f93a0073b60 2024-09-21T04:36:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.300+0000 7f939effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93a0073b60 0x7f93a0118eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.301+0000 7f939effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93a0073b60 0x7f93a0118eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:59374/0 (socket says 172.21.15.7:59374) 2024-09-21T04:36:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.301+0000 7f939effd640 1 -- 172.21.15.7:0/3234520193 learned_addr learned my addr 172.21.15.7:0/3234520193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.301+0000 7f939effd640 1 -- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93a0109ab0 msgr2=0x7f93a01193f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.301+0000 7f939effd640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93a0109ab0 0x7f93a01193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.301+0000 7f939effd640 1 -- 172.21.15.7:0/3234520193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a0119a70 con 0x7f93a0073b60 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.301+0000 7f939effd640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93a0073b60 0x7f93a0118eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9398005c00 tx=0x7f9398003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.302+0000 7f9387fff640 1 -- 172.21.15.7:0/3234520193 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9398031560 con 0x7f93a0073b60 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.302+0000 7f9387fff640 1 -- 172.21.15.7:0/3234520193 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9398003dc0 con 0x7f93a0073b60 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.302+0000 7f9387fff640 1 -- 172.21.15.7:0/3234520193 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9398004300 con 0x7f93a0073b60 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.302+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a011c4a0 con 0x7f93a0073b60 2024-09-21T04:36:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.302+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a011cab0 con 0x7f93a0073b60 2024-09-21T04:36:13.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.303+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a0109570 con 0x7f93a0073b60 2024-09-21T04:36:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.305+0000 7f9387fff640 1 -- 172.21.15.7:0/3234520193 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7f9398031d30 con 0x7f93a0073b60 2024-09-21T04:36:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.305+0000 7f9387fff640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f938807b100 0x7f938807d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.306+0000 7f9387fff640 1 -- 172.21.15.7:0/3234520193 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f93980bf2d0 con 0x7f93a0073b60 2024-09-21T04:36:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.306+0000 7f939e7fc640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f938807b100 0x7f938807d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:13.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.307+0000 7f939e7fc640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f938807b100 0x7f938807d5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f93a011a360 tx=0x7f9394009630 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:13.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.312+0000 7f9387fff640 1 -- 172.21.15.7:0/3234520193 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f939808a6e0 con 0x7f93a0073b60 2024-09-21T04:36:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.626+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f93a011ce20 con 0x7f938807b100 2024-09-21T04:36:13.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.627+0000 7f9387fff640 1 -- 172.21.15.7:0/3234520193 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+329 (secure 0 0 0) 0x7f93a011ce20 con 0x7f938807b100 2024-09-21T04:36:13.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f938807b100 msgr2=0x7f938807d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:13.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f938807b100 0x7f938807d5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f93a011a360 tx=0x7f9394009630 comp rx=0 tx=0).stop 2024-09-21T04:36:13.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93a0073b60 msgr2=0x7f93a0118eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:13.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93a0073b60 0x7f93a0118eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9398005c00 tx=0x7f9398003ba0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 shutdown_connections 2024-09-21T04:36:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f938807b100 0x7f938807d5c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93a0109ab0 0x7f93a01193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 --2- 172.21.15.7:0/3234520193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93a0073b60 0x7f93a0118eb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 >> 172.21.15.7:0/3234520193 conn(0x7f93a006ed60 msgr2=0x7f93a0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.629+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 shutdown_connections 2024-09-21T04:36:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.630+0000 7f93a56a5640 1 -- 172.21.15.7:0/3234520193 wait complete. 2024-09-21T04:36:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 -- 172.21.15.7:0/4210097280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb8104d20 msgr2=0x7efeb8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 --2- 172.21.15.7:0/4210097280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb8104d20 0x7efeb8107110 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efeac0098b0 tx=0x7efeac02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 -- 172.21.15.7:0/4210097280 shutdown_connections 2024-09-21T04:36:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 --2- 172.21.15.7:0/4210097280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb8104d20 0x7efeb8107110 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 --2- 172.21.15.7:0/4210097280 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb81023f0 0x7efeb81047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 -- 172.21.15.7:0/4210097280 >> 172.21.15.7:0/4210097280 conn(0x7efeb80fc010 msgr2=0x7efeb80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 -- 172.21.15.7:0/4210097280 shutdown_connections 2024-09-21T04:36:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 -- 172.21.15.7:0/4210097280 wait complete. 2024-09-21T04:36:13.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.743+0000 7efebe67b640 1 Processor -- start 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efebe67b640 1 -- start start 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efebe67b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb81023f0 0x7efeb8197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efebe67b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb8104d20 0x7efeb8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efebe67b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efeb810a7b0 con 0x7efeb81023f0 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efebe67b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efeb810a630 con 0x7efeb8104d20 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efeb7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb81023f0 0x7efeb8197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efeb77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb8104d20 0x7efeb8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efeb7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb81023f0 0x7efeb8197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55978/0 (socket says 172.21.15.7:55978) 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.744+0000 7efeb7fff640 1 -- 172.21.15.7:0/3039064049 learned_addr learned my addr 172.21.15.7:0/3039064049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.745+0000 7efeb7fff640 1 -- 172.21.15.7:0/3039064049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb8104d20 msgr2=0x7efeb8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.745+0000 7efeb7fff640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb8104d20 0x7efeb8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.745+0000 7efeb7fff640 1 -- 172.21.15.7:0/3039064049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb81980f0 con 0x7efeb81023f0 2024-09-21T04:36:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.745+0000 7efeb77fe640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb8104d20 0x7efeb8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:36:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.745+0000 7efeb7fff640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb81023f0 0x7efeb8197530 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efea800e970 tx=0x7efea800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.745+0000 7efeb57fa640 1 -- 172.21.15.7:0/3039064049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efea800cac0 con 0x7efeb81023f0 2024-09-21T04:36:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.745+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb819cb30 con 0x7efeb81023f0 2024-09-21T04:36:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.745+0000 7efeb57fa640 1 -- 172.21.15.7:0/3039064049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efea80054e0 con 0x7efeb81023f0 2024-09-21T04:36:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.746+0000 7efeb57fa640 1 -- 172.21.15.7:0/3039064049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efea8010430 con 0x7efeb81023f0 2024-09-21T04:36:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.746+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb819d090 con 0x7efeb81023f0 2024-09-21T04:36:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.747+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb810af90 con 0x7efeb81023f0 2024-09-21T04:36:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.747+0000 7efeb57fa640 1 -- 172.21.15.7:0/3039064049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7efea8004f00 con 0x7efeb81023f0 2024-09-21T04:36:13.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.748+0000 7efeb57fa640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efea007b0b0 0x7efea007d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.748+0000 7efeb57fa640 1 -- 172.21.15.7:0/3039064049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7efea809e830 con 0x7efeb81023f0 2024-09-21T04:36:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.748+0000 7efeb77fe640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efea007b0b0 0x7efea007d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:13.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.748+0000 7efeb77fe640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efea007b0b0 0x7efea007d570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efeac009880 tx=0x7efeac0058d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:13.755+0000 7efeb57fa640 1 -- 172.21.15.7:0/3039064049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7efea8069ce0 con 0x7efeb81023f0 2024-09-21T04:36:13.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:13 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:36:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.008+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7efeb8105780 con 0x7efea007b0b0 2024-09-21T04:36:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.029+0000 7efeb57fa640 1 -- 172.21.15.7:0/3039064049 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7efeb8105780 con 0x7efea007b0b0 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (7m) 9s ago 9m 24.2M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (82s) 9s ago 9m 8544k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (59s) 11s ago 8m 8372k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (9m) 9s ago 9m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (8m) 11s ago 8m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (7m) 9s ago 8m 82.9M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (3m) 9s ago 3m 242M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (3m) 9s ago 3m 278M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (3m) 11s ago 3m 25.7M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (3m) 11s ago 3m 30.7M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:36:14.032 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (28s) 9s ago 10m 490M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (2m) 11s ago 8m 139M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7d35aae8ab52 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (10m) 9s ago 10m 57.6M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (8m) 11s ago 8m 48.5M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (9m) 9s ago 9m 16.1M - 1.5.0 0da6a335fe13 b4a8255a594a 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (8m) 11s ago 8m 16.6M - 1.5.0 0da6a335fe13 88246b15ca5d 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 9s ago 7m 174M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (6m) 9s ago 6m 182M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (6m) 9s ago 6m 160M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (5m) 11s ago 5m 208M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (5m) 11s ago 5m 183M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (4m) 11s ago 4m 166M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:36:14.033 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (71s) 9s ago 8m 58.5M - 2.43.0 a07b618ecd1d 0634b307248e 2024-09-21T04:36:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.032+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efea007b0b0 msgr2=0x7efea007d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.032+0000 7efebe67b640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efea007b0b0 0x7efea007d570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efeac009880 tx=0x7efeac0058d0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.032+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb81023f0 msgr2=0x7efeb8197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.032+0000 7efebe67b640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb81023f0 0x7efeb8197530 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7efea800e970 tx=0x7efea800ed70 comp rx=0 tx=0).stop 2024-09-21T04:36:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.033+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 shutdown_connections 2024-09-21T04:36:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.033+0000 7efebe67b640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efea007b0b0 0x7efea007d570 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.033+0000 7efebe67b640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb8104d20 0x7efeb8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.033+0000 7efebe67b640 1 --2- 172.21.15.7:0/3039064049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb81023f0 0x7efeb8197530 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.033+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 >> 172.21.15.7:0/3039064049 conn(0x7efeb80fc010 msgr2=0x7efeb80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.033+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 shutdown_connections 2024-09-21T04:36:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.033+0000 7efebe67b640 1 -- 172.21.15.7:0/3039064049 wait complete. 2024-09-21T04:36:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.147+0000 7faa735b1640 1 -- 172.21.15.7:0/2172727435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 msgr2=0x7faa6c101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.147+0000 7faa735b1640 1 --2- 172.21.15.7:0/2172727435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 0x7faa6c101ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009880 tx=0x7faa5c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.147+0000 7faa735b1640 1 -- 172.21.15.7:0/2172727435 shutdown_connections 2024-09-21T04:36:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.147+0000 7faa735b1640 1 --2- 172.21.15.7:0/2172727435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 0x7faa6c101ee0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.147+0000 7faa735b1640 1 --2- 172.21.15.7:0/2172727435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c100860 0x7faa6c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.147+0000 7faa735b1640 1 -- 172.21.15.7:0/2172727435 >> 172.21.15.7:0/2172727435 conn(0x7faa6c0fc010 msgr2=0x7faa6c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.147+0000 7faa735b1640 1 -- 172.21.15.7:0/2172727435 shutdown_connections 2024-09-21T04:36:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.147+0000 7faa735b1640 1 -- 172.21.15.7:0/2172727435 wait complete. 2024-09-21T04:36:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa735b1640 1 Processor -- start 2024-09-21T04:36:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa735b1640 1 -- start start 2024-09-21T04:36:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa735b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c100860 0x7faa6c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa735b1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 0x7faa6c196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa735b1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c109680 con 0x7faa6c101a60 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa735b1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c109500 con 0x7faa6c100860 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa70b4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 0x7faa6c196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa70b4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 0x7faa6c196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55992/0 (socket says 172.21.15.7:55992) 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.148+0000 7faa70b4c640 1 -- 172.21.15.7:0/2583572958 learned_addr learned my addr 172.21.15.7:0/2583572958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.149+0000 7faa7134d640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c100860 0x7faa6c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.149+0000 7faa7134d640 1 -- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 msgr2=0x7faa6c196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.149+0000 7faa7134d640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 0x7faa6c196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.149+0000 7faa7134d640 1 -- 172.21.15.7:0/2583572958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c196f80 con 0x7faa6c100860 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.149+0000 7faa70b4c640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 0x7faa6c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.149+0000 7faa7134d640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c100860 0x7faa6c1963c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa6800e940 tx=0x7faa6800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.149+0000 7faa627fc640 1 -- 172.21.15.7:0/2583572958 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faa6800cac0 con 0x7faa6c100860 2024-09-21T04:36:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.150+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c069d00 con 0x7faa6c100860 2024-09-21T04:36:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.150+0000 7faa627fc640 1 -- 172.21.15.7:0/2583572958 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa6800cc60 con 0x7faa6c100860 2024-09-21T04:36:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.150+0000 7faa627fc640 1 -- 172.21.15.7:0/2583572958 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faa68003e10 con 0x7faa6c100860 2024-09-21T04:36:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.150+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c06a290 con 0x7faa6c100860 2024-09-21T04:36:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.151+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c100c60 con 0x7faa6c100860 2024-09-21T04:36:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.152+0000 7faa627fc640 1 -- 172.21.15.7:0/2583572958 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7faa68004000 con 0x7faa6c100860 2024-09-21T04:36:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.153+0000 7faa627fc640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa5807b050 0x7faa5807d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.153+0000 7faa627fc640 1 -- 172.21.15.7:0/2583572958 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7faa6809e910 con 0x7faa6c100860 2024-09-21T04:36:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.153+0000 7faa70b4c640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa5807b050 0x7faa5807d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.154+0000 7faa70b4c640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa5807b050 0x7faa5807d510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0096c0 tx=0x7faa5c005a20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.157+0000 7faa627fc640 1 -- 172.21.15.7:0/2583572958 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7faa68069d60 con 0x7faa6c100860 2024-09-21T04:36:14.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:13 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:36:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.474+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7faa6c06a600 con 0x7faa6c100860 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.475+0000 7faa627fc640 1 -- 172.21.15.7:0/2583572958 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7faa6806ec10 con 0x7faa6c100860 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:36:14.478 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 12, 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:36:14.479 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:36:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa5807b050 msgr2=0x7faa5807d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa5807b050 0x7faa5807d510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0096c0 tx=0x7faa5c005a20 comp rx=0 tx=0).stop 2024-09-21T04:36:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c100860 msgr2=0x7faa6c1963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c100860 0x7faa6c1963c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa6800e940 tx=0x7faa6800ee00 comp rx=0 tx=0).stop 2024-09-21T04:36:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 shutdown_connections 2024-09-21T04:36:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa5807b050 0x7faa5807d510 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6c101a60 0x7faa6c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 --2- 172.21.15.7:0/2583572958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c100860 0x7faa6c1963c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.477+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 >> 172.21.15.7:0/2583572958 conn(0x7faa6c0fc010 msgr2=0x7faa6c104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.478+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 shutdown_connections 2024-09-21T04:36:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.478+0000 7faa735b1640 1 -- 172.21.15.7:0/2583572958 wait complete. 2024-09-21T04:36:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.568+0000 7f401fc2a640 1 -- 172.21.15.7:0/4021130554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 msgr2=0x7f4018101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.568+0000 7f401fc2a640 1 --2- 172.21.15.7:0/4021130554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 0x7f4018101ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4008009880 tx=0x7f400802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.568+0000 7f401fc2a640 1 -- 172.21.15.7:0/4021130554 shutdown_connections 2024-09-21T04:36:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.568+0000 7f401fc2a640 1 --2- 172.21.15.7:0/4021130554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 0x7f4018101ee0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.568+0000 7f401fc2a640 1 --2- 172.21.15.7:0/4021130554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4018100860 0x7f4018100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.568+0000 7f401fc2a640 1 -- 172.21.15.7:0/4021130554 >> 172.21.15.7:0/4021130554 conn(0x7f40180fc010 msgr2=0x7f40180fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.568+0000 7f401fc2a640 1 -- 172.21.15.7:0/4021130554 shutdown_connections 2024-09-21T04:36:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.568+0000 7f401fc2a640 1 -- 172.21.15.7:0/4021130554 wait complete. 2024-09-21T04:36:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401fc2a640 1 Processor -- start 2024-09-21T04:36:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401fc2a640 1 -- start start 2024-09-21T04:36:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401fc2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4018100860 0x7f40181963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401fc2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 0x7f4018196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401fc2a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40181096a0 con 0x7f4018101a60 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401fc2a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4018109520 con 0x7f4018100860 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401d1c5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 0x7f4018196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401d9c6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4018100860 0x7f40181963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401d1c5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 0x7f4018196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56002/0 (socket says 172.21.15.7:56002) 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.569+0000 7f401d1c5640 1 -- 172.21.15.7:0/2437477040 learned_addr learned my addr 172.21.15.7:0/2437477040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f401d1c5640 1 -- 172.21.15.7:0/2437477040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4018100860 msgr2=0x7f40181963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f401d1c5640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4018100860 0x7f40181963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f401d1c5640 1 -- 172.21.15.7:0/2437477040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4018196fa0 con 0x7f4018101a60 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f401d9c6640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4018100860 0x7f40181963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:36:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f401d1c5640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 0x7f4018196920 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4008006010 tx=0x7f4008004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f400effd640 1 -- 172.21.15.7:0/2437477040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f400803f070 con 0x7f4018101a60 2024-09-21T04:36:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f400effd640 1 -- 172.21.15.7:0/2437477040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f40080049d0 con 0x7f4018101a60 2024-09-21T04:36:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f400effd640 1 -- 172.21.15.7:0/2437477040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4008035030 con 0x7f4018101a60 2024-09-21T04:36:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4018069ca0 con 0x7f4018101a60 2024-09-21T04:36:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.570+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401806a230 con 0x7f4018101a60 2024-09-21T04:36:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.572+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4018100c60 con 0x7f4018101a60 2024-09-21T04:36:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.572+0000 7f400effd640 1 -- 172.21.15.7:0/2437477040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7f4008042070 con 0x7f4018101a60 2024-09-21T04:36:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.573+0000 7f400effd640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f400407d2c0 0x7f400407f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.573+0000 7f400effd640 1 -- 172.21.15.7:0/2437477040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f40080becf0 con 0x7f4018101a60 2024-09-21T04:36:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.573+0000 7f401d9c6640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f400407d2c0 0x7f400407f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.574+0000 7f401d9c6640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f400407d2c0 0x7f400407f780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40181018c0 tx=0x7f4014005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.578+0000 7f400effd640 1 -- 172.21.15.7:0/2437477040 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f400808a1a0 con 0x7f4018101a60 2024-09-21T04:36:14.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.839+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f401806a5a0 con 0x7f400407d2c0 2024-09-21T04:36:14.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.841+0000 7f400effd640 1 -- 172.21.15.7:0/2437477040 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+329 (secure 0 0 0) 0x7f401806a5a0 con 0x7f400407d2c0 2024-09-21T04:36:14.843 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/2 daemons upgraded", 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:36:14.844 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:36:14.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f400407d2c0 msgr2=0x7f400407f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f400407d2c0 0x7f400407f780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40181018c0 tx=0x7f4014005f90 comp rx=0 tx=0).stop 2024-09-21T04:36:14.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 msgr2=0x7f4018196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:14.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 0x7f4018196920 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4008006010 tx=0x7f4008004570 comp rx=0 tx=0).stop 2024-09-21T04:36:14.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 shutdown_connections 2024-09-21T04:36:14.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f400407d2c0 0x7f400407f780 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4018101a60 0x7f4018196920 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 --2- 172.21.15.7:0/2437477040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4018100860 0x7f40181963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:14.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.843+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 >> 172.21.15.7:0/2437477040 conn(0x7f40180fc010 msgr2=0x7f4018104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:14.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.844+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 shutdown_connections 2024-09-21T04:36:14.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:14.844+0000 7f401fc2a640 1 -- 172.21.15.7:0/2437477040 wait complete. 2024-09-21T04:36:15.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:14 smithi007 ceph-mon[27954]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:15.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:14 smithi007 ceph-mon[27954]: from='client.24545 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:15.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:14 smithi007 ceph-mon[27954]: pgmap v11: 65 pgs: 65 active+clean; 286 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-09-21T04:36:15.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:14 smithi007 ceph-mon[27954]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:15.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:14 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2583572958' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:15.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:14 smithi053 ceph-mon[35554]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:14 smithi053 ceph-mon[35554]: from='client.24545 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:14 smithi053 ceph-mon[35554]: pgmap v11: 65 pgs: 65 active+clean; 286 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-09-21T04:36:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:14 smithi053 ceph-mon[35554]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:14 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2583572958' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:16.169 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:15 smithi007 ceph-mon[27954]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:16.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:15 smithi053 ceph-mon[35554]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:17.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:16 smithi053 ceph-mon[35554]: pgmap v12: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 140 op/s 2024-09-21T04:36:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:16 smithi007 ceph-mon[27954]: pgmap v12: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 140 op/s 2024-09-21T04:36:19.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:18 smithi053 ceph-mon[35554]: pgmap v13: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-21T04:36:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:18 smithi007 ceph-mon[27954]: pgmap v13: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-21T04:36:21.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:20 smithi053 ceph-mon[35554]: pgmap v14: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-21T04:36:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:20 smithi007 ceph-mon[27954]: pgmap v14: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-21T04:36:22.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:22 smithi007 ceph-mon[27954]: pgmap v15: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 87 op/s 2024-09-21T04:36:22.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:22 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:22.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:22 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:22.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:22 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:36:22.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:22 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:36:23.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:22 smithi053 ceph-mon[35554]: pgmap v15: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 87 op/s 2024-09-21T04:36:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:22 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:22 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:22 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:36:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:22 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:36:23.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-21T04:36:23.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-21T04:36:23.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:36:23.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:36:23.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:36:23.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:23.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:23.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:23.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:23.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:23 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:23 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:24 smithi007 ceph-mon[27954]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:36:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:24 smithi007 ceph-mon[27954]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:36:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:24 smithi007 ceph-mon[27954]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:36:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:24 smithi007 ceph-mon[27954]: pgmap v16: 65 pgs: 65 active+clean; 297 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 114 op/s 2024-09-21T04:36:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:24 smithi007 ceph-mon[27954]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-09-21T04:36:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:24 smithi053 ceph-mon[35554]: Updating smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:36:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:24 smithi053 ceph-mon[35554]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-21T04:36:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:24 smithi053 ceph-mon[35554]: Updating smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T04:36:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:24 smithi053 ceph-mon[35554]: pgmap v16: 65 pgs: 65 active+clean; 297 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 114 op/s 2024-09-21T04:36:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:24 smithi053 ceph-mon[35554]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-09-21T04:36:25.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:25 smithi007 ceph-mon[27954]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-09-21T04:36:26.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:25 smithi053 ceph-mon[35554]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-09-21T04:36:27.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:26 smithi053 ceph-mon[35554]: pgmap v17: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-21T04:36:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:26 smithi007 ceph-mon[27954]: pgmap v17: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-09-21T04:36:29.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:28 smithi053 ceph-mon[35554]: pgmap v18: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-09-21T04:36:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:28 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:36:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:28 smithi007 ceph-mon[27954]: pgmap v18: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-09-21T04:36:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:28 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:36:31.111 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:30 smithi007 ceph-mon[27954]: pgmap v19: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-09-21T04:36:31.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:30 smithi053 ceph-mon[35554]: pgmap v19: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-09-21T04:36:33.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:32 smithi053 ceph-mon[35554]: pgmap v20: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 83 op/s 2024-09-21T04:36:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:32 smithi007 ceph-mon[27954]: pgmap v20: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 83 op/s 2024-09-21T04:36:35.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:34 smithi053 ceph-mon[35554]: pgmap v21: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-09-21T04:36:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:34 smithi007 ceph-mon[27954]: pgmap v21: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-09-21T04:36:37.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:36 smithi053 ceph-mon[35554]: pgmap v22: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-09-21T04:36:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:36 smithi007 ceph-mon[27954]: pgmap v22: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-09-21T04:36:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:38 smithi007 ceph-mon[27954]: pgmap v23: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-09-21T04:36:39.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:38 smithi053 ceph-mon[35554]: pgmap v23: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-09-21T04:36:40.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:40 smithi007 ceph-mon[27954]: pgmap v24: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 399 op/s 2024-09-21T04:36:40.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:40 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:40.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:40 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:40.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:40 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:36:40.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:40 smithi007 ceph-mon[27954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:36:40.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:40 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:36:41.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:40 smithi053 ceph-mon[35554]: pgmap v24: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 399 op/s 2024-09-21T04:36:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:40 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:40 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:40 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:36:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:40 smithi053 ceph-mon[35554]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:36:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:40 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:36:42.134 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:41 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:36:42.134 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:41 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:36:42.134 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:41 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:36:42.134 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:41 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:36:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:41 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:36:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:41 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi053.lnadao", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-21T04:36:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:41 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-21T04:36:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:41 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:36:43.129 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:42 smithi053 ceph-mon[35554]: Upgrade: Updating mgr.smithi053.lnadao 2024-09-21T04:36:43.129 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:42 smithi053 ceph-mon[35554]: Deploying daemon mgr.smithi053.lnadao on smithi053 2024-09-21T04:36:43.129 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:42 smithi053 ceph-mon[35554]: pgmap v25: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 356 op/s 2024-09-21T04:36:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:42 smithi007 ceph-mon[27954]: Upgrade: Updating mgr.smithi053.lnadao 2024-09-21T04:36:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:42 smithi007 ceph-mon[27954]: Deploying daemon mgr.smithi053.lnadao on smithi053 2024-09-21T04:36:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:42 smithi007 ceph-mon[27954]: pgmap v25: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 356 op/s 2024-09-21T04:36:44.098 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:43 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:36:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:43 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:36:44.911 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:44 smithi053 ceph-mon[35554]: pgmap v26: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 420 op/s 2024-09-21T04:36:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.956+0000 7f2e70736640 1 -- 172.21.15.7:0/3876974322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 msgr2=0x7f2e68100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.956+0000 7f2e70736640 1 --2- 172.21.15.7:0/3876974322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 0x7f2e68100020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009880 tx=0x7f2e6402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.956+0000 7f2e70736640 1 -- 172.21.15.7:0/3876974322 shutdown_connections 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.956+0000 7f2e70736640 1 --2- 172.21.15.7:0/3876974322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e68102c00 0x7f2e68103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.956+0000 7f2e70736640 1 --2- 172.21.15.7:0/3876974322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 0x7f2e68100020 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.956+0000 7f2e70736640 1 -- 172.21.15.7:0/3876974322 >> 172.21.15.7:0/3876974322 conn(0x7f2e680fb770 msgr2=0x7f2e680fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.956+0000 7f2e70736640 1 -- 172.21.15.7:0/3876974322 shutdown_connections 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.956+0000 7f2e70736640 1 -- 172.21.15.7:0/3876974322 wait complete. 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e70736640 1 Processor -- start 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e70736640 1 -- start start 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e70736640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 0x7f2e681024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e70736640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e68102c00 0x7f2e68100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e70736640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6810a780 con 0x7f2e680ffc20 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e70736640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6810a600 con 0x7f2e68102c00 2024-09-21T04:36:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e6e4d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 0x7f2e681024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e6e4d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 0x7f2e681024f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33114/0 (socket says 172.21.15.7:33114) 2024-09-21T04:36:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e6e4d2640 1 -- 172.21.15.7:0/3113560377 learned_addr learned my addr 172.21.15.7:0/3113560377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.957+0000 7f2e6dcd1640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e68102c00 0x7f2e68100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e6dcd1640 1 -- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 msgr2=0x7f2e681024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e6dcd1640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 0x7f2e681024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e6dcd1640 1 -- 172.21.15.7:0/3113560377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e681010f0 con 0x7f2e68102c00 2024-09-21T04:36:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e6e4d2640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 0x7f2e681024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:36:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e6dcd1640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e68102c00 0x7f2e68100bb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800e970 tx=0x7f2e5800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e5f7fe640 1 -- 172.21.15.7:0/3113560377 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2e5800cac0 con 0x7f2e68102c00 2024-09-21T04:36:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e681013e0 con 0x7f2e68102c00 2024-09-21T04:36:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e68101970 con 0x7f2e68102c00 2024-09-21T04:36:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e5f7fe640 1 -- 172.21.15.7:0/3113560377 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2e5800cc60 con 0x7f2e68102c00 2024-09-21T04:36:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.958+0000 7f2e5f7fe640 1 -- 172.21.15.7:0/3113560377 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2e58018720 con 0x7f2e68102c00 2024-09-21T04:36:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.960+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e68100020 con 0x7f2e68102c00 2024-09-21T04:36:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.960+0000 7f2e5f7fe640 1 -- 172.21.15.7:0/3113560377 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7f2e58010430 con 0x7f2e68102c00 2024-09-21T04:36:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.961+0000 7f2e5f7fe640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2e5407b0b0 0x7f2e5407d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.961+0000 7f2e5f7fe640 1 -- 172.21.15.7:0/3113560377 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f2e5809e6c0 con 0x7f2e68102c00 2024-09-21T04:36:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.961+0000 7f2e6e4d2640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2e5407b0b0 0x7f2e5407d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.962+0000 7f2e6e4d2640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2e5407b0b0 0x7f2e5407d570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2e640096c0 tx=0x7f2e640058d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:44.966+0000 7f2e5f7fe640 1 -- 172.21.15.7:0/3113560377 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f2e58069b10 con 0x7f2e68102c00 2024-09-21T04:36:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:44 smithi007 ceph-mon[27954]: pgmap v26: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 420 op/s 2024-09-21T04:36:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.225+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2e68107210 con 0x7f2e5407b0b0 2024-09-21T04:36:45.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.226+0000 7f2e5f7fe640 1 -- 172.21.15.7:0/3113560377 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f2e68107210 con 0x7f2e5407b0b0 2024-09-21T04:36:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2e5407b0b0 msgr2=0x7f2e5407d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2e5407b0b0 0x7f2e5407d570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2e640096c0 tx=0x7f2e640058d0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e68102c00 msgr2=0x7f2e68100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e68102c00 0x7f2e68100bb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800e970 tx=0x7f2e5800ee30 comp rx=0 tx=0).stop 2024-09-21T04:36:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 shutdown_connections 2024-09-21T04:36:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2e5407b0b0 0x7f2e5407d570 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e68102c00 0x7f2e68100bb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 --2- 172.21.15.7:0/3113560377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e680ffc20 0x7f2e681024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 >> 172.21.15.7:0/3113560377 conn(0x7f2e680fb770 msgr2=0x7f2e680fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 shutdown_connections 2024-09-21T04:36:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.228+0000 7f2e70736640 1 -- 172.21.15.7:0/3113560377 wait complete. 2024-09-21T04:36:45.242 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:36:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.349+0000 7f76dab47640 1 -- 172.21.15.7:0/3456902645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 msgr2=0x7f76d4101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.349+0000 7f76dab47640 1 --2- 172.21.15.7:0/3456902645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 0x7f76d4101d70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009880 tx=0x7f76c802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.349+0000 7f76dab47640 1 -- 172.21.15.7:0/3456902645 shutdown_connections 2024-09-21T04:36:45.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.349+0000 7f76dab47640 1 --2- 172.21.15.7:0/3456902645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4102b70 0x7f76d4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.349+0000 7f76dab47640 1 --2- 172.21.15.7:0/3456902645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 0x7f76d4101d70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.349+0000 7f76dab47640 1 -- 172.21.15.7:0/3456902645 >> 172.21.15.7:0/3456902645 conn(0x7f76d40fd140 msgr2=0x7f76d40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.349+0000 7f76dab47640 1 -- 172.21.15.7:0/3456902645 shutdown_connections 2024-09-21T04:36:45.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.349+0000 7f76dab47640 1 -- 172.21.15.7:0/3456902645 wait complete. 2024-09-21T04:36:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.350+0000 7f76dab47640 1 Processor -- start 2024-09-21T04:36:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.350+0000 7f76dab47640 1 -- start start 2024-09-21T04:36:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.350+0000 7f76dab47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 0x7f76d41974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:45.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.350+0000 7f76dab47640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4102b70 0x7f76d41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.350+0000 7f76dab47640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410a7b0 con 0x7f76d4101970 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.350+0000 7f76dab47640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76d410a630 con 0x7f76d4102b70 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.351+0000 7f76d88e3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 0x7f76d41974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.351+0000 7f76d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4102b70 0x7f76d41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.351+0000 7f76d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4102b70 0x7f76d41979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:60932/0 (socket says 172.21.15.7:60932) 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.351+0000 7f76d3fff640 1 -- 172.21.15.7:0/1299810878 learned_addr learned my addr 172.21.15.7:0/1299810878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.351+0000 7f76d88e3640 1 -- 172.21.15.7:0/1299810878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4102b70 msgr2=0x7f76d41979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.351+0000 7f76d88e3640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4102b70 0x7f76d41979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.351+0000 7f76d88e3640 1 -- 172.21.15.7:0/1299810878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d4198060 con 0x7f76d4101970 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.352+0000 7f76d88e3640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 0x7f76d41974a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009230 tx=0x7f76c80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.352+0000 7f76d1ffb640 1 -- 172.21.15.7:0/1299810878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f76c803f070 con 0x7f76d4101970 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.352+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d419ca40 con 0x7f76d4101970 2024-09-21T04:36:45.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.352+0000 7f76d1ffb640 1 -- 172.21.15.7:0/1299810878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f76c80049d0 con 0x7f76d4101970 2024-09-21T04:36:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.352+0000 7f76d1ffb640 1 -- 172.21.15.7:0/1299810878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f76c8035030 con 0x7f76d4101970 2024-09-21T04:36:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.352+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d419cfd0 con 0x7f76d4101970 2024-09-21T04:36:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.354+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d4101d70 con 0x7f76d4101970 2024-09-21T04:36:45.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.355+0000 7f76d1ffb640 1 -- 172.21.15.7:0/1299810878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7f76c8042070 con 0x7f76d4101970 2024-09-21T04:36:45.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.355+0000 7f76d1ffb640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76c407b0a0 0x7f76c407d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:45.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.355+0000 7f76d1ffb640 1 -- 172.21.15.7:0/1299810878 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f76c80beaa0 con 0x7f76d4101970 2024-09-21T04:36:45.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.355+0000 7f76d3fff640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76c407b0a0 0x7f76c407d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:45.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.356+0000 7f76d3fff640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76c407b0a0 0x7f76c407d560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f76d4198950 tx=0x7f76c0005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.362+0000 7f76d1ffb640 1 -- 172.21.15.7:0/1299810878 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f76c8089f50 con 0x7f76d4101970 2024-09-21T04:36:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.729+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f76d4062630 con 0x7f76c407b0a0 2024-09-21T04:36:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.731+0000 7f76d1ffb640 1 -- 172.21.15.7:0/1299810878 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+321 (secure 0 0 0) 0x7f76d4062630 con 0x7f76c407b0a0 2024-09-21T04:36:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76c407b0a0 msgr2=0x7f76c407d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76c407b0a0 0x7f76c407d560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f76d4198950 tx=0x7f76c0005f90 comp rx=0 tx=0).stop 2024-09-21T04:36:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 msgr2=0x7f76d41974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 0x7f76d41974a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009230 tx=0x7f76c80045e0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 shutdown_connections 2024-09-21T04:36:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76c407b0a0 0x7f76c407d560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76d4102b70 0x7f76d41979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 --2- 172.21.15.7:0/1299810878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d4101970 0x7f76d41974a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 >> 172.21.15.7:0/1299810878 conn(0x7f76d40fd140 msgr2=0x7f76d4106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 shutdown_connections 2024-09-21T04:36:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.735+0000 7f76dab47640 1 -- 172.21.15.7:0/1299810878 wait complete. 2024-09-21T04:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.846+0000 7f2bc718a640 1 -- 172.21.15.7:0/3941397673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 msgr2=0x7f2bc0102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.846+0000 7f2bc718a640 1 --2- 172.21.15.7:0/3941397673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 0x7f2bc0102fe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009880 tx=0x7f2bac02a4b0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.846+0000 7f2bc718a640 1 -- 172.21.15.7:0/3941397673 shutdown_connections 2024-09-21T04:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.846+0000 7f2bc718a640 1 --2- 172.21.15.7:0/3941397673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 0x7f2bc0102fe0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.846+0000 7f2bc718a640 1 --2- 172.21.15.7:0/3941397673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bc0101960 0x7f2bc0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.846+0000 7f2bc718a640 1 -- 172.21.15.7:0/3941397673 >> 172.21.15.7:0/3941397673 conn(0x7f2bc00fd0f0 msgr2=0x7f2bc00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.847+0000 7f2bc718a640 1 -- 172.21.15.7:0/3941397673 shutdown_connections 2024-09-21T04:36:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.847+0000 7f2bc718a640 1 -- 172.21.15.7:0/3941397673 wait complete. 2024-09-21T04:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.847+0000 7f2bc718a640 1 Processor -- start 2024-09-21T04:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.847+0000 7f2bc718a640 1 -- start start 2024-09-21T04:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bc718a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bc0101960 0x7f2bc01952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bc718a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 0x7f2bc01957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bc718a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc01085b0 con 0x7f2bc0102b60 2024-09-21T04:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bc718a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0108430 con 0x7f2bc0101960 2024-09-21T04:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 0x7f2bc01957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bc4f26640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bc0101960 0x7f2bc01952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 0x7f2bc01957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33164/0 (socket says 172.21.15.7:33164) 2024-09-21T04:36:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bc4f26640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bc0101960 0x7f2bc01952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:60954/0 (socket says 172.21.15.7:60954) 2024-09-21T04:36:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.848+0000 7f2bbffff640 1 -- 172.21.15.7:0/1156753563 learned_addr learned my addr 172.21.15.7:0/1156753563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.849+0000 7f2bc4f26640 1 -- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 msgr2=0x7f2bc01957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.849+0000 7f2bc4f26640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 0x7f2bc01957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.849+0000 7f2bc4f26640 1 -- 172.21.15.7:0/1156753563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc0195e60 con 0x7f2bc0101960 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.849+0000 7f2bc4f26640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bc0101960 0x7f2bc01952a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400e940 tx=0x7f2bb400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.849+0000 7f2bbdffb640 1 -- 172.21.15.7:0/1156753563 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bb400cac0 con 0x7f2bc0101960 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.849+0000 7f2bbdffb640 1 -- 172.21.15.7:0/1156753563 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bb400cc60 con 0x7f2bc0101960 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.849+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc019a8a0 con 0x7f2bc0101960 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.849+0000 7f2bbdffb640 1 -- 172.21.15.7:0/1156753563 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bb4003e10 con 0x7f2bc0101960 2024-09-21T04:36:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.850+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc019ae30 con 0x7f2bc0101960 2024-09-21T04:36:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.851+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc0101d60 con 0x7f2bc0101960 2024-09-21T04:36:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.852+0000 7f2bbdffb640 1 -- 172.21.15.7:0/1156753563 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7f2bb4004000 con 0x7f2bc0101960 2024-09-21T04:36:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.853+0000 7f2bbdffb640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bb007f4c0 0x7f2bb0081980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.853+0000 7f2bbdffb640 1 -- 172.21.15.7:0/1156753563 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f2bb409e7e0 con 0x7f2bc0101960 2024-09-21T04:36:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.853+0000 7f2bbffff640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bb007f4c0 0x7f2bb0081980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.854+0000 7f2bbffff640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bb007f4c0 0x7f2bb0081980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0096c0 tx=0x7f2bac005a20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:45.862+0000 7f2bbdffb640 1 -- 172.21.15.7:0/1156753563 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f2bb4069bf0 con 0x7f2bc0101960 2024-09-21T04:36:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.145+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2bc0062630 con 0x7f2bb007f4c0 2024-09-21T04:36:46.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.160+0000 7f2bbdffb640 1 -- 172.21.15.7:0/1156753563 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f2bc0062630 con 0x7f2bb007f4c0 2024-09-21T04:36:46.163 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (8m) 41s ago 9m 24.2M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (114s) 41s ago 9m 8544k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (91s) 43s ago 9m 8372k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (9m) 41s ago 9m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (9m) 43s ago 9m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (8m) 41s ago 9m 82.9M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (4m) 41s ago 4m 242M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (4m) 41s ago 4m 278M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (4m) 43s ago 4m 25.7M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (4m) 43s ago 4m 30.7M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (60s) 41s ago 11m 490M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 starting - - - - 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (11m) 41s ago 11m 57.6M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:36:46.164 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (8m) 43s ago 8m 48.5M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (9m) 41s ago 9m 16.1M - 1.5.0 0da6a335fe13 b4a8255a594a 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (8m) 43s ago 8m 16.6M - 1.5.0 0da6a335fe13 88246b15ca5d 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 41s ago 7m 174M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (7m) 41s ago 7m 182M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (6m) 41s ago 6m 160M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (6m) 43s ago 6m 208M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (5m) 43s ago 5m 183M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (5m) 43s ago 5m 166M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:36:46.165 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 starting - - - - 2024-09-21T04:36:46.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bb007f4c0 msgr2=0x7f2bb0081980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:46.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bb007f4c0 0x7f2bb0081980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0096c0 tx=0x7f2bac005a20 comp rx=0 tx=0).stop 2024-09-21T04:36:46.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bc0101960 msgr2=0x7f2bc01952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:46.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bc0101960 0x7f2bc01952a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400e940 tx=0x7f2bb400ee00 comp rx=0 tx=0).stop 2024-09-21T04:36:46.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 shutdown_connections 2024-09-21T04:36:46.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bb007f4c0 0x7f2bb0081980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc0102b60 0x7f2bc01957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 --2- 172.21.15.7:0/1156753563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bc0101960 0x7f2bc01952a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 >> 172.21.15.7:0/1156753563 conn(0x7f2bc00fd0f0 msgr2=0x7f2bc0106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 shutdown_connections 2024-09-21T04:36:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.163+0000 7f2bc718a640 1 -- 172.21.15.7:0/1156753563 wait complete. 2024-09-21T04:36:46.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.259+0000 7feb988ee640 1 -- 172.21.15.7:0/3324705163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 msgr2=0x7feb94101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:46.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.259+0000 7feb988ee640 1 --2- 172.21.15.7:0/3324705163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 0x7feb94101ee0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009880 tx=0x7feb7c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.259+0000 7feb988ee640 1 -- 172.21.15.7:0/3324705163 shutdown_connections 2024-09-21T04:36:46.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.259+0000 7feb988ee640 1 --2- 172.21.15.7:0/3324705163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 0x7feb94101ee0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.259+0000 7feb988ee640 1 --2- 172.21.15.7:0/3324705163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb94100860 0x7feb94100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.259+0000 7feb988ee640 1 -- 172.21.15.7:0/3324705163 >> 172.21.15.7:0/3324705163 conn(0x7feb940fc010 msgr2=0x7feb940fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:46.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.259+0000 7feb988ee640 1 -- 172.21.15.7:0/3324705163 shutdown_connections 2024-09-21T04:36:46.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.259+0000 7feb988ee640 1 -- 172.21.15.7:0/3324705163 wait complete. 2024-09-21T04:36:46.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.260+0000 7feb988ee640 1 Processor -- start 2024-09-21T04:36:46.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.260+0000 7feb988ee640 1 -- start start 2024-09-21T04:36:46.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.260+0000 7feb988ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb94100860 0x7feb94104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:46.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.260+0000 7feb988ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 0x7feb94102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:46.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.260+0000 7feb988ee640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb941096a0 con 0x7feb94101a60 2024-09-21T04:36:46.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.260+0000 7feb988ee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb94109520 con 0x7feb94100860 2024-09-21T04:36:46.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.260+0000 7feb8bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 0x7feb94102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb8bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 0x7feb94102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33180/0 (socket says 172.21.15.7:33180) 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb9259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb94100860 0x7feb94104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb8bfff640 1 -- 172.21.15.7:0/3490282619 learned_addr learned my addr 172.21.15.7:0/3490282619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb9259c640 1 -- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 msgr2=0x7feb94102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb9259c640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 0x7feb94102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb9259c640 1 -- 172.21.15.7:0/3490282619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb94103380 con 0x7feb94100860 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb9259c640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb94100860 0x7feb94104780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb8400e940 tx=0x7feb8400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb8b7fe640 1 -- 172.21.15.7:0/3490282619 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feb8400cac0 con 0x7feb94100860 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb94103670 con 0x7feb94100860 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.261+0000 7feb8b7fe640 1 -- 172.21.15.7:0/3490282619 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feb8400cc60 con 0x7feb94100860 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.262+0000 7feb8b7fe640 1 -- 172.21.15.7:0/3490282619 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feb84003df0 con 0x7feb94100860 2024-09-21T04:36:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.262+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb94103c00 con 0x7feb94100860 2024-09-21T04:36:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.263+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb94100c60 con 0x7feb94100860 2024-09-21T04:36:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.263+0000 7feb8b7fe640 1 -- 172.21.15.7:0/3490282619 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7feb84004000 con 0x7feb94100860 2024-09-21T04:36:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.264+0000 7feb8b7fe640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7feb8007b040 0x7feb8007d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:46.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.264+0000 7feb8b7fe640 1 -- 172.21.15.7:0/3490282619 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7feb8409e7a0 con 0x7feb94100860 2024-09-21T04:36:46.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.264+0000 7feb8bfff640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7feb8007b040 0x7feb8007d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:46.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.265+0000 7feb8bfff640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7feb8007b040 0x7feb8007d500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0096c0 tx=0x7feb7c005a20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:46.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.269+0000 7feb8b7fe640 1 -- 172.21.15.7:0/3490282619 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7feb84069b30 con 0x7feb94100860 2024-09-21T04:36:46.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:46 smithi007 ceph-mon[27954]: from='client.24561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:46.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:46 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:46.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:46 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:46.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:46 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:36:46.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:46 smithi007 ceph-mon[27954]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:46.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:46 smithi007 ceph-mon[27954]: pgmap v27: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 420 op/s 2024-09-21T04:36:46.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:46 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:46.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:46 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:46 smithi053 ceph-mon[35554]: from='client.24561 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:46 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:46 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:46 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:36:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:46 smithi053 ceph-mon[35554]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:46 smithi053 ceph-mon[35554]: pgmap v27: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 420 op/s 2024-09-21T04:36:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:46 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:46 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.708+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7feb9419f600 con 0x7feb94100860 2024-09-21T04:36:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.709+0000 7feb8b7fe640 1 -- 172.21.15.7:0/3490282619 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7feb8406ce20 con 0x7feb94100860 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:36:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:36:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:36:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:36:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 12, 2024-09-21T04:36:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:36:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:36:46.713 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:36:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7feb8007b040 msgr2=0x7feb8007d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7feb8007b040 0x7feb8007d500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0096c0 tx=0x7feb7c005a20 comp rx=0 tx=0).stop 2024-09-21T04:36:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb94100860 msgr2=0x7feb94104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb94100860 0x7feb94104780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feb8400e940 tx=0x7feb8400ee00 comp rx=0 tx=0).stop 2024-09-21T04:36:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 shutdown_connections 2024-09-21T04:36:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7feb8007b040 0x7feb8007d500 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb94101a60 0x7feb94102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 --2- 172.21.15.7:0/3490282619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb94100860 0x7feb94104780 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 >> 172.21.15.7:0/3490282619 conn(0x7feb940fc010 msgr2=0x7feb94069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 shutdown_connections 2024-09-21T04:36:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.712+0000 7feb988ee640 1 -- 172.21.15.7:0/3490282619 wait complete. 2024-09-21T04:36:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.827+0000 7f9d88032640 1 -- 172.21.15.7:0/2684628309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 msgr2=0x7f9d80100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.827+0000 7f9d88032640 1 --2- 172.21.15.7:0/2684628309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 0x7f9d80100c60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009880 tx=0x7f9d7c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.827+0000 7f9d88032640 1 -- 172.21.15.7:0/2684628309 shutdown_connections 2024-09-21T04:36:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.827+0000 7f9d88032640 1 --2- 172.21.15.7:0/2684628309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d80101a60 0x7f9d80101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.827+0000 7f9d88032640 1 --2- 172.21.15.7:0/2684628309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 0x7f9d80100c60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.827+0000 7f9d88032640 1 -- 172.21.15.7:0/2684628309 >> 172.21.15.7:0/2684628309 conn(0x7f9d800fc010 msgr2=0x7f9d800fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.827+0000 7f9d88032640 1 -- 172.21.15.7:0/2684628309 shutdown_connections 2024-09-21T04:36:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.828+0000 7f9d88032640 1 -- 172.21.15.7:0/2684628309 wait complete. 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.828+0000 7f9d88032640 1 Processor -- start 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.828+0000 7f9d88032640 1 -- start start 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d88032640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 0x7f9d80104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d88032640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d80101a60 0x7f9d80102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d88032640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801096a0 con 0x7f9d80100860 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d88032640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80109520 con 0x7f9d80101a60 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d85dce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 0x7f9d80104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d855cd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d80101a60 0x7f9d80102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d85dce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 0x7f9d80104780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33200/0 (socket says 172.21.15.7:33200) 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d85dce640 1 -- 172.21.15.7:0/2008031920 learned_addr learned my addr 172.21.15.7:0/2008031920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d855cd640 1 -- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 msgr2=0x7f9d80104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d855cd640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 0x7f9d80104780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d855cd640 1 -- 172.21.15.7:0/2008031920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d80103380 con 0x7f9d80101a60 2024-09-21T04:36:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.829+0000 7f9d855cd640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d80101a60 0x7f9d80102e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000e970 tx=0x7f9d7000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.830+0000 7f9d76ffd640 1 -- 172.21.15.7:0/2008031920 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9d7000cac0 con 0x7f9d80101a60 2024-09-21T04:36:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.830+0000 7f9d76ffd640 1 -- 172.21.15.7:0/2008031920 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9d7000cc60 con 0x7f9d80101a60 2024-09-21T04:36:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.830+0000 7f9d76ffd640 1 -- 172.21.15.7:0/2008031920 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9d700047a0 con 0x7f9d80101a60 2024-09-21T04:36:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.830+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d80103670 con 0x7f9d80101a60 2024-09-21T04:36:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.830+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d80103c00 con 0x7f9d80101a60 2024-09-21T04:36:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.831+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d80100c60 con 0x7f9d80101a60 2024-09-21T04:36:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.832+0000 7f9d76ffd640 1 -- 172.21.15.7:0/2008031920 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) ==== 103420+0+0 (secure 0 0 0) 0x7f9d70004a50 con 0x7f9d80101a60 2024-09-21T04:36:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.832+0000 7f9d76ffd640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9d6c07b050 0x7f9d6c07d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:36:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.832+0000 7f9d76ffd640 1 -- 172.21.15.7:0/2008031920 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f9d7009e770 con 0x7f9d80101a60 2024-09-21T04:36:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.833+0000 7f9d85dce640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9d6c07b050 0x7f9d6c07d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:36:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.833+0000 7f9d85dce640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9d6c07b050 0x7f9d6c07d510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c000c00 tx=0x7f9d7c005a20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:36:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:46.837+0000 7f9d76ffd640 1 -- 172.21.15.7:0/2008031920 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f9d70069b00 con 0x7f9d80101a60 2024-09-21T04:36:47.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.132+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9d8019f5e0 con 0x7f9d6c07b050 2024-09-21T04:36:47.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.134+0000 7f9d76ffd640 1 -- 172.21.15.7:0/2008031920 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+321 (secure 0 0 0) 0x7f9d8019f5e0 con 0x7f9d6c07b050 2024-09-21T04:36:47.136 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:36:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:36:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:36:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:36:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-09-21T04:36:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "", 2024-09-21T04:36:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mgr daemons", 2024-09-21T04:36:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:36:47.137 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9d6c07b050 msgr2=0x7f9d6c07d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9d6c07b050 0x7f9d6c07d510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c000c00 tx=0x7f9d7c005a20 comp rx=0 tx=0).stop 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d80101a60 msgr2=0x7f9d80102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d80101a60 0x7f9d80102e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000e970 tx=0x7f9d7000ee30 comp rx=0 tx=0).stop 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 shutdown_connections 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9d6c07b050 0x7f9d6c07d510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d80101a60 0x7f9d80102e40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 --2- 172.21.15.7:0/2008031920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d80100860 0x7f9d80104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:36:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 >> 172.21.15.7:0/2008031920 conn(0x7f9d800fc010 msgr2=0x7f9d80069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:36:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 shutdown_connections 2024-09-21T04:36:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:36:47.136+0000 7f9d88032640 1 -- 172.21.15.7:0/2008031920 wait complete. 2024-09-21T04:36:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:47 smithi007 ceph-mon[27954]: from='client.24573 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:47.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:47 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3490282619' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:47.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:47 smithi053 ceph-mon[35554]: from='client.24573 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:47.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:47 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3490282619' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:48.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:48 smithi053 ceph-mon[35554]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:48.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:48 smithi053 ceph-mon[35554]: pgmap v28: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 393 op/s 2024-09-21T04:36:48.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:48 smithi007 ceph-mon[27954]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:36:48.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:48 smithi007 ceph-mon[27954]: pgmap v28: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 393 op/s 2024-09-21T04:36:49.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:49 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:49.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:49 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:49.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:49 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:49.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:49 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:49.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:49 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:49.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:49 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:49.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:49 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:49.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:49 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:50.917 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:50 smithi007 ceph-mon[27954]: pgmap v29: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 522 op/s 2024-09-21T04:36:50.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:50 smithi053 ceph-mon[35554]: pgmap v29: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 522 op/s 2024-09-21T04:36:51.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:51 smithi053 ceph-mon[35554]: Standby manager daemon smithi053.lnadao restarted 2024-09-21T04:36:51.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:51 smithi053 ceph-mon[35554]: Standby manager daemon smithi053.lnadao started 2024-09-21T04:36:51.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:51 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/1724664973' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/crt"}]: dispatch 2024-09-21T04:36:51.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:51 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/1724664973' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:36:51.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:51 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/1724664973' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/key"}]: dispatch 2024-09-21T04:36:51.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:51 smithi053 ceph-mon[35554]: from='mgr.? 172.21.15.53:0/1724664973' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:36:51.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:51 smithi007 ceph-mon[27954]: Standby manager daemon smithi053.lnadao restarted 2024-09-21T04:36:51.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:51 smithi007 ceph-mon[27954]: Standby manager daemon smithi053.lnadao started 2024-09-21T04:36:51.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:51 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/1724664973' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/crt"}]: dispatch 2024-09-21T04:36:51.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:51 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/1724664973' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-21T04:36:51.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:51 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/1724664973' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi053.lnadao/key"}]: dispatch 2024-09-21T04:36:51.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:51 smithi007 ceph-mon[27954]: from='mgr.? 172.21.15.53:0/1724664973' entity='mgr.smithi053.lnadao' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-21T04:36:52.933 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:52 smithi053 ceph-mon[35554]: mgrmap e32: smithi007.huohfl(active, since 53s), standbys: smithi053.lnadao 2024-09-21T04:36:52.933 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:52 smithi053 ceph-mon[35554]: pgmap v30: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-09-21T04:36:52.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:52 smithi007 ceph-mon[27954]: mgrmap e32: smithi007.huohfl(active, since 53s), standbys: smithi053.lnadao 2024-09-21T04:36:52.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:52 smithi007 ceph-mon[27954]: pgmap v30: 65 pgs: 65 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-09-21T04:36:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:54 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:54 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:54 smithi053 ceph-mon[35554]: pgmap v31: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-21T04:36:54.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:54 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:54.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:54 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:54.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:54 smithi007 ceph-mon[27954]: pgmap v31: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 284 op/s 2024-09-21T04:36:57.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:56 smithi053 ceph-mon[35554]: pgmap v32: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 265 op/s 2024-09-21T04:36:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:56 smithi007 ceph-mon[27954]: pgmap v32: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 265 op/s 2024-09-21T04:36:57.795 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-09-21T04:36:57.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-21T04:36:58.122 DEBUG:teuthology.parallel:result is None 2024-09-21T04:36:58.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: Upgrade: Setting container_image for all mgr 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.huohfl"}]: dispatch 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.huohfl"}]: dispatch 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.huohfl"}]': finished 2024-09-21T04:36:58.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi053.lnadao"}]: dispatch 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi053.lnadao"}]: dispatch 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi053.lnadao"}]': finished 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: Upgrade: Setting container_image for all rgw 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: Upgrade: Setting container_image for all rbd-mirror 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: Upgrade: Setting container_image for all iscsi 2024-09-21T04:36:58.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: Upgrade: Setting container_image for all nfs 2024-09-21T04:36:58.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: Upgrade: Setting container_image for all nvmeof 2024-09-21T04:36:58.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:58 smithi007 ceph-mon[27954]: pgmap v33: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 233 op/s 2024-09-21T04:36:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: Upgrade: Setting container_image for all mgr 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.huohfl"}]: dispatch 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.huohfl"}]: dispatch 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.huohfl"}]': finished 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi053.lnadao"}]: dispatch 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi053.lnadao"}]: dispatch 2024-09-21T04:36:58.446 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi053.lnadao"}]': finished 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: Upgrade: Setting container_image for all rgw 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: Upgrade: Setting container_image for all rbd-mirror 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: Upgrade: Setting container_image for all iscsi 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.447 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: Upgrade: Setting container_image for all nfs 2024-09-21T04:36:58.448 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.448 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:36:58.448 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: Upgrade: Setting container_image for all nvmeof 2024-09-21T04:36:58.448 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:36:58.448 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:58 smithi053 ceph-mon[35554]: pgmap v33: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 233 op/s 2024-09-21T04:36:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:59 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:36:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:59 smithi007 ceph-mon[27954]: Upgrade: Updating node-exporter.smithi007 (1/2) 2024-09-21T04:36:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:36:59 smithi007 ceph-mon[27954]: Deploying daemon node-exporter.smithi007 on smithi007 2024-09-21T04:36:59.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:59 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:36:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:59 smithi053 ceph-mon[35554]: Upgrade: Updating node-exporter.smithi007 (1/2) 2024-09-21T04:36:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:36:59 smithi053 ceph-mon[35554]: Deploying daemon node-exporter.smithi007 on smithi007 2024-09-21T04:37:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:00 smithi007 ceph-mon[27954]: pgmap v34: 65 pgs: 65 active+clean; 282 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 304 op/s 2024-09-21T04:37:00.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:00 smithi053 ceph-mon[35554]: pgmap v34: 65 pgs: 65 active+clean; 282 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 304 op/s 2024-09-21T04:37:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:02 smithi007 ceph-mon[27954]: pgmap v35: 65 pgs: 65 active+clean; 282 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-21T04:37:03.153 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:02 smithi053 ceph-mon[35554]: pgmap v35: 65 pgs: 65 active+clean; 282 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-21T04:37:04.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:04 smithi007 ceph-mon[27954]: pgmap v36: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2024-09-21T04:37:05.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:04 smithi053 ceph-mon[35554]: pgmap v36: 65 pgs: 65 active+clean; 281 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2024-09-21T04:37:06.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:06 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:06.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:06 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:06.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:06 smithi053 ceph-mon[35554]: pgmap v37: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-09-21T04:37:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:06 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:06 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:06 smithi007 ceph-mon[27954]: pgmap v37: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-09-21T04:37:07.358 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:07 smithi007 ceph-mon[27954]: Upgrade: Updating node-exporter.smithi053 (2/2) 2024-09-21T04:37:07.358 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:07 smithi007 ceph-mon[27954]: Deploying daemon node-exporter.smithi053 on smithi053 2024-09-21T04:37:07.360 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-09-21T04:37:07.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T04:37:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:07 smithi053 ceph-mon[35554]: Upgrade: Updating node-exporter.smithi053 (2/2) 2024-09-21T04:37:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:07 smithi053 ceph-mon[35554]: Deploying daemon node-exporter.smithi053 on smithi053 2024-09-21T04:37:07.706 DEBUG:teuthology.parallel:result is None 2024-09-21T04:37:07.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T04:37:07.751 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T04:37:07.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-21T04:37:07.786 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-21T04:37:07.786 DEBUG:teuthology.parallel:result is None 2024-09-21T04:37:08.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:08 smithi053 ceph-mon[35554]: pgmap v38: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-09-21T04:37:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:08 smithi007 ceph-mon[27954]: pgmap v38: 65 pgs: 65 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-09-21T04:37:11.072 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:10 smithi053 ceph-mon[35554]: pgmap v39: 65 pgs: 65 active+clean; 278 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2024-09-21T04:37:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:10 smithi007 ceph-mon[27954]: pgmap v39: 65 pgs: 65 active+clean; 278 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2024-09-21T04:37:12.488 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:12 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:12.489 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:12 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:12.489 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:12 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:37:12.489 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:12 smithi053 ceph-mon[35554]: pgmap v40: 65 pgs: 65 active+clean; 278 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 892 KiB/s rd, 863 KiB/s wr, 140 op/s 2024-09-21T04:37:12.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:12 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:12.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:12 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:12.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:12 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:37:12.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:12 smithi007 ceph-mon[27954]: pgmap v40: 65 pgs: 65 active+clean; 278 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 892 KiB/s rd, 863 KiB/s wr, 140 op/s 2024-09-21T04:37:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:13 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:37:13.859 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:13 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:37:14.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:14 smithi053 ceph-mon[35554]: pgmap v41: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 892 KiB/s rd, 863 KiB/s wr, 149 op/s 2024-09-21T04:37:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:14 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:14 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:14 smithi007 ceph-mon[27954]: pgmap v41: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 892 KiB/s rd, 863 KiB/s wr, 149 op/s 2024-09-21T04:37:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:14 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:14 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:16.292 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:15 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:16.292 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:15 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:16.374 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:15 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:16.374 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:15 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:16.979 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:16 smithi053 ceph-mon[35554]: pgmap v42: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 794 KiB/s rd, 668 KiB/s wr, 114 op/s 2024-09-21T04:37:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.237+0000 7f899e7a3640 1 -- 172.21.15.7:0/2719635439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 msgr2=0x7f89981047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.237+0000 7f899e7a3640 1 --2- 172.21.15.7:0/2719635439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 0x7f89981047e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f89880098b0 tx=0x7f898802a4d0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.240 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:16 smithi007 ceph-mon[27954]: pgmap v42: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 794 KiB/s rd, 668 KiB/s wr, 114 op/s 2024-09-21T04:37:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.237+0000 7f899e7a3640 1 -- 172.21.15.7:0/2719635439 shutdown_connections 2024-09-21T04:37:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.237+0000 7f899e7a3640 1 --2- 172.21.15.7:0/2719635439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8998104d20 0x7f8998107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.237+0000 7f899e7a3640 1 --2- 172.21.15.7:0/2719635439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 0x7f89981047e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.237+0000 7f899e7a3640 1 -- 172.21.15.7:0/2719635439 >> 172.21.15.7:0/2719635439 conn(0x7f89980fc010 msgr2=0x7f89980fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.237+0000 7f899e7a3640 1 -- 172.21.15.7:0/2719635439 shutdown_connections 2024-09-21T04:37:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.237+0000 7f899e7a3640 1 -- 172.21.15.7:0/2719635439 wait complete. 2024-09-21T04:37:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.238+0000 7f899e7a3640 1 Processor -- start 2024-09-21T04:37:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.238+0000 7f899e7a3640 1 -- start start 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.238+0000 7f899e7a3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 0x7f8998197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.238+0000 7f899e7a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8998104d20 0x7f8998197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.238+0000 7f899e7a3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f899810a7e0 con 0x7f89981023f0 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.238+0000 7f899e7a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f899810a660 con 0x7f8998104d20 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f8997fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 0x7f8998197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f89977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8998104d20 0x7f8998197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f8997fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 0x7f8998197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42234/0 (socket says 172.21.15.7:42234) 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f8997fff640 1 -- 172.21.15.7:0/3494847088 learned_addr learned my addr 172.21.15.7:0/3494847088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f89977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8998104d20 0x7f8998197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:48590/0 (socket says 172.21.15.7:48590) 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f89977fe640 1 -- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 msgr2=0x7f8998197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f89977fe640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 0x7f8998197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f89977fe640 1 -- 172.21.15.7:0/3494847088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89981980c0 con 0x7f8998104d20 2024-09-21T04:37:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f8997fff640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 0x7f8998197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:37:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f89977fe640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8998104d20 0x7f8998197a40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e940 tx=0x7f898c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f89957fa640 1 -- 172.21.15.7:0/3494847088 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f898c00cac0 con 0x7f8998104d20 2024-09-21T04:37:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819cb00 con 0x7f8998104d20 2024-09-21T04:37:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.239+0000 7f89957fa640 1 -- 172.21.15.7:0/3494847088 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f898c00cc60 con 0x7f8998104d20 2024-09-21T04:37:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.240+0000 7f89957fa640 1 -- 172.21.15.7:0/3494847088 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f898c0047a0 con 0x7f8998104d20 2024-09-21T04:37:17.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.240+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899819d090 con 0x7f8998104d20 2024-09-21T04:37:17.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.241+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89981027d0 con 0x7f8998104d20 2024-09-21T04:37:17.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.242+0000 7f89957fa640 1 -- 172.21.15.7:0/3494847088 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 32) ==== 103420+0+0 (secure 0 0 0) 0x7f898c004a50 con 0x7f8998104d20 2024-09-21T04:37:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.242+0000 7f89957fa640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898007b050 0x7f898007d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.243+0000 7f89957fa640 1 -- 172.21.15.7:0/3494847088 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f898c09e680 con 0x7f8998104d20 2024-09-21T04:37:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.243+0000 7f8997fff640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898007b050 0x7f898007d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.243+0000 7f8997fff640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898007b050 0x7f898007d510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f89880096c0 tx=0x7f8988005a20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:17.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.247+0000 7f89957fa640 1 -- 172.21.15.7:0/3494847088 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f898c069ab0 con 0x7f8998104d20 2024-09-21T04:37:17.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.478+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8998105680 con 0x7f898007b050 2024-09-21T04:37:17.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.480+0000 7f89957fa640 1 -- 172.21.15.7:0/3494847088 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+370 (secure 0 0 0) 0x7f8998105680 con 0x7f898007b050 2024-09-21T04:37:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898007b050 msgr2=0x7f898007d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898007b050 0x7f898007d510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f89880096c0 tx=0x7f8988005a20 comp rx=0 tx=0).stop 2024-09-21T04:37:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8998104d20 msgr2=0x7f8998197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8998104d20 0x7f8998197a40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e940 tx=0x7f898c00ee00 comp rx=0 tx=0).stop 2024-09-21T04:37:17.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 shutdown_connections 2024-09-21T04:37:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898007b050 0x7f898007d510 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8998104d20 0x7f8998197a40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 --2- 172.21.15.7:0/3494847088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89981023f0 0x7f8998197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 >> 172.21.15.7:0/3494847088 conn(0x7f89980fc010 msgr2=0x7f89980fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.482+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 shutdown_connections 2024-09-21T04:37:17.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.483+0000 7f899e7a3640 1 -- 172.21.15.7:0/3494847088 wait complete. 2024-09-21T04:37:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:37:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.570+0000 7f58e366f640 1 -- 172.21.15.7:0/2121700366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 msgr2=0x7f58dc101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.570+0000 7f58e366f640 1 --2- 172.21.15.7:0/2121700366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 0x7f58dc101ee0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009880 tx=0x7f58cc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.570+0000 7f58e366f640 1 -- 172.21.15.7:0/2121700366 shutdown_connections 2024-09-21T04:37:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.570+0000 7f58e366f640 1 --2- 172.21.15.7:0/2121700366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 0x7f58dc101ee0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.570+0000 7f58e366f640 1 --2- 172.21.15.7:0/2121700366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc100860 0x7f58dc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.570+0000 7f58e366f640 1 -- 172.21.15.7:0/2121700366 >> 172.21.15.7:0/2121700366 conn(0x7f58dc0fc010 msgr2=0x7f58dc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.570+0000 7f58e366f640 1 -- 172.21.15.7:0/2121700366 shutdown_connections 2024-09-21T04:37:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.570+0000 7f58e366f640 1 -- 172.21.15.7:0/2121700366 wait complete. 2024-09-21T04:37:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e366f640 1 Processor -- start 2024-09-21T04:37:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e366f640 1 -- start start 2024-09-21T04:37:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e366f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc100860 0x7f58dc1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e366f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 0x7f58dc196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e366f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1096a0 con 0x7f58dc101a60 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e366f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc109520 con 0x7f58dc100860 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e0c0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 0x7f58dc196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e0c0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 0x7f58dc196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42246/0 (socket says 172.21.15.7:42246) 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e0c0a640 1 -- 172.21.15.7:0/303953284 learned_addr learned my addr 172.21.15.7:0/303953284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.571+0000 7f58e140b640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc100860 0x7f58dc1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58e0c0a640 1 -- 172.21.15.7:0/303953284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc100860 msgr2=0x7f58dc1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58e0c0a640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc100860 0x7f58dc1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58e0c0a640 1 -- 172.21.15.7:0/303953284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc196fa0 con 0x7f58dc101a60 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58e140b640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc100860 0x7f58dc1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:37:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58e0c0a640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 0x7f58dc196920 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f58cc006010 tx=0x7f58cc004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58d27fc640 1 -- 172.21.15.7:0/303953284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f58cc035070 con 0x7f58dc101a60 2024-09-21T04:37:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc069ca0 con 0x7f58dc101a60 2024-09-21T04:37:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58d27fc640 1 -- 172.21.15.7:0/303953284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58cc0035f0 con 0x7f58dc101a60 2024-09-21T04:37:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58d27fc640 1 -- 172.21.15.7:0/303953284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f58cc031810 con 0x7f58dc101a60 2024-09-21T04:37:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.572+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc06a230 con 0x7f58dc101a60 2024-09-21T04:37:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.573+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc109e80 con 0x7f58dc101a60 2024-09-21T04:37:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.574+0000 7f58d27fc640 1 -- 172.21.15.7:0/303953284 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) ==== 103420+0+0 (secure 0 0 0) 0x7f58cc0037a0 con 0x7f58dc101a60 2024-09-21T04:37:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.574+0000 7f58d27fc640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f58c807b050 0x7f58c807d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.575+0000 7f58d27fc640 1 -- 172.21.15.7:0/303953284 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f58cc03e070 con 0x7f58dc101a60 2024-09-21T04:37:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.575+0000 7f58e140b640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f58c807b050 0x7f58c807d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.575+0000 7f58e140b640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f58c807b050 0x7f58c807d510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f58dc1018c0 tx=0x7f58d8005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.579+0000 7f58d27fc640 1 -- 172.21.15.7:0/303953284 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f58cc08bc00 con 0x7f58dc101a60 2024-09-21T04:37:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.789+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f58dc190520 con 0x7f58c807b050 2024-09-21T04:37:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.791+0000 7f58d27fc640 1 -- 172.21.15.7:0/303953284 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+370 (secure 0 0 0) 0x7f58d800f480 con 0x7f58c807b050 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f58c807b050 msgr2=0x7f58c807d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f58c807b050 0x7f58c807d510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f58dc1018c0 tx=0x7f58d8005f90 comp rx=0 tx=0).stop 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 msgr2=0x7f58dc196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 0x7f58dc196920 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f58cc006010 tx=0x7f58cc004570 comp rx=0 tx=0).stop 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 shutdown_connections 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f58c807b050 0x7f58c807d510 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58dc101a60 0x7f58dc196920 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 --2- 172.21.15.7:0/303953284 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58dc100860 0x7f58dc1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 >> 172.21.15.7:0/303953284 conn(0x7f58dc0fc010 msgr2=0x7f58dc104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 shutdown_connections 2024-09-21T04:37:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.793+0000 7f58e366f640 1 -- 172.21.15.7:0/303953284 wait complete. 2024-09-21T04:37:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.879+0000 7fa06b39f640 1 -- 172.21.15.7:0/3355933759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 msgr2=0x7fa0641047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.879+0000 7fa06b39f640 1 --2- 172.21.15.7:0/3355933759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 0x7fa0641047e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa0600098b0 tx=0x7fa06002a4d0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.879+0000 7fa06b39f640 1 -- 172.21.15.7:0/3355933759 shutdown_connections 2024-09-21T04:37:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.879+0000 7fa06b39f640 1 --2- 172.21.15.7:0/3355933759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa064104d20 0x7fa064107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.879+0000 7fa06b39f640 1 --2- 172.21.15.7:0/3355933759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 0x7fa0641047e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.879+0000 7fa06b39f640 1 -- 172.21.15.7:0/3355933759 >> 172.21.15.7:0/3355933759 conn(0x7fa0640fc010 msgr2=0x7fa0640fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.880+0000 7fa06b39f640 1 -- 172.21.15.7:0/3355933759 shutdown_connections 2024-09-21T04:37:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.880+0000 7fa06b39f640 1 -- 172.21.15.7:0/3355933759 wait complete. 2024-09-21T04:37:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.880+0000 7fa06b39f640 1 Processor -- start 2024-09-21T04:37:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.880+0000 7fa06b39f640 1 -- start start 2024-09-21T04:37:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.880+0000 7fa06b39f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 0x7fa064199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.880+0000 7fa06b39f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa064104d20 0x7fa064199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.880+0000 7fa06b39f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa06410a7e0 con 0x7fa0641023f0 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.880+0000 7fa06b39f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa06410a660 con 0x7fa064104d20 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06913b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 0x7fa064199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06913b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 0x7fa064199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42260/0 (socket says 172.21.15.7:42260) 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06913b640 1 -- 172.21.15.7:0/2410260007 learned_addr learned my addr 172.21.15.7:0/2410260007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06893a640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa064104d20 0x7fa064199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06913b640 1 -- 172.21.15.7:0/2410260007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa064104d20 msgr2=0x7fa064199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06913b640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa064104d20 0x7fa064199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06913b640 1 -- 172.21.15.7:0/2410260007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06419a300 con 0x7fa0641023f0 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06913b640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 0x7fa064199740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa0600099d0 tx=0x7fa0600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa05a7fc640 1 -- 172.21.15.7:0/2410260007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa06003f070 con 0x7fa0641023f0 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06419ece0 con 0x7fa0641023f0 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa05a7fc640 1 -- 172.21.15.7:0/2410260007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa060004a40 con 0x7fa0641023f0 2024-09-21T04:37:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.881+0000 7fa05a7fc640 1 -- 172.21.15.7:0/2410260007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa060035030 con 0x7fa0641023f0 2024-09-21T04:37:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.882+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06419f1c0 con 0x7fa0641023f0 2024-09-21T04:37:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.883+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0641027d0 con 0x7fa0641023f0 2024-09-21T04:37:17.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.883+0000 7fa05a7fc640 1 -- 172.21.15.7:0/2410260007 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) ==== 103420+0+0 (secure 0 0 0) 0x7fa060042070 con 0x7fa0641023f0 2024-09-21T04:37:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.884+0000 7fa05a7fc640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa05007b050 0x7fa05007d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.884+0000 7fa05a7fc640 1 -- 172.21.15.7:0/2410260007 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fa0600bedc0 con 0x7fa0641023f0 2024-09-21T04:37:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.884+0000 7fa06893a640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa05007b050 0x7fa05007d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:17.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.885+0000 7fa06893a640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa05007b050 0x7fa05007d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa06419abf0 tx=0x7fa05400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:17.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:17.889+0000 7fa05a7fc640 1 -- 172.21.15.7:0/2410260007 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fa06008a1f0 con 0x7fa0641023f0 2024-09-21T04:37:18.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.097+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa0640626f0 con 0x7fa05007b050 2024-09-21T04:37:18.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.106+0000 7fa05a7fc640 1 -- 172.21.15.7:0/2410260007 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fa0640626f0 con 0x7fa05007b050 2024-09-21T04:37:18.109 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:37:18.109 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (8m) 0s ago 10m 23.8M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:37:18.109 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (2m) 0s ago 10m 8644k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (2m) 3s ago 9m 8514k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (10m) 0s ago 10m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (9m) 3s ago 9m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (8m) 0s ago 10m 81.6M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (4m) 0s ago 4m 156M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (4m) 0s ago 4m 127M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (4m) 3s ago 4m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (4m) 3s ago 4m 30.9M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (92s) 0s ago 11m 514M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (33s) 3s ago 9m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (11m) 0s ago 11m 56.0M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (9m) 3s ago 9m 47.3M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (13s) 0s ago 10m 7889k - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:37:18.110 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (7s) 3s ago 9m 3405k - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:37:18.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (8m) 0s ago 8m 211M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:37:18.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (7m) 0s ago 7m 213M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:37:18.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (7m) 0s ago 7m 193M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:37:18.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (6m) 3s ago 6m 241M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:37:18.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (6m) 3s ago 6m 210M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:37:18.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (5m) 3s ago 5m 181M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:37:18.111 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (42s) 0s ago 9m 51.1M - 2.43.0 a07b618ecd1d a3e261b55f06 2024-09-21T04:37:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa05007b050 msgr2=0x7fa05007d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa05007b050 0x7fa05007d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa06419abf0 tx=0x7fa05400b390 comp rx=0 tx=0).stop 2024-09-21T04:37:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 msgr2=0x7fa064199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 0x7fa064199740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa0600099d0 tx=0x7fa0600046c0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 shutdown_connections 2024-09-21T04:37:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa05007b050 0x7fa05007d510 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa064104d20 0x7fa064199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 --2- 172.21.15.7:0/2410260007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0641023f0 0x7fa064199740 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.109+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 >> 172.21.15.7:0/2410260007 conn(0x7fa0640fc010 msgr2=0x7fa064101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.110+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 shutdown_connections 2024-09-21T04:37:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.110+0000 7fa06b39f640 1 -- 172.21.15.7:0/2410260007 wait complete. 2024-09-21T04:37:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2365296384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 msgr2=0x7f2b18104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 --2- 172.21.15.7:0/2365296384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 0x7f2b18104860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009880 tx=0x7f2b0c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2365296384 shutdown_connections 2024-09-21T04:37:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 --2- 172.21.15.7:0/2365296384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b18104da0 0x7f2b18107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 --2- 172.21.15.7:0/2365296384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 0x7f2b18104860 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2365296384 >> 172.21.15.7:0/2365296384 conn(0x7f2b180fc010 msgr2=0x7f2b180fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2365296384 shutdown_connections 2024-09-21T04:37:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2365296384 wait complete. 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 Processor -- start 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.194+0000 7f2b1ed0b640 1 -- start start 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1ed0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 0x7f2b18197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1ed0b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b18104da0 0x7f2b18197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1ed0b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1810a7b0 con 0x7f2b18102470 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1ed0b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1810a630 con 0x7f2b18104da0 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1caa7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 0x7f2b18197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1caa7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 0x7f2b18197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42286/0 (socket says 172.21.15.7:42286) 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1caa7640 1 -- 172.21.15.7:0/2640077497 learned_addr learned my addr 172.21.15.7:0/2640077497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b17fff640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b18104da0 0x7f2b18197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1caa7640 1 -- 172.21.15.7:0/2640077497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b18104da0 msgr2=0x7f2b18197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1caa7640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b18104da0 0x7f2b18197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1caa7640 1 -- 172.21.15.7:0/2640077497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b181980c0 con 0x7f2b18102470 2024-09-21T04:37:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b17fff640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b18104da0 0x7f2b18197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.195+0000 7f2b1caa7640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 0x7f2b18197500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006010 tx=0x7f2b0c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.196+0000 7f2b15ffb640 1 -- 172.21.15.7:0/2640077497 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2b0c03f070 con 0x7f2b18102470 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.196+0000 7f2b15ffb640 1 -- 172.21.15.7:0/2640077497 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2b0c0049d0 con 0x7f2b18102470 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.196+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1819caa0 con 0x7f2b18102470 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.196+0000 7f2b15ffb640 1 -- 172.21.15.7:0/2640077497 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2b0c035030 con 0x7f2b18102470 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.196+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1819d030 con 0x7f2b18102470 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.197+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b18102850 con 0x7f2b18102470 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.198+0000 7f2b15ffb640 1 -- 172.21.15.7:0/2640077497 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) ==== 103420+0+0 (secure 0 0 0) 0x7f2b0c042070 con 0x7f2b18102470 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.198+0000 7f2b15ffb640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2b0807b0b0 0x7f2b0807d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.198+0000 7f2b15ffb640 1 -- 172.21.15.7:0/2640077497 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f2b0c0bedf0 con 0x7f2b18102470 2024-09-21T04:37:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.198+0000 7f2b17fff640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2b0807b0b0 0x7f2b0807d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.199+0000 7f2b17fff640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2b0807b0b0 0x7f2b0807d570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2b181989b0 tx=0x7f2b04005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:18.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.203+0000 7f2b15ffb640 1 -- 172.21.15.7:0/2640077497 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f2b0c08a2a0 con 0x7f2b18102470 2024-09-21T04:37:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.488+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f2b1819d320 con 0x7f2b18102470 2024-09-21T04:37:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.488+0000 7f2b15ffb640 1 -- 172.21.15.7:0/2640077497 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7f2b0c08f150 con 0x7f2b18102470 2024-09-21T04:37:18.491 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:37:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:37:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 12, 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:37:18.492 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:37:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2b0807b0b0 msgr2=0x7f2b0807d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2b0807b0b0 0x7f2b0807d570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2b181989b0 tx=0x7f2b04005f90 comp rx=0 tx=0).stop 2024-09-21T04:37:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 msgr2=0x7f2b18197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 0x7f2b18197500 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006010 tx=0x7f2b0c004570 comp rx=0 tx=0).stop 2024-09-21T04:37:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 shutdown_connections 2024-09-21T04:37:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2b0807b0b0 0x7f2b0807d570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b18104da0 0x7f2b18197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 --2- 172.21.15.7:0/2640077497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b18102470 0x7f2b18197500 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.490+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 >> 172.21.15.7:0/2640077497 conn(0x7f2b180fc010 msgr2=0x7f2b180fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.491+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 shutdown_connections 2024-09-21T04:37:18.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.491+0000 7f2b1ed0b640 1 -- 172.21.15.7:0/2640077497 wait complete. 2024-09-21T04:37:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.576+0000 7f2d48bf2640 1 -- 172.21.15.7:0/2394917519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 msgr2=0x7f2d44102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.576+0000 7f2d48bf2640 1 --2- 172.21.15.7:0/2394917519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 0x7f2d44102fe0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009880 tx=0x7f2d3802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.576+0000 7f2d48bf2640 1 -- 172.21.15.7:0/2394917519 shutdown_connections 2024-09-21T04:37:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.576+0000 7f2d48bf2640 1 --2- 172.21.15.7:0/2394917519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 0x7f2d44102fe0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.576+0000 7f2d48bf2640 1 --2- 172.21.15.7:0/2394917519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d44101960 0x7f2d44101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.576+0000 7f2d48bf2640 1 -- 172.21.15.7:0/2394917519 >> 172.21.15.7:0/2394917519 conn(0x7f2d440fd0f0 msgr2=0x7f2d440ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.576+0000 7f2d48bf2640 1 -- 172.21.15.7:0/2394917519 shutdown_connections 2024-09-21T04:37:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.577+0000 7f2d48bf2640 1 -- 172.21.15.7:0/2394917519 wait complete. 2024-09-21T04:37:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.577+0000 7f2d48bf2640 1 Processor -- start 2024-09-21T04:37:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.577+0000 7f2d48bf2640 1 -- start start 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.577+0000 7f2d48bf2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d44101960 0x7f2d441974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.577+0000 7f2d48bf2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 0x7f2d44197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.577+0000 7f2d48bf2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4410a7a0 con 0x7f2d44102b60 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.577+0000 7f2d48bf2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4410a620 con 0x7f2d44101960 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d41d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 0x7f2d44197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d41d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 0x7f2d44197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42300/0 (socket says 172.21.15.7:42300) 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d41d9b640 1 -- 172.21.15.7:0/4286484959 learned_addr learned my addr 172.21.15.7:0/4286484959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d4259c640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d44101960 0x7f2d441974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d41d9b640 1 -- 172.21.15.7:0/4286484959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d44101960 msgr2=0x7f2d441974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d41d9b640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d44101960 0x7f2d441974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d41d9b640 1 -- 172.21.15.7:0/4286484959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d44198080 con 0x7f2d44102b60 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d41d9b640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 0x7f2d44197a00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d38006010 tx=0x7f2d380048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d337fe640 1 -- 172.21.15.7:0/4286484959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2d3803f070 con 0x7f2d44102b60 2024-09-21T04:37:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d337fe640 1 -- 172.21.15.7:0/4286484959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d380044d0 con 0x7f2d44102b60 2024-09-21T04:37:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4419ca60 con 0x7f2d44102b60 2024-09-21T04:37:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d337fe640 1 -- 172.21.15.7:0/4286484959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2d38035030 con 0x7f2d44102b60 2024-09-21T04:37:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.578+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4419cff0 con 0x7f2d44102b60 2024-09-21T04:37:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.579+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d44101d60 con 0x7f2d44102b60 2024-09-21T04:37:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.580+0000 7f2d337fe640 1 -- 172.21.15.7:0/4286484959 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) ==== 103420+0+0 (secure 0 0 0) 0x7f2d38042070 con 0x7f2d44102b60 2024-09-21T04:37:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.581+0000 7f2d337fe640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2d2c07b0b0 0x7f2d2c07d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.581+0000 7f2d337fe640 1 -- 172.21.15.7:0/4286484959 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f2d380bee40 con 0x7f2d44102b60 2024-09-21T04:37:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.581+0000 7f2d4259c640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2d2c07b0b0 0x7f2d2c07d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.581+0000 7f2d4259c640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2d2c07b0b0 0x7f2d2c07d570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2d340058c0 tx=0x7f2d3400a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:18.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.585+0000 7f2d337fe640 1 -- 172.21.15.7:0/4286484959 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f2d3808a2f0 con 0x7f2d44102b60 2024-09-21T04:37:18.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: from='client.24585 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: pgmap v43: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 529 KiB/s rd, 449 KiB/s wr, 84 op/s 2024-09-21T04:37:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:18 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: from='client.24585 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: pgmap v43: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 529 KiB/s rd, 449 KiB/s wr, 84 op/s 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:18 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.794+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2d44062630 con 0x7f2d2c07b0b0 2024-09-21T04:37:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.796+0000 7f2d337fe640 1 -- 172.21.15.7:0/4286484959 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+370 (secure 0 0 0) 0x7f2d44062630 con 0x7f2d2c07b0b0 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/2 daemons upgraded", 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading node-exporter daemons", 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:37:18.799 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:37:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2d2c07b0b0 msgr2=0x7f2d2c07d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2d2c07b0b0 0x7f2d2c07d570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2d340058c0 tx=0x7f2d3400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 msgr2=0x7f2d44197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 0x7f2d44197a00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d38006010 tx=0x7f2d380048f0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 shutdown_connections 2024-09-21T04:37:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2d2c07b0b0 0x7f2d2c07d570 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d44102b60 0x7f2d44197a00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 --2- 172.21.15.7:0/4286484959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d44101960 0x7f2d441974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 >> 172.21.15.7:0/4286484959 conn(0x7f2d440fd0f0 msgr2=0x7f2d44106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 shutdown_connections 2024-09-21T04:37:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:18.798+0000 7f2d48bf2640 1 -- 172.21.15.7:0/4286484959 wait complete. 2024-09-21T04:37:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:19 smithi053 ceph-mon[35554]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:19 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2640077497' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:19 smithi053 ceph-mon[35554]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:19 smithi007 ceph-mon[27954]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:19 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2640077497' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:19 smithi007 ceph-mon[27954]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:20.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:20 smithi053 ceph-mon[35554]: pgmap v44: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 530 KiB/s rd, 449 KiB/s wr, 87 op/s 2024-09-21T04:37:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:20 smithi007 ceph-mon[27954]: pgmap v44: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 530 KiB/s rd, 449 KiB/s wr, 87 op/s 2024-09-21T04:37:22.913 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:22 smithi007 ceph-mon[27954]: pgmap v45: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-21T04:37:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:22 smithi053 ceph-mon[35554]: pgmap v45: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-21T04:37:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: pgmap v46: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:24.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:24.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:24.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:24 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: pgmap v46: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-09-21T04:37:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:37:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:37:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:37:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:24 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:26 smithi053 ceph-mon[35554]: Upgrade: Updating prometheus.smithi007 2024-09-21T04:37:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:26 smithi053 ceph-mon[35554]: Deploying daemon prometheus.smithi007 on smithi007 2024-09-21T04:37:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:26 smithi053 ceph-mon[35554]: pgmap v47: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-09-21T04:37:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:26 smithi007 ceph-mon[27954]: Upgrade: Updating prometheus.smithi007 2024-09-21T04:37:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:26 smithi007 ceph-mon[27954]: Deploying daemon prometheus.smithi007 on smithi007 2024-09-21T04:37:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:26 smithi007 ceph-mon[27954]: pgmap v47: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-09-21T04:37:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:28 smithi053 ceph-mon[35554]: pgmap v48: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-21T04:37:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:28 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:37:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:28 smithi007 ceph-mon[27954]: pgmap v48: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-21T04:37:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:28 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:37:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:29 smithi053 ceph-mon[35554]: mgrmap e33: smithi007.huohfl(active, since 91s), standbys: smithi053.lnadao 2024-09-21T04:37:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:29 smithi007 ceph-mon[27954]: mgrmap e33: smithi007.huohfl(active, since 91s), standbys: smithi053.lnadao 2024-09-21T04:37:31.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:30 smithi053 ceph-mon[35554]: pgmap v49: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T04:37:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:30 smithi007 ceph-mon[27954]: pgmap v49: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T04:37:33.138 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:32 smithi007 ceph-mon[27954]: pgmap v50: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:37:33.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:32 smithi053 ceph-mon[35554]: pgmap v50: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:37:35.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:34 smithi007 ceph-mon[27954]: pgmap v51: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T04:37:35.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:34 smithi053 ceph-mon[35554]: pgmap v51: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T04:37:37.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:36 smithi053 ceph-mon[35554]: pgmap v52: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T04:37:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:36 smithi007 ceph-mon[27954]: pgmap v52: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T04:37:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:38 smithi053 ceph-mon[35554]: pgmap v53: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:37:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:38 smithi007 ceph-mon[27954]: pgmap v53: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-21T04:37:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:40 smithi053 ceph-mon[35554]: pgmap v54: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T04:37:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:40 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:40 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:40 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:37:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:40 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:40 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:40 smithi007 ceph-mon[27954]: pgmap v54: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-21T04:37:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:40 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:40 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:40 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:37:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:40 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:40 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:42 smithi053 ceph-mon[35554]: pgmap v55: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-21T04:37:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:42 smithi007 ceph-mon[27954]: pgmap v55: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-21T04:37:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:43 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:37:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:43 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:37:45.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:44 smithi053 ceph-mon[35554]: pgmap v56: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:37:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:44 smithi007 ceph-mon[27954]: pgmap v56: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:37:46.304 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:46 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:46.305 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:46 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:46.305 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:46 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:46.305 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:46 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:46 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:46 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:46 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:46 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:47.294 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:47 smithi007 ceph-mon[27954]: pgmap v57: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:37:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:47 smithi053 ceph-mon[35554]: pgmap v57: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:37:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 -- 172.21.15.7:0/2849176601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938100860 msgr2=0x7ff938100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 --2- 172.21.15.7:0/2849176601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938100860 0x7ff938100c60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff934009880 tx=0x7ff93402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 -- 172.21.15.7:0/2849176601 shutdown_connections 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 --2- 172.21.15.7:0/2849176601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff938101a60 0x7ff938101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 --2- 172.21.15.7:0/2849176601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938100860 0x7ff938100c60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 -- 172.21.15.7:0/2849176601 >> 172.21.15.7:0/2849176601 conn(0x7ff9380fc010 msgr2=0x7ff9380fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 -- 172.21.15.7:0/2849176601 shutdown_connections 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 -- 172.21.15.7:0/2849176601 wait complete. 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 Processor -- start 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.897+0000 7ff93ffee640 1 -- start start 2024-09-21T04:37:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93ffee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff938100860 0x7ff9381974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93ffee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938101a60 0x7ff938197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93ffee640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff93810a780 con 0x7ff938101a60 2024-09-21T04:37:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93ffee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff93810a600 con 0x7ff938100860 2024-09-21T04:37:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93d589640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938101a60 0x7ff938197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93d589640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938101a60 0x7ff938197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40192/0 (socket says 172.21.15.7:40192) 2024-09-21T04:37:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93d589640 1 -- 172.21.15.7:0/2791551310 learned_addr learned my addr 172.21.15.7:0/2791551310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93dd8a640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff938100860 0x7ff9381974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93d589640 1 -- 172.21.15.7:0/2791551310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff938100860 msgr2=0x7ff9381974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93d589640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff938100860 0x7ff9381974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93d589640 1 -- 172.21.15.7:0/2791551310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff938198080 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93d589640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938101a60 0x7ff938197a00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff92800d8d0 tx=0x7ff92800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff92effd640 1 -- 172.21.15.7:0/2791551310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff928017070 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93819cac0 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff92effd640 1 -- 172.21.15.7:0/2791551310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff928010460 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.898+0000 7ff92effd640 1 -- 172.21.15.7:0/2791551310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff92800f5f0 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.899+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93819d050 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.899+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff938100c60 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.900+0000 7ff92effd640 1 -- 172.21.15.7:0/2791551310 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff928009a30 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.901+0000 7ff92effd640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff92407af30 0x7ff92407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.901+0000 7ff92effd640 1 -- 172.21.15.7:0/2791551310 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7ff92809ddd0 con 0x7ff938101a60 2024-09-21T04:37:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.901+0000 7ff93dd8a640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff92407af30 0x7ff92407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.901+0000 7ff93dd8a640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff92407af30 0x7ff92407d3f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff934009230 tx=0x7ff9340058d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:48.907+0000 7ff92effd640 1 -- 172.21.15.7:0/2791551310 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7ff928069420 con 0x7ff938101a60 2024-09-21T04:37:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.118+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff9380626f0 con 0x7ff92407af30 2024-09-21T04:37:49.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: pgmap v58: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:37:49.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:49.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:49.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:37:49.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:37:49.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:49.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:49 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.119+0000 7ff92effd640 1 -- 172.21.15.7:0/2791551310 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+369 (secure 0 0 0) 0x7ff9380626f0 con 0x7ff92407af30 2024-09-21T04:37:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff92407af30 msgr2=0x7ff92407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff92407af30 0x7ff92407d3f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff934009230 tx=0x7ff9340058d0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938101a60 msgr2=0x7ff938197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938101a60 0x7ff938197a00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff92800d8d0 tx=0x7ff92800dd90 comp rx=0 tx=0).stop 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 shutdown_connections 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff92407af30 0x7ff92407d3f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff938101a60 0x7ff938197a00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 --2- 172.21.15.7:0/2791551310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff938100860 0x7ff9381974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.121+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 >> 172.21.15.7:0/2791551310 conn(0x7ff9380fc010 msgr2=0x7ff938104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.122+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 shutdown_connections 2024-09-21T04:37:49.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.122+0000 7ff93ffee640 1 -- 172.21.15.7:0/2791551310 wait complete. 2024-09-21T04:37:49.133 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:37:49.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.213+0000 7f309f5ad640 1 -- 172.21.15.7:0/2065530573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098101960 msgr2=0x7f3098101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.213+0000 7f309f5ad640 1 --2- 172.21.15.7:0/2065530573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098101960 0x7f3098101d60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3094009880 tx=0x7f309402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.213+0000 7f309f5ad640 1 -- 172.21.15.7:0/2065530573 shutdown_connections 2024-09-21T04:37:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.213+0000 7f309f5ad640 1 --2- 172.21.15.7:0/2065530573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3098102b60 0x7f3098102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.213+0000 7f309f5ad640 1 --2- 172.21.15.7:0/2065530573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098101960 0x7f3098101d60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.213+0000 7f309f5ad640 1 -- 172.21.15.7:0/2065530573 >> 172.21.15.7:0/2065530573 conn(0x7f30980fd110 msgr2=0x7f30980ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:49.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.213+0000 7f309f5ad640 1 -- 172.21.15.7:0/2065530573 shutdown_connections 2024-09-21T04:37:49.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.214+0000 7f309f5ad640 1 -- 172.21.15.7:0/2065530573 wait complete. 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.214+0000 7f309f5ad640 1 Processor -- start 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.214+0000 7f309f5ad640 1 -- start start 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.214+0000 7f309f5ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3098101960 0x7f30981996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.214+0000 7f309f5ad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098102b60 0x7f3098199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.214+0000 7f309f5ad640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f309810a7a0 con 0x7f3098102b60 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.214+0000 7f309f5ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f309810a620 con 0x7f3098101960 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309cb48640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098102b60 0x7f3098199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309cb48640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098102b60 0x7f3098199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40200/0 (socket says 172.21.15.7:40200) 2024-09-21T04:37:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309cb48640 1 -- 172.21.15.7:0/1201094872 learned_addr learned my addr 172.21.15.7:0/1201094872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309d349640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3098101960 0x7f30981996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309cb48640 1 -- 172.21.15.7:0/1201094872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3098101960 msgr2=0x7f30981996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309cb48640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3098101960 0x7f30981996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309cb48640 1 -- 172.21.15.7:0/1201094872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309819a2b0 con 0x7f3098102b60 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309cb48640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098102b60 0x7f3098199c30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f308800c8f0 tx=0x7f308800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f308e7fc640 1 -- 172.21.15.7:0/1201094872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f30880079b0 con 0x7f3098102b60 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309819ecf0 con 0x7f3098102b60 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309819f220 con 0x7f3098102b60 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.215+0000 7f308e7fc640 1 -- 172.21.15.7:0/1201094872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f30880054e0 con 0x7f3098102b60 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.216+0000 7f308e7fc640 1 -- 172.21.15.7:0/1201094872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3088004dd0 con 0x7f3098102b60 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.216+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f307c0052c0 con 0x7f3098102b60 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.217+0000 7f308e7fc640 1 -- 172.21.15.7:0/1201094872 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f308800f4b0 con 0x7f3098102b60 2024-09-21T04:37:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.218+0000 7f308e7fc640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f308407af30 0x7f308407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.218+0000 7f308e7fc640 1 -- 172.21.15.7:0/1201094872 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f308809ddf0 con 0x7f3098102b60 2024-09-21T04:37:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.218+0000 7f309d349640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f308407af30 0x7f308407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.219+0000 7f309d349640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f308407af30 0x7f308407d3f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3094009230 tx=0x7f30940058d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:49.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.223+0000 7f308e7fc640 1 -- 172.21.15.7:0/1201094872 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f3088014030 con 0x7f3098102b60 2024-09-21T04:37:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.433+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f307c000bf0 con 0x7f308407af30 2024-09-21T04:37:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.435+0000 7f308e7fc640 1 -- 172.21.15.7:0/1201094872 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+369 (secure 0 0 0) 0x7f307c000bf0 con 0x7f308407af30 2024-09-21T04:37:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f308407af30 msgr2=0x7f308407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f308407af30 0x7f308407d3f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3094009230 tx=0x7f30940058d0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098102b60 msgr2=0x7f3098199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098102b60 0x7f3098199c30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f308800c8f0 tx=0x7f308800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 shutdown_connections 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f308407af30 0x7f308407d3f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3098102b60 0x7f3098199c30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 --2- 172.21.15.7:0/1201094872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3098101960 0x7f30981996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 >> 172.21.15.7:0/1201094872 conn(0x7f30980fd110 msgr2=0x7f30980746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 shutdown_connections 2024-09-21T04:37:49.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.437+0000 7f309f5ad640 1 -- 172.21.15.7:0/1201094872 wait complete. 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: pgmap v58: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:49 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 -- 172.21.15.7:0/200113170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 msgr2=0x7f98bc101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 --2- 172.21.15.7:0/200113170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 0x7f98bc101ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009880 tx=0x7f98ac02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 -- 172.21.15.7:0/200113170 shutdown_connections 2024-09-21T04:37:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 --2- 172.21.15.7:0/200113170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 0x7f98bc101ee0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 --2- 172.21.15.7:0/200113170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc100860 0x7f98bc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 -- 172.21.15.7:0/200113170 >> 172.21.15.7:0/200113170 conn(0x7f98bc0fc010 msgr2=0x7f98bc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 -- 172.21.15.7:0/200113170 shutdown_connections 2024-09-21T04:37:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 -- 172.21.15.7:0/200113170 wait complete. 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 Processor -- start 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.525+0000 7f98c32d2640 1 -- start start 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c32d2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc100860 0x7f98bc197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c32d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 0x7f98bc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c32d2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc109720 con 0x7f98bc101a60 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c32d2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1095a0 con 0x7f98bc100860 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c086d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 0x7f98bc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c086d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 0x7f98bc197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40220/0 (socket says 172.21.15.7:40220) 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c086d640 1 -- 172.21.15.7:0/3778900185 learned_addr learned my addr 172.21.15.7:0/3778900185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c106e640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc100860 0x7f98bc197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c086d640 1 -- 172.21.15.7:0/3778900185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc100860 msgr2=0x7f98bc197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c086d640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc100860 0x7f98bc197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c086d640 1 -- 172.21.15.7:0/3778900185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc198150 con 0x7f98bc101a60 2024-09-21T04:37:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.526+0000 7f98c086d640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 0x7f98bc197ad0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f98ac006010 tx=0x7f98ac0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.527+0000 7f98b27fc640 1 -- 172.21.15.7:0/3778900185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98ac03f070 con 0x7f98bc101a60 2024-09-21T04:37:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.527+0000 7f98b27fc640 1 -- 172.21.15.7:0/3778900185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98ac0044d0 con 0x7f98bc101a60 2024-09-21T04:37:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.527+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc19cb30 con 0x7f98bc101a60 2024-09-21T04:37:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.527+0000 7f98b27fc640 1 -- 172.21.15.7:0/3778900185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98ac035030 con 0x7f98bc101a60 2024-09-21T04:37:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.527+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc19d140 con 0x7f98bc101a60 2024-09-21T04:37:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.528+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc100c40 con 0x7f98bc101a60 2024-09-21T04:37:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.528+0000 7f98b27fc640 1 -- 172.21.15.7:0/3778900185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f98ac042070 con 0x7f98bc101a60 2024-09-21T04:37:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.529+0000 7f98b27fc640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98a807af80 0x7f98a807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.529+0000 7f98b27fc640 1 -- 172.21.15.7:0/3778900185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f98ac0beba0 con 0x7f98bc101a60 2024-09-21T04:37:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.529+0000 7f98c106e640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98a807af80 0x7f98a807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.530+0000 7f98c106e640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98a807af80 0x7f98a807d440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f98bc1018c0 tx=0x7f98b8009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.534+0000 7f98b27fc640 1 -- 172.21.15.7:0/3778900185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f98ac08a270 con 0x7f98bc101a60 2024-09-21T04:37:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.734+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f98bc06a910 con 0x7f98a807af80 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.744+0000 7f98b27fc640 1 -- 172.21.15.7:0/3778900185 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f98bc06a910 con 0x7f98a807af80 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (9m) 4s ago 10m 23.8M - 0.25.0 c8568f914cd2 ba81c202deaf 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (2m) 4s ago 11m 8678k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (2m) 35s ago 10m 8514k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (10m) 4s ago 10m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (10m) 35s ago 10m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (9m) 4s ago 10m 79.3M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:37:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (5m) 4s ago 5m 146M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (5m) 4s ago 5m 89.0M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (5m) 35s ago 5m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (5m) 35s ago 5m 30.9M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (2m) 4s ago 12m 516M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (64s) 35s ago 9m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (12m) 4s ago 12m 58.2M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (9m) 35s ago 9m 47.3M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (45s) 4s ago 10m 8246k - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (38s) 35s ago 10m 3405k - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (8m) 4s ago 8m 211M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (8m) 4s ago 8m 212M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (7m) 4s ago 7m 192M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (7m) 35s ago 7m 241M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:37:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (6m) 35s ago 6m 210M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:37:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (6m) 35s ago 6m 181M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:37:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (11s) 4s ago 10m 40.9M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:37:49.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98a807af80 msgr2=0x7f98a807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98a807af80 0x7f98a807d440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f98bc1018c0 tx=0x7f98b8009110 comp rx=0 tx=0).stop 2024-09-21T04:37:49.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 msgr2=0x7f98bc197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 0x7f98bc197ad0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f98ac006010 tx=0x7f98ac0048f0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 shutdown_connections 2024-09-21T04:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98a807af80 0x7f98a807d440 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98bc101a60 0x7f98bc197ad0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 --2- 172.21.15.7:0/3778900185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98bc100860 0x7f98bc197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 >> 172.21.15.7:0/3778900185 conn(0x7f98bc0fc010 msgr2=0x7f98bc06a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 shutdown_connections 2024-09-21T04:37:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.746+0000 7f98c32d2640 1 -- 172.21.15.7:0/3778900185 wait complete. 2024-09-21T04:37:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.829+0000 7fef7759e640 1 -- 172.21.15.7:0/1226734210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781017a0 msgr2=0x7fef78101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.829+0000 7fef7759e640 1 --2- 172.21.15.7:0/1226734210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781017a0 0x7fef78101ba0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fef60009880 tx=0x7fef6002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.829+0000 7fef7759e640 1 -- 172.21.15.7:0/1226734210 shutdown_connections 2024-09-21T04:37:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.829+0000 7fef7759e640 1 --2- 172.21.15.7:0/1226734210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef781029a0 0x7fef78102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.829+0000 7fef7759e640 1 --2- 172.21.15.7:0/1226734210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781017a0 0x7fef78101ba0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.829+0000 7fef7759e640 1 -- 172.21.15.7:0/1226734210 >> 172.21.15.7:0/1226734210 conn(0x7fef780fcf30 msgr2=0x7fef780ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.830+0000 7fef7759e640 1 -- 172.21.15.7:0/1226734210 shutdown_connections 2024-09-21T04:37:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.830+0000 7fef7759e640 1 -- 172.21.15.7:0/1226734210 wait complete. 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.830+0000 7fef7759e640 1 Processor -- start 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.830+0000 7fef7759e640 1 -- start start 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef7759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef781017a0 0x7fef781972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef7759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781029a0 0x7fef78197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef7759e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810a5c0 con 0x7fef781029a0 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef7759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810a440 con 0x7fef781017a0 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef75d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781029a0 0x7fef78197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef75d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781029a0 0x7fef78197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40236/0 (socket says 172.21.15.7:40236) 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef75d9b640 1 -- 172.21.15.7:0/1466862346 learned_addr learned my addr 172.21.15.7:0/1466862346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef7659c640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef781017a0 0x7fef781972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef75d9b640 1 -- 172.21.15.7:0/1466862346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef781017a0 msgr2=0x7fef781972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef75d9b640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef781017a0 0x7fef781972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef75d9b640 1 -- 172.21.15.7:0/1466862346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef78197e80 con 0x7fef781029a0 2024-09-21T04:37:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef75d9b640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781029a0 0x7fef78197800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c810 tx=0x7fef6800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef6f7fe640 1 -- 172.21.15.7:0/1466862346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fef68004020 con 0x7fef781029a0 2024-09-21T04:37:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef6f7fe640 1 -- 172.21.15.7:0/1466862346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fef680041c0 con 0x7fef781029a0 2024-09-21T04:37:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7819c8c0 con 0x7fef781029a0 2024-09-21T04:37:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.831+0000 7fef6f7fe640 1 -- 172.21.15.7:0/1466862346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fef680047a0 con 0x7fef781029a0 2024-09-21T04:37:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.832+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7819ce50 con 0x7fef781029a0 2024-09-21T04:37:49.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.833+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef78101ba0 con 0x7fef781029a0 2024-09-21T04:37:49.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.833+0000 7fef6f7fe640 1 -- 172.21.15.7:0/1466862346 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fef680079b0 con 0x7fef781029a0 2024-09-21T04:37:49.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.834+0000 7fef6f7fe640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fef6407af80 0x7fef6407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:49.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.834+0000 7fef6f7fe640 1 -- 172.21.15.7:0/1466862346 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fef6809dd80 con 0x7fef781029a0 2024-09-21T04:37:49.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.834+0000 7fef7659c640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fef6407af80 0x7fef6407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:49.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.835+0000 7fef7659c640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fef6407af80 0x7fef6407d440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef60009230 tx=0x7fef600058d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:49.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:49.838+0000 7fef6f7fe640 1 -- 172.21.15.7:0/1466862346 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fef68014030 con 0x7fef781029a0 2024-09-21T04:37:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.113+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fef78062df0 con 0x7fef781029a0 2024-09-21T04:37:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.113+0000 7fef6f7fe640 1 -- 172.21.15.7:0/1466862346 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7fef68069450 con 0x7fef781029a0 2024-09-21T04:37:50.116 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:37:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 12, 2024-09-21T04:37:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:37:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:37:50.118 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fef6407af80 msgr2=0x7fef6407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fef6407af80 0x7fef6407d440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef60009230 tx=0x7fef600058d0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781029a0 msgr2=0x7fef78197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781029a0 0x7fef78197800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c810 tx=0x7fef6800ccd0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 shutdown_connections 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fef6407af80 0x7fef6407d440 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef781029a0 0x7fef78197800 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 --2- 172.21.15.7:0/1466862346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef781017a0 0x7fef781972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 >> 172.21.15.7:0/1466862346 conn(0x7fef780fcf30 msgr2=0x7fef78105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 shutdown_connections 2024-09-21T04:37:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.116+0000 7fef7759e640 1 -- 172.21.15.7:0/1466862346 wait complete. 2024-09-21T04:37:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.202+0000 7fe7b1d76640 1 -- 172.21.15.7:0/1536806045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 msgr2=0x7fe7ac1071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.202+0000 7fe7b1d76640 1 --2- 172.21.15.7:0/1536806045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 0x7fe7ac1071b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009880 tx=0x7fe7a002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.202+0000 7fe7b1d76640 1 -- 172.21.15.7:0/1536806045 shutdown_connections 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.202+0000 7fe7b1d76640 1 --2- 172.21.15.7:0/1536806045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 0x7fe7ac1071b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.202+0000 7fe7b1d76640 1 --2- 172.21.15.7:0/1536806045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac102490 0x7fe7ac104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.202+0000 7fe7b1d76640 1 -- 172.21.15.7:0/1536806045 >> 172.21.15.7:0/1536806045 conn(0x7fe7ac0fc010 msgr2=0x7fe7ac0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.202+0000 7fe7b1d76640 1 -- 172.21.15.7:0/1536806045 shutdown_connections 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.202+0000 7fe7b1d76640 1 -- 172.21.15.7:0/1536806045 wait complete. 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7b1d76640 1 Processor -- start 2024-09-21T04:37:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7b1d76640 1 -- start start 2024-09-21T04:37:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7b1d76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac102490 0x7fe7ac101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7b1d76640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 0x7fe7ac100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7b1d76640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac10a800 con 0x7fe7ac104dc0 2024-09-21T04:37:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7b1d76640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac10a680 con 0x7fe7ac102490 2024-09-21T04:37:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7aaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 0x7fe7ac100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7aaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 0x7fe7ac100440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40264/0 (socket says 172.21.15.7:40264) 2024-09-21T04:37:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7aaffd640 1 -- 172.21.15.7:0/4210422042 learned_addr learned my addr 172.21.15.7:0/4210422042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:37:50.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.203+0000 7fe7ab7fe640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac102490 0x7fe7ac101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7aaffd640 1 -- 172.21.15.7:0/4210422042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac102490 msgr2=0x7fe7ac101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7aaffd640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac102490 0x7fe7ac101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7aaffd640 1 -- 172.21.15.7:0/4210422042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac100980 con 0x7fe7ac104dc0 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7aaffd640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 0x7fe7ac100440 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006010 tx=0x7fe7a00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7a8ff9640 1 -- 172.21.15.7:0/4210422042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe7a003f070 con 0x7fe7ac104dc0 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7a8ff9640 1 -- 172.21.15.7:0/4210422042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7a00044d0 con 0x7fe7ac104dc0 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac100c10 con 0x7fe7ac104dc0 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7a8ff9640 1 -- 172.21.15.7:0/4210422042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe7a0035030 con 0x7fe7ac104dc0 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.204+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac1011c0 con 0x7fe7ac104dc0 2024-09-21T04:37:50.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.205+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac102870 con 0x7fe7ac104dc0 2024-09-21T04:37:50.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.206+0000 7fe7a8ff9640 1 -- 172.21.15.7:0/4210422042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe7a0042070 con 0x7fe7ac104dc0 2024-09-21T04:37:50.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.206+0000 7fe7a8ff9640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe79407af80 0x7fe79407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:37:50.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.207+0000 7fe7a8ff9640 1 -- 172.21.15.7:0/4210422042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fe7a00beb30 con 0x7fe7ac104dc0 2024-09-21T04:37:50.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.207+0000 7fe7ab7fe640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe79407af80 0x7fe79407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:37:50.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.207+0000 7fe7ab7fe640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe79407af80 0x7fe79407d440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009990 tx=0x7fe79c009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:37:50.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.211+0000 7fe7a8ff9640 1 -- 172.21.15.7:0/4210422042 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fe7a008a200 con 0x7fe7ac104dc0 2024-09-21T04:37:50.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:50 smithi007 ceph-mon[27954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:37:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.421+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe7ac105740 con 0x7fe79407af80 2024-09-21T04:37:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.423+0000 7fe7a8ff9640 1 -- 172.21.15.7:0/4210422042 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+369 (secure 0 0 0) 0x7fe7ac105740 con 0x7fe79407af80 2024-09-21T04:37:50.425 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/2 daemons upgraded", 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading alertmanager daemons", 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:37:50.426 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:37:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe79407af80 msgr2=0x7fe79407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe79407af80 0x7fe79407d440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009990 tx=0x7fe79c009110 comp rx=0 tx=0).stop 2024-09-21T04:37:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 msgr2=0x7fe7ac100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:37:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 0x7fe7ac100440 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006010 tx=0x7fe7a00048f0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 shutdown_connections 2024-09-21T04:37:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe79407af80 0x7fe79407d440 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7ac104dc0 0x7fe7ac100440 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 --2- 172.21.15.7:0/4210422042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac102490 0x7fe7ac101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:37:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 >> 172.21.15.7:0/4210422042 conn(0x7fe7ac0fc010 msgr2=0x7fe7ac0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:37:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 shutdown_connections 2024-09-21T04:37:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:37:50.425+0000 7fe7b1d76640 1 -- 172.21.15.7:0/4210422042 wait complete. 2024-09-21T04:37:50.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:50 smithi053 ceph-mon[35554]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T04:37:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:51 smithi053 ceph-mon[35554]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:51 smithi053 ceph-mon[35554]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:51 smithi053 ceph-mon[35554]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:51 smithi053 ceph-mon[35554]: pgmap v59: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:37:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:51 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1466862346' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:51 smithi007 ceph-mon[27954]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:51 smithi007 ceph-mon[27954]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:51 smithi007 ceph-mon[27954]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:51 smithi007 ceph-mon[27954]: pgmap v59: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:37:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:51 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1466862346' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:37:52.345 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:52 smithi007 ceph-mon[27954]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:52.345 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:52 smithi007 ceph-mon[27954]: Upgrade: Updating alertmanager.smithi007 2024-09-21T04:37:52.345 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:52 smithi007 ceph-mon[27954]: Deploying daemon alertmanager.smithi007 on smithi007 2024-09-21T04:37:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:52 smithi053 ceph-mon[35554]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:37:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:52 smithi053 ceph-mon[35554]: Upgrade: Updating alertmanager.smithi007 2024-09-21T04:37:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:52 smithi053 ceph-mon[35554]: Deploying daemon alertmanager.smithi007 on smithi007 2024-09-21T04:37:53.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:53 smithi053 ceph-mon[35554]: pgmap v60: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:37:53.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:53 smithi007 ceph-mon[27954]: pgmap v60: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:37:55.353 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:55 smithi053 ceph-mon[35554]: pgmap v61: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:37:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:55 smithi007 ceph-mon[27954]: pgmap v61: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:37:56.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:56 smithi053 ceph-mon[35554]: pgmap v62: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:37:56.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:56 smithi007 ceph-mon[27954]: pgmap v62: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:37:58.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:58 smithi007 ceph-mon[27954]: pgmap v63: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:37:58.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:37:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:37:59.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:58 smithi053 ceph-mon[35554]: pgmap v63: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:37:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:37:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:01.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:00 smithi007 ceph-mon[27954]: pgmap v64: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:01.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:00 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:01.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:00 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:01.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:00 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:01.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:00 smithi053 ceph-mon[35554]: pgmap v64: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:00 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:00 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:00 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:03.052 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:02 smithi007 ceph-mon[27954]: pgmap v65: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:03.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:02 smithi053 ceph-mon[35554]: pgmap v65: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:04 smithi053 ceph-mon[35554]: pgmap v66: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:38:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:04 smithi007 ceph-mon[27954]: pgmap v66: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:38:06.340 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:05 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:06.340 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:05 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:06.340 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:05 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:06.340 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:05 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:06.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:05 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:06.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:05 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:06.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:05 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:06.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:05 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:07.295 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:06 smithi007 ceph-mon[27954]: pgmap v67: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:38:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:06 smithi053 ceph-mon[35554]: pgmap v67: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:38:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: pgmap v68: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:09.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:09.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:09.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:38:09.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:38:09.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:09.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:38:09.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:09.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:08 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: pgmap v68: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:09.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:09.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:09.417 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:09.418 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:08 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:10.269 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:09 smithi007 ceph-mon[27954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:38:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:09 smithi053 ceph-mon[35554]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T04:38:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:10 smithi053 ceph-mon[35554]: pgmap v69: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:11.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:10 smithi007 ceph-mon[27954]: pgmap v69: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:12.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:11 smithi053 ceph-mon[35554]: Upgrade: Updating grafana.smithi007 2024-09-21T04:38:12.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:11 smithi053 ceph-mon[35554]: Deploying daemon grafana.smithi007 on smithi007 2024-09-21T04:38:12.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:11 smithi007 ceph-mon[27954]: Upgrade: Updating grafana.smithi007 2024-09-21T04:38:12.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:11 smithi007 ceph-mon[27954]: Deploying daemon grafana.smithi007 on smithi007 2024-09-21T04:38:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:13 smithi053 ceph-mon[35554]: pgmap v70: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:12 smithi007 ceph-mon[27954]: pgmap v70: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:14 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:14 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:15 smithi053 ceph-mon[35554]: pgmap v71: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:15 smithi007 ceph-mon[27954]: pgmap v71: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:17 smithi053 ceph-mon[35554]: pgmap v72: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-21T04:38:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:17 smithi007 ceph-mon[27954]: pgmap v72: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-21T04:38:19.139 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:19 smithi007 ceph-mon[27954]: pgmap v73: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:19 smithi053 ceph-mon[35554]: pgmap v73: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.515+0000 7efe97b6c640 1 -- 172.21.15.7:0/1630624167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe900761a0 msgr2=0x7efe90074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.515+0000 7efe97b6c640 1 --2- 172.21.15.7:0/1630624167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe900761a0 0x7efe90074600 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efe80009c20 tx=0x7efe8002a970 comp rx=0 tx=0).stop 2024-09-21T04:38:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.515+0000 7efe97b6c640 1 -- 172.21.15.7:0/1630624167 shutdown_connections 2024-09-21T04:38:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.515+0000 7efe97b6c640 1 --2- 172.21.15.7:0/1630624167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90074b40 0x7efe90074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.515+0000 7efe97b6c640 1 --2- 172.21.15.7:0/1630624167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe900761a0 0x7efe90074600 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.515+0000 7efe97b6c640 1 -- 172.21.15.7:0/1630624167 >> 172.21.15.7:0/1630624167 conn(0x7efe900fd140 msgr2=0x7efe900ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.515+0000 7efe97b6c640 1 -- 172.21.15.7:0/1630624167 shutdown_connections 2024-09-21T04:38:20.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.515+0000 7efe97b6c640 1 -- 172.21.15.7:0/1630624167 wait complete. 2024-09-21T04:38:20.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe97b6c640 1 Processor -- start 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe97b6c640 1 -- start start 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe97b6c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe90074b40 0x7efe901996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe97b6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe900761a0 0x7efe90199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe97b6c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe9010c9a0 con 0x7efe90074b40 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe97b6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe9010c820 con 0x7efe900761a0 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe95908640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe90074b40 0x7efe901996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe95908640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe90074b40 0x7efe901996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54520/0 (socket says 172.21.15.7:54520) 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe95908640 1 -- 172.21.15.7:0/3379778537 learned_addr learned my addr 172.21.15.7:0/3379778537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe95107640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe900761a0 0x7efe90199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe95908640 1 -- 172.21.15.7:0/3379778537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe900761a0 msgr2=0x7efe90199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe95908640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe900761a0 0x7efe90199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.516+0000 7efe95908640 1 -- 172.21.15.7:0/3379778537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9019a290 con 0x7efe90074b40 2024-09-21T04:38:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.517+0000 7efe95107640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe900761a0 0x7efe90199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:38:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.517+0000 7efe95908640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe90074b40 0x7efe901996d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efe80009d40 tx=0x7efe8002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.517+0000 7efe8effd640 1 -- 172.21.15.7:0/3379778537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe80047070 con 0x7efe90074b40 2024-09-21T04:38:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.517+0000 7efe8effd640 1 -- 172.21.15.7:0/3379778537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe80003e30 con 0x7efe90074b40 2024-09-21T04:38:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.517+0000 7efe8effd640 1 -- 172.21.15.7:0/3379778537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe80034070 con 0x7efe90074b40 2024-09-21T04:38:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.517+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9019ec70 con 0x7efe90074b40 2024-09-21T04:38:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.517+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9019f1b0 con 0x7efe90074b40 2024-09-21T04:38:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.518+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe90074fa0 con 0x7efe90074b40 2024-09-21T04:38:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.519+0000 7efe8effd640 1 -- 172.21.15.7:0/3379778537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7efe8004a070 con 0x7efe90074b40 2024-09-21T04:38:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.519+0000 7efe8effd640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7807af30 0x7efe7807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.520+0000 7efe8effd640 1 -- 172.21.15.7:0/3379778537 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7efe800bef30 con 0x7efe90074b40 2024-09-21T04:38:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.520+0000 7efe95107640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7807af30 0x7efe7807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.520+0000 7efe95107640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7807af30 0x7efe7807d3f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe9019ab80 tx=0x7efe84005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.529+0000 7efe8effd640 1 -- 172.21.15.7:0/3379778537 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7efe8003f070 con 0x7efe90074b40 2024-09-21T04:38:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.769+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7efe900626f0 con 0x7efe7807af30 2024-09-21T04:38:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.771+0000 7efe8effd640 1 -- 172.21.15.7:0/3379778537 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+364 (secure 0 0 0) 0x7efe900626f0 con 0x7efe7807af30 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7807af30 msgr2=0x7efe7807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7807af30 0x7efe7807d3f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe9019ab80 tx=0x7efe84005f90 comp rx=0 tx=0).stop 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe90074b40 msgr2=0x7efe901996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe90074b40 0x7efe901996d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efe80009d40 tx=0x7efe8002f5d0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 shutdown_connections 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7807af30 0x7efe7807d3f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe900761a0 0x7efe90199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 --2- 172.21.15.7:0/3379778537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe90074b40 0x7efe901996d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 >> 172.21.15.7:0/3379778537 conn(0x7efe900fd140 msgr2=0x7efe90107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 shutdown_connections 2024-09-21T04:38:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.773+0000 7efe97b6c640 1 -- 172.21.15.7:0/3379778537 wait complete. 2024-09-21T04:38:20.785 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.859+0000 7f4a77e40640 1 -- 172.21.15.7:0/729657018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 msgr2=0x7f4a70107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.859+0000 7f4a77e40640 1 --2- 172.21.15.7:0/729657018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 0x7f4a70107110 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0098b0 tx=0x7f4a6c02a4d0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.859+0000 7f4a77e40640 1 -- 172.21.15.7:0/729657018 shutdown_connections 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.859+0000 7f4a77e40640 1 --2- 172.21.15.7:0/729657018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 0x7f4a70107110 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.859+0000 7f4a77e40640 1 --2- 172.21.15.7:0/729657018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a701023f0 0x7f4a701047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.859+0000 7f4a77e40640 1 -- 172.21.15.7:0/729657018 >> 172.21.15.7:0/729657018 conn(0x7f4a700fc010 msgr2=0x7f4a700fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a77e40640 1 -- 172.21.15.7:0/729657018 shutdown_connections 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a77e40640 1 -- 172.21.15.7:0/729657018 wait complete. 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a77e40640 1 Processor -- start 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a77e40640 1 -- start start 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a77e40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a701023f0 0x7f4a70101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:20.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a77e40640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 0x7f4a701003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a77e40640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010a7b0 con 0x7f4a70104d20 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a77e40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010a630 con 0x7f4a701023f0 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a753db640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 0x7f4a701003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a753db640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 0x7f4a701003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54550/0 (socket says 172.21.15.7:54550) 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.860+0000 7f4a753db640 1 -- 172.21.15.7:0/1286949689 learned_addr learned my addr 172.21.15.7:0/1286949689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a75bdc640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a701023f0 0x7f4a70101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a753db640 1 -- 172.21.15.7:0/1286949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a701023f0 msgr2=0x7f4a70101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a753db640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a701023f0 0x7f4a70101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a753db640 1 -- 172.21.15.7:0/1286949689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a701008e0 con 0x7f4a70104d20 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a75bdc640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a701023f0 0x7f4a70101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:38:20.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a753db640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 0x7f4a701003a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009230 tx=0x7f4a6c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:20.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a66ffd640 1 -- 172.21.15.7:0/1286949689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a6c03f070 con 0x7f4a70104d20 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a66ffd640 1 -- 172.21.15.7:0/1286949689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a6c004a50 con 0x7f4a70104d20 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a70100b70 con 0x7f4a70104d20 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a66ffd640 1 -- 172.21.15.7:0/1286949689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a6c035030 con 0x7f4a70104d20 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.861+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a701010a0 con 0x7f4a70104d20 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.862+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a701027d0 con 0x7f4a70104d20 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.863+0000 7f4a66ffd640 1 -- 172.21.15.7:0/1286949689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4a6c042070 con 0x7f4a70104d20 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.864+0000 7f4a66ffd640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a5c07af80 0x7f4a5c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.864+0000 7f4a66ffd640 1 -- 172.21.15.7:0/1286949689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f4a6c0bea60 con 0x7f4a70104d20 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.864+0000 7f4a75bdc640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a5c07af80 0x7f4a5c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:20.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.864+0000 7f4a75bdc640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a5c07af80 0x7f4a5c07d440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a600058c0 tx=0x7f4a60005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:20.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:20.868+0000 7f4a66ffd640 1 -- 172.21.15.7:0/1286949689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f4a6c08a130 con 0x7f4a70104d20 2024-09-21T04:38:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.081+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4a70105760 con 0x7f4a5c07af80 2024-09-21T04:38:21.084 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:21 smithi007 ceph-mon[27954]: pgmap v74: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.082+0000 7f4a66ffd640 1 -- 172.21.15.7:0/1286949689 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+364 (secure 0 0 0) 0x7f4a70105760 con 0x7f4a5c07af80 2024-09-21T04:38:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a5c07af80 msgr2=0x7f4a5c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a5c07af80 0x7f4a5c07d440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a600058c0 tx=0x7f4a60005850 comp rx=0 tx=0).stop 2024-09-21T04:38:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 msgr2=0x7f4a701003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 0x7f4a701003a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009230 tx=0x7f4a6c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 shutdown_connections 2024-09-21T04:38:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a5c07af80 0x7f4a5c07d440 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a70104d20 0x7f4a701003a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 --2- 172.21.15.7:0/1286949689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a701023f0 0x7f4a70101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.084+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 >> 172.21.15.7:0/1286949689 conn(0x7f4a700fc010 msgr2=0x7f4a700fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.085+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 shutdown_connections 2024-09-21T04:38:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.085+0000 7f4a77e40640 1 -- 172.21.15.7:0/1286949689 wait complete. 2024-09-21T04:38:21.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.170+0000 7f505759e640 1 -- 172.21.15.7:0/3018426960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 msgr2=0x7f5058101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.170+0000 7f505759e640 1 --2- 172.21.15.7:0/3018426960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 0x7f5058101dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5048009c20 tx=0x7f504802a970 comp rx=0 tx=0).stop 2024-09-21T04:38:21.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.170+0000 7f505759e640 1 -- 172.21.15.7:0/3018426960 shutdown_connections 2024-09-21T04:38:21.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.170+0000 7f505759e640 1 --2- 172.21.15.7:0/3018426960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 0x7f5058101dc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.170+0000 7f505759e640 1 --2- 172.21.15.7:0/3018426960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5058100740 0x7f5058100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.170+0000 7f505759e640 1 -- 172.21.15.7:0/3018426960 >> 172.21.15.7:0/3018426960 conn(0x7f50580fbed0 msgr2=0x7f50580fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:21.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.170+0000 7f505759e640 1 -- 172.21.15.7:0/3018426960 shutdown_connections 2024-09-21T04:38:21.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.170+0000 7f505759e640 1 -- 172.21.15.7:0/3018426960 wait complete. 2024-09-21T04:38:21.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f505759e640 1 Processor -- start 2024-09-21T04:38:21.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f505759e640 1 -- start start 2024-09-21T04:38:21.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f505759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5058100740 0x7f5058197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f505759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 0x7f5058197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f505759e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5058109560 con 0x7f5058101940 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f505759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f50581093e0 con 0x7f5058100740 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f5055d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 0x7f5058197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f5055d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 0x7f5058197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54566/0 (socket says 172.21.15.7:54566) 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f5055d9b640 1 -- 172.21.15.7:0/2038332643 learned_addr learned my addr 172.21.15.7:0/2038332643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.171+0000 7f505659c640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5058100740 0x7f5058197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f5055d9b640 1 -- 172.21.15.7:0/2038332643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5058100740 msgr2=0x7f5058197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f5055d9b640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5058100740 0x7f5058197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f5055d9b640 1 -- 172.21.15.7:0/2038332643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5058197f00 con 0x7f5058101940 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f5055d9b640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 0x7f5058197880 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5048009d40 tx=0x7f504802f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f504f7fe640 1 -- 172.21.15.7:0/2038332643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5048047070 con 0x7f5058101940 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505819c8e0 con 0x7f5058101940 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f504f7fe640 1 -- 172.21.15.7:0/2038332643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5048003e30 con 0x7f5058101940 2024-09-21T04:38:21.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f504f7fe640 1 -- 172.21.15.7:0/2038332643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5048034070 con 0x7f5058101940 2024-09-21T04:38:21.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.172+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505819ce70 con 0x7f5058101940 2024-09-21T04:38:21.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.173+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5058100b40 con 0x7f5058101940 2024-09-21T04:38:21.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.174+0000 7f504f7fe640 1 -- 172.21.15.7:0/2038332643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f504804a070 con 0x7f5058101940 2024-09-21T04:38:21.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.174+0000 7f504f7fe640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f504407af80 0x7f504407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.174+0000 7f504f7fe640 1 -- 172.21.15.7:0/2038332643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f50480bee60 con 0x7f5058101940 2024-09-21T04:38:21.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.175+0000 7f505659c640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f504407af80 0x7f504407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.175+0000 7f505659c640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f504407af80 0x7f504407d440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5040005fd0 tx=0x7f5040005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:21.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.179+0000 7f504f7fe640 1 -- 172.21.15.7:0/2038332643 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f504803f070 con 0x7f5058101940 2024-09-21T04:38:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.376+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f505806a8c0 con 0x7f504407af80 2024-09-21T04:38:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.386+0000 7f504f7fe640 1 -- 172.21.15.7:0/2038332643 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f505806a8c0 con 0x7f504407af80 2024-09-21T04:38:21.389 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:38:21.389 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (22s) 16s ago 11m 17.4M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:38:21.389 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (3m) 16s ago 11m 8702k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (3m) 67s ago 10m 8514k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (11m) 16s ago 11m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (10m) 67s ago 10m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (9m) 16s ago 11m 80.1M - 9.4.12 2bacad6d85d8 6274bc742d37 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (5m) 16s ago 5m 146M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (6m) 16s ago 6m 89.0M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (5m) 67s ago 5m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (5m) 67s ago 5m 30.9M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (2m) 16s ago 12m 517M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (96s) 67s ago 10m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:38:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (12m) 16s ago 12m 59.5M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (10m) 67s ago 10m 47.3M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (76s) 16s ago 11m 11.5M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (70s) 67s ago 10m 3405k - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (9m) 16s ago 9m 211M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (8m) 16s ago 8m 212M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (8m) 16s ago 8m 193M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (7m) 67s ago 7m 241M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (7m) 67s ago 7m 210M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (7m) 67s ago 7m 181M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:38:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (43s) 16s ago 10m 41.9M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f504407af80 msgr2=0x7f504407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f504407af80 0x7f504407d440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5040005fd0 tx=0x7f5040005850 comp rx=0 tx=0).stop 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 msgr2=0x7f5058197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 0x7f5058197880 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5048009d40 tx=0x7f504802f730 comp rx=0 tx=0).stop 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 shutdown_connections 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f504407af80 0x7f504407d440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5058101940 0x7f5058197880 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 --2- 172.21.15.7:0/2038332643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5058100740 0x7f5058197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 >> 172.21.15.7:0/2038332643 conn(0x7f50580fbed0 msgr2=0x7f505806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.388+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 shutdown_connections 2024-09-21T04:38:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.389+0000 7f505759e640 1 -- 172.21.15.7:0/2038332643 wait complete. 2024-09-21T04:38:21.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:21 smithi053 ceph-mon[35554]: pgmap v74: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.471+0000 7f8d00e58640 1 -- 172.21.15.7:0/992291589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 msgr2=0x7f8cfc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.471+0000 7f8d00e58640 1 --2- 172.21.15.7:0/992291589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 0x7f8cfc100c60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009880 tx=0x7f8cec02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.471+0000 7f8d00e58640 1 -- 172.21.15.7:0/992291589 shutdown_connections 2024-09-21T04:38:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.471+0000 7f8d00e58640 1 --2- 172.21.15.7:0/992291589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cfc101a60 0x7f8cfc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.471+0000 7f8d00e58640 1 --2- 172.21.15.7:0/992291589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 0x7f8cfc100c60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.471+0000 7f8d00e58640 1 -- 172.21.15.7:0/992291589 >> 172.21.15.7:0/992291589 conn(0x7f8cfc0fc010 msgr2=0x7f8cfc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.471+0000 7f8d00e58640 1 -- 172.21.15.7:0/992291589 shutdown_connections 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.471+0000 7f8d00e58640 1 -- 172.21.15.7:0/992291589 wait complete. 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8d00e58640 1 Processor -- start 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8d00e58640 1 -- start start 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8d00e58640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 0x7f8cfc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8d00e58640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cfc101a60 0x7f8cfc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8d00e58640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc109700 con 0x7f8cfc100860 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8d00e58640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc109580 con 0x7f8cfc101a60 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 0x7f8cfc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 0x7f8cfc1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54590/0 (socket says 172.21.15.7:54590) 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.472+0000 7f8cfa59c640 1 -- 172.21.15.7:0/2280973750 learned_addr learned my addr 172.21.15.7:0/2280973750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8cf3fff640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cfc101a60 0x7f8cfc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8cfa59c640 1 -- 172.21.15.7:0/2280973750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cfc101a60 msgr2=0x7f8cfc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8cfa59c640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cfc101a60 0x7f8cfc197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8cfa59c640 1 -- 172.21.15.7:0/2280973750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc1981b0 con 0x7f8cfc100860 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8cfa59c640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 0x7f8cfc1975f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8cec006010 tx=0x7f8cec0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8cf37fe640 1 -- 172.21.15.7:0/2280973750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cec03f070 con 0x7f8cfc100860 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc19cb20 con 0x7f8cfc100860 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8cf37fe640 1 -- 172.21.15.7:0/2280973750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cec0044d0 con 0x7f8cfc100860 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8cf37fe640 1 -- 172.21.15.7:0/2280973750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cec035030 con 0x7f8cfc100860 2024-09-21T04:38:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.473+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc19d0b0 con 0x7f8cfc100860 2024-09-21T04:38:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.474+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cfc100c60 con 0x7f8cfc100860 2024-09-21T04:38:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.475+0000 7f8cf37fe640 1 -- 172.21.15.7:0/2280973750 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8cec042070 con 0x7f8cfc100860 2024-09-21T04:38:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.476+0000 7f8cf37fe640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ce807af30 0x7f8ce807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.476+0000 7f8cf37fe640 1 -- 172.21.15.7:0/2280973750 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f8cec0bf080 con 0x7f8cfc100860 2024-09-21T04:38:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.476+0000 7f8cf3fff640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ce807af30 0x7f8ce807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.476+0000 7f8cf3fff640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ce807af30 0x7f8ce807d3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc198a30 tx=0x7f8ce4009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.480+0000 7f8cf37fe640 1 -- 172.21.15.7:0/2280973750 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f8cec08a6b0 con 0x7f8cfc100860 2024-09-21T04:38:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.760+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f8cfc19d540 con 0x7f8cfc100860 2024-09-21T04:38:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.760+0000 7f8cf37fe640 1 -- 172.21.15.7:0/2280973750 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7f8cec08f560 con 0x7f8cfc100860 2024-09-21T04:38:21.763 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:38:21.763 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:38:21.763 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:38:21.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:38:21.763 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:38:21.763 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:38:21.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 12, 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:38:21.764 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ce807af30 msgr2=0x7f8ce807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ce807af30 0x7f8ce807d3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc198a30 tx=0x7f8ce4009110 comp rx=0 tx=0).stop 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 msgr2=0x7f8cfc1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 0x7f8cfc1975f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8cec006010 tx=0x7f8cec0048f0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 shutdown_connections 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ce807af30 0x7f8ce807d3f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cfc101a60 0x7f8cfc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 --2- 172.21.15.7:0/2280973750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc100860 0x7f8cfc1975f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 >> 172.21.15.7:0/2280973750 conn(0x7f8cfc0fc010 msgr2=0x7f8cfc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 shutdown_connections 2024-09-21T04:38:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.762+0000 7f8d00e58640 1 -- 172.21.15.7:0/2280973750 wait complete. 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 -- 172.21.15.7:0/1439910372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 msgr2=0x7f207c104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 --2- 172.21.15.7:0/1439910372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 0x7f207c104840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2074009880 tx=0x7f207402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 -- 172.21.15.7:0/1439910372 shutdown_connections 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 --2- 172.21.15.7:0/1439910372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f207c104d80 0x7f207c107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 --2- 172.21.15.7:0/1439910372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 0x7f207c104840 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 -- 172.21.15.7:0/1439910372 >> 172.21.15.7:0/1439910372 conn(0x7f207c0fc010 msgr2=0x7f207c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 -- 172.21.15.7:0/1439910372 shutdown_connections 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 -- 172.21.15.7:0/1439910372 wait complete. 2024-09-21T04:38:21.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.859+0000 7f208312d640 1 Processor -- start 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f208312d640 1 -- start start 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f208312d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 0x7f207c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f208312d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f207c104d80 0x7f207c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f208312d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f207c10a770 con 0x7f207c102450 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f208312d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f207c10a5f0 con 0x7f207c104d80 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f2080ec9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 0x7f207c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f2080ec9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 0x7f207c197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54604/0 (socket says 172.21.15.7:54604) 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f2080ec9640 1 -- 172.21.15.7:0/3616140258 learned_addr learned my addr 172.21.15.7:0/3616140258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f207bfff640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f207c104d80 0x7f207c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f2080ec9640 1 -- 172.21.15.7:0/3616140258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f207c104d80 msgr2=0x7f207c1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f2080ec9640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f207c104d80 0x7f207c1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:21.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f2080ec9640 1 -- 172.21.15.7:0/3616140258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207c198030 con 0x7f207c102450 2024-09-21T04:38:21.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.860+0000 7f2080ec9640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 0x7f207c197470 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2074006010 tx=0x7f20740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:21.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.861+0000 7f2079ffb640 1 -- 172.21.15.7:0/3616140258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f207403f070 con 0x7f207c102450 2024-09-21T04:38:21.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.861+0000 7f2079ffb640 1 -- 172.21.15.7:0/3616140258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20740044d0 con 0x7f207c102450 2024-09-21T04:38:21.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.861+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207c19ca70 con 0x7f207c102450 2024-09-21T04:38:21.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.861+0000 7f2079ffb640 1 -- 172.21.15.7:0/3616140258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2074035030 con 0x7f207c102450 2024-09-21T04:38:21.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.861+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f207c19d000 con 0x7f207c102450 2024-09-21T04:38:21.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.862+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f207c102830 con 0x7f207c102450 2024-09-21T04:38:21.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.863+0000 7f2079ffb640 1 -- 172.21.15.7:0/3616140258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2074042070 con 0x7f207c102450 2024-09-21T04:38:21.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.863+0000 7f2079ffb640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f206807af80 0x7f206807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:21.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.863+0000 7f2079ffb640 1 -- 172.21.15.7:0/3616140258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f20740bebe0 con 0x7f207c102450 2024-09-21T04:38:21.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.863+0000 7f207bfff640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f206807af80 0x7f206807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:21.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.864+0000 7f207bfff640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f206807af80 0x7f206807d440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f207c198920 tx=0x7f206400a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:21.868+0000 7f2079ffb640 1 -- 172.21.15.7:0/3616140258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f207408a250 con 0x7f207c102450 2024-09-21T04:38:22.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.087+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f207c105640 con 0x7f206807af80 2024-09-21T04:38:22.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.088+0000 7f2079ffb640 1 -- 172.21.15.7:0/3616140258 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+364 (secure 0 0 0) 0x7f207c105640 con 0x7f206807af80 2024-09-21T04:38:22.091 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:38:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:38:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:38:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-21T04:38:22.091 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:38:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-09-21T04:38:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:38:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/2 daemons upgraded", 2024-09-21T04:38:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading grafana daemons", 2024-09-21T04:38:22.092 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:38:22.092 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:38:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f206807af80 msgr2=0x7f206807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f206807af80 0x7f206807d440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f207c198920 tx=0x7f206400a250 comp rx=0 tx=0).stop 2024-09-21T04:38:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 msgr2=0x7f207c197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 0x7f207c197470 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2074006010 tx=0x7f20740048f0 comp rx=0 tx=0).stop 2024-09-21T04:38:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 shutdown_connections 2024-09-21T04:38:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f206807af80 0x7f206807d440 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f207c104d80 0x7f207c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 --2- 172.21.15.7:0/3616140258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f207c102450 0x7f207c197470 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 >> 172.21.15.7:0/3616140258 conn(0x7f207c0fc010 msgr2=0x7f207c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 shutdown_connections 2024-09-21T04:38:22.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:22.091+0000 7f208312d640 1 -- 172.21.15.7:0/3616140258 wait complete. 2024-09-21T04:38:22.316 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:22 smithi007 ceph-mon[27954]: from='client.14882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:22.317 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:22 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2280973750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:22 smithi053 ceph-mon[35554]: from='client.14882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:22 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2280973750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:23.119 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:23 smithi007 ceph-mon[27954]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:23.119 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:23 smithi007 ceph-mon[27954]: from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:23.120 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:23 smithi007 ceph-mon[27954]: pgmap v75: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:23.120 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:23 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:23.120 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:23 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:23.120 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:23 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:23 smithi053 ceph-mon[35554]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:23 smithi053 ceph-mon[35554]: from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:23 smithi053 ceph-mon[35554]: pgmap v75: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:23 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:23 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:23 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:24.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:24 smithi007 ceph-mon[27954]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:24 smithi053 ceph-mon[35554]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:25 smithi007 ceph-mon[27954]: pgmap v76: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:25 smithi053 ceph-mon[35554]: pgmap v76: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:27 smithi053 ceph-mon[35554]: pgmap v77: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:27 smithi007 ceph-mon[27954]: pgmap v77: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:28.915 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:28 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:28.915 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:28 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:28.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:28 smithi007 ceph-mon[27954]: pgmap v78: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:28.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:28 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:28.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:28 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:28.916 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:28 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:28 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:28 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:28 smithi053 ceph-mon[35554]: pgmap v78: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:28 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:28 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:28 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:30 smithi053 ceph-mon[35554]: pgmap v79: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:30 smithi007 ceph-mon[27954]: pgmap v79: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:32.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:32.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:32.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:38:32.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:38:32.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:32.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:38:32.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:38:32.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: pgmap v80: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: Upgrade: Finalizing container_image settings 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-21T04:38:32.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-21T04:38:32.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: Upgrade: Complete! 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:38:32.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:32 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:33.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: pgmap v80: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: Upgrade: Finalizing container_image settings 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: Upgrade: Complete! 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:33.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:38:33.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:38:33.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:33.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:38:33.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:38:33.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:38:33.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:32 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:34.887 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:34 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:34.887 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:34 smithi053 ceph-mon[35554]: pgmap v81: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:34.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:34 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:38:34.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:34 smithi007 ceph-mon[27954]: pgmap v81: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:37.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:36 smithi053 ceph-mon[35554]: pgmap v82: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:36 smithi007 ceph-mon[27954]: pgmap v82: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:38 smithi053 ceph-mon[35554]: pgmap v83: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:38 smithi007 ceph-mon[27954]: pgmap v83: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:40 smithi053 ceph-mon[35554]: pgmap v84: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:40 smithi007 ceph-mon[27954]: pgmap v84: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:42 smithi053 ceph-mon[35554]: pgmap v85: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:42 smithi007 ceph-mon[27954]: pgmap v85: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:43 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:43 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:45.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:44 smithi053 ceph-mon[35554]: pgmap v86: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:44 smithi007 ceph-mon[27954]: pgmap v86: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:47.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:46 smithi053 ceph-mon[35554]: pgmap v87: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:46 smithi007 ceph-mon[27954]: pgmap v87: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:49.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:48 smithi053 ceph-mon[35554]: pgmap v88: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:49.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:48 smithi007 ceph-mon[27954]: pgmap v88: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:51.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:50 smithi053 ceph-mon[35554]: pgmap v89: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:51.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:50 smithi007 ceph-mon[27954]: pgmap v89: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.190+0000 7fdc290cb640 1 -- 172.21.15.7:0/3595914037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc240ffc40 msgr2=0x7fdc24100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.190+0000 7fdc290cb640 1 --2- 172.21.15.7:0/3595914037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc240ffc40 0x7fdc24100040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009880 tx=0x7fdc1402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:38:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.191+0000 7fdc290cb640 1 -- 172.21.15.7:0/3595914037 shutdown_connections 2024-09-21T04:38:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.191+0000 7fdc290cb640 1 --2- 172.21.15.7:0/3595914037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc24102cc0 0x7fdc24103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.191+0000 7fdc290cb640 1 --2- 172.21.15.7:0/3595914037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc240ffc40 0x7fdc24100040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.191+0000 7fdc290cb640 1 -- 172.21.15.7:0/3595914037 >> 172.21.15.7:0/3595914037 conn(0x7fdc240fb770 msgr2=0x7fdc240fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.191+0000 7fdc290cb640 1 -- 172.21.15.7:0/3595914037 shutdown_connections 2024-09-21T04:38:52.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.191+0000 7fdc290cb640 1 -- 172.21.15.7:0/3595914037 wait complete. 2024-09-21T04:38:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.191+0000 7fdc290cb640 1 Processor -- start 2024-09-21T04:38:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc290cb640 1 -- start start 2024-09-21T04:38:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc290cb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc240ffc40 0x7fdc241025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc290cb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24102cc0 0x7fdc24100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc290cb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410a780 con 0x7fdc24102cc0 2024-09-21T04:38:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc290cb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410a600 con 0x7fdc240ffc40 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc2259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24102cc0 0x7fdc24100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc2259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24102cc0 0x7fdc24100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41520/0 (socket says 172.21.15.7:41520) 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc2259c640 1 -- 172.21.15.7:0/3725369157 learned_addr learned my addr 172.21.15.7:0/3725369157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.192+0000 7fdc22d9d640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc240ffc40 0x7fdc241025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc2259c640 1 -- 172.21.15.7:0/3725369157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc240ffc40 msgr2=0x7fdc241025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc2259c640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc240ffc40 0x7fdc241025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc2259c640 1 -- 172.21.15.7:0/3725369157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc241011b0 con 0x7fdc24102cc0 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc2259c640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24102cc0 0x7fdc24100c70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d8d0 tx=0x7fdc1800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc0bfff640 1 -- 172.21.15.7:0/3725369157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdc18017070 con 0x7fdc24102cc0 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc0bfff640 1 -- 172.21.15.7:0/3725369157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc18010460 con 0x7fdc24102cc0 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc241014a0 con 0x7fdc24102cc0 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc0bfff640 1 -- 172.21.15.7:0/3725369157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdc1800f5f0 con 0x7fdc24102cc0 2024-09-21T04:38:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.193+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc24101a30 con 0x7fdc24102cc0 2024-09-21T04:38:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.194+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc24100040 con 0x7fdc24102cc0 2024-09-21T04:38:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.195+0000 7fdc0bfff640 1 -- 172.21.15.7:0/3725369157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdc18009a30 con 0x7fdc24102cc0 2024-09-21T04:38:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.195+0000 7fdc0bfff640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc0c07af30 0x7fdc0c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.196+0000 7fdc0bfff640 1 -- 172.21.15.7:0/3725369157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fdc1809dde0 con 0x7fdc24102cc0 2024-09-21T04:38:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.196+0000 7fdc22d9d640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc0c07af30 0x7fdc0c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.196+0000 7fdc22d9d640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc0c07af30 0x7fdc0c07d3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009230 tx=0x7fdc140058d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.200+0000 7fdc0bfff640 1 -- 172.21.15.7:0/3725369157 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fdc18069430 con 0x7fdc24102cc0 2024-09-21T04:38:52.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.405+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdc24062740 con 0x7fdc0c07af30 2024-09-21T04:38:52.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.407+0000 7fdc0bfff640 1 -- 172.21.15.7:0/3725369157 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+175 (secure 0 0 0) 0x7fdc24062740 con 0x7fdc0c07af30 2024-09-21T04:38:52.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc0c07af30 msgr2=0x7fdc0c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc0c07af30 0x7fdc0c07d3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009230 tx=0x7fdc140058d0 comp rx=0 tx=0).stop 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24102cc0 msgr2=0x7fdc24100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24102cc0 0x7fdc24100c70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d8d0 tx=0x7fdc1800dd90 comp rx=0 tx=0).stop 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 shutdown_connections 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc0c07af30 0x7fdc0c07d3f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc24102cc0 0x7fdc24100c70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 --2- 172.21.15.7:0/3725369157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc240ffc40 0x7fdc241025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 >> 172.21.15.7:0/3725369157 conn(0x7fdc240fb770 msgr2=0x7fdc24106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 shutdown_connections 2024-09-21T04:38:52.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:52.409+0000 7fdc290cb640 1 -- 172.21.15.7:0/3725369157 wait complete. 2024-09-21T04:38:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:52 smithi053 ceph-mon[35554]: pgmap v90: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:52 smithi007 ceph-mon[27954]: pgmap v90: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:53.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-09-21T04:38:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:38:53.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.989+0000 7ff1628f7640 1 -- 172.21.15.7:0/3482329686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 msgr2=0x7ff15c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:53.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.989+0000 7ff1628f7640 1 --2- 172.21.15.7:0/3482329686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 0x7ff15c100c50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff144009880 tx=0x7ff14402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.989+0000 7ff1628f7640 1 -- 172.21.15.7:0/3482329686 shutdown_connections 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.989+0000 7ff1628f7640 1 --2- 172.21.15.7:0/3482329686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 0x7ff15c100c50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.989+0000 7ff1628f7640 1 --2- 172.21.15.7:0/3482329686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff15c1067f0 0x7ff15c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.989+0000 7ff1628f7640 1 -- 172.21.15.7:0/3482329686 >> 172.21.15.7:0/3482329686 conn(0x7ff15c0fc550 msgr2=0x7ff15c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.989+0000 7ff1628f7640 1 -- 172.21.15.7:0/3482329686 shutdown_connections 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.989+0000 7ff1628f7640 1 -- 172.21.15.7:0/3482329686 wait complete. 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff1628f7640 1 Processor -- start 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff1628f7640 1 -- start start 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff1628f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 0x7ff15c0fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff1628f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff15c1067f0 0x7ff15c0fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff1628f7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c10dd20 con 0x7ff15c1007f0 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff1628f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c10dba0 con 0x7ff15c1067f0 2024-09-21T04:38:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 0x7ff15c0fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff15bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 0x7ff15c0fccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57946/0 (socket says 172.21.15.7:57946) 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff15bfff640 1 -- 172.21.15.7:0/1520777244 learned_addr learned my addr 172.21.15.7:0/1520777244 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff153fff640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff15c1067f0 0x7ff15c0fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff15bfff640 1 -- 172.21.15.7:0/1520777244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff15c1067f0 msgr2=0x7ff15c0fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff15bfff640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff15c1067f0 0x7ff15c0fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff15bfff640 1 -- 172.21.15.7:0/1520777244 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c0fd750 con 0x7ff15c1007f0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.990+0000 7ff153fff640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff15c1067f0 0x7ff15c0fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.991+0000 7ff15bfff640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 0x7ff15c0fccd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00b420 tx=0x7ff14c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.991+0000 7ff159ffb640 1 -- 172.21.15.7:0/1520777244 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff14c011630 con 0x7ff15c1007f0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.991+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15c0fda40 con 0x7ff15c1007f0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.991+0000 7ff159ffb640 1 -- 172.21.15.7:0/1520777244 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff14c016070 con 0x7ff15c1007f0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.991+0000 7ff159ffb640 1 -- 172.21.15.7:0/1520777244 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff14c010d40 con 0x7ff15c1007f0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.991+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15c1a3f50 con 0x7ff15c1007f0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.992+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15c101f30 con 0x7ff15c1007f0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.993+0000 7ff159ffb640 1 -- 172.21.15.7:0/1520777244 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff14c01b460 con 0x7ff15c1007f0 2024-09-21T04:38:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.993+0000 7ff159ffb640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff14807af40 0x7ff14807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.993+0000 7ff159ffb640 1 -- 172.21.15.7:0/1520777244 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7ff14c09d800 con 0x7ff15c1007f0 2024-09-21T04:38:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.993+0000 7ff153fff640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff14807af40 0x7ff14807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.994+0000 7ff153fff640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff14807af40 0x7ff14807d400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff144031460 tx=0x7ff144004280 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:53.998+0000 7ff159ffb640 1 -- 172.21.15.7:0/1520777244 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7ff14c021250 con 0x7ff15c1007f0 2024-09-21T04:38:54.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:53 smithi053 ceph-mon[35554]: from='client.14902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:53 smithi007 ceph-mon[27954]: from='client.14902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:38:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.257+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ff15c109d90 con 0x7ff15c1007f0 2024-09-21T04:38:54.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.258+0000 7ff159ffb640 1 -- 172.21.15.7:0/1520777244 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7ff14c068db0 con 0x7ff15c1007f0 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff14807af40 msgr2=0x7ff14807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff14807af40 0x7ff14807d400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff144031460 tx=0x7ff144004280 comp rx=0 tx=0).stop 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 msgr2=0x7ff15c0fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 0x7ff15c0fccd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00b420 tx=0x7ff14c00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 shutdown_connections 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff14807af40 0x7ff14807d400 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff15c1067f0 0x7ff15c0fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 --2- 172.21.15.7:0/1520777244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff15c1007f0 0x7ff15c0fccd0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:54.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 >> 172.21.15.7:0/1520777244 conn(0x7ff15c0fc550 msgr2=0x7ff15c069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 shutdown_connections 2024-09-21T04:38:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:54.260+0000 7ff1628f7640 1 -- 172.21.15.7:0/1520777244 wait complete. 2024-09-21T04:38:54.272 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:38:54.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-09-21T04:38:55.187 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:38:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:54 smithi053 ceph-mon[35554]: pgmap v91: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:54 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1520777244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:54 smithi007 ceph-mon[27954]: pgmap v91: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:38:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:54 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1520777244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:55.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.653+0000 7f68e71aa640 1 -- 172.21.15.7:0/1543950894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 msgr2=0x7f68e0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:55.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.653+0000 7f68e71aa640 1 --2- 172.21.15.7:0/1543950894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 0x7f68e0101d60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009880 tx=0x7f68cc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:38:55.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.653+0000 7f68e71aa640 1 -- 172.21.15.7:0/1543950894 shutdown_connections 2024-09-21T04:38:55.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.653+0000 7f68e71aa640 1 --2- 172.21.15.7:0/1543950894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 0x7f68e0101d60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:55.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.653+0000 7f68e71aa640 1 --2- 172.21.15.7:0/1543950894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e0107900 0x7f68e0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:55.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.653+0000 7f68e71aa640 1 -- 172.21.15.7:0/1543950894 >> 172.21.15.7:0/1543950894 conn(0x7f68e00fd600 msgr2=0x7f68e00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:55.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.653+0000 7f68e71aa640 1 -- 172.21.15.7:0/1543950894 shutdown_connections 2024-09-21T04:38:55.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.653+0000 7f68e71aa640 1 -- 172.21.15.7:0/1543950894 wait complete. 2024-09-21T04:38:55.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e71aa640 1 Processor -- start 2024-09-21T04:38:55.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e71aa640 1 -- start start 2024-09-21T04:38:55.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e71aa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 0x7f68e019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:55.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e71aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e0107900 0x7f68e019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:55.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e71aa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010ee30 con 0x7f68e0101900 2024-09-21T04:38:55.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e71aa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010ecb0 con 0x7f68e0107900 2024-09-21T04:38:55.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e4f46640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 0x7f68e019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:55.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e4f46640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 0x7f68e019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57968/0 (socket says 172.21.15.7:57968) 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e4f46640 1 -- 172.21.15.7:0/960706075 learned_addr learned my addr 172.21.15.7:0/960706075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68dffff640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e0107900 0x7f68e019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e4f46640 1 -- 172.21.15.7:0/960706075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e0107900 msgr2=0x7f68e019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e4f46640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e0107900 0x7f68e019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.654+0000 7f68e4f46640 1 -- 172.21.15.7:0/960706075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e0197840 con 0x7f68e0101900 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.655+0000 7f68e4f46640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 0x7f68e019d6c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f68d400b420 tx=0x7f68d400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.655+0000 7f68ddffb640 1 -- 172.21.15.7:0/960706075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f68d4011630 con 0x7f68e0101900 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.655+0000 7f68ddffb640 1 -- 172.21.15.7:0/960706075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f68d4016070 con 0x7f68e0101900 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.655+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e0197b30 con 0x7f68e0101900 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.655+0000 7f68ddffb640 1 -- 172.21.15.7:0/960706075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f68d4010d40 con 0x7f68e0101900 2024-09-21T04:38:55.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.655+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e0198090 con 0x7f68e0101900 2024-09-21T04:38:55.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.656+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e0103040 con 0x7f68e0101900 2024-09-21T04:38:55.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.657+0000 7f68ddffb640 1 -- 172.21.15.7:0/960706075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f68d401b460 con 0x7f68e0101900 2024-09-21T04:38:55.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.657+0000 7f68ddffb640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f68d007af80 0x7f68d007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:55.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.657+0000 7f68ddffb640 1 -- 172.21.15.7:0/960706075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f68d409d5a0 con 0x7f68e0101900 2024-09-21T04:38:55.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.657+0000 7f68dffff640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f68d007af80 0x7f68d007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:55.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.658+0000 7f68dffff640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f68d007af80 0x7f68d007d440 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f68e0198b90 tx=0x7f68cc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.662+0000 7f68ddffb640 1 -- 172.21.15.7:0/960706075 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f68d4068c70 con 0x7f68e0101900 2024-09-21T04:38:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.933+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f68e0109e90 con 0x7f68e0101900 2024-09-21T04:38:55.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.934+0000 7f68ddffb640 1 -- 172.21.15.7:0/960706075 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7f68d406db20 con 0x7f68e0101900 2024-09-21T04:38:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f68d007af80 msgr2=0x7f68d007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:55.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f68d007af80 0x7f68d007d440 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f68e0198b90 tx=0x7f68cc031400 comp rx=0 tx=0).stop 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 msgr2=0x7f68e019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 0x7f68e019d6c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f68d400b420 tx=0x7f68d400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 shutdown_connections 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f68d007af80 0x7f68d007d440 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68e0107900 0x7f68e019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 --2- 172.21.15.7:0/960706075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68e0101900 0x7f68e019d6c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.936+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 >> 172.21.15.7:0/960706075 conn(0x7f68e00fd600 msgr2=0x7f68e00fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.937+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 shutdown_connections 2024-09-21T04:38:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:55.937+0000 7f68e71aa640 1 -- 172.21.15.7:0/960706075 wait complete. 2024-09-21T04:38:55.951 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)" 2024-09-21T04:38:56.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-09-21T04:38:56.881 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:38:57.139 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:56 smithi007 ceph-mon[27954]: pgmap v92: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:57.139 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:56 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/960706075' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:56 smithi053 ceph-mon[35554]: pgmap v92: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:56 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/960706075' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.371+0000 7ffad9704640 1 -- 172.21.15.7:0/2377314256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074f40 msgr2=0x7ffad410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.371+0000 7ffad9704640 1 --2- 172.21.15.7:0/2377314256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074f40 0x7ffad410dc00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009880 tx=0x7ffabc02a530 comp rx=0 tx=0).stop 2024-09-21T04:38:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.371+0000 7ffad9704640 1 -- 172.21.15.7:0/2377314256 shutdown_connections 2024-09-21T04:38:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.371+0000 7ffad9704640 1 --2- 172.21.15.7:0/2377314256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074f40 0x7ffad410dc00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.371+0000 7ffad9704640 1 --2- 172.21.15.7:0/2377314256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad4074620 0x7ffad4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.371+0000 7ffad9704640 1 -- 172.21.15.7:0/2377314256 >> 172.21.15.7:0/2377314256 conn(0x7ffad40fd600 msgr2=0x7ffad40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.371+0000 7ffad9704640 1 -- 172.21.15.7:0/2377314256 shutdown_connections 2024-09-21T04:38:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.371+0000 7ffad9704640 1 -- 172.21.15.7:0/2377314256 wait complete. 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.372+0000 7ffad9704640 1 Processor -- start 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.372+0000 7ffad9704640 1 -- start start 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.372+0000 7ffad9704640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074620 0x7ffad419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.372+0000 7ffad9704640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad4074f40 0x7ffad419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.372+0000 7ffad9704640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4111050 con 0x7ffad4074620 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.372+0000 7ffad9704640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4110ed0 con 0x7ffad4074f40 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.372+0000 7ffad2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074620 0x7ffad419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074620 0x7ffad419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57986/0 (socket says 172.21.15.7:57986) 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad2ffd640 1 -- 172.21.15.7:0/2355595725 learned_addr learned my addr 172.21.15.7:0/2355595725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffaca7fc640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad4074f40 0x7ffad419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad2ffd640 1 -- 172.21.15.7:0/2355595725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad4074f40 msgr2=0x7ffad419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad2ffd640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad4074f40 0x7ffad419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad2ffd640 1 -- 172.21.15.7:0/2355595725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad4199a60 con 0x7ffad4074620 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad2ffd640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074620 0x7ffad419f8a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b420 tx=0x7ffac400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad0ff9640 1 -- 172.21.15.7:0/2355595725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffac4011630 con 0x7ffad4074620 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad4199d50 con 0x7ffad4074620 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad0ff9640 1 -- 172.21.15.7:0/2355595725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffac4016070 con 0x7ffad4074620 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad0ff9640 1 -- 172.21.15.7:0/2355595725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffac4010d40 con 0x7ffad4074620 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.373+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad419a2b0 con 0x7ffad4074620 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.374+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad4075cd0 con 0x7ffad4074620 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.375+0000 7ffad0ff9640 1 -- 172.21.15.7:0/2355595725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ffac401b460 con 0x7ffad4074620 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.376+0000 7ffad0ff9640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffac007af30 0x7ffac007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.376+0000 7ffaca7fc640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffac007af30 0x7ffac007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.376+0000 7ffad0ff9640 1 -- 172.21.15.7:0/2355595725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7ffac409d710 con 0x7ffad4074620 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.376+0000 7ffaca7fc640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffac007af30 0x7ffac007d3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffad419b250 tx=0x7ffabc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.380+0000 7ffad0ff9640 1 -- 172.21.15.7:0/2355595725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7ffac4068cc0 con 0x7ffad4074620 2024-09-21T04:38:57.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.644+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ffad419af90 con 0x7ffad4074620 2024-09-21T04:38:57.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.645+0000 7ffad0ff9640 1 -- 172.21.15.7:0/2355595725 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7ffac406db70 con 0x7ffad4074620 2024-09-21T04:38:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffac007af30 msgr2=0x7ffac007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffac007af30 0x7ffac007d3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffad419b250 tx=0x7ffabc031400 comp rx=0 tx=0).stop 2024-09-21T04:38:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074620 msgr2=0x7ffad419f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074620 0x7ffad419f8a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffac400b420 tx=0x7ffac400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:38:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 shutdown_connections 2024-09-21T04:38:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffac007af30 0x7ffac007d3f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:57.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad4074f40 0x7ffad419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:57.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 --2- 172.21.15.7:0/2355595725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad4074620 0x7ffad419f8a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:57.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 >> 172.21.15.7:0/2355595725 conn(0x7ffad40fd600 msgr2=0x7ffad410b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:57.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 shutdown_connections 2024-09-21T04:38:57.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:57.647+0000 7ffad9704640 1 -- 172.21.15.7:0/2355595725 wait complete. 2024-09-21T04:38:57.659 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:38:57.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:57 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/2355595725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:58.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:57 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/2355595725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:38:58.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-09-21T04:38:58.659 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:38:59.049 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:58 smithi007 ceph-mon[27954]: pgmap v93: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:59.049 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:38:58 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.131+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3890387359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 msgr2=0x7fe59410b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.131+0000 7fe59b3d7640 1 --2- 172.21.15.7:0/3890387359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 0x7fe59410b8c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe588009c20 tx=0x7fe58802aa00 comp rx=0 tx=0).stop 2024-09-21T04:38:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.131+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3890387359 shutdown_connections 2024-09-21T04:38:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.131+0000 7fe59b3d7640 1 --2- 172.21.15.7:0/3890387359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 0x7fe59410b8c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.131+0000 7fe59b3d7640 1 --2- 172.21.15.7:0/3890387359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe594102ab0 0x7fe594102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.131+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3890387359 >> 172.21.15.7:0/3890387359 conn(0x7fe5940fc550 msgr2=0x7fe5940fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:38:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.132+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3890387359 shutdown_connections 2024-09-21T04:38:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.132+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3890387359 wait complete. 2024-09-21T04:38:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.132+0000 7fe59b3d7640 1 Processor -- start 2024-09-21T04:38:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.132+0000 7fe59b3d7640 1 -- start start 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe59b3d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe594102ab0 0x7fe59419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe59b3d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 0x7fe59419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe59b3d7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe59410edc0 con 0x7fe5941033d0 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe59b3d7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe59410ec40 con 0x7fe594102ab0 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe598972640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 0x7fe59419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe598972640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 0x7fe59419db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57994/0 (socket says 172.21.15.7:57994) 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe598972640 1 -- 172.21.15.7:0/3706871279 learned_addr learned my addr 172.21.15.7:0/3706871279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe599173640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe594102ab0 0x7fe59419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe598972640 1 -- 172.21.15.7:0/3706871279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe594102ab0 msgr2=0x7fe59419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe598972640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe594102ab0 0x7fe59419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe598972640 1 -- 172.21.15.7:0/3706871279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe594197790 con 0x7fe5941033d0 2024-09-21T04:38:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.133+0000 7fe598972640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 0x7fe59419db50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe58802af20 tx=0x7fe58802ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.134+0000 7fe5927fc640 1 -- 172.21.15.7:0/3706871279 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe588047070 con 0x7fe5941033d0 2024-09-21T04:38:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.134+0000 7fe5927fc640 1 -- 172.21.15.7:0/3706871279 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe58802eeb0 con 0x7fe5941033d0 2024-09-21T04:38:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.134+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe594197a20 con 0x7fe5941033d0 2024-09-21T04:38:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.134+0000 7fe5927fc640 1 -- 172.21.15.7:0/3706871279 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe588033050 con 0x7fe5941033d0 2024-09-21T04:38:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.134+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe594197f50 con 0x7fe5941033d0 2024-09-21T04:38:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.135+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59406a580 con 0x7fe5941033d0 2024-09-21T04:38:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.136+0000 7fe5927fc640 1 -- 172.21.15.7:0/3706871279 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe58804a070 con 0x7fe5941033d0 2024-09-21T04:38:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.136+0000 7fe5927fc640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe57c07af90 0x7fe57c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:38:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.136+0000 7fe5927fc640 1 -- 172.21.15.7:0/3706871279 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fe58803e030 con 0x7fe5941033d0 2024-09-21T04:38:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.136+0000 7fe599173640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe57c07af90 0x7fe57c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:38:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.137+0000 7fe599173640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe57c07af90 0x7fe57c07d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe58c005d00 tx=0x7fe58c00b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:38:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.141+0000 7fe5927fc640 1 -- 172.21.15.7:0/3706871279 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fe58808a670 con 0x7fe5941033d0 2024-09-21T04:38:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:58 smithi053 ceph-mon[35554]: pgmap v93: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:38:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:38:58 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:38:59.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:38:59.344+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}) -- 0x7fe594198200 con 0x7fe57c07af90 2024-09-21T04:39:01.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.106+0000 7fe5927fc640 1 -- 172.21.15.7:0/3706871279 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+5741 (secure 0 0 0) 0x7fe594198200 con 0x7fe57c07af90 2024-09-21T04:39:01.109 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:00 smithi007 ceph-mon[27954]: from='client.14918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:01.109 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:00 smithi007 ceph-mon[27954]: pgmap v94: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe57c07af90 msgr2=0x7fe57c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe57c07af90 0x7fe57c07d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe58c005d00 tx=0x7fe58c00b410 comp rx=0 tx=0).stop 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 msgr2=0x7fe59419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 0x7fe59419db50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe58802af20 tx=0x7fe58802ed40 comp rx=0 tx=0).stop 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 shutdown_connections 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe57c07af90 0x7fe57c07d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5941033d0 0x7fe59419db50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 --2- 172.21.15.7:0/3706871279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe594102ab0 0x7fe59419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 >> 172.21.15.7:0/3706871279 conn(0x7fe5940fc550 msgr2=0x7fe59410aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 shutdown_connections 2024-09-21T04:39:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:01.108+0000 7fe59b3d7640 1 -- 172.21.15.7:0/3706871279 wait complete. 2024-09-21T04:39:01.121 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:39:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:00 smithi053 ceph-mon[35554]: from='client.14918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:00 smithi053 ceph-mon[35554]: pgmap v94: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:01.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch ps' 2024-09-21T04:39:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:39:02.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.504+0000 7f5acebb0640 1 -- 172.21.15.7:0/2028392185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 msgr2=0x7f5ac810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:02.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.504+0000 7f5acebb0640 1 --2- 172.21.15.7:0/2028392185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 0x7f5ac810b970 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009880 tx=0x7f5ab002a530 comp rx=0 tx=0).stop 2024-09-21T04:39:02.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.504+0000 7f5acebb0640 1 -- 172.21.15.7:0/2028392185 shutdown_connections 2024-09-21T04:39:02.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.504+0000 7f5acebb0640 1 --2- 172.21.15.7:0/2028392185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 0x7f5ac810b970 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:02.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.504+0000 7f5acebb0640 1 --2- 172.21.15.7:0/2028392185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac8102b60 0x7f5ac8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:02.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.504+0000 7f5acebb0640 1 -- 172.21.15.7:0/2028392185 >> 172.21.15.7:0/2028392185 conn(0x7f5ac80fc520 msgr2=0x7f5ac80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.504+0000 7f5acebb0640 1 -- 172.21.15.7:0/2028392185 shutdown_connections 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.504+0000 7f5acebb0640 1 -- 172.21.15.7:0/2028392185 wait complete. 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5acebb0640 1 Processor -- start 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5acebb0640 1 -- start start 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5acebb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac8102b60 0x7f5ac81007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5acebb0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 0x7f5ac8100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5acebb0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac810edf0 con 0x7f5ac8103480 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5acebb0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac810ec70 con 0x7f5ac8102b60 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 0x7f5ac8100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 0x7f5ac8100d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58002/0 (socket says 172.21.15.7:58002) 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5ac7fff640 1 -- 172.21.15.7:0/4238277598 learned_addr learned my addr 172.21.15.7:0/4238277598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5acc94c640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac8102b60 0x7f5ac81007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5ac7fff640 1 -- 172.21.15.7:0/4238277598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac8102b60 msgr2=0x7f5ac81007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5ac7fff640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac8102b60 0x7f5ac81007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5ac7fff640 1 -- 172.21.15.7:0/4238277598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac8101240 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.505+0000 7f5ac7fff640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 0x7f5ac8100d00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0004f90 tx=0x7f5ab00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.506+0000 7f5ac5ffb640 1 -- 172.21.15.7:0/4238277598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5ab003f070 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.506+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac81014d0 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.506+0000 7f5ac5ffb640 1 -- 172.21.15.7:0/4238277598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ab0004c70 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.506+0000 7f5ac5ffb640 1 -- 172.21.15.7:0/4238277598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5ab0035030 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.506+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81a3da0 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.507+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac806a9f0 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.508+0000 7f5ac5ffb640 1 -- 172.21.15.7:0/4238277598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5ab0042070 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.508+0000 7f5ac5ffb640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ab407af30 0x7f5ab407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.508+0000 7f5ac5ffb640 1 -- 172.21.15.7:0/4238277598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f5ab00be9e0 con 0x7f5ac8103480 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.508+0000 7f5acc94c640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ab407af30 0x7f5ab407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:02.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.509+0000 7f5acc94c640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ab407af30 0x7f5ab407d3f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005c40 tx=0x7f5ab8005bd0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:02.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.513+0000 7f5ac5ffb640 1 -- 172.21.15.7:0/4238277598 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f5ab008a030 con 0x7f5ac8103480 2024-09-21T04:39:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.708+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5ac8104fe0 con 0x7f5ab407af30 2024-09-21T04:39:02.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.719+0000 7f5ac5ffb640 1 -- 172.21.15.7:0/4238277598 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f5ab8009dc0 con 0x7f5ab407af30 2024-09-21T04:39:02.721 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (63s) 35s ago 11m 17.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (4m) 35s ago 12m 8740k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (3m) 108s ago 11m 8514k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (12m) 35s ago 12m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (11m) 108s ago 11m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (40s) 35s ago 11m 48.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (6m) 35s ago 6m 146M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (6m) 35s ago 6m 89.3M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (6m) 108s ago 6m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (6m) 108s ago 6m 30.9M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:39:02.722 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (3m) 35s ago 13m 525M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (2m) 108s ago 11m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (13m) 35s ago 13m 60.1M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (11m) 108s ago 11m 47.3M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (118s) 35s ago 12m 11.0M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (111s) 108s ago 11m 3405k - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 35s ago 10m 211M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (9m) 35s ago 9m 213M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (9m) 35s ago 9m 193M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (8m) 108s ago 8m 241M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (8m) 108s ago 8m 210M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (7m) 108s ago 7m 181M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:39:02.723 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (84s) 35s ago 11m 42.9M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ab407af30 msgr2=0x7f5ab407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ab407af30 0x7f5ab407d3f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005c40 tx=0x7f5ab8005bd0 comp rx=0 tx=0).stop 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 msgr2=0x7f5ac8100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 0x7f5ac8100d00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0004f90 tx=0x7f5ab00048f0 comp rx=0 tx=0).stop 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 shutdown_connections 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ab407af30 0x7f5ab407d3f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ac8103480 0x7f5ac8100d00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 --2- 172.21.15.7:0/4238277598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac8102b60 0x7f5ac81007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:02.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 >> 172.21.15.7:0/4238277598 conn(0x7f5ac80fc520 msgr2=0x7f5ac8107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 shutdown_connections 2024-09-21T04:39:02.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:02.721+0000 7f5acebb0640 1 -- 172.21.15.7:0/4238277598 wait complete. 2024-09-21T04:39:03.027 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:02 smithi007 ceph-mon[27954]: pgmap v95: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:03.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:02 smithi053 ceph-mon[35554]: pgmap v95: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:03.374 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-21T04:39:03.374 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:39:03.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-09-21T04:39:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:39:03.919 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:03 smithi007 ceph-mon[27954]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/2592057599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08106800 msgr2=0x7f0f08106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 --2- 172.21.15.7:0/2592057599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08106800 0x7f0f08106be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009880 tx=0x7f0ef802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/2592057599 shutdown_connections 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 --2- 172.21.15.7:0/2592057599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f08100800 0x7f0f08100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 --2- 172.21.15.7:0/2592057599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08106800 0x7f0f08106be0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/2592057599 >> 172.21.15.7:0/2592057599 conn(0x7f0f080fc4e0 msgr2=0x7f0f080fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/2592057599 shutdown_connections 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/2592057599 wait complete. 2024-09-21T04:39:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.111+0000 7f0f0e0d7640 1 Processor -- start 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f0e0d7640 1 -- start start 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f0e0d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08100800 0x7f0f08198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f0e0d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f08106800 0x7f0f08198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f0e0d7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810dcd0 con 0x7f0f08106800 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f0e0d7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810db50 con 0x7f0f08100800 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f06ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f08106800 0x7f0f08198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f06ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f08106800 0x7f0f08198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36936/0 (socket says 172.21.15.7:36936) 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f06ffd640 1 -- 172.21.15.7:0/4093961560 learned_addr learned my addr 172.21.15.7:0/4093961560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f077fe640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08100800 0x7f0f08198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f06ffd640 1 -- 172.21.15.7:0/4093961560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08100800 msgr2=0x7f0f08198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f06ffd640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08100800 0x7f0f08198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f06ffd640 1 -- 172.21.15.7:0/4093961560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f08199390 con 0x7f0f08106800 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f077fe640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08100800 0x7f0f08198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:39:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.112+0000 7f0f06ffd640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f08106800 0x7f0f08198d40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00e970 tx=0x7f0efc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.113+0000 7f0f04ff9640 1 -- 172.21.15.7:0/4093961560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0efc00cac0 con 0x7f0f08106800 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.113+0000 7f0f04ff9640 1 -- 172.21.15.7:0/4093961560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0efc0054e0 con 0x7f0f08106800 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.113+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f081a1870 con 0x7f0f08106800 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.113+0000 7f0f04ff9640 1 -- 172.21.15.7:0/4093961560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0efc010430 con 0x7f0f08106800 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.113+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081a1c70 con 0x7f0f08106800 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.114+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f08101f40 con 0x7f0f08106800 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.115+0000 7f0f04ff9640 1 -- 172.21.15.7:0/4093961560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0efc004e80 con 0x7f0f08106800 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.115+0000 7f0f04ff9640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ef007af30 0x7f0ef007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.115+0000 7f0f04ff9640 1 -- 172.21.15.7:0/4093961560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f0efc09e700 con 0x7f0f08106800 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.115+0000 7f0f077fe640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ef007af30 0x7f0ef007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.116+0000 7f0f077fe640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ef007af30 0x7f0ef007d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.120+0000 7f0f04ff9640 1 -- 172.21.15.7:0/4093961560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f0efc069cf0 con 0x7f0f08106800 2024-09-21T04:39:04.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:03 smithi053 ceph-mon[35554]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:04.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.308+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) -- 0x7f0f08100c60 con 0x7f0f08106800 2024-09-21T04:39:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.312+0000 7f0f04ff9640 1 -- 172.21.15.7:0/4093961560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v63)=0 v63) ==== 125+0+0 (secure 0 0 0) 0x7f0efc06eba0 con 0x7f0f08106800 2024-09-21T04:39:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ef007af30 msgr2=0x7f0ef007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ef007af30 0x7f0ef007d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef8031400 comp rx=0 tx=0).stop 2024-09-21T04:39:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f08106800 msgr2=0x7f0f08198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f08106800 0x7f0f08198d40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00e970 tx=0x7f0efc00ed70 comp rx=0 tx=0).stop 2024-09-21T04:39:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 shutdown_connections 2024-09-21T04:39:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ef007af30 0x7f0ef007d3f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f08106800 0x7f0f08198d40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 --2- 172.21.15.7:0/4093961560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f08100800 0x7f0f08198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.314+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 >> 172.21.15.7:0/4093961560 conn(0x7f0f080fc4e0 msgr2=0x7f0f080fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.315+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 shutdown_connections 2024-09-21T04:39:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:04.315+0000 7f0f0e0d7640 1 -- 172.21.15.7:0/4093961560 wait complete. 2024-09-21T04:39:04.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:04 smithi007 ceph-mon[27954]: pgmap v96: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:04.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:04 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/4093961560' entity='client.admin' 2024-09-21T04:39:04.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:04 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:04.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:04 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:04.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:04 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:39:04.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:04 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:04.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:04 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:39:04.982 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-21T04:39:04.982 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:39:04.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-21T04:39:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:04 smithi053 ceph-mon[35554]: pgmap v96: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:04 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/4093961560' entity='client.admin' 2024-09-21T04:39:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:04 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:04 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:04 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:39:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:04 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:04 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:39:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:39:05.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 -- 172.21.15.7:0/283742206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30102b30 msgr2=0x7fda30102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:05.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 --2- 172.21.15.7:0/283742206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30102b30 0x7fda30102f10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fda28009880 tx=0x7fda2802a530 comp rx=0 tx=0).stop 2024-09-21T04:39:05.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 -- 172.21.15.7:0/283742206 shutdown_connections 2024-09-21T04:39:05.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 --2- 172.21.15.7:0/283742206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30103450 0x7fda3010b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:05.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 --2- 172.21.15.7:0/283742206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30102b30 0x7fda30102f10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 -- 172.21.15.7:0/283742206 >> 172.21.15.7:0/283742206 conn(0x7fda300fc550 msgr2=0x7fda300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 -- 172.21.15.7:0/283742206 shutdown_connections 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 -- 172.21.15.7:0/283742206 wait complete. 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 Processor -- start 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.744+0000 7fda3734a640 1 -- start start 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda3734a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30102b30 0x7fda3019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda3734a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30103450 0x7fda3019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda3734a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda3010ede0 con 0x7fda30103450 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda3734a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda3010ec60 con 0x7fda30102b30 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda348e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30103450 0x7fda3019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda348e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30103450 0x7fda3019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36956/0 (socket says 172.21.15.7:36956) 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda348e5640 1 -- 172.21.15.7:0/2795767177 learned_addr learned my addr 172.21.15.7:0/2795767177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda350e6640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30102b30 0x7fda3019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda348e5640 1 -- 172.21.15.7:0/2795767177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30102b30 msgr2=0x7fda3019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:05.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda348e5640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30102b30 0x7fda3019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:05.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda348e5640 1 -- 172.21.15.7:0/2795767177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda301977f0 con 0x7fda30103450 2024-09-21T04:39:05.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.745+0000 7fda348e5640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30103450 0x7fda3019dbb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fda2400d6f0 tx=0x7fda2400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.746+0000 7fda2e7fc640 1 -- 172.21.15.7:0/2795767177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda24017070 con 0x7fda30103450 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.746+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda30197ae0 con 0x7fda30103450 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.746+0000 7fda2e7fc640 1 -- 172.21.15.7:0/2795767177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fda2400de00 con 0x7fda30103450 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.746+0000 7fda2e7fc640 1 -- 172.21.15.7:0/2795767177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda24004bb0 con 0x7fda30103450 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.746+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda30198070 con 0x7fda30103450 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.747+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3006a500 con 0x7fda30103450 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.747+0000 7fda2e7fc640 1 -- 172.21.15.7:0/2795767177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fda24004e60 con 0x7fda30103450 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.748+0000 7fda2e7fc640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda1807af30 0x7fda1807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.748+0000 7fda2e7fc640 1 -- 172.21.15.7:0/2795767177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fda2409dbf0 con 0x7fda30103450 2024-09-21T04:39:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.748+0000 7fda350e6640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda1807af30 0x7fda1807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:05.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.748+0000 7fda350e6640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda1807af30 0x7fda1807d3f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fda280096c0 tx=0x7fda28004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:05.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.753+0000 7fda2e7fc640 1 -- 172.21.15.7:0/2795767177 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fda240691a0 con 0x7fda30103450 2024-09-21T04:39:05.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.938+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) -- 0x7fda30106000 con 0x7fda30103450 2024-09-21T04:39:05.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:05 smithi007 ceph-mon[27954]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:39:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.938+0000 7fda2e7fc640 1 -- 172.21.15.7:0/2795767177 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v63)=0 v63) ==== 155+0+0 (secure 0 0 0) 0x7fda2406e050 con 0x7fda30103450 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.941+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda1807af30 msgr2=0x7fda1807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.941+0000 7fda3734a640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda1807af30 0x7fda1807d3f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fda280096c0 tx=0x7fda28004f30 comp rx=0 tx=0).stop 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.941+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30103450 msgr2=0x7fda3019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.941+0000 7fda3734a640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30103450 0x7fda3019dbb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fda2400d6f0 tx=0x7fda2400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.942+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 shutdown_connections 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.942+0000 7fda3734a640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda1807af30 0x7fda1807d3f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.942+0000 7fda3734a640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda30103450 0x7fda3019dbb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.942+0000 7fda3734a640 1 --2- 172.21.15.7:0/2795767177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30102b30 0x7fda3019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.942+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 >> 172.21.15.7:0/2795767177 conn(0x7fda300fc550 msgr2=0x7fda30103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.942+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 shutdown_connections 2024-09-21T04:39:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:05.942+0000 7fda3734a640 1 -- 172.21.15.7:0/2795767177 wait complete. 2024-09-21T04:39:06.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:05 smithi053 ceph-mon[35554]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:39:06.618 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-21T04:39:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:39:07.139 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:06 smithi007 ceph-mon[27954]: pgmap v97: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:07.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:06 smithi053 ceph-mon[35554]: pgmap v97: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 -- 172.21.15.7:0/548046482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 msgr2=0x7f52e8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 --2- 172.21.15.7:0/548046482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 0x7f52e8100c60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009880 tx=0x7f52dc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 -- 172.21.15.7:0/548046482 shutdown_connections 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 --2- 172.21.15.7:0/548046482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 0x7f52e8100c60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 --2- 172.21.15.7:0/548046482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e8106800 0x7f52e8106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 -- 172.21.15.7:0/548046482 >> 172.21.15.7:0/548046482 conn(0x7f52e80fc520 msgr2=0x7f52e80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 -- 172.21.15.7:0/548046482 shutdown_connections 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 -- 172.21.15.7:0/548046482 wait complete. 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.366+0000 7f52eef0f640 1 Processor -- start 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eef0f640 1 -- start start 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eef0f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 0x7f52e819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eef0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e8106800 0x7f52e819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eef0f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52e810dcb0 con 0x7f52e8100800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eef0f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52e810db30 con 0x7f52e8106800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eccab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 0x7f52e819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eccab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 0x7f52e819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36988/0 (socket says 172.21.15.7:36988) 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eccab640 1 -- 172.21.15.7:0/2125982469 learned_addr learned my addr 172.21.15.7:0/2125982469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eccab640 1 -- 172.21.15.7:0/2125982469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e8106800 msgr2=0x7f52e819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eccab640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e8106800 0x7f52e819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eccab640 1 -- 172.21.15.7:0/2125982469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e81966c0 con 0x7f52e8100800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.367+0000 7f52eccab640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 0x7f52e819c5f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f52d800d8d0 tx=0x7f52d800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.368+0000 7f52e5ffb640 1 -- 172.21.15.7:0/2125982469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52d8017070 con 0x7f52e8100800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.368+0000 7f52e5ffb640 1 -- 172.21.15.7:0/2125982469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52d8010460 con 0x7f52e8100800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.368+0000 7f52e5ffb640 1 -- 172.21.15.7:0/2125982469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52d800f5f0 con 0x7f52e8100800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.368+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e81969b0 con 0x7f52e8100800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.368+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e8196f10 con 0x7f52e8100800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.369+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e8101f40 con 0x7f52e8100800 2024-09-21T04:39:07.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.369+0000 7f52e5ffb640 1 -- 172.21.15.7:0/2125982469 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f52d8009a30 con 0x7f52e8100800 2024-09-21T04:39:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.370+0000 7f52e5ffb640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52d007af30 0x7f52d007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.370+0000 7f52e5ffb640 1 -- 172.21.15.7:0/2125982469 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f52d809df30 con 0x7f52e8100800 2024-09-21T04:39:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.370+0000 7f52e7fff640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52d007af30 0x7f52d007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.371+0000 7f52e7fff640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52d007af30 0x7f52d007d3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0096c0 tx=0x7f52dc000bc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.375+0000 7f52e5ffb640 1 -- 172.21.15.7:0/2125982469 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f52d80694e0 con 0x7f52e8100800 2024-09-21T04:39:07.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.561+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) -- 0x7f52e8108d90 con 0x7f52e8100800 2024-09-21T04:39:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.561+0000 7f52e5ffb640 1 -- 172.21.15.7:0/2125982469 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v63)=0 v63) ==== 163+0+0 (secure 0 0 0) 0x7f52d806e390 con 0x7f52e8100800 2024-09-21T04:39:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.563+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52d007af30 msgr2=0x7f52d007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.563+0000 7f52eef0f640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52d007af30 0x7f52d007d3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0096c0 tx=0x7f52dc000bc0 comp rx=0 tx=0).stop 2024-09-21T04:39:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.563+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 msgr2=0x7f52e819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.563+0000 7f52eef0f640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 0x7f52e819c5f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f52d800d8d0 tx=0x7f52d800dd90 comp rx=0 tx=0).stop 2024-09-21T04:39:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.563+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 shutdown_connections 2024-09-21T04:39:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.564+0000 7f52eef0f640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52d007af30 0x7f52d007d3f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:07.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.564+0000 7f52eef0f640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e8106800 0x7f52e819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:07.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.564+0000 7f52eef0f640 1 --2- 172.21.15.7:0/2125982469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52e8100800 0x7f52e819c5f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:07.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.564+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 >> 172.21.15.7:0/2125982469 conn(0x7f52e80fc520 msgr2=0x7f52e80fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:07.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.564+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 shutdown_connections 2024-09-21T04:39:07.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:07.564+0000 7f52eef0f640 1 -- 172.21.15.7:0/2125982469 wait complete. 2024-09-21T04:39:08.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-21T04:39:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:39:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.966+0000 7f268ffdc640 1 -- 172.21.15.7:0/1005504579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 msgr2=0x7f2688100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.966+0000 7f268ffdc640 1 --2- 172.21.15.7:0/1005504579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 0x7f2688100a30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2684009880 tx=0x7f268402a490 comp rx=0 tx=0).stop 2024-09-21T04:39:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.966+0000 7f268ffdc640 1 -- 172.21.15.7:0/1005504579 shutdown_connections 2024-09-21T04:39:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.966+0000 7f268ffdc640 1 --2- 172.21.15.7:0/1005504579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2688101000 0x7f268810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.966+0000 7f268ffdc640 1 --2- 172.21.15.7:0/1005504579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 0x7f2688100a30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.966+0000 7f268ffdc640 1 -- 172.21.15.7:0/1005504579 >> 172.21.15.7:0/1005504579 conn(0x7f26880fc520 msgr2=0x7f26880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268ffdc640 1 -- 172.21.15.7:0/1005504579 shutdown_connections 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268ffdc640 1 -- 172.21.15.7:0/1005504579 wait complete. 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268ffdc640 1 Processor -- start 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268ffdc640 1 -- start start 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268ffdc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 0x7f268819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268ffdc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2688101000 0x7f268819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268ffdc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2688110fb0 con 0x7f2688100650 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268ffdc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2688110e30 con 0x7f2688101000 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.967+0000 7f268dd78640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 0x7f268819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268dd78640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 0x7f268819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37010/0 (socket says 172.21.15.7:37010) 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268dd78640 1 -- 172.21.15.7:0/1874035112 learned_addr learned my addr 172.21.15.7:0/1874035112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:08.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268d577640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2688101000 0x7f268819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268dd78640 1 -- 172.21.15.7:0/1874035112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2688101000 msgr2=0x7f268819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268dd78640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2688101000 0x7f268819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268dd78640 1 -- 172.21.15.7:0/1874035112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26881999c0 con 0x7f2688100650 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268d577640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2688101000 0x7f268819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268dd78640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 0x7f268819f800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2684031460 tx=0x7f26840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f267effd640 1 -- 172.21.15.7:0/1874035112 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f268403f070 con 0x7f2688100650 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f267effd640 1 -- 172.21.15.7:0/1874035112 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2684004cf0 con 0x7f2688100650 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f267effd640 1 -- 172.21.15.7:0/1874035112 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2684035030 con 0x7f2688100650 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2688199c50 con 0x7f2688100650 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.968+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268819a1e0 con 0x7f2688100650 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.969+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268806aa90 con 0x7f2688100650 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.970+0000 7f267effd640 1 -- 172.21.15.7:0/1874035112 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2684042070 con 0x7f2688100650 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.970+0000 7f267effd640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f267407af30 0x7f267407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.971+0000 7f267effd640 1 -- 172.21.15.7:0/1874035112 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f26840beb10 con 0x7f2688100650 2024-09-21T04:39:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.971+0000 7f268d577640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f267407af30 0x7f267407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.971+0000 7f268d577640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f267407af30 0x7f267407d3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2678005fd0 tx=0x7f2678005ea0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:08.975+0000 7f267effd640 1 -- 172.21.15.7:0/1874035112 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f268408a0c0 con 0x7f2688100650 2024-09-21T04:39:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.161+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) -- 0x7f268819aef0 con 0x7f2688100650 2024-09-21T04:39:09.164 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:08 smithi007 ceph-mon[27954]: pgmap v98: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.161+0000 7f267effd640 1 -- 172.21.15.7:0/1874035112 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v63)=0 v63) ==== 135+0+0 (secure 0 0 0) 0x7f268408ef70 con 0x7f2688100650 2024-09-21T04:39:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f267407af30 msgr2=0x7f267407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f267407af30 0x7f267407d3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2678005fd0 tx=0x7f2678005ea0 comp rx=0 tx=0).stop 2024-09-21T04:39:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 msgr2=0x7f268819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 0x7f268819f800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2684031460 tx=0x7f26840048f0 comp rx=0 tx=0).stop 2024-09-21T04:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 shutdown_connections 2024-09-21T04:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f267407af30 0x7f267407d3f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2688101000 0x7f268819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 --2- 172.21.15.7:0/1874035112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2688100650 0x7f268819f800 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 >> 172.21.15.7:0/1874035112 conn(0x7f26880fc520 msgr2=0x7f26880fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 shutdown_connections 2024-09-21T04:39:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:09.163+0000 7f268ffdc640 1 -- 172.21.15.7:0/1874035112 wait complete. 2024-09-21T04:39:09.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:08 smithi053 ceph-mon[35554]: pgmap v98: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:09.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-09-21T04:39:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:39:10.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.557+0000 7faaac944640 1 -- 172.21.15.7:0/2772969425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 msgr2=0x7faaa810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.557+0000 7faaac944640 1 --2- 172.21.15.7:0/2772969425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 0x7faaa810b8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009880 tx=0x7faa9c02a530 comp rx=0 tx=0).stop 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.558+0000 7faaac944640 1 -- 172.21.15.7:0/2772969425 shutdown_connections 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.558+0000 7faaac944640 1 --2- 172.21.15.7:0/2772969425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 0x7faaa810b8d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.558+0000 7faaac944640 1 --2- 172.21.15.7:0/2772969425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa8102ac0 0x7faaa8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.558+0000 7faaac944640 1 -- 172.21.15.7:0/2772969425 >> 172.21.15.7:0/2772969425 conn(0x7faaa80fc520 msgr2=0x7faaa80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.558+0000 7faaac944640 1 -- 172.21.15.7:0/2772969425 shutdown_connections 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.558+0000 7faaac944640 1 -- 172.21.15.7:0/2772969425 wait complete. 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.558+0000 7faaac944640 1 Processor -- start 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.558+0000 7faaac944640 1 -- start start 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaac944640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa8102ac0 0x7faaa819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaac944640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 0x7faaa819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaac944640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faaa810edf0 con 0x7faaa81033e0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaac944640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faaa810ec70 con 0x7faaa8102ac0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaa5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 0x7faaa819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaa5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 0x7faaa819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37034/0 (socket says 172.21.15.7:37034) 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaa5d9b640 1 -- 172.21.15.7:0/3847602952 learned_addr learned my addr 172.21.15.7:0/3847602952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaa659c640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa8102ac0 0x7faaa819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaa5d9b640 1 -- 172.21.15.7:0/3847602952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa8102ac0 msgr2=0x7faaa819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaa5d9b640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa8102ac0 0x7faaa819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaa5d9b640 1 -- 172.21.15.7:0/3847602952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa8197800 con 0x7faaa81033e0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faaa5d9b640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 0x7faaa819dbc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faa9c031460 tx=0x7faa9c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faa977fe640 1 -- 172.21.15.7:0/3847602952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faa9c03f070 con 0x7faaa81033e0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.559+0000 7faa977fe640 1 -- 172.21.15.7:0/3847602952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa9c0035f0 con 0x7faaa81033e0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.560+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa8197a90 con 0x7faaa81033e0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.560+0000 7faa977fe640 1 -- 172.21.15.7:0/3847602952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faa9c035030 con 0x7faaa81033e0 2024-09-21T04:39:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.560+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa8197fc0 con 0x7faaa81033e0 2024-09-21T04:39:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.561+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa806aa90 con 0x7faaa81033e0 2024-09-21T04:39:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.561+0000 7faa977fe640 1 -- 172.21.15.7:0/3847602952 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7faa9c042070 con 0x7faaa81033e0 2024-09-21T04:39:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.562+0000 7faa977fe640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa9007af30 0x7faa9007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.562+0000 7faaa659c640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa9007af30 0x7faa9007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.562+0000 7faa977fe640 1 -- 172.21.15.7:0/3847602952 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7faa9c0be7f0 con 0x7faaa81033e0 2024-09-21T04:39:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.563+0000 7faaa659c640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa9007af30 0x7faa9007d3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa98009bd0 tx=0x7faa98005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.567+0000 7faa977fe640 1 -- 172.21.15.7:0/3847602952 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7faa9c089e40 con 0x7faaa81033e0 2024-09-21T04:39:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.773+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}) -- 0x7faaa8198270 con 0x7faa9007af30 2024-09-21T04:39:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.779+0000 7faa977fe640 1 -- 172.21.15.7:0/3847602952 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+89 (secure 0 0 0) 0x7faaa8198270 con 0x7faa9007af30 2024-09-21T04:39:10.783 INFO:teuthology.orchestra.run.smithi007.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:39:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa9007af30 msgr2=0x7faa9007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa9007af30 0x7faa9007d3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa98009bd0 tx=0x7faa98005f90 comp rx=0 tx=0).stop 2024-09-21T04:39:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 msgr2=0x7faaa819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 0x7faaa819dbc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faa9c031460 tx=0x7faa9c0045c0 comp rx=0 tx=0).stop 2024-09-21T04:39:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 shutdown_connections 2024-09-21T04:39:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faa9007af30 0x7faa9007d3f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa81033e0 0x7faaa819dbc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 --2- 172.21.15.7:0/3847602952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faaa8102ac0 0x7faaa819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 >> 172.21.15.7:0/3847602952 conn(0x7faaa80fc520 msgr2=0x7faaa8108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 shutdown_connections 2024-09-21T04:39:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:10.782+0000 7faaac944640 1 -- 172.21.15.7:0/3847602952 wait complete. 2024-09-21T04:39:11.087 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:10 smithi007 ceph-mon[27954]: pgmap v99: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:11.087 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:10 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:11.087 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:10 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:11.087 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:10 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:11.087 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:10 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:39:11.087 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:10 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:10 smithi053 ceph-mon[35554]: pgmap v99: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:10 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:10 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:10 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:10 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:39:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:10 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:11.426 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-21T04:39:11.426 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T04:39:11.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-09-21T04:39:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T04:39:12.131 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:11 smithi007 ceph-mon[27954]: from='client.14942 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:12.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:11 smithi007 ceph-mon[27954]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:39:12.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:11 smithi007 ceph-mon[27954]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:39:12.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:11 smithi053 ceph-mon[35554]: from='client.14942 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:12.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:11 smithi053 ceph-mon[35554]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:39:12.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:11 smithi053 ceph-mon[35554]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 2024-09-21T04:39:12.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.552+0000 7f216ff6c640 1 -- 172.21.15.7:0/3092350392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681067f0 msgr2=0x7f2168106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.552+0000 7f216ff6c640 1 --2- 172.21.15.7:0/3092350392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681067f0 0x7f2168106bd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2164009880 tx=0x7f216402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.552+0000 7f216ff6c640 1 -- 172.21.15.7:0/3092350392 shutdown_connections 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.552+0000 7f216ff6c640 1 --2- 172.21.15.7:0/3092350392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21681007f0 0x7f2168100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.552+0000 7f216ff6c640 1 --2- 172.21.15.7:0/3092350392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681067f0 0x7f2168106bd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.552+0000 7f216ff6c640 1 -- 172.21.15.7:0/3092350392 >> 172.21.15.7:0/3092350392 conn(0x7f21680fc550 msgr2=0x7f21680fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.552+0000 7f216ff6c640 1 -- 172.21.15.7:0/3092350392 shutdown_connections 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.552+0000 7f216ff6c640 1 -- 172.21.15.7:0/3092350392 wait complete. 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216ff6c640 1 Processor -- start 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216ff6c640 1 -- start start 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216ff6c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681007f0 0x7f21680fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216ff6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21681067f0 0x7f21680fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216ff6c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f216810dcc0 con 0x7f21681007f0 2024-09-21T04:39:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216ff6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f216810db40 con 0x7f21681067f0 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216dd08640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681007f0 0x7f21680fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216dd08640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681007f0 0x7f21680fcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37046/0 (socket says 172.21.15.7:37046) 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216dd08640 1 -- 172.21.15.7:0/2142888140 learned_addr learned my addr 172.21.15.7:0/2142888140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216d507640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21681067f0 0x7f21680fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216dd08640 1 -- 172.21.15.7:0/2142888140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21681067f0 msgr2=0x7f21680fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216dd08640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21681067f0 0x7f21680fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216dd08640 1 -- 172.21.15.7:0/2142888140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21680fd7b0 con 0x7f21681007f0 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.553+0000 7f216dd08640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681007f0 0x7f21680fcd30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2164031460 tx=0x7f21640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.554+0000 7f215effd640 1 -- 172.21.15.7:0/2142888140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f216403f070 con 0x7f21681007f0 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.554+0000 7f215effd640 1 -- 172.21.15.7:0/2142888140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2164004cf0 con 0x7f21681007f0 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.554+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21680fda40 con 0x7f21681007f0 2024-09-21T04:39:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.554+0000 7f215effd640 1 -- 172.21.15.7:0/2142888140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2164035030 con 0x7f21681007f0 2024-09-21T04:39:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.554+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21681a3d70 con 0x7f21681007f0 2024-09-21T04:39:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.555+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2168101f30 con 0x7f21681007f0 2024-09-21T04:39:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.556+0000 7f215effd640 1 -- 172.21.15.7:0/2142888140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2164042070 con 0x7f21681007f0 2024-09-21T04:39:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.556+0000 7f215effd640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f215407af80 0x7f215407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.556+0000 7f215effd640 1 -- 172.21.15.7:0/2142888140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f21640bea30 con 0x7f21681007f0 2024-09-21T04:39:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.556+0000 7f216d507640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f215407af80 0x7f215407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.557+0000 7f216d507640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f215407af80 0x7f215407d440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2158005fd0 tx=0x7f2158005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:12.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.561+0000 7f215effd640 1 -- 172.21.15.7:0/2142888140 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f21640c0e20 con 0x7f21681007f0 2024-09-21T04:39:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.767+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2168062630 con 0x7f215407af80 2024-09-21T04:39:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.768+0000 7f215effd640 1 -- 172.21.15.7:0/2142888140 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+358 (secure 0 0 0) 0x7f2168062630 con 0x7f215407af80 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f215407af80 msgr2=0x7f215407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f215407af80 0x7f215407d440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2158005fd0 tx=0x7f2158005e20 comp rx=0 tx=0).stop 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681007f0 msgr2=0x7f21680fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681007f0 0x7f21680fcd30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2164031460 tx=0x7f21640048f0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 shutdown_connections 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f215407af80 0x7f215407d440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21681067f0 0x7f21680fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 --2- 172.21.15.7:0/2142888140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21681007f0 0x7f21680fcd30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 >> 172.21.15.7:0/2142888140 conn(0x7f21680fc550 msgr2=0x7f2168069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 shutdown_connections 2024-09-21T04:39:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.771+0000 7f216ff6c640 1 -- 172.21.15.7:0/2142888140 wait complete. 2024-09-21T04:39:12.783 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:39:12.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.856+0000 7f0434af3640 1 -- 172.21.15.7:0/1354996395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430107900 msgr2=0x7f0430107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:12.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.856+0000 7f0434af3640 1 --2- 172.21.15.7:0/1354996395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430107900 0x7f0430107ce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0420009880 tx=0x7f042002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.856+0000 7f0434af3640 1 -- 172.21.15.7:0/1354996395 shutdown_connections 2024-09-21T04:39:12.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.856+0000 7f0434af3640 1 --2- 172.21.15.7:0/1354996395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430101900 0x7f0430101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.856+0000 7f0434af3640 1 --2- 172.21.15.7:0/1354996395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430107900 0x7f0430107ce0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.856+0000 7f0434af3640 1 -- 172.21.15.7:0/1354996395 >> 172.21.15.7:0/1354996395 conn(0x7f04300fd600 msgr2=0x7f04300ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:12.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.856+0000 7f0434af3640 1 -- 172.21.15.7:0/1354996395 shutdown_connections 2024-09-21T04:39:12.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.856+0000 7f0434af3640 1 -- 172.21.15.7:0/1354996395 wait complete. 2024-09-21T04:39:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f0434af3640 1 Processor -- start 2024-09-21T04:39:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f0434af3640 1 -- start start 2024-09-21T04:39:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f0434af3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430101900 0x7f043019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f0434af3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430107900 0x7f043019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f0434af3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f043010ee30 con 0x7f0430101900 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f0434af3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f043010ecb0 con 0x7f0430107900 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f042e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430101900 0x7f043019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f042e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430101900 0x7f043019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37050/0 (socket says 172.21.15.7:37050) 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.857+0000 7f042e59c640 1 -- 172.21.15.7:0/1753711625 learned_addr learned my addr 172.21.15.7:0/1753711625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f042dd9b640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430107900 0x7f043019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f042e59c640 1 -- 172.21.15.7:0/1753711625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430107900 msgr2=0x7f043019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f042e59c640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430107900 0x7f043019dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f042e59c640 1 -- 172.21.15.7:0/1753711625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0430197810 con 0x7f0430101900 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f042dd9b640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430107900 0x7f043019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f042e59c640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430101900 0x7f043019d690 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0420031460 tx=0x7f04200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f041f7fe640 1 -- 172.21.15.7:0/1753711625 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f042003f070 con 0x7f0430101900 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f041f7fe640 1 -- 172.21.15.7:0/1753711625 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0420004cf0 con 0x7f0430101900 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0430197aa0 con 0x7f0430101900 2024-09-21T04:39:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f041f7fe640 1 -- 172.21.15.7:0/1753711625 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0420035030 con 0x7f0430101900 2024-09-21T04:39:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.858+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0430197fd0 con 0x7f0430101900 2024-09-21T04:39:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.859+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0430103040 con 0x7f0430101900 2024-09-21T04:39:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.860+0000 7f041f7fe640 1 -- 172.21.15.7:0/1753711625 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0420042070 con 0x7f0430101900 2024-09-21T04:39:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.860+0000 7f041f7fe640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f041807af80 0x7f041807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.861+0000 7f041f7fe640 1 -- 172.21.15.7:0/1753711625 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f04200be990 con 0x7f0430101900 2024-09-21T04:39:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.861+0000 7f042dd9b640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f041807af80 0x7f041807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.861+0000 7f042dd9b640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f041807af80 0x7f041807d440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0424006fd0 tx=0x7f0424008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:12.865+0000 7f041f7fe640 1 -- 172.21.15.7:0/1753711625 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f042008a060 con 0x7f0430101900 2024-09-21T04:39:13.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:12 smithi007 ceph-mon[27954]: pgmap v100: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.071+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0430198280 con 0x7f041807af80 2024-09-21T04:39:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.072+0000 7f041f7fe640 1 -- 172.21.15.7:0/1753711625 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+358 (secure 0 0 0) 0x7f0430198280 con 0x7f041807af80 2024-09-21T04:39:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f041807af80 msgr2=0x7f041807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f041807af80 0x7f041807d440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0424006fd0 tx=0x7f0424008040 comp rx=0 tx=0).stop 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430101900 msgr2=0x7f043019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430101900 0x7f043019d690 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0420031460 tx=0x7f04200048f0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 shutdown_connections 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f041807af80 0x7f041807d440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430107900 0x7f043019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 --2- 172.21.15.7:0/1753711625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0430101900 0x7f043019d690 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.074+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 >> 172.21.15.7:0/1753711625 conn(0x7f04300fd600 msgr2=0x7f04300fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.075+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 shutdown_connections 2024-09-21T04:39:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.075+0000 7f0434af3640 1 -- 172.21.15.7:0/1753711625 wait complete. 2024-09-21T04:39:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 -- 172.21.15.7:0/2337573556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 msgr2=0x7f87c0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 --2- 172.21.15.7:0/2337573556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 0x7f87c0107ce0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009880 tx=0x7f87b002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 -- 172.21.15.7:0/2337573556 shutdown_connections 2024-09-21T04:39:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 --2- 172.21.15.7:0/2337573556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0101900 0x7f87c0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 --2- 172.21.15.7:0/2337573556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 0x7f87c0107ce0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 -- 172.21.15.7:0/2337573556 >> 172.21.15.7:0/2337573556 conn(0x7f87c00fd600 msgr2=0x7f87c00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 -- 172.21.15.7:0/2337573556 shutdown_connections 2024-09-21T04:39:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 -- 172.21.15.7:0/2337573556 wait complete. 2024-09-21T04:39:13.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.161+0000 7f87c71ee640 1 Processor -- start 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87c71ee640 1 -- start start 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87c71ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0101900 0x7f87c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87c71ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 0x7f87c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87c71ee640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87c010ee30 con 0x7f87c0107900 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87c71ee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f87c010ecb0 con 0x7f87c0101900 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 0x7f87c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 0x7f87c0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37072/0 (socket says 172.21.15.7:37072) 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87bffff640 1 -- 172.21.15.7:0/864013450 learned_addr learned my addr 172.21.15.7:0/864013450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87c4f8a640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0101900 0x7f87c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87bffff640 1 -- 172.21.15.7:0/864013450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0101900 msgr2=0x7f87c00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87bffff640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0101900 0x7f87c00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87bffff640 1 -- 172.21.15.7:0/864013450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c0074f70 con 0x7f87c0107900 2024-09-21T04:39:13.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87c4f8a640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0101900 0x7f87c00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.162+0000 7f87bffff640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 0x7f87c0074a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f87b400b420 tx=0x7f87b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.163+0000 7f87bdffb640 1 -- 172.21.15.7:0/864013450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f87b4011630 con 0x7f87c0107900 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.163+0000 7f87bdffb640 1 -- 172.21.15.7:0/864013450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87b4016070 con 0x7f87c0107900 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.163+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c0075260 con 0x7f87c0107900 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.163+0000 7f87bdffb640 1 -- 172.21.15.7:0/864013450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f87b4010d40 con 0x7f87c0107900 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.163+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c01a3ee0 con 0x7f87c0107900 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.164+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c0103040 con 0x7f87c0107900 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.164+0000 7f87bdffb640 1 -- 172.21.15.7:0/864013450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f87b401b460 con 0x7f87c0107900 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.165+0000 7f87bdffb640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87a807af80 0x7f87a807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.165+0000 7f87bdffb640 1 -- 172.21.15.7:0/864013450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f87b409d650 con 0x7f87c0107900 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.165+0000 7f87c4f8a640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87a807af80 0x7f87a807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.166+0000 7f87c4f8a640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87a807af80 0x7f87a807d440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87b0006010 tx=0x7f87b00095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:13.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.169+0000 7f87bdffb640 1 -- 172.21.15.7:0/864013450 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f87b4021250 con 0x7f87c0107900 2024-09-21T04:39:13.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:12 smithi053 ceph-mon[35554]: pgmap v100: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.366+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f87c0062630 con 0x7f87a807af80 2024-09-21T04:39:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.375+0000 7f87bdffb640 1 -- 172.21.15.7:0/864013450 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f87c0062630 con 0x7f87a807af80 2024-09-21T04:39:13.379 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:39:13.379 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (74s) 45s ago 12m 17.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (4m) 45s ago 12m 8740k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (3m) 119s ago 11m 8514k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (12m) 45s ago 12m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (11m) 119s ago 11m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (51s) 45s ago 11m 48.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (6m) 45s ago 6m 146M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (6m) 45s ago 6m 89.3M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (6m) 119s ago 6m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (6m) 119s ago 6m 30.9M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (3m) 45s ago 13m 525M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (2m) 119s ago 11m 453M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (13m) 45s ago 13m 60.1M 2048M 19.1.1-373-g4e45e6b4 164465d2334a 1fc3cec404f2 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (11m) 119s ago 11m 47.3M 2048M 19.1.1-373-g4e45e6b4 164465d2334a d92ed5c08206 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (2m) 45s ago 12m 11.0M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:39:13.380 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (2m) 119s ago 11m 3405k - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:39:13.381 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 45s ago 10m 211M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:39:13.381 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (9m) 45s ago 9m 213M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:39:13.381 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (9m) 45s ago 9m 193M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:39:13.381 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (8m) 119s ago 8m 241M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:39:13.381 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (8m) 119s ago 8m 210M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:39:13.381 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (7m) 119s ago 7m 181M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:39:13.381 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (95s) 45s ago 11m 42.9M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87a807af80 msgr2=0x7f87a807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87a807af80 0x7f87a807d440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87b0006010 tx=0x7f87b00095f0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 msgr2=0x7f87c0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 0x7f87c0074a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f87b400b420 tx=0x7f87b400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 shutdown_connections 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87a807af80 0x7f87a807d440 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c0107900 0x7f87c0074a30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 --2- 172.21.15.7:0/864013450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c0101900 0x7f87c00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 >> 172.21.15.7:0/864013450 conn(0x7f87c00fd600 msgr2=0x7f87c00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 shutdown_connections 2024-09-21T04:39:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.378+0000 7f87c71ee640 1 -- 172.21.15.7:0/864013450 wait complete. 2024-09-21T04:39:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/2582612504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4106800 msgr2=0x7fbaf4106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 --2- 172.21.15.7:0/2582612504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4106800 0x7fbaf4106be0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009880 tx=0x7fbae402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/2582612504 shutdown_connections 2024-09-21T04:39:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 --2- 172.21.15.7:0/2582612504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaf4100800 0x7fbaf4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 --2- 172.21.15.7:0/2582612504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4106800 0x7fbaf4106be0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/2582612504 >> 172.21.15.7:0/2582612504 conn(0x7fbaf40fc520 msgr2=0x7fbaf40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/2582612504 shutdown_connections 2024-09-21T04:39:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/2582612504 wait complete. 2024-09-21T04:39:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.463+0000 7fbaf9c9b640 1 Processor -- start 2024-09-21T04:39:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf9c9b640 1 -- start start 2024-09-21T04:39:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf9c9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4100800 0x7fbaf419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf9c9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaf4106800 0x7fbaf419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf9c9b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf410dcd0 con 0x7fbaf4100800 2024-09-21T04:39:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf9c9b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf410db50 con 0x7fbaf4106800 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4100800 0x7fbaf419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4100800 0x7fbaf419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59050/0 (socket says 172.21.15.7:59050) 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf37fe640 1 -- 172.21.15.7:0/1934032987 learned_addr learned my addr 172.21.15.7:0/1934032987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf2ffd640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaf4106800 0x7fbaf419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf37fe640 1 -- 172.21.15.7:0/1934032987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaf4106800 msgr2=0x7fbaf419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf37fe640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaf4106800 0x7fbaf419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf37fe640 1 -- 172.21.15.7:0/1934032987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf41977c0 con 0x7fbaf4100800 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.464+0000 7fbaf2ffd640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaf4106800 0x7fbaf419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.465+0000 7fbaf37fe640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4100800 0x7fbaf419d640 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbae4031460 tx=0x7fbae40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.465+0000 7fbaf0ff9640 1 -- 172.21.15.7:0/1934032987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbae403f070 con 0x7fbaf4100800 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.465+0000 7fbaf0ff9640 1 -- 172.21.15.7:0/1934032987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbae40035f0 con 0x7fbaf4100800 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.465+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf4197a50 con 0x7fbaf4100800 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.465+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf4197f80 con 0x7fbaf4100800 2024-09-21T04:39:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.465+0000 7fbaf0ff9640 1 -- 172.21.15.7:0/1934032987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbae4035030 con 0x7fbaf4100800 2024-09-21T04:39:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.466+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae0005290 con 0x7fbaf4100800 2024-09-21T04:39:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.467+0000 7fbaf0ff9640 1 -- 172.21.15.7:0/1934032987 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbae4042070 con 0x7fbaf4100800 2024-09-21T04:39:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.467+0000 7fbaf0ff9640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbadc07af80 0x7fbadc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.468+0000 7fbaf0ff9640 1 -- 172.21.15.7:0/1934032987 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fbae40be810 con 0x7fbaf4100800 2024-09-21T04:39:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.468+0000 7fbaf2ffd640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbadc07af80 0x7fbadc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.468+0000 7fbaf2ffd640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbadc07af80 0x7fbadc07d440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbae80058c0 tx=0x7fbae8005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.472+0000 7fbaf0ff9640 1 -- 172.21.15.7:0/1934032987 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fbae4089ee0 con 0x7fbaf4100800 2024-09-21T04:39:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.741+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fbae0005d90 con 0x7fbaf4100800 2024-09-21T04:39:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.741+0000 7fbaf0ff9640 1 -- 172.21.15.7:0/1934032987 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7fbae408ed90 con 0x7fbaf4100800 2024-09-21T04:39:13.745 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:39:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 12, 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:39:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:39:13.747 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:39:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbadc07af80 msgr2=0x7fbadc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbadc07af80 0x7fbadc07d440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbae80058c0 tx=0x7fbae8005850 comp rx=0 tx=0).stop 2024-09-21T04:39:13.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4100800 msgr2=0x7fbaf419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4100800 0x7fbaf419d640 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbae4031460 tx=0x7fbae40045e0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 shutdown_connections 2024-09-21T04:39:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbadc07af80 0x7fbadc07d440 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbaf4106800 0x7fbaf419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 --2- 172.21.15.7:0/1934032987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf4100800 0x7fbaf419d640 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 >> 172.21.15.7:0/1934032987 conn(0x7fbaf40fc520 msgr2=0x7fbaf406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 shutdown_connections 2024-09-21T04:39:13.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.744+0000 7fbaf9c9b640 1 -- 172.21.15.7:0/1934032987 wait complete. 2024-09-21T04:39:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 -- 172.21.15.7:0/2670976845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c106810 msgr2=0x7f176c106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 --2- 172.21.15.7:0/2670976845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c106810 0x7f176c106bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f175c009880 tx=0x7f175c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 -- 172.21.15.7:0/2670976845 shutdown_connections 2024-09-21T04:39:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 --2- 172.21.15.7:0/2670976845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c100810 0x7f176c100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 --2- 172.21.15.7:0/2670976845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c106810 0x7f176c106bf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 -- 172.21.15.7:0/2670976845 >> 172.21.15.7:0/2670976845 conn(0x7f176c0fc510 msgr2=0x7f176c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 -- 172.21.15.7:0/2670976845 shutdown_connections 2024-09-21T04:39:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 -- 172.21.15.7:0/2670976845 wait complete. 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.829+0000 7f1771b0c640 1 Processor -- start 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f1771b0c640 1 -- start start 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f1771b0c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c100810 0x7f176c0fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f1771b0c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c106810 0x7f176c0fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f1771b0c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f176c10dce0 con 0x7f176c100810 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f1771b0c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f176c10db60 con 0x7f176c106810 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f176b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c100810 0x7f176c0fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f176b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c100810 0x7f176c0fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59054/0 (socket says 172.21.15.7:59054) 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f176b7fe640 1 -- 172.21.15.7:0/3663431636 learned_addr learned my addr 172.21.15.7:0/3663431636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f176affd640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c106810 0x7f176c0fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f176b7fe640 1 -- 172.21.15.7:0/3663431636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c106810 msgr2=0x7f176c0fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f176b7fe640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c106810 0x7f176c0fd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f176b7fe640 1 -- 172.21.15.7:0/3663431636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c0fd670 con 0x7f176c100810 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.830+0000 7f176b7fe640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c100810 0x7f176c0fcbf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f175c031460 tx=0x7f175c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.831+0000 7f1768ff9640 1 -- 172.21.15.7:0/3663431636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f175c03f070 con 0x7f176c100810 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.831+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c0fd900 con 0x7f176c100810 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.831+0000 7f1768ff9640 1 -- 172.21.15.7:0/3663431636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f175c0035f0 con 0x7f176c100810 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.831+0000 7f1768ff9640 1 -- 172.21.15.7:0/3663431636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f175c035030 con 0x7f176c100810 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.831+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c1a3e10 con 0x7f176c100810 2024-09-21T04:39:13.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.832+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176c101f50 con 0x7f176c100810 2024-09-21T04:39:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.833+0000 7f1768ff9640 1 -- 172.21.15.7:0/3663431636 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f175c042070 con 0x7f176c100810 2024-09-21T04:39:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.834+0000 7f1768ff9640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f175407af30 0x7f175407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.834+0000 7f1768ff9640 1 -- 172.21.15.7:0/3663431636 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f175c0be920 con 0x7f176c100810 2024-09-21T04:39:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.834+0000 7f176affd640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f175407af30 0x7f175407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.835+0000 7f176affd640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f175407af30 0x7f175407d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f176c0fe680 tx=0x7f176000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:13.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:13.840+0000 7f1768ff9640 1 -- 172.21.15.7:0/3663431636 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f175c089f70 con 0x7f176c100810 2024-09-21T04:39:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.097+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f176c02d7c0 con 0x7f176c100810 2024-09-21T04:39:14.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:13 smithi007 ceph-mon[27954]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:14.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:13 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:39:14.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:13 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1934032987' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:39:14.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:13 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:14.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:13 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:14.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:13 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:39:14.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:13 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:14.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:13 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.098+0000 7f1768ff9640 1 -- 172.21.15.7:0/3663431636 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2038 (secure 0 0 0) 0x7f175c08ee20 con 0x7f176c100810 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:39:14.103 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 44 2024-09-21T04:39:14.104 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14578,1=24327} 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:14.105 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:14.106 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:39:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:39:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:39:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.100+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f175407af30 msgr2=0x7f175407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.100+0000 7f1771b0c640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f175407af30 0x7f175407d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f176c0fe680 tx=0x7f176000b390 comp rx=0 tx=0).stop 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.100+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c100810 msgr2=0x7f176c0fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.100+0000 7f1771b0c640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c100810 0x7f176c0fcbf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f175c031460 tx=0x7f175c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.100+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 shutdown_connections 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.101+0000 7f1771b0c640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f175407af30 0x7f175407d3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.101+0000 7f1771b0c640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f176c106810 0x7f176c0fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.101+0000 7f1771b0c640 1 --2- 172.21.15.7:0/3663431636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176c100810 0x7f176c0fcbf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.101+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 >> 172.21.15.7:0/3663431636 conn(0x7f176c0fc510 msgr2=0x7f176c06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.101+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 shutdown_connections 2024-09-21T04:39:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.101+0000 7f1771b0c640 1 -- 172.21.15.7:0/3663431636 wait complete. 2024-09-21T04:39:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:13 smithi053 ceph-mon[35554]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:13 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:39:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:13 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1934032987' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:39:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:13 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:13 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:13 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:39:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:13 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:13 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.208+0000 7f1fab15b640 1 -- 172.21.15.7:0/3202737974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4101900 msgr2=0x7f1fa4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.208+0000 7f1fab15b640 1 --2- 172.21.15.7:0/3202737974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4101900 0x7f1fa4101d60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009880 tx=0x7f1f9002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.208+0000 7f1fab15b640 1 -- 172.21.15.7:0/3202737974 shutdown_connections 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.208+0000 7f1fab15b640 1 --2- 172.21.15.7:0/3202737974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4101900 0x7f1fa4101d60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.208+0000 7f1fab15b640 1 --2- 172.21.15.7:0/3202737974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa4107900 0x7f1fa4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.208+0000 7f1fab15b640 1 -- 172.21.15.7:0/3202737974 >> 172.21.15.7:0/3202737974 conn(0x7f1fa40fd600 msgr2=0x7f1fa40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.208+0000 7f1fab15b640 1 -- 172.21.15.7:0/3202737974 shutdown_connections 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.208+0000 7f1fab15b640 1 -- 172.21.15.7:0/3202737974 wait complete. 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fab15b640 1 Processor -- start 2024-09-21T04:39:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fab15b640 1 -- start start 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fab15b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa4101900 0x7f1fa419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fab15b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4107900 0x7f1fa419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fab15b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa410edb0 con 0x7f1fa4107900 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fab15b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa410ec30 con 0x7f1fa4101900 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fa3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4107900 0x7f1fa419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fa3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4107900 0x7f1fa419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59072/0 (socket says 172.21.15.7:59072) 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.209+0000 7f1fa3fff640 1 -- 172.21.15.7:0/927624695 learned_addr learned my addr 172.21.15.7:0/927624695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fa8ef7640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa4101900 0x7f1fa419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fa3fff640 1 -- 172.21.15.7:0/927624695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa4101900 msgr2=0x7f1fa419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fa3fff640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa4101900 0x7f1fa419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fa3fff640 1 -- 172.21.15.7:0/927624695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa4199a30 con 0x7f1fa4107900 2024-09-21T04:39:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fa3fff640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4107900 0x7f1fa419fdb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1f90031460 tx=0x7f1f900045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fa1ffb640 1 -- 172.21.15.7:0/927624695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1f9003f070 con 0x7f1fa4107900 2024-09-21T04:39:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fa1ffb640 1 -- 172.21.15.7:0/927624695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f900035f0 con 0x7f1fa4107900 2024-09-21T04:39:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa4199cc0 con 0x7f1fa4107900 2024-09-21T04:39:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.210+0000 7f1fa1ffb640 1 -- 172.21.15.7:0/927624695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1f90035030 con 0x7f1fa4107900 2024-09-21T04:39:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.211+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa419a1f0 con 0x7f1fa4107900 2024-09-21T04:39:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.211+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa4103040 con 0x7f1fa4107900 2024-09-21T04:39:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.212+0000 7f1fa1ffb640 1 -- 172.21.15.7:0/927624695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1f90042070 con 0x7f1fa4107900 2024-09-21T04:39:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.213+0000 7f1fa1ffb640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f9407af80 0x7f1f9407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.213+0000 7f1fa1ffb640 1 -- 172.21.15.7:0/927624695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f1f900be810 con 0x7f1fa4107900 2024-09-21T04:39:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.213+0000 7f1fa8ef7640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f9407af80 0x7f1f9407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.213+0000 7f1fa8ef7640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f9407af80 0x7f1f9407d440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1fa419ab70 tx=0x7f1f9800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.217+0000 7f1fa1ffb640 1 -- 172.21.15.7:0/927624695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7f1f90089ee0 con 0x7f1fa4107900 2024-09-21T04:39:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.429+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1fa419b060 con 0x7f1f9407af80 2024-09-21T04:39:14.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.430+0000 7f1fa1ffb640 1 -- 172.21.15.7:0/927624695 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+368 (secure 0 0 0) 0x7f1f98010be0 con 0x7f1f9407af80 2024-09-21T04:39:14.434 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:39:14.434 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:39:14.434 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:39:14.435 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:39:14.435 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:39:14.435 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-09-21T04:39:14.435 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:39:14.435 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/23 daemons upgraded", 2024-09-21T04:39:14.435 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mon daemons", 2024-09-21T04:39:14.435 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:39:14.435 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:39:14.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.432+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f9407af80 msgr2=0x7f1f9407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.432+0000 7f1fab15b640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f9407af80 0x7f1f9407d440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1fa419ab70 tx=0x7f1f9800b390 comp rx=0 tx=0).stop 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.432+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4107900 msgr2=0x7f1fa419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.432+0000 7f1fab15b640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4107900 0x7f1fa419fdb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1f90031460 tx=0x7f1f900045e0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.433+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 shutdown_connections 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.433+0000 7f1fab15b640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f9407af80 0x7f1f9407d440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.433+0000 7f1fab15b640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fa4107900 0x7f1fa419fdb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.433+0000 7f1fab15b640 1 --2- 172.21.15.7:0/927624695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa4101900 0x7f1fa419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.433+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 >> 172.21.15.7:0/927624695 conn(0x7f1fa40fd600 msgr2=0x7f1fa4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.433+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 shutdown_connections 2024-09-21T04:39:14.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.433+0000 7f1fab15b640 1 -- 172.21.15.7:0/927624695 wait complete. 2024-09-21T04:39:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.516+0000 7fabf893a640 1 -- 172.21.15.7:0/2301941212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4107900 msgr2=0x7fabf4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.516+0000 7fabf893a640 1 --2- 172.21.15.7:0/2301941212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4107900 0x7fabf4107ce0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009c20 tx=0x7fabe402a970 comp rx=0 tx=0).stop 2024-09-21T04:39:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.517+0000 7fabf893a640 1 -- 172.21.15.7:0/2301941212 shutdown_connections 2024-09-21T04:39:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.517+0000 7fabf893a640 1 --2- 172.21.15.7:0/2301941212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf4101900 0x7fabf4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.517+0000 7fabf893a640 1 --2- 172.21.15.7:0/2301941212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4107900 0x7fabf4107ce0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.517+0000 7fabf893a640 1 -- 172.21.15.7:0/2301941212 >> 172.21.15.7:0/2301941212 conn(0x7fabf40fd600 msgr2=0x7fabf40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.517+0000 7fabf893a640 1 -- 172.21.15.7:0/2301941212 shutdown_connections 2024-09-21T04:39:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.517+0000 7fabf893a640 1 -- 172.21.15.7:0/2301941212 wait complete. 2024-09-21T04:39:14.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.517+0000 7fabf893a640 1 Processor -- start 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf893a640 1 -- start start 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf893a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4101900 0x7fabf419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf893a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf4107900 0x7fabf419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf893a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabf410edb0 con 0x7fabf4101900 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf893a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fabf410ec30 con 0x7fabf4107900 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4101900 0x7fabf419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4101900 0x7fabf419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59096/0 (socket says 172.21.15.7:59096) 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf259c640 1 -- 172.21.15.7:0/1933743150 learned_addr learned my addr 172.21.15.7:0/1933743150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf1d9b640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf4107900 0x7fabf419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf259c640 1 -- 172.21.15.7:0/1933743150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf4107900 msgr2=0x7fabf419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf259c640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf4107900 0x7fabf419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf259c640 1 -- 172.21.15.7:0/1933743150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41977c0 con 0x7fabf4101900 2024-09-21T04:39:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf1d9b640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf4107900 0x7fabf419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:39:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.518+0000 7fabf259c640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4101900 0x7fabf419d640 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fabe4005cc0 tx=0x7fabe402f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.519+0000 7fabeb7fe640 1 -- 172.21.15.7:0/1933743150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fabe4047070 con 0x7fabf4101900 2024-09-21T04:39:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.519+0000 7fabeb7fe640 1 -- 172.21.15.7:0/1933743150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fabe402f7e0 con 0x7fabf4101900 2024-09-21T04:39:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.519+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf4197a50 con 0x7fabf4101900 2024-09-21T04:39:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.519+0000 7fabeb7fe640 1 -- 172.21.15.7:0/1933743150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fabe4034070 con 0x7fabf4101900 2024-09-21T04:39:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.519+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf4197f80 con 0x7fabf4101900 2024-09-21T04:39:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.520+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf4103040 con 0x7fabf4101900 2024-09-21T04:39:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.520+0000 7fabeb7fe640 1 -- 172.21.15.7:0/1933743150 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fabe404a070 con 0x7fabf4101900 2024-09-21T04:39:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.521+0000 7fabeb7fe640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabe007af30 0x7fabe007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.521+0000 7fabeb7fe640 1 -- 172.21.15.7:0/1933743150 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fabe40bee90 con 0x7fabf4101900 2024-09-21T04:39:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.521+0000 7fabf1d9b640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabe007af30 0x7fabe007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:14.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.522+0000 7fabf1d9b640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabe007af30 0x7fabe007d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fabf4100e00 tx=0x7fabdc005d90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:14.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.527+0000 7fabeb7fe640 1 -- 172.21.15.7:0/1933743150 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196976 (secure 0 0 0) 0x7fabe403f070 con 0x7fabf4101900 2024-09-21T04:39:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.792+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fabf4198cf0 con 0x7fabf4101900 2024-09-21T04:39:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.792+0000 7fabeb7fe640 1 -- 172.21.15.7:0/1933743150 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7fabe408a4e0 con 0x7fabf4101900 2024-09-21T04:39:14.797 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:14.797 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:14.797 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabe007af30 msgr2=0x7fabe007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabe007af30 0x7fabe007d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fabf4100e00 tx=0x7fabdc005d90 comp rx=0 tx=0).stop 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4101900 msgr2=0x7fabf419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4101900 0x7fabf419d640 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fabe4005cc0 tx=0x7fabe402f5f0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 shutdown_connections 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabe007af30 0x7fabe007d3f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabf4107900 0x7fabf419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 --2- 172.21.15.7:0/1933743150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf4101900 0x7fabf419d640 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 >> 172.21.15.7:0/1933743150 conn(0x7fabf40fd600 msgr2=0x7fabf40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 shutdown_connections 2024-09-21T04:39:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:14.795+0000 7fabf893a640 1 -- 172.21.15.7:0/1933743150 wait complete. 2024-09-21T04:39:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:14 smithi053 ceph-mon[35554]: from='client.14950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:14 smithi053 ceph-mon[35554]: from='client.14954 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:14 smithi053 ceph-mon[35554]: pgmap v101: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:14 smithi053 ceph-mon[35554]: Upgrade: Target is version 19.3.0-5103-gf36283e7 (squid) 2024-09-21T04:39:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:14 smithi053 ceph-mon[35554]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00'] 2024-09-21T04:39:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:14 smithi053 ceph-mon[35554]: Upgrade: Setting container_image for all mgr 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:14 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/3663431636' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:14 smithi053 ceph-mon[35554]: from='client.? 172.21.15.7:0/1933743150' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:14 smithi007 ceph-mon[27954]: from='client.14950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:14 smithi007 ceph-mon[27954]: from='client.14954 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:14 smithi007 ceph-mon[27954]: pgmap v101: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:14 smithi007 ceph-mon[27954]: Upgrade: Target is version 19.3.0-5103-gf36283e7 (squid) 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:14 smithi007 ceph-mon[27954]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00'] 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:14 smithi007 ceph-mon[27954]: Upgrade: Setting container_image for all mgr 2024-09-21T04:39:15.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:14 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/3663431636' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-21T04:39:15.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:14 smithi007 ceph-mon[27954]: from='client.? 172.21.15.7:0/1933743150' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-21T04:39:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:15 smithi053 ceph-mon[35554]: from='client.14966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:15 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:15 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:39:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:15 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:39:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:15 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:15 smithi007 ceph-mon[27954]: from='client.14966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:15 smithi007 ceph-mon[27954]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:15 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:39:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:15 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:39:16.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:15 smithi007 ceph-mon[27954]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:16.893 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:16 smithi007 systemd[1]: Stopping Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:39:17.151 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:16 smithi007 ceph-mon[27954]: Upgrade: Updating mon.smithi007 2024-09-21T04:39:17.152 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:16 smithi007 ceph-mon[27954]: Deploying daemon mon.smithi007 on smithi007 2024-09-21T04:39:17.152 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:16 smithi007 ceph-mon[27954]: pgmap v102: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:17.152 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[27950]: 2024-09-21T04:39:17.086+0000 7fa0c8c40640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi007 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:39:17.152 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[27950]: 2024-09-21T04:39:17.086+0000 7fa0c8c40640 -1 mon.smithi007@0(leader) e2 *** Got Signal Terminated *** 2024-09-21T04:39:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:16 smithi053 ceph-mon[35554]: Upgrade: Updating mon.smithi007 2024-09-21T04:39:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:16 smithi053 ceph-mon[35554]: Deploying daemon mon.smithi007 on smithi007 2024-09-21T04:39:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:16 smithi053 ceph-mon[35554]: pgmap v102: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:17.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 podman[108078]: 2024-09-21 04:39:17.149636993 +0000 UTC m=+0.163041087 container died 1fc3cec404f2f7bde6b5e049958af9b1b1ed36ce79fd506ebec4da41abb9a88b (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-4e45e6b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:39:17.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 podman[108078]: 2024-09-21 04:39:17.529157842 +0000 UTC m=+0.542561935 container cleanup 1fc3cec404f2f7bde6b5e049958af9b1b1ed36ce79fd506ebec4da41abb9a88b (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-4e45e6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T04:39:17.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 bash[108078]: ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007 2024-09-21T04:39:18.183 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 podman[108090]: 2024-09-21 04:39:17.797555758 +0000 UTC m=+0.642458473 container remove 1fc3cec404f2f7bde6b5e049958af9b1b1ed36ce79fd506ebec4da41abb9a88b (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-4e45e6b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T04:39:18.183 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service: Deactivated successfully. 2024-09-21T04:39:18.183 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 systemd[1]: Stopped Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:39:18.183 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:17 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service: Consumed 14.678s CPU time. 2024-09-21T04:39:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 systemd[1]: Starting Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:39:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 podman[108219]: 2024-09-21 04:39:18.296360286 +0000 UTC m=+0.016489366 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:39:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 podman[108219]: 2024-09-21 04:39:18.425995251 +0000 UTC m=+0.146124372 container create 1ac9cb9a20ec6c2bdf3e219bf56b8702e6cc90b48c9226b34a3286780fb8b239 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:39:18.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 podman[108219]: 2024-09-21 04:39:18.660542297 +0000 UTC m=+0.380671390 container init 1ac9cb9a20ec6c2bdf3e219bf56b8702e6cc90b48c9226b34a3286780fb8b239 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-21T04:39:18.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 podman[108219]: 2024-09-21 04:39:18.663860971 +0000 UTC m=+0.383990053 container start 1ac9cb9a20ec6c2bdf3e219bf56b8702e6cc90b48c9226b34a3286780fb8b239 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev), process ceph-mon, pid 2 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: pidfile_write: ignore empty --pid-file 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: load: jerasure load: lrc 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: RocksDB version: 7.9.2 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Git sha 0 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Compile date 2024-09-20 23:32:13 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: DB SUMMARY 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: DB Session ID: KNCCUQ5HDXDGWG12RQRX 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: CURRENT file: CURRENT 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1519 Bytes 2024-09-21T04:39:18.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi007/store.db dir, Total Num: 1, files: 000032.sst 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi007/store.db: 000030.log size: 2478009 ; 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.error_if_exists: 0 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.create_if_missing: 0 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.paranoid_checks: 1 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.env: 0x555cbccb4120 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.info_log: 0x555cbf3a98e0 2024-09-21T04:39:18.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.statistics: (nil) 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.use_fsync: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_log_file_size: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.allow_fallocate: 1 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.use_direct_reads: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.db_log_dir: 2024-09-21T04:39:18.948 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.wal_dir: 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.write_buffer_manager: 0x555cbf3adb80 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T04:39:18.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.unordered_write: 0 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.row_cache: None 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.wal_filter: None 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.two_write_queues: 0 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.wal_compression: 0 2024-09-21T04:39:18.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.atomic_flush: 0 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.log_readahead_size: 0 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_background_jobs: 2 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_background_compactions: -1 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_subcompactions: 1 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T04:39:18.951 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_open_files: -1 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_background_flushes: -1 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Compression algorithms supported: 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: kZSTD supported: 0 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: kXpressCompression supported: 0 2024-09-21T04:39:18.952 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: kBZip2Compression supported: 0 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: kLZ4Compression supported: 1 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: kZlibCompression supported: 1 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: kSnappyCompression supported: 1 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi007/store.db/MANIFEST-000015 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.merge_operator: 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_filter: None 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T04:39:18.953 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555cbf3a9540) 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_type: 0 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_block_index_type: 0 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_shortening: 1 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T04:39:18.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: checksum: 4 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: no_block_cache: 0 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache: 0x555cbf39d610 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_options: 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: capacity : 536870912 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: num_shard_bits : 4 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: strict_capacity_limit : 0 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_compressed: (nil) 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: persistent_cache: (nil) 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_size: 4096 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_size_deviation: 10 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_restart_interval: 16 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_block_restart_interval: 1 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: metadata_block_size: 4096 2024-09-21T04:39:18.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: partition_filters: 0 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: use_delta_encoding: 1 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: filter_policy: bloomfilter 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: whole_key_filtering: 1 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: verify_compression: 0 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: format_version: 5 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: enable_index_compression: 1 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_align: 0 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_auto_readahead_size: 262144 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: prepopulate_block_cache: 0 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: initial_auto_readahead_size: 8192 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T04:39:18.956 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression: NoCompression 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.num_levels: 7 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T04:39:18.957 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T04:39:18.958 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T04:39:18.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T04:39:18.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.inplace_update_support: 0 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.bloom_locality: 0 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.max_successive_merges: 0 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T04:39:18.961 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.ttl: 2592000 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.enable_blob_files: false 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.min_blob_size: 0 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T04:39:18.962 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi007/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12065, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f38080b9-bf82-48bb-b516-5795a3936218 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726893558691434, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726893558702129, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 2169507, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12066, "largest_seqno": 13867, "table_properties": {"data_size": 2161050, "index_size": 5363, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2053, "raw_key_size": 18520, "raw_average_key_size": 23, "raw_value_size": 2143986, "raw_average_value_size": 2679, "num_data_blocks": 245, "num_entries": 800, "num_filter_entries": 800, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726893558, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f38080b9-bf82-48bb-b516-5795a3936218", "db_session_id": "KNCCUQ5HDXDGWG12RQRX", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726893558702269, "job": 1, "event": "recovery_finished"} 2024-09-21T04:39:18.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi007/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555cbf3cce00 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: DB pointer 0x555cbf4ce000 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** DB Stats ** 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:39:18.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: L0 1/0 2.07 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 521.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: L6 1/0 7.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Sum 2/0 9.50 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 521.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 521.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 521.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-09-21T04:39:18.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative compaction: 0.00 GB write, 151.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval compaction: 0.00 GB write, 151.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Block cache BinnedLRUCache@0x555cbf39d610#2 capacity: 512.00 MB usage: 8.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.28 KB,0.000435114%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: starting mon.smithi007 rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi007 fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:39:18.966 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: mon.smithi007@-1(???) e2 preinit fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:39:18.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: mon.smithi007@-1(???).mds e19 new map 2024-09-21T04:39:18.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: mon.smithi007@-1(???).mds e19 print_map 2024-09-21T04:39:18.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: e19 2024-09-21T04:39:18.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:39:18.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:39:18.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:39:18.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: legacy client fscid: 1 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Filesystem 'cephfs' (1) 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: fs_name cephfs 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: epoch 19 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: tableserver 0 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: root 0 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: session_timeout 60 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: session_autoclose 300 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_file_size 1099511627776 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_xattr_size 65536 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: required_client_features {} 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: last_failure 0 2024-09-21T04:39:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: last_failure_osd_epoch 44 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_mds 2 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: in 0,1 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: up {0=14578,1=24327} 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: failed 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: damaged 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: stopped 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_pools [3] 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: metadata_pool 2 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: inline_data enabled 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: balancer 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: bal_rank_mask -1 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: standby_count_wanted 1 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: [mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:18.969 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: [mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: [mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: [mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: mon.smithi007@-1(???).osd e46 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: mon.smithi007@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: mon.smithi007@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: mon.smithi007@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 ceph-mon[108238]: mon.smithi007@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 bash[108219]: 1ac9cb9a20ec6c2bdf3e219bf56b8702e6cc90b48c9226b34a3286780fb8b239 2024-09-21T04:39:18.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:18 smithi007 systemd[1]: Started Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:39:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: pgmap v103: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: pgmap v104: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: mon.smithi007 calling monitor election 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: mon.smithi007 is new leader, mons smithi007,smithi053 in quorum (ranks 0,1) 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: monmap epoch 2 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: last_changed 2024-09-21T04:27:55.080724+0000 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: created 2024-09-21T04:25:29.508493+0000 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: min_mon_release 19 (squid) 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: election_strategy: 1 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:39:20.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: osdmap e46: 6 total, 6 up, 6 in 2024-09-21T04:39:20.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: mgrmap e33: smithi007.huohfl(active, since 3m), standbys: smithi053.lnadao 2024-09-21T04:39:20.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:20.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:20.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:39:20.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='' 2024-09-21T04:39:20.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:20.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:20 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:20.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: pgmap v103: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:20.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: pgmap v104: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:20.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: mon.smithi007 calling monitor election 2024-09-21T04:39:20.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: mon.smithi007 is new leader, mons smithi007,smithi053 in quorum (ranks 0,1) 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: monmap epoch 2 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: last_changed 2024-09-21T04:27:55.080724+0000 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: created 2024-09-21T04:25:29.508493+0000 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: min_mon_release 19 (squid) 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: election_strategy: 1 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: osdmap e46: 6 total, 6 up, 6 in 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: mgrmap e33: smithi007.huohfl(active, since 3m), standbys: smithi053.lnadao 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:20.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:39:20.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='' 2024-09-21T04:39:20.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:20.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:20 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:23.057 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:22 smithi007 ceph-mon[108238]: pgmap v105: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:23.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:22 smithi053 ceph-mon[35554]: pgmap v105: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:25.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:24 smithi053 ceph-mon[35554]: pgmap v106: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:24 smithi007 ceph-mon[108238]: pgmap v106: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:26.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:26 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:26.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:26 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:26.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:26 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:26.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:26 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:26.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:26 smithi053 ceph-mon[35554]: pgmap v107: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:26.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:26 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:26.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:26 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:26.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:26 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:26.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:26 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:26.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:26 smithi007 ceph-mon[108238]: pgmap v107: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:29.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:28 smithi053 ceph-mon[35554]: pgmap v108: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:28 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:39:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:28 smithi007 ceph-mon[108238]: pgmap v108: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-21T04:39:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:29 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:29 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:29 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:29 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:39:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:29 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:29 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:30.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:29 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:39:30.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:29 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-21T04:39:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:29 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:29 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-21T04:39:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:29 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:30.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-21T04:39:30.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "versions"}]: dispatch 2024-09-21T04:39:30.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-21T04:39:31.051 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:30 smithi053 ceph-mon[35554]: pgmap v109: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:31.051 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:30 smithi053 ceph-mon[35554]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:31.051 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:30 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:39:31.051 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:30 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:39:31.052 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:30 smithi053 ceph-mon[35554]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:30 smithi007 ceph-mon[108238]: pgmap v109: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:30 smithi007 ceph-mon[108238]: from='mgr.14772 ' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-21T04:39:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-21T04:39:31.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-21T04:39:32.127 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:31 smithi053 ceph-mon[35554]: Upgrade: Updating mon.smithi053 2024-09-21T04:39:32.127 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:31 smithi053 ceph-mon[35554]: Deploying daemon mon.smithi053 on smithi053 2024-09-21T04:39:32.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:31 smithi007 ceph-mon[108238]: Upgrade: Updating mon.smithi053 2024-09-21T04:39:32.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:31 smithi007 ceph-mon[108238]: Deploying daemon mon.smithi053 on smithi053 2024-09-21T04:39:32.411 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:32 smithi053 systemd[1]: Stopping Ceph mon.smithi053 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:39:32.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:32 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053[35550]: 2024-09-21T04:39:32.407+0000 7f91407f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi053 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:39:32.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:32 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053[35550]: 2024-09-21T04:39:32.407+0000 7f91407f1640 -1 mon.smithi053@1(peon) e2 *** Got Signal Terminated *** 2024-09-21T04:39:32.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:32 smithi053 podman[80012]: 2024-09-21 04:39:32.47939177 +0000 UTC m=+0.204794965 container died d92ed5c082065a3311191cd98d52b2abcfdfccb14074eb6b0d4748d4380fe7f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-4e45e6b, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T04:39:33.002 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:32 smithi053 podman[80012]: 2024-09-21 04:39:32.807514616 +0000 UTC m=+0.532917851 container cleanup d92ed5c082065a3311191cd98d52b2abcfdfccb14074eb6b0d4748d4380fe7f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-4e45e6b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T04:39:33.002 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:32 smithi053 bash[80012]: ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053 2024-09-21T04:39:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 podman[80026]: 2024-09-21 04:39:33.000582209 +0000 UTC m=+0.522274138 container remove d92ed5c082065a3311191cd98d52b2abcfdfccb14074eb6b0d4748d4380fe7f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053, org.label-schema.license=GPLv2, RELEASE=squid-4e45e6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:39:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi053.service: Deactivated successfully. 2024-09-21T04:39:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 systemd[1]: Stopped Ceph mon.smithi053 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:39:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi053.service: Consumed 8.065s CPU time. 2024-09-21T04:39:33.803 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 systemd[1]: Starting Ceph mon.smithi053 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:39:33.804 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 podman[80157]: 2024-09-21 04:39:33.570582927 +0000 UTC m=+0.016498575 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:39:33.804 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 podman[80157]: 2024-09-21 04:39:33.685319006 +0000 UTC m=+0.131234651 container create fdf38e86da7b2e75065ab2df90bfc93419d828de2611519ecb98ddc48b067ff3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T04:39:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 podman[80157]: 2024-09-21 04:39:33.929851571 +0000 UTC m=+0.375767229 container init fdf38e86da7b2e75065ab2df90bfc93419d828de2611519ecb98ddc48b067ff3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T04:39:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 podman[80157]: 2024-09-21 04:39:33.933181914 +0000 UTC m=+0.379097559 container start fdf38e86da7b2e75065ab2df90bfc93419d828de2611519ecb98ddc48b067ff3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T04:39:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T04:39:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev), process ceph-mon, pid 2 2024-09-21T04:39:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: pidfile_write: ignore empty --pid-file 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: load: jerasure load: lrc 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: RocksDB version: 7.9.2 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Git sha 0 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Compile date 2024-09-20 23:32:13 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: DB SUMMARY 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: DB Session ID: 4QYOMP8A253MZ9IVKA0B 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: CURRENT file: CURRENT 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1412 Bytes 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi053/store.db dir, Total Num: 1, files: 000027.sst 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi053/store.db: 000025.log size: 4288869 ; 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.error_if_exists: 0 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.create_if_missing: 0 2024-09-21T04:39:34.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.paranoid_checks: 1 2024-09-21T04:39:34.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T04:39:34.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T04:39:34.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T04:39:34.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.env: 0x5593a1140120 2024-09-21T04:39:34.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T04:39:34.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.info_log: 0x5593a30798e0 2024-09-21T04:39:34.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T04:39:34.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.statistics: (nil) 2024-09-21T04:39:34.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.use_fsync: 0 2024-09-21T04:39:34.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_log_file_size: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.allow_fallocate: 1 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.use_direct_reads: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.db_log_dir: 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.wal_dir: 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T04:39:34.197 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.write_buffer_manager: 0x5593a307db80 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T04:39:34.198 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.unordered_write: 0 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.row_cache: None 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.wal_filter: None 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.two_write_queues: 0 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.wal_compression: 0 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.atomic_flush: 0 2024-09-21T04:39:34.199 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.log_readahead_size: 0 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_background_jobs: 2 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_background_compactions: -1 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_subcompactions: 1 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T04:39:34.200 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_open_files: -1 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_background_flushes: -1 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Compression algorithms supported: 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: kZSTD supported: 0 2024-09-21T04:39:34.201 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: kXpressCompression supported: 0 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: kBZip2Compression supported: 0 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: kLZ4Compression supported: 1 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: kZlibCompression supported: 1 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: kSnappyCompression supported: 1 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi053/store.db/MANIFEST-000010 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T04:39:34.202 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.merge_operator: 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_filter: None 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5593a3079540) 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: cache_index_and_filter_blocks: 1 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: pin_top_level_index_and_filter: 1 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: index_type: 0 2024-09-21T04:39:34.203 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: data_block_index_type: 0 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: index_shortening: 1 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: checksum: 4 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: no_block_cache: 0 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_cache: 0x5593a306d610 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_cache_name: BinnedLRUCache 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_cache_options: 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: capacity : 536870912 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: num_shard_bits : 4 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: strict_capacity_limit : 0 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: high_pri_pool_ratio: 0.000 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_cache_compressed: (nil) 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: persistent_cache: (nil) 2024-09-21T04:39:34.204 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_size: 4096 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_size_deviation: 10 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_restart_interval: 16 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: index_block_restart_interval: 1 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: metadata_block_size: 4096 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: partition_filters: 0 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: use_delta_encoding: 1 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: filter_policy: bloomfilter 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: whole_key_filtering: 1 2024-09-21T04:39:34.205 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: verify_compression: 0 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: read_amp_bytes_per_bit: 0 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: format_version: 5 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: enable_index_compression: 1 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: block_align: 0 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: max_auto_readahead_size: 262144 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: prepopulate_block_cache: 0 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: initial_auto_readahead_size: 8192 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression: NoCompression 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T04:39:34.206 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.num_levels: 7 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T04:39:34.207 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T04:39:34.208 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T04:39:34.209 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T04:39:34.210 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.inplace_update_support: 0 2024-09-21T04:39:34.211 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.bloom_locality: 0 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.max_successive_merges: 0 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T04:39:34.212 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.ttl: 2592000 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.enable_blob_files: false 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.min_blob_size: 0 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T04:39:34.213 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi053/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12243, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-09-21T04:39:34.214 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b1486bb7-033a-4d3f-bebb-e7766d02f414 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726893573960575, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726893573971926, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 2483178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12248, "largest_seqno": 14523, "table_properties": {"data_size": 2473741, "index_size": 6087, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2309, "raw_key_size": 21011, "raw_average_key_size": 23, "raw_value_size": 2454366, "raw_average_value_size": 2709, "num_data_blocks": 277, "num_entries": 906, "num_filter_entries": 906, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726893573, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b1486bb7-033a-4d3f-bebb-e7766d02f414", "db_session_id": "4QYOMP8A253MZ9IVKA0B", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726893573972073, "job": 1, "event": "recovery_finished"} 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi053/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5593a309ce00 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: DB pointer 0x5593a319e000 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ** DB Stats ** 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:39:34.215 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ** Compaction Stats [default] ** 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: L0 1/0 2.37 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 511.6 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: L6 1/0 7.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Sum 2/0 9.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 511.6 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 511.6 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-09-21T04:39:34.216 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ** Compaction Stats [default] ** 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 511.6 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T04:39:34.217 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Cumulative compaction: 0.00 GB write, 168.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Interval compaction: 0.00 GB write, 168.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Block cache BinnedLRUCache@0x5593a306d610#2 capacity: 512.00 MB usage: 8.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.53 KB,0.000482798%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: starting mon.smithi053 rank 1 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi053 fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: mon.smithi053@-1(???) e2 preinit fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: mon.smithi053@-1(???).mds e19 new map 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: mon.smithi053@-1(???).mds e19 print_map 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: e19 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:39:34.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: legacy client fscid: 1 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: Filesystem 'cephfs' (1) 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: fs_name cephfs 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: epoch 19 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: tableserver 0 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: root 0 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: session_timeout 60 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: session_autoclose 300 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: max_file_size 1099511627776 2024-09-21T04:39:34.219 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: max_xattr_size 65536 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: required_client_features {} 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: last_failure 0 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: last_failure_osd_epoch 44 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: max_mds 2 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: in 0,1 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: up {0=14578,1=24327} 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: failed 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: damaged 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: stopped 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: data_pools [3] 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: metadata_pool 2 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: inline_data enabled 2024-09-21T04:39:34.220 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: balancer 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: bal_rank_mask -1 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: standby_count_wanted 1 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: [mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: [mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: [mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: [mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout: 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: mon.smithi053@-1(???).osd e46 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: mon.smithi053@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: mon.smithi053@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: mon.smithi053@-1(???).osd e46 crush map has features 288514051259236352, adjusting msgr requires 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:33 smithi053 ceph-mon[80176]: mon.smithi053@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-09-21T04:39:34.221 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:34 smithi053 bash[80157]: fdf38e86da7b2e75065ab2df90bfc93419d828de2611519ecb98ddc48b067ff3 2024-09-21T04:39:34.222 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:34 smithi053 systemd[1]: Started Ceph mon.smithi053 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:39:34.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:34 smithi053 ceph-mon[80176]: pgmap v110: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:34.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:34 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mon metadata", "id": "smithi007"} : dispatch 2024-09-21T04:39:34.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:34 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mon metadata", "id": "smithi053"} : dispatch 2024-09-21T04:39:34.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:34 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T04:39:34.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:34 smithi007 ceph-mon[108238]: pgmap v110: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:34.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:34 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mon metadata", "id": "smithi007"} : dispatch 2024-09-21T04:39:34.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:34 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mon metadata", "id": "smithi053"} : dispatch 2024-09-21T04:39:34.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:34 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T04:39:35.631 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:39:35.631 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: pgmap v111: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:35.631 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: mon.smithi053 calling monitor election 2024-09-21T04:39:35.631 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: mon.smithi007 calling monitor election 2024-09-21T04:39:35.631 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: mon.smithi007 is new leader, mons smithi007,smithi053 in quorum (ranks 0,1) 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: monmap epoch 2 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: last_changed 2024-09-21T04:27:55.080724+0000 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: created 2024-09-21T04:25:29.508493+0000 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: min_mon_release 19 (squid) 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: election_strategy: 1 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: osdmap e46: 6 total, 6 up, 6 in 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: mgrmap e33: smithi007.huohfl(active, since 3m), standbys: smithi053.lnadao 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:35.632 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:39:35.633 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:35.633 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:35.633 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:35 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:39:35.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T04:39:35.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: pgmap v111: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:35.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: mon.smithi053 calling monitor election 2024-09-21T04:39:35.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: mon.smithi007 calling monitor election 2024-09-21T04:39:35.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: mon.smithi007 is new leader, mons smithi007,smithi053 in quorum (ranks 0,1) 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: monmap epoch 2 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: fsid 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: last_changed 2024-09-21T04:27:55.080724+0000 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: created 2024-09-21T04:25:29.508493+0000 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: min_mon_release 19 (squid) 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: election_strategy: 1 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: 1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.smithi053 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 2 up:standby-replay 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: osdmap e46: 6 total, 6 up, 6 in 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: mgrmap e33: smithi007.huohfl(active, since 3m), standbys: smithi053.lnadao 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:35.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:39:35.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:35.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:35.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:35 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:39:36.625 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:36 smithi053 ceph-mon[80176]: pgmap v112: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:36 smithi007 ceph-mon[108238]: pgmap v112: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:38.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:38.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:38.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:38.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:38.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:38 smithi007 ceph-mon[108238]: pgmap v113: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:38.380 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:38.380 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:38.380 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:38.381 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:38.381 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:38 smithi053 ceph-mon[80176]: pgmap v113: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:40 smithi007 ceph-mon[108238]: pgmap v114: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:40 smithi053 ceph-mon[80176]: pgmap v114: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi007"} : dispatch 2024-09-21T04:39:42.345 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi007"}]': finished 2024-09-21T04:39:42.346 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi053"} : dispatch 2024-09-21T04:39:42.346 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi053"}]': finished 2024-09-21T04:39:42.435 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:42.435 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:42.435 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:39:42.435 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:39:42.435 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:42.435 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:39:42.435 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:39:42.435 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:39:42.436 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:42.436 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi007"} : dispatch 2024-09-21T04:39:42.436 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi007"}]': finished 2024-09-21T04:39:42.436 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi053"} : dispatch 2024-09-21T04:39:42.436 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi053"}]': finished 2024-09-21T04:39:43.121 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:42 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all mon 2024-09-21T04:39:43.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:42 smithi007 ceph-mon[108238]: pgmap v115: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:43.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:43.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:39:43.122 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:39:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:42 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all mon 2024-09-21T04:39:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:42 smithi053 ceph-mon[80176]: pgmap v115: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:39:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:39:44.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:43 smithi007 ceph-mon[108238]: Upgrade: Updating crash.smithi007 (1/2) 2024-09-21T04:39:44.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:43 smithi007 ceph-mon[108238]: Deploying daemon crash.smithi007 on smithi007 2024-09-21T04:39:44.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:39:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:43 smithi053 ceph-mon[80176]: Upgrade: Updating crash.smithi007 (1/2) 2024-09-21T04:39:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:43 smithi053 ceph-mon[80176]: Deploying daemon crash.smithi007 on smithi007 2024-09-21T04:39:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:39:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.885+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2931969918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100ff0 msgr2=0x7fa2ec10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.885+0000 7fa2f2f83640 1 --2- 172.21.15.7:0/2931969918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100ff0 0x7fa2ec10dcb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009880 tx=0x7fa2d802a490 comp rx=0 tx=0).stop 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.885+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2931969918 shutdown_connections 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.885+0000 7fa2f2f83640 1 --2- 172.21.15.7:0/2931969918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100ff0 0x7fa2ec10dcb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.885+0000 7fa2f2f83640 1 --2- 172.21.15.7:0/2931969918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2ec100640 0x7fa2ec100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.885+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2931969918 >> 172.21.15.7:0/2931969918 conn(0x7fa2ec0fc550 msgr2=0x7fa2ec0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.885+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2931969918 shutdown_connections 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.885+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2931969918 wait complete. 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.886+0000 7fa2f2f83640 1 Processor -- start 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.886+0000 7fa2f2f83640 1 -- start start 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.886+0000 7fa2f2f83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100640 0x7fa2ec19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.886+0000 7fa2f2f83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2ec100ff0 0x7fa2ec19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.886+0000 7fa2f2f83640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec110ff0 con 0x7fa2ec100640 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.886+0000 7fa2f2f83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec110e70 con 0x7fa2ec100ff0 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2f0d1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100640 0x7fa2ec19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2f0d1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100640 0x7fa2ec19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54162/0 (socket says 172.21.15.7:54162) 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2f0d1f640 1 -- 172.21.15.7:0/2196667940 learned_addr learned my addr 172.21.15.7:0/2196667940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2ebfff640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2ec100ff0 0x7fa2ec19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2f0d1f640 1 -- 172.21.15.7:0/2196667940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2ec100ff0 msgr2=0x7fa2ec19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2f0d1f640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2ec100ff0 0x7fa2ec19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2f0d1f640 1 -- 172.21.15.7:0/2196667940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec199a60 con 0x7fa2ec100640 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2ebfff640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2ec100ff0 0x7fa2ec19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2f0d1f640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100640 0x7fa2ec19f8a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000b420 tx=0x7fa2e000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2e9ffb640 1 -- 172.21.15.7:0/2196667940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa2e0011630 con 0x7fa2ec100640 2024-09-21T04:39:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2e9ffb640 1 -- 172.21.15.7:0/2196667940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa2e0016070 con 0x7fa2ec100640 2024-09-21T04:39:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec199d50 con 0x7fa2ec100640 2024-09-21T04:39:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.887+0000 7fa2e9ffb640 1 -- 172.21.15.7:0/2196667940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa2e0010d40 con 0x7fa2ec100640 2024-09-21T04:39:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.888+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec19a2b0 con 0x7fa2ec100640 2024-09-21T04:39:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.889+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ec06a580 con 0x7fa2ec100640 2024-09-21T04:39:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.889+0000 7fa2e9ffb640 1 -- 172.21.15.7:0/2196667940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa2e001b460 con 0x7fa2ec100640 2024-09-21T04:39:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.890+0000 7fa2e9ffb640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2dc07af30 0x7fa2dc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.890+0000 7fa2e9ffb640 1 -- 172.21.15.7:0/2196667940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fa2e009d6f0 con 0x7fa2ec100640 2024-09-21T04:39:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.890+0000 7fa2ebfff640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2dc07af30 0x7fa2dc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.890+0000 7fa2ebfff640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2dc07af30 0x7fa2dc07d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec19b250 tx=0x7fa2d8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:44.894+0000 7fa2e9ffb640 1 -- 172.21.15.7:0/2196667940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2e0021250 con 0x7fa2ec100640 2024-09-21T04:39:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.116+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa2ec19af90 con 0x7fa2dc07af30 2024-09-21T04:39:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.117+0000 7fa2e9ffb640 1 -- 172.21.15.7:0/2196667940 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+385 (secure 0 0 0) 0x7fa2ec19af90 con 0x7fa2dc07af30 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2dc07af30 msgr2=0x7fa2dc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2dc07af30 0x7fa2dc07d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec19b250 tx=0x7fa2d8031400 comp rx=0 tx=0).stop 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100640 msgr2=0x7fa2ec19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100640 0x7fa2ec19f8a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000b420 tx=0x7fa2e000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 shutdown_connections 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2dc07af30 0x7fa2dc07d3f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2ec100ff0 0x7fa2ec19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 --2- 172.21.15.7:0/2196667940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2ec100640 0x7fa2ec19f8a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 >> 172.21.15.7:0/2196667940 conn(0x7fa2ec0fc550 msgr2=0x7fa2ec0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 shutdown_connections 2024-09-21T04:39:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.120+0000 7fa2f2f83640 1 -- 172.21.15.7:0/2196667940 wait complete. 2024-09-21T04:39:45.131 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:39:45.154 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:45 smithi007 ceph-mon[108238]: pgmap v116: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.218+0000 7ff1767e3640 1 -- 172.21.15.7:0/1322739900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 msgr2=0x7ff17010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.218+0000 7ff1767e3640 1 --2- 172.21.15.7:0/1322739900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 0x7ff17010dcb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff164009880 tx=0x7ff16402a490 comp rx=0 tx=0).stop 2024-09-21T04:39:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.219+0000 7ff1767e3640 1 -- 172.21.15.7:0/1322739900 shutdown_connections 2024-09-21T04:39:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.219+0000 7ff1767e3640 1 --2- 172.21.15.7:0/1322739900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 0x7ff17010dcb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.219+0000 7ff1767e3640 1 --2- 172.21.15.7:0/1322739900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff170100640 0x7ff170100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.219+0000 7ff1767e3640 1 -- 172.21.15.7:0/1322739900 >> 172.21.15.7:0/1322739900 conn(0x7ff1700fc550 msgr2=0x7ff1700fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.219+0000 7ff1767e3640 1 -- 172.21.15.7:0/1322739900 shutdown_connections 2024-09-21T04:39:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.219+0000 7ff1767e3640 1 -- 172.21.15.7:0/1322739900 wait complete. 2024-09-21T04:39:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.219+0000 7ff1767e3640 1 Processor -- start 2024-09-21T04:39:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.219+0000 7ff1767e3640 1 -- start start 2024-09-21T04:39:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.220+0000 7ff1767e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff170100640 0x7ff17019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.220+0000 7ff1767e3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 0x7ff17019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.220+0000 7ff1767e3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff170110f70 con 0x7ff170100ff0 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.220+0000 7ff1767e3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff170110df0 con 0x7ff170100640 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.220+0000 7ff16f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 0x7ff17019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.220+0000 7ff16f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 0x7ff17019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54178/0 (socket says 172.21.15.7:54178) 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.220+0000 7ff16f7fe640 1 -- 172.21.15.7:0/3835607883 learned_addr learned my addr 172.21.15.7:0/3835607883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.220+0000 7ff16ffff640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff170100640 0x7ff17019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff16f7fe640 1 -- 172.21.15.7:0/3835607883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff170100640 msgr2=0x7ff17019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff16f7fe640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff170100640 0x7ff17019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff16f7fe640 1 -- 172.21.15.7:0/3835607883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1701a03d0 con 0x7ff170100ff0 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff16f7fe640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 0x7ff17019fd20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff164031460 tx=0x7ff1640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff16d7fa640 1 -- 172.21.15.7:0/3835607883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff16403f070 con 0x7ff170100ff0 2024-09-21T04:39:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff170199a60 con 0x7ff170100ff0 2024-09-21T04:39:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff16d7fa640 1 -- 172.21.15.7:0/3835607883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff164004cf0 con 0x7ff170100ff0 2024-09-21T04:39:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff16d7fa640 1 -- 172.21.15.7:0/3835607883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff164035030 con 0x7ff170100ff0 2024-09-21T04:39:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.221+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff170199fc0 con 0x7ff170100ff0 2024-09-21T04:39:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.223+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17006a580 con 0x7ff170100ff0 2024-09-21T04:39:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.223+0000 7ff16d7fa640 1 -- 172.21.15.7:0/3835607883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff164042070 con 0x7ff170100ff0 2024-09-21T04:39:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.224+0000 7ff16d7fa640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff15807af80 0x7ff15807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.224+0000 7ff16d7fa640 1 -- 172.21.15.7:0/3835607883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7ff1640be8e0 con 0x7ff170100ff0 2024-09-21T04:39:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.224+0000 7ff16ffff640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff15807af80 0x7ff15807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.225+0000 7ff16ffff640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff15807af80 0x7ff15807d440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff17019aae0 tx=0x7ff16000a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.229+0000 7ff16d7fa640 1 -- 172.21.15.7:0/3835607883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff164089fb0 con 0x7ff170100ff0 2024-09-21T04:39:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:45 smithi053 ceph-mon[80176]: pgmap v116: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:45.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.460+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff17019af40 con 0x7ff15807af80 2024-09-21T04:39:45.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.462+0000 7ff16d7fa640 1 -- 172.21.15.7:0/3835607883 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+385 (secure 0 0 0) 0x7ff17019af40 con 0x7ff15807af80 2024-09-21T04:39:45.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff15807af80 msgr2=0x7ff15807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff15807af80 0x7ff15807d440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff17019aae0 tx=0x7ff16000a2d0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 msgr2=0x7ff17019fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 0x7ff17019fd20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff164031460 tx=0x7ff1640048f0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 shutdown_connections 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff15807af80 0x7ff15807d440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff170100ff0 0x7ff17019fd20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 --2- 172.21.15.7:0/3835607883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff170100640 0x7ff17019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 >> 172.21.15.7:0/3835607883 conn(0x7ff1700fc550 msgr2=0x7ff1700fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 shutdown_connections 2024-09-21T04:39:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.464+0000 7ff1767e3640 1 -- 172.21.15.7:0/3835607883 wait complete. 2024-09-21T04:39:45.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.553+0000 7fa11e7b0640 1 -- 172.21.15.7:0/745982404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa118100510 msgr2=0x7fa118106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.553+0000 7fa11e7b0640 1 --2- 172.21.15.7:0/745982404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa118100510 0x7fa118106570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009880 tx=0x7fa10c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.553+0000 7fa11e7b0640 1 -- 172.21.15.7:0/745982404 shutdown_connections 2024-09-21T04:39:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.553+0000 7fa11e7b0640 1 --2- 172.21.15.7:0/745982404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa118100510 0x7fa118106570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.553+0000 7fa11e7b0640 1 --2- 172.21.15.7:0/745982404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1180ffb60 0x7fa1180fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.553+0000 7fa11e7b0640 1 -- 172.21.15.7:0/745982404 >> 172.21.15.7:0/745982404 conn(0x7fa1180fb870 msgr2=0x7fa1180fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.553+0000 7fa11e7b0640 1 -- 172.21.15.7:0/745982404 shutdown_connections 2024-09-21T04:39:45.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.553+0000 7fa11e7b0640 1 -- 172.21.15.7:0/745982404 wait complete. 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.554+0000 7fa11e7b0640 1 Processor -- start 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.554+0000 7fa11e7b0640 1 -- start start 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.554+0000 7fa11e7b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1180ffb60 0x7fa118101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.554+0000 7fa11e7b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa118100510 0x7fa1181018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa11e7b0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa11810eda0 con 0x7fa1180ffb60 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa11e7b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa11810ec20 con 0x7fa118100510 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa117fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1180ffb60 0x7fa118101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa117fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1180ffb60 0x7fa118101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54184/0 (socket says 172.21.15.7:54184) 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa117fff640 1 -- 172.21.15.7:0/415045245 learned_addr learned my addr 172.21.15.7:0/415045245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa1177fe640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa118100510 0x7fa1181018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa117fff640 1 -- 172.21.15.7:0/415045245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa118100510 msgr2=0x7fa1181018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa117fff640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa118100510 0x7fa1181018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa117fff640 1 -- 172.21.15.7:0/415045245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa118101df0 con 0x7fa1180ffb60 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa1177fe640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa118100510 0x7fa1181018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa117fff640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1180ffb60 0x7fa118101370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa10800c8f0 tx=0x7fa10800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa1157fa640 1 -- 172.21.15.7:0/415045245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa1080079b0 con 0x7fa1180ffb60 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa1157fa640 1 -- 172.21.15.7:0/415045245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1080054e0 con 0x7fa1180ffb60 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1181020e0 con 0x7fa1180ffb60 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa1157fa640 1 -- 172.21.15.7:0/415045245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa108004dd0 con 0x7fa1180ffb60 2024-09-21T04:39:45.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.555+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1181a3e70 con 0x7fa1180ffb60 2024-09-21T04:39:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.556+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa118100e10 con 0x7fa1180ffb60 2024-09-21T04:39:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.557+0000 7fa1157fa640 1 -- 172.21.15.7:0/415045245 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa108005020 con 0x7fa1180ffb60 2024-09-21T04:39:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.558+0000 7fa1157fa640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa10007af30 0x7fa10007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.558+0000 7fa1177fe640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa10007af30 0x7fa10007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.558+0000 7fa1157fa640 1 -- 172.21.15.7:0/415045245 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7fa10809dea0 con 0x7fa1180ffb60 2024-09-21T04:39:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.558+0000 7fa1177fe640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa10007af30 0x7fa10007d3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa10c031460 tx=0x7fa10c0095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.562+0000 7fa1157fa640 1 -- 172.21.15.7:0/415045245 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa108014030 con 0x7fa1180ffb60 2024-09-21T04:39:45.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.760+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa118062630 con 0x7fa10007af30 2024-09-21T04:39:45.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.771+0000 7fa1157fa640 1 -- 172.21.15.7:0/415045245 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7fa118062630 con 0x7fa10007af30 2024-09-21T04:39:45.775 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:39:45.775 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (106s) 20s ago 12m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:39:45.775 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (4m) 20s ago 12m 8824k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:39:45.775 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (4m) 8s ago 12m 8724k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:39:45.775 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (12m) 20s ago 12m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a 5e062d04556d 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (12m) 8s ago 12m 6895k - 19.1.1-373-g4e45e6b4 164465d2334a ed6a1e630193 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (83s) 20s ago 12m 75.5M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (7m) 20s ago 7m 146M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (7m) 20s ago 7m 89.2M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (7m) 8s ago 7m 26.1M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (7m) 8s ago 7m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (4m) 20s ago 14m 525M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (3m) 8s ago 11m 455M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (27s) 20s ago 14m 23.6M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (12s) 8s ago 11m 21.2M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:39:45.776 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (2m) 20s ago 12m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:39:45.777 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (2m) 8s ago 11m 11.2M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:39:45.777 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 20s ago 10m 211M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 1941886275ba 2024-09-21T04:39:45.777 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (10m) 20s ago 10m 214M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:39:45.777 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (9m) 20s ago 9m 194M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:39:45.777 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (9m) 8s ago 9m 243M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:39:45.777 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (8m) 8s ago 8m 213M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:39:45.777 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (8m) 8s ago 8m 184M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:39:45.777 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (2m) 20s ago 12m 49.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.773+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa10007af30 msgr2=0x7fa10007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.773+0000 7fa11e7b0640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa10007af30 0x7fa10007d3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa10c031460 tx=0x7fa10c0095f0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.773+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1180ffb60 msgr2=0x7fa118101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.773+0000 7fa11e7b0640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1180ffb60 0x7fa118101370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa10800c8f0 tx=0x7fa10800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.774+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 shutdown_connections 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.774+0000 7fa11e7b0640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa10007af30 0x7fa10007d3f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.774+0000 7fa11e7b0640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa118100510 0x7fa1181018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.774+0000 7fa11e7b0640 1 --2- 172.21.15.7:0/415045245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1180ffb60 0x7fa118101370 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.774+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 >> 172.21.15.7:0/415045245 conn(0x7fa1180fb870 msgr2=0x7fa1180fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.774+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 shutdown_connections 2024-09-21T04:39:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.774+0000 7fa11e7b0640 1 -- 172.21.15.7:0/415045245 wait complete. 2024-09-21T04:39:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.858+0000 7f9e2551e640 1 -- 172.21.15.7:0/491693503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201007f0 msgr2=0x7f9e20100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.858+0000 7f9e2551e640 1 --2- 172.21.15.7:0/491693503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201007f0 0x7f9e20100c50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009880 tx=0x7f9e0802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.859+0000 7f9e2551e640 1 -- 172.21.15.7:0/491693503 shutdown_connections 2024-09-21T04:39:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.859+0000 7f9e2551e640 1 --2- 172.21.15.7:0/491693503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201007f0 0x7f9e20100c50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.859+0000 7f9e2551e640 1 --2- 172.21.15.7:0/491693503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e201067f0 0x7f9e20106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.859+0000 7f9e2551e640 1 -- 172.21.15.7:0/491693503 >> 172.21.15.7:0/491693503 conn(0x7f9e200fc550 msgr2=0x7f9e200fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.859+0000 7f9e2551e640 1 -- 172.21.15.7:0/491693503 shutdown_connections 2024-09-21T04:39:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.859+0000 7f9e2551e640 1 -- 172.21.15.7:0/491693503 wait complete. 2024-09-21T04:39:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.859+0000 7f9e2551e640 1 Processor -- start 2024-09-21T04:39:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.859+0000 7f9e2551e640 1 -- start start 2024-09-21T04:39:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e2551e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e201007f0 0x7f9e201987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e2551e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201067f0 0x7f9e20198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e2551e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2010dca0 con 0x7f9e201067f0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e2551e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2010db20 con 0x7f9e201007f0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e1e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201067f0 0x7f9e20198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e1e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201067f0 0x7f9e20198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54202/0 (socket says 172.21.15.7:54202) 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e1e7fc640 1 -- 172.21.15.7:0/524649366 learned_addr learned my addr 172.21.15.7:0/524649366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e1effd640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e201007f0 0x7f9e201987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e1e7fc640 1 -- 172.21.15.7:0/524649366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e201007f0 msgr2=0x7f9e201987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e1e7fc640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e201007f0 0x7f9e201987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e1e7fc640 1 -- 172.21.15.7:0/524649366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e201992f0 con 0x7f9e201067f0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.860+0000 7f9e1e7fc640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201067f0 0x7f9e20198cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9e08031460 tx=0x7f9e080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.861+0000 7f9e17fff640 1 -- 172.21.15.7:0/524649366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9e0803f070 con 0x7f9e201067f0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.861+0000 7f9e17fff640 1 -- 172.21.15.7:0/524649366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e080035f0 con 0x7f9e201067f0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.861+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e20199580 con 0x7f9e201067f0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.861+0000 7f9e17fff640 1 -- 172.21.15.7:0/524649366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9e08035030 con 0x7f9e201067f0 2024-09-21T04:39:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.861+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e201a1b30 con 0x7f9e201067f0 2024-09-21T04:39:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.862+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e20101f30 con 0x7f9e201067f0 2024-09-21T04:39:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.863+0000 7f9e17fff640 1 -- 172.21.15.7:0/524649366 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9e08042070 con 0x7f9e201067f0 2024-09-21T04:39:45.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.863+0000 7f9e17fff640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9e0c07af80 0x7f9e0c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.863+0000 7f9e17fff640 1 -- 172.21.15.7:0/524649366 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f9e080be810 con 0x7f9e201067f0 2024-09-21T04:39:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.863+0000 7f9e1effd640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9e0c07af80 0x7f9e0c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.864+0000 7f9e1effd640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9e0c07af80 0x7f9e0c07d440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e10005fd0 tx=0x7f9e10009760 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:45.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:45.868+0000 7f9e17fff640 1 -- 172.21.15.7:0/524649366 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e08089ee0 con 0x7f9e201067f0 2024-09-21T04:39:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.134+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f9e201a1ea0 con 0x7f9e201067f0 2024-09-21T04:39:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.134+0000 7f9e17fff640 1 -- 172.21.15.7:0/524649366 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+733 (secure 0 0 0) 0x7f9e0808ed90 con 0x7f9e201067f0 2024-09-21T04:39:46.137 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:39:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 10, 2024-09-21T04:39:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:39:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:39:46.139 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:39:46.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.136+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9e0c07af80 msgr2=0x7f9e0c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.136+0000 7f9e2551e640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9e0c07af80 0x7f9e0c07d440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e10005fd0 tx=0x7f9e10009760 comp rx=0 tx=0).stop 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.136+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201067f0 msgr2=0x7f9e20198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.136+0000 7f9e2551e640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201067f0 0x7f9e20198cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9e08031460 tx=0x7f9e080045e0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.137+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 shutdown_connections 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.137+0000 7f9e2551e640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9e0c07af80 0x7f9e0c07d440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.137+0000 7f9e2551e640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e201067f0 0x7f9e20198cf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.137+0000 7f9e2551e640 1 --2- 172.21.15.7:0/524649366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e201007f0 0x7f9e201987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.137+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 >> 172.21.15.7:0/524649366 conn(0x7f9e200fc550 msgr2=0x7f9e200fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.137+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 shutdown_connections 2024-09-21T04:39:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.137+0000 7f9e2551e640 1 -- 172.21.15.7:0/524649366 wait complete. 2024-09-21T04:39:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.221+0000 7f8d5d510640 1 -- 172.21.15.7:0/3738850888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58107900 msgr2=0x7f8d58107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.221+0000 7f8d5d510640 1 --2- 172.21.15.7:0/3738850888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58107900 0x7f8d58107ce0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009880 tx=0x7f8d4802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.221+0000 7f8d5d510640 1 -- 172.21.15.7:0/3738850888 shutdown_connections 2024-09-21T04:39:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.221+0000 7f8d5d510640 1 --2- 172.21.15.7:0/3738850888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d58101900 0x7f8d58101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.221+0000 7f8d5d510640 1 --2- 172.21.15.7:0/3738850888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58107900 0x7f8d58107ce0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.221+0000 7f8d5d510640 1 -- 172.21.15.7:0/3738850888 >> 172.21.15.7:0/3738850888 conn(0x7f8d580fd600 msgr2=0x7f8d580ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:46.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.221+0000 7f8d5d510640 1 -- 172.21.15.7:0/3738850888 shutdown_connections 2024-09-21T04:39:46.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.221+0000 7f8d5d510640 1 -- 172.21.15.7:0/3738850888 wait complete. 2024-09-21T04:39:46.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d5d510640 1 Processor -- start 2024-09-21T04:39:46.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d5d510640 1 -- start start 2024-09-21T04:39:46.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d5d510640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58101900 0x7f8d580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d5d510640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d58107900 0x7f8d58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d5d510640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5810edb0 con 0x7f8d58101900 2024-09-21T04:39:46.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d5d510640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5810ec30 con 0x7f8d58107900 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d56ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58101900 0x7f8d580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d56ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58101900 0x7f8d580744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54228/0 (socket says 172.21.15.7:54228) 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d56ffd640 1 -- 172.21.15.7:0/1812486520 learned_addr learned my addr 172.21.15.7:0/1812486520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.222+0000 7f8d567fc640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d58107900 0x7f8d58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d56ffd640 1 -- 172.21.15.7:0/1812486520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d58107900 msgr2=0x7f8d58074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d56ffd640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d58107900 0x7f8d58074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d56ffd640 1 -- 172.21.15.7:0/1812486520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d58074f70 con 0x7f8d58101900 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d56ffd640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58101900 0x7f8d580744f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d48031460 tx=0x7f8d480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1812486520 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8d4803f070 con 0x7f8d58101900 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d58075200 con 0x7f8d58101900 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d581a3d80 con 0x7f8d58101900 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1812486520 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d480035f0 con 0x7f8d58101900 2024-09-21T04:39:46.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.223+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1812486520 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8d48035030 con 0x7f8d58101900 2024-09-21T04:39:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.224+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d44005290 con 0x7f8d58101900 2024-09-21T04:39:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.225+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1812486520 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8d48042070 con 0x7f8d58101900 2024-09-21T04:39:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.226+0000 7f8d3ffff640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d4007af80 0x7f8d4007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.226+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1812486520 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f8d480be830 con 0x7f8d58101900 2024-09-21T04:39:46.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.226+0000 7f8d567fc640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d4007af80 0x7f8d4007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.226+0000 7f8d567fc640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d4007af80 0x7f8d4007d440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8d58075f80 tx=0x7f8d4c00b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:46.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.230+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1812486520 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d48089e60 con 0x7f8d58101900 2024-09-21T04:39:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.458+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8d44005d90 con 0x7f8d58101900 2024-09-21T04:39:46.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.458+0000 7f8d3ffff640 1 -- 172.21.15.7:0/1812486520 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2038 (secure 0 0 0) 0x7f8d4808ed10 con 0x7f8d58101900 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:39:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 44 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:39:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14578,1=24327} 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:39:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:39:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:39:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:39:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d4007af80 msgr2=0x7f8d4007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d4007af80 0x7f8d4007d440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8d58075f80 tx=0x7f8d4c00b410 comp rx=0 tx=0).stop 2024-09-21T04:39:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58101900 msgr2=0x7f8d580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58101900 0x7f8d580744f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d48031460 tx=0x7f8d480045e0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 shutdown_connections 2024-09-21T04:39:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d4007af80 0x7f8d4007d440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d58107900 0x7f8d58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 --2- 172.21.15.7:0/1812486520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d58101900 0x7f8d580744f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 >> 172.21.15.7:0/1812486520 conn(0x7f8d580fd600 msgr2=0x7f8d580ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 shutdown_connections 2024-09-21T04:39:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.461+0000 7f8d5d510640 1 -- 172.21.15.7:0/1812486520 wait complete. 2024-09-21T04:39:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 -- 172.21.15.7:0/210915771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 msgr2=0x7f455010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 --2- 172.21.15.7:0/210915771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 0x7f455010b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4540009880 tx=0x7f454002a530 comp rx=0 tx=0).stop 2024-09-21T04:39:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 -- 172.21.15.7:0/210915771 shutdown_connections 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 --2- 172.21.15.7:0/210915771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 0x7f455010b8d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 --2- 172.21.15.7:0/210915771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4550102ac0 0x7f4550102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 -- 172.21.15.7:0/210915771 >> 172.21.15.7:0/210915771 conn(0x7f45500fc520 msgr2=0x7f45500fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 -- 172.21.15.7:0/210915771 shutdown_connections 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 -- 172.21.15.7:0/210915771 wait complete. 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 Processor -- start 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.547+0000 7f4556af1640 1 -- start start 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f4556af1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4550102ac0 0x7f455019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f4556af1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 0x7f455019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f4556af1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f455010edf0 con 0x7f45501033e0 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f4556af1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f455010ec70 con 0x7f4550102ac0 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 0x7f455019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 0x7f455019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54252/0 (socket says 172.21.15.7:54252) 2024-09-21T04:39:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454ffff640 1 -- 172.21.15.7:0/1538357720 learned_addr learned my addr 172.21.15.7:0/1538357720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f455488d640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4550102ac0 0x7f455019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454ffff640 1 -- 172.21.15.7:0/1538357720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4550102ac0 msgr2=0x7f455019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454ffff640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4550102ac0 0x7f455019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454ffff640 1 -- 172.21.15.7:0/1538357720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4550199a20 con 0x7f45501033e0 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454ffff640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 0x7f455019fda0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4540031460 tx=0x7f4540004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454dffb640 1 -- 172.21.15.7:0/1538357720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f454003f070 con 0x7f45501033e0 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454dffb640 1 -- 172.21.15.7:0/1538357720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4540004a90 con 0x7f45501033e0 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4550199cb0 con 0x7f45501033e0 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.548+0000 7f454dffb640 1 -- 172.21.15.7:0/1538357720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4540035030 con 0x7f45501033e0 2024-09-21T04:39:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.549+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455019a1e0 con 0x7f45501033e0 2024-09-21T04:39:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.550+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455006aa90 con 0x7f45501033e0 2024-09-21T04:39:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.550+0000 7f454dffb640 1 -- 172.21.15.7:0/1538357720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4540042070 con 0x7f45501033e0 2024-09-21T04:39:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.551+0000 7f454dffb640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f453c07af90 0x7f453c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.551+0000 7f454dffb640 1 -- 172.21.15.7:0/1538357720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f45400be970 con 0x7f45501033e0 2024-09-21T04:39:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.551+0000 7f455488d640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f453c07af90 0x7f453c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.551+0000 7f455488d640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f453c07af90 0x7f453c07d450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f455019ab60 tx=0x7f453800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:46.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.555+0000 7f454dffb640 1 -- 172.21.15.7:0/1538357720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45400c0e20 con 0x7f45501033e0 2024-09-21T04:39:46.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.759+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f455019af50 con 0x7f453c07af90 2024-09-21T04:39:46.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.760+0000 7f454dffb640 1 -- 172.21.15.7:0/1538357720 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+385 (secure 0 0 0) 0x7f455019af50 con 0x7f453c07af90 2024-09-21T04:39:46.763 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:39:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:39:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:39:46.763 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:39:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:39:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:39:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-09-21T04:39:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:39:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "4/23 daemons upgraded", 2024-09-21T04:39:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading crash daemons", 2024-09-21T04:39:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:39:46.764 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:39:46.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f453c07af90 msgr2=0x7f453c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f453c07af90 0x7f453c07d450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f455019ab60 tx=0x7f453800b390 comp rx=0 tx=0).stop 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 msgr2=0x7f455019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 0x7f455019fda0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4540031460 tx=0x7f4540004530 comp rx=0 tx=0).stop 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 shutdown_connections 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f453c07af90 0x7f453c07d450 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45501033e0 0x7f455019fda0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 --2- 172.21.15.7:0/1538357720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4550102ac0 0x7f455019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 >> 172.21.15.7:0/1538357720 conn(0x7f45500fc520 msgr2=0x7f4550100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 shutdown_connections 2024-09-21T04:39:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.763+0000 7f4556af1640 1 -- 172.21.15.7:0/1538357720 wait complete. 2024-09-21T04:39:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.846+0000 7f43623ad640 1 -- 172.21.15.7:0/521472713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 msgr2=0x7f435c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.846+0000 7f43623ad640 1 --2- 172.21.15.7:0/521472713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 0x7f435c10b8c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4350009880 tx=0x7f435002a530 comp rx=0 tx=0).stop 2024-09-21T04:39:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.847+0000 7f43623ad640 1 -- 172.21.15.7:0/521472713 shutdown_connections 2024-09-21T04:39:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.847+0000 7f43623ad640 1 --2- 172.21.15.7:0/521472713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 0x7f435c10b8c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.847+0000 7f43623ad640 1 --2- 172.21.15.7:0/521472713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f435c102ab0 0x7f435c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.847+0000 7f43623ad640 1 -- 172.21.15.7:0/521472713 >> 172.21.15.7:0/521472713 conn(0x7f435c0fc550 msgr2=0x7f435c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.847+0000 7f43623ad640 1 -- 172.21.15.7:0/521472713 shutdown_connections 2024-09-21T04:39:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.847+0000 7f43623ad640 1 -- 172.21.15.7:0/521472713 wait complete. 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.847+0000 7f43623ad640 1 Processor -- start 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.847+0000 7f43623ad640 1 -- start start 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f43623ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f435c102ab0 0x7f435c100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f43623ad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 0x7f435c100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f43623ad640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f435c10edc0 con 0x7f435c1033d0 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f43623ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f435c10ec40 con 0x7f435c102ab0 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 0x7f435c100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 0x7f435c100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54256/0 (socket says 172.21.15.7:54256) 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f435b7fe640 1 -- 172.21.15.7:0/3790018515 learned_addr learned my addr 172.21.15.7:0/3790018515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f435bfff640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f435c102ab0 0x7f435c100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f435b7fe640 1 -- 172.21.15.7:0/3790018515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f435c102ab0 msgr2=0x7f435c100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f435b7fe640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f435c102ab0 0x7f435c100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f435b7fe640 1 -- 172.21.15.7:0/3790018515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c101190 con 0x7f435c1033d0 2024-09-21T04:39:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.848+0000 7f435b7fe640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 0x7f435c100c50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4350031460 tx=0x7f43500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.849+0000 7f43597fa640 1 -- 172.21.15.7:0/3790018515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f435003f070 con 0x7f435c1033d0 2024-09-21T04:39:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.849+0000 7f43597fa640 1 -- 172.21.15.7:0/3790018515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4350004ac0 con 0x7f435c1033d0 2024-09-21T04:39:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.849+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c101420 con 0x7f435c1033d0 2024-09-21T04:39:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.849+0000 7f43597fa640 1 -- 172.21.15.7:0/3790018515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4350035030 con 0x7f435c1033d0 2024-09-21T04:39:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.849+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1a3d00 con 0x7f435c1033d0 2024-09-21T04:39:46.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.850+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435c06a580 con 0x7f435c1033d0 2024-09-21T04:39:46.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.851+0000 7f43597fa640 1 -- 172.21.15.7:0/3790018515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4350042070 con 0x7f435c1033d0 2024-09-21T04:39:46.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.851+0000 7f43597fa640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f434407af30 0x7f434407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:39:46.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.851+0000 7f43597fa640 1 -- 172.21.15.7:0/3790018515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6070+0+0 (secure 0 0 0) 0x7f43500be980 con 0x7f435c1033d0 2024-09-21T04:39:46.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.851+0000 7f435bfff640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f434407af30 0x7f434407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:39:46.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.852+0000 7f435bfff640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f434407af30 0x7f434407d3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f434c00abe0 tx=0x7f434c00a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:39:46.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:46.855+0000 7f43597fa640 1 -- 172.21.15.7:0/3790018515 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4350089f30 con 0x7f435c1033d0 2024-09-21T04:39:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.119+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f435c1a3f60 con 0x7f435c1033d0 2024-09-21T04:39:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.119+0000 7f43597fa640 1 -- 172.21.15.7:0/3790018515 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7f435008ede0 con 0x7f435c1033d0 2024-09-21T04:39:47.123 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:47.123 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:39:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:39:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f434407af30 msgr2=0x7f434407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f434407af30 0x7f434407d3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f434c00abe0 tx=0x7f434c00a2d0 comp rx=0 tx=0).stop 2024-09-21T04:39:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 msgr2=0x7f435c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:39:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 0x7f435c100c50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4350031460 tx=0x7f43500046c0 comp rx=0 tx=0).stop 2024-09-21T04:39:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 shutdown_connections 2024-09-21T04:39:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f434407af30 0x7f434407d3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435c1033d0 0x7f435c100c50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 --2- 172.21.15.7:0/3790018515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f435c102ab0 0x7f435c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:39:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 >> 172.21.15.7:0/3790018515 conn(0x7f435c0fc550 msgr2=0x7f435c10aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:39:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 shutdown_connections 2024-09-21T04:39:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:39:47.123+0000 7f43623ad640 1 -- 172.21.15.7:0/3790018515 wait complete. 2024-09-21T04:39:47.317 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:47 smithi007 ceph-mon[108238]: from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:47.318 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:47 smithi007 ceph-mon[108238]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:47.318 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:47 smithi007 ceph-mon[108238]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:47.318 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:47 smithi007 ceph-mon[108238]: pgmap v117: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:47.318 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:47 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:47.318 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:47 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:47.318 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:47 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/524649366' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:39:47.318 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:47 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1812486520' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:39:47.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:47 smithi053 ceph-mon[80176]: from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:47.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:47 smithi053 ceph-mon[80176]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:47.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:47 smithi053 ceph-mon[80176]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:47.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:47 smithi053 ceph-mon[80176]: pgmap v117: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:47.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:47 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:47.409 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:47 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:47.410 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:47 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/524649366' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:39:47.410 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:47 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1812486520' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:39:48.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:48 smithi007 ceph-mon[108238]: from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:48.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:48 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3790018515' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:39:48.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:48 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:48.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:48 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:39:48.319 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:48 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:39:48.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:48 smithi053 ceph-mon[80176]: from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:39:48.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:48 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3790018515' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:39:48.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:48 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:48.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:48 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi053", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-21T04:39:48.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:48 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:39:49.313 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:49 smithi053 ceph-mon[80176]: Upgrade: Updating crash.smithi053 (2/2) 2024-09-21T04:39:49.313 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:49 smithi053 ceph-mon[80176]: Deploying daemon crash.smithi053 on smithi053 2024-09-21T04:39:49.313 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:49 smithi053 ceph-mon[80176]: pgmap v118: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:49 smithi007 ceph-mon[108238]: Upgrade: Updating crash.smithi053 (2/2) 2024-09-21T04:39:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:49 smithi007 ceph-mon[108238]: Deploying daemon crash.smithi053 on smithi053 2024-09-21T04:39:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:49 smithi007 ceph-mon[108238]: pgmap v118: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:51.338 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:51 smithi053 ceph-mon[80176]: pgmap v119: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:51.338 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:51 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:51.338 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:51 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:51.338 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:51 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:39:51.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:51 smithi007 ceph-mon[108238]: pgmap v119: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:51.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:51 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:51.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:51 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:51.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:51 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:39:53.148 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:53 smithi007 ceph-mon[108238]: pgmap v120: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:53 smithi053 ceph-mon[80176]: pgmap v120: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:54 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:54 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:54 smithi007 ceph-mon[108238]: pgmap v121: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:54 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:54 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:54 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:54 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:54 smithi053 ceph-mon[80176]: pgmap v121: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:39:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:54 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:54 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:57.065 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:56 smithi007 ceph-mon[108238]: pgmap v122: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:57.065 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:56 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:57.065 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:56 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:57.065 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:56 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:57.065 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:56 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:57.170 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:56 smithi053 ceph-mon[80176]: pgmap v122: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:57.171 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:56 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:57.171 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:56 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:57.171 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:56 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:57.171 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:56 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:58 smithi007 ceph-mon[108238]: pgmap v123: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:39:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:39:58.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:58.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:39:58.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:58 smithi053 ceph-mon[80176]: pgmap v123: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:39:58.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:39:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: pgmap v124: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:40:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:40:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:40:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all crash 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi007"} : dispatch 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi007"}]': finished 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi053"} : dispatch 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi053"}]': finished 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16} : dispatch 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: Upgrade: osd.0 is safe to restart 2024-09-21T04:40:01.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:00 smithi053 ceph-mon[80176]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: pgmap v124: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all crash 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi007"} : dispatch 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi007"}]': finished 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi053"} : dispatch 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi053"}]': finished 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16} : dispatch 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: Upgrade: osd.0 is safe to restart 2024-09-21T04:40:01.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:00 smithi007 ceph-mon[108238]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:40:02.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:02 smithi053 ceph-mon[80176]: Upgrade: Updating osd.0 2024-09-21T04:40:02.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:02 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:02.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:02 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T04:40:02.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:02 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:40:02.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:02 smithi053 ceph-mon[80176]: Deploying daemon osd.0 on smithi007 2024-09-21T04:40:02.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:02 smithi053 ceph-mon[80176]: pgmap v125: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:40:02.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:02 smithi007 ceph-mon[108238]: Upgrade: Updating osd.0 2024-09-21T04:40:02.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:02 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:02.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:02 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T04:40:02.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:02 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:40:02.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:02 smithi007 ceph-mon[108238]: Deploying daemon osd.0 on smithi007 2024-09-21T04:40:02.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:02 smithi007 ceph-mon[108238]: pgmap v125: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:40:04.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:04 smithi007 systemd[1]: Stopping Ceph osd.0 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:40:04.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:04 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[48982]: 2024-09-21T04:40:04.654+0000 7f639bd7b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:40:04.695 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:04 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[48982]: 2024-09-21T04:40:04.654+0000 7f639bd7b640 -1 osd.0 46 *** Got signal Terminated *** 2024-09-21T04:40:04.695 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:04 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[48982]: 2024-09-21T04:40:04.654+0000 7f639bd7b640 -1 osd.0 46 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:40:05.188 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:04 smithi007 ceph-mon[108238]: pgmap v126: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:40:05.188 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:04 smithi007 ceph-mon[108238]: osd.0 marked itself down and dead 2024-09-21T04:40:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:04 smithi053 ceph-mon[80176]: pgmap v126: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:40:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:04 smithi053 ceph-mon[80176]: osd.0 marked itself down and dead 2024-09-21T04:40:05.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:05 smithi007 podman[116196]: 2024-09-21 04:40:05.186254111 +0000 UTC m=+0.624769300 container died 1941886275ba90be23449eff5eff96dde7eb459f51dda00ef346e3e63cfc08ab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, RELEASE=squid-4e45e6b, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:40:05.809 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:05 smithi007 podman[116196]: 2024-09-21 04:40:05.613220693 +0000 UTC m=+1.051735889 container cleanup 1941886275ba90be23449eff5eff96dde7eb459f51dda00ef346e3e63cfc08ab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-4e45e6b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-21T04:40:05.809 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:05 smithi007 bash[116196]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0 2024-09-21T04:40:06.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:05 smithi053 ceph-mon[80176]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:40:06.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:05 smithi053 ceph-mon[80176]: osdmap e47: 6 total, 5 up, 6 in 2024-09-21T04:40:06.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:05 smithi053 ceph-mon[80176]: osdmap e48: 6 total, 5 up, 6 in 2024-09-21T04:40:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:05 smithi007 ceph-mon[108238]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:40:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:05 smithi007 ceph-mon[108238]: osdmap e47: 6 total, 5 up, 6 in 2024-09-21T04:40:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:05 smithi007 ceph-mon[108238]: osdmap e48: 6 total, 5 up, 6 in 2024-09-21T04:40:06.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:05 smithi007 podman[116212]: 2024-09-21 04:40:05.806399062 +0000 UTC m=+0.618807517 container remove 1941886275ba90be23449eff5eff96dde7eb459f51dda00ef346e3e63cfc08ab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, RELEASE=squid-4e45e6b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:40:06.678 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:06 smithi007 podman[116302]: 2024-09-21 04:40:06.241145733 +0000 UTC m=+0.106692097 container create 743fa5dd5f1320c423cd53854c2bc213f3f0eaeac33ef017c160830063f881b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T04:40:06.678 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:06 smithi007 podman[116302]: 2024-09-21 04:40:06.151308281 +0000 UTC m=+0.016854657 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:40:06.678 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:06 smithi007 podman[116302]: 2024-09-21 04:40:06.460466265 +0000 UTC m=+0.326012641 container init 743fa5dd5f1320c423cd53854c2bc213f3f0eaeac33ef017c160830063f881b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:40:06.679 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:06 smithi007 podman[116302]: 2024-09-21 04:40:06.463849657 +0000 UTC m=+0.329396019 container start 743fa5dd5f1320c423cd53854c2bc213f3f0eaeac33ef017c160830063f881b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T04:40:06.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:06 smithi053 ceph-mon[80176]: pgmap v128: 65 pgs: 9 stale+active+clean, 56 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:40:06.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:06 smithi007 ceph-mon[108238]: pgmap v128: 65 pgs: 9 stale+active+clean, 56 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:40:06.944 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:06 smithi007 podman[116302]: 2024-09-21 04:40:06.754018609 +0000 UTC m=+0.619564980 container attach 743fa5dd5f1320c423cd53854c2bc213f3f0eaeac33ef017c160830063f881b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731) 2024-09-21T04:40:06.944 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:06 smithi007 podman[116302]: 2024-09-21 04:40:06.754241378 +0000 UTC m=+0.619787736 container died 743fa5dd5f1320c423cd53854c2bc213f3f0eaeac33ef017c160830063f881b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T04:40:07.657 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:07 smithi007 podman[116325]: 2024-09-21 04:40:07.397586997 +0000 UTC m=+0.713895687 container remove 743fa5dd5f1320c423cd53854c2bc213f3f0eaeac33ef017c160830063f881b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T04:40:07.657 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:07 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.0.service: Deactivated successfully. 2024-09-21T04:40:07.657 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:07 smithi007 systemd[1]: Stopped Ceph osd.0 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:40:07.658 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:07 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.0.service: Consumed 55.253s CPU time. 2024-09-21T04:40:07.929 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:07 smithi007 systemd[1]: Starting Ceph osd.0 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:40:07.929 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:07 smithi007 podman[116426]: 2024-09-21 04:40:07.865670528 +0000 UTC m=+0.109206634 container create 7cc356b0df887d1ad0547f486b771dd3a718ac3e3db402789c35023be92c5b4d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:40:07.929 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:07 smithi007 podman[116426]: 2024-09-21 04:40:07.773164523 +0000 UTC m=+0.016700587 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:40:08.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:08 smithi007 podman[116426]: 2024-09-21 04:40:08.101962755 +0000 UTC m=+0.345498819 container init 7cc356b0df887d1ad0547f486b771dd3a718ac3e3db402789c35023be92c5b4d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T04:40:08.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:08 smithi007 podman[116426]: 2024-09-21 04:40:08.105478812 +0000 UTC m=+0.349014864 container start 7cc356b0df887d1ad0547f486b771dd3a718ac3e3db402789c35023be92c5b4d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T04:40:08.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:08 smithi007 podman[116426]: 2024-09-21 04:40:08.207412365 +0000 UTC m=+0.450948412 container attach 7cc356b0df887d1ad0547f486b771dd3a718ac3e3db402789c35023be92c5b4d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T04:40:08.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:08 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:08.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:08 smithi007 bash[116426]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:08.695 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:08 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:08.695 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:08 smithi007 bash[116426]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:08 smithi053 ceph-mon[80176]: pgmap v130: 65 pgs: 9 stale+active+clean, 56 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T04:40:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:08 smithi007 ceph-mon[108238]: pgmap v130: 65 pgs: 9 stale+active+clean, 56 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T04:40:10.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:09 smithi053 ceph-mon[80176]: Health check failed: Degraded data redundancy: 65/333 objects degraded (19.520%), 20 pgs degraded (PG_DEGRADED) 2024-09-21T04:40:10.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:09 smithi007 ceph-mon[108238]: Health check failed: Degraded data redundancy: 65/333 objects degraded (19.520%), 20 pgs degraded (PG_DEGRADED) 2024-09-21T04:40:11.182 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:10 smithi007 ceph-mon[108238]: pgmap v131: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:10 smithi053 ceph-mon[80176]: pgmap v131: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:11.446 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-09-21T04:40:11.447 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-09-21T04:40:11.944 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-21T04:40:11.944 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-21T04:40:11.944 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-09-21T04:40:11.945 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-09-21T04:40:11.945 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T04:40:11.945 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T04:40:11.945 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T04:40:11.945 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T04:40:11.945 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate[116440]: --> ceph-volume lvm activate successful for osd ID: 0 2024-09-21T04:40:11.945 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 bash[116426]: --> ceph-volume lvm activate successful for osd ID: 0 2024-09-21T04:40:11.945 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:11 smithi007 podman[116426]: 2024-09-21 04:40:11.570621153 +0000 UTC m=+3.814157207 container died 7cc356b0df887d1ad0547f486b771dd3a718ac3e3db402789c35023be92c5b4d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:40:12.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:12 smithi007 podman[116668]: 2024-09-21 04:40:12.397718489 +0000 UTC m=+0.815335125 container remove 7cc356b0df887d1ad0547f486b771dd3a718ac3e3db402789c35023be92c5b4d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:40:12.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:12 smithi007 podman[116727]: 2024-09-21 04:40:12.524397058 +0000 UTC m=+0.017656092 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:40:12.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:12 smithi007 podman[116727]: 2024-09-21 04:40:12.623972349 +0000 UTC m=+0.117231413 container create 9ec59c74f4bd9763e1616c0aa7ff5093c4feff2af2a651601b057ba7baaab39c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T04:40:13.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:12 smithi053 ceph-mon[80176]: pgmap v132: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:12 smithi007 ceph-mon[108238]: pgmap v132: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:13.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:12 smithi007 podman[116727]: 2024-09-21 04:40:12.851299531 +0000 UTC m=+0.344558554 container init 9ec59c74f4bd9763e1616c0aa7ff5093c4feff2af2a651601b057ba7baaab39c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T04:40:13.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:12 smithi007 podman[116727]: 2024-09-21 04:40:12.854704027 +0000 UTC m=+0.347963056 container start 9ec59c74f4bd9763e1616c0aa7ff5093c4feff2af2a651601b057ba7baaab39c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:40:13.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:12 smithi007 ceph-osd[116746]: -- 172.21.15.7:0/1385914257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557046b6e1e0 con 0x557045d09c00 2024-09-21T04:40:13.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:12 smithi007 bash[116727]: 9ec59c74f4bd9763e1616c0aa7ff5093c4feff2af2a651601b057ba7baaab39c 2024-09-21T04:40:13.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:12 smithi007 systemd[1]: Started Ceph osd.0 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:40:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:40:14.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:14.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:40:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:14.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:14 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[116742]: 2024-09-21T04:40:14.511+0000 7f7d4ba15640 -1 Falling back to public interface 2024-09-21T04:40:15.281 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:15 smithi007 ceph-mon[108238]: pgmap v133: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:15 smithi053 ceph-mon[80176]: pgmap v133: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:17 smithi053 ceph-mon[80176]: pgmap v134: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:17 smithi007 ceph-mon[108238]: pgmap v134: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.220+0000 7f7177d1c640 1 -- 172.21.15.7:0/1710576747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701067f0 msgr2=0x7f7170106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.220+0000 7f7177d1c640 1 --2- 172.21.15.7:0/1710576747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701067f0 0x7f7170106bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f716c009880 tx=0x7f716c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.220+0000 7f7177d1c640 1 -- 172.21.15.7:0/1710576747 shutdown_connections 2024-09-21T04:40:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.220+0000 7f7177d1c640 1 --2- 172.21.15.7:0/1710576747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71701007f0 0x7f7170100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.220+0000 7f7177d1c640 1 --2- 172.21.15.7:0/1710576747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701067f0 0x7f7170106bd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.220+0000 7f7177d1c640 1 -- 172.21.15.7:0/1710576747 >> 172.21.15.7:0/1710576747 conn(0x7f71700fc550 msgr2=0x7f71700fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.221+0000 7f7177d1c640 1 -- 172.21.15.7:0/1710576747 shutdown_connections 2024-09-21T04:40:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.221+0000 7f7177d1c640 1 -- 172.21.15.7:0/1710576747 wait complete. 2024-09-21T04:40:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.221+0000 7f7177d1c640 1 Processor -- start 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.221+0000 7f7177d1c640 1 -- start start 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7177d1c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701007f0 0x7f717010b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7177d1c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71701067f0 0x7f717010bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7177d1c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f717010edc0 con 0x7f71701007f0 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7177d1c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f717010ec40 con 0x7f71701067f0 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7175ab8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701007f0 0x7f717010b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7175ab8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701007f0 0x7f717010b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50196/0 (socket says 172.21.15.7:50196) 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7175ab8640 1 -- 172.21.15.7:0/3053601072 learned_addr learned my addr 172.21.15.7:0/3053601072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f71752b7640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71701067f0 0x7f717010bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7175ab8640 1 -- 172.21.15.7:0/3053601072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71701067f0 msgr2=0x7f717010bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7175ab8640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71701067f0 0x7f717010bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7175ab8640 1 -- 172.21.15.7:0/3053601072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717010c200 con 0x7f71701007f0 2024-09-21T04:40:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7175ab8640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701007f0 0x7f717010b780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f716c031460 tx=0x7f716c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7166ffd640 1 -- 172.21.15.7:0/3053601072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f716c03f070 con 0x7f71701007f0 2024-09-21T04:40:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717010c490 con 0x7f71701007f0 2024-09-21T04:40:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7166ffd640 1 -- 172.21.15.7:0/3053601072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f716c004b10 con 0x7f71701007f0 2024-09-21T04:40:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.222+0000 7f7166ffd640 1 -- 172.21.15.7:0/3053601072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f716c035030 con 0x7f71701007f0 2024-09-21T04:40:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.223+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71701a3e70 con 0x7f71701007f0 2024-09-21T04:40:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.223+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7170101f30 con 0x7f71701007f0 2024-09-21T04:40:17.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.224+0000 7f7166ffd640 1 -- 172.21.15.7:0/3053601072 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f716c042070 con 0x7f71701007f0 2024-09-21T04:40:17.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.225+0000 7f7166ffd640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f715c07af40 0x7f715c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.225+0000 7f7166ffd640 1 -- 172.21.15.7:0/3053601072 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6070+0+0 (secure 0 0 0) 0x7f716c0beaf0 con 0x7f71701007f0 2024-09-21T04:40:17.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.225+0000 7f71752b7640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f715c07af40 0x7f715c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.225+0000 7f71752b7640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f715c07af40 0x7f715c07d400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f717010d210 tx=0x7f716000b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.230+0000 7f7166ffd640 1 -- 172.21.15.7:0/3053601072 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f716c08a0a0 con 0x7f71701007f0 2024-09-21T04:40:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.436+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7170062630 con 0x7f715c07af40 2024-09-21T04:40:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.437+0000 7f7166ffd640 1 -- 172.21.15.7:0/3053601072 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f7170062630 con 0x7f715c07af40 2024-09-21T04:40:17.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f715c07af40 msgr2=0x7f715c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f715c07af40 0x7f715c07d400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f717010d210 tx=0x7f716000b410 comp rx=0 tx=0).stop 2024-09-21T04:40:17.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701007f0 msgr2=0x7f717010b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701007f0 0x7f717010b780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f716c031460 tx=0x7f716c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 shutdown_connections 2024-09-21T04:40:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f715c07af40 0x7f715c07d400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71701067f0 0x7f717010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 --2- 172.21.15.7:0/3053601072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71701007f0 0x7f717010b780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 >> 172.21.15.7:0/3053601072 conn(0x7f71700fc550 msgr2=0x7f71700fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 shutdown_connections 2024-09-21T04:40:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.439+0000 7f7177d1c640 1 -- 172.21.15.7:0/3053601072 wait complete. 2024-09-21T04:40:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:40:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.527+0000 7f74ea1c3640 1 -- 172.21.15.7:0/2943262807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4102b10 msgr2=0x7f74e4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.527+0000 7f74ea1c3640 1 --2- 172.21.15.7:0/2943262807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4102b10 0x7f74e4102ef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f74d40098b0 tx=0x7f74d402a560 comp rx=0 tx=0).stop 2024-09-21T04:40:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.527+0000 7f74ea1c3640 1 -- 172.21.15.7:0/2943262807 shutdown_connections 2024-09-21T04:40:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.527+0000 7f74ea1c3640 1 --2- 172.21.15.7:0/2943262807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e4103430 0x7f74e410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.527+0000 7f74ea1c3640 1 --2- 172.21.15.7:0/2943262807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4102b10 0x7f74e4102ef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.527+0000 7f74ea1c3640 1 -- 172.21.15.7:0/2943262807 >> 172.21.15.7:0/2943262807 conn(0x7f74e40fc550 msgr2=0x7f74e40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.527+0000 7f74ea1c3640 1 -- 172.21.15.7:0/2943262807 shutdown_connections 2024-09-21T04:40:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.527+0000 7f74ea1c3640 1 -- 172.21.15.7:0/2943262807 wait complete. 2024-09-21T04:40:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.528+0000 7f74ea1c3640 1 Processor -- start 2024-09-21T04:40:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.528+0000 7f74ea1c3640 1 -- start start 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74ea1c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e4102b10 0x7f74e419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74ea1c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4103430 0x7f74e419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74ea1c3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74e410ede0 con 0x7f74e4103430 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74ea1c3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74e410ec60 con 0x7f74e4102b10 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74e2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4103430 0x7f74e419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74e2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4103430 0x7f74e419dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50216/0 (socket says 172.21.15.7:50216) 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74e2ffd640 1 -- 172.21.15.7:0/593208759 learned_addr learned my addr 172.21.15.7:0/593208759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74e37fe640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e4102b10 0x7f74e419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74e2ffd640 1 -- 172.21.15.7:0/593208759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e4102b10 msgr2=0x7f74e419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74e2ffd640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e4102b10 0x7f74e419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74e2ffd640 1 -- 172.21.15.7:0/593208759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e4197870 con 0x7f74e4103430 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.529+0000 7f74e2ffd640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4103430 0x7f74e419dbf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e970 tx=0x7f74d800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.530+0000 7f74e0ff9640 1 -- 172.21.15.7:0/593208759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74d800cac0 con 0x7f74e4103430 2024-09-21T04:40:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.530+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e4197b60 con 0x7f74e4103430 2024-09-21T04:40:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.530+0000 7f74e0ff9640 1 -- 172.21.15.7:0/593208759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f74d80054e0 con 0x7f74e4103430 2024-09-21T04:40:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.530+0000 7f74e0ff9640 1 -- 172.21.15.7:0/593208759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74d8010430 con 0x7f74e4103430 2024-09-21T04:40:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.530+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e41980f0 con 0x7f74e4103430 2024-09-21T04:40:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.531+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e406a520 con 0x7f74e4103430 2024-09-21T04:40:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.532+0000 7f74e0ff9640 1 -- 172.21.15.7:0/593208759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f74d8004e80 con 0x7f74e4103430 2024-09-21T04:40:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.532+0000 7f74e0ff9640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74cc07af30 0x7f74cc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.533+0000 7f74e37fe640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74cc07af30 0x7f74cc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.533+0000 7f74e0ff9640 1 -- 172.21.15.7:0/593208759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6070+0+0 (secure 0 0 0) 0x7f74d809e640 con 0x7f74e4103430 2024-09-21T04:40:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.533+0000 7f74e37fe640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74cc07af30 0x7f74cc07d3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009230 tx=0x7f74d4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.537+0000 7f74e0ff9640 1 -- 172.21.15.7:0/593208759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74d8069c30 con 0x7f74e4103430 2024-09-21T04:40:17.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.755+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f74e4198da0 con 0x7f74cc07af30 2024-09-21T04:40:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.756+0000 7f74e0ff9640 1 -- 172.21.15.7:0/593208759 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f74e4198da0 con 0x7f74cc07af30 2024-09-21T04:40:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74cc07af30 msgr2=0x7f74cc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74cc07af30 0x7f74cc07d3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009230 tx=0x7f74d4031400 comp rx=0 tx=0).stop 2024-09-21T04:40:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4103430 msgr2=0x7f74e419dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4103430 0x7f74e419dbf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e970 tx=0x7f74d800ed70 comp rx=0 tx=0).stop 2024-09-21T04:40:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 shutdown_connections 2024-09-21T04:40:17.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74cc07af30 0x7f74cc07d3f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e4103430 0x7f74e419dbf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 --2- 172.21.15.7:0/593208759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e4102b10 0x7f74e419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 >> 172.21.15.7:0/593208759 conn(0x7f74e40fc550 msgr2=0x7f74e4107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:17.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 shutdown_connections 2024-09-21T04:40:17.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.759+0000 7f74ea1c3640 1 -- 172.21.15.7:0/593208759 wait complete. 2024-09-21T04:40:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.848+0000 7f31948a5640 1 -- 172.21.15.7:0/871745665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901067f0 msgr2=0x7f3190106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.848+0000 7f31948a5640 1 --2- 172.21.15.7:0/871745665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901067f0 0x7f3190106bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3180009880 tx=0x7f318002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.848+0000 7f31948a5640 1 -- 172.21.15.7:0/871745665 shutdown_connections 2024-09-21T04:40:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.848+0000 7f31948a5640 1 --2- 172.21.15.7:0/871745665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31901007f0 0x7f3190100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.848+0000 7f31948a5640 1 --2- 172.21.15.7:0/871745665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901067f0 0x7f3190106bd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.848+0000 7f31948a5640 1 -- 172.21.15.7:0/871745665 >> 172.21.15.7:0/871745665 conn(0x7f31900fc550 msgr2=0x7f31900fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.848+0000 7f31948a5640 1 -- 172.21.15.7:0/871745665 shutdown_connections 2024-09-21T04:40:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.848+0000 7f31948a5640 1 -- 172.21.15.7:0/871745665 wait complete. 2024-09-21T04:40:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.849+0000 7f31948a5640 1 Processor -- start 2024-09-21T04:40:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.849+0000 7f31948a5640 1 -- start start 2024-09-21T04:40:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f31948a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901007f0 0x7f319019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f31948a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31901067f0 0x7f319019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f31948a5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f319010dcc0 con 0x7f31901007f0 2024-09-21T04:40:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f31948a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f319010db40 con 0x7f31901067f0 2024-09-21T04:40:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f318e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901007f0 0x7f319019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f318e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901007f0 0x7f319019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50224/0 (socket says 172.21.15.7:50224) 2024-09-21T04:40:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f318e59c640 1 -- 172.21.15.7:0/1985305060 learned_addr learned my addr 172.21.15.7:0/1985305060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f318dd9b640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31901067f0 0x7f319019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f318e59c640 1 -- 172.21.15.7:0/1985305060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31901067f0 msgr2=0x7f319019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f318e59c640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31901067f0 0x7f319019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f318e59c640 1 -- 172.21.15.7:0/1985305060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31901977f0 con 0x7f31901007f0 2024-09-21T04:40:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.850+0000 7f318e59c640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901007f0 0x7f319019d670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3180031460 tx=0x7f31800045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.851+0000 7f317f7fe640 1 -- 172.21.15.7:0/1985305060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3180035070 con 0x7f31901007f0 2024-09-21T04:40:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.851+0000 7f317f7fe640 1 -- 172.21.15.7:0/1985305060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f318003b6e0 con 0x7f31901007f0 2024-09-21T04:40:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.851+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3190197a80 con 0x7f31901007f0 2024-09-21T04:40:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.851+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3190197fb0 con 0x7f31901007f0 2024-09-21T04:40:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.851+0000 7f317f7fe640 1 -- 172.21.15.7:0/1985305060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3180003b50 con 0x7f31901007f0 2024-09-21T04:40:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.852+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3190101f30 con 0x7f31901007f0 2024-09-21T04:40:17.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.853+0000 7f317f7fe640 1 -- 172.21.15.7:0/1985305060 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3180003cf0 con 0x7f31901007f0 2024-09-21T04:40:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.854+0000 7f317f7fe640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f317807af80 0x7f317807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.854+0000 7f317f7fe640 1 -- 172.21.15.7:0/1985305060 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6070+0+0 (secure 0 0 0) 0x7f31800c2800 con 0x7f31901007f0 2024-09-21T04:40:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.854+0000 7f318dd9b640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f317807af80 0x7f317807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:17.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.855+0000 7f318dd9b640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f317807af80 0x7f317807d440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3190105fc0 tx=0x7f318400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:17.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:17.857+0000 7f317f7fe640 1 -- 172.21.15.7:0/1985305060 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f318008df70 con 0x7f31901007f0 2024-09-21T04:40:18.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.060+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3190198260 con 0x7f317807af80 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.071+0000 7f317f7fe640 1 -- 172.21.15.7:0/1985305060 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f3190198260 con 0x7f317807af80 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (2m) 0s ago 13m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (5m) 0s ago 13m 8887k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (5m) 24s ago 12m 8744k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (32s) 0s ago 13m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (27s) 24s ago 12m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (115s) 0s ago 13m 77.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (7m) 0s ago 7m 145M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (7m) 0s ago 7m 89.5M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:40:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (7m) 24s ago 7m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (7m) 24s ago 7m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (4m) 0s ago 14m 526M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (3m) 24s ago 12m 455M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (59s) 0s ago 14m 38.7M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (44s) 24s ago 12m 25.9M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (3m) 0s ago 13m 10.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (3m) 24s ago 12m 10.7M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (5s) 0s ago 11m 12.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (10m) 0s ago 10m 214M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (10m) 0s ago 10m 193M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (9m) 24s ago 9m 243M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (9m) 24s ago 9m 212M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (8m) 24s ago 8m 184M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:40:18.076 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (2m) 0s ago 12m 51.6M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:40:18.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f317807af80 msgr2=0x7f317807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f317807af80 0x7f317807d440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3190105fc0 tx=0x7f318400b390 comp rx=0 tx=0).stop 2024-09-21T04:40:18.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901007f0 msgr2=0x7f319019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901007f0 0x7f319019d670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3180031460 tx=0x7f31800045e0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 shutdown_connections 2024-09-21T04:40:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f317807af80 0x7f317807d440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31901067f0 0x7f319019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 --2- 172.21.15.7:0/1985305060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31901007f0 0x7f319019d670 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 >> 172.21.15.7:0/1985305060 conn(0x7f31900fc550 msgr2=0x7f3190069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.073+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 shutdown_connections 2024-09-21T04:40:18.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.074+0000 7f31948a5640 1 -- 172.21.15.7:0/1985305060 wait complete. 2024-09-21T04:40:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.166+0000 7faceeb67640 1 -- 172.21.15.7:0/2941779128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 msgr2=0x7face8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.166+0000 7faceeb67640 1 --2- 172.21.15.7:0/2941779128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 0x7face8100c50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7facd4009880 tx=0x7facd402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.166+0000 7faceeb67640 1 -- 172.21.15.7:0/2941779128 shutdown_connections 2024-09-21T04:40:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.166+0000 7faceeb67640 1 --2- 172.21.15.7:0/2941779128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 0x7face8100c50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.166+0000 7faceeb67640 1 --2- 172.21.15.7:0/2941779128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face81067f0 0x7face8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.166+0000 7faceeb67640 1 -- 172.21.15.7:0/2941779128 >> 172.21.15.7:0/2941779128 conn(0x7face80fc550 msgr2=0x7face80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.167+0000 7faceeb67640 1 -- 172.21.15.7:0/2941779128 shutdown_connections 2024-09-21T04:40:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.167+0000 7faceeb67640 1 -- 172.21.15.7:0/2941779128 wait complete. 2024-09-21T04:40:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.167+0000 7faceeb67640 1 Processor -- start 2024-09-21T04:40:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.167+0000 7faceeb67640 1 -- start start 2024-09-21T04:40:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7faceeb67640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 0x7face80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7faceeb67640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face81067f0 0x7face80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7faceeb67640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7face810dca0 con 0x7face81007f0 2024-09-21T04:40:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7faceeb67640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7face810db20 con 0x7face81067f0 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7facec903640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 0x7face80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7facec903640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 0x7face80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50254/0 (socket says 172.21.15.7:50254) 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7facec903640 1 -- 172.21.15.7:0/3376376722 learned_addr learned my addr 172.21.15.7:0/3376376722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7face7fff640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face81067f0 0x7face80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7face7fff640 1 -- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 msgr2=0x7face80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7face7fff640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 0x7face80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.168+0000 7face7fff640 1 -- 172.21.15.7:0/3376376722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80fd700 con 0x7face81067f0 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.169+0000 7face7fff640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face81067f0 0x7face80fd1c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7facd4006010 tx=0x7facd40045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.169+0000 7face5ffb640 1 -- 172.21.15.7:0/3376376722 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7facd403f070 con 0x7face81067f0 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.169+0000 7face5ffb640 1 -- 172.21.15.7:0/3376376722 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7facd40035f0 con 0x7face81067f0 2024-09-21T04:40:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.169+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face80fd990 con 0x7face81067f0 2024-09-21T04:40:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.169+0000 7face5ffb640 1 -- 172.21.15.7:0/3376376722 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7facd4035030 con 0x7face81067f0 2024-09-21T04:40:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.169+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face81a3d40 con 0x7face81067f0 2024-09-21T04:40:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.170+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face8101f30 con 0x7face81067f0 2024-09-21T04:40:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.171+0000 7face5ffb640 1 -- 172.21.15.7:0/3376376722 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7facd4042070 con 0x7face81067f0 2024-09-21T04:40:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.172+0000 7face5ffb640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7facd807af80 0x7facd807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.172+0000 7face5ffb640 1 -- 172.21.15.7:0/3376376722 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6070+0+0 (secure 0 0 0) 0x7facd40be890 con 0x7face81067f0 2024-09-21T04:40:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.172+0000 7facec903640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7facd807af80 0x7facd807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.173+0000 7facec903640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7facd807af80 0x7facd807d440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facdc009c20 tx=0x7facdc005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.177+0000 7face5ffb640 1 -- 172.21.15.7:0/3376376722 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7facd4089ec0 con 0x7face81067f0 2024-09-21T04:40:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.505+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7face802d7c0 con 0x7face81067f0 2024-09-21T04:40:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.506+0000 7face5ffb640 1 -- 172.21.15.7:0/3376376722 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+732 (secure 0 0 0) 0x7facd408e030 con 0x7face81067f0 2024-09-21T04:40:18.510 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:40:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:40:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:40:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 5 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 9, 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:40:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:40:18.512 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:40:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7facd807af80 msgr2=0x7facd807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7facd807af80 0x7facd807d440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facdc009c20 tx=0x7facdc005f90 comp rx=0 tx=0).stop 2024-09-21T04:40:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face81067f0 msgr2=0x7face80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face81067f0 0x7face80fd1c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7facd4006010 tx=0x7facd40045e0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 shutdown_connections 2024-09-21T04:40:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7facd807af80 0x7facd807d440 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face81067f0 0x7face80fd1c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 --2- 172.21.15.7:0/3376376722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face81007f0 0x7face80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 >> 172.21.15.7:0/3376376722 conn(0x7face80fc550 msgr2=0x7face8069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 shutdown_connections 2024-09-21T04:40:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.509+0000 7faceeb67640 1 -- 172.21.15.7:0/3376376722 wait complete. 2024-09-21T04:40:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.607+0000 7f6373408640 1 -- 172.21.15.7:0/3083427438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1067f0 msgr2=0x7f636c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.607+0000 7f6373408640 1 --2- 172.21.15.7:0/3083427438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1067f0 0x7f636c106bd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f635c009880 tx=0x7f635c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.607+0000 7f6373408640 1 -- 172.21.15.7:0/3083427438 shutdown_connections 2024-09-21T04:40:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.607+0000 7f6373408640 1 --2- 172.21.15.7:0/3083427438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f636c1007f0 0x7f636c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.607+0000 7f6373408640 1 --2- 172.21.15.7:0/3083427438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1067f0 0x7f636c106bd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.607+0000 7f6373408640 1 -- 172.21.15.7:0/3083427438 >> 172.21.15.7:0/3083427438 conn(0x7f636c0fc550 msgr2=0x7f636c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.607+0000 7f6373408640 1 -- 172.21.15.7:0/3083427438 shutdown_connections 2024-09-21T04:40:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.607+0000 7f6373408640 1 -- 172.21.15.7:0/3083427438 wait complete. 2024-09-21T04:40:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f6373408640 1 Processor -- start 2024-09-21T04:40:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f6373408640 1 -- start start 2024-09-21T04:40:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f6373408640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1007f0 0x7f636c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f6373408640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f636c1067f0 0x7f636c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f6373408640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f636c10dca0 con 0x7f636c1007f0 2024-09-21T04:40:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f6373408640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f636c10db20 con 0x7f636c1067f0 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f63711a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1007f0 0x7f636c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f63711a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1007f0 0x7f636c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50278/0 (socket says 172.21.15.7:50278) 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f63711a4640 1 -- 172.21.15.7:0/2574709668 learned_addr learned my addr 172.21.15.7:0/2574709668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.608+0000 7f63709a3640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f636c1067f0 0x7f636c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f63711a4640 1 -- 172.21.15.7:0/2574709668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f636c1067f0 msgr2=0x7f636c198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f63711a4640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f636c1067f0 0x7f636c198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f63711a4640 1 -- 172.21.15.7:0/2574709668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c199410 con 0x7f636c1007f0 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f63711a4640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1007f0 0x7f636c1988d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f635c031460 tx=0x7f635c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f63627fc640 1 -- 172.21.15.7:0/2574709668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f635c03f070 con 0x7f636c1007f0 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c1996a0 con 0x7f636c1007f0 2024-09-21T04:40:18.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f63627fc640 1 -- 172.21.15.7:0/2574709668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f635c0035f0 con 0x7f636c1007f0 2024-09-21T04:40:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f63627fc640 1 -- 172.21.15.7:0/2574709668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f635c035030 con 0x7f636c1007f0 2024-09-21T04:40:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.609+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c1a1c50 con 0x7f636c1007f0 2024-09-21T04:40:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.610+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c101f30 con 0x7f636c1007f0 2024-09-21T04:40:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.611+0000 7f63627fc640 1 -- 172.21.15.7:0/2574709668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f635c042070 con 0x7f636c1007f0 2024-09-21T04:40:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.611+0000 7f63627fc640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f635807af80 0x7f635807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.611+0000 7f63627fc640 1 -- 172.21.15.7:0/2574709668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6070+0+0 (secure 0 0 0) 0x7f635c0be7c0 con 0x7f636c1007f0 2024-09-21T04:40:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.612+0000 7f63709a3640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f635807af80 0x7f635807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.612+0000 7f63709a3640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f635807af80 0x7f635807d440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f636c06a7f0 tx=0x7f636800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.616+0000 7f63627fc640 1 -- 172.21.15.7:0/2574709668 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f635c089e90 con 0x7f636c1007f0 2024-09-21T04:40:18.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.850+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f636c06a290 con 0x7f636c1007f0 2024-09-21T04:40:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.851+0000 7f63627fc640 1 -- 172.21.15.7:0/2574709668 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2038 (secure 0 0 0) 0x7f635c08ed40 con 0x7f636c1007f0 2024-09-21T04:40:18.855 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-09-21T04:40:18.855 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:40:18.855 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:40:18.855 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:40:18.855 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:40:18.856 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 44 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14578,1=24327} 2024-09-21T04:40:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:40:18.858 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:40:18.859 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:40:18.859 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:40:18.859 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:40:18.859 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:40:18.859 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:40:18.859 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f635807af80 msgr2=0x7f635807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f635807af80 0x7f635807d440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f636c06a7f0 tx=0x7f636800b390 comp rx=0 tx=0).stop 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1007f0 msgr2=0x7f636c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1007f0 0x7f636c1988d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f635c031460 tx=0x7f635c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 shutdown_connections 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f635807af80 0x7f635807d440 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f636c1067f0 0x7f636c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 --2- 172.21.15.7:0/2574709668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f636c1007f0 0x7f636c1988d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 >> 172.21.15.7:0/2574709668 conn(0x7f636c0fc550 msgr2=0x7f636c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 shutdown_connections 2024-09-21T04:40:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.853+0000 7f6373408640 1 -- 172.21.15.7:0/2574709668 wait complete. 2024-09-21T04:40:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 -- 172.21.15.7:0/3438167019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0101900 msgr2=0x7f61c0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61bf7fe640 1 -- 172.21.15.7:0/3438167019 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f61b4035020 con 0x7f61c0101900 2024-09-21T04:40:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 --2- 172.21.15.7:0/3438167019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0101900 0x7f61c0101d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009880 tx=0x7f61b402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 -- 172.21.15.7:0/3438167019 shutdown_connections 2024-09-21T04:40:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 --2- 172.21.15.7:0/3438167019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0101900 0x7f61c0101d60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 --2- 172.21.15.7:0/3438167019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c0107900 0x7f61c0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 -- 172.21.15.7:0/3438167019 >> 172.21.15.7:0/3438167019 conn(0x7f61c00fd600 msgr2=0x7f61c00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 -- 172.21.15.7:0/3438167019 shutdown_connections 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 -- 172.21.15.7:0/3438167019 wait complete. 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 Processor -- start 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.950+0000 7f61c6ca6640 1 -- start start 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61c6ca6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c0101900 0x7f61c019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61c6ca6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0107900 0x7f61c019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61c6ca6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61c010edb0 con 0x7f61c0107900 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61c6ca6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61c010ec30 con 0x7f61c0101900 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0107900 0x7f61c019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0107900 0x7f61c019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50296/0 (socket says 172.21.15.7:50296) 2024-09-21T04:40:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61bffff640 1 -- 172.21.15.7:0/1804258604 learned_addr learned my addr 172.21.15.7:0/1804258604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61c4a42640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c0101900 0x7f61c019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61c4a42640 1 -- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0107900 msgr2=0x7f61c019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61c4a42640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0107900 0x7f61c019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.951+0000 7f61c4a42640 1 -- 172.21.15.7:0/1804258604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01a0220 con 0x7f61c0101900 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.952+0000 7f61c4a42640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c0101900 0x7f61c019a4d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61b000d700 tx=0x7f61b000dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.952+0000 7f61bdffb640 1 -- 172.21.15.7:0/1804258604 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f61b0017070 con 0x7f61c0101900 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.952+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c01a0510 con 0x7f61c0101900 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.952+0000 7f61bdffb640 1 -- 172.21.15.7:0/1804258604 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61b00054e0 con 0x7f61c0101900 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.952+0000 7f61bdffb640 1 -- 172.21.15.7:0/1804258604 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f61b0009a60 con 0x7f61c0101900 2024-09-21T04:40:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.952+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01a0aa0 con 0x7f61c0101900 2024-09-21T04:40:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.953+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c0103040 con 0x7f61c0101900 2024-09-21T04:40:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.954+0000 7f61bdffb640 1 -- 172.21.15.7:0/1804258604 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f61b0009ce0 con 0x7f61c0101900 2024-09-21T04:40:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.954+0000 7f61bdffb640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61a807af90 0x7f61a807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.954+0000 7f61bdffb640 1 -- 172.21.15.7:0/1804258604 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6070+0+0 (secure 0 0 0) 0x7f61b009dab0 con 0x7f61c0101900 2024-09-21T04:40:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.954+0000 7f61bffff640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61a807af90 0x7f61a807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.955+0000 7f61bffff640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61a807af90 0x7f61a807d450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61b40096c0 tx=0x7f61b402a9a0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:18.959+0000 7f61bdffb640 1 -- 172.21.15.7:0/1804258604 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61b0069100 con 0x7f61c0101900 2024-09-21T04:40:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.172+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f61c0074b30 con 0x7f61a807af90 2024-09-21T04:40:19.176 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:19.176 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:19.176 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: pgmap v135: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:19.177 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:19.177 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:19.177 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3376376722' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:19.177 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:19.177 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:19.177 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:18 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2574709668' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:40:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.173+0000 7f61bdffb640 1 -- 172.21.15.7:0/1804258604 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f61c0074b30 con 0x7f61a807af90 2024-09-21T04:40:19.177 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "7/23 daemons upgraded", 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:40:19.178 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:40:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.175+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61a807af90 msgr2=0x7f61a807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.175+0000 7f61c6ca6640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61a807af90 0x7f61a807d450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61b40096c0 tx=0x7f61b402a9a0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.175+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c0101900 msgr2=0x7f61c019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.175+0000 7f61c6ca6640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c0101900 0x7f61c019a4d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61b000d700 tx=0x7f61b000dbc0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.176+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 shutdown_connections 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.176+0000 7f61c6ca6640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61a807af90 0x7f61a807d450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.176+0000 7f61c6ca6640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61c0107900 0x7f61c019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.176+0000 7f61c6ca6640 1 --2- 172.21.15.7:0/1804258604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c0101900 0x7f61c019a4d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.176+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 >> 172.21.15.7:0/1804258604 conn(0x7f61c00fd600 msgr2=0x7f61c0074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.176+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 shutdown_connections 2024-09-21T04:40:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.176+0000 7f61c6ca6640 1 -- 172.21.15.7:0/1804258604 wait complete. 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: pgmap v135: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3376376722' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:19.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:18 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2574709668' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.265+0000 7f7c34f2b640 1 -- 172.21.15.7:0/857758358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100250 msgr2=0x7f7c30100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.265+0000 7f7c34f2b640 1 --2- 172.21.15.7:0/857758358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100250 0x7f7c30100630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000a990 tx=0x7f7c2002b7d0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.265+0000 7f7c34f2b640 1 -- 172.21.15.7:0/857758358 shutdown_connections 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.265+0000 7f7c34f2b640 1 --2- 172.21.15.7:0/857758358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c30100c00 0x7f7c30104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.265+0000 7f7c34f2b640 1 --2- 172.21.15.7:0/857758358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100250 0x7f7c30100630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.265+0000 7f7c34f2b640 1 -- 172.21.15.7:0/857758358 >> 172.21.15.7:0/857758358 conn(0x7f7c300fbc90 msgr2=0x7f7c300fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.265+0000 7f7c34f2b640 1 -- 172.21.15.7:0/857758358 shutdown_connections 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.265+0000 7f7c34f2b640 1 -- 172.21.15.7:0/857758358 wait complete. 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c34f2b640 1 Processor -- start 2024-09-21T04:40:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c34f2b640 1 -- start start 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c34f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c30100250 0x7f7c3019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c34f2b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100c00 0x7f7c3019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c34f2b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3010cbd0 con 0x7f7c30100c00 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c34f2b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3010ca50 con 0x7f7c30100250 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c2dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100c00 0x7f7c3019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c2dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100c00 0x7f7c3019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50300/0 (socket says 172.21.15.7:50300) 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.266+0000 7f7c2dd9b640 1 -- 172.21.15.7:0/1897512579 learned_addr learned my addr 172.21.15.7:0/1897512579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c2e59c640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c30100250 0x7f7c3019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c2dd9b640 1 -- 172.21.15.7:0/1897512579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c30100250 msgr2=0x7f7c3019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c2dd9b640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c30100250 0x7f7c3019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c2dd9b640 1 -- 172.21.15.7:0/1897512579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c301977f0 con 0x7f7c30100c00 2024-09-21T04:40:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c2e59c640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c30100250 0x7f7c3019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:40:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c2dd9b640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100c00 0x7f7c3019dbb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800c8f0 tx=0x7f7c1800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c277fe640 1 -- 172.21.15.7:0/1897512579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7c180079b0 con 0x7f7c30100c00 2024-09-21T04:40:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c277fe640 1 -- 172.21.15.7:0/1897512579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c180054e0 con 0x7f7c30100c00 2024-09-21T04:40:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c30197ae0 con 0x7f7c30100c00 2024-09-21T04:40:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c277fe640 1 -- 172.21.15.7:0/1897512579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7c18004dd0 con 0x7f7c30100c00 2024-09-21T04:40:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.267+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c30198070 con 0x7f7c30100c00 2024-09-21T04:40:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.268+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c30101730 con 0x7f7c30100c00 2024-09-21T04:40:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.269+0000 7f7c277fe640 1 -- 172.21.15.7:0/1897512579 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7c18005020 con 0x7f7c30100c00 2024-09-21T04:40:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.269+0000 7f7c277fe640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c1c07af30 0x7f7c1c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.270+0000 7f7c277fe640 1 -- 172.21.15.7:0/1897512579 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6070+0+0 (secure 0 0 0) 0x7f7c1809de50 con 0x7f7c30100c00 2024-09-21T04:40:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.270+0000 7f7c2e59c640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c1c07af30 0x7f7c1c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.270+0000 7f7c2e59c640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c1c07af30 0x7f7c1c07d3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c20000c00 tx=0x7f7c20008750 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.274+0000 7f7c277fe640 1 -- 172.21.15.7:0/1897512579 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c18014030 con 0x7f7c30100c00 2024-09-21T04:40:19.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.569+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f7c30108560 con 0x7f7c30100c00 2024-09-21T04:40:19.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.569+0000 7f7c277fe640 1 -- 172.21.15.7:0/1897512579 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+1573 (secure 0 0 0) 0x7f7c18069520 con 0x7f7c30100c00 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 65/333 objects degraded (19.520%), 20 pgs degraded 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: osd.0 (root=default,host=smithi007) is down 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 65/333 objects degraded (19.520%), 20 pgs degraded 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-09-21T04:40:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-09-21T04:40:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-09-21T04:40:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c1c07af30 msgr2=0x7f7c1c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c1c07af30 0x7f7c1c07d3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c20000c00 tx=0x7f7c20008750 comp rx=0 tx=0).stop 2024-09-21T04:40:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100c00 msgr2=0x7f7c3019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100c00 0x7f7c3019dbb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800c8f0 tx=0x7f7c1800cdb0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 shutdown_connections 2024-09-21T04:40:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c1c07af30 0x7f7c1c07d3f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c30100c00 0x7f7c3019dbb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 --2- 172.21.15.7:0/1897512579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c30100250 0x7f7c3019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 >> 172.21.15.7:0/1897512579 conn(0x7f7c300fbc90 msgr2=0x7f7c300fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 shutdown_connections 2024-09-21T04:40:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:19.572+0000 7f7c34f2b640 1 -- 172.21.15.7:0/1897512579 wait complete. 2024-09-21T04:40:20.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:19 smithi053 ceph-mon[80176]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:20.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:19 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1897512579' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:40:20.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:19 smithi053 ceph-mon[80176]: from='osd.0 [v2:172.21.15.7:6802/3855647303,v1:172.21.15.7:6803/3855647303]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T04:40:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:19 smithi007 ceph-mon[108238]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:19 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1897512579' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:40:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:19 smithi007 ceph-mon[108238]: from='osd.0 [v2:172.21.15.7:6802/3855647303,v1:172.21.15.7:6803/3855647303]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T04:40:20.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:19 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[116742]: 2024-09-21T04:40:19.907+0000 7f7d4ba15640 -1 osd.0 46 log_to_monitors true 2024-09-21T04:40:20.194 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 04:40:19 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[116742]: 2024-09-21T04:40:19.956+0000 7f7d437bf640 -1 osd.0 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:40:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:20 smithi007 ceph-mon[108238]: from='client.44141 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:20 smithi007 ceph-mon[108238]: pgmap v136: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:21.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:20 smithi007 ceph-mon[108238]: from='osd.0 [v2:172.21.15.7:6802/3855647303,v1:172.21.15.7:6803/3855647303]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T04:40:21.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:20 smithi007 ceph-mon[108238]: osdmap e49: 6 total, 5 up, 6 in 2024-09-21T04:40:21.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:20 smithi007 ceph-mon[108238]: from='osd.0 [v2:172.21.15.7:6802/3855647303,v1:172.21.15.7:6803/3855647303]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]} : dispatch 2024-09-21T04:40:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:20 smithi053 ceph-mon[80176]: from='client.44141 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:20 smithi053 ceph-mon[80176]: pgmap v136: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:20 smithi053 ceph-mon[80176]: from='osd.0 [v2:172.21.15.7:6802/3855647303,v1:172.21.15.7:6803/3855647303]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T04:40:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:20 smithi053 ceph-mon[80176]: osdmap e49: 6 total, 5 up, 6 in 2024-09-21T04:40:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:20 smithi053 ceph-mon[80176]: from='osd.0 [v2:172.21.15.7:6802/3855647303,v1:172.21.15.7:6803/3855647303]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]} : dispatch 2024-09-21T04:40:22.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:21 smithi053 ceph-mon[80176]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:40:22.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:21 smithi053 ceph-mon[80176]: osd.0 [v2:172.21.15.7:6802/3855647303,v1:172.21.15.7:6803/3855647303] boot 2024-09-21T04:40:22.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:21 smithi053 ceph-mon[80176]: osdmap e50: 6 total, 6 up, 6 in 2024-09-21T04:40:22.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:21 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T04:40:22.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:21 smithi053 ceph-mon[80176]: osdmap e51: 6 total, 6 up, 6 in 2024-09-21T04:40:22.231 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:21 smithi007 ceph-mon[108238]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:40:22.231 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:21 smithi007 ceph-mon[108238]: osd.0 [v2:172.21.15.7:6802/3855647303,v1:172.21.15.7:6803/3855647303] boot 2024-09-21T04:40:22.231 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:21 smithi007 ceph-mon[108238]: osdmap e50: 6 total, 6 up, 6 in 2024-09-21T04:40:22.231 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:21 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T04:40:22.231 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:21 smithi007 ceph-mon[108238]: osdmap e51: 6 total, 6 up, 6 in 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: pgmap v139: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:23.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:23.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-21T04:40:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: pgmap v139: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 65/333 objects degraded (19.520%) 2024-09-21T04:40:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:40:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:40:23.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:23.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:40:23.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:23.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:23.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:23.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-21T04:40:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:23 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-21T04:40:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:23 smithi053 ceph-mon[80176]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-21T04:40:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:23 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-21T04:40:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:23 smithi007 ceph-mon[108238]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-21T04:40:25.240 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:24 smithi007 ceph-mon[108238]: pgmap v141: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 56 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 6 op/s; 9/333 objects degraded (2.703%) 2024-09-21T04:40:25.240 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:24 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 9/333 objects degraded (2.703%), 3 pgs degraded (PG_DEGRADED) 2024-09-21T04:40:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:24 smithi053 ceph-mon[80176]: pgmap v141: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 56 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 6 op/s; 9/333 objects degraded (2.703%) 2024-09-21T04:40:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:24 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 9/333 objects degraded (2.703%), 3 pgs degraded (PG_DEGRADED) 2024-09-21T04:40:26.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:26 smithi053 ceph-mon[80176]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/333 objects degraded (2.703%), 3 pgs degraded) 2024-09-21T04:40:26.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:26 smithi007 ceph-mon[108238]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/333 objects degraded (2.703%), 3 pgs degraded) 2024-09-21T04:40:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:27 smithi053 ceph-mon[80176]: pgmap v142: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-21T04:40:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:27 smithi007 ceph-mon[108238]: pgmap v142: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-21T04:40:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:29 smithi007 ceph-mon[108238]: pgmap v143: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-21T04:40:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:29 smithi053 ceph-mon[80176]: pgmap v143: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-21T04:40:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:31 smithi053 ceph-mon[80176]: pgmap v144: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-21T04:40:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:31 smithi007 ceph-mon[108238]: pgmap v144: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-21T04:40:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:33 smithi053 ceph-mon[80176]: pgmap v145: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:40:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:33 smithi007 ceph-mon[108238]: pgmap v145: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:40:34.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:34 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:34.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:34 smithi007 ceph-mon[108238]: pgmap v146: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:40:34.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:34 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:34.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:34 smithi053 ceph-mon[80176]: pgmap v146: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:40:37.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:36 smithi053 ceph-mon[80176]: pgmap v147: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:40:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:36 smithi007 ceph-mon[108238]: pgmap v147: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:40:38.025 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:37 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-21T04:40:38.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:37 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-21T04:40:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:38 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-21T04:40:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:38 smithi053 ceph-mon[80176]: Upgrade: osd.1 is safe to restart 2024-09-21T04:40:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:38 smithi053 ceph-mon[80176]: pgmap v148: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:40:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:38 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-21T04:40:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:38 smithi007 ceph-mon[108238]: Upgrade: osd.1 is safe to restart 2024-09-21T04:40:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:38 smithi007 ceph-mon[108238]: pgmap v148: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:40:40.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:39 smithi053 ceph-mon[80176]: Upgrade: Updating osd.1 2024-09-21T04:40:40.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:39 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:40.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:39 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T04:40:40.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:39 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:40:40.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:39 smithi053 ceph-mon[80176]: Deploying daemon osd.1 on smithi007 2024-09-21T04:40:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:39 smithi007 ceph-mon[108238]: Upgrade: Updating osd.1 2024-09-21T04:40:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:39 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:39 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T04:40:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:39 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:40:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:39 smithi007 ceph-mon[108238]: Deploying daemon osd.1 on smithi007 2024-09-21T04:40:41.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:40 smithi053 ceph-mon[80176]: pgmap v149: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:40:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:40 smithi007 ceph-mon[108238]: pgmap v149: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:40:42.318 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:42 smithi007 systemd[1]: Stopping Ceph osd.1 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:40:42.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:42 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[56123]: 2024-09-21T04:40:42.315+0000 7f555e710640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:40:42.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:42 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[56123]: 2024-09-21T04:40:42.315+0000 7f555e710640 -1 osd.1 51 *** Got signal Terminated *** 2024-09-21T04:40:42.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:42 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[56123]: 2024-09-21T04:40:42.315+0000 7f555e710640 -1 osd.1 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:40:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:42 smithi053 ceph-mon[80176]: pgmap v150: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:40:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:42 smithi053 ceph-mon[80176]: osd.1 marked itself down and dead 2024-09-21T04:40:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:42 smithi007 ceph-mon[108238]: pgmap v150: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:40:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:42 smithi007 ceph-mon[108238]: osd.1 marked itself down and dead 2024-09-21T04:40:43.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:43 smithi007 podman[121888]: 2024-09-21 04:40:43.410604242 +0000 UTC m=+1.211516656 container died 4d8bf68bf697f50ad59ef4033a247439056ff93038babe8fdcc9338dfaff4a8a (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-4e45e6b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:40:44.061 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:43 smithi007 podman[121888]: 2024-09-21 04:40:43.790597244 +0000 UTC m=+1.591509655 container cleanup 4d8bf68bf697f50ad59ef4033a247439056ff93038babe8fdcc9338dfaff4a8a (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-4e45e6b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T04:40:44.061 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:43 smithi007 bash[121888]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1 2024-09-21T04:40:44.061 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:43 smithi007 ceph-mon[108238]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:40:44.062 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:43 smithi007 ceph-mon[108238]: osdmap e52: 6 total, 5 up, 6 in 2024-09-21T04:40:44.062 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:44.062 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:44.062 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:43 smithi007 ceph-mon[108238]: osdmap e53: 6 total, 5 up, 6 in 2024-09-21T04:40:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:43 smithi053 ceph-mon[80176]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:40:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:43 smithi053 ceph-mon[80176]: osdmap e52: 6 total, 5 up, 6 in 2024-09-21T04:40:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:43 smithi053 ceph-mon[80176]: osdmap e53: 6 total, 5 up, 6 in 2024-09-21T04:40:44.387 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:44 smithi007 podman[121905]: 2024-09-21 04:40:44.059043726 +0000 UTC m=+0.649504744 container remove 4d8bf68bf697f50ad59ef4033a247439056ff93038babe8fdcc9338dfaff4a8a (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-4e45e6b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:40:44.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:44 smithi007 podman[121994]: 2024-09-21 04:40:44.384981368 +0000 UTC m=+0.109272479 container create 219784fff4e1a26d0a890f93824dd955e9b8a7cd16be4e87bd9d375813be1fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T04:40:44.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:44 smithi007 podman[121994]: 2024-09-21 04:40:44.292824675 +0000 UTC m=+0.017115797 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:40:44.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:44 smithi007 podman[121994]: 2024-09-21 04:40:44.621233492 +0000 UTC m=+0.345524612 container init 219784fff4e1a26d0a890f93824dd955e9b8a7cd16be4e87bd9d375813be1fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux ) 2024-09-21T04:40:44.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:44 smithi007 podman[121994]: 2024-09-21 04:40:44.625059484 +0000 UTC m=+0.349350605 container start 219784fff4e1a26d0a890f93824dd955e9b8a7cd16be4e87bd9d375813be1fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T04:40:45.186 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:44 smithi007 ceph-mon[108238]: pgmap v152: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-09-21T04:40:45.186 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:44 smithi007 ceph-mon[108238]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-09-21T04:40:45.186 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:44 smithi007 podman[121994]: 2024-09-21 04:40:44.712592937 +0000 UTC m=+0.436884068 container attach 219784fff4e1a26d0a890f93824dd955e9b8a7cd16be4e87bd9d375813be1fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:40:45.186 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:44 smithi007 podman[121994]: 2024-09-21 04:40:44.825938965 +0000 UTC m=+0.550230076 container died 219784fff4e1a26d0a890f93824dd955e9b8a7cd16be4e87bd9d375813be1fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T04:40:45.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:44 smithi053 ceph-mon[80176]: pgmap v152: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-09-21T04:40:45.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:44 smithi053 ceph-mon[80176]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-09-21T04:40:45.761 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:45 smithi007 podman[122017]: 2024-09-21 04:40:45.500274751 +0000 UTC m=+0.667945389 container remove 219784fff4e1a26d0a890f93824dd955e9b8a7cd16be4e87bd9d375813be1fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:40:45.761 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:45 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.1.service: Deactivated successfully. 2024-09-21T04:40:45.761 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:45 smithi007 systemd[1]: Stopped Ceph osd.1 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:40:45.761 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:45 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.1.service: Consumed 1min 1.009s CPU time. 2024-09-21T04:40:46.102 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:45 smithi007 ceph-mon[108238]: Health check failed: Degraded data redundancy: 4/333 objects degraded (1.201%), 2 pgs degraded (PG_DEGRADED) 2024-09-21T04:40:46.102 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:45 smithi007 systemd[1]: Starting Ceph osd.1 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:40:46.102 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:45 smithi007 podman[122115]: 2024-09-21 04:40:45.87622332 +0000 UTC m=+0.016758849 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:40:46.102 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:45 smithi007 podman[122115]: 2024-09-21 04:40:45.976625655 +0000 UTC m=+0.117161158 container create c46a7504ff41c13f0daeee5c7b79a7d55c7a4d4273ed04b679c5824971c0a25c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:40:46.361 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:46 smithi007 podman[122115]: 2024-09-21 04:40:46.22106161 +0000 UTC m=+0.361597133 container init c46a7504ff41c13f0daeee5c7b79a7d55c7a4d4273ed04b679c5824971c0a25c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T04:40:46.361 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:46 smithi007 podman[122115]: 2024-09-21 04:40:46.224714349 +0000 UTC m=+0.365249858 container start c46a7504ff41c13f0daeee5c7b79a7d55c7a4d4273ed04b679c5824971c0a25c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T04:40:46.361 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:46 smithi007 podman[122115]: 2024-09-21 04:40:46.304407833 +0000 UTC m=+0.444943361 container attach c46a7504ff41c13f0daeee5c7b79a7d55c7a4d4273ed04b679c5824971c0a25c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T04:40:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:45 smithi053 ceph-mon[80176]: Health check failed: Degraded data redundancy: 4/333 objects degraded (1.201%), 2 pgs degraded (PG_DEGRADED) 2024-09-21T04:40:46.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:46 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:46.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:46 smithi007 bash[122115]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:46.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:46 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:46.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:46 smithi007 bash[122115]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:46 smithi007 ceph-mon[108238]: pgmap v154: 65 pgs: 3 active+undersized, 29 peering, 2 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 4/333 objects degraded (1.201%) 2024-09-21T04:40:47.390 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:46 smithi053 ceph-mon[80176]: pgmap v154: 65 pgs: 3 active+undersized, 29 peering, 2 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 4/333 objects degraded (1.201%) 2024-09-21T04:40:49.311 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:48 smithi007 ceph-mon[108238]: pgmap v155: 65 pgs: 3 active+undersized, 29 peering, 2 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 4/333 objects degraded (1.201%) 2024-09-21T04:40:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:48 smithi053 ceph-mon[80176]: pgmap v155: 65 pgs: 3 active+undersized, 29 peering, 2 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 4/333 objects degraded (1.201%) 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2024-09-21T04:40:49.666 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.662+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/3515191131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc1033e0 msgr2=0x7fc3bc10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.662+0000 7fc3c0bdd640 1 --2- 172.21.15.7:0/3515191131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc1033e0 0x7fc3bc10b8d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009c20 tx=0x7fc3b002aa00 comp rx=0 tx=0).stop 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.662+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/3515191131 shutdown_connections 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.662+0000 7fc3c0bdd640 1 --2- 172.21.15.7:0/3515191131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc1033e0 0x7fc3bc10b8d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.662+0000 7fc3c0bdd640 1 --2- 172.21.15.7:0/3515191131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc102ac0 0x7fc3bc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.662+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/3515191131 >> 172.21.15.7:0/3515191131 conn(0x7fc3bc0fc520 msgr2=0x7fc3bc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.662+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/3515191131 shutdown_connections 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.662+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/3515191131 wait complete. 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3c0bdd640 1 Processor -- start 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3c0bdd640 1 -- start start 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3c0bdd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc102ac0 0x7fc3bc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3c0bdd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1033e0 0x7fc3bc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3c0bdd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc10edf0 con 0x7fc3bc102ac0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3c0bdd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc10ec70 con 0x7fc3bc1033e0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3ba59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc102ac0 0x7fc3bc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3ba59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc102ac0 0x7fc3bc19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40006/0 (socket says 172.21.15.7:40006) 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3ba59c640 1 -- 172.21.15.7:0/153621698 learned_addr learned my addr 172.21.15.7:0/153621698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3b9d9b640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1033e0 0x7fc3bc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3ba59c640 1 -- 172.21.15.7:0/153621698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1033e0 msgr2=0x7fc3bc19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3ba59c640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1033e0 0x7fc3bc19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.663+0000 7fc3ba59c640 1 -- 172.21.15.7:0/153621698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc197820 con 0x7fc3bc102ac0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.664+0000 7fc3ba59c640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc102ac0 0x7fc3bc19d6a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00d6f0 tx=0x7fc3ac00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.664+0000 7fc3ab7fe640 1 -- 172.21.15.7:0/153621698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3ac017070 con 0x7fc3bc102ac0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.664+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3bc197b10 con 0x7fc3bc102ac0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.664+0000 7fc3ab7fe640 1 -- 172.21.15.7:0/153621698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3ac00de00 con 0x7fc3bc102ac0 2024-09-21T04:40:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.664+0000 7fc3ab7fe640 1 -- 172.21.15.7:0/153621698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3ac004bb0 con 0x7fc3bc102ac0 2024-09-21T04:40:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.664+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3bc1980a0 con 0x7fc3bc102ac0 2024-09-21T04:40:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.665+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3bc06aa90 con 0x7fc3bc102ac0 2024-09-21T04:40:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.666+0000 7fc3ab7fe640 1 -- 172.21.15.7:0/153621698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc3ac004e60 con 0x7fc3bc102ac0 2024-09-21T04:40:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.666+0000 7fc3ab7fe640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3a407af40 0x7fc3a407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.666+0000 7fc3ab7fe640 1 -- 172.21.15.7:0/153621698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6070+0+0 (secure 0 0 0) 0x7fc3ac09dbf0 con 0x7fc3bc102ac0 2024-09-21T04:40:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.667+0000 7fc3b9d9b640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3a407af40 0x7fc3a407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.667+0000 7fc3b9d9b640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3a407af40 0x7fc3a407d400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc1088d0 tx=0x7fc3b003a040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:49.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.671+0000 7fc3ab7fe640 1 -- 172.21.15.7:0/153621698 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3ac0691a0 con 0x7fc3bc102ac0 2024-09-21T04:40:49.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.881+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc3bc198410 con 0x7fc3a407af40 2024-09-21T04:40:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.883+0000 7fc3ab7fe640 1 -- 172.21.15.7:0/153621698 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fc3bc198410 con 0x7fc3a407af40 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3a407af40 msgr2=0x7fc3a407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3a407af40 0x7fc3a407d400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc1088d0 tx=0x7fc3b003a040 comp rx=0 tx=0).stop 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc102ac0 msgr2=0x7fc3bc19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc102ac0 0x7fc3bc19d6a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00d6f0 tx=0x7fc3ac00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 shutdown_connections 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3a407af40 0x7fc3a407d400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1033e0 0x7fc3bc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 --2- 172.21.15.7:0/153621698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3bc102ac0 0x7fc3bc19d6a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 >> 172.21.15.7:0/153621698 conn(0x7fc3bc0fc520 msgr2=0x7fc3bc108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 shutdown_connections 2024-09-21T04:40:49.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.885+0000 7fc3c0bdd640 1 -- 172.21.15.7:0/153621698 wait complete. 2024-09-21T04:40:49.896 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T04:40:49.942 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate[122129]: --> ceph-volume lvm activate successful for osd ID: 1 2024-09-21T04:40:49.943 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 bash[122115]: --> ceph-volume lvm activate successful for osd ID: 1 2024-09-21T04:40:49.943 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:49 smithi007 podman[122115]: 2024-09-21 04:40:49.706754574 +0000 UTC m=+3.847290085 container died c46a7504ff41c13f0daeee5c7b79a7d55c7a4d4273ed04b679c5824971c0a25c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T04:40:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.981+0000 7fd8908b5640 1 -- 172.21.15.7:0/1421116759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c107900 msgr2=0x7fd88c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.981+0000 7fd8908b5640 1 --2- 172.21.15.7:0/1421116759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c107900 0x7fd88c107ce0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009880 tx=0x7fd87c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.981+0000 7fd8908b5640 1 -- 172.21.15.7:0/1421116759 shutdown_connections 2024-09-21T04:40:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.981+0000 7fd8908b5640 1 --2- 172.21.15.7:0/1421116759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c101900 0x7fd88c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.981+0000 7fd8908b5640 1 --2- 172.21.15.7:0/1421116759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c107900 0x7fd88c107ce0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.981+0000 7fd8908b5640 1 -- 172.21.15.7:0/1421116759 >> 172.21.15.7:0/1421116759 conn(0x7fd88c0fd600 msgr2=0x7fd88c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.981+0000 7fd8908b5640 1 -- 172.21.15.7:0/1421116759 shutdown_connections 2024-09-21T04:40:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.981+0000 7fd8908b5640 1 -- 172.21.15.7:0/1421116759 wait complete. 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.982+0000 7fd8908b5640 1 Processor -- start 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.982+0000 7fd8908b5640 1 -- start start 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.982+0000 7fd8908b5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c101900 0x7fd88c19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.982+0000 7fd8908b5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c107900 0x7fd88c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.982+0000 7fd8908b5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c10ee30 con 0x7fd88c101900 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.982+0000 7fd8908b5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c10ecb0 con 0x7fd88c107900 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd88a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c101900 0x7fd88c19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd88a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c101900 0x7fd88c19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40022/0 (socket says 172.21.15.7:40022) 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd88a59c640 1 -- 172.21.15.7:0/2295078669 learned_addr learned my addr 172.21.15.7:0/2295078669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd889d9b640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c107900 0x7fd88c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd88a59c640 1 -- 172.21.15.7:0/2295078669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c107900 msgr2=0x7fd88c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd88a59c640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c107900 0x7fd88c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd88a59c640 1 -- 172.21.15.7:0/2295078669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c199a30 con 0x7fd88c101900 2024-09-21T04:40:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd88a59c640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c101900 0x7fd88c19f870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd87c031460 tx=0x7fd87c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd87b7fe640 1 -- 172.21.15.7:0/2295078669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd87c03f070 con 0x7fd88c101900 2024-09-21T04:40:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c199cc0 con 0x7fd88c101900 2024-09-21T04:40:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd87b7fe640 1 -- 172.21.15.7:0/2295078669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd87c004cf0 con 0x7fd88c101900 2024-09-21T04:40:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd87b7fe640 1 -- 172.21.15.7:0/2295078669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd87c035030 con 0x7fd88c101900 2024-09-21T04:40:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.983+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c19a1f0 con 0x7fd88c101900 2024-09-21T04:40:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.984+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88c103040 con 0x7fd88c101900 2024-09-21T04:40:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.985+0000 7fd87b7fe640 1 -- 172.21.15.7:0/2295078669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd87c042070 con 0x7fd88c101900 2024-09-21T04:40:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.986+0000 7fd87b7fe640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd87407af30 0x7fd87407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.986+0000 7fd87b7fe640 1 -- 172.21.15.7:0/2295078669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6070+0+0 (secure 0 0 0) 0x7fd87c0beae0 con 0x7fd88c101900 2024-09-21T04:40:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.986+0000 7fd889d9b640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd87407af30 0x7fd87407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.986+0000 7fd889d9b640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd87407af30 0x7fd87407d3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd88c075f50 tx=0x7fd880009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:49.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:49.990+0000 7fd87b7fe640 1 -- 172.21.15.7:0/2295078669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd87c08a090 con 0x7fd88c101900 2024-09-21T04:40:50.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:50 smithi007 ceph-mon[108238]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-09-21T04:40:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.199+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd88c19a4a0 con 0x7fd87407af30 2024-09-21T04:40:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.201+0000 7fd87b7fe640 1 -- 172.21.15.7:0/2295078669 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fd88c19a4a0 con 0x7fd87407af30 2024-09-21T04:40:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.202+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd87407af30 msgr2=0x7fd87407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.202+0000 7fd8908b5640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd87407af30 0x7fd87407d3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd88c075f50 tx=0x7fd880009110 comp rx=0 tx=0).stop 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c101900 msgr2=0x7fd88c19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c101900 0x7fd88c19f870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd87c031460 tx=0x7fd87c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 shutdown_connections 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd87407af30 0x7fd87407d3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c107900 0x7fd88c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 --2- 172.21.15.7:0/2295078669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c101900 0x7fd88c19f870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 >> 172.21.15.7:0/2295078669 conn(0x7fd88c0fd600 msgr2=0x7fd88c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 shutdown_connections 2024-09-21T04:40:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.203+0000 7fd8908b5640 1 -- 172.21.15.7:0/2295078669 wait complete. 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.287+0000 7f652d3f3640 1 -- 172.21.15.7:0/2196536541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 msgr2=0x7f6528101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.287+0000 7f652d3f3640 1 --2- 172.21.15.7:0/2196536541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 0x7f6528101d60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f651c009880 tx=0x7f651c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.287+0000 7f652d3f3640 1 -- 172.21.15.7:0/2196536541 shutdown_connections 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.287+0000 7f652d3f3640 1 --2- 172.21.15.7:0/2196536541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 0x7f6528101d60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.287+0000 7f652d3f3640 1 --2- 172.21.15.7:0/2196536541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6528107900 0x7f6528107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.287+0000 7f652d3f3640 1 -- 172.21.15.7:0/2196536541 >> 172.21.15.7:0/2196536541 conn(0x7f65280fd600 msgr2=0x7f65280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.287+0000 7f652d3f3640 1 -- 172.21.15.7:0/2196536541 shutdown_connections 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.287+0000 7f652d3f3640 1 -- 172.21.15.7:0/2196536541 wait complete. 2024-09-21T04:40:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.288+0000 7f652d3f3640 1 Processor -- start 2024-09-21T04:40:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.288+0000 7f652d3f3640 1 -- start start 2024-09-21T04:40:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.288+0000 7f652d3f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 0x7f652819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.288+0000 7f652d3f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6528107900 0x7f652819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.288+0000 7f652d3f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f652810edd0 con 0x7f6528101900 2024-09-21T04:40:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.288+0000 7f652d3f3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f652810ec50 con 0x7f6528107900 2024-09-21T04:40:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 0x7f652819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 0x7f652819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40028/0 (socket says 172.21.15.7:40028) 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f6526ffd640 1 -- 172.21.15.7:0/1044174327 learned_addr learned my addr 172.21.15.7:0/1044174327 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f65267fc640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6528107900 0x7f652819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f6526ffd640 1 -- 172.21.15.7:0/1044174327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6528107900 msgr2=0x7f652819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f6526ffd640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6528107900 0x7f652819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f6526ffd640 1 -- 172.21.15.7:0/1044174327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6528197800 con 0x7f6528101900 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f6526ffd640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 0x7f652819d680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f651800e970 tx=0x7f651800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f650ffff640 1 -- 172.21.15.7:0/1044174327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f651800cac0 con 0x7f6528101900 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f650ffff640 1 -- 172.21.15.7:0/1044174327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65180054e0 con 0x7f6528101900 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6528197af0 con 0x7f6528101900 2024-09-21T04:40:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f650ffff640 1 -- 172.21.15.7:0/1044174327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6518010430 con 0x7f6528101900 2024-09-21T04:40:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.289+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6528198050 con 0x7f6528101900 2024-09-21T04:40:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.290+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6528103040 con 0x7f6528101900 2024-09-21T04:40:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.291+0000 7f650ffff640 1 -- 172.21.15.7:0/1044174327 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6518004ea0 con 0x7f6528101900 2024-09-21T04:40:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.291+0000 7f650ffff640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f651007af30 0x7f651007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.292+0000 7f650ffff640 1 -- 172.21.15.7:0/1044174327 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6070+0+0 (secure 0 0 0) 0x7f651809e6f0 con 0x7f6528101900 2024-09-21T04:40:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.292+0000 7f65267fc640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f651007af30 0x7f651007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.292+0000 7f65267fc640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f651007af30 0x7f651007d3f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6528198b50 tx=0x7f651c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.296+0000 7f650ffff640 1 -- 172.21.15.7:0/1044174327 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6518069d40 con 0x7f6528101900 2024-09-21T04:40:50.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:50 smithi053 ceph-mon[80176]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-09-21T04:40:50.523 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:50 smithi007 podman[122362]: 2024-09-21 04:40:50.334097617 +0000 UTC m=+0.620459177 container remove c46a7504ff41c13f0daeee5c7b79a7d55c7a4d4273ed04b679c5824971c0a25c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-activate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T04:40:50.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.519+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f65281983c0 con 0x7f651007af30 2024-09-21T04:40:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.529+0000 7f650ffff640 1 -- 172.21.15.7:0/1044174327 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f65281983c0 con 0x7f651007af30 2024-09-21T04:40:50.532 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:40:50.532 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (2m) 32s ago 13m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (5m) 32s ago 14m 8887k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (5m) 56s ago 13m 8744k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (65s) 32s ago 13m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (60s) 56s ago 13m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (2m) 32s ago 13m 77.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (8m) 32s ago 8m 145M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (8m) 32s ago 8m 89.5M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (8m) 56s ago 8m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (8m) 56s ago 8m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (5m) 32s ago 15m 526M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (4m) 56s ago 12m 455M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (92s) 32s ago 15m 38.7M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:40:50.533 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (76s) 56s ago 12m 25.9M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (3m) 32s ago 13m 10.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (3m) 56s ago 13m 10.7M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (37s) 32s ago 11m 12.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (11m) 32s ago 11m 214M 1217M 19.1.1-373-g4e45e6b4 164465d2334a 4d8bf68bf697 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (10m) 32s ago 10m 193M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (10m) 56s ago 10m 243M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (9m) 56s ago 9m 212M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (9m) 56s ago 9m 184M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:40:50.534 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (3m) 32s ago 13m 51.6M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.531+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f651007af30 msgr2=0x7f651007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.531+0000 7f652d3f3640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f651007af30 0x7f651007d3f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6528198b50 tx=0x7f651c031400 comp rx=0 tx=0).stop 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.531+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 msgr2=0x7f652819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.531+0000 7f652d3f3640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 0x7f652819d680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f651800e970 tx=0x7f651800ed70 comp rx=0 tx=0).stop 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.532+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 shutdown_connections 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.532+0000 7f652d3f3640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f651007af30 0x7f651007d3f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.532+0000 7f652d3f3640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6528107900 0x7f652819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.532+0000 7f652d3f3640 1 --2- 172.21.15.7:0/1044174327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528101900 0x7f652819d680 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.532+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 >> 172.21.15.7:0/1044174327 conn(0x7f65280fd600 msgr2=0x7f65280fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.532+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 shutdown_connections 2024-09-21T04:40:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.532+0000 7f652d3f3640 1 -- 172.21.15.7:0/1044174327 wait complete. 2024-09-21T04:40:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.615+0000 7fdfaea1b640 1 -- 172.21.15.7:0/1065458467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa80ffdc0 msgr2=0x7fdfa8100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.615+0000 7fdfaea1b640 1 --2- 172.21.15.7:0/1065458467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa80ffdc0 0x7fdfa8100220 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009880 tx=0x7fdf9c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.615+0000 7fdfaea1b640 1 -- 172.21.15.7:0/1065458467 shutdown_connections 2024-09-21T04:40:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.615+0000 7fdfaea1b640 1 --2- 172.21.15.7:0/1065458467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa80ffdc0 0x7fdfa8100220 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.615+0000 7fdfaea1b640 1 --2- 172.21.15.7:0/1065458467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfa81069a0 0x7fdfa8106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.615+0000 7fdfaea1b640 1 -- 172.21.15.7:0/1065458467 >> 172.21.15.7:0/1065458467 conn(0x7fdfa80fbc90 msgr2=0x7fdfa80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.615+0000 7fdfaea1b640 1 -- 172.21.15.7:0/1065458467 shutdown_connections 2024-09-21T04:40:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.615+0000 7fdfaea1b640 1 -- 172.21.15.7:0/1065458467 wait complete. 2024-09-21T04:40:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfaea1b640 1 Processor -- start 2024-09-21T04:40:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfaea1b640 1 -- start start 2024-09-21T04:40:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfaea1b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfa80ffdc0 0x7fdfa819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfaea1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81069a0 0x7fdfa819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfaea1b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa810ef40 con 0x7fdfa81069a0 2024-09-21T04:40:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfaea1b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa810edc0 con 0x7fdfa80ffdc0 2024-09-21T04:40:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfa77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81069a0 0x7fdfa819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfa77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81069a0 0x7fdfa819dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40036/0 (socket says 172.21.15.7:40036) 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfa77fe640 1 -- 172.21.15.7:0/3097773681 learned_addr learned my addr 172.21.15.7:0/3097773681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.616+0000 7fdfa7fff640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfa80ffdc0 0x7fdfa819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfa77fe640 1 -- 172.21.15.7:0/3097773681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfa80ffdc0 msgr2=0x7fdfa819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfa77fe640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfa80ffdc0 0x7fdfa819d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfa77fe640 1 -- 172.21.15.7:0/3097773681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa81978f0 con 0x7fdfa81069a0 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfa77fe640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81069a0 0x7fdfa819dc70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c031460 tx=0x7fdf9c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfa57fa640 1 -- 172.21.15.7:0/3097773681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf9c03f070 con 0x7fdfa81069a0 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfa57fa640 1 -- 172.21.15.7:0/3097773681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf9c004b10 con 0x7fdfa81069a0 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfa57fa640 1 -- 172.21.15.7:0/3097773681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf9c035030 con 0x7fdfa81069a0 2024-09-21T04:40:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa8197b80 con 0x7fdfa81069a0 2024-09-21T04:40:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.617+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa81980b0 con 0x7fdfa81069a0 2024-09-21T04:40:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.618+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa81030e0 con 0x7fdfa81069a0 2024-09-21T04:40:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.619+0000 7fdfa57fa640 1 -- 172.21.15.7:0/3097773681 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdf9c042070 con 0x7fdfa81069a0 2024-09-21T04:40:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.619+0000 7fdfa57fa640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf9007af90 0x7fdf9007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.619+0000 7fdfa57fa640 1 -- 172.21.15.7:0/3097773681 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6070+0+0 (secure 0 0 0) 0x7fdf9c0beae0 con 0x7fdfa81069a0 2024-09-21T04:40:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.619+0000 7fdfa7fff640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf9007af90 0x7fdf9007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.620+0000 7fdfa7fff640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf9007af90 0x7fdf9007d450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005fd0 tx=0x7fdf98005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.624+0000 7fdfa57fa640 1 -- 172.21.15.7:0/3097773681 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdf9c08a1b0 con 0x7fdfa81069a0 2024-09-21T04:40:50.780 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:50 smithi007 podman[122468]: 2024-09-21 04:40:50.465699581 +0000 UTC m=+0.025781122 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:40:50.780 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:50 smithi007 podman[122468]: 2024-09-21 04:40:50.669166642 +0000 UTC m=+0.229248163 container create 55f4289c127550ca7ca03d3ea36cdd3f9ac7f89254ac6e1d93ba1c4976182d48 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:40:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.905+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fdfa8109ee0 con 0x7fdfa81069a0 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.905+0000 7fdfa57fa640 1 -- 172.21.15.7:0/3097773681 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7fdf9c08f060 con 0x7fdfa81069a0 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:40:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:40:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4, 2024-09-21T04:40:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 1 2024-09-21T04:40:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:40:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:40:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:40:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:40:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:40:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 8, 2024-09-21T04:40:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 5 2024-09-21T04:40:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:40:50.911 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:40:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf9007af90 msgr2=0x7fdf9007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf9007af90 0x7fdf9007d450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005fd0 tx=0x7fdf98005850 comp rx=0 tx=0).stop 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81069a0 msgr2=0x7fdfa819dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81069a0 0x7fdfa819dc70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c031460 tx=0x7fdf9c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 shutdown_connections 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf9007af90 0x7fdf9007d450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa81069a0 0x7fdfa819dc70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 --2- 172.21.15.7:0/3097773681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfa80ffdc0 0x7fdfa819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 >> 172.21.15.7:0/3097773681 conn(0x7fdfa80fbc90 msgr2=0x7fdfa80fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 shutdown_connections 2024-09-21T04:40:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.908+0000 7fdfaea1b640 1 -- 172.21.15.7:0/3097773681 wait complete. 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.994+0000 7f21c1af4640 1 -- 172.21.15.7:0/2396997007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 msgr2=0x7f21bc102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.994+0000 7f21c1af4640 1 --2- 172.21.15.7:0/2396997007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 0x7f21bc102f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009880 tx=0x7f21ac02a530 comp rx=0 tx=0).stop 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.994+0000 7f21c1af4640 1 -- 172.21.15.7:0/2396997007 shutdown_connections 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.994+0000 7f21c1af4640 1 --2- 172.21.15.7:0/2396997007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21bc103450 0x7f21bc10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.994+0000 7f21c1af4640 1 --2- 172.21.15.7:0/2396997007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 0x7f21bc102f10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.994+0000 7f21c1af4640 1 -- 172.21.15.7:0/2396997007 >> 172.21.15.7:0/2396997007 conn(0x7f21bc0fc550 msgr2=0x7f21bc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.994+0000 7f21c1af4640 1 -- 172.21.15.7:0/2396997007 shutdown_connections 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.994+0000 7f21c1af4640 1 -- 172.21.15.7:0/2396997007 wait complete. 2024-09-21T04:40:51.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21c1af4640 1 Processor -- start 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21c1af4640 1 -- start start 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21c1af4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 0x7f21bc19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21c1af4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21bc103450 0x7f21bc19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21c1af4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10edc0 con 0x7f21bc102b30 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21c1af4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10ec40 con 0x7f21bc103450 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 0x7f21bc19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 0x7f21bc19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40054/0 (socket says 172.21.15.7:40054) 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.995+0000 7f21bb7fe640 1 -- 172.21.15.7:0/2569014947 learned_addr learned my addr 172.21.15.7:0/2569014947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21b2ffd640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21bc103450 0x7f21bc19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21bb7fe640 1 -- 172.21.15.7:0/2569014947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21bc103450 msgr2=0x7f21bc19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21bb7fe640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21bc103450 0x7f21bc19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21bb7fe640 1 -- 172.21.15.7:0/2569014947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc1977b0 con 0x7f21bc102b30 2024-09-21T04:40:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21bb7fe640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 0x7f21bc19d630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21ac004f90 tx=0x7f21ac0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21b97fa640 1 -- 172.21.15.7:0/2569014947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f21ac03f070 con 0x7f21bc102b30 2024-09-21T04:40:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21b97fa640 1 -- 172.21.15.7:0/2569014947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21ac004c70 con 0x7f21bc102b30 2024-09-21T04:40:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc197a40 con 0x7f21bc102b30 2024-09-21T04:40:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21b97fa640 1 -- 172.21.15.7:0/2569014947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f21ac035030 con 0x7f21bc102b30 2024-09-21T04:40:51.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.996+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc197f70 con 0x7f21bc102b30 2024-09-21T04:40:51.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.997+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21bc06a500 con 0x7f21bc102b30 2024-09-21T04:40:51.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.998+0000 7f21b97fa640 1 -- 172.21.15.7:0/2569014947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f21ac042070 con 0x7f21bc102b30 2024-09-21T04:40:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.999+0000 7f21b97fa640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f219c07af80 0x7f219c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.999+0000 7f21b97fa640 1 -- 172.21.15.7:0/2569014947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6070+0+0 (secure 0 0 0) 0x7f21ac0be8e0 con 0x7f21bc102b30 2024-09-21T04:40:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.999+0000 7f21b2ffd640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f219c07af80 0x7f219c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:50.999+0000 7f21b2ffd640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f219c07af80 0x7f219c07d440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21bc108900 tx=0x7f21a400b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.003+0000 7f21b97fa640 1 -- 172.21.15.7:0/2569014947 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f21ac089f10 con 0x7f21bc102b30 2024-09-21T04:40:51.033 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:51 smithi007 ceph-mon[108238]: pgmap v156: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-09-21T04:40:51.033 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:51 smithi007 ceph-mon[108238]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:51.033 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:51 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3097773681' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:51.033 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:50 smithi007 podman[122468]: 2024-09-21 04:40:50.930537189 +0000 UTC m=+0.490618723 container init 55f4289c127550ca7ca03d3ea36cdd3f9ac7f89254ac6e1d93ba1c4976182d48 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T04:40:51.033 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:50 smithi007 podman[122468]: 2024-09-21 04:40:50.934372422 +0000 UTC m=+0.494453944 container start 55f4289c127550ca7ca03d3ea36cdd3f9ac7f89254ac6e1d93ba1c4976182d48 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T04:40:51.033 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:50 smithi007 ceph-osd[122511]: -- 172.21.15.7:0/3705002820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56141d4ae1e0 con 0x56141d4aa000 2024-09-21T04:40:51.034 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:51 smithi007 bash[122468]: 55f4289c127550ca7ca03d3ea36cdd3f9ac7f89254ac6e1d93ba1c4976182d48 2024-09-21T04:40:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.256+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f21bc105f90 con 0x7f21bc102b30 2024-09-21T04:40:51.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.257+0000 7f21b97fa640 1 -- 172.21.15.7:0/2569014947 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2038 (secure 0 0 0) 0x7f21ac08edc0 con 0x7f21bc102b30 2024-09-21T04:40:51.260 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:40:51.261 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 44 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:40:51.262 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14578,1=24327} 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:40:51.263 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:40:51.264 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:40:51.264 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:40:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:40:51.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.260+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f219c07af80 msgr2=0x7f219c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.260+0000 7f21c1af4640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f219c07af80 0x7f219c07d440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21bc108900 tx=0x7f21a400b410 comp rx=0 tx=0).stop 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.260+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 msgr2=0x7f21bc19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.260+0000 7f21c1af4640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 0x7f21bc19d630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21ac004f90 tx=0x7f21ac0048f0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.261+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 shutdown_connections 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.261+0000 7f21c1af4640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f219c07af80 0x7f219c07d440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.261+0000 7f21c1af4640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21bc103450 0x7f21bc19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.261+0000 7f21c1af4640 1 --2- 172.21.15.7:0/2569014947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21bc102b30 0x7f21bc19d630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.261+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 >> 172.21.15.7:0/2569014947 conn(0x7f21bc0fc550 msgr2=0x7f21bc108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.261+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 shutdown_connections 2024-09-21T04:40:51.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.261+0000 7f21c1af4640 1 -- 172.21.15.7:0/2569014947 wait complete. 2024-09-21T04:40:51.316 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:51 smithi007 systemd[1]: Started Ceph osd.1 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:40:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc4067fc640 1 -- 172.21.15.7:0/4240036998 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc031740 con 0x7fc408100800 2024-09-21T04:40:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc40df8d640 1 -- 172.21.15.7:0/4240036998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408100800 msgr2=0x7fc408100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc40df8d640 1 --2- 172.21.15.7:0/4240036998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408100800 0x7fc408100c60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009880 tx=0x7fc3fc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc40df8d640 1 -- 172.21.15.7:0/4240036998 shutdown_connections 2024-09-21T04:40:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc40df8d640 1 --2- 172.21.15.7:0/4240036998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408100800 0x7fc408100c60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc40df8d640 1 --2- 172.21.15.7:0/4240036998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc408106800 0x7fc408106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc40df8d640 1 -- 172.21.15.7:0/4240036998 >> 172.21.15.7:0/4240036998 conn(0x7fc4080fc520 msgr2=0x7fc4080fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc40df8d640 1 -- 172.21.15.7:0/4240036998 shutdown_connections 2024-09-21T04:40:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.359+0000 7fc40df8d640 1 -- 172.21.15.7:0/4240036998 wait complete. 2024-09-21T04:40:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc40df8d640 1 Processor -- start 2024-09-21T04:40:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc40df8d640 1 -- start start 2024-09-21T04:40:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc40df8d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc408100800 0x7fc4081965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc40df8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408106800 0x7fc408196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc40df8d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810dcb0 con 0x7fc408106800 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc40df8d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810db30 con 0x7fc408100800 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408106800 0x7fc408196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408106800 0x7fc408196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40080/0 (socket says 172.21.15.7:40080) 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc406ffd640 1 -- 172.21.15.7:0/3561755442 learned_addr learned my addr 172.21.15.7:0/3561755442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.360+0000 7fc4077fe640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc408100800 0x7fc4081965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc406ffd640 1 -- 172.21.15.7:0/3561755442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc408100800 msgr2=0x7fc4081965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc406ffd640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc408100800 0x7fc4081965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc406ffd640 1 -- 172.21.15.7:0/3561755442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc408197030 con 0x7fc408106800 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc406ffd640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408106800 0x7fc408196af0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc031460 tx=0x7fc3fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc404ff9640 1 -- 172.21.15.7:0/3561755442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc03f070 con 0x7fc408106800 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc404ff9640 1 -- 172.21.15.7:0/3561755442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3fc0035f0 con 0x7fc408106800 2024-09-21T04:40:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4081972c0 con 0x7fc408106800 2024-09-21T04:40:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc404ff9640 1 -- 172.21.15.7:0/3561755442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc035030 con 0x7fc408106800 2024-09-21T04:40:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.361+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a3df0 con 0x7fc408106800 2024-09-21T04:40:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.362+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc408101f40 con 0x7fc408106800 2024-09-21T04:40:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.364+0000 7fc404ff9640 1 -- 172.21.15.7:0/3561755442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc3fc042070 con 0x7fc408106800 2024-09-21T04:40:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.364+0000 7fc404ff9640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3f007afd0 0x7fc3f007d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.364+0000 7fc404ff9640 1 -- 172.21.15.7:0/3561755442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6070+0+0 (secure 0 0 0) 0x7fc3fc0bef30 con 0x7fc408106800 2024-09-21T04:40:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.365+0000 7fc4077fe640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3f007afd0 0x7fc3f007d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.365+0000 7fc4077fe640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3f007afd0 0x7fc3f007d490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8005d00 tx=0x7fc3f800b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.372+0000 7fc404ff9640 1 -- 172.21.15.7:0/3561755442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3fc08a5e0 con 0x7fc408106800 2024-09-21T04:40:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:51 smithi053 ceph-mon[80176]: pgmap v156: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-09-21T04:40:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:51 smithi053 ceph-mon[80176]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:51 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3097773681' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:40:51.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.608+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc408062630 con 0x7fc3f007afd0 2024-09-21T04:40:51.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.610+0000 7fc404ff9640 1 -- 172.21.15.7:0/3561755442 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fc408062630 con 0x7fc3f007afd0 2024-09-21T04:40:51.613 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:40:51.613 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:40:51.613 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:40:51.613 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:40:51.613 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:40:51.613 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:40:51.613 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:40:51.614 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:40:51.614 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:40:51.614 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "7/23 daemons upgraded", 2024-09-21T04:40:51.614 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:40:51.614 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:40:51.614 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:40:51.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3f007afd0 msgr2=0x7fc3f007d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3f007afd0 0x7fc3f007d490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8005d00 tx=0x7fc3f800b410 comp rx=0 tx=0).stop 2024-09-21T04:40:51.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408106800 msgr2=0x7fc408196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408106800 0x7fc408196af0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc031460 tx=0x7fc3fc0045e0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 shutdown_connections 2024-09-21T04:40:51.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3f007afd0 0x7fc3f007d490 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc408106800 0x7fc408196af0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 --2- 172.21.15.7:0/3561755442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc408100800 0x7fc4081965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.612+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 >> 172.21.15.7:0/3561755442 conn(0x7fc4080fc520 msgr2=0x7fc4080fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.613+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 shutdown_connections 2024-09-21T04:40:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.613+0000 7fc40df8d640 1 -- 172.21.15.7:0/3561755442 wait complete. 2024-09-21T04:40:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 -- 172.21.15.7:0/2544914841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 msgr2=0x7fdd00106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 --2- 172.21.15.7:0/2544914841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 0x7fdd00106bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009880 tx=0x7fdce802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 -- 172.21.15.7:0/2544914841 shutdown_connections 2024-09-21T04:40:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 --2- 172.21.15.7:0/2544914841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd001007f0 0x7fdd00100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 --2- 172.21.15.7:0/2544914841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 0x7fdd00106bd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 -- 172.21.15.7:0/2544914841 >> 172.21.15.7:0/2544914841 conn(0x7fdd000fc550 msgr2=0x7fdd000fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 -- 172.21.15.7:0/2544914841 shutdown_connections 2024-09-21T04:40:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 -- 172.21.15.7:0/2544914841 wait complete. 2024-09-21T04:40:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 Processor -- start 2024-09-21T04:40:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.699+0000 7fdd0720e640 1 -- start start 2024-09-21T04:40:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdd0720e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd001007f0 0x7fdd0019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdd0720e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 0x7fdd0019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdd0720e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0010dd20 con 0x7fdd001067f0 2024-09-21T04:40:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdd0720e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0010dba0 con 0x7fdd001007f0 2024-09-21T04:40:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdcfffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 0x7fdd0019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdcfffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 0x7fdd0019ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40110/0 (socket says 172.21.15.7:40110) 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdcfffff640 1 -- 172.21.15.7:0/1796742747 learned_addr learned my addr 172.21.15.7:0/1796742747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdd04faa640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd001007f0 0x7fdd0019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdcfffff640 1 -- 172.21.15.7:0/1796742747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd001007f0 msgr2=0x7fdd0019fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdcfffff640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd001007f0 0x7fdd0019fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdcfffff640 1 -- 172.21.15.7:0/1796742747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00199b10 con 0x7fdd001067f0 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdcfffff640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 0x7fdd0019ff80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000ba50 tx=0x7fdcf000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.700+0000 7fdcfdffb640 1 -- 172.21.15.7:0/1796742747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdcf0019070 con 0x7fdd001067f0 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.701+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd00199e00 con 0x7fdd001067f0 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.701+0000 7fdcfdffb640 1 -- 172.21.15.7:0/1796742747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdcf000ce20 con 0x7fdd001067f0 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.701+0000 7fdcfdffb640 1 -- 172.21.15.7:0/1796742747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdcf00075a0 con 0x7fdd001067f0 2024-09-21T04:40:51.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.701+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0019a390 con 0x7fdd001067f0 2024-09-21T04:40:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.702+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd00101f30 con 0x7fdd001067f0 2024-09-21T04:40:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.702+0000 7fdcfdffb640 1 -- 172.21.15.7:0/1796742747 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdcf001c070 con 0x7fdd001067f0 2024-09-21T04:40:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.703+0000 7fdcfdffb640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdcec07af30 0x7fdcec07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:40:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.703+0000 7fdcfdffb640 1 -- 172.21.15.7:0/1796742747 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6070+0+0 (secure 0 0 0) 0x7fdcf009d7f0 con 0x7fdd001067f0 2024-09-21T04:40:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.703+0000 7fdd04faa640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdcec07af30 0x7fdcec07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:40:51.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.703+0000 7fdd04faa640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdcec07af30 0x7fdcec07d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdce80096c0 tx=0x7fdce8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:40:51.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.707+0000 7fdcfdffb640 1 -- 172.21.15.7:0/1796742747 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdcf0068e40 con 0x7fdd001067f0 2024-09-21T04:40:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.996+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fdd0019afd0 con 0x7fdd001067f0 2024-09-21T04:40:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.996+0000 7fdcfdffb640 1 -- 172.21.15.7:0/1796742747 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+1294 (secure 0 0 0) 0x7fdcf006dcf0 con 0x7fdd001067f0 2024-09-21T04:40:52.000 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 49/333 objects degraded (14.715%), 15 pgs degraded 2024-09-21T04:40:52.001 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:40:52.001 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:40:52.001 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-21T04:40:52.001 INFO:teuthology.orchestra.run.smithi007.stdout: osd.1 (root=default,host=smithi007) is down 2024-09-21T04:40:52.001 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 49/333 objects degraded (14.715%), 15 pgs degraded 2024-09-21T04:40:52.001 INFO:teuthology.orchestra.run.smithi007.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-09-21T04:40:52.002 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-09-21T04:40:52.002 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-09-21T04:40:52.002 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-09-21T04:40:52.002 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-09-21T04:40:52.002 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-09-21T04:40:52.002 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-09-21T04:40:52.002 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-09-21T04:40:52.003 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-09-21T04:40:52.003 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-09-21T04:40:52.003 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-09-21T04:40:52.003 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-09-21T04:40:52.003 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-09-21T04:40:52.003 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-09-21T04:40:52.004 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-09-21T04:40:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdcec07af30 msgr2=0x7fdcec07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdcec07af30 0x7fdcec07d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdce80096c0 tx=0x7fdce8031400 comp rx=0 tx=0).stop 2024-09-21T04:40:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 msgr2=0x7fdd0019ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:40:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 0x7fdd0019ff80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000ba50 tx=0x7fdcf000bf10 comp rx=0 tx=0).stop 2024-09-21T04:40:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 shutdown_connections 2024-09-21T04:40:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdcec07af30 0x7fdcec07d3f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd001067f0 0x7fdd0019ff80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 --2- 172.21.15.7:0/1796742747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd001007f0 0x7fdd0019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:40:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.998+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 >> 172.21.15.7:0/1796742747 conn(0x7fdd000fc550 msgr2=0x7fdd00069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:40:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.999+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 shutdown_connections 2024-09-21T04:40:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:40:51.999+0000 7fdd0720e640 1 -- 172.21.15.7:0/1796742747 wait complete. 2024-09-21T04:40:52.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:52 smithi007 ceph-mon[108238]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:52.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:52 smithi007 ceph-mon[108238]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:52.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:52 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:52.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:52 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:52.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:52 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:40:52.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:52 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2569014947' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:40:52.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:52 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1796742747' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:40:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:52 smithi053 ceph-mon[80176]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:52 smithi053 ceph-mon[80176]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:52 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:52 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:52 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:40:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:52 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2569014947' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:40:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:52 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1796742747' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:40:52.868 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:52 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[122504]: 2024-09-21T04:40:52.835+0000 7f7803f97640 -1 Falling back to public interface 2024-09-21T04:40:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:53 smithi007 ceph-mon[108238]: from='client.34182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:53 smithi007 ceph-mon[108238]: pgmap v157: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-09-21T04:40:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:53 smithi053 ceph-mon[80176]: from='client.34182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:40:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:53 smithi053 ceph-mon[80176]: pgmap v157: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-09-21T04:40:54.335 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:54 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 49/333 objects degraded (14.715%), 15 pgs degraded (PG_DEGRADED) 2024-09-21T04:40:54.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:54 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 49/333 objects degraded (14.715%), 15 pgs degraded (PG_DEGRADED) 2024-09-21T04:40:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:55 smithi007 ceph-mon[108238]: pgmap v158: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-09-21T04:40:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:55 smithi053 ceph-mon[80176]: pgmap v158: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-09-21T04:40:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:57 smithi053 ceph-mon[80176]: pgmap v159: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-09-21T04:40:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:57 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:57 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:57 smithi007 ceph-mon[108238]: pgmap v159: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-09-21T04:40:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:57 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:57 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:58.072 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:57 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[122504]: 2024-09-21T04:40:57.843+0000 7f7803f97640 -1 osd.1 51 log_to_monitors true 2024-09-21T04:40:58.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:58.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:58.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:58 smithi053 ceph-mon[80176]: from='osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T04:40:58.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:58.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:58.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:58 smithi007 ceph-mon[108238]: from='osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T04:40:59.074 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 04:40:58 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[122504]: 2024-09-21T04:40:58.798+0000 7f77f7d39640 -1 osd.1 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:40:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:59 smithi053 ceph-mon[80176]: pgmap v160: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 49/333 objects degraded (14.715%) 2024-09-21T04:40:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:59 smithi053 ceph-mon[80176]: from='osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T04:40:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:59 smithi053 ceph-mon[80176]: osdmap e54: 6 total, 5 up, 6 in 2024-09-21T04:40:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:59 smithi053 ceph-mon[80176]: from='osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]} : dispatch 2024-09-21T04:40:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:59.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:40:59 smithi053 ceph-mon[80176]: from='osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290]' entity='osd.1' 2024-09-21T04:40:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:59 smithi007 ceph-mon[108238]: pgmap v160: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 49/333 objects degraded (14.715%) 2024-09-21T04:40:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:59 smithi007 ceph-mon[108238]: from='osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T04:40:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:59 smithi007 ceph-mon[108238]: osdmap e54: 6 total, 5 up, 6 in 2024-09-21T04:40:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:59 smithi007 ceph-mon[108238]: from='osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]} : dispatch 2024-09-21T04:40:59.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:40:59.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:40:59.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:40:59 smithi007 ceph-mon[108238]: from='osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290]' entity='osd.1' 2024-09-21T04:41:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:00 smithi053 ceph-mon[80176]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:41:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:00 smithi053 ceph-mon[80176]: pgmap v162: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 49/333 objects degraded (14.715%) 2024-09-21T04:41:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:00 smithi053 ceph-mon[80176]: osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290] boot 2024-09-21T04:41:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:00 smithi053 ceph-mon[80176]: osdmap e55: 6 total, 6 up, 6 in 2024-09-21T04:41:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T04:41:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:00 smithi007 ceph-mon[108238]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:41:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:00 smithi007 ceph-mon[108238]: pgmap v162: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 49/333 objects degraded (14.715%) 2024-09-21T04:41:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:00 smithi007 ceph-mon[108238]: osd.1 [v2:172.21.15.7:6810/537664290,v1:172.21.15.7:6811/537664290] boot 2024-09-21T04:41:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:00 smithi007 ceph-mon[108238]: osdmap e55: 6 total, 6 up, 6 in 2024-09-21T04:41:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:01.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:01.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-21T04:41:01.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-21T04:41:01.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-21T04:41:01.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:01 smithi007 ceph-mon[108238]: osdmap e56: 6 total, 6 up, 6 in 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-21T04:41:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-21T04:41:01.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:01 smithi053 ceph-mon[80176]: osdmap e56: 6 total, 6 up, 6 in 2024-09-21T04:41:02.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:02 smithi053 ceph-mon[80176]: pgmap v165: 65 pgs: 2 peering, 17 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 49/333 objects degraded (14.715%) 2024-09-21T04:41:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:02 smithi007 ceph-mon[108238]: pgmap v165: 65 pgs: 2 peering, 17 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 49/333 objects degraded (14.715%) 2024-09-21T04:41:04.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:03 smithi053 ceph-mon[80176]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/333 objects degraded (14.715%), 15 pgs degraded) 2024-09-21T04:41:04.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:03 smithi007 ceph-mon[108238]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/333 objects degraded (14.715%), 15 pgs degraded) 2024-09-21T04:41:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:04 smithi053 ceph-mon[80176]: pgmap v166: 65 pgs: 2 peering, 63 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T04:41:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:04 smithi007 ceph-mon[108238]: pgmap v166: 65 pgs: 2 peering, 63 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T04:41:07.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:06 smithi053 ceph-mon[80176]: pgmap v167: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:41:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:06 smithi007 ceph-mon[108238]: pgmap v167: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:41:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:08 smithi053 ceph-mon[80176]: pgmap v168: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-09-21T04:41:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:08 smithi007 ceph-mon[108238]: pgmap v168: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-09-21T04:41:11.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:10 smithi053 ceph-mon[80176]: pgmap v169: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:11.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:10 smithi007 ceph-mon[108238]: pgmap v169: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:12 smithi053 ceph-mon[80176]: pgmap v170: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-21T04:41:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:12 smithi007 ceph-mon[108238]: pgmap v170: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-21T04:41:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:41:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:14 smithi053 ceph-mon[80176]: pgmap v171: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:41:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:41:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:14 smithi007 ceph-mon[108238]: pgmap v171: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:41:15.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:15 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-21T04:41:16.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:15 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-21T04:41:17.088 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:16 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-21T04:41:17.088 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:16 smithi007 ceph-mon[108238]: Upgrade: osd.2 is safe to restart 2024-09-21T04:41:17.088 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:16 smithi007 ceph-mon[108238]: pgmap v172: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:16 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-21T04:41:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:16 smithi053 ceph-mon[80176]: Upgrade: osd.2 is safe to restart 2024-09-21T04:41:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:16 smithi053 ceph-mon[80176]: pgmap v172: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:18.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:18 smithi007 ceph-mon[108238]: Upgrade: Updating osd.2 2024-09-21T04:41:18.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:18.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T04:41:18.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:41:18.492 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:18 smithi007 ceph-mon[108238]: Deploying daemon osd.2 on smithi007 2024-09-21T04:41:18.492 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:18 smithi007 ceph-mon[108238]: pgmap v173: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:18 smithi053 ceph-mon[80176]: Upgrade: Updating osd.2 2024-09-21T04:41:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T04:41:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:41:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:18 smithi053 ceph-mon[80176]: Deploying daemon osd.2 on smithi007 2024-09-21T04:41:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:18 smithi053 ceph-mon[80176]: pgmap v173: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:20.396 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:20 smithi007 systemd[1]: Stopping Ceph osd.2 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:41:20.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:20 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[63454]: 2024-09-21T04:41:20.393+0000 7fd8123ba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:41:20.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:20 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[63454]: 2024-09-21T04:41:20.393+0000 7fd8123ba640 -1 osd.2 56 *** Got signal Terminated *** 2024-09-21T04:41:20.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:20 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[63454]: 2024-09-21T04:41:20.393+0000 7fd8123ba640 -1 osd.2 56 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:41:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:20 smithi053 ceph-mon[80176]: pgmap v174: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:41:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:20 smithi053 ceph-mon[80176]: osd.2 marked itself down and dead 2024-09-21T04:41:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:20 smithi007 ceph-mon[108238]: pgmap v174: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:41:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:20 smithi007 ceph-mon[108238]: osd.2 marked itself down and dead 2024-09-21T04:41:21.640 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:21 smithi007 podman[127120]: 2024-09-21 04:41:21.385815298 +0000 UTC m=+1.101817546 container died e408323deeab9626d3f69a02f03b1f9f0d928c096e492424f4361542ec9c2815 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-4e45e6b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T04:41:21.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:21 smithi007 ceph-mon[108238]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:41:21.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:21 smithi007 ceph-mon[108238]: osdmap e57: 6 total, 5 up, 6 in 2024-09-21T04:41:21.944 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:21 smithi007 podman[127120]: 2024-09-21 04:41:21.735353608 +0000 UTC m=+1.451355844 container cleanup e408323deeab9626d3f69a02f03b1f9f0d928c096e492424f4361542ec9c2815 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, RELEASE=squid-4e45e6b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T04:41:21.944 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:21 smithi007 bash[127120]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.091+0000 7efe980f7640 1 -- 172.21.15.7:0/2754386689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 msgr2=0x7efe9010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.091+0000 7efe980f7640 1 --2- 172.21.15.7:0/2754386689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 0x7efe9010b9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe84009880 tx=0x7efe8402a530 comp rx=0 tx=0).stop 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.092+0000 7efe980f7640 1 -- 172.21.15.7:0/2754386689 shutdown_connections 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.092+0000 7efe980f7640 1 --2- 172.21.15.7:0/2754386689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 0x7efe9010b9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.092+0000 7efe980f7640 1 --2- 172.21.15.7:0/2754386689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90102b90 0x7efe90102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.092+0000 7efe980f7640 1 -- 172.21.15.7:0/2754386689 >> 172.21.15.7:0/2754386689 conn(0x7efe900fc510 msgr2=0x7efe900fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.092+0000 7efe980f7640 1 -- 172.21.15.7:0/2754386689 shutdown_connections 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.092+0000 7efe980f7640 1 -- 172.21.15.7:0/2754386689 wait complete. 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.092+0000 7efe980f7640 1 Processor -- start 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.092+0000 7efe980f7640 1 -- start start 2024-09-21T04:41:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe980f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90102b90 0x7efe901007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe980f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 0x7efe90100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe980f7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe9010ee00 con 0x7efe901034b0 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe980f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe9010ec80 con 0x7efe90102b90 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe95e93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90102b90 0x7efe901007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe95e93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90102b90 0x7efe901007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:52664/0 (socket says 172.21.15.7:52664) 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe95e93640 1 -- 172.21.15.7:0/649224097 learned_addr learned my addr 172.21.15.7:0/649224097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe95692640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 0x7efe90100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe95e93640 1 -- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 msgr2=0x7efe90100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe95e93640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 0x7efe90100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe95e93640 1 -- 172.21.15.7:0/649224097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe90101270 con 0x7efe90102b90 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.093+0000 7efe95692640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 0x7efe90100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.094+0000 7efe95e93640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90102b90 0x7efe901007f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00d6f0 tx=0x7efe8c00dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.094+0000 7efe82ffd640 1 -- 172.21.15.7:0/649224097 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe8c017070 con 0x7efe90102b90 2024-09-21T04:41:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.094+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe90101560 con 0x7efe90102b90 2024-09-21T04:41:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.094+0000 7efe82ffd640 1 -- 172.21.15.7:0/649224097 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe8c00de00 con 0x7efe90102b90 2024-09-21T04:41:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.094+0000 7efe82ffd640 1 -- 172.21.15.7:0/649224097 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe8c004bb0 con 0x7efe90102b90 2024-09-21T04:41:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.094+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe901a3e70 con 0x7efe90102b90 2024-09-21T04:41:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.095+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe9006ace0 con 0x7efe90102b90 2024-09-21T04:41:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.096+0000 7efe82ffd640 1 -- 172.21.15.7:0/649224097 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7efe8c004e60 con 0x7efe90102b90 2024-09-21T04:41:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.096+0000 7efe82ffd640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7c07af40 0x7efe7c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.096+0000 7efe82ffd640 1 -- 172.21.15.7:0/649224097 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6070+0+0 (secure 0 0 0) 0x7efe8c09dbe0 con 0x7efe90102b90 2024-09-21T04:41:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.096+0000 7efe95692640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7c07af40 0x7efe7c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.097+0000 7efe95692640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7c07af40 0x7efe7c07d400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe84004f90 tx=0x7efe840041a0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:22.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.101+0000 7efe82ffd640 1 -- 172.21.15.7:0/649224097 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe8c069230 con 0x7efe90102b90 2024-09-21T04:41:22.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:21 smithi053 ceph-mon[80176]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:41:22.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:21 smithi053 ceph-mon[80176]: osdmap e57: 6 total, 5 up, 6 in 2024-09-21T04:41:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.368+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7efe90062630 con 0x7efe7c07af40 2024-09-21T04:41:22.371 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:22 smithi007 podman[127135]: 2024-09-21 04:41:22.028852918 +0000 UTC m=+0.645315203 container remove e408323deeab9626d3f69a02f03b1f9f0d928c096e492424f4361542ec9c2815 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-4e45e6b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T04:41:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.369+0000 7efe82ffd640 1 -- 172.21.15.7:0/649224097 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7efe90062630 con 0x7efe7c07af40 2024-09-21T04:41:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7c07af40 msgr2=0x7efe7c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7c07af40 0x7efe7c07d400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe84004f90 tx=0x7efe840041a0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90102b90 msgr2=0x7efe901007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90102b90 0x7efe901007f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00d6f0 tx=0x7efe8c00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 shutdown_connections 2024-09-21T04:41:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efe7c07af40 0x7efe7c07d400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe901034b0 0x7efe90100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 --2- 172.21.15.7:0/649224097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe90102b90 0x7efe901007f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 >> 172.21.15.7:0/649224097 conn(0x7efe900fc510 msgr2=0x7efe90103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 shutdown_connections 2024-09-21T04:41:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.371+0000 7efe980f7640 1 -- 172.21.15.7:0/649224097 wait complete. 2024-09-21T04:41:22.383 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:41:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.457+0000 7fcb17617640 1 -- 172.21.15.7:0/2044818378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 msgr2=0x7fcb1010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.457+0000 7fcb17617640 1 --2- 172.21.15.7:0/2044818378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 0x7fcb1010b8d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009880 tx=0x7fcb0002a530 comp rx=0 tx=0).stop 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.457+0000 7fcb17617640 1 -- 172.21.15.7:0/2044818378 shutdown_connections 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.457+0000 7fcb17617640 1 --2- 172.21.15.7:0/2044818378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 0x7fcb1010b8d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.457+0000 7fcb17617640 1 --2- 172.21.15.7:0/2044818378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb10102ac0 0x7fcb10102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.457+0000 7fcb17617640 1 -- 172.21.15.7:0/2044818378 >> 172.21.15.7:0/2044818378 conn(0x7fcb100fc520 msgr2=0x7fcb100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.457+0000 7fcb17617640 1 -- 172.21.15.7:0/2044818378 shutdown_connections 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.457+0000 7fcb17617640 1 -- 172.21.15.7:0/2044818378 wait complete. 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb17617640 1 Processor -- start 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb17617640 1 -- start start 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb17617640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb10102ac0 0x7fcb1019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb17617640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 0x7fcb1019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb17617640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010edd0 con 0x7fcb101033e0 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb17617640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010ec50 con 0x7fcb10102ac0 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb14bb2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 0x7fcb1019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb14bb2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 0x7fcb1019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34938/0 (socket says 172.21.15.7:34938) 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb14bb2640 1 -- 172.21.15.7:0/3780732447 learned_addr learned my addr 172.21.15.7:0/3780732447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb153b3640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb10102ac0 0x7fcb1019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb14bb2640 1 -- 172.21.15.7:0/3780732447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb10102ac0 msgr2=0x7fcb1019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb14bb2640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb10102ac0 0x7fcb1019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb14bb2640 1 -- 172.21.15.7:0/3780732447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb101977e0 con 0x7fcb101033e0 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.458+0000 7fcb153b3640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb10102ac0 0x7fcb1019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.459+0000 7fcb14bb2640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 0x7fcb1019dba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb00031460 tx=0x7fcb000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.459+0000 7fcb067fc640 1 -- 172.21.15.7:0/3780732447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcb0003f070 con 0x7fcb101033e0 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.459+0000 7fcb067fc640 1 -- 172.21.15.7:0/3780732447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb00004ac0 con 0x7fcb101033e0 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.459+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb10197a70 con 0x7fcb101033e0 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.459+0000 7fcb067fc640 1 -- 172.21.15.7:0/3780732447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcb00035030 con 0x7fcb101033e0 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.459+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb10197fa0 con 0x7fcb101033e0 2024-09-21T04:41:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.460+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1006aa90 con 0x7fcb101033e0 2024-09-21T04:41:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.461+0000 7fcb067fc640 1 -- 172.21.15.7:0/3780732447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fcb00042070 con 0x7fcb101033e0 2024-09-21T04:41:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.461+0000 7fcb067fc640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcafc07af30 0x7fcafc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.461+0000 7fcb067fc640 1 -- 172.21.15.7:0/3780732447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6070+0+0 (secure 0 0 0) 0x7fcb000bea10 con 0x7fcb101033e0 2024-09-21T04:41:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.461+0000 7fcb153b3640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcafc07af30 0x7fcafc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.462+0000 7fcb153b3640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcafc07af30 0x7fcafc07d3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00abe0 tx=0x7fcb0c00a5c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.466+0000 7fcb067fc640 1 -- 172.21.15.7:0/3780732447 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb0008a060 con 0x7fcb101033e0 2024-09-21T04:41:22.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.689+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fcb10198250 con 0x7fcafc07af30 2024-09-21T04:41:22.693 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:22 smithi007 podman[127246]: 2024-09-21 04:41:22.334307008 +0000 UTC m=+0.017327954 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:41:22.693 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:22 smithi007 podman[127246]: 2024-09-21 04:41:22.472285255 +0000 UTC m=+0.155306189 container create b7c845091178c858837d5231697ca812dc44e331b92e56a22322e5f4e764e9d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD) 2024-09-21T04:41:22.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.691+0000 7fcb067fc640 1 -- 172.21.15.7:0/3780732447 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fcb10198250 con 0x7fcafc07af30 2024-09-21T04:41:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcafc07af30 msgr2=0x7fcafc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcafc07af30 0x7fcafc07d3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00abe0 tx=0x7fcb0c00a5c0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 msgr2=0x7fcb1019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 0x7fcb1019dba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb00031460 tx=0x7fcb000046c0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 shutdown_connections 2024-09-21T04:41:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcafc07af30 0x7fcafc07d3f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb101033e0 0x7fcb1019dba0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 --2- 172.21.15.7:0/3780732447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb10102ac0 0x7fcb1019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 >> 172.21.15.7:0/3780732447 conn(0x7fcb100fc520 msgr2=0x7fcb10108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 shutdown_connections 2024-09-21T04:41:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.694+0000 7fcb17617640 1 -- 172.21.15.7:0/3780732447 wait complete. 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.780+0000 7f2c5004c640 1 -- 172.21.15.7:0/2659276586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481007f0 msgr2=0x7f2c48100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.780+0000 7f2c5004c640 1 --2- 172.21.15.7:0/2659276586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481007f0 0x7f2c48100c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009880 tx=0x7f2c3c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.780+0000 7f2c5004c640 1 -- 172.21.15.7:0/2659276586 shutdown_connections 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.780+0000 7f2c5004c640 1 --2- 172.21.15.7:0/2659276586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481007f0 0x7f2c48100c50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.780+0000 7f2c5004c640 1 --2- 172.21.15.7:0/2659276586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c481067f0 0x7f2c48106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.780+0000 7f2c5004c640 1 -- 172.21.15.7:0/2659276586 >> 172.21.15.7:0/2659276586 conn(0x7f2c480fc550 msgr2=0x7f2c480fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.780+0000 7f2c5004c640 1 -- 172.21.15.7:0/2659276586 shutdown_connections 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.780+0000 7f2c5004c640 1 -- 172.21.15.7:0/2659276586 wait complete. 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c5004c640 1 Processor -- start 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c5004c640 1 -- start start 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c5004c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c481007f0 0x7f2c4819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c5004c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481067f0 0x7f2c4819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c5004c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810dcc0 con 0x7f2c481067f0 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c5004c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4810db40 con 0x7f2c481007f0 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c4d5e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481067f0 0x7f2c4819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c4d5e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481067f0 0x7f2c4819fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34952/0 (socket says 172.21.15.7:34952) 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c4d5e7640 1 -- 172.21.15.7:0/2750928137 learned_addr learned my addr 172.21.15.7:0/2750928137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c4dde8640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c481007f0 0x7f2c4819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c4d5e7640 1 -- 172.21.15.7:0/2750928137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c481007f0 msgr2=0x7f2c4819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c4d5e7640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c481007f0 0x7f2c4819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c4d5e7640 1 -- 172.21.15.7:0/2750928137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c48199a10 con 0x7f2c481067f0 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.781+0000 7f2c4dde8640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c481007f0 0x7f2c4819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:41:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.782+0000 7f2c4d5e7640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481067f0 0x7f2c4819fd90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c031460 tx=0x7f2c3c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.782+0000 7f2c3affd640 1 -- 172.21.15.7:0/2750928137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2c3c03f070 con 0x7f2c481067f0 2024-09-21T04:41:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.782+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c48199ca0 con 0x7f2c481067f0 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.782+0000 7f2c3affd640 1 -- 172.21.15.7:0/2750928137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c3c0035f0 con 0x7f2c481067f0 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.782+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4819a1d0 con 0x7f2c481067f0 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.782+0000 7f2c3affd640 1 -- 172.21.15.7:0/2750928137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2c3c035030 con 0x7f2c481067f0 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.783+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c48101f30 con 0x7f2c481067f0 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.784+0000 7f2c3affd640 1 -- 172.21.15.7:0/2750928137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2c3c042070 con 0x7f2c481067f0 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.784+0000 7f2c3affd640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2c3407af30 0x7f2c3407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.784+0000 7f2c3affd640 1 -- 172.21.15.7:0/2750928137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6070+0+0 (secure 0 0 0) 0x7f2c3c0be950 con 0x7f2c481067f0 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.785+0000 7f2c4dde8640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2c3407af30 0x7f2c3407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.785+0000 7f2c4dde8640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2c3407af30 0x7f2c3407d3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c4819ab50 tx=0x7f2c44005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.789+0000 7f2c3affd640 1 -- 172.21.15.7:0/2750928137 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2c3c089f00 con 0x7f2c481067f0 2024-09-21T04:41:22.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:22 smithi007 ceph-mon[108238]: pgmap v176: 65 pgs: 9 stale+active+clean, 56 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:41:22.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:22 smithi007 ceph-mon[108238]: osdmap e58: 6 total, 5 up, 6 in 2024-09-21T04:41:22.945 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:22 smithi007 podman[127246]: 2024-09-21 04:41:22.736032015 +0000 UTC m=+0.419052965 container init b7c845091178c858837d5231697ca812dc44e331b92e56a22322e5f4e764e9d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=) 2024-09-21T04:41:22.945 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:22 smithi007 podman[127246]: 2024-09-21 04:41:22.741157859 +0000 UTC m=+0.424178803 container start b7c845091178c858837d5231697ca812dc44e331b92e56a22322e5f4e764e9d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T04:41:22.945 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:22 smithi007 podman[127246]: 2024-09-21 04:41:22.841807485 +0000 UTC m=+0.524828435 container attach b7c845091178c858837d5231697ca812dc44e331b92e56a22322e5f4e764e9d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T04:41:22.945 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:22 smithi007 podman[127246]: 2024-09-21 04:41:22.935964339 +0000 UTC m=+0.618985292 container died b7c845091178c858837d5231697ca812dc44e331b92e56a22322e5f4e764e9d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T04:41:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:22.991+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2c4819a480 con 0x7f2c3407af30 2024-09-21T04:41:23.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.001+0000 7f2c3affd640 1 -- 172.21.15.7:0/2750928137 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f2c4819a480 con 0x7f2c3407af30 2024-09-21T04:41:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:41:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (3m) 26s ago 14m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:41:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (6m) 26s ago 14m 8967k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:41:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (6m) 89s ago 13m 8744k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:41:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (97s) 26s ago 14m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:41:23.005 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (92s) 89s ago 13m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:41:23.005 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (3m) 26s ago 14m 77.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:41:23.005 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (8m) 26s ago 8m 145M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:41:23.005 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (9m) 26s ago 9m 89.7M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:41:23.005 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (8m) 89s ago 8m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:41:23.005 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (9m) 89s ago 9m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:41:23.005 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (5m) 26s ago 15m 527M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:41:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (4m) 89s ago 13m 455M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:41:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (2m) 26s ago 15m 42.3M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:41:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (109s) 89s ago 13m 25.9M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:41:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (4m) 26s ago 14m 11.3M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:41:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (4m) 89s ago 13m 10.7M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:41:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (70s) 26s ago 12m 86.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:41:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (32s) 26s ago 11m 11.4M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:41:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (11m) 26s ago 11m 190M 1217M 19.1.1-373-g4e45e6b4 164465d2334a e408323deeab 2024-09-21T04:41:23.007 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (10m) 89s ago 10m 243M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:41:23.007 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (10m) 89s ago 10m 212M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:41:23.007 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (10m) 89s ago 10m 184M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:41:23.007 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (3m) 26s ago 13m 56.0M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:41:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2c3407af30 msgr2=0x7f2c3407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2c3407af30 0x7f2c3407d3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c4819ab50 tx=0x7f2c44005f90 comp rx=0 tx=0).stop 2024-09-21T04:41:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481067f0 msgr2=0x7f2c4819fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481067f0 0x7f2c4819fd90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c031460 tx=0x7f2c3c004530 comp rx=0 tx=0).stop 2024-09-21T04:41:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 shutdown_connections 2024-09-21T04:41:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2c3407af30 0x7f2c3407d3f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c481067f0 0x7f2c4819fd90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 --2- 172.21.15.7:0/2750928137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c481007f0 0x7f2c4819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.003+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 >> 172.21.15.7:0/2750928137 conn(0x7f2c480fc550 msgr2=0x7f2c48069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.004+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 shutdown_connections 2024-09-21T04:41:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.004+0000 7f2c5004c640 1 -- 172.21.15.7:0/2750928137 wait complete. 2024-09-21T04:41:23.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.087+0000 7f79f363f640 1 -- 172.21.15.7:0/2966647695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec106810 msgr2=0x7f79ec106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.087+0000 7f79f363f640 1 --2- 172.21.15.7:0/2966647695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec106810 0x7f79ec106bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009880 tx=0x7f79e802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.087+0000 7f79f363f640 1 -- 172.21.15.7:0/2966647695 shutdown_connections 2024-09-21T04:41:23.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.087+0000 7f79f363f640 1 --2- 172.21.15.7:0/2966647695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ec100810 0x7f79ec100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.087+0000 7f79f363f640 1 --2- 172.21.15.7:0/2966647695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec106810 0x7f79ec106bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.087+0000 7f79f363f640 1 -- 172.21.15.7:0/2966647695 >> 172.21.15.7:0/2966647695 conn(0x7f79ec0fc510 msgr2=0x7f79ec0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:23.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.087+0000 7f79f363f640 1 -- 172.21.15.7:0/2966647695 shutdown_connections 2024-09-21T04:41:23.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.087+0000 7f79f363f640 1 -- 172.21.15.7:0/2966647695 wait complete. 2024-09-21T04:41:23.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.088+0000 7f79f363f640 1 Processor -- start 2024-09-21T04:41:23.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.088+0000 7f79f363f640 1 -- start start 2024-09-21T04:41:23.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.088+0000 7f79f363f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec100810 0x7f79ec19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.088+0000 7f79f363f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ec106810 0x7f79ec19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.088+0000 7f79f363f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec10cbd0 con 0x7f79ec100810 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.088+0000 7f79f363f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec10ca50 con 0x7f79ec106810 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f13db640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec100810 0x7f79ec19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f13db640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec100810 0x7f79ec19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34978/0 (socket says 172.21.15.7:34978) 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f13db640 1 -- 172.21.15.7:0/1806362655 learned_addr learned my addr 172.21.15.7:0/1806362655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f0bda640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ec106810 0x7f79ec19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f13db640 1 -- 172.21.15.7:0/1806362655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ec106810 msgr2=0x7f79ec19b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f13db640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ec106810 0x7f79ec19b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f13db640 1 -- 172.21.15.7:0/1806362655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec1955c0 con 0x7f79ec100810 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f13db640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec100810 0x7f79ec19b400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79e8031460 tx=0x7f79e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79e27fc640 1 -- 172.21.15.7:0/1806362655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f79e803f070 con 0x7f79ec100810 2024-09-21T04:41:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec195850 con 0x7f79ec100810 2024-09-21T04:41:23.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79e27fc640 1 -- 172.21.15.7:0/1806362655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f79e8004b10 con 0x7f79ec100810 2024-09-21T04:41:23.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79e27fc640 1 -- 172.21.15.7:0/1806362655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f79e8035030 con 0x7f79ec100810 2024-09-21T04:41:23.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.089+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec195d80 con 0x7f79ec100810 2024-09-21T04:41:23.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.090+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ec101f50 con 0x7f79ec100810 2024-09-21T04:41:23.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.091+0000 7f79e27fc640 1 -- 172.21.15.7:0/1806362655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f79e8042070 con 0x7f79ec100810 2024-09-21T04:41:23.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.092+0000 7f79e27fc640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f79d807d150 0x7f79d807f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.092+0000 7f79e27fc640 1 -- 172.21.15.7:0/1806362655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6070+0+0 (secure 0 0 0) 0x7f79e80bea50 con 0x7f79ec100810 2024-09-21T04:41:23.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.092+0000 7f79f0bda640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f79d807d150 0x7f79d807f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.092+0000 7f79f0bda640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f79d807d150 0x7f79d807f610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0ffc40 tx=0x7f79dc00b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:23.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.096+0000 7f79e27fc640 1 -- 172.21.15.7:0/1806362655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79e808a0a0 con 0x7f79ec100810 2024-09-21T04:41:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:22 smithi053 ceph-mon[80176]: pgmap v176: 65 pgs: 9 stale+active+clean, 56 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:41:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:22 smithi053 ceph-mon[80176]: osdmap e58: 6 total, 5 up, 6 in 2024-09-21T04:41:23.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.362+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f79ec103a20 con 0x7f79ec100810 2024-09-21T04:41:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.363+0000 7f79e27fc640 1 -- 172.21.15.7:0/1806362655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7f79e808c5c0 con 0x7f79ec100810 2024-09-21T04:41:23.366 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:41:23.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:41:23.366 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:41:23.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:41:23.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 3, 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 7, 2024-09-21T04:41:23.367 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:41:23.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:41:23.368 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:41:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f79d807d150 msgr2=0x7f79d807f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f79d807d150 0x7f79d807f610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0ffc40 tx=0x7f79dc00b410 comp rx=0 tx=0).stop 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec100810 msgr2=0x7f79ec19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec100810 0x7f79ec19b400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79e8031460 tx=0x7f79e80045e0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 shutdown_connections 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f79d807d150 0x7f79d807f610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ec106810 0x7f79ec19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 --2- 172.21.15.7:0/1806362655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ec100810 0x7f79ec19b400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 >> 172.21.15.7:0/1806362655 conn(0x7f79ec0fc510 msgr2=0x7f79ec0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 shutdown_connections 2024-09-21T04:41:23.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.365+0000 7f79f363f640 1 -- 172.21.15.7:0/1806362655 wait complete. 2024-09-21T04:41:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 -- 172.21.15.7:0/441302862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 msgr2=0x7f598410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 --2- 172.21.15.7:0/441302862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 0x7f598410b8d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5978009880 tx=0x7f597802a530 comp rx=0 tx=0).stop 2024-09-21T04:41:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 -- 172.21.15.7:0/441302862 shutdown_connections 2024-09-21T04:41:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 --2- 172.21.15.7:0/441302862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 0x7f598410b8d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 --2- 172.21.15.7:0/441302862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5984102ac0 0x7f5984102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 -- 172.21.15.7:0/441302862 >> 172.21.15.7:0/441302862 conn(0x7f59840fc520 msgr2=0x7f59840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 -- 172.21.15.7:0/441302862 shutdown_connections 2024-09-21T04:41:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 -- 172.21.15.7:0/441302862 wait complete. 2024-09-21T04:41:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.450+0000 7f5988a6b640 1 Processor -- start 2024-09-21T04:41:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5988a6b640 1 -- start start 2024-09-21T04:41:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5988a6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5984102ac0 0x7f5984100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5988a6b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 0x7f5984100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5988a6b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f598410edd0 con 0x7f59841033e0 2024-09-21T04:41:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5988a6b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f598410ec50 con 0x7f5984102ac0 2024-09-21T04:41:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5981d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 0x7f5984100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5981d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 0x7f5984100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39380/0 (socket says 172.21.15.7:39380) 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5981d9b640 1 -- 172.21.15.7:0/1441062602 learned_addr learned my addr 172.21.15.7:0/1441062602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f598259c640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5984102ac0 0x7f5984100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5981d9b640 1 -- 172.21.15.7:0/1441062602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5984102ac0 msgr2=0x7f5984100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5981d9b640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5984102ac0 0x7f5984100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.451+0000 7f5981d9b640 1 -- 172.21.15.7:0/1441062602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59841011a0 con 0x7f59841033e0 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.452+0000 7f5981d9b640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 0x7f5984100c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5978031460 tx=0x7f5978004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.452+0000 7f59737fe640 1 -- 172.21.15.7:0/1441062602 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f597803f070 con 0x7f59841033e0 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.452+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5984101430 con 0x7f59841033e0 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.452+0000 7f59737fe640 1 -- 172.21.15.7:0/1441062602 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59780035f0 con 0x7f59841033e0 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.452+0000 7f59737fe640 1 -- 172.21.15.7:0/1441062602 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5978035030 con 0x7f59841033e0 2024-09-21T04:41:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.452+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841a3d80 con 0x7f59841033e0 2024-09-21T04:41:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.453+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f598406aa90 con 0x7f59841033e0 2024-09-21T04:41:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.454+0000 7f59737fe640 1 -- 172.21.15.7:0/1441062602 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5978042070 con 0x7f59841033e0 2024-09-21T04:41:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.454+0000 7f59737fe640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f596c07af30 0x7f596c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.454+0000 7f598259c640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f596c07af30 0x7f596c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.454+0000 7f59737fe640 1 -- 172.21.15.7:0/1441062602 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6070+0+0 (secure 0 0 0) 0x7f59780be960 con 0x7f59841033e0 2024-09-21T04:41:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.455+0000 7f598259c640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f596c07af30 0x7f596c07d3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5974005fd0 tx=0x7f5974005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:23.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.459+0000 7f59737fe640 1 -- 172.21.15.7:0/1441062602 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5978089fb0 con 0x7f59841033e0 2024-09-21T04:41:23.599 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:23 smithi007 podman[127315]: 2024-09-21 04:41:23.596791541 +0000 UTC m=+0.654967704 container remove b7c845091178c858837d5231697ca812dc44e331b92e56a22322e5f4e764e9d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T04:41:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.702+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5984105f60 con 0x7f59841033e0 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.703+0000 7f59737fe640 1 -- 172.21.15.7:0/1441062602 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2038 (secure 0 0 0) 0x7f597808ee60 con 0x7f59841033e0 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:41:23.706 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 44 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:41:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14578,1=24327} 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:41:23.708 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:41:23.709 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:41:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:41:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:41:23.709 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:41:23.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.706+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f596c07af30 msgr2=0x7f596c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.706+0000 7f5988a6b640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f596c07af30 0x7f596c07d3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5974005fd0 tx=0x7f5974005850 comp rx=0 tx=0).stop 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.706+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 msgr2=0x7f5984100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.706+0000 7f5988a6b640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 0x7f5984100c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5978031460 tx=0x7f5978004530 comp rx=0 tx=0).stop 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.707+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 shutdown_connections 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.707+0000 7f5988a6b640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f596c07af30 0x7f596c07d3f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.707+0000 7f5988a6b640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59841033e0 0x7f5984100c60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.707+0000 7f5988a6b640 1 --2- 172.21.15.7:0/1441062602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5984102ac0 0x7f5984100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.707+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 >> 172.21.15.7:0/1441062602 conn(0x7f59840fc520 msgr2=0x7f5984108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.707+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 shutdown_connections 2024-09-21T04:41:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.707+0000 7f5988a6b640 1 -- 172.21.15.7:0/1441062602 wait complete. 2024-09-21T04:41:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.799+0000 7fc42b149640 1 -- 172.21.15.7:0/400544656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 msgr2=0x7fc424100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.799+0000 7fc42b149640 1 --2- 172.21.15.7:0/400544656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 0x7fc424100c50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc414009c20 tx=0x7fc41402a970 comp rx=0 tx=0).stop 2024-09-21T04:41:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.799+0000 7fc42b149640 1 -- 172.21.15.7:0/400544656 shutdown_connections 2024-09-21T04:41:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.799+0000 7fc42b149640 1 --2- 172.21.15.7:0/400544656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 0x7fc424100c50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.799+0000 7fc42b149640 1 --2- 172.21.15.7:0/400544656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4241067f0 0x7fc424106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.799+0000 7fc42b149640 1 -- 172.21.15.7:0/400544656 >> 172.21.15.7:0/400544656 conn(0x7fc4240fc550 msgr2=0x7fc4240fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.799+0000 7fc42b149640 1 -- 172.21.15.7:0/400544656 shutdown_connections 2024-09-21T04:41:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.799+0000 7fc42b149640 1 -- 172.21.15.7:0/400544656 wait complete. 2024-09-21T04:41:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc42b149640 1 Processor -- start 2024-09-21T04:41:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc42b149640 1 -- start start 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc42b149640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 0x7fc42419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc42b149640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4241067f0 0x7fc42419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc42b149640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc42410dca0 con 0x7fc4241007f0 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc42b149640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc42410db20 con 0x7fc4241067f0 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc428ee5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 0x7fc42419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc428ee5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 0x7fc42419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39398/0 (socket says 172.21.15.7:39398) 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc428ee5640 1 -- 172.21.15.7:0/2827692264 learned_addr learned my addr 172.21.15.7:0/2827692264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.800+0000 7fc423fff640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4241067f0 0x7fc42419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc428ee5640 1 -- 172.21.15.7:0/2827692264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4241067f0 msgr2=0x7fc42419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc428ee5640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4241067f0 0x7fc42419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc428ee5640 1 -- 172.21.15.7:0/2827692264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc424199990 con 0x7fc4241007f0 2024-09-21T04:41:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc428ee5640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 0x7fc42419f7d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00d8d0 tx=0x7fc40c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc421ffb640 1 -- 172.21.15.7:0/2827692264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc40c017070 con 0x7fc4241007f0 2024-09-21T04:41:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc421ffb640 1 -- 172.21.15.7:0/2827692264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc40c010460 con 0x7fc4241007f0 2024-09-21T04:41:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc421ffb640 1 -- 172.21.15.7:0/2827692264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc40c00f5f0 con 0x7fc4241007f0 2024-09-21T04:41:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc424199c80 con 0x7fc4241007f0 2024-09-21T04:41:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.801+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42419a210 con 0x7fc4241007f0 2024-09-21T04:41:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.802+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc424101f30 con 0x7fc4241007f0 2024-09-21T04:41:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.804+0000 7fc421ffb640 1 -- 172.21.15.7:0/2827692264 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc40c009a30 con 0x7fc4241007f0 2024-09-21T04:41:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.804+0000 7fc421ffb640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc41007af30 0x7fc41007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.804+0000 7fc421ffb640 1 -- 172.21.15.7:0/2827692264 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6070+0+0 (secure 0 0 0) 0x7fc40c09dde0 con 0x7fc4241007f0 2024-09-21T04:41:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.804+0000 7fc423fff640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc41007af30 0x7fc41007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.805+0000 7fc423fff640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc41007af30 0x7fc41007d3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc4240feaa0 tx=0x7fc41403c040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:23.809+0000 7fc421ffb640 1 -- 172.21.15.7:0/2827692264 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc40c069430 con 0x7fc4241007f0 2024-09-21T04:41:23.867 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:23 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.2.service: Deactivated successfully. 2024-09-21T04:41:23.867 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:23 smithi007 systemd[1]: Stopped Ceph osd.2 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:41:23.867 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:23 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.2.service: Consumed 51.523s CPU time. 2024-09-21T04:41:24.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.042+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc42419a4c0 con 0x7fc41007af30 2024-09-21T04:41:24.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.044+0000 7fc421ffb640 1 -- 172.21.15.7:0/2827692264 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fc42419a4c0 con 0x7fc41007af30 2024-09-21T04:41:24.047 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:41:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:41:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:41:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:41:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:41:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:41:24.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:41:24.048 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:41:24.048 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:41:24.048 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "8/23 daemons upgraded", 2024-09-21T04:41:24.048 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:41:24.048 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:41:24.048 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:41:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc41007af30 msgr2=0x7fc41007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc41007af30 0x7fc41007d3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc4240feaa0 tx=0x7fc41403c040 comp rx=0 tx=0).stop 2024-09-21T04:41:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 msgr2=0x7fc42419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 0x7fc42419f7d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00d8d0 tx=0x7fc40c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:41:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 shutdown_connections 2024-09-21T04:41:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc41007af30 0x7fc41007d3f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4241067f0 0x7fc42419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 --2- 172.21.15.7:0/2827692264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4241007f0 0x7fc42419f7d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 >> 172.21.15.7:0/2827692264 conn(0x7fc4240fc550 msgr2=0x7fc424069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 shutdown_connections 2024-09-21T04:41:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.047+0000 7fc42b149640 1 -- 172.21.15.7:0/2827692264 wait complete. 2024-09-21T04:41:24.125 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:23 smithi007 systemd[1]: Starting Ceph osd.2 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:41:24.125 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 podman[127479]: 2024-09-21 04:41:24.008020315 +0000 UTC m=+0.019166711 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:41:24.125 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:23 smithi007 ceph-mon[108238]: from='client.44175 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:24.125 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:23 smithi007 ceph-mon[108238]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:24.126 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:23 smithi007 ceph-mon[108238]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:24.126 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1806362655' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:24.126 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1441062602' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:41:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.135+0000 7faaee6d3640 1 -- 172.21.15.7:0/2405940902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae81033d0 msgr2=0x7faae810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.135+0000 7faaee6d3640 1 --2- 172.21.15.7:0/2405940902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae81033d0 0x7faae810b8c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faadc009880 tx=0x7faadc02a530 comp rx=0 tx=0).stop 2024-09-21T04:41:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.136+0000 7faaee6d3640 1 -- 172.21.15.7:0/2405940902 shutdown_connections 2024-09-21T04:41:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.136+0000 7faaee6d3640 1 --2- 172.21.15.7:0/2405940902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae81033d0 0x7faae810b8c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.136+0000 7faaee6d3640 1 --2- 172.21.15.7:0/2405940902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faae8102ab0 0x7faae8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.136+0000 7faaee6d3640 1 -- 172.21.15.7:0/2405940902 >> 172.21.15.7:0/2405940902 conn(0x7faae80fc550 msgr2=0x7faae80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.136+0000 7faaee6d3640 1 -- 172.21.15.7:0/2405940902 shutdown_connections 2024-09-21T04:41:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.136+0000 7faaee6d3640 1 -- 172.21.15.7:0/2405940902 wait complete. 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.136+0000 7faaee6d3640 1 Processor -- start 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.136+0000 7faaee6d3640 1 -- start start 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faaee6d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae8102ab0 0x7faae8100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faaee6d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faae81033d0 0x7faae8100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faaee6d3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faae810ede0 con 0x7faae8102ab0 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faaee6d3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faae810ec60 con 0x7faae81033d0 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae8102ab0 0x7faae8100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae8102ab0 0x7faae8100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39408/0 (socket says 172.21.15.7:39408) 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae7fff640 1 -- 172.21.15.7:0/2514924328 learned_addr learned my addr 172.21.15.7:0/2514924328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae77fe640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faae81033d0 0x7faae8100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae7fff640 1 -- 172.21.15.7:0/2514924328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faae81033d0 msgr2=0x7faae8100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae7fff640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faae81033d0 0x7faae8100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae7fff640 1 -- 172.21.15.7:0/2514924328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae8101190 con 0x7faae8102ab0 2024-09-21T04:41:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae77fe640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faae81033d0 0x7faae8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:41:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.137+0000 7faae7fff640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae8102ab0 0x7faae8100710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faad800d6f0 tx=0x7faad800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.138+0000 7faae57fa640 1 -- 172.21.15.7:0/2514924328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faad8017070 con 0x7faae8102ab0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.138+0000 7faae57fa640 1 -- 172.21.15.7:0/2514924328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faad800de00 con 0x7faae8102ab0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.138+0000 7faae57fa640 1 -- 172.21.15.7:0/2514924328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faad8004bb0 con 0x7faae8102ab0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.138+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae8101480 con 0x7faae8102ab0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.138+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae81a3e90 con 0x7faae8102ab0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.139+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae806a580 con 0x7faae8102ab0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.139+0000 7faae57fa640 1 -- 172.21.15.7:0/2514924328 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7faad8004e60 con 0x7faae8102ab0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.140+0000 7faae57fa640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faad007af40 0x7faad007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.140+0000 7faae77fe640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faad007af40 0x7faad007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.140+0000 7faae57fa640 1 -- 172.21.15.7:0/2514924328 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6070+0+0 (secure 0 0 0) 0x7faad809dbe0 con 0x7faae8102ab0 2024-09-21T04:41:24.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.141+0000 7faae77fe640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faad007af40 0x7faad007d400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faae81021a0 tx=0x7faadc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.145+0000 7faae57fa640 1 -- 172.21.15.7:0/2514924328 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faad8069230 con 0x7faae8102ab0 2024-09-21T04:41:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:23 smithi053 ceph-mon[80176]: from='client.44175 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:23 smithi053 ceph-mon[80176]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:23 smithi053 ceph-mon[80176]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1806362655' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:24.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1441062602' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:41:24.404 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 podman[127479]: 2024-09-21 04:41:24.123159801 +0000 UTC m=+0.134306162 container create 0aa3adb27d93eed96d71f1496218394bf2617d717f64c2e360a143fb28b896d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T04:41:24.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.441+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7faae8102e90 con 0x7faae8102ab0 2024-09-21T04:41:24.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.442+0000 7faae57fa640 1 -- 172.21.15.7:0/2514924328 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+959 (secure 0 0 0) 0x7faad806e0e0 con 0x7faae8102ab0 2024-09-21T04:41:24.445 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 31/333 objects degraded (9.309%), 9 pgs degraded 2024-09-21T04:41:24.445 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:41:24.445 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:41:24.445 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-21T04:41:24.445 INFO:teuthology.orchestra.run.smithi007.stdout: osd.2 (root=default,host=smithi007) is down 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 31/333 objects degraded (9.309%), 9 pgs degraded 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.17 is active+undersized+degraded, acting [5,1] 2024-09-21T04:41:24.446 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-09-21T04:41:24.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faad007af40 msgr2=0x7faad007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:24.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faad007af40 0x7faad007d400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faae81021a0 tx=0x7faadc031400 comp rx=0 tx=0).stop 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae8102ab0 msgr2=0x7faae8100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae8102ab0 0x7faae8100710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faad800d6f0 tx=0x7faad800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 shutdown_connections 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faad007af40 0x7faad007d400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faae81033d0 0x7faae8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 --2- 172.21.15.7:0/2514924328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae8102ab0 0x7faae8100710 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.444+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 >> 172.21.15.7:0/2514924328 conn(0x7faae80fc550 msgr2=0x7faae8108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.445+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 shutdown_connections 2024-09-21T04:41:24.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:24.445+0000 7faaee6d3640 1 -- 172.21.15.7:0/2514924328 wait complete. 2024-09-21T04:41:24.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 podman[127479]: 2024-09-21 04:41:24.401499917 +0000 UTC m=+0.412646309 container init 0aa3adb27d93eed96d71f1496218394bf2617d717f64c2e360a143fb28b896d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T04:41:24.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 podman[127479]: 2024-09-21 04:41:24.406828924 +0000 UTC m=+0.417975293 container start 0aa3adb27d93eed96d71f1496218394bf2617d717f64c2e360a143fb28b896d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T04:41:24.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 podman[127479]: 2024-09-21 04:41:24.49277228 +0000 UTC m=+0.503918642 container attach 0aa3adb27d93eed96d71f1496218394bf2617d717f64c2e360a143fb28b896d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:41:24.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:41:24.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 bash[127479]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:41:24.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:41:24.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:24 smithi007 bash[127479]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:41:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:24 smithi053 ceph-mon[80176]: pgmap v178: 65 pgs: 10 active+undersized, 3 stale+active+clean, 9 active+undersized+degraded, 43 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 31/333 objects degraded (9.309%) 2024-09-21T04:41:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:24 smithi053 ceph-mon[80176]: Health check failed: Degraded data redundancy: 31/333 objects degraded (9.309%), 9 pgs degraded (PG_DEGRADED) 2024-09-21T04:41:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:24 smithi053 ceph-mon[80176]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:24 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2514924328' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:41:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:24 smithi007 ceph-mon[108238]: pgmap v178: 65 pgs: 10 active+undersized, 3 stale+active+clean, 9 active+undersized+degraded, 43 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 31/333 objects degraded (9.309%) 2024-09-21T04:41:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:24 smithi007 ceph-mon[108238]: Health check failed: Degraded data redundancy: 31/333 objects degraded (9.309%), 9 pgs degraded (PG_DEGRADED) 2024-09-21T04:41:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:24 smithi007 ceph-mon[108238]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:24 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2514924328' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:41:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:26 smithi053 ceph-mon[80176]: pgmap v179: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:26 smithi007 ceph-mon[108238]: pgmap v179: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:27.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:41:27.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:41:27.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:41:27.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:41:27.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:41:27.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:41:27.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T04:41:27.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T04:41:27.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2024-09-21T04:41:27.695 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2024-09-21T04:41:28.195 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate[127523]: --> ceph-volume lvm activate successful for osd ID: 2 2024-09-21T04:41:28.290 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 bash[127479]: --> ceph-volume lvm activate successful for osd ID: 2 2024-09-21T04:41:28.291 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:27 smithi007 podman[127479]: 2024-09-21 04:41:27.834708724 +0000 UTC m=+3.845855102 container died 0aa3adb27d93eed96d71f1496218394bf2617d717f64c2e360a143fb28b896d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T04:41:28.863 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:28 smithi007 podman[127751]: 2024-09-21 04:41:28.505275852 +0000 UTC m=+0.663279647 container remove 0aa3adb27d93eed96d71f1496218394bf2617d717f64c2e360a143fb28b896d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T04:41:28.863 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:28 smithi007 podman[127813]: 2024-09-21 04:41:28.626669421 +0000 UTC m=+0.017127470 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:41:28.863 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:28 smithi007 podman[127813]: 2024-09-21 04:41:28.731101433 +0000 UTC m=+0.121559473 container create fb138708649cf2ea9b2a382e0eab986e3cbd9f5880c4ce543f29d3571274b789 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:41:29.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:28 smithi053 ceph-mon[80176]: pgmap v180: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:41:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:28 smithi007 ceph-mon[108238]: pgmap v180: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:41:29.194 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:28 smithi007 podman[127813]: 2024-09-21 04:41:28.984223808 +0000 UTC m=+0.374681864 container init fb138708649cf2ea9b2a382e0eab986e3cbd9f5880c4ce543f29d3571274b789 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:41:29.194 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:28 smithi007 podman[127813]: 2024-09-21 04:41:28.98808485 +0000 UTC m=+0.378542892 container start fb138708649cf2ea9b2a382e0eab986e3cbd9f5880c4ce543f29d3571274b789 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-21T04:41:29.194 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:29 smithi007 ceph-osd[127831]: -- 172.21.15.7:0/3474111437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609a820a1e0 con 0x5609a819a000 2024-09-21T04:41:29.195 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:29 smithi007 bash[127813]: fb138708649cf2ea9b2a382e0eab986e3cbd9f5880c4ce543f29d3571274b789 2024-09-21T04:41:29.195 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:29 smithi007 systemd[1]: Started Ceph osd.2 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:41:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:30 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:30 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:30 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:41:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:30 smithi053 ceph-mon[80176]: pgmap v181: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:30.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:30.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:30.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:41:30.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:30 smithi007 ceph-mon[108238]: pgmap v181: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:30.944 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:30 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[127827]: 2024-09-21T04:41:30.605+0000 7ff7059e0640 -1 Falling back to public interface 2024-09-21T04:41:33.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:32 smithi053 ceph-mon[80176]: pgmap v182: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:32 smithi007 ceph-mon[108238]: pgmap v182: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:34.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:33 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 43/333 objects degraded (12.913%), 13 pgs degraded (PG_DEGRADED) 2024-09-21T04:41:34.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:33 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 43/333 objects degraded (12.913%), 13 pgs degraded (PG_DEGRADED) 2024-09-21T04:41:35.148 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:34 smithi007 ceph-mon[108238]: pgmap v183: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:35.148 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:34 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:35.148 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:34 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:35.148 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:34 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:35.148 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:34 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:34 smithi053 ceph-mon[80176]: pgmap v183: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-09-21T04:41:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:34 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:34 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:34 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:34 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:35.944 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:35 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[127827]: 2024-09-21T04:41:35.857+0000 7ff7059e0640 -1 osd.2 56 log_to_monitors true 2024-09-21T04:41:37.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:36 smithi053 ceph-mon[80176]: pgmap v184: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 43/333 objects degraded (12.913%) 2024-09-21T04:41:37.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:36 smithi053 ceph-mon[80176]: from='osd.2 [v2:172.21.15.7:6818/2399513129,v1:172.21.15.7:6819/2399513129]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T04:41:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:36 smithi007 ceph-mon[108238]: pgmap v184: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 43/333 objects degraded (12.913%) 2024-09-21T04:41:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:36 smithi007 ceph-mon[108238]: from='osd.2 [v2:172.21.15.7:6818/2399513129,v1:172.21.15.7:6819/2399513129]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T04:41:37.194 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 04:41:36 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[127827]: 2024-09-21T04:41:36.879+0000 7ff6f9782640 -1 osd.2 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:41:37.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:37 smithi007 ceph-mon[108238]: from='osd.2 [v2:172.21.15.7:6818/2399513129,v1:172.21.15.7:6819/2399513129]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T04:41:37.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:37 smithi007 ceph-mon[108238]: osdmap e59: 6 total, 5 up, 6 in 2024-09-21T04:41:37.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:37 smithi007 ceph-mon[108238]: from='osd.2 [v2:172.21.15.7:6818/2399513129,v1:172.21.15.7:6819/2399513129]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]} : dispatch 2024-09-21T04:41:38.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:37 smithi053 ceph-mon[80176]: from='osd.2 [v2:172.21.15.7:6818/2399513129,v1:172.21.15.7:6819/2399513129]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T04:41:38.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:37 smithi053 ceph-mon[80176]: osdmap e59: 6 total, 5 up, 6 in 2024-09-21T04:41:38.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:37 smithi053 ceph-mon[80176]: from='osd.2 [v2:172.21.15.7:6818/2399513129,v1:172.21.15.7:6819/2399513129]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]} : dispatch 2024-09-21T04:41:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: pgmap v186: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 43/333 objects degraded (12.913%) 2024-09-21T04:41:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:41:39.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: osd.2 [v2:172.21.15.7:6818/2399513129,v1:172.21.15.7:6819/2399513129] boot 2024-09-21T04:41:39.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: osdmap e60: 6 total, 6 up, 6 in 2024-09-21T04:41:39.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T04:41:39.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:39.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:39.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:41:39.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:41:39.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:39.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:41:39.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:39.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:39.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:39.196 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-21T04:41:39.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: pgmap v186: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 43/333 objects degraded (12.913%) 2024-09-21T04:41:39.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:41:39.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: osd.2 [v2:172.21.15.7:6818/2399513129,v1:172.21.15.7:6819/2399513129] boot 2024-09-21T04:41:39.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: osdmap e60: 6 total, 6 up, 6 in 2024-09-21T04:41:39.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T04:41:39.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:39.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:39.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:41:39.198 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:41:39.198 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:39.198 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:41:39.198 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:39.198 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:39.198 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:39.199 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-21T04:41:40.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:39 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-21T04:41:40.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:39 smithi053 ceph-mon[80176]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-21T04:41:40.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:39 smithi053 ceph-mon[80176]: osdmap e61: 6 total, 6 up, 6 in 2024-09-21T04:41:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:39 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-21T04:41:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:39 smithi007 ceph-mon[108238]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-21T04:41:40.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:39 smithi007 ceph-mon[108238]: osdmap e61: 6 total, 6 up, 6 in 2024-09-21T04:41:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:40 smithi053 ceph-mon[80176]: pgmap v189: 65 pgs: 13 peering, 7 active+undersized, 7 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 24/333 objects degraded (7.207%) 2024-09-21T04:41:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:40 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 24/333 objects degraded (7.207%), 7 pgs degraded (PG_DEGRADED) 2024-09-21T04:41:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:40 smithi007 ceph-mon[108238]: pgmap v189: 65 pgs: 13 peering, 7 active+undersized, 7 active+undersized+degraded, 38 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 24/333 objects degraded (7.207%) 2024-09-21T04:41:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:40 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 24/333 objects degraded (7.207%), 7 pgs degraded (PG_DEGRADED) 2024-09-21T04:41:43.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:42 smithi053 ceph-mon[80176]: pgmap v190: 65 pgs: 13 peering, 4 active+undersized, 5 active+undersized+degraded, 43 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 18/333 objects degraded (5.405%) 2024-09-21T04:41:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:42 smithi007 ceph-mon[108238]: pgmap v190: 65 pgs: 13 peering, 4 active+undersized, 5 active+undersized+degraded, 43 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 18/333 objects degraded (5.405%) 2024-09-21T04:41:44.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:44.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:41:44.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:44 smithi053 ceph-mon[80176]: pgmap v191: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 2 op/s 2024-09-21T04:41:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:41:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:44 smithi007 ceph-mon[108238]: pgmap v191: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 2 op/s 2024-09-21T04:41:45.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:45 smithi053 ceph-mon[80176]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/333 objects degraded (5.405%), 5 pgs degraded) 2024-09-21T04:41:45.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:45 smithi007 ceph-mon[108238]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/333 objects degraded (5.405%), 5 pgs degraded) 2024-09-21T04:41:46.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:46 smithi007 ceph-mon[108238]: pgmap v192: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-09-21T04:41:46.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:46 smithi053 ceph-mon[80176]: pgmap v192: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-09-21T04:41:49.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:48 smithi053 ceph-mon[80176]: pgmap v193: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-09-21T04:41:49.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:48 smithi007 ceph-mon[108238]: pgmap v193: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-09-21T04:41:51.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:50 smithi053 ceph-mon[80176]: pgmap v194: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-21T04:41:51.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:50 smithi007 ceph-mon[108238]: pgmap v194: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-21T04:41:53.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:52 smithi053 ceph-mon[80176]: pgmap v195: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-09-21T04:41:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:52 smithi007 ceph-mon[108238]: pgmap v195: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-09-21T04:41:54.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:53 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-21T04:41:54.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:53 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:53 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-21T04:41:54.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:53 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.545+0000 7fc7f9db2640 1 -- 172.21.15.7:0/1714666231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f4102ac0 msgr2=0x7fc7f4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.545+0000 7fc7f9db2640 1 --2- 172.21.15.7:0/1714666231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f4102ac0 0x7fc7f4102ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009880 tx=0x7fc7e402a530 comp rx=0 tx=0).stop 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.545+0000 7fc7f9db2640 1 -- 172.21.15.7:0/1714666231 shutdown_connections 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.545+0000 7fc7f9db2640 1 --2- 172.21.15.7:0/1714666231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f41033e0 0x7fc7f410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.545+0000 7fc7f9db2640 1 --2- 172.21.15.7:0/1714666231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f4102ac0 0x7fc7f4102ea0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.545+0000 7fc7f9db2640 1 -- 172.21.15.7:0/1714666231 >> 172.21.15.7:0/1714666231 conn(0x7fc7f40fc520 msgr2=0x7fc7f40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.545+0000 7fc7f9db2640 1 -- 172.21.15.7:0/1714666231 shutdown_connections 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.545+0000 7fc7f9db2640 1 -- 172.21.15.7:0/1714666231 wait complete. 2024-09-21T04:41:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f9db2640 1 Processor -- start 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f9db2640 1 -- start start 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f9db2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f4102ac0 0x7fc7f419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f9db2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f41033e0 0x7fc7f419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f9db2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f410edf0 con 0x7fc7f41033e0 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f9db2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f410ec70 con 0x7fc7f4102ac0 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f41033e0 0x7fc7f419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f41033e0 0x7fc7f419db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36978/0 (socket says 172.21.15.7:36978) 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f4102ac0 0x7fc7f419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.546+0000 7fc7f2ffd640 1 -- 172.21.15.7:0/3527158185 learned_addr learned my addr 172.21.15.7:0/3527158185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f2ffd640 1 -- 172.21.15.7:0/3527158185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f4102ac0 msgr2=0x7fc7f419d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f2ffd640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f4102ac0 0x7fc7f419d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f2ffd640 1 -- 172.21.15.7:0/3527158185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f4197780 con 0x7fc7f41033e0 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f37fe640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f4102ac0 0x7fc7f419d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:41:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f2ffd640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f41033e0 0x7fc7f419db40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800d6d0 tx=0x7fc7e800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:54.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f0ff9640 1 -- 172.21.15.7:0/3527158185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc7e8017070 con 0x7fc7f41033e0 2024-09-21T04:41:54.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f0ff9640 1 -- 172.21.15.7:0/3527158185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc7e80054e0 con 0x7fc7f41033e0 2024-09-21T04:41:54.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f0ff9640 1 -- 172.21.15.7:0/3527158185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc7e8004ae0 con 0x7fc7f41033e0 2024-09-21T04:41:54.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f4197a70 con 0x7fc7f41033e0 2024-09-21T04:41:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.547+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f4198000 con 0x7fc7f41033e0 2024-09-21T04:41:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.549+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f406aa90 con 0x7fc7f41033e0 2024-09-21T04:41:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.549+0000 7fc7f0ff9640 1 -- 172.21.15.7:0/3527158185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc7e8004d90 con 0x7fc7f41033e0 2024-09-21T04:41:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.550+0000 7fc7f0ff9640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc7dc07af80 0x7fc7dc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.550+0000 7fc7f37fe640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc7dc07af80 0x7fc7dc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.550+0000 7fc7f0ff9640 1 -- 172.21.15.7:0/3527158185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7fc7e809da50 con 0x7fc7f41033e0 2024-09-21T04:41:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.550+0000 7fc7f37fe640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc7dc07af80 0x7fc7dc07d440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40096c0 tx=0x7fc7e4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.554+0000 7fc7f0ff9640 1 -- 172.21.15.7:0/3527158185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7e8069120 con 0x7fc7f41033e0 2024-09-21T04:41:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.760+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc7f41982b0 con 0x7fc7dc07af80 2024-09-21T04:41:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.762+0000 7fc7f0ff9640 1 -- 172.21.15.7:0/3527158185 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fc7f41982b0 con 0x7fc7dc07af80 2024-09-21T04:41:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc7dc07af80 msgr2=0x7fc7dc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc7dc07af80 0x7fc7dc07d440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40096c0 tx=0x7fc7e4031400 comp rx=0 tx=0).stop 2024-09-21T04:41:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f41033e0 msgr2=0x7fc7f419db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f41033e0 0x7fc7f419db40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800d6d0 tx=0x7fc7e800db90 comp rx=0 tx=0).stop 2024-09-21T04:41:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 shutdown_connections 2024-09-21T04:41:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc7dc07af80 0x7fc7dc07d440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7f41033e0 0x7fc7f419db40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 --2- 172.21.15.7:0/3527158185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7f4102ac0 0x7fc7f419d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 >> 172.21.15.7:0/3527158185 conn(0x7fc7f40fc520 msgr2=0x7fc7f410af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 shutdown_connections 2024-09-21T04:41:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.764+0000 7fc7f9db2640 1 -- 172.21.15.7:0/3527158185 wait complete. 2024-09-21T04:41:54.776 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:41:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.850+0000 7f95615ac640 1 -- 172.21.15.7:0/816440557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 msgr2=0x7f955c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.850+0000 7f95615ac640 1 --2- 172.21.15.7:0/816440557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 0x7f955c101d60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9544009880 tx=0x7f954402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.850+0000 7f95615ac640 1 -- 172.21.15.7:0/816440557 shutdown_connections 2024-09-21T04:41:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.850+0000 7f95615ac640 1 --2- 172.21.15.7:0/816440557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 0x7f955c101d60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.850+0000 7f95615ac640 1 --2- 172.21.15.7:0/816440557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f955c107900 0x7f955c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.850+0000 7f95615ac640 1 -- 172.21.15.7:0/816440557 >> 172.21.15.7:0/816440557 conn(0x7f955c0fd600 msgr2=0x7f955c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.850+0000 7f95615ac640 1 -- 172.21.15.7:0/816440557 shutdown_connections 2024-09-21T04:41:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.850+0000 7f95615ac640 1 -- 172.21.15.7:0/816440557 wait complete. 2024-09-21T04:41:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f95615ac640 1 Processor -- start 2024-09-21T04:41:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f95615ac640 1 -- start start 2024-09-21T04:41:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f95615ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 0x7f955c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f95615ac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f955c107900 0x7f955c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f95615ac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f955c10edd0 con 0x7f955c101900 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f95615ac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f955c10ec50 con 0x7f955c107900 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f955affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 0x7f955c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f955affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 0x7f955c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37004/0 (socket says 172.21.15.7:37004) 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f955affd640 1 -- 172.21.15.7:0/3465752462 learned_addr learned my addr 172.21.15.7:0/3465752462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f955a7fc640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f955c107900 0x7f955c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f955affd640 1 -- 172.21.15.7:0/3465752462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f955c107900 msgr2=0x7f955c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f955affd640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f955c107900 0x7f955c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.851+0000 7f955affd640 1 -- 172.21.15.7:0/3465752462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c074f70 con 0x7f955c101900 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.852+0000 7f955affd640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 0x7f955c0744f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f954c00e970 tx=0x7f954c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.852+0000 7f9553fff640 1 -- 172.21.15.7:0/3465752462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f954c00cac0 con 0x7f955c101900 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.852+0000 7f9553fff640 1 -- 172.21.15.7:0/3465752462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f954c0054e0 con 0x7f955c101900 2024-09-21T04:41:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.852+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c075260 con 0x7f955c101900 2024-09-21T04:41:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.852+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c1a3e60 con 0x7f955c101900 2024-09-21T04:41:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.852+0000 7f9553fff640 1 -- 172.21.15.7:0/3465752462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f954c010430 con 0x7f955c101900 2024-09-21T04:41:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.853+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953c005290 con 0x7f955c101900 2024-09-21T04:41:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.854+0000 7f9553fff640 1 -- 172.21.15.7:0/3465752462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f954c00cc60 con 0x7f955c101900 2024-09-21T04:41:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.854+0000 7f9553fff640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f954807af80 0x7f954807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.855+0000 7f955a7fc640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f954807af80 0x7f954807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:54.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.855+0000 7f9553fff640 1 -- 172.21.15.7:0/3465752462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7f954c09e4b0 con 0x7f955c101900 2024-09-21T04:41:54.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.855+0000 7f955a7fc640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f954807af80 0x7f954807d440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f955c075f80 tx=0x7f9544031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:54.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:54.859+0000 7f9553fff640 1 -- 172.21.15.7:0/3465752462 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f954c069b80 con 0x7f955c101900 2024-09-21T04:41:55.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.068+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f953c000bc0 con 0x7f954807af80 2024-09-21T04:41:55.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.069+0000 7f9553fff640 1 -- 172.21.15.7:0/3465752462 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f953c000bc0 con 0x7f954807af80 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.071+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f954807af80 msgr2=0x7f954807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.071+0000 7f95615ac640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f954807af80 0x7f954807d440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f955c075f80 tx=0x7f9544031400 comp rx=0 tx=0).stop 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.071+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 msgr2=0x7f955c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.071+0000 7f95615ac640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 0x7f955c0744f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f954c00e970 tx=0x7f954c00ed70 comp rx=0 tx=0).stop 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.072+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 shutdown_connections 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.072+0000 7f95615ac640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f954807af80 0x7f954807d440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.072+0000 7f95615ac640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f955c107900 0x7f955c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.072+0000 7f95615ac640 1 --2- 172.21.15.7:0/3465752462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f955c101900 0x7f955c0744f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.072+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 >> 172.21.15.7:0/3465752462 conn(0x7f955c0fd600 msgr2=0x7f955c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:55.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.072+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 shutdown_connections 2024-09-21T04:41:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.072+0000 7f95615ac640 1 -- 172.21.15.7:0/3465752462 wait complete. 2024-09-21T04:41:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.156+0000 7f2f66575640 1 -- 172.21.15.7:0/1168390890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60103440 msgr2=0x7f2f6010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.156+0000 7f2f66575640 1 --2- 172.21.15.7:0/1168390890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60103440 0x7f2f6010b930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098b0 tx=0x7f2f5402a560 comp rx=0 tx=0).stop 2024-09-21T04:41:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.156+0000 7f2f66575640 1 -- 172.21.15.7:0/1168390890 shutdown_connections 2024-09-21T04:41:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.156+0000 7f2f66575640 1 --2- 172.21.15.7:0/1168390890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60103440 0x7f2f6010b930 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.156+0000 7f2f66575640 1 --2- 172.21.15.7:0/1168390890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f60102b20 0x7f2f60102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.156+0000 7f2f66575640 1 -- 172.21.15.7:0/1168390890 >> 172.21.15.7:0/1168390890 conn(0x7f2f600fc520 msgr2=0x7f2f600fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:55.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:54 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-21T04:41:55.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:54 smithi007 ceph-mon[108238]: Upgrade: osd.3 is safe to restart 2024-09-21T04:41:55.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:54 smithi007 ceph-mon[108238]: pgmap v196: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-09-21T04:41:55.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:54 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:55.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:54 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T04:41:55.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:54 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.156+0000 7f2f66575640 1 -- 172.21.15.7:0/1168390890 shutdown_connections 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.156+0000 7f2f66575640 1 -- 172.21.15.7:0/1168390890 wait complete. 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f66575640 1 Processor -- start 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f66575640 1 -- start start 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f66575640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60102b20 0x7f2f60100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f66575640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f60103440 0x7f2f60100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f66575640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6010edf0 con 0x7f2f60102b20 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f66575640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6010ec70 con 0x7f2f60103440 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f5ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60102b20 0x7f2f60100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f5ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60102b20 0x7f2f60100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37020/0 (socket says 172.21.15.7:37020) 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f5ffff640 1 -- 172.21.15.7:0/4195055690 learned_addr learned my addr 172.21.15.7:0/4195055690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.157+0000 7f2f5f7fe640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f60103440 0x7f2f60100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f5ffff640 1 -- 172.21.15.7:0/4195055690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f60103440 msgr2=0x7f2f60100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f5ffff640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f60103440 0x7f2f60100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f5ffff640 1 -- 172.21.15.7:0/4195055690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f60101200 con 0x7f2f60102b20 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f5ffff640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60102b20 0x7f2f60100780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000e970 tx=0x7f2f5000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f5d7fa640 1 -- 172.21.15.7:0/4195055690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2f5000cac0 con 0x7f2f60102b20 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f5d7fa640 1 -- 172.21.15.7:0/4195055690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2f500054e0 con 0x7f2f60102b20 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f601014f0 con 0x7f2f60102b20 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f5d7fa640 1 -- 172.21.15.7:0/4195055690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2f50010430 con 0x7f2f60102b20 2024-09-21T04:41:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.158+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f601a3f20 con 0x7f2f60102b20 2024-09-21T04:41:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.159+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6006aa30 con 0x7f2f60102b20 2024-09-21T04:41:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.160+0000 7f2f5d7fa640 1 -- 172.21.15.7:0/4195055690 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2f50004f00 con 0x7f2f60102b20 2024-09-21T04:41:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.160+0000 7f2f5d7fa640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f4807af30 0x7f2f4807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.161+0000 7f2f5d7fa640 1 -- 172.21.15.7:0/4195055690 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7f2f5009e6f0 con 0x7f2f60102b20 2024-09-21T04:41:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.161+0000 7f2f5f7fe640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f4807af30 0x7f2f4807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.161+0000 7f2f5f7fe640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f4807af30 0x7f2f4807d3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2f54000c00 tx=0x7f2f54031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.165+0000 7f2f5d7fa640 1 -- 172.21.15.7:0/4195055690 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f50069d40 con 0x7f2f60102b20 2024-09-21T04:41:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:54 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-21T04:41:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:54 smithi053 ceph-mon[80176]: Upgrade: osd.3 is safe to restart 2024-09-21T04:41:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:54 smithi053 ceph-mon[80176]: pgmap v196: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s 2024-09-21T04:41:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:54 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:54 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T04:41:55.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:54 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:41:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.362+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2f60062630 con 0x7f2f4807af30 2024-09-21T04:41:55.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.372+0000 7f2f5d7fa640 1 -- 172.21.15.7:0/4195055690 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f2f60062630 con 0x7f2f4807af30 2024-09-21T04:41:55.375 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:41:55.375 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (3m) 21s ago 14m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:41:55.375 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (7m) 21s ago 15m 9013k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:41:55.375 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (6m) 2m ago 14m 8744k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (2m) 21s ago 15m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (2m) 2m ago 14m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (3m) 21s ago 14m 77.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (9m) 21s ago 9m 144M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (9m) 21s ago 9m 89.8M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (9m) 2m ago 9m 26.0M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (9m) 2m ago 9m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (6m) 21s ago 16m 528M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (5m) 2m ago 14m 455M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (2m) 21s ago 16m 43.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (2m) 2m ago 14m 25.9M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (4m) 21s ago 14m 10.7M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (4m) 2m ago 14m 10.7M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:41:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (102s) 21s ago 12m 86.1M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:41:55.377 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (64s) 21s ago 12m 90.7M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:41:55.377 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (26s) 21s ago 11m 14.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:41:55.377 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (11m) 2m ago 11m 243M 2241M 19.1.1-373-g4e45e6b4 164465d2334a b545d5073061 2024-09-21T04:41:55.377 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (11m) 2m ago 11m 212M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:41:55.377 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (10m) 2m ago 10m 184M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:41:55.377 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (4m) 21s ago 14m 56.1M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:41:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f4807af30 msgr2=0x7f2f4807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f4807af30 0x7f2f4807d3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2f54000c00 tx=0x7f2f54031400 comp rx=0 tx=0).stop 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60102b20 msgr2=0x7f2f60100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60102b20 0x7f2f60100780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000e970 tx=0x7f2f5000ed70 comp rx=0 tx=0).stop 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 shutdown_connections 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f4807af30 0x7f2f4807d3f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f60103440 0x7f2f60100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 --2- 172.21.15.7:0/4195055690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f60102b20 0x7f2f60100780 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.374+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 >> 172.21.15.7:0/4195055690 conn(0x7f2f600fc520 msgr2=0x7f2f60103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.375+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 shutdown_connections 2024-09-21T04:41:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.375+0000 7f2f66575640 1 -- 172.21.15.7:0/4195055690 wait complete. 2024-09-21T04:41:55.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 -- 172.21.15.7:0/2421820998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 msgr2=0x7f7d0410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 --2- 172.21.15.7:0/2421820998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 0x7f7d0410b8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009880 tx=0x7f7cf802a530 comp rx=0 tx=0).stop 2024-09-21T04:41:55.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 -- 172.21.15.7:0/2421820998 shutdown_connections 2024-09-21T04:41:55.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 --2- 172.21.15.7:0/2421820998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 0x7f7d0410b8d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 --2- 172.21.15.7:0/2421820998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d04102ac0 0x7f7d04102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 -- 172.21.15.7:0/2421820998 >> 172.21.15.7:0/2421820998 conn(0x7f7d040fc520 msgr2=0x7f7d040fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:55.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 -- 172.21.15.7:0/2421820998 shutdown_connections 2024-09-21T04:41:55.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 -- 172.21.15.7:0/2421820998 wait complete. 2024-09-21T04:41:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 Processor -- start 2024-09-21T04:41:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.459+0000 7f7d0b182640 1 -- start start 2024-09-21T04:41:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d0b182640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d0b182640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 0x7f7d0419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d0b182640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410edd0 con 0x7f7d041033e0 2024-09-21T04:41:55.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d0b182640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410ec50 con 0x7f7d04102ac0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 0x7f7d0419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 0x7f7d0419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37034/0 (socket says 172.21.15.7:37034) 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d03fff640 1 -- 172.21.15.7:0/2107274767 learned_addr learned my addr 172.21.15.7:0/2107274767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d08f1e640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d03fff640 1 -- 172.21.15.7:0/2107274767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d04102ac0 msgr2=0x7f7d0419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d03fff640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d03fff640 1 -- 172.21.15.7:0/2107274767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d041977c0 con 0x7f7d041033e0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d08f1e640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.460+0000 7f7d03fff640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 0x7f7d0419db80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8031460 tx=0x7f7cf8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.461+0000 7f7d01ffb640 1 -- 172.21.15.7:0/2107274767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cf803f070 con 0x7f7d041033e0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.461+0000 7f7d01ffb640 1 -- 172.21.15.7:0/2107274767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cf80035f0 con 0x7f7d041033e0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.461+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d04197a50 con 0x7f7d041033e0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.461+0000 7f7d01ffb640 1 -- 172.21.15.7:0/2107274767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cf8035030 con 0x7f7d041033e0 2024-09-21T04:41:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.461+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d04197f80 con 0x7f7d041033e0 2024-09-21T04:41:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.462+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0406aa90 con 0x7f7d041033e0 2024-09-21T04:41:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.463+0000 7f7d01ffb640 1 -- 172.21.15.7:0/2107274767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7cf8042070 con 0x7f7d041033e0 2024-09-21T04:41:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.463+0000 7f7d01ffb640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cec07af30 0x7f7cec07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.463+0000 7f7d01ffb640 1 -- 172.21.15.7:0/2107274767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7f7cf80be960 con 0x7f7d041033e0 2024-09-21T04:41:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.463+0000 7f7d08f1e640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cec07af30 0x7f7cec07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.464+0000 7f7d08f1e640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cec07af30 0x7f7cec07d3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4005fd0 tx=0x7f7cf400b5f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.467+0000 7f7d01ffb640 1 -- 172.21.15.7:0/2107274767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7cf8089fb0 con 0x7f7d041033e0 2024-09-21T04:41:55.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.731+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7d04105f60 con 0x7f7d041033e0 2024-09-21T04:41:55.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.732+0000 7f7d01ffb640 1 -- 172.21.15.7:0/2107274767 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7f7cf808ee60 con 0x7f7d041033e0 2024-09-21T04:41:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:41:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:41:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:41:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:41:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:41:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:41:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 3, 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 7, 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 7 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:41:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:41:55.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cec07af30 msgr2=0x7f7cec07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cec07af30 0x7f7cec07d3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4005fd0 tx=0x7f7cf400b5f0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 msgr2=0x7f7d0419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 0x7f7d0419db80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8031460 tx=0x7f7cf8004530 comp rx=0 tx=0).stop 2024-09-21T04:41:55.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 shutdown_connections 2024-09-21T04:41:55.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cec07af30 0x7f7cec07d3f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d041033e0 0x7f7d0419db80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 --2- 172.21.15.7:0/2107274767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 >> 172.21.15.7:0/2107274767 conn(0x7f7d040fc520 msgr2=0x7f7d04108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:55.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.734+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 shutdown_connections 2024-09-21T04:41:55.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.735+0000 7f7d0b182640 1 -- 172.21.15.7:0/2107274767 wait complete. 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.819+0000 7fb5c2054640 1 -- 172.21.15.7:0/440161403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 msgr2=0x7fb5bc102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.819+0000 7fb5c2054640 1 --2- 172.21.15.7:0/440161403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 0x7fb5bc102f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0098b0 tx=0x7fb5ac02a560 comp rx=0 tx=0).stop 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.819+0000 7fb5c2054640 1 -- 172.21.15.7:0/440161403 shutdown_connections 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.819+0000 7fb5c2054640 1 --2- 172.21.15.7:0/440161403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5bc103440 0x7fb5bc10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.819+0000 7fb5c2054640 1 --2- 172.21.15.7:0/440161403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 0x7fb5bc102f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.819+0000 7fb5c2054640 1 -- 172.21.15.7:0/440161403 >> 172.21.15.7:0/440161403 conn(0x7fb5bc0fc520 msgr2=0x7fb5bc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.819+0000 7fb5c2054640 1 -- 172.21.15.7:0/440161403 shutdown_connections 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.819+0000 7fb5c2054640 1 -- 172.21.15.7:0/440161403 wait complete. 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.820+0000 7fb5c2054640 1 Processor -- start 2024-09-21T04:41:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.820+0000 7fb5c2054640 1 -- start start 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.820+0000 7fb5c2054640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 0x7fb5bc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.820+0000 7fb5c2054640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5bc103440 0x7fb5bc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.820+0000 7fb5c2054640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10edf0 con 0x7fb5bc102b20 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.820+0000 7fb5c2054640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc10ec70 con 0x7fb5bc103440 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 0x7fb5bc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 0x7fb5bc19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37046/0 (socket says 172.21.15.7:37046) 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5bb7fe640 1 -- 172.21.15.7:0/500434935 learned_addr learned my addr 172.21.15.7:0/500434935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5baffd640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5bc103440 0x7fb5bc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5bb7fe640 1 -- 172.21.15.7:0/500434935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5bc103440 msgr2=0x7fb5bc19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5bb7fe640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5bc103440 0x7fb5bc19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5bb7fe640 1 -- 172.21.15.7:0/500434935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc197820 con 0x7fb5bc102b20 2024-09-21T04:41:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5bb7fe640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 0x7fb5bc19d6a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac031460 tx=0x7fb5ac0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5b8ff9640 1 -- 172.21.15.7:0/500434935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb5ac03f070 con 0x7fb5bc102b20 2024-09-21T04:41:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5b8ff9640 1 -- 172.21.15.7:0/500434935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb5ac0035f0 con 0x7fb5bc102b20 2024-09-21T04:41:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc197ab0 con 0x7fb5bc102b20 2024-09-21T04:41:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5b8ff9640 1 -- 172.21.15.7:0/500434935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb5ac035030 con 0x7fb5bc102b20 2024-09-21T04:41:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.821+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc197fe0 con 0x7fb5bc102b20 2024-09-21T04:41:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.822+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc06aa30 con 0x7fb5bc102b20 2024-09-21T04:41:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.823+0000 7fb5b8ff9640 1 -- 172.21.15.7:0/500434935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb5ac042070 con 0x7fb5bc102b20 2024-09-21T04:41:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.824+0000 7fb5b8ff9640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5a407af80 0x7fb5a407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.824+0000 7fb5b8ff9640 1 -- 172.21.15.7:0/500434935 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7fb5ac0be870 con 0x7fb5bc102b20 2024-09-21T04:41:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.824+0000 7fb5baffd640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5a407af80 0x7fb5a407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.824+0000 7fb5baffd640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5a407af80 0x7fb5a407d440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0005fd0 tx=0x7fb5b0005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:55.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:55.828+0000 7fb5b8ff9640 1 -- 172.21.15.7:0/500434935 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5ac089f40 con 0x7fb5bc102b20 2024-09-21T04:41:56.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.056+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb5bc105ff0 con 0x7fb5bc102b20 2024-09-21T04:41:56.060 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:55 smithi007 ceph-mon[108238]: Upgrade: Updating osd.3 2024-09-21T04:41:56.060 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:55 smithi007 ceph-mon[108238]: Deploying daemon osd.3 on smithi053 2024-09-21T04:41:56.060 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:55 smithi007 ceph-mon[108238]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:56.060 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:55 smithi007 ceph-mon[108238]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:56.060 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:55 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2107274767' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:56.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.057+0000 7fb5b8ff9640 1 -- 172.21.15.7:0/500434935 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2038 (secure 0 0 0) 0x7fb5ac08edf0 con 0x7fb5bc102b20 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:41:56.061 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 44 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14578,1=24327} 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:41:56.062 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:41:56.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:41:56.064 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:41:56.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5a407af80 msgr2=0x7fb5a407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5a407af80 0x7fb5a407d440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0005fd0 tx=0x7fb5b0005850 comp rx=0 tx=0).stop 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 msgr2=0x7fb5bc19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 0x7fb5bc19d6a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac031460 tx=0x7fb5ac0045c0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 shutdown_connections 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5a407af80 0x7fb5a407d440 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5bc103440 0x7fb5bc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 --2- 172.21.15.7:0/500434935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5bc102b20 0x7fb5bc19d6a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.059+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 >> 172.21.15.7:0/500434935 conn(0x7fb5bc0fc520 msgr2=0x7fb5bc103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.060+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 shutdown_connections 2024-09-21T04:41:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.060+0000 7fb5c2054640 1 -- 172.21.15.7:0/500434935 wait complete. 2024-09-21T04:41:56.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:55 smithi053 ceph-mon[80176]: Upgrade: Updating osd.3 2024-09-21T04:41:56.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:55 smithi053 ceph-mon[80176]: Deploying daemon osd.3 on smithi053 2024-09-21T04:41:56.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:55 smithi053 ceph-mon[80176]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:56.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:55 smithi053 ceph-mon[80176]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:56.076 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:55 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2107274767' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:41:56.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.145+0000 7fd1c803d640 1 -- 172.21.15.7:0/3031812475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c0102b80 msgr2=0x7fd1c0102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.145+0000 7fd1c803d640 1 --2- 172.21.15.7:0/3031812475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c0102b80 0x7fd1c0102f60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009880 tx=0x7fd1bc02a530 comp rx=0 tx=0).stop 2024-09-21T04:41:56.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.145+0000 7fd1c803d640 1 -- 172.21.15.7:0/3031812475 shutdown_connections 2024-09-21T04:41:56.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.145+0000 7fd1c803d640 1 --2- 172.21.15.7:0/3031812475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c01034a0 0x7fd1c010b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.145+0000 7fd1c803d640 1 --2- 172.21.15.7:0/3031812475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c0102b80 0x7fd1c0102f60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.145+0000 7fd1c803d640 1 -- 172.21.15.7:0/3031812475 >> 172.21.15.7:0/3031812475 conn(0x7fd1c00fc520 msgr2=0x7fd1c00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:56.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.145+0000 7fd1c803d640 1 -- 172.21.15.7:0/3031812475 shutdown_connections 2024-09-21T04:41:56.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.145+0000 7fd1c803d640 1 -- 172.21.15.7:0/3031812475 wait complete. 2024-09-21T04:41:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c803d640 1 Processor -- start 2024-09-21T04:41:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c803d640 1 -- start start 2024-09-21T04:41:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c803d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c0102b80 0x7fd1c019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c803d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c01034a0 0x7fd1c019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c803d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c010edf0 con 0x7fd1c01034a0 2024-09-21T04:41:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c803d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c010ec70 con 0x7fd1c0102b80 2024-09-21T04:41:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c55d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c01034a0 0x7fd1c019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c55d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c01034a0 0x7fd1c019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37058/0 (socket says 172.21.15.7:37058) 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c55d8640 1 -- 172.21.15.7:0/3689503334 learned_addr learned my addr 172.21.15.7:0/3689503334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.146+0000 7fd1c5dd9640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c0102b80 0x7fd1c019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1c55d8640 1 -- 172.21.15.7:0/3689503334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c0102b80 msgr2=0x7fd1c019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1c55d8640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c0102b80 0x7fd1c019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1c55d8640 1 -- 172.21.15.7:0/3689503334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c0199a10 con 0x7fd1c01034a0 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1c55d8640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c01034a0 0x7fd1c019fd90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000e970 tx=0x7fd1b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1b6ffd640 1 -- 172.21.15.7:0/3689503334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd1b000cac0 con 0x7fd1c01034a0 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c0199d00 con 0x7fd1c01034a0 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1b6ffd640 1 -- 172.21.15.7:0/3689503334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd1b00054e0 con 0x7fd1c01034a0 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1b6ffd640 1 -- 172.21.15.7:0/3689503334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd1b0010430 con 0x7fd1c01034a0 2024-09-21T04:41:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.147+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c019a290 con 0x7fd1c01034a0 2024-09-21T04:41:56.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.148+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c006a9d0 con 0x7fd1c01034a0 2024-09-21T04:41:56.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.149+0000 7fd1b6ffd640 1 -- 172.21.15.7:0/3689503334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd1b0004ec0 con 0x7fd1c01034a0 2024-09-21T04:41:56.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.149+0000 7fd1b6ffd640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1ac07af30 0x7fd1ac07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:56.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.149+0000 7fd1b6ffd640 1 -- 172.21.15.7:0/3689503334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7fd1b009e670 con 0x7fd1c01034a0 2024-09-21T04:41:56.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.149+0000 7fd1c5dd9640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1ac07af30 0x7fd1ac07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:56.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.150+0000 7fd1c5dd9640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1ac07af30 0x7fd1ac07d3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc000c00 tx=0x7fd1bc004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:56.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.154+0000 7fd1b6ffd640 1 -- 172.21.15.7:0/3689503334 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1b0069cc0 con 0x7fd1c01034a0 2024-09-21T04:41:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.361+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd1c019af40 con 0x7fd1ac07af30 2024-09-21T04:41:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.362+0000 7fd1b6ffd640 1 -- 172.21.15.7:0/3689503334 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fd1c019af40 con 0x7fd1ac07af30 2024-09-21T04:41:56.365 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:41:56.365 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:41:56.365 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:41:56.365 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:41:56.365 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:41:56.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:41:56.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:41:56.366 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:41:56.366 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:41:56.366 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "9/23 daemons upgraded", 2024-09-21T04:41:56.366 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:41:56.366 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:41:56.366 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:41:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1ac07af30 msgr2=0x7fd1ac07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1ac07af30 0x7fd1ac07d3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc000c00 tx=0x7fd1bc004f30 comp rx=0 tx=0).stop 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c01034a0 msgr2=0x7fd1c019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c01034a0 0x7fd1c019fd90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000e970 tx=0x7fd1b000ed70 comp rx=0 tx=0).stop 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 shutdown_connections 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1ac07af30 0x7fd1ac07d3f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1c01034a0 0x7fd1c019fd90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 --2- 172.21.15.7:0/3689503334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1c0102b80 0x7fd1c019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 >> 172.21.15.7:0/3689503334 conn(0x7fd1c00fc520 msgr2=0x7fd1c0100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 shutdown_connections 2024-09-21T04:41:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.364+0000 7fd1c803d640 1 -- 172.21.15.7:0/3689503334 wait complete. 2024-09-21T04:41:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.448+0000 7fc27022c640 1 -- 172.21.15.7:0/4030318120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268100650 msgr2=0x7fc268100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.448+0000 7fc27022c640 1 --2- 172.21.15.7:0/4030318120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268100650 0x7fc268100a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc264009880 tx=0x7fc26402a490 comp rx=0 tx=0).stop 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.448+0000 7fc27022c640 1 -- 172.21.15.7:0/4030318120 shutdown_connections 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.448+0000 7fc27022c640 1 --2- 172.21.15.7:0/4030318120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268101000 0x7fc26810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.448+0000 7fc27022c640 1 --2- 172.21.15.7:0/4030318120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268100650 0x7fc268100a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.448+0000 7fc27022c640 1 -- 172.21.15.7:0/4030318120 >> 172.21.15.7:0/4030318120 conn(0x7fc2680fc520 msgr2=0x7fc2680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.448+0000 7fc27022c640 1 -- 172.21.15.7:0/4030318120 shutdown_connections 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.448+0000 7fc27022c640 1 -- 172.21.15.7:0/4030318120 wait complete. 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc27022c640 1 Processor -- start 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc27022c640 1 -- start start 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc27022c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268100650 0x7fc26819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc27022c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101000 0x7fc26819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc27022c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc268110fb0 con 0x7fc268101000 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc27022c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc268110e30 con 0x7fc268100650 2024-09-21T04:41:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc26d7c7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101000 0x7fc26819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc26d7c7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101000 0x7fc26819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37074/0 (socket says 172.21.15.7:37074) 2024-09-21T04:41:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc26d7c7640 1 -- 172.21.15.7:0/2124958211 learned_addr learned my addr 172.21.15.7:0/2124958211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:41:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.449+0000 7fc26dfc8640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268100650 0x7fc26819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc26d7c7640 1 -- 172.21.15.7:0/2124958211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268100650 msgr2=0x7fc26819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc26d7c7640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268100650 0x7fc26819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc26d7c7640 1 -- 172.21.15.7:0/2124958211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc268199a40 con 0x7fc268101000 2024-09-21T04:41:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc26d7c7640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101000 0x7fc26819fdc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e970 tx=0x7fc25800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc25effd640 1 -- 172.21.15.7:0/2124958211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc25800cac0 con 0x7fc268101000 2024-09-21T04:41:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc25effd640 1 -- 172.21.15.7:0/2124958211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc2580054e0 con 0x7fc268101000 2024-09-21T04:41:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc268199d30 con 0x7fc268101000 2024-09-21T04:41:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc25effd640 1 -- 172.21.15.7:0/2124958211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc258010430 con 0x7fc268101000 2024-09-21T04:41:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.450+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26819a2c0 con 0x7fc268101000 2024-09-21T04:41:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.451+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26806aa90 con 0x7fc268101000 2024-09-21T04:41:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.452+0000 7fc25effd640 1 -- 172.21.15.7:0/2124958211 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc258004f00 con 0x7fc268101000 2024-09-21T04:41:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.452+0000 7fc25effd640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25407af90 0x7fc25407d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:41:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.453+0000 7fc26dfc8640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25407af90 0x7fc25407d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:41:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.453+0000 7fc25effd640 1 -- 172.21.15.7:0/2124958211 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7fc25809e550 con 0x7fc268101000 2024-09-21T04:41:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.453+0000 7fc26dfc8640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25407af90 0x7fc25407d450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc264000c00 tx=0x7fc264031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:41:56.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.457+0000 7fc25effd640 1 -- 172.21.15.7:0/2124958211 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc258069c20 con 0x7fc268101000 2024-09-21T04:41:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.719+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fc26810b3a0 con 0x7fc268101000 2024-09-21T04:41:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.719+0000 7fc25effd640 1 -- 172.21.15.7:0/2124958211 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7fc25806ead0 con 0x7fc268101000 2024-09-21T04:41:56.722 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:41:56.722 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:41:56.722 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:41:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25407af90 msgr2=0x7fc25407d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25407af90 0x7fc25407d450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc264000c00 tx=0x7fc264031400 comp rx=0 tx=0).stop 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101000 msgr2=0x7fc26819fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101000 0x7fc26819fdc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e970 tx=0x7fc25800ed70 comp rx=0 tx=0).stop 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 shutdown_connections 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25407af90 0x7fc25407d450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101000 0x7fc26819fdc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 --2- 172.21.15.7:0/2124958211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268100650 0x7fc26819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.721+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 >> 172.21.15.7:0/2124958211 conn(0x7fc2680fc520 msgr2=0x7fc2680fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.722+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 shutdown_connections 2024-09-21T04:41:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:41:56.722+0000 7fc27022c640 1 -- 172.21.15.7:0/2124958211 wait complete. 2024-09-21T04:41:56.988 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:56 smithi053 ceph-mon[80176]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:56.988 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:56 smithi053 ceph-mon[80176]: pgmap v197: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:56.988 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:56 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/500434935' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:41:56.988 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:56 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2124958211' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:41:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:56 smithi007 ceph-mon[108238]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:56 smithi007 ceph-mon[108238]: pgmap v197: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:57.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:56 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/500434935' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:41:57.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:56 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2124958211' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:41:57.931 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:57 smithi053 ceph-mon[80176]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:57.931 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:41:57 smithi053 systemd[1]: Stopping Ceph osd.3 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:41:58.193 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:41:57 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[41479]: 2024-09-21T04:41:57.927+0000 7fe465992640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:41:58.193 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:41:57 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[41479]: 2024-09-21T04:41:57.927+0000 7fe465992640 -1 osd.3 61 *** Got signal Terminated *** 2024-09-21T04:41:58.193 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:41:57 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[41479]: 2024-09-21T04:41:57.927+0000 7fe465992640 -1 osd.3 61 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:41:58.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:57 smithi007 ceph-mon[108238]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:41:59.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:58 smithi053 ceph-mon[80176]: pgmap v198: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:58 smithi053 ceph-mon[80176]: osd.3 marked itself down and dead 2024-09-21T04:41:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:59.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:41:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:58 smithi007 ceph-mon[108238]: pgmap v198: 65 pgs: 65 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:41:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:58 smithi007 ceph-mon[108238]: osd.3 marked itself down and dead 2024-09-21T04:41:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:41:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:41:59.565 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:41:59 smithi053 podman[88032]: 2024-09-21 04:41:59.301976006 +0000 UTC m=+1.473601718 container died b545d5073061908244038577d39b91afe0714e3cb1f27ffcb1404cb4f6df5612 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-4e45e6b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:41:59.900 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:41:59 smithi053 podman[88032]: 2024-09-21 04:41:59.653221904 +0000 UTC m=+1.824847601 container cleanup b545d5073061908244038577d39b91afe0714e3cb1f27ffcb1404cb4f6df5612 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, RELEASE=squid-4e45e6b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T04:41:59.900 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:41:59 smithi053 bash[88032]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3 2024-09-21T04:42:00.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:59 smithi053 ceph-mon[80176]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:42:00.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:41:59 smithi053 ceph-mon[80176]: osdmap e62: 6 total, 5 up, 6 in 2024-09-21T04:42:00.193 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:41:59 smithi053 podman[88060]: 2024-09-21 04:41:59.940961921 +0000 UTC m=+0.633604551 container remove b545d5073061908244038577d39b91afe0714e3cb1f27ffcb1404cb4f6df5612 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-4e45e6b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T04:42:00.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:59 smithi007 ceph-mon[108238]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:42:00.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:41:59 smithi007 ceph-mon[108238]: osdmap e62: 6 total, 5 up, 6 in 2024-09-21T04:42:00.561 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:00 smithi053 podman[88151]: 2024-09-21 04:42:00.177042653 +0000 UTC m=+0.016641309 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:42:00.561 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:00 smithi053 podman[88151]: 2024-09-21 04:42:00.305831852 +0000 UTC m=+0.145430544 container create 9fe8e7ceea3cae128ab56d4a9a939451e97762cf4aa6c893d863af7563174aaa (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T04:42:00.561 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:00 smithi053 podman[88151]: 2024-09-21 04:42:00.55877167 +0000 UTC m=+0.398370326 container init 9fe8e7ceea3cae128ab56d4a9a939451e97762cf4aa6c893d863af7563174aaa (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:42:00.901 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:00 smithi053 podman[88151]: 2024-09-21 04:42:00.562395004 +0000 UTC m=+0.401993651 container start 9fe8e7ceea3cae128ab56d4a9a939451e97762cf4aa6c893d863af7563174aaa (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T04:42:00.901 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:00 smithi053 podman[88151]: 2024-09-21 04:42:00.650296854 +0000 UTC m=+0.489895512 container attach 9fe8e7ceea3cae128ab56d4a9a939451e97762cf4aa6c893d863af7563174aaa (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T04:42:00.901 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:00 smithi053 podman[88151]: 2024-09-21 04:42:00.767192617 +0000 UTC m=+0.606791264 container died 9fe8e7ceea3cae128ab56d4a9a939451e97762cf4aa6c893d863af7563174aaa (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T04:42:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:00 smithi053 ceph-mon[80176]: pgmap v200: 65 pgs: 16 stale+active+clean, 49 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:42:01.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:00 smithi053 ceph-mon[80176]: osdmap e63: 6 total, 5 up, 6 in 2024-09-21T04:42:01.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:00 smithi007 ceph-mon[108238]: pgmap v200: 65 pgs: 16 stale+active+clean, 49 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:42:01.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:00 smithi007 ceph-mon[108238]: osdmap e63: 6 total, 5 up, 6 in 2024-09-21T04:42:01.692 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:01 smithi053 podman[88173]: 2024-09-21 04:42:01.397001787 +0000 UTC m=+0.623168455 container remove 9fe8e7ceea3cae128ab56d4a9a939451e97762cf4aa6c893d863af7563174aaa (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:42:01.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:01 smithi053 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.3.service: Deactivated successfully. 2024-09-21T04:42:01.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:01 smithi053 systemd[1]: Stopped Ceph osd.3 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:42:01.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:01 smithi053 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.3.service: Consumed 1min 5.960s CPU time. 2024-09-21T04:42:01.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:01 smithi053 systemd[1]: Starting Ceph osd.3 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:42:02.111 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:01 smithi053 ceph-mon[80176]: Health check failed: Degraded data redundancy: 10/333 objects degraded (3.003%), 3 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:02.111 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:01 smithi053 podman[88274]: 2024-09-21 04:42:01.84883801 +0000 UTC m=+0.112874029 container create bcf510e219f79c018326670208869b34b8651cb247b58981e8eb6a4d1ebb31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T04:42:02.112 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:01 smithi053 podman[88274]: 2024-09-21 04:42:01.752440506 +0000 UTC m=+0.016476478 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:42:02.112 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:02 smithi053 podman[88274]: 2024-09-21 04:42:02.043162224 +0000 UTC m=+0.307198178 container init bcf510e219f79c018326670208869b34b8651cb247b58981e8eb6a4d1ebb31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T04:42:02.112 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:02 smithi053 podman[88274]: 2024-09-21 04:42:02.046549123 +0000 UTC m=+0.310585080 container start bcf510e219f79c018326670208869b34b8651cb247b58981e8eb6a4d1ebb31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T04:42:02.112 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:02 smithi053 podman[88274]: 2024-09-21 04:42:02.109535018 +0000 UTC m=+0.373570974 container attach bcf510e219f79c018326670208869b34b8651cb247b58981e8eb6a4d1ebb31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T04:42:02.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:01 smithi007 ceph-mon[108238]: Health check failed: Degraded data redundancy: 10/333 objects degraded (3.003%), 3 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:02.443 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:02 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:02.443 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:02 smithi053 bash[88274]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:02.443 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:02 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:02.443 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:02 smithi053 bash[88274]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:03.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:02 smithi007 ceph-mon[108238]: pgmap v202: 65 pgs: 4 active+undersized, 13 stale+active+clean, 3 active+undersized+degraded, 45 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 10/333 objects degraded (3.003%) 2024-09-21T04:42:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:02 smithi053 ceph-mon[80176]: pgmap v202: 65 pgs: 4 active+undersized, 13 stale+active+clean, 3 active+undersized+degraded, 45 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 10/333 objects degraded (3.003%) 2024-09-21T04:42:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:04 smithi007 ceph-mon[108238]: pgmap v203: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:05.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:04 smithi053 ceph-mon[80176]: pgmap v203: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:05.429 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:42:05.429 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:05.429 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:42:05.429 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:05.429 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:05.429 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:05.429 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T04:42:05.430 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T04:42:05.430 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2024-09-21T04:42:05.430 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2024-09-21T04:42:05.430 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-21T04:42:05.430 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-21T04:42:05.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2024-09-21T04:42:05.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2024-09-21T04:42:05.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T04:42:05.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T04:42:05.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T04:42:05.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T04:42:05.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate[88289]: --> ceph-volume lvm activate successful for osd ID: 3 2024-09-21T04:42:05.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 bash[88274]: --> ceph-volume lvm activate successful for osd ID: 3 2024-09-21T04:42:05.694 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:05 smithi053 podman[88274]: 2024-09-21 04:42:05.455810139 +0000 UTC m=+3.719846100 container died bcf510e219f79c018326670208869b34b8651cb247b58981e8eb6a4d1ebb31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True) 2024-09-21T04:42:06.443 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:06 smithi053 podman[88496]: 2024-09-21 04:42:06.14095634 +0000 UTC m=+0.677993211 container remove bcf510e219f79c018326670208869b34b8651cb247b58981e8eb6a4d1ebb31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T04:42:06.443 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:06 smithi053 podman[88556]: 2024-09-21 04:42:06.257991779 +0000 UTC m=+0.016501703 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:42:06.443 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:06 smithi053 podman[88556]: 2024-09-21 04:42:06.366763288 +0000 UTC m=+0.125273240 container create 1a3233caf51a33faa85403126b8d03f758a6782aa5f4d5aa4c8014df2cfd2485 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS) 2024-09-21T04:42:06.943 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:06 smithi053 podman[88556]: 2024-09-21 04:42:06.611031791 +0000 UTC m=+0.369541720 container init 1a3233caf51a33faa85403126b8d03f758a6782aa5f4d5aa4c8014df2cfd2485 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T04:42:06.943 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:06 smithi053 podman[88556]: 2024-09-21 04:42:06.614544782 +0000 UTC m=+0.373054688 container start 1a3233caf51a33faa85403126b8d03f758a6782aa5f4d5aa4c8014df2cfd2485 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:42:06.943 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:06 smithi053 ceph-osd[88575]: -- 172.21.15.53:0/3504201980 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a275001e0 con 0x562a2669fc00 2024-09-21T04:42:06.943 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:06 smithi053 bash[88556]: 1a3233caf51a33faa85403126b8d03f758a6782aa5f4d5aa4c8014df2cfd2485 2024-09-21T04:42:06.943 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:06 smithi053 systemd[1]: Started Ceph osd.3 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:42:07.352 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:06 smithi053 ceph-mon[80176]: pgmap v204: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:07.352 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:06 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:07.352 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:06 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:07.352 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:06 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:42:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:06 smithi007 ceph-mon[108238]: pgmap v204: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:06 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:07.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:06 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:07.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:06 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:42:08.693 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:08 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T04:42:08.489+0000 7f5334474640 -1 Falling back to public interface 2024-09-21T04:42:09.135 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:08 smithi053 ceph-mon[80176]: pgmap v205: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:09.135 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:08 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 70/333 objects degraded (21.021%), 19 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:08 smithi007 ceph-mon[108238]: pgmap v205: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:08 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 70/333 objects degraded (21.021%), 19 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:10.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:10 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:10.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:10 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:10.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:10 smithi053 ceph-mon[80176]: pgmap v206: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:10.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:10 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:10.834 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:10 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:10.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:10 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:10.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:10 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:10.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:10 smithi007 ceph-mon[108238]: pgmap v206: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:10.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:10 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:10.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:10 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:12 smithi053 ceph-mon[80176]: pgmap v207: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:12 smithi007 ceph-mon[108238]: pgmap v207: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:14.601 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T04:42:14.224+0000 7f5334474640 -1 osd.3 61 log_to_monitors true 2024-09-21T04:42:14.604 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T04:42:14.597+0000 7f532c21e640 -1 osd.3 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:14.605 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:14.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-21T04:42:14.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-21T04:42:14.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-21T04:42:14.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: pgmap v208: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:14.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='osd.3 [v2:172.21.15.53:6800/1863065362,v1:172.21.15.53:6801/1863065362]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T04:42:14.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:14 smithi053 ceph-mon[80176]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T04:42:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:42:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:42:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:42:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: pgmap v208: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 70/333 objects degraded (21.021%) 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='osd.3 [v2:172.21.15.53:6800/1863065362,v1:172.21.15.53:6801/1863065362]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T04:42:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:14 smithi007 ceph-mon[108238]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T04:42:15.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:15 smithi053 ceph-mon[80176]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T04:42:15.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:15 smithi053 ceph-mon[80176]: from='osd.3 [v2:172.21.15.53:6800/1863065362,v1:172.21.15.53:6801/1863065362]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:42:15.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:15 smithi053 ceph-mon[80176]: osdmap e64: 6 total, 5 up, 6 in 2024-09-21T04:42:15.944 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:15 smithi053 ceph-mon[80176]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:42:15.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:15 smithi007 ceph-mon[108238]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T04:42:15.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:15 smithi007 ceph-mon[108238]: from='osd.3 [v2:172.21.15.53:6800/1863065362,v1:172.21.15.53:6801/1863065362]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:42:15.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:15 smithi007 ceph-mon[108238]: osdmap e64: 6 total, 5 up, 6 in 2024-09-21T04:42:15.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:15 smithi007 ceph-mon[108238]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:42:16.928 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:16 smithi007 ceph-mon[108238]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:42:16.928 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:16 smithi007 ceph-mon[108238]: osd.3 [v2:172.21.15.53:6800/1863065362,v1:172.21.15.53:6801/1863065362] boot 2024-09-21T04:42:16.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:16 smithi007 ceph-mon[108238]: osdmap e65: 6 total, 6 up, 6 in 2024-09-21T04:42:16.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T04:42:16.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:16 smithi007 ceph-mon[108238]: pgmap v211: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 70/333 objects degraded (21.021%) 2024-09-21T04:42:16.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:16 smithi053 ceph-mon[80176]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:42:16.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:16 smithi053 ceph-mon[80176]: osd.3 [v2:172.21.15.53:6800/1863065362,v1:172.21.15.53:6801/1863065362] boot 2024-09-21T04:42:16.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:16 smithi053 ceph-mon[80176]: osdmap e65: 6 total, 6 up, 6 in 2024-09-21T04:42:16.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T04:42:16.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:16 smithi053 ceph-mon[80176]: pgmap v211: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 70/333 objects degraded (21.021%) 2024-09-21T04:42:17.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:17 smithi053 ceph-mon[80176]: osdmap e66: 6 total, 6 up, 6 in 2024-09-21T04:42:17.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:17 smithi007 ceph-mon[108238]: osdmap e66: 6 total, 6 up, 6 in 2024-09-21T04:42:18.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:18 smithi053 ceph-mon[80176]: pgmap v213: 65 pgs: 18 active+undersized, 17 active+undersized+degraded, 30 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 66/333 objects degraded (19.820%) 2024-09-21T04:42:18.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:18 smithi007 ceph-mon[108238]: pgmap v213: 65 pgs: 18 active+undersized, 17 active+undersized+degraded, 30 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 66/333 objects degraded (19.820%) 2024-09-21T04:42:19.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:19 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 66/333 objects degraded (19.820%), 17 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:19.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:19 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 66/333 objects degraded (19.820%), 17 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:20.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:20 smithi053 ceph-mon[80176]: pgmap v214: 65 pgs: 3 active+undersized, 1 active+undersized+degraded, 61 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 3/333 objects degraded (0.901%) 2024-09-21T04:42:20.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:20 smithi007 ceph-mon[108238]: pgmap v214: 65 pgs: 3 active+undersized, 1 active+undersized+degraded, 61 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 3/333 objects degraded (0.901%) 2024-09-21T04:42:22.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:21 smithi053 ceph-mon[80176]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/333 objects degraded (0.901%), 1 pg degraded) 2024-09-21T04:42:22.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:21 smithi007 ceph-mon[108238]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/333 objects degraded (0.901%), 1 pg degraded) 2024-09-21T04:42:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:22 smithi053 ceph-mon[80176]: pgmap v215: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T04:42:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:22 smithi007 ceph-mon[108238]: pgmap v215: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T04:42:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:24 smithi053 ceph-mon[80176]: pgmap v216: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T04:42:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:24 smithi007 ceph-mon[108238]: pgmap v216: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T04:42:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.819+0000 7f5a638fa640 1 -- 172.21.15.7:0/1323844720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c102ac0 msgr2=0x7f5a5c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.819+0000 7f5a638fa640 1 --2- 172.21.15.7:0/1323844720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c102ac0 0x7f5a5c102ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009880 tx=0x7f5a5802a530 comp rx=0 tx=0).stop 2024-09-21T04:42:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.819+0000 7f5a638fa640 1 -- 172.21.15.7:0/1323844720 shutdown_connections 2024-09-21T04:42:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.819+0000 7f5a638fa640 1 --2- 172.21.15.7:0/1323844720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a5c1033e0 0x7f5a5c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.819+0000 7f5a638fa640 1 --2- 172.21.15.7:0/1323844720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c102ac0 0x7f5a5c102ea0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.819+0000 7f5a638fa640 1 -- 172.21.15.7:0/1323844720 >> 172.21.15.7:0/1323844720 conn(0x7f5a5c0fc520 msgr2=0x7f5a5c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.820+0000 7f5a638fa640 1 -- 172.21.15.7:0/1323844720 shutdown_connections 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.820+0000 7f5a638fa640 1 -- 172.21.15.7:0/1323844720 wait complete. 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.820+0000 7f5a638fa640 1 Processor -- start 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.820+0000 7f5a638fa640 1 -- start start 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a638fa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a5c102ac0 0x7f5a5c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a638fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c1033e0 0x7f5a5c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a638fa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c10edd0 con 0x7f5a5c1033e0 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a638fa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c10ec50 con 0x7f5a5c102ac0 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a60e95640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c1033e0 0x7f5a5c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a60e95640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c1033e0 0x7f5a5c19dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54764/0 (socket says 172.21.15.7:54764) 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a60e95640 1 -- 172.21.15.7:0/1890075530 learned_addr learned my addr 172.21.15.7:0/1890075530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a61696640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a5c102ac0 0x7f5a5c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a60e95640 1 -- 172.21.15.7:0/1890075530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a5c102ac0 msgr2=0x7f5a5c19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a60e95640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a5c102ac0 0x7f5a5c19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a60e95640 1 -- 172.21.15.7:0/1890075530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c1978e0 con 0x7f5a5c1033e0 2024-09-21T04:42:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a60e95640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c1033e0 0x7f5a5c19dc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00d6c0 tx=0x7f5a4c00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.821+0000 7f5a527fc640 1 -- 172.21.15.7:0/1890075530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5a4c017070 con 0x7f5a5c1033e0 2024-09-21T04:42:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.822+0000 7f5a527fc640 1 -- 172.21.15.7:0/1890075530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a4c009ce0 con 0x7f5a5c1033e0 2024-09-21T04:42:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.822+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5c197bd0 con 0x7f5a5c1033e0 2024-09-21T04:42:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.822+0000 7f5a527fc640 1 -- 172.21.15.7:0/1890075530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5a4c00f3c0 con 0x7f5a5c1033e0 2024-09-21T04:42:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.822+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5c198160 con 0x7f5a5c1033e0 2024-09-21T04:42:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.823+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a5c06aa90 con 0x7f5a5c1033e0 2024-09-21T04:42:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.824+0000 7f5a527fc640 1 -- 172.21.15.7:0/1890075530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5a4c010460 con 0x7f5a5c1033e0 2024-09-21T04:42:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.824+0000 7f5a527fc640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a4807af90 0x7f5a4807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.824+0000 7f5a61696640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a4807af90 0x7f5a4807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.824+0000 7f5a527fc640 1 -- 172.21.15.7:0/1890075530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7f5a4c09dc00 con 0x7f5a5c1033e0 2024-09-21T04:42:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.825+0000 7f5a61696640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a4807af90 0x7f5a4807d450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a58000c00 tx=0x7f5a58031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:26.829+0000 7f5a527fc640 1 -- 172.21.15.7:0/1890075530 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a4c0a3020 con 0x7f5a5c1033e0 2024-09-21T04:42:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.041+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5a5c106000 con 0x7f5a4807af90 2024-09-21T04:42:27.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.042+0000 7f5a527fc640 1 -- 172.21.15.7:0/1890075530 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7f5a58004260 con 0x7f5a4807af90 2024-09-21T04:42:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a4807af90 msgr2=0x7f5a4807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a4807af90 0x7f5a4807d450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a58000c00 tx=0x7f5a58031400 comp rx=0 tx=0).stop 2024-09-21T04:42:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c1033e0 msgr2=0x7f5a5c19dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c1033e0 0x7f5a5c19dc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00d6c0 tx=0x7f5a4c00db80 comp rx=0 tx=0).stop 2024-09-21T04:42:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 shutdown_connections 2024-09-21T04:42:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a4807af90 0x7f5a4807d450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a5c1033e0 0x7f5a5c19dc60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 --2- 172.21.15.7:0/1890075530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a5c102ac0 0x7f5a5c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 >> 172.21.15.7:0/1890075530 conn(0x7f5a5c0fc520 msgr2=0x7f5a5c108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 shutdown_connections 2024-09-21T04:42:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.044+0000 7f5a638fa640 1 -- 172.21.15.7:0/1890075530 wait complete. 2024-09-21T04:42:27.056 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:42:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.130+0000 7fea96fa0640 1 -- 172.21.15.7:0/4282375487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90107750 msgr2=0x7fea90107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.130+0000 7fea96fa0640 1 --2- 172.21.15.7:0/4282375487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90107750 0x7fea90107b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea88009880 tx=0x7fea8802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.130+0000 7fea96fa0640 1 -- 172.21.15.7:0/4282375487 shutdown_connections 2024-09-21T04:42:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.130+0000 7fea96fa0640 1 --2- 172.21.15.7:0/4282375487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea90101750 0x7fea90101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.130+0000 7fea96fa0640 1 --2- 172.21.15.7:0/4282375487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90107750 0x7fea90107b30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.130+0000 7fea96fa0640 1 -- 172.21.15.7:0/4282375487 >> 172.21.15.7:0/4282375487 conn(0x7fea900fd470 msgr2=0x7fea900ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:27.134 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:26 smithi007 ceph-mon[108238]: pgmap v217: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-21T04:42:27.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.130+0000 7fea96fa0640 1 -- 172.21.15.7:0/4282375487 shutdown_connections 2024-09-21T04:42:27.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.130+0000 7fea96fa0640 1 -- 172.21.15.7:0/4282375487 wait complete. 2024-09-21T04:42:27.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea96fa0640 1 Processor -- start 2024-09-21T04:42:27.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea96fa0640 1 -- start start 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea96fa0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90101750 0x7fea900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea96fa0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea90107750 0x7fea90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea96fa0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea9010ec80 con 0x7fea90101750 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea96fa0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fea9010eb00 con 0x7fea90107750 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea95f9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90101750 0x7fea900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea95f9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90101750 0x7fea900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54788/0 (socket says 172.21.15.7:54788) 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea95f9e640 1 -- 172.21.15.7:0/810408655 learned_addr learned my addr 172.21.15.7:0/810408655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea9579d640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea90107750 0x7fea90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea95f9e640 1 -- 172.21.15.7:0/810408655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea90107750 msgr2=0x7fea90074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea95f9e640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea90107750 0x7fea90074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.131+0000 7fea95f9e640 1 -- 172.21.15.7:0/810408655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea90074f70 con 0x7fea90101750 2024-09-21T04:42:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.132+0000 7fea9579d640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea90107750 0x7fea90074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:42:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.132+0000 7fea95f9e640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90101750 0x7fea900744f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea88031460 tx=0x7fea880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.132+0000 7fea8effd640 1 -- 172.21.15.7:0/810408655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fea8803f070 con 0x7fea90101750 2024-09-21T04:42:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.132+0000 7fea8effd640 1 -- 172.21.15.7:0/810408655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea880044d0 con 0x7fea90101750 2024-09-21T04:42:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.132+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea90075200 con 0x7fea90101750 2024-09-21T04:42:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.132+0000 7fea8effd640 1 -- 172.21.15.7:0/810408655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fea88035030 con 0x7fea90101750 2024-09-21T04:42:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.132+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901a3c10 con 0x7fea90101750 2024-09-21T04:42:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.133+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea90102e90 con 0x7fea90101750 2024-09-21T04:42:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.134+0000 7fea8effd640 1 -- 172.21.15.7:0/810408655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fea88042070 con 0x7fea90101750 2024-09-21T04:42:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.134+0000 7fea8effd640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea7807af40 0x7fea7807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.134+0000 7fea8effd640 1 -- 172.21.15.7:0/810408655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7fea880bebb0 con 0x7fea90101750 2024-09-21T04:42:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.134+0000 7fea9579d640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea7807af40 0x7fea7807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.135+0000 7fea9579d640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea7807af40 0x7fea7807d400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea84006fd0 tx=0x7fea84008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:27.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.139+0000 7fea8effd640 1 -- 172.21.15.7:0/810408655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea880c5080 con 0x7fea90101750 2024-09-21T04:42:27.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:26 smithi053 ceph-mon[80176]: pgmap v217: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-21T04:42:27.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.343+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fea90062630 con 0x7fea7807af40 2024-09-21T04:42:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.344+0000 7fea8effd640 1 -- 172.21.15.7:0/810408655 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7fea90062630 con 0x7fea7807af40 2024-09-21T04:42:27.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea7807af40 msgr2=0x7fea7807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea7807af40 0x7fea7807d400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea84006fd0 tx=0x7fea84008040 comp rx=0 tx=0).stop 2024-09-21T04:42:27.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90101750 msgr2=0x7fea900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90101750 0x7fea900744f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea88031460 tx=0x7fea880048f0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 shutdown_connections 2024-09-21T04:42:27.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea7807af40 0x7fea7807d400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea90107750 0x7fea90074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 --2- 172.21.15.7:0/810408655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90101750 0x7fea900744f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.346+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 >> 172.21.15.7:0/810408655 conn(0x7fea900fd470 msgr2=0x7fea900fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:27.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.347+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 shutdown_connections 2024-09-21T04:42:27.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.347+0000 7fea96fa0640 1 -- 172.21.15.7:0/810408655 wait complete. 2024-09-21T04:42:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.431+0000 7fd249d06640 1 -- 172.21.15.7:0/3265096283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 msgr2=0x7fd244101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.431+0000 7fd249d06640 1 --2- 172.21.15.7:0/3265096283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 0x7fd244101d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd238009880 tx=0x7fd23802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.431+0000 7fd249d06640 1 -- 172.21.15.7:0/3265096283 shutdown_connections 2024-09-21T04:42:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.431+0000 7fd249d06640 1 --2- 172.21.15.7:0/3265096283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 0x7fd244101d60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.431+0000 7fd249d06640 1 --2- 172.21.15.7:0/3265096283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd244107900 0x7fd244107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.431+0000 7fd249d06640 1 -- 172.21.15.7:0/3265096283 >> 172.21.15.7:0/3265096283 conn(0x7fd2440fd600 msgr2=0x7fd2440ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.431+0000 7fd249d06640 1 -- 172.21.15.7:0/3265096283 shutdown_connections 2024-09-21T04:42:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.431+0000 7fd249d06640 1 -- 172.21.15.7:0/3265096283 wait complete. 2024-09-21T04:42:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd249d06640 1 Processor -- start 2024-09-21T04:42:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd249d06640 1 -- start start 2024-09-21T04:42:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd249d06640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 0x7fd2440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd249d06640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd244107900 0x7fd244074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd249d06640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410ee30 con 0x7fd244101900 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd249d06640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410ecb0 con 0x7fd244107900 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 0x7fd2440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 0x7fd2440744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54806/0 (socket says 172.21.15.7:54806) 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.432+0000 7fd2437fe640 1 -- 172.21.15.7:0/1823818352 learned_addr learned my addr 172.21.15.7:0/1823818352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd242ffd640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd244107900 0x7fd244074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd2437fe640 1 -- 172.21.15.7:0/1823818352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd244107900 msgr2=0x7fd244074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd2437fe640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd244107900 0x7fd244074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd2437fe640 1 -- 172.21.15.7:0/1823818352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd244074f70 con 0x7fd244101900 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd2437fe640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 0x7fd2440744f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd23400b420 tx=0x7fd23400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd240ff9640 1 -- 172.21.15.7:0/1823818352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd234011630 con 0x7fd244101900 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd240ff9640 1 -- 172.21.15.7:0/1823818352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd234016070 con 0x7fd244101900 2024-09-21T04:42:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd244075260 con 0x7fd244101900 2024-09-21T04:42:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd240ff9640 1 -- 172.21.15.7:0/1823818352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd234010d40 con 0x7fd244101900 2024-09-21T04:42:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.433+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2441a3ec0 con 0x7fd244101900 2024-09-21T04:42:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.434+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd244103040 con 0x7fd244101900 2024-09-21T04:42:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.435+0000 7fd240ff9640 1 -- 172.21.15.7:0/1823818352 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd23401b480 con 0x7fd244101900 2024-09-21T04:42:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.435+0000 7fd240ff9640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd22c07af30 0x7fd22c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.436+0000 7fd240ff9640 1 -- 172.21.15.7:0/1823818352 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7fd23409d790 con 0x7fd244101900 2024-09-21T04:42:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.436+0000 7fd242ffd640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd22c07af30 0x7fd22c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.436+0000 7fd242ffd640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd22c07af30 0x7fd22c07d3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd244075f80 tx=0x7fd238031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.440+0000 7fd240ff9640 1 -- 172.21.15.7:0/1823818352 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd234068d40 con 0x7fd244101900 2024-09-21T04:42:27.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.641+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd244062630 con 0x7fd22c07af30 2024-09-21T04:42:27.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.651+0000 7fd240ff9640 1 -- 172.21.15.7:0/1823818352 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fd244062630 con 0x7fd22c07af30 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (4m) 53s ago 15m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (7m) 53s ago 15m 9013k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (7m) 18s ago 14m 8929k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (2m) 53s ago 15m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (2m) 18s ago 14m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (4m) 53s ago 15m 77.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (10m) 53s ago 10m 144M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (10m) 53s ago 10m 89.8M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (9m) 17s ago 9m 26.1M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:42:27.654 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (10m) 17s ago 10m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (6m) 53s ago 16m 528M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (5m) 18s ago 14m 456M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (3m) 53s ago 16m 43.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (2m) 18s ago 14m 36.7M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (5m) 53s ago 15m 10.7M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (5m) 18s ago 14m 10.5M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (2m) 53s ago 13m 86.1M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (97s) 53s ago 12m 90.7M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (59s) 53s ago 12m 14.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (21s) 18s ago 12m 11.5M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:42:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (11m) 18s ago 11m 215M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 4d77c6dc889b 2024-09-21T04:42:27.656 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (11m) 17s ago 11m 185M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:42:27.656 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (4m) 53s ago 15m 56.1M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:42:27.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd22c07af30 msgr2=0x7fd22c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd22c07af30 0x7fd22c07d3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd244075f80 tx=0x7fd238031400 comp rx=0 tx=0).stop 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 msgr2=0x7fd2440744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 0x7fd2440744f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd23400b420 tx=0x7fd23400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 shutdown_connections 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd22c07af30 0x7fd22c07d3f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd244107900 0x7fd244074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 --2- 172.21.15.7:0/1823818352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd244101900 0x7fd2440744f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 >> 172.21.15.7:0/1823818352 conn(0x7fd2440fd600 msgr2=0x7fd2440fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 shutdown_connections 2024-09-21T04:42:27.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.653+0000 7fd249d06640 1 -- 172.21.15.7:0/1823818352 wait complete. 2024-09-21T04:42:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.735+0000 7f8f15efa640 1 -- 172.21.15.7:0/2479776596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10102b10 msgr2=0x7f8f10102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.735+0000 7f8f15efa640 1 --2- 172.21.15.7:0/2479776596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10102b10 0x7f8f10102ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f000098b0 tx=0x7f8f0002a560 comp rx=0 tx=0).stop 2024-09-21T04:42:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.736+0000 7f8f15efa640 1 -- 172.21.15.7:0/2479776596 shutdown_connections 2024-09-21T04:42:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.736+0000 7f8f15efa640 1 --2- 172.21.15.7:0/2479776596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f10103430 0x7f8f1010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.736+0000 7f8f15efa640 1 --2- 172.21.15.7:0/2479776596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10102b10 0x7f8f10102ef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.736+0000 7f8f15efa640 1 -- 172.21.15.7:0/2479776596 >> 172.21.15.7:0/2479776596 conn(0x7f8f100fc550 msgr2=0x7f8f100fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.736+0000 7f8f15efa640 1 -- 172.21.15.7:0/2479776596 shutdown_connections 2024-09-21T04:42:27.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.736+0000 7f8f15efa640 1 -- 172.21.15.7:0/2479776596 wait complete. 2024-09-21T04:42:27.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.736+0000 7f8f15efa640 1 Processor -- start 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.736+0000 7f8f15efa640 1 -- start start 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f15efa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f10102b10 0x7f8f1019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f15efa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10103430 0x7f8f1019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f15efa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010ee40 con 0x7f8f10103430 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f15efa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1010ecc0 con 0x7f8f10102b10 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10103430 0x7f8f1019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10103430 0x7f8f1019dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54816/0 (socket says 172.21.15.7:54816) 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0effd640 1 -- 172.21.15.7:0/1605115548 learned_addr learned my addr 172.21.15.7:0/1605115548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0f7fe640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f10102b10 0x7f8f1019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0effd640 1 -- 172.21.15.7:0/1605115548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f10102b10 msgr2=0x7f8f1019d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0effd640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f10102b10 0x7f8f1019d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0effd640 1 -- 172.21.15.7:0/1605115548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f10197850 con 0x7f8f10103430 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0f7fe640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f10102b10 0x7f8f1019d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.737+0000 7f8f0effd640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10103430 0x7f8f1019dc10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400ba50 tx=0x7f8f0400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.738+0000 7f8f0cff9640 1 -- 172.21.15.7:0/1605115548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f04019070 con 0x7f8f10103430 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.738+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f10197b40 con 0x7f8f10103430 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.738+0000 7f8f0cff9640 1 -- 172.21.15.7:0/1605115548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f0400ce20 con 0x7f8f10103430 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.738+0000 7f8f0cff9640 1 -- 172.21.15.7:0/1605115548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f040075a0 con 0x7f8f10103430 2024-09-21T04:42:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.738+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101980d0 con 0x7f8f10103430 2024-09-21T04:42:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.739+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1006a520 con 0x7f8f10103430 2024-09-21T04:42:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.739+0000 7f8f0cff9640 1 -- 172.21.15.7:0/1605115548 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8f0401c070 con 0x7f8f10103430 2024-09-21T04:42:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.740+0000 7f8f0cff9640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ef807af30 0x7f8ef807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.740+0000 7f8f0cff9640 1 -- 172.21.15.7:0/1605115548 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7f8f0409d8a0 con 0x7f8f10103430 2024-09-21T04:42:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.740+0000 7f8f0f7fe640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ef807af30 0x7f8ef807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.740+0000 7f8f0f7fe640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ef807af30 0x7f8ef807d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f00000c00 tx=0x7f8f00031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:27.744+0000 7f8f0cff9640 1 -- 172.21.15.7:0/1605115548 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f04068ef0 con 0x7f8f10103430 2024-09-21T04:42:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.009+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f8f10198d80 con 0x7f8f10103430 2024-09-21T04:42:28.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:27 smithi007 ceph-mon[108238]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.010+0000 7f8f0cff9640 1 -- 172.21.15.7:0/1605115548 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7f8f0406dda0 con 0x7f8f10103430 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:42:28.013 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 2, 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 6, 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 8 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:42:28.014 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:42:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ef807af30 msgr2=0x7f8ef807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ef807af30 0x7f8ef807d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f00000c00 tx=0x7f8f00031400 comp rx=0 tx=0).stop 2024-09-21T04:42:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10103430 msgr2=0x7f8f1019dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10103430 0x7f8f1019dc10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400ba50 tx=0x7f8f0400bf10 comp rx=0 tx=0).stop 2024-09-21T04:42:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 shutdown_connections 2024-09-21T04:42:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8ef807af30 0x7f8ef807d3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f10103430 0x7f8f1019dc10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 --2- 172.21.15.7:0/1605115548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f10102b10 0x7f8f1019d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 >> 172.21.15.7:0/1605115548 conn(0x7f8f100fc550 msgr2=0x7f8f10107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.013+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 shutdown_connections 2024-09-21T04:42:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.014+0000 7f8f15efa640 1 -- 172.21.15.7:0/1605115548 wait complete. 2024-09-21T04:42:28.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.098+0000 7f89a9e69640 1 -- 172.21.15.7:0/3870593522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4101900 msgr2=0x7f89a4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.098+0000 7f89a9e69640 1 --2- 172.21.15.7:0/3870593522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4101900 0x7f89a4101d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8998009880 tx=0x7f899802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.098+0000 7f89a9e69640 1 -- 172.21.15.7:0/3870593522 shutdown_connections 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.098+0000 7f89a9e69640 1 --2- 172.21.15.7:0/3870593522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4101900 0x7f89a4101d60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.098+0000 7f89a9e69640 1 --2- 172.21.15.7:0/3870593522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89a4107900 0x7f89a4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.098+0000 7f89a9e69640 1 -- 172.21.15.7:0/3870593522 >> 172.21.15.7:0/3870593522 conn(0x7f89a40fd600 msgr2=0x7f89a40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.098+0000 7f89a9e69640 1 -- 172.21.15.7:0/3870593522 shutdown_connections 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.098+0000 7f89a9e69640 1 -- 172.21.15.7:0/3870593522 wait complete. 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a9e69640 1 Processor -- start 2024-09-21T04:42:28.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a9e69640 1 -- start start 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a9e69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89a4101900 0x7f89a419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a9e69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4107900 0x7f89a419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a9e69640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89a410edb0 con 0x7f89a4107900 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a9e69640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f89a410ec30 con 0x7f89a4101900 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4107900 0x7f89a419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4107900 0x7f89a419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54830/0 (socket says 172.21.15.7:54830) 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.099+0000 7f89a2ffd640 1 -- 172.21.15.7:0/172510764 learned_addr learned my addr 172.21.15.7:0/172510764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a37fe640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89a4101900 0x7f89a419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a2ffd640 1 -- 172.21.15.7:0/172510764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89a4101900 msgr2=0x7f89a419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a2ffd640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89a4101900 0x7f89a419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a2ffd640 1 -- 172.21.15.7:0/172510764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a41a0400 con 0x7f89a4107900 2024-09-21T04:42:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a2ffd640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4107900 0x7f89a419fd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8998031460 tx=0x7f89980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a0ff9640 1 -- 172.21.15.7:0/172510764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f899803f070 con 0x7f89a4107900 2024-09-21T04:42:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a4199a90 con 0x7f89a4107900 2024-09-21T04:42:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a0ff9640 1 -- 172.21.15.7:0/172510764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8998004cf0 con 0x7f89a4107900 2024-09-21T04:42:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a0ff9640 1 -- 172.21.15.7:0/172510764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8998035030 con 0x7f89a4107900 2024-09-21T04:42:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.100+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a4199f90 con 0x7f89a4107900 2024-09-21T04:42:28.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.101+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a4103040 con 0x7f89a4107900 2024-09-21T04:42:28.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.102+0000 7f89a0ff9640 1 -- 172.21.15.7:0/172510764 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8998042070 con 0x7f89a4107900 2024-09-21T04:42:28.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.102+0000 7f89a0ff9640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898c07af30 0x7f898c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.102+0000 7f89a0ff9640 1 -- 172.21.15.7:0/172510764 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7f89980bea70 con 0x7f89a4107900 2024-09-21T04:42:28.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.102+0000 7f89a37fe640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898c07af30 0x7f898c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.103+0000 7f89a37fe640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898c07af30 0x7f898c07d3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8994005fd0 tx=0x7f8994005ce0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:28.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.107+0000 7f89a0ff9640 1 -- 172.21.15.7:0/172510764 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f899808a0c0 con 0x7f89a4107900 2024-09-21T04:42:28.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:27 smithi053 ceph-mon[80176]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:42:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.334+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f89a419a300 con 0x7f89a4107900 2024-09-21T04:42:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.335+0000 7f89a0ff9640 1 -- 172.21.15.7:0/172510764 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2038 (secure 0 0 0) 0x7f899808ef70 con 0x7f89a4107900 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-09-21T04:42:28.338 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 44 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:42:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14578,1=24327} 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:42:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:42:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:42:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:42:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898c07af30 msgr2=0x7f898c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898c07af30 0x7f898c07d3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8994005fd0 tx=0x7f8994005ce0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4107900 msgr2=0x7f89a419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4107900 0x7f89a419fd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8998031460 tx=0x7f89980048f0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 shutdown_connections 2024-09-21T04:42:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f898c07af30 0x7f898c07d3f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a4107900 0x7f89a419fd50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 --2- 172.21.15.7:0/172510764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89a4101900 0x7f89a419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 >> 172.21.15.7:0/172510764 conn(0x7f89a40fd600 msgr2=0x7f89a4074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 shutdown_connections 2024-09-21T04:42:28.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.337+0000 7f89a9e69640 1 -- 172.21.15.7:0/172510764 wait complete. 2024-09-21T04:42:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 -- 172.21.15.7:0/2140528611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c103430 msgr2=0x7f148c10b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 --2- 172.21.15.7:0/2140528611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c103430 0x7f148c10b920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f14740098b0 tx=0x7f147402a560 comp rx=0 tx=0).stop 2024-09-21T04:42:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 -- 172.21.15.7:0/2140528611 shutdown_connections 2024-09-21T04:42:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 --2- 172.21.15.7:0/2140528611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c103430 0x7f148c10b920 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 --2- 172.21.15.7:0/2140528611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c102b10 0x7f148c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 -- 172.21.15.7:0/2140528611 >> 172.21.15.7:0/2140528611 conn(0x7f148c0fc550 msgr2=0x7f148c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 -- 172.21.15.7:0/2140528611 shutdown_connections 2024-09-21T04:42:28.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 -- 172.21.15.7:0/2140528611 wait complete. 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.423+0000 7f1490bf0640 1 Processor -- start 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f1490bf0640 1 -- start start 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f1490bf0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c102b10 0x7f148c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f1490bf0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c103430 0x7f148c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f1490bf0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f148c10ee40 con 0x7f148c102b10 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f1490bf0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f148c10ecc0 con 0x7f148c103430 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f148a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c102b10 0x7f148c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f148a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c102b10 0x7f148c19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54844/0 (socket says 172.21.15.7:54844) 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f148a59c640 1 -- 172.21.15.7:0/3476010110 learned_addr learned my addr 172.21.15.7:0/3476010110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f1483fff640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c103430 0x7f148c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f148a59c640 1 -- 172.21.15.7:0/3476010110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c103430 msgr2=0x7f148c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f148a59c640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c103430 0x7f148c19dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f148a59c640 1 -- 172.21.15.7:0/3476010110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c197870 con 0x7f148c102b10 2024-09-21T04:42:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.424+0000 7f148a59c640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c102b10 0x7f148c19d6b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f147c00b420 tx=0x7f147c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.425+0000 7f14837fe640 1 -- 172.21.15.7:0/3476010110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f147c011630 con 0x7f148c102b10 2024-09-21T04:42:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.425+0000 7f14837fe640 1 -- 172.21.15.7:0/3476010110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f147c016070 con 0x7f148c102b10 2024-09-21T04:42:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.425+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148c197b60 con 0x7f148c102b10 2024-09-21T04:42:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.425+0000 7f14837fe640 1 -- 172.21.15.7:0/3476010110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f147c010d40 con 0x7f148c102b10 2024-09-21T04:42:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.425+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148c1980f0 con 0x7f148c102b10 2024-09-21T04:42:28.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.426+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148c06a520 con 0x7f148c102b10 2024-09-21T04:42:28.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.426+0000 7f14837fe640 1 -- 172.21.15.7:0/3476010110 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f147c01b460 con 0x7f148c102b10 2024-09-21T04:42:28.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.427+0000 7f14837fe640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147807af30 0x7f147807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.427+0000 7f1483fff640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147807af30 0x7f147807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.427+0000 7f14837fe640 1 -- 172.21.15.7:0/3476010110 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7f147c09d790 con 0x7f148c102b10 2024-09-21T04:42:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.427+0000 7f1483fff640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147807af30 0x7f147807d3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1474000c00 tx=0x7f1474031420 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.431+0000 7f14837fe640 1 -- 172.21.15.7:0/3476010110 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f147c021250 con 0x7f148c102b10 2024-09-21T04:42:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.636+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f148c198da0 con 0x7f147807af30 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.638+0000 7f14837fe640 1 -- 172.21.15.7:0/3476010110 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7f148c198da0 con 0x7f147807af30 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:42:28.641 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "10/23 daemons upgraded", 2024-09-21T04:42:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:42:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:42:28.642 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147807af30 msgr2=0x7f147807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147807af30 0x7f147807d3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1474000c00 tx=0x7f1474031420 comp rx=0 tx=0).stop 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c102b10 msgr2=0x7f148c19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c102b10 0x7f148c19d6b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f147c00b420 tx=0x7f147c00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 shutdown_connections 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147807af30 0x7f147807d3f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c103430 0x7f148c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 --2- 172.21.15.7:0/3476010110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f148c102b10 0x7f148c19d6b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 >> 172.21.15.7:0/3476010110 conn(0x7f148c0fc550 msgr2=0x7f148c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 shutdown_connections 2024-09-21T04:42:28.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.641+0000 7f1490bf0640 1 -- 172.21.15.7:0/3476010110 wait complete. 2024-09-21T04:42:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.726+0000 7f9b30de2640 1 -- 172.21.15.7:0/738314186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c103480 msgr2=0x7f9b2c10b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.726+0000 7f9b30de2640 1 --2- 172.21.15.7:0/738314186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c103480 0x7f9b2c10b970 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009880 tx=0x7f9b1402a530 comp rx=0 tx=0).stop 2024-09-21T04:42:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.726+0000 7f9b30de2640 1 -- 172.21.15.7:0/738314186 shutdown_connections 2024-09-21T04:42:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.726+0000 7f9b30de2640 1 --2- 172.21.15.7:0/738314186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c103480 0x7f9b2c10b970 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.726+0000 7f9b30de2640 1 --2- 172.21.15.7:0/738314186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b2c102b60 0x7f9b2c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.726+0000 7f9b30de2640 1 -- 172.21.15.7:0/738314186 >> 172.21.15.7:0/738314186 conn(0x7f9b2c0fc520 msgr2=0x7f9b2c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.726+0000 7f9b30de2640 1 -- 172.21.15.7:0/738314186 shutdown_connections 2024-09-21T04:42:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.726+0000 7f9b30de2640 1 -- 172.21.15.7:0/738314186 wait complete. 2024-09-21T04:42:28.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.727+0000 7f9b30de2640 1 Processor -- start 2024-09-21T04:42:28.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.727+0000 7f9b30de2640 1 -- start start 2024-09-21T04:42:28.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.727+0000 7f9b30de2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c102b60 0x7f9b2c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.727+0000 7f9b30de2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b2c103480 0x7f9b2c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.727+0000 7f9b30de2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c10edd0 con 0x7f9b2c102b60 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.727+0000 7f9b30de2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c10ec50 con 0x7f9b2c103480 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b2a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c102b60 0x7f9b2c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b2a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c102b60 0x7f9b2c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54848/0 (socket says 172.21.15.7:54848) 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b2a59c640 1 -- 172.21.15.7:0/3946302857 learned_addr learned my addr 172.21.15.7:0/3946302857 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b23fff640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b2c103480 0x7f9b2c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b2a59c640 1 -- 172.21.15.7:0/3946302857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b2c103480 msgr2=0x7f9b2c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b2a59c640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b2c103480 0x7f9b2c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b2a59c640 1 -- 172.21.15.7:0/3946302857 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c1977c0 con 0x7f9b2c102b60 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b2a59c640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c102b60 0x7f9b2c19d640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00d8d0 tx=0x7f9b1c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b237fe640 1 -- 172.21.15.7:0/3946302857 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b1c017070 con 0x7f9b2c102b60 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c197ab0 con 0x7f9b2c102b60 2024-09-21T04:42:28.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b237fe640 1 -- 172.21.15.7:0/3946302857 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b1c010460 con 0x7f9b2c102b60 2024-09-21T04:42:28.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b237fe640 1 -- 172.21.15.7:0/3946302857 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b1c00f5f0 con 0x7f9b2c102b60 2024-09-21T04:42:28.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.728+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c198010 con 0x7f9b2c102b60 2024-09-21T04:42:28.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.730+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2c06a9f0 con 0x7f9b2c102b60 2024-09-21T04:42:28.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.730+0000 7f9b237fe640 1 -- 172.21.15.7:0/3946302857 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9b1c009a30 con 0x7f9b2c102b60 2024-09-21T04:42:28.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.731+0000 7f9b237fe640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9b1807af30 0x7f9b1807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:28.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.731+0000 7f9b237fe640 1 -- 172.21.15.7:0/3946302857 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7f9b1c09dde0 con 0x7f9b2c102b60 2024-09-21T04:42:28.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.731+0000 7f9b23fff640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9b1807af30 0x7f9b1807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:28.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.731+0000 7f9b23fff640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9b1807af30 0x7f9b1807d3f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c198b10 tx=0x7f9b14004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:28.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:28.735+0000 7f9b237fe640 1 -- 172.21.15.7:0/3946302857 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9b1c069430 con 0x7f9b2c102b60 2024-09-21T04:42:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.001+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9b2c105ff0 con 0x7f9b2c102b60 2024-09-21T04:42:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.002+0000 7f9b237fe640 1 -- 172.21.15.7:0/3946302857 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7f9b1c06e2e0 con 0x7f9b2c102b60 2024-09-21T04:42:29.005 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:42:29.005 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:42:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:42:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9b1807af30 msgr2=0x7f9b1807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9b1807af30 0x7f9b1807d3f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c198b10 tx=0x7f9b14004f30 comp rx=0 tx=0).stop 2024-09-21T04:42:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c102b60 msgr2=0x7f9b2c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c102b60 0x7f9b2c19d640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00d8d0 tx=0x7f9b1c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:42:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 shutdown_connections 2024-09-21T04:42:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9b1807af30 0x7f9b1807d3f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b2c103480 0x7f9b2c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 --2- 172.21.15.7:0/3946302857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b2c102b60 0x7f9b2c19d640 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 >> 172.21.15.7:0/3946302857 conn(0x7f9b2c0fc520 msgr2=0x7f9b2c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 shutdown_connections 2024-09-21T04:42:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:29.004+0000 7f9b30de2640 1 -- 172.21.15.7:0/3946302857 wait complete. 2024-09-21T04:42:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:42:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:42:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: pgmap v218: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:42:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1605115548' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/172510764' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:42:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: pgmap v218: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1605115548' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/172510764' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:42:29.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:29.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-21T04:42:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:29 smithi053 ceph-mon[80176]: from='client.34270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:42:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:29 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-21T04:42:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:29 smithi053 ceph-mon[80176]: Upgrade: osd.4 is safe to restart 2024-09-21T04:42:30.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:29 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3946302857' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:42:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:29 smithi007 ceph-mon[108238]: from='client.34270 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:42:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:29 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-21T04:42:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:29 smithi007 ceph-mon[108238]: Upgrade: osd.4 is safe to restart 2024-09-21T04:42:30.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:29 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3946302857' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:42:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:30 smithi053 ceph-mon[80176]: pgmap v219: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:42:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:30 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:30 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T04:42:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:30 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:42:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:30 smithi007 ceph-mon[108238]: pgmap v219: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:42:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T04:42:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:30 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:42:32.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:31 smithi007 ceph-mon[108238]: Upgrade: Updating osd.4 2024-09-21T04:42:32.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:31 smithi007 ceph-mon[108238]: Deploying daemon osd.4 on smithi053 2024-09-21T04:42:32.230 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:31 smithi053 ceph-mon[80176]: Upgrade: Updating osd.4 2024-09-21T04:42:32.230 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:31 smithi053 ceph-mon[80176]: Deploying daemon osd.4 on smithi053 2024-09-21T04:42:33.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:32 smithi053 ceph-mon[80176]: pgmap v220: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:42:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:32 smithi007 ceph-mon[108238]: pgmap v220: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:42:33.500 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:33 smithi053 systemd[1]: Stopping Ceph osd.4 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:42:33.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:33 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[47937]: 2024-09-21T04:42:33.496+0000 7fb5a2699640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:42:33.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:33 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[47937]: 2024-09-21T04:42:33.496+0000 7fb5a2699640 -1 osd.4 66 *** Got signal Terminated *** 2024-09-21T04:42:33.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:33 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[47937]: 2024-09-21T04:42:33.496+0000 7fb5a2699640 -1 osd.4 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:42:34.363 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:33 smithi053 ceph-mon[80176]: osd.4 marked itself down and dead 2024-09-21T04:42:34.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:33 smithi007 ceph-mon[108238]: osd.4 marked itself down and dead 2024-09-21T04:42:34.693 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:34 smithi053 podman[93233]: 2024-09-21 04:42:34.498452154 +0000 UTC m=+1.118552497 container died 4d77c6dc889baa820282052cd51e66269820918574f4924861b9d6590810fdc3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-4e45e6b, maintainer=Guillaume Abrioux ) 2024-09-21T04:42:35.135 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:34 smithi053 ceph-mon[80176]: pgmap v221: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:42:35.135 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:34 smithi053 ceph-mon[80176]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:42:35.135 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:34 smithi053 ceph-mon[80176]: osdmap e67: 6 total, 5 up, 6 in 2024-09-21T04:42:35.136 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:34 smithi053 podman[93233]: 2024-09-21 04:42:34.93967137 +0000 UTC m=+1.559771717 container cleanup 4d77c6dc889baa820282052cd51e66269820918574f4924861b9d6590810fdc3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-4e45e6b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T04:42:35.136 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:34 smithi053 bash[93233]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4 2024-09-21T04:42:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:34 smithi007 ceph-mon[108238]: pgmap v221: 65 pgs: 65 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-21T04:42:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:34 smithi007 ceph-mon[108238]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:42:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:34 smithi007 ceph-mon[108238]: osdmap e67: 6 total, 5 up, 6 in 2024-09-21T04:42:35.442 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:35 smithi053 podman[93250]: 2024-09-21 04:42:35.133177521 +0000 UTC m=+0.629334401 container remove 4d77c6dc889baa820282052cd51e66269820918574f4924861b9d6590810fdc3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-4e45e6b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:42:35.855 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:35 smithi053 podman[93338]: 2024-09-21 04:42:35.567418555 +0000 UTC m=+0.110217726 container create 7f3ae2c25eb47a83b973492b598d77e03d4cc0696e88438c9459ea71672c4a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:42:35.855 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:35 smithi053 podman[93338]: 2024-09-21 04:42:35.474208909 +0000 UTC m=+0.017008047 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:42:35.855 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:35 smithi053 podman[93338]: 2024-09-21 04:42:35.795095575 +0000 UTC m=+0.337894706 container init 7f3ae2c25eb47a83b973492b598d77e03d4cc0696e88438c9459ea71672c4a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731) 2024-09-21T04:42:35.856 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:35 smithi053 podman[93338]: 2024-09-21 04:42:35.798858556 +0000 UTC m=+0.341657684 container start 7f3ae2c25eb47a83b973492b598d77e03d4cc0696e88438c9459ea71672c4a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T04:42:35.856 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:35 smithi053 podman[93338]: 2024-09-21 04:42:35.853328583 +0000 UTC m=+0.396127725 container attach 7f3ae2c25eb47a83b973492b598d77e03d4cc0696e88438c9459ea71672c4a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T04:42:36.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:35 smithi053 ceph-mon[80176]: osdmap e68: 6 total, 5 up, 6 in 2024-09-21T04:42:36.193 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:35 smithi053 podman[93338]: 2024-09-21 04:42:35.994357335 +0000 UTC m=+0.537156464 container died 7f3ae2c25eb47a83b973492b598d77e03d4cc0696e88438c9459ea71672c4a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:42:36.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:35 smithi007 ceph-mon[108238]: osdmap e68: 6 total, 5 up, 6 in 2024-09-21T04:42:36.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:36 smithi053 podman[93361]: 2024-09-21 04:42:36.607496887 +0000 UTC m=+0.606985998 container remove 7f3ae2c25eb47a83b973492b598d77e03d4cc0696e88438c9459ea71672c4a80 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T04:42:36.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:36 smithi053 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.4.service: Deactivated successfully. 2024-09-21T04:42:36.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:36 smithi053 systemd[1]: Stopped Ceph osd.4 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:42:36.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:36 smithi053 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.4.service: Consumed 57.086s CPU time. 2024-09-21T04:42:36.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:36 smithi053 systemd[1]: Starting Ceph osd.4 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:42:37.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:36 smithi053 ceph-mon[80176]: pgmap v224: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:42:37.218 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:36 smithi053 ceph-mon[80176]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-21T04:42:37.218 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 podman[93459]: 2024-09-21 04:42:36.963763829 +0000 UTC m=+0.016863228 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:42:37.218 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 podman[93459]: 2024-09-21 04:42:37.100497615 +0000 UTC m=+0.153597059 container create 5770ef132a10277817b0487241580ca8489b9513bc87c090cc4bd1266e46480d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T04:42:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:36 smithi007 ceph-mon[108238]: pgmap v224: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:42:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:36 smithi007 ceph-mon[108238]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-21T04:42:37.471 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 podman[93459]: 2024-09-21 04:42:37.361331654 +0000 UTC m=+0.414431052 container init 5770ef132a10277817b0487241580ca8489b9513bc87c090cc4bd1266e46480d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T04:42:37.471 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 podman[93459]: 2024-09-21 04:42:37.365176913 +0000 UTC m=+0.418276312 container start 5770ef132a10277817b0487241580ca8489b9513bc87c090cc4bd1266e46480d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:42:37.472 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 podman[93459]: 2024-09-21 04:42:37.469609255 +0000 UTC m=+0.522708656 container attach 5770ef132a10277817b0487241580ca8489b9513bc87c090cc4bd1266e46480d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:42:37.942 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:37.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 bash[93459]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:37.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:37.943 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:37 smithi053 bash[93459]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:38 smithi053 ceph-mon[80176]: pgmap v225: 65 pgs: 2 active+undersized, 19 peering, 2 stale+active+clean, 42 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-09-21T04:42:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:38 smithi007 ceph-mon[108238]: pgmap v225: 65 pgs: 2 active+undersized, 19 peering, 2 stale+active+clean, 42 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-09-21T04:42:40.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-mon[80176]: Health check failed: Degraded data redundancy: 42/333 objects degraded (12.613%), 11 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:40.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:40 smithi007 ceph-mon[108238]: Health check failed: Degraded data redundancy: 42/333 objects degraded (12.613%), 11 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:40.844 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-21T04:42:40.845 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-21T04:42:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:41 smithi053 ceph-mon[80176]: pgmap v226: 65 pgs: 15 active+undersized, 8 peering, 11 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s; 42/333 objects degraded (12.613%) 2024-09-21T04:42:41.193 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-09-21T04:42:41.193 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-09-21T04:42:41.194 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-21T04:42:41.194 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-21T04:42:41.194 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T04:42:41.194 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T04:42:41.194 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate[93473]: --> ceph-volume lvm activate successful for osd ID: 4 2024-09-21T04:42:41.194 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 bash[93459]: --> ceph-volume lvm activate successful for osd ID: 4 2024-09-21T04:42:41.194 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:40 smithi053 podman[93459]: 2024-09-21 04:42:40.87199403 +0000 UTC m=+3.925093439 container died 5770ef132a10277817b0487241580ca8489b9513bc87c090cc4bd1266e46480d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:42:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:41 smithi007 ceph-mon[108238]: pgmap v226: 65 pgs: 15 active+undersized, 8 peering, 11 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s; 42/333 objects degraded (12.613%) 2024-09-21T04:42:41.822 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:41 smithi053 podman[93696]: 2024-09-21 04:42:41.509057896 +0000 UTC m=+0.627048197 container remove 5770ef132a10277817b0487241580ca8489b9513bc87c090cc4bd1266e46480d (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T04:42:41.823 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:41 smithi053 podman[93757]: 2024-09-21 04:42:41.718215518 +0000 UTC m=+0.104774971 container create 632b5d2947ba1362876d08888b629ec9c9358a4f4bdcfde86669c36b496e4ed7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:42:41.823 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:41 smithi053 podman[93757]: 2024-09-21 04:42:41.630096955 +0000 UTC m=+0.016656409 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:42:42.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:42 smithi053 ceph-mon[80176]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-21T04:42:42.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:42 smithi053 podman[93757]: 2024-09-21 04:42:42.078932708 +0000 UTC m=+0.465492156 container init 632b5d2947ba1362876d08888b629ec9c9358a4f4bdcfde86669c36b496e4ed7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T04:42:42.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:42 smithi053 podman[93757]: 2024-09-21 04:42:42.082409368 +0000 UTC m=+0.468968822 container start 632b5d2947ba1362876d08888b629ec9c9358a4f4bdcfde86669c36b496e4ed7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:42:42.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:42 smithi053 ceph-osd[93775]: -- 172.21.15.53:0/3776142736 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e327681e0 con 0x563e31907c00 2024-09-21T04:42:42.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:42 smithi053 bash[93757]: 632b5d2947ba1362876d08888b629ec9c9358a4f4bdcfde86669c36b496e4ed7 2024-09-21T04:42:42.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:42 smithi053 systemd[1]: Started Ceph osd.4 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:42:42.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:42 smithi007 ceph-mon[108238]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-21T04:42:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:43 smithi053 ceph-mon[80176]: pgmap v227: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:42:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:43 smithi007 ceph-mon[108238]: pgmap v227: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:42:43.819 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:43 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T04:42:43.729+0000 7ffa7f39e640 -1 Falling back to public interface 2024-09-21T04:42:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:42:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:44 smithi053 ceph-mon[80176]: pgmap v228: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:42:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:44 smithi007 ceph-mon[108238]: pgmap v228: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:46 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:46 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:46 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:46 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:46.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:46 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:46.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:46 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:46.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:46 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:46.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:46 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:47 smithi007 ceph-mon[108238]: pgmap v229: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:47.424 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:47 smithi053 ceph-mon[80176]: pgmap v229: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:49 smithi053 ceph-mon[80176]: pgmap v230: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:49 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 54/333 objects degraded (16.216%), 15 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:49.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:49 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T04:42:49.155+0000 7ffa7f39e640 -1 osd.4 66 log_to_monitors true 2024-09-21T04:42:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:49 smithi007 ceph-mon[108238]: pgmap v230: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:49 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 54/333 objects degraded (16.216%), 15 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:50.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='osd.4 [v2:172.21.15.53:6808/908167253,v1:172.21.15.53:6809/908167253]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:50.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:50.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-21T04:42:50.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-21T04:42:50.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-21T04:42:50.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:50 smithi007 ceph-mon[108238]: pgmap v231: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:50.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T04:42:50.136+0000 7ffa77148640 -1 osd.4 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:42:50.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:50.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:50.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:42:50.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:42:50.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:50.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='osd.4 [v2:172.21.15.53:6808/908167253,v1:172.21.15.53:6809/908167253]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T04:42:50.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T04:42:50.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:42:50.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:50.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:50.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:42:50.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-21T04:42:50.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-21T04:42:50.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-21T04:42:50.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:50 smithi053 ceph-mon[80176]: pgmap v231: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:51 smithi053 ceph-mon[80176]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T04:42:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:51 smithi053 ceph-mon[80176]: osdmap e69: 6 total, 5 up, 6 in 2024-09-21T04:42:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:51 smithi053 ceph-mon[80176]: from='osd.4 [v2:172.21.15.53:6808/908167253,v1:172.21.15.53:6809/908167253]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:42:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:51 smithi053 ceph-mon[80176]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:42:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:51 smithi007 ceph-mon[108238]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T04:42:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:51 smithi007 ceph-mon[108238]: osdmap e69: 6 total, 5 up, 6 in 2024-09-21T04:42:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:51 smithi007 ceph-mon[108238]: from='osd.4 [v2:172.21.15.53:6808/908167253,v1:172.21.15.53:6809/908167253]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:42:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:51 smithi007 ceph-mon[108238]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:42:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:52 smithi053 ceph-mon[80176]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:42:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:52 smithi053 ceph-mon[80176]: osd.4 [v2:172.21.15.53:6808/908167253,v1:172.21.15.53:6809/908167253] boot 2024-09-21T04:42:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:52 smithi053 ceph-mon[80176]: osdmap e70: 6 total, 6 up, 6 in 2024-09-21T04:42:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:52 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T04:42:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:52 smithi053 ceph-mon[80176]: pgmap v234: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:52 smithi007 ceph-mon[108238]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:42:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:52 smithi007 ceph-mon[108238]: osd.4 [v2:172.21.15.53:6808/908167253,v1:172.21.15.53:6809/908167253] boot 2024-09-21T04:42:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:52 smithi007 ceph-mon[108238]: osdmap e70: 6 total, 6 up, 6 in 2024-09-21T04:42:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:52 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T04:42:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:52 smithi007 ceph-mon[108238]: pgmap v234: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 54/333 objects degraded (16.216%) 2024-09-21T04:42:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:53 smithi007 ceph-mon[108238]: osdmap e71: 6 total, 6 up, 6 in 2024-09-21T04:42:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:53 smithi053 ceph-mon[80176]: osdmap e71: 6 total, 6 up, 6 in 2024-09-21T04:42:54.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:54 smithi053 ceph-mon[80176]: pgmap v236: 65 pgs: 1 peering, 12 active+undersized, 10 active+undersized+degraded, 42 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 39/333 objects degraded (11.712%) 2024-09-21T04:42:54.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:54 smithi007 ceph-mon[108238]: pgmap v236: 65 pgs: 1 peering, 12 active+undersized, 10 active+undersized+degraded, 42 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 39/333 objects degraded (11.712%) 2024-09-21T04:42:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:55 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 39/333 objects degraded (11.712%), 10 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:55 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 39/333 objects degraded (11.712%), 10 pgs degraded (PG_DEGRADED) 2024-09-21T04:42:56.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:56 smithi007 ceph-mon[108238]: pgmap v237: 65 pgs: 1 peering, 64 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-21T04:42:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:56 smithi053 ceph-mon[80176]: pgmap v237: 65 pgs: 1 peering, 64 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-21T04:42:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:57 smithi053 ceph-mon[80176]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 39/333 objects degraded (11.712%), 10 pgs degraded) 2024-09-21T04:42:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:57 smithi007 ceph-mon[108238]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 39/333 objects degraded (11.712%), 10 pgs degraded) 2024-09-21T04:42:58.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:58 smithi053 ceph-mon[80176]: pgmap v238: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-09-21T04:42:58.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:58 smithi007 ceph-mon[108238]: pgmap v238: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-09-21T04:42:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.102+0000 7f5312983640 1 -- 172.21.15.7:0/3680091304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 msgr2=0x7f530c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.102+0000 7f5312983640 1 --2- 172.21.15.7:0/3680091304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 0x7f530c102ea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009880 tx=0x7f52fc02a530 comp rx=0 tx=0).stop 2024-09-21T04:42:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.102+0000 7f5312983640 1 -- 172.21.15.7:0/3680091304 shutdown_connections 2024-09-21T04:42:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.102+0000 7f5312983640 1 --2- 172.21.15.7:0/3680091304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f530c1033e0 0x7f530c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.102+0000 7f5312983640 1 --2- 172.21.15.7:0/3680091304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 0x7f530c102ea0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.102+0000 7f5312983640 1 -- 172.21.15.7:0/3680091304 >> 172.21.15.7:0/3680091304 conn(0x7f530c0fc520 msgr2=0x7f530c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.102+0000 7f5312983640 1 -- 172.21.15.7:0/3680091304 shutdown_connections 2024-09-21T04:42:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.102+0000 7f5312983640 1 -- 172.21.15.7:0/3680091304 wait complete. 2024-09-21T04:42:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f5312983640 1 Processor -- start 2024-09-21T04:42:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f5312983640 1 -- start start 2024-09-21T04:42:59.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f5312983640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 0x7f530c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f5312983640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f530c1033e0 0x7f530c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f5312983640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f530c10edf0 con 0x7f530c102ac0 2024-09-21T04:42:59.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f5312983640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f530c10ec70 con 0x7f530c1033e0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f530bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 0x7f530c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f530bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 0x7f530c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36262/0 (socket says 172.21.15.7:36262) 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.103+0000 7f530bfff640 1 -- 172.21.15.7:0/283425190 learned_addr learned my addr 172.21.15.7:0/283425190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f530b7fe640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f530c1033e0 0x7f530c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f530bfff640 1 -- 172.21.15.7:0/283425190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f530c1033e0 msgr2=0x7f530c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f530bfff640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f530c1033e0 0x7f530c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f530bfff640 1 -- 172.21.15.7:0/283425190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c1977c0 con 0x7f530c102ac0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f530bfff640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 0x7f530c19d640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52fc031460 tx=0x7f52fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f53097fa640 1 -- 172.21.15.7:0/283425190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52fc03f070 con 0x7f530c102ac0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530c197a50 con 0x7f530c102ac0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f53097fa640 1 -- 172.21.15.7:0/283425190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52fc0035f0 con 0x7f530c102ac0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f53097fa640 1 -- 172.21.15.7:0/283425190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52fc035030 con 0x7f530c102ac0 2024-09-21T04:42:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.104+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530c197f80 con 0x7f530c102ac0 2024-09-21T04:42:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.106+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f530c06aa90 con 0x7f530c102ac0 2024-09-21T04:42:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.106+0000 7f53097fa640 1 -- 172.21.15.7:0/283425190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f52fc042070 con 0x7f530c102ac0 2024-09-21T04:42:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.107+0000 7f53097fa640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52f407af30 0x7f52f407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.107+0000 7f53097fa640 1 -- 172.21.15.7:0/283425190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6070+0+0 (secure 0 0 0) 0x7f52fc0be9d0 con 0x7f530c102ac0 2024-09-21T04:42:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.107+0000 7f530b7fe640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52f407af30 0x7f52f407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.107+0000 7f530b7fe640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52f407af30 0x7f52f407d3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53000058c0 tx=0x7f5300005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.111+0000 7f53097fa640 1 -- 172.21.15.7:0/283425190 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52fc08a020 con 0x7f530c102ac0 2024-09-21T04:42:59.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.316+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f530c1982f0 con 0x7f52f407af30 2024-09-21T04:42:59.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.318+0000 7f53097fa640 1 -- 172.21.15.7:0/283425190 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7f530c1982f0 con 0x7f52f407af30 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52f407af30 msgr2=0x7f52f407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52f407af30 0x7f52f407d3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53000058c0 tx=0x7f5300005850 comp rx=0 tx=0).stop 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 msgr2=0x7f530c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 0x7f530c19d640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52fc031460 tx=0x7f52fc0045e0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 shutdown_connections 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52f407af30 0x7f52f407d3f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f530c1033e0 0x7f530c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 --2- 172.21.15.7:0/283425190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f530c102ac0 0x7f530c19d640 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 >> 172.21.15.7:0/283425190 conn(0x7f530c0fc520 msgr2=0x7f530c10af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:59.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 shutdown_connections 2024-09-21T04:42:59.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.320+0000 7f5312983640 1 -- 172.21.15.7:0/283425190 wait complete. 2024-09-21T04:42:59.332 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:42:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.406+0000 7fb79b720640 1 -- 172.21.15.7:0/186962744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 msgr2=0x7fb794100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.406+0000 7fb79b720640 1 --2- 172.21.15.7:0/186962744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 0x7fb794100c90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb790009880 tx=0x7fb79002a530 comp rx=0 tx=0).stop 2024-09-21T04:42:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.406+0000 7fb79b720640 1 -- 172.21.15.7:0/186962744 shutdown_connections 2024-09-21T04:42:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.406+0000 7fb79b720640 1 --2- 172.21.15.7:0/186962744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7941011d0 0x7fb7941056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.406+0000 7fb79b720640 1 --2- 172.21.15.7:0/186962744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 0x7fb794100c90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.406+0000 7fb79b720640 1 -- 172.21.15.7:0/186962744 >> 172.21.15.7:0/186962744 conn(0x7fb7940fc520 msgr2=0x7fb7940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.406+0000 7fb79b720640 1 -- 172.21.15.7:0/186962744 shutdown_connections 2024-09-21T04:42:59.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.406+0000 7fb79b720640 1 -- 172.21.15.7:0/186962744 wait complete. 2024-09-21T04:42:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb79b720640 1 Processor -- start 2024-09-21T04:42:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb79b720640 1 -- start start 2024-09-21T04:42:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb79b720640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 0x7fb79419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb79b720640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7941011d0 0x7fb79419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb79b720640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb794115450 con 0x7fb7941008b0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb79b720640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb7941152d0 con 0x7fb7941011d0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb7994bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 0x7fb79419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb7994bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 0x7fb79419e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36270/0 (socket says 172.21.15.7:36270) 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb7994bc640 1 -- 172.21.15.7:0/1804128942 learned_addr learned my addr 172.21.15.7:0/1804128942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb7994bc640 1 -- 172.21.15.7:0/1804128942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7941011d0 msgr2=0x7fb79419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb798cbb640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7941011d0 0x7fb79419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb7994bc640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7941011d0 0x7fb79419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb7994bc640 1 -- 172.21.15.7:0/1804128942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7941a2a00 con 0x7fb7941008b0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.407+0000 7fb798cbb640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7941011d0 0x7fb79419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.408+0000 7fb7994bc640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 0x7fb79419e130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb790031460 tx=0x7fb7900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.408+0000 7fb78a7fc640 1 -- 172.21.15.7:0/1804128942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb79003f070 con 0x7fb7941008b0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.408+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7941a2c90 con 0x7fb7941008b0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.408+0000 7fb78a7fc640 1 -- 172.21.15.7:0/1804128942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb790004cf0 con 0x7fb7941008b0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.408+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7941a31c0 con 0x7fb7941008b0 2024-09-21T04:42:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.408+0000 7fb78a7fc640 1 -- 172.21.15.7:0/1804128942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb790035030 con 0x7fb7941008b0 2024-09-21T04:42:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.409+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb778005290 con 0x7fb7941008b0 2024-09-21T04:42:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.410+0000 7fb78a7fc640 1 -- 172.21.15.7:0/1804128942 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb790042070 con 0x7fb7941008b0 2024-09-21T04:42:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.410+0000 7fb78a7fc640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb78007af30 0x7fb78007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.410+0000 7fb78a7fc640 1 -- 172.21.15.7:0/1804128942 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6070+0+0 (secure 0 0 0) 0x7fb7900beaa0 con 0x7fb7941008b0 2024-09-21T04:42:59.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.410+0000 7fb798cbb640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb78007af30 0x7fb78007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.411+0000 7fb798cbb640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb78007af30 0x7fb78007d3f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb784006fd0 tx=0x7fb784008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:59.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.415+0000 7fb78a7fc640 1 -- 172.21.15.7:0/1804128942 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb79008a0f0 con 0x7fb7941008b0 2024-09-21T04:42:59.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.620+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb778000bc0 con 0x7fb78007af30 2024-09-21T04:42:59.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:59.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:42:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:42:59.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.622+0000 7fb78a7fc640 1 -- 172.21.15.7:0/1804128942 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7fb778000bc0 con 0x7fb78007af30 2024-09-21T04:42:59.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb78007af30 msgr2=0x7fb78007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb78007af30 0x7fb78007d3f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb784006fd0 tx=0x7fb784008040 comp rx=0 tx=0).stop 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 msgr2=0x7fb79419e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 0x7fb79419e130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb790031460 tx=0x7fb7900048f0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 shutdown_connections 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb78007af30 0x7fb78007d3f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7941011d0 0x7fb79419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 --2- 172.21.15.7:0/1804128942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7941008b0 0x7fb79419e130 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.625+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 >> 172.21.15.7:0/1804128942 conn(0x7fb7940fc520 msgr2=0x7fb794112b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.626+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 shutdown_connections 2024-09-21T04:42:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.626+0000 7fb79b720640 1 -- 172.21.15.7:0/1804128942 wait complete. 2024-09-21T04:42:59.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.710+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/2837552140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8101900 msgr2=0x7fa6d8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.710+0000 7fa6dfa0f640 1 --2- 172.21.15.7:0/2837552140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8101900 0x7fa6d8101d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009880 tx=0x7fa6c802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.710+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/2837552140 shutdown_connections 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.710+0000 7fa6dfa0f640 1 --2- 172.21.15.7:0/2837552140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8101900 0x7fa6d8101d60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.710+0000 7fa6dfa0f640 1 --2- 172.21.15.7:0/2837552140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d8107900 0x7fa6d8107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.710+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/2837552140 >> 172.21.15.7:0/2837552140 conn(0x7fa6d80fd600 msgr2=0x7fa6d80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.710+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/2837552140 shutdown_connections 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.710+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/2837552140 wait complete. 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dfa0f640 1 Processor -- start 2024-09-21T04:42:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dfa0f640 1 -- start start 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dfa0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d8101900 0x7fa6d819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dfa0f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8107900 0x7fa6d819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dfa0f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d810ee30 con 0x7fa6d8107900 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dfa0f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d810ecb0 con 0x7fa6d8101900 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dcfaa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8107900 0x7fa6d819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dcfaa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8107900 0x7fa6d819fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36290/0 (socket says 172.21.15.7:36290) 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dcfaa640 1 -- 172.21.15.7:0/1552824103 learned_addr learned my addr 172.21.15.7:0/1552824103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.711+0000 7fa6dd7ab640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d8101900 0x7fa6d819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6dcfaa640 1 -- 172.21.15.7:0/1552824103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d8101900 msgr2=0x7fa6d819f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6dcfaa640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d8101900 0x7fa6d819f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6dcfaa640 1 -- 172.21.15.7:0/1552824103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d8199a90 con 0x7fa6d8107900 2024-09-21T04:42:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6dcfaa640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8107900 0x7fa6d819fe10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8031460 tx=0x7fa6c80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6ce7fc640 1 -- 172.21.15.7:0/1552824103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa6c803f070 con 0x7fa6d8107900 2024-09-21T04:42:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d8199d20 con 0x7fa6d8107900 2024-09-21T04:42:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6ce7fc640 1 -- 172.21.15.7:0/1552824103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6c8004cf0 con 0x7fa6d8107900 2024-09-21T04:42:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6ce7fc640 1 -- 172.21.15.7:0/1552824103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa6c8035030 con 0x7fa6d8107900 2024-09-21T04:42:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.712+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d819a250 con 0x7fa6d8107900 2024-09-21T04:42:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.713+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d8103040 con 0x7fa6d8107900 2024-09-21T04:42:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.714+0000 7fa6ce7fc640 1 -- 172.21.15.7:0/1552824103 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa6c8042070 con 0x7fa6d8107900 2024-09-21T04:42:59.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.714+0000 7fa6ce7fc640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6c407af30 0x7fa6c407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:42:59.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.714+0000 7fa6dd7ab640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6c407af30 0x7fa6c407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:42:59.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.714+0000 7fa6ce7fc640 1 -- 172.21.15.7:0/1552824103 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6070+0+0 (secure 0 0 0) 0x7fa6c80beab0 con 0x7fa6d8107900 2024-09-21T04:42:59.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.715+0000 7fa6dd7ab640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6c407af30 0x7fa6c407d3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa6d819abd0 tx=0x7fa6d4008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:42:59.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.719+0000 7fa6ce7fc640 1 -- 172.21.15.7:0/1552824103 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa6c808a100 con 0x7fa6d8107900 2024-09-21T04:42:59.793 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:42:59.793 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:42:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:42:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.915+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa6d819b060 con 0x7fa6c407af30 2024-09-21T04:42:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.926+0000 7fa6ce7fc640 1 -- 172.21.15.7:0/1552824103 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fa6d819b060 con 0x7fa6c407af30 2024-09-21T04:42:59.929 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:42:59.929 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (5m) 85s ago 15m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:42:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (8m) 85s ago 16m 9013k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:42:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (7m) 14s ago 15m 8975k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:42:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (3m) 85s ago 16m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:42:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (3m) 14s ago 15m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:42:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (4m) 85s ago 15m 77.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:42:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (10m) 85s ago 10m 144M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:42:59.931 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (10m) 85s ago 10m 89.8M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:42:59.931 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (10m) 14s ago 10m 26.1M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:42:59.931 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (10m) 14s ago 10m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:42:59.931 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (7m) 85s ago 17m 528M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:42:59.931 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (6m) 14s ago 15m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:42:59.931 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (3m) 85s ago 17m 43.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:42:59.931 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (3m) 14s ago 15m 37.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:42:59.932 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (5m) 85s ago 16m 10.7M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:42:59.932 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (5m) 14s ago 15m 10.2M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:42:59.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (2m) 85s ago 14m 86.1M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:42:59.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (2m) 85s ago 13m 90.7M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:42:59.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (91s) 85s ago 12m 14.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:42:59.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (53s) 14s ago 12m 93.2M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:42:59.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (18s) 14s ago 12m 12.2M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:42:59.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (11m) 14s ago 11m 186M 2241M 19.1.1-373-g4e45e6b4 164465d2334a 0eda65c5b4fd 2024-09-21T04:42:59.933 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (5m) 85s ago 15m 56.1M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6c407af30 msgr2=0x7fa6c407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6c407af30 0x7fa6c407d3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa6d819abd0 tx=0x7fa6d4008040 comp rx=0 tx=0).stop 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8107900 msgr2=0x7fa6d819fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8107900 0x7fa6d819fe10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8031460 tx=0x7fa6c80048f0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 shutdown_connections 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6c407af30 0x7fa6c407d3f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6d8107900 0x7fa6d819fe10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 --2- 172.21.15.7:0/1552824103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6d8101900 0x7fa6d819f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:42:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 >> 172.21.15.7:0/1552824103 conn(0x7fa6d80fd600 msgr2=0x7fa6d8074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:42:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 shutdown_connections 2024-09-21T04:42:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:42:59.928+0000 7fa6dfa0f640 1 -- 172.21.15.7:0/1552824103 wait complete. 2024-09-21T04:43:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.012+0000 7f31f520b640 1 -- 172.21.15.7:0/620118504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0107900 msgr2=0x7f31f0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.012+0000 7f31f520b640 1 --2- 172.21.15.7:0/620118504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0107900 0x7f31f0107ce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009880 tx=0x7f31e002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.012+0000 7f31f520b640 1 -- 172.21.15.7:0/620118504 shutdown_connections 2024-09-21T04:43:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.012+0000 7f31f520b640 1 --2- 172.21.15.7:0/620118504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0101900 0x7f31f0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.012+0000 7f31f520b640 1 --2- 172.21.15.7:0/620118504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0107900 0x7f31f0107ce0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.012+0000 7f31f520b640 1 -- 172.21.15.7:0/620118504 >> 172.21.15.7:0/620118504 conn(0x7f31f00fd600 msgr2=0x7f31f00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.012+0000 7f31f520b640 1 -- 172.21.15.7:0/620118504 shutdown_connections 2024-09-21T04:43:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.012+0000 7f31f520b640 1 -- 172.21.15.7:0/620118504 wait complete. 2024-09-21T04:43:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.013+0000 7f31f520b640 1 Processor -- start 2024-09-21T04:43:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.013+0000 7f31f520b640 1 -- start start 2024-09-21T04:43:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.013+0000 7f31f520b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0101900 0x7f31f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.013+0000 7f31f520b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107900 0x7f31f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.013+0000 7f31f520b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31f010edb0 con 0x7f31f0101900 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.013+0000 7f31f520b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f31f010ec30 con 0x7f31f0107900 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31eed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0101900 0x7f31f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31eed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0101900 0x7f31f00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36318/0 (socket says 172.21.15.7:36318) 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31eed9d640 1 -- 172.21.15.7:0/3227262821 learned_addr learned my addr 172.21.15.7:0/3227262821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31ee59c640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107900 0x7f31f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31eed9d640 1 -- 172.21.15.7:0/3227262821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107900 msgr2=0x7f31f0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31eed9d640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107900 0x7f31f0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31eed9d640 1 -- 172.21.15.7:0/3227262821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f0074f70 con 0x7f31f0101900 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31eed9d640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0101900 0x7f31f00744f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f31e0031460 tx=0x7f31e00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31d7fff640 1 -- 172.21.15.7:0/3227262821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f31e003f070 con 0x7f31f0101900 2024-09-21T04:43:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f0075200 con 0x7f31f0101900 2024-09-21T04:43:00.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31d7fff640 1 -- 172.21.15.7:0/3227262821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f31e0004b10 con 0x7f31f0101900 2024-09-21T04:43:00.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31d7fff640 1 -- 172.21.15.7:0/3227262821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f31e0035030 con 0x7f31f0101900 2024-09-21T04:43:00.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.014+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f01a3d80 con 0x7f31f0101900 2024-09-21T04:43:00.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.015+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f0103040 con 0x7f31f0101900 2024-09-21T04:43:00.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.016+0000 7f31d7fff640 1 -- 172.21.15.7:0/3227262821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f31e0042070 con 0x7f31f0101900 2024-09-21T04:43:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.017+0000 7f31d7fff640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d807af40 0x7f31d807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.017+0000 7f31d7fff640 1 -- 172.21.15.7:0/3227262821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6070+0+0 (secure 0 0 0) 0x7f31e00bea50 con 0x7f31f0101900 2024-09-21T04:43:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.017+0000 7f31ee59c640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d807af40 0x7f31d807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.017+0000 7f31ee59c640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d807af40 0x7f31d807d400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31f0075f80 tx=0x7f31e400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.021+0000 7f31d7fff640 1 -- 172.21.15.7:0/3227262821 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31e008a0a0 con 0x7f31f0101900 2024-09-21T04:43:00.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.292+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f31f002d7c0 con 0x7f31f0101900 2024-09-21T04:43:00.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.292+0000 7f31d7fff640 1 -- 172.21.15.7:0/3227262821 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7f31e008c5c0 con 0x7f31f0101900 2024-09-21T04:43:00.296 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:43:00.296 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:43:00.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:43:00.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:43:00.296 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:43:00.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:43:00.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 5 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 4 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:43:00.297 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 5, 2024-09-21T04:43:00.298 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 9 2024-09-21T04:43:00.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:43:00.298 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:43:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d807af40 msgr2=0x7f31d807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d807af40 0x7f31d807d400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31f0075f80 tx=0x7f31e400b390 comp rx=0 tx=0).stop 2024-09-21T04:43:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0101900 msgr2=0x7f31f00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0101900 0x7f31f00744f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f31e0031460 tx=0x7f31e00045e0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 shutdown_connections 2024-09-21T04:43:00.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d807af40 0x7f31d807d400 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31f0107900 0x7f31f0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 --2- 172.21.15.7:0/3227262821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f0101900 0x7f31f00744f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 >> 172.21.15.7:0/3227262821 conn(0x7f31f00fd600 msgr2=0x7f31f00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 shutdown_connections 2024-09-21T04:43:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.295+0000 7f31f520b640 1 -- 172.21.15.7:0/3227262821 wait complete. 2024-09-21T04:43:00.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.380+0000 7fb83015b640 1 -- 172.21.15.7:0/1755234458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828107900 msgr2=0x7fb828107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.380+0000 7fb83015b640 1 --2- 172.21.15.7:0/1755234458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828107900 0x7fb828107ce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb824009880 tx=0x7fb82402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.380+0000 7fb83015b640 1 -- 172.21.15.7:0/1755234458 shutdown_connections 2024-09-21T04:43:00.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.380+0000 7fb83015b640 1 --2- 172.21.15.7:0/1755234458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828101900 0x7fb828101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.380+0000 7fb83015b640 1 --2- 172.21.15.7:0/1755234458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828107900 0x7fb828107ce0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.380+0000 7fb83015b640 1 -- 172.21.15.7:0/1755234458 >> 172.21.15.7:0/1755234458 conn(0x7fb8280fd600 msgr2=0x7fb8280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:00.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.380+0000 7fb83015b640 1 -- 172.21.15.7:0/1755234458 shutdown_connections 2024-09-21T04:43:00.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.380+0000 7fb83015b640 1 -- 172.21.15.7:0/1755234458 wait complete. 2024-09-21T04:43:00.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb83015b640 1 Processor -- start 2024-09-21T04:43:00.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb83015b640 1 -- start start 2024-09-21T04:43:00.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb83015b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828101900 0x7fb82819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb83015b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828107900 0x7fb82819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb83015b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb82810edd0 con 0x7fb828101900 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb83015b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb82810ec50 con 0x7fb828107900 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb82def7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828101900 0x7fb82819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb82def7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828101900 0x7fb82819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36338/0 (socket says 172.21.15.7:36338) 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.381+0000 7fb82def7640 1 -- 172.21.15.7:0/1590350892 learned_addr learned my addr 172.21.15.7:0/1590350892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb82def7640 1 -- 172.21.15.7:0/1590350892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828107900 msgr2=0x7fb82819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb82d6f6640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828107900 0x7fb82819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb82def7640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828107900 0x7fb82819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb82def7640 1 -- 172.21.15.7:0/1590350892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8281977a0 con 0x7fb828101900 2024-09-21T04:43:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb82def7640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828101900 0x7fb82819d620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb824031460 tx=0x7fb8240045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb81effd640 1 -- 172.21.15.7:0/1590350892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb82403f070 con 0x7fb828101900 2024-09-21T04:43:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb81effd640 1 -- 172.21.15.7:0/1590350892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb8240035f0 con 0x7fb828101900 2024-09-21T04:43:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb828197a30 con 0x7fb828101900 2024-09-21T04:43:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb81effd640 1 -- 172.21.15.7:0/1590350892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb824035030 con 0x7fb828101900 2024-09-21T04:43:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.382+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb828197f60 con 0x7fb828101900 2024-09-21T04:43:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.383+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb828103040 con 0x7fb828101900 2024-09-21T04:43:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.384+0000 7fb81effd640 1 -- 172.21.15.7:0/1590350892 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb824042070 con 0x7fb828101900 2024-09-21T04:43:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.384+0000 7fb81effd640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81407af30 0x7fb81407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.385+0000 7fb82d6f6640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81407af30 0x7fb81407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.385+0000 7fb81effd640 1 -- 172.21.15.7:0/1590350892 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6070+0+0 (secure 0 0 0) 0x7fb8240bea70 con 0x7fb828101900 2024-09-21T04:43:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.385+0000 7fb82d6f6640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81407af30 0x7fb81407d3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb818005fd0 tx=0x7fb818005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:00.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.389+0000 7fb81effd640 1 -- 172.21.15.7:0/1590350892 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb82408a020 con 0x7fb828101900 2024-09-21T04:43:00.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.613+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb828109e70 con 0x7fb828101900 2024-09-21T04:43:00.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:00 smithi007 ceph-mon[108238]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:00.616 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:00 smithi007 ceph-mon[108238]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:00.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:00 smithi007 ceph-mon[108238]: pgmap v239: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:43:00.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:00 smithi007 ceph-mon[108238]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:00.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:00 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3227262821' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:00.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.614+0000 7fb81effd640 1 -- 172.21.15.7:0/1590350892 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2038 (secure 0 0 0) 0x7fb82408c020 con 0x7fb828101900 2024-09-21T04:43:00.617 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-09-21T04:43:00.617 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:32:33:892197+0000 2024-09-21T04:43:00.617 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:32:33.892184+0000 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:43:00.618 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 44 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14578,1=24327} 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:43:00.619 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14578 members: 24327,14578 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{0:14578} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/406067991,v1:172.21.15.7:6829/406067991] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:14594} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3426051215,v1:172.21.15.7:6827/3426051215] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{1:24327} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.53:6824/2105303008,v1:172.21.15.53:6825/2105303008] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:24345} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.53:6826/2642617317,v1:172.21.15.53:6827/2642617317] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:43:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:43:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-21T04:43:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81407af30 msgr2=0x7fb81407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81407af30 0x7fb81407d3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb818005fd0 tx=0x7fb818005850 comp rx=0 tx=0).stop 2024-09-21T04:43:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828101900 msgr2=0x7fb82819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828101900 0x7fb82819d620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb824031460 tx=0x7fb8240045e0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 shutdown_connections 2024-09-21T04:43:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81407af30 0x7fb81407d3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb828107900 0x7fb82819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 --2- 172.21.15.7:0/1590350892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb828101900 0x7fb82819d620 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 >> 172.21.15.7:0/1590350892 conn(0x7fb8280fd600 msgr2=0x7fb8280fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 shutdown_connections 2024-09-21T04:43:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.616+0000 7fb83015b640 1 -- 172.21.15.7:0/1590350892 wait complete. 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.701+0000 7fc26df53640 1 -- 172.21.15.7:0/2858137547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 msgr2=0x7fc268101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.701+0000 7fc26df53640 1 --2- 172.21.15.7:0/2858137547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 0x7fc268101d60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009880 tx=0x7fc25c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.701+0000 7fc26df53640 1 -- 172.21.15.7:0/2858137547 shutdown_connections 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.701+0000 7fc26df53640 1 --2- 172.21.15.7:0/2858137547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 0x7fc268101d60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.701+0000 7fc26df53640 1 --2- 172.21.15.7:0/2858137547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268107900 0x7fc268107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.701+0000 7fc26df53640 1 -- 172.21.15.7:0/2858137547 >> 172.21.15.7:0/2858137547 conn(0x7fc2680fd600 msgr2=0x7fc2680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.701+0000 7fc26df53640 1 -- 172.21.15.7:0/2858137547 shutdown_connections 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.701+0000 7fc26df53640 1 -- 172.21.15.7:0/2858137547 wait complete. 2024-09-21T04:43:00.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc26df53640 1 Processor -- start 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc26df53640 1 -- start start 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc26df53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 0x7fc26819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc26df53640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268107900 0x7fc26819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc26df53640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc26810ee30 con 0x7fc268101900 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc26df53640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc26810ecb0 con 0x7fc268107900 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 0x7fc26819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc2677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 0x7fc26819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36352/0 (socket says 172.21.15.7:36352) 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.702+0000 7fc2677fe640 1 -- 172.21.15.7:0/708268686 learned_addr learned my addr 172.21.15.7:0/708268686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc266ffd640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268107900 0x7fc26819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc2677fe640 1 -- 172.21.15.7:0/708268686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268107900 msgr2=0x7fc26819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc2677fe640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268107900 0x7fc26819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc2677fe640 1 -- 172.21.15.7:0/708268686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc268197840 con 0x7fc268101900 2024-09-21T04:43:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc2677fe640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 0x7fc26819d6c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc25800b420 tx=0x7fc25800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc264ff9640 1 -- 172.21.15.7:0/708268686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc258011630 con 0x7fc268101900 2024-09-21T04:43:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc264ff9640 1 -- 172.21.15.7:0/708268686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc258016070 con 0x7fc268101900 2024-09-21T04:43:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc268197b30 con 0x7fc268101900 2024-09-21T04:43:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc264ff9640 1 -- 172.21.15.7:0/708268686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc258010d40 con 0x7fc268101900 2024-09-21T04:43:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.703+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc268198090 con 0x7fc268101900 2024-09-21T04:43:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.704+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc268103040 con 0x7fc268101900 2024-09-21T04:43:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.705+0000 7fc264ff9640 1 -- 172.21.15.7:0/708268686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc25801b460 con 0x7fc268101900 2024-09-21T04:43:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.706+0000 7fc264ff9640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25007af80 0x7fc25007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.706+0000 7fc266ffd640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25007af80 0x7fc25007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.706+0000 7fc264ff9640 1 -- 172.21.15.7:0/708268686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6070+0+0 (secure 0 0 0) 0x7fc25809d6a0 con 0x7fc268101900 2024-09-21T04:43:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.706+0000 7fc266ffd640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25007af80 0x7fc25007d440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc268198b90 tx=0x7fc25c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.710+0000 7fc264ff9640 1 -- 172.21.15.7:0/708268686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc258021250 con 0x7fc268101900 2024-09-21T04:43:00.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.917+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc268198400 con 0x7fc25007af80 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.918+0000 7fc264ff9640 1 -- 172.21.15.7:0/708268686 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7fc268198400 con 0x7fc25007af80 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:43:00.921 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "11/23 daemons upgraded", 2024-09-21T04:43:00.922 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-09-21T04:43:00.922 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:43:00.922 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25007af80 msgr2=0x7fc25007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25007af80 0x7fc25007d440 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc268198b90 tx=0x7fc25c031400 comp rx=0 tx=0).stop 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 msgr2=0x7fc26819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 0x7fc26819d6c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc25800b420 tx=0x7fc25800b8e0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 shutdown_connections 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc25007af80 0x7fc25007d440 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc268107900 0x7fc26819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 --2- 172.21.15.7:0/708268686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc268101900 0x7fc26819d6c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.921+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 >> 172.21.15.7:0/708268686 conn(0x7fc2680fd600 msgr2=0x7fc2680fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.922+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 shutdown_connections 2024-09-21T04:43:00.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:00.922+0000 7fc26df53640 1 -- 172.21.15.7:0/708268686 wait complete. 2024-09-21T04:43:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:00 smithi053 ceph-mon[80176]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:00 smithi053 ceph-mon[80176]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:00 smithi053 ceph-mon[80176]: pgmap v239: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-21T04:43:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:00 smithi053 ceph-mon[80176]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:00 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3227262821' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.006+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/1961255984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 msgr2=0x7f4ce0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.006+0000 7f4ce6bc8640 1 --2- 172.21.15.7:0/1961255984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 0x7f4ce0100c60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009880 tx=0x7f4ccc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:43:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.006+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/1961255984 shutdown_connections 2024-09-21T04:43:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.006+0000 7f4ce6bc8640 1 --2- 172.21.15.7:0/1961255984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 0x7f4ce0100c60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:01.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.006+0000 7f4ce6bc8640 1 --2- 172.21.15.7:0/1961255984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ce0106800 0x7f4ce0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.006+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/1961255984 >> 172.21.15.7:0/1961255984 conn(0x7f4ce00fc520 msgr2=0x7f4ce00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.006+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/1961255984 shutdown_connections 2024-09-21T04:43:01.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.006+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/1961255984 wait complete. 2024-09-21T04:43:01.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce6bc8640 1 Processor -- start 2024-09-21T04:43:01.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce6bc8640 1 -- start start 2024-09-21T04:43:01.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce6bc8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 0x7f4ce019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:01.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce6bc8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ce0106800 0x7f4ce019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:01.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce6bc8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce010dcd0 con 0x7f4ce0100800 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce6bc8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce010db50 con 0x7f4ce0106800 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce4964640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 0x7f4ce019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce4964640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 0x7f4ce019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36368/0 (socket says 172.21.15.7:36368) 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4ce4964640 1 -- 172.21.15.7:0/876215592 learned_addr learned my addr 172.21.15.7:0/876215592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.007+0000 7f4cdffff640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ce0106800 0x7f4ce019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4ce4964640 1 -- 172.21.15.7:0/876215592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ce0106800 msgr2=0x7f4ce019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4ce4964640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ce0106800 0x7f4ce019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4ce4964640 1 -- 172.21.15.7:0/876215592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce01966e0 con 0x7f4ce0100800 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4cdffff640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ce0106800 0x7f4ce019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4ce4964640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 0x7f4ce019c610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e970 tx=0x7f4cd400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4cddffb640 1 -- 172.21.15.7:0/876215592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4cd400cac0 con 0x7f4ce0100800 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4cddffb640 1 -- 172.21.15.7:0/876215592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4cd40054e0 con 0x7f4ce0100800 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce01969d0 con 0x7f4ce0100800 2024-09-21T04:43:01.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4cddffb640 1 -- 172.21.15.7:0/876215592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4cd4010430 con 0x7f4ce0100800 2024-09-21T04:43:01.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.008+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce0196f30 con 0x7f4ce0100800 2024-09-21T04:43:01.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.009+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce0101f40 con 0x7f4ce0100800 2024-09-21T04:43:01.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.010+0000 7f4cddffb640 1 -- 172.21.15.7:0/876215592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4cd4004ec0 con 0x7f4ce0100800 2024-09-21T04:43:01.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.011+0000 7f4cddffb640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4cd007af30 0x7f4cd007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:01.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.011+0000 7f4cdffff640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4cd007af30 0x7f4cd007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:01.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.011+0000 7f4cddffb640 1 -- 172.21.15.7:0/876215592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6070+0+0 (secure 0 0 0) 0x7f4cd409e7a0 con 0x7f4ce0100800 2024-09-21T04:43:01.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.011+0000 7f4cdffff640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4cd007af30 0x7f4cd007d3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0197f30 tx=0x7f4ccc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.015+0000 7f4cddffb640 1 -- 172.21.15.7:0/876215592 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4cd4069d50 con 0x7f4ce0100800 2024-09-21T04:43:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.276+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f4ce0108d70 con 0x7f4ce0100800 2024-09-21T04:43:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.277+0000 7f4cddffb640 1 -- 172.21.15.7:0/876215592 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7f4cd406ec00 con 0x7f4ce0100800 2024-09-21T04:43:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T04:43:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:43:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:43:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4cd007af30 msgr2=0x7f4cd007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4cd007af30 0x7f4cd007d3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0197f30 tx=0x7f4ccc031400 comp rx=0 tx=0).stop 2024-09-21T04:43:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 msgr2=0x7f4ce019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 0x7f4ce019c610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e970 tx=0x7f4cd400ed70 comp rx=0 tx=0).stop 2024-09-21T04:43:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 shutdown_connections 2024-09-21T04:43:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4cd007af30 0x7f4cd007d3f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ce0106800 0x7f4ce019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 --2- 172.21.15.7:0/876215592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ce0100800 0x7f4ce019c610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 >> 172.21.15.7:0/876215592 conn(0x7f4ce00fc520 msgr2=0x7f4ce00fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 shutdown_connections 2024-09-21T04:43:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:01.279+0000 7f4ce6bc8640 1 -- 172.21.15.7:0/876215592 wait complete. 2024-09-21T04:43:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:01 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1590350892' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:43:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:01 smithi007 ceph-mon[108238]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:01 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/876215592' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:43:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:01 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1590350892' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:43:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:01 smithi053 ceph-mon[80176]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:01.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:01 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/876215592' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:43:02.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:02 smithi053 ceph-mon[80176]: pgmap v240: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-21T04:43:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:02 smithi007 ceph-mon[108238]: pgmap v240: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-21T04:43:04.861 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:04 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:04.861 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:04 smithi053 ceph-mon[80176]: pgmap v241: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-21T04:43:04.861 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:04 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-21T04:43:04.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:04 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:04.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:04 smithi007 ceph-mon[108238]: pgmap v241: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-21T04:43:04.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:04 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-21T04:43:05.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:05 smithi053 ceph-mon[80176]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-21T04:43:05.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:05 smithi053 ceph-mon[80176]: Upgrade: osd.5 is safe to restart 2024-09-21T04:43:05.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:05 smithi007 ceph-mon[108238]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-21T04:43:05.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:05 smithi007 ceph-mon[108238]: Upgrade: osd.5 is safe to restart 2024-09-21T04:43:07.189 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:06 smithi053 ceph-mon[80176]: Upgrade: Updating osd.5 2024-09-21T04:43:07.189 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:06 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:07.189 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:06 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T04:43:07.189 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:06 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:07.189 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:06 smithi053 ceph-mon[80176]: Deploying daemon osd.5 on smithi053 2024-09-21T04:43:07.189 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:06 smithi053 ceph-mon[80176]: pgmap v242: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:43:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:06 smithi007 ceph-mon[108238]: Upgrade: Updating osd.5 2024-09-21T04:43:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:06 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:06 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T04:43:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:06 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:06 smithi007 ceph-mon[108238]: Deploying daemon osd.5 on smithi053 2024-09-21T04:43:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:06 smithi007 ceph-mon[108238]: pgmap v242: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-21T04:43:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:08 smithi053 ceph-mon[80176]: pgmap v243: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:43:09.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:08 smithi053 systemd[1]: Stopping Ceph osd.5 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:43:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:08 smithi007 ceph-mon[108238]: pgmap v243: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T04:43:09.693 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:09 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[54429]: 2024-09-21T04:43:09.239+0000 7efedb3a9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:43:09.693 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:09 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[54429]: 2024-09-21T04:43:09.239+0000 7efedb3a9640 -1 osd.5 71 *** Got signal Terminated *** 2024-09-21T04:43:09.693 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:09 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[54429]: 2024-09-21T04:43:09.239+0000 7efedb3a9640 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:43:10.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:09 smithi053 ceph-mon[80176]: osd.5 marked itself down and dead 2024-09-21T04:43:10.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:09 smithi007 ceph-mon[108238]: osd.5 marked itself down and dead 2024-09-21T04:43:10.543 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:10 smithi053 podman[98403]: 2024-09-21 04:43:10.246961028 +0000 UTC m=+1.123979345 container died 0eda65c5b4fdcfb17fab8d8682a088019d4bd77526cd96746bea3b2115fcb148 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-4e45e6b) 2024-09-21T04:43:10.869 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:10 smithi053 podman[98403]: 2024-09-21 04:43:10.640513659 +0000 UTC m=+1.517531966 container cleanup 0eda65c5b4fdcfb17fab8d8682a088019d4bd77526cd96746bea3b2115fcb148 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-4e45e6b) 2024-09-21T04:43:10.869 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:10 smithi053 bash[98403]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5 2024-09-21T04:43:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:10 smithi053 ceph-mon[80176]: pgmap v244: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:43:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:10 smithi053 ceph-mon[80176]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:43:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:10 smithi053 ceph-mon[80176]: osdmap e72: 6 total, 5 up, 6 in 2024-09-21T04:43:11.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:10 smithi053 podman[98432]: 2024-09-21 04:43:10.866643292 +0000 UTC m=+0.622074042 container remove 0eda65c5b4fdcfb17fab8d8682a088019d4bd77526cd96746bea3b2115fcb148 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3429f9fdc6da1a3bba9b3b7353043861d7430d80bfe651414b12f783f35cd64, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, RELEASE=squid-4e45e6b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T04:43:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:10 smithi007 ceph-mon[108238]: pgmap v244: 65 pgs: 65 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:43:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:10 smithi007 ceph-mon[108238]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T04:43:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:10 smithi007 ceph-mon[108238]: osdmap e72: 6 total, 5 up, 6 in 2024-09-21T04:43:11.473 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:11 smithi053 podman[98521]: 2024-09-21 04:43:11.192393771 +0000 UTC m=+0.112744202 container create 397c16341a409631897eb13dea81aa5443ae5d682f78a31beaa16f2f26a9b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-21T04:43:11.474 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:11 smithi053 podman[98521]: 2024-09-21 04:43:11.096008836 +0000 UTC m=+0.016359281 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:43:11.474 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:11 smithi053 podman[98521]: 2024-09-21 04:43:11.471304323 +0000 UTC m=+0.391654766 container init 397c16341a409631897eb13dea81aa5443ae5d682f78a31beaa16f2f26a9b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T04:43:11.899 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:11 smithi053 podman[98521]: 2024-09-21 04:43:11.475442786 +0000 UTC m=+0.395793217 container start 397c16341a409631897eb13dea81aa5443ae5d682f78a31beaa16f2f26a9b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T04:43:11.899 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:11 smithi053 podman[98521]: 2024-09-21 04:43:11.553461539 +0000 UTC m=+0.473811984 container attach 397c16341a409631897eb13dea81aa5443ae5d682f78a31beaa16f2f26a9b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T04:43:11.899 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:11 smithi053 podman[98521]: 2024-09-21 04:43:11.665227543 +0000 UTC m=+0.585577983 container died 397c16341a409631897eb13dea81aa5443ae5d682f78a31beaa16f2f26a9b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T04:43:12.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:11 smithi053 ceph-mon[80176]: osdmap e73: 6 total, 5 up, 6 in 2024-09-21T04:43:12.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:11 smithi007 ceph-mon[108238]: osdmap e73: 6 total, 5 up, 6 in 2024-09-21T04:43:12.480 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 podman[98545]: 2024-09-21 04:43:12.232450532 +0000 UTC m=+0.559985808 container remove 397c16341a409631897eb13dea81aa5443ae5d682f78a31beaa16f2f26a9b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T04:43:12.480 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.5.service: Deactivated successfully. 2024-09-21T04:43:12.480 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 systemd[1]: Stopped Ceph osd.5 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:43:12.480 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.5.service: Consumed 53.587s CPU time. 2024-09-21T04:43:12.818 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 systemd[1]: Starting Ceph osd.5 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T04:43:12.818 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 podman[98645]: 2024-09-21 04:43:12.594494626 +0000 UTC m=+0.016156817 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:43:12.818 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 podman[98645]: 2024-09-21 04:43:12.708387455 +0000 UTC m=+0.130049631 container create 65cd861d67e670aa4a5d85914fbbaa0a9f8b6c91e83b7d698699301e0791cfbf (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T04:43:13.102 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:12 smithi053 ceph-mon[80176]: pgmap v247: 65 pgs: 10 peering, 7 stale+active+clean, 48 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-21T04:43:13.102 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:12 smithi053 ceph-mon[80176]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-09-21T04:43:13.102 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 podman[98645]: 2024-09-21 04:43:12.961047117 +0000 UTC m=+0.382709314 container init 65cd861d67e670aa4a5d85914fbbaa0a9f8b6c91e83b7d698699301e0791cfbf (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T04:43:13.102 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:12 smithi053 podman[98645]: 2024-09-21 04:43:12.964888299 +0000 UTC m=+0.386550478 container start 65cd861d67e670aa4a5d85914fbbaa0a9f8b6c91e83b7d698699301e0791cfbf (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T04:43:13.102 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:13 smithi053 podman[98645]: 2024-09-21 04:43:13.035818607 +0000 UTC m=+0.457480788 container attach 65cd861d67e670aa4a5d85914fbbaa0a9f8b6c91e83b7d698699301e0791cfbf (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:43:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:12 smithi007 ceph-mon[108238]: pgmap v247: 65 pgs: 10 peering, 7 stale+active+clean, 48 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-21T04:43:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:12 smithi007 ceph-mon[108238]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-09-21T04:43:13.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:13 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:43:13.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:13 smithi053 bash[98645]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:43:13.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:13 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:43:13.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:13 smithi053 bash[98645]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:43:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:43:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:14 smithi053 ceph-mon[80176]: pgmap v248: 65 pgs: 3 active+undersized, 10 peering, 3 stale+active+clean, 5 active+undersized+degraded, 44 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/333 objects degraded (4.204%) 2024-09-21T04:43:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:43:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:14 smithi007 ceph-mon[108238]: pgmap v248: 65 pgs: 3 active+undersized, 10 peering, 3 stale+active+clean, 5 active+undersized+degraded, 44 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/333 objects degraded (4.204%) 2024-09-21T04:43:15.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:15 smithi053 ceph-mon[80176]: Health check failed: Degraded data redundancy: 14/333 objects degraded (4.204%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T04:43:15.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:15 smithi007 ceph-mon[108238]: Health check failed: Degraded data redundancy: 14/333 objects degraded (4.204%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2024-09-21T04:43:16.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2024-09-21T04:43:16.783 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-mon[80176]: pgmap v249: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate[98660]: --> ceph-volume lvm activate successful for osd ID: 5 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 bash[98645]: --> ceph-volume lvm activate successful for osd ID: 5 2024-09-21T04:43:16.784 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:16 smithi053 podman[98882]: 2024-09-21 04:43:16.527418266 +0000 UTC m=+0.017878005 container died 65cd861d67e670aa4a5d85914fbbaa0a9f8b6c91e83b7d698699301e0791cfbf (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T04:43:16.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:16 smithi007 ceph-mon[108238]: pgmap v249: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:17.342 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:17 smithi053 podman[98882]: 2024-09-21 04:43:17.089792867 +0000 UTC m=+0.580252598 container remove 65cd861d67e670aa4a5d85914fbbaa0a9f8b6c91e83b7d698699301e0791cfbf (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T04:43:17.343 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:17 smithi053 podman[98943]: 2024-09-21 04:43:17.211917603 +0000 UTC m=+0.017310332 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T04:43:17.343 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:17 smithi053 podman[98943]: 2024-09-21 04:43:17.340656002 +0000 UTC m=+0.146048722 container create d8e874d488eb9be29e0641ad862e68030549a56efaf4329cd1b5d323e13ca5a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T04:43:17.613 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:17 smithi053 ceph-mon[80176]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-09-21T04:43:17.613 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:17 smithi053 podman[98943]: 2024-09-21 04:43:17.585068646 +0000 UTC m=+0.390461380 container init d8e874d488eb9be29e0641ad862e68030549a56efaf4329cd1b5d323e13ca5a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T04:43:17.614 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:17 smithi053 podman[98943]: 2024-09-21 04:43:17.588481477 +0000 UTC m=+0.393874191 container start d8e874d488eb9be29e0641ad862e68030549a56efaf4329cd1b5d323e13ca5a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T04:43:17.942 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:17 smithi053 ceph-osd[98962]: -- 172.21.15.53:0/2533222389 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614eba801e0 con 0x5614eac1fc00 2024-09-21T04:43:17.943 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:17 smithi053 bash[98943]: d8e874d488eb9be29e0641ad862e68030549a56efaf4329cd1b5d323e13ca5a3 2024-09-21T04:43:17.943 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:17 smithi053 systemd[1]: Started Ceph osd.5 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T04:43:17.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:17 smithi007 ceph-mon[108238]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs peering) 2024-09-21T04:43:18.996 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:18.996 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:18.996 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:18.996 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:18 smithi053 ceph-mon[80176]: pgmap v250: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:18 smithi007 ceph-mon[108238]: pgmap v250: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:19.252 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:18 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T04:43:18.993+0000 7f791ce1a640 -1 Falling back to public interface 2024-09-21T04:43:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:20 smithi053 ceph-mon[80176]: pgmap v251: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:20 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:21.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:20 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:21.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:20 smithi007 ceph-mon[108238]: pgmap v251: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:21.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:20 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:21.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:20 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:22 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:22 smithi053 ceph-mon[80176]: pgmap v252: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:22.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:22.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:22 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:22.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:22 smithi007 ceph-mon[108238]: pgmap v252: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:24.172 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:23 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 52/333 objects degraded (15.616%), 14 pgs degraded (PG_DEGRADED) 2024-09-21T04:43:24.173 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:23 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T04:43:23.815+0000 7f791ce1a640 -1 osd.5 71 log_to_monitors true 2024-09-21T04:43:24.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:23 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 52/333 objects degraded (15.616%), 14 pgs degraded (PG_DEGRADED) 2024-09-21T04:43:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='osd.5 [v2:172.21.15.53:6816/879599655,v1:172.21.15.53:6817/879599655]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T04:43:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T04:43:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: pgmap v253: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:43:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.0"} : dispatch 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.1"} : dispatch 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.2"} : dispatch 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-21T04:43:25.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.3"} : dispatch 2024-09-21T04:43:25.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-21T04:43:25.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.4"} : dispatch 2024-09-21T04:43:25.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-21T04:43:25.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.5"} : dispatch 2024-09-21T04:43:25.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-21T04:43:25.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:24 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"} : dispatch 2024-09-21T04:43:25.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='osd.5 [v2:172.21.15.53:6816/879599655,v1:172.21.15.53:6817/879599655]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: pgmap v253: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.196 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.0"} : dispatch 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.1"} : dispatch 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.2"} : dispatch 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.3"} : dispatch 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.4"} : dispatch 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.5"} : dispatch 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.197 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:24 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"} : dispatch 2024-09-21T04:43:25.818 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T04:43:25.521+0000 7f7910bbc640 -1 osd.5 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:43:25.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:43:25.692+0000 7f54cce67640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:43:25.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all osd 2024-09-21T04:43:25.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-09-21T04:43:25.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T04:43:25.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: osdmap e74: 6 total, 5 up, 6 in 2024-09-21T04:43:25.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='osd.5 [v2:172.21.15.53:6816/879599655,v1:172.21.15.53:6817/879599655]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:43:25.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:43:25.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='osd.5 ' entity='osd.5' 2024-09-21T04:43:25.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-21T04:43:25.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-09-21T04:43:25.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 2024-09-21T04:43:25.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:25.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:25.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:43:25.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:43:25.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:43:25.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-21T04:43:25.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T04:43:25.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: osd.5 [v2:172.21.15.53:6816/879599655,v1:172.21.15.53:6817/879599655] boot 2024-09-21T04:43:25.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: osdmap e75: 6 total, 6 up, 6 in 2024-09-21T04:43:25.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] up:boot 2024-09-21T04:43:25.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6826/269195221,v1:172.21.15.7:6827/269195221] up:boot 2024-09-21T04:43:25.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-21T04:43:25.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"} : dispatch 2024-09-21T04:43:25.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"} : dispatch 2024-09-21T04:43:25.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T04:43:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all osd 2024-09-21T04:43:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-09-21T04:43:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T04:43:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: osdmap e74: 6 total, 5 up, 6 in 2024-09-21T04:43:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='osd.5 [v2:172.21.15.53:6816/879599655,v1:172.21.15.53:6817/879599655]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:43:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]} : dispatch 2024-09-21T04:43:26.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='osd.5 ' entity='osd.5' 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: fsmap cephfs:2 {0=cephfs.smithi007.eyygpg=up:active,1=cephfs.smithi053.wdnfat=up:active} 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:26.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: osd.5 [v2:172.21.15.53:6816/879599655,v1:172.21.15.53:6817/879599655] boot 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: osdmap e75: 6 total, 6 up, 6 in 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] up:boot 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6826/269195221,v1:172.21.15.7:6827/269195221] up:boot 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"} : dispatch 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"} : dispatch 2024-09-21T04:43:26.195 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T04:43:27.077 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:26 smithi007 ceph-mon[108238]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-21T04:43:27.078 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:26 smithi007 ceph-mon[108238]: pgmap v256: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 52/333 objects degraded (15.616%) 2024-09-21T04:43:27.078 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:26 smithi007 ceph-mon[108238]: osdmap e76: 6 total, 6 up, 6 in 2024-09-21T04:43:27.078 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:26 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.53:6824/1786424362,v1:172.21.15.53:6825/1786424362] up:boot 2024-09-21T04:43:27.078 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:26 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6828/840879872,v1:172.21.15.7:6829/840879872] up:boot 2024-09-21T04:43:27.078 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:26 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:43:27.078 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:26 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"} : dispatch 2024-09-21T04:43:27.078 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:26 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"} : dispatch 2024-09-21T04:43:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:26 smithi053 ceph-mon[80176]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-21T04:43:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:26 smithi053 ceph-mon[80176]: pgmap v256: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 52/333 objects degraded (15.616%) 2024-09-21T04:43:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:26 smithi053 ceph-mon[80176]: osdmap e76: 6 total, 6 up, 6 in 2024-09-21T04:43:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:26 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.53:6824/1786424362,v1:172.21.15.53:6825/1786424362] up:boot 2024-09-21T04:43:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:26 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6828/840879872,v1:172.21.15.7:6829/840879872] up:boot 2024-09-21T04:43:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:26 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:43:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:26 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"} : dispatch 2024-09-21T04:43:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:26 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"} : dispatch 2024-09-21T04:43:28.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:28 smithi007 ceph-mon[108238]: Upgrade: Updating mds.cephfs.smithi007.oxcnqk 2024-09-21T04:43:28.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:28.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.oxcnqk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:43:28.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:28.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:28 smithi007 ceph-mon[108238]: Deploying daemon mds.cephfs.smithi007.oxcnqk on smithi007 2024-09-21T04:43:28.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:28 smithi007 ceph-mon[108238]: pgmap v258: 65 pgs: 13 active+undersized, 13 active+undersized+degraded, 39 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 49/333 objects degraded (14.715%) 2024-09-21T04:43:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:28 smithi053 ceph-mon[80176]: Upgrade: Updating mds.cephfs.smithi007.oxcnqk 2024-09-21T04:43:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.oxcnqk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:43:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:28 smithi053 ceph-mon[80176]: Deploying daemon mds.cephfs.smithi007.oxcnqk on smithi007 2024-09-21T04:43:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:28 smithi053 ceph-mon[80176]: pgmap v258: 65 pgs: 13 active+undersized, 13 active+undersized+degraded, 39 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 49/333 objects degraded (14.715%) 2024-09-21T04:43:29.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:29.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:43:29.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:29 smithi053 ceph-mon[80176]: Health check update: Degraded data redundancy: 49/333 objects degraded (14.715%), 13 pgs degraded (PG_DEGRADED) 2024-09-21T04:43:29.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:29.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:43:29.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:29 smithi007 ceph-mon[108238]: Health check update: Degraded data redundancy: 49/333 objects degraded (14.715%), 13 pgs degraded (PG_DEGRADED) 2024-09-21T04:43:30.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:30 smithi053 ceph-mon[80176]: osdmap e77: 6 total, 6 up, 6 in 2024-09-21T04:43:30.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:30 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-21T04:43:30.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:30 smithi053 ceph-mon[80176]: pgmap v260: 65 pgs: 2 active+undersized, 2 active+undersized+degraded, 61 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5/333 objects degraded (1.502%) 2024-09-21T04:43:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:30 smithi007 ceph-mon[108238]: osdmap e77: 6 total, 6 up, 6 in 2024-09-21T04:43:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:30 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-21T04:43:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:30 smithi007 ceph-mon[108238]: pgmap v260: 65 pgs: 2 active+undersized, 2 active+undersized+degraded, 61 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5/333 objects degraded (1.502%) 2024-09-21T04:43:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.378+0000 7fd533810640 1 -- 172.21.15.7:0/380082038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1007f0 msgr2=0x7fd52c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.378+0000 7fd533810640 1 --2- 172.21.15.7:0/380082038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1007f0 0x7fd52c100c50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009880 tx=0x7fd51c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.378+0000 7fd533810640 1 -- 172.21.15.7:0/380082038 shutdown_connections 2024-09-21T04:43:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.378+0000 7fd533810640 1 --2- 172.21.15.7:0/380082038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1007f0 0x7fd52c100c50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.378+0000 7fd533810640 1 --2- 172.21.15.7:0/380082038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c1067f0 0x7fd52c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.378+0000 7fd533810640 1 -- 172.21.15.7:0/380082038 >> 172.21.15.7:0/380082038 conn(0x7fd52c0fc550 msgr2=0x7fd52c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.379+0000 7fd533810640 1 -- 172.21.15.7:0/380082038 shutdown_connections 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.379+0000 7fd533810640 1 -- 172.21.15.7:0/380082038 wait complete. 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.379+0000 7fd533810640 1 Processor -- start 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.379+0000 7fd533810640 1 -- start start 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd533810640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c1007f0 0x7fd52c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd533810640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1067f0 0x7fd52c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd533810640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c10edc0 con 0x7fd52c1067f0 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd533810640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c10ec40 con 0x7fd52c1007f0 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd530dab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1067f0 0x7fd52c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd530dab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1067f0 0x7fd52c19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39782/0 (socket says 172.21.15.7:39782) 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd530dab640 1 -- 172.21.15.7:0/2207461888 learned_addr learned my addr 172.21.15.7:0/2207461888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd5315ac640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c1007f0 0x7fd52c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd530dab640 1 -- 172.21.15.7:0/2207461888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c1007f0 msgr2=0x7fd52c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd530dab640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c1007f0 0x7fd52c19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.380+0000 7fd530dab640 1 -- 172.21.15.7:0/2207461888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c197790 con 0x7fd52c1067f0 2024-09-21T04:43:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.381+0000 7fd530dab640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1067f0 0x7fd52c19db50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd51c031460 tx=0x7fd51c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.381+0000 7fd5227fc640 1 -- 172.21.15.7:0/2207461888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd51c03f070 con 0x7fd52c1067f0 2024-09-21T04:43:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.381+0000 7fd5227fc640 1 -- 172.21.15.7:0/2207461888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd51c0035f0 con 0x7fd52c1067f0 2024-09-21T04:43:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.381+0000 7fd5227fc640 1 -- 172.21.15.7:0/2207461888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd51c035030 con 0x7fd52c1067f0 2024-09-21T04:43:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.381+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c197a20 con 0x7fd52c1067f0 2024-09-21T04:43:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.381+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c197f50 con 0x7fd52c1067f0 2024-09-21T04:43:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.382+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c101f30 con 0x7fd52c1067f0 2024-09-21T04:43:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.383+0000 7fd5227fc640 1 -- 172.21.15.7:0/2207461888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd51c042070 con 0x7fd52c1067f0 2024-09-21T04:43:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.383+0000 7fd5227fc640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd51807af80 0x7fd51807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.384+0000 7fd5227fc640 1 -- 172.21.15.7:0/2207461888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6242+0+0 (secure 0 0 0) 0x7fd51c0be970 con 0x7fd52c1067f0 2024-09-21T04:43:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.384+0000 7fd5315ac640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd51807af80 0x7fd51807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.384+0000 7fd5315ac640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd51807af80 0x7fd51807d440 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd528005fd0 tx=0x7fd528005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.388+0000 7fd5227fc640 1 -- 172.21.15.7:0/2207461888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd51c0c0e20 con 0x7fd52c1067f0 2024-09-21T04:43:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.593+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd52c198200 con 0x7fd51807af80 2024-09-21T04:43:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.595+0000 7fd5227fc640 1 -- 172.21.15.7:0/2207461888 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd52c198200 con 0x7fd51807af80 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd51807af80 msgr2=0x7fd51807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd51807af80 0x7fd51807d440 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd528005fd0 tx=0x7fd528005850 comp rx=0 tx=0).stop 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1067f0 msgr2=0x7fd52c19db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1067f0 0x7fd52c19db50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd51c031460 tx=0x7fd51c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 shutdown_connections 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd51807af80 0x7fd51807d440 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52c1067f0 0x7fd52c19db50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 --2- 172.21.15.7:0/2207461888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c1007f0 0x7fd52c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 >> 172.21.15.7:0/2207461888 conn(0x7fd52c0fc550 msgr2=0x7fd52c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 shutdown_connections 2024-09-21T04:43:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.597+0000 7fd533810640 1 -- 172.21.15.7:0/2207461888 wait complete. 2024-09-21T04:43:31.609 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:43:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.682+0000 7ff89db91640 1 -- 172.21.15.7:0/4121794247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 msgr2=0x7ff898102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.682+0000 7ff89db91640 1 --2- 172.21.15.7:0/4121794247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 0x7ff898102f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff888009880 tx=0x7ff88802a530 comp rx=0 tx=0).stop 2024-09-21T04:43:31.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.682+0000 7ff89db91640 1 -- 172.21.15.7:0/4121794247 shutdown_connections 2024-09-21T04:43:31.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.682+0000 7ff89db91640 1 --2- 172.21.15.7:0/4121794247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff898103460 0x7ff89810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.682+0000 7ff89db91640 1 --2- 172.21.15.7:0/4121794247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 0x7ff898102f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.682+0000 7ff89db91640 1 -- 172.21.15.7:0/4121794247 >> 172.21.15.7:0/4121794247 conn(0x7ff8980fc520 msgr2=0x7ff8980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:31.686 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:31 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"} : dispatch 2024-09-21T04:43:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.683+0000 7ff89db91640 1 -- 172.21.15.7:0/4121794247 shutdown_connections 2024-09-21T04:43:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.683+0000 7ff89db91640 1 -- 172.21.15.7:0/4121794247 wait complete. 2024-09-21T04:43:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.683+0000 7ff89db91640 1 Processor -- start 2024-09-21T04:43:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.683+0000 7ff89db91640 1 -- start start 2024-09-21T04:43:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff89db91640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 0x7ff89819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff89db91640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff898103460 0x7ff89819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff89db91640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810edd0 con 0x7ff898102b40 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff89db91640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810ec50 con 0x7ff898103460 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff8977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 0x7ff89819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff8977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 0x7ff89819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39800/0 (socket says 172.21.15.7:39800) 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff8977fe640 1 -- 172.21.15.7:0/2200082636 learned_addr learned my addr 172.21.15.7:0/2200082636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff896ffd640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff898103460 0x7ff89819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff8977fe640 1 -- 172.21.15.7:0/2200082636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff898103460 msgr2=0x7ff89819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff8977fe640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff898103460 0x7ff89819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff8977fe640 1 -- 172.21.15.7:0/2200082636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8981977c0 con 0x7ff898102b40 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.684+0000 7ff8977fe640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 0x7ff89819d640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff888004f90 tx=0x7ff8880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.685+0000 7ff894ff9640 1 -- 172.21.15.7:0/2200082636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff88803f070 con 0x7ff898102b40 2024-09-21T04:43:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.685+0000 7ff894ff9640 1 -- 172.21.15.7:0/2200082636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff888004c70 con 0x7ff898102b40 2024-09-21T04:43:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.685+0000 7ff894ff9640 1 -- 172.21.15.7:0/2200082636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff888035030 con 0x7ff898102b40 2024-09-21T04:43:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.685+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff898197a50 con 0x7ff898102b40 2024-09-21T04:43:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.685+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff898197f80 con 0x7ff898102b40 2024-09-21T04:43:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.686+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89806aa10 con 0x7ff898102b40 2024-09-21T04:43:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.687+0000 7ff894ff9640 1 -- 172.21.15.7:0/2200082636 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff888042070 con 0x7ff898102b40 2024-09-21T04:43:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.688+0000 7ff894ff9640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff88007af80 0x7ff88007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.688+0000 7ff894ff9640 1 -- 172.21.15.7:0/2200082636 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6242+0+0 (secure 0 0 0) 0x7ff8880be9a0 con 0x7ff898102b40 2024-09-21T04:43:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.688+0000 7ff896ffd640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff88007af80 0x7ff88007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.688+0000 7ff896ffd640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff88007af80 0x7ff88007d440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff88c005fd0 tx=0x7ff88c005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.692+0000 7ff894ff9640 1 -- 172.21.15.7:0/2200082636 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff888089fc0 con 0x7ff898102b40 2024-09-21T04:43:31.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.899+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff8981982f0 con 0x7ff88007af80 2024-09-21T04:43:31.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.900+0000 7ff894ff9640 1 -- 172.21.15.7:0/2200082636 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7ff8981982f0 con 0x7ff88007af80 2024-09-21T04:43:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff88007af80 msgr2=0x7ff88007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff88007af80 0x7ff88007d440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff88c005fd0 tx=0x7ff88c005850 comp rx=0 tx=0).stop 2024-09-21T04:43:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 msgr2=0x7ff89819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 0x7ff89819d640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff888004f90 tx=0x7ff8880048f0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 shutdown_connections 2024-09-21T04:43:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff88007af80 0x7ff88007d440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff898103460 0x7ff89819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 --2- 172.21.15.7:0/2200082636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff898102b40 0x7ff89819d640 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 >> 172.21.15.7:0/2200082636 conn(0x7ff8980fc520 msgr2=0x7ff898108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 shutdown_connections 2024-09-21T04:43:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.902+0000 7ff89db91640 1 -- 172.21.15.7:0/2200082636 wait complete. 2024-09-21T04:43:31.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:31 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"} : dispatch 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.989+0000 7f60a452b640 1 -- 172.21.15.7:0/2265165666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c106800 msgr2=0x7f609c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.989+0000 7f60a452b640 1 --2- 172.21.15.7:0/2265165666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c106800 0x7f609c106be0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6098009880 tx=0x7f609802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.989+0000 7f60a452b640 1 -- 172.21.15.7:0/2265165666 shutdown_connections 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.989+0000 7f60a452b640 1 --2- 172.21.15.7:0/2265165666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f609c100800 0x7f609c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.989+0000 7f60a452b640 1 --2- 172.21.15.7:0/2265165666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c106800 0x7f609c106be0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.989+0000 7f60a452b640 1 -- 172.21.15.7:0/2265165666 >> 172.21.15.7:0/2265165666 conn(0x7f609c0fc520 msgr2=0x7f609c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.989+0000 7f60a452b640 1 -- 172.21.15.7:0/2265165666 shutdown_connections 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.989+0000 7f60a452b640 1 -- 172.21.15.7:0/2265165666 wait complete. 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a452b640 1 Processor -- start 2024-09-21T04:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a452b640 1 -- start start 2024-09-21T04:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a452b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c100800 0x7f609c107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a452b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f609c106800 0x7f609c107c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a452b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f609c10dcb0 con 0x7f609c100800 2024-09-21T04:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a452b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f609c10db30 con 0x7f609c106800 2024-09-21T04:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a22c7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c100800 0x7f609c107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a22c7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c100800 0x7f609c107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39832/0 (socket says 172.21.15.7:39832) 2024-09-21T04:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a22c7640 1 -- 172.21.15.7:0/3851236760 learned_addr learned my addr 172.21.15.7:0/3851236760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a22c7640 1 -- 172.21.15.7:0/3851236760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f609c106800 msgr2=0x7f609c107c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a1ac6640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f609c106800 0x7f609c107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a22c7640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f609c106800 0x7f609c107c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a22c7640 1 -- 172.21.15.7:0/3851236760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c1081b0 con 0x7f609c100800 2024-09-21T04:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.990+0000 7f60a1ac6640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f609c106800 0x7f609c107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.991+0000 7f60a22c7640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c100800 0x7f609c107730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6098031460 tx=0x7f60980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.991+0000 7f60937fe640 1 -- 172.21.15.7:0/3851236760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f609803f070 con 0x7f609c100800 2024-09-21T04:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.991+0000 7f60937fe640 1 -- 172.21.15.7:0/3851236760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6098004cf0 con 0x7f609c100800 2024-09-21T04:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.991+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c108440 con 0x7f609c100800 2024-09-21T04:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.991+0000 7f60937fe640 1 -- 172.21.15.7:0/3851236760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6098035030 con 0x7f609c100800 2024-09-21T04:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.991+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c1a3da0 con 0x7f609c100800 2024-09-21T04:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.992+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609c101f40 con 0x7f609c100800 2024-09-21T04:43:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.993+0000 7f60937fe640 1 -- 172.21.15.7:0/3851236760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6098042070 con 0x7f609c100800 2024-09-21T04:43:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.993+0000 7f60937fe640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f608807af30 0x7f608807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.993+0000 7f60937fe640 1 -- 172.21.15.7:0/3851236760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6242+0+0 (secure 0 0 0) 0x7f60980beb20 con 0x7f609c100800 2024-09-21T04:43:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.993+0000 7f60a1ac6640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f608807af30 0x7f608807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.994+0000 7f60a1ac6640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f608807af30 0x7f608807d3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f609c1091c0 tx=0x7f608c005d30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:31.998+0000 7f60937fe640 1 -- 172.21.15.7:0/3851236760 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f609808a0c0 con 0x7f609c100800 2024-09-21T04:43:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.194+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f609c06a9e0 con 0x7f608807af30 2024-09-21T04:43:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.204+0000 7f60937fe640 1 -- 172.21.15.7:0/3851236760 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f609c06a9e0 con 0x7f608807af30 2024-09-21T04:43:32.207 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:43:32.207 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (5m) 118s ago 16m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:43:32.207 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (8m) 118s ago 16m 9013k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:43:32.207 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (8m) 11s ago 15m 9042k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (3m) 118s ago 16m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (3m) 11s ago 15m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (5m) 118s ago 16m 77.0M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (11m) 118s ago 11m 144M - 19.1.1-373-g4e45e6b4 164465d2334a d3efb70d0f17 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (11m) 118s ago 11m 89.8M - 19.1.1-373-g4e45e6b4 164465d2334a 41153fcf9a11 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (11m) 11s ago 11m 26.1M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (11m) 11s ago 11m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (7m) 118s ago 17m 528M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (6m) 11s ago 15m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (4m) 118s ago 17m 43.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (3m) 11s ago 15m 37.1M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (6m) 118s ago 16m 10.7M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (6m) 11s ago 15m 10.4M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (3m) 118s ago 14m 86.1M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:43:32.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (2m) 118s ago 14m 90.7M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:43:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (2m) 118s ago 13m 14.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:43:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (85s) 11s ago 13m 95.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:43:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (50s) 11s ago 12m 85.5M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:43:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (14s) 11s ago 12m 12.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:43:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (5m) 118s ago 16m 56.1M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:43:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.206+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f608807af30 msgr2=0x7f608807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.206+0000 7f60a452b640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f608807af30 0x7f608807d3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f609c1091c0 tx=0x7f608c005d30 comp rx=0 tx=0).stop 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.206+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c100800 msgr2=0x7f609c107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.206+0000 7f60a452b640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c100800 0x7f609c107730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6098031460 tx=0x7f60980048f0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.207+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 shutdown_connections 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.207+0000 7f60a452b640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f608807af30 0x7f608807d3f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.207+0000 7f60a452b640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f609c106800 0x7f609c107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.207+0000 7f60a452b640 1 --2- 172.21.15.7:0/3851236760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609c100800 0x7f609c107730 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.207+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 >> 172.21.15.7:0/3851236760 conn(0x7f609c0fc520 msgr2=0x7f609c06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.207+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 shutdown_connections 2024-09-21T04:43:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.207+0000 7f60a452b640 1 -- 172.21.15.7:0/3851236760 wait complete. 2024-09-21T04:43:32.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.289+0000 7fa978b3e640 1 -- 172.21.15.7:0/2119139495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 msgr2=0x7fa974100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.289+0000 7fa978b3e640 1 --2- 172.21.15.7:0/2119139495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 0x7fa974100c60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa968009880 tx=0x7fa96802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.290+0000 7fa978b3e640 1 -- 172.21.15.7:0/2119139495 shutdown_connections 2024-09-21T04:43:32.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.290+0000 7fa978b3e640 1 --2- 172.21.15.7:0/2119139495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 0x7fa974100c60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.290+0000 7fa978b3e640 1 --2- 172.21.15.7:0/2119139495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa974106800 0x7fa974106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.290+0000 7fa978b3e640 1 -- 172.21.15.7:0/2119139495 >> 172.21.15.7:0/2119139495 conn(0x7fa9740fc520 msgr2=0x7fa9740fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:32.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.290+0000 7fa978b3e640 1 -- 172.21.15.7:0/2119139495 shutdown_connections 2024-09-21T04:43:32.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.290+0000 7fa978b3e640 1 -- 172.21.15.7:0/2119139495 wait complete. 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.290+0000 7fa978b3e640 1 Processor -- start 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.290+0000 7fa978b3e640 1 -- start start 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa978b3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 0x7fa974198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa978b3e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa974106800 0x7fa974198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa978b3e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa97410dcb0 con 0x7fa974100800 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa978b3e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa97410db30 con 0x7fa974106800 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa97259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 0x7fa974198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa97259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 0x7fa974198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39860/0 (socket says 172.21.15.7:39860) 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa97259c640 1 -- 172.21.15.7:0/825673881 learned_addr learned my addr 172.21.15.7:0/825673881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa971d9b640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa974106800 0x7fa974198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa97259c640 1 -- 172.21.15.7:0/825673881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa974106800 msgr2=0x7fa974198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa97259c640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa974106800 0x7fa974198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa97259c640 1 -- 172.21.15.7:0/825673881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa974199340 con 0x7fa974100800 2024-09-21T04:43:32.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa97259c640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 0x7fa974198800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa96400d8d0 tx=0x7fa96400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:32.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa9637fe640 1 -- 172.21.15.7:0/825673881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa964017070 con 0x7fa974100800 2024-09-21T04:43:32.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa9637fe640 1 -- 172.21.15.7:0/825673881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa964010460 con 0x7fa974100800 2024-09-21T04:43:32.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa974199630 con 0x7fa974100800 2024-09-21T04:43:32.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.291+0000 7fa9637fe640 1 -- 172.21.15.7:0/825673881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa96400f5f0 con 0x7fa974100800 2024-09-21T04:43:32.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.292+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741a1c30 con 0x7fa974100800 2024-09-21T04:43:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.293+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa974101f40 con 0x7fa974100800 2024-09-21T04:43:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.293+0000 7fa9637fe640 1 -- 172.21.15.7:0/825673881 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa964009a30 con 0x7fa974100800 2024-09-21T04:43:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.294+0000 7fa9637fe640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa95c07af90 0x7fa95c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.294+0000 7fa9637fe640 1 -- 172.21.15.7:0/825673881 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6242+0+0 (secure 0 0 0) 0x7fa96409dda0 con 0x7fa974100800 2024-09-21T04:43:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.294+0000 7fa971d9b640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa95c07af90 0x7fa95c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.294+0000 7fa971d9b640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa95c07af90 0x7fa95c07d450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa97406ad00 tx=0x7fa968031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.298+0000 7fa9637fe640 1 -- 172.21.15.7:0/825673881 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa974199630 con 0x7fa974100800 2024-09-21T04:43:32.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.563+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fa974108d90 con 0x7fa974100800 2024-09-21T04:43:32.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.563+0000 7fa9637fe640 1 -- 172.21.15.7:0/825673881 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+731 (secure 0 0 0) 0x7fa9640693c0 con 0x7fa974100800 2024-09-21T04:43:32.566 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:43:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:43:32.566 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 3 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 3, 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 10 2024-09-21T04:43:32.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:43:32.568 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa95c07af90 msgr2=0x7fa95c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa95c07af90 0x7fa95c07d450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa97406ad00 tx=0x7fa968031400 comp rx=0 tx=0).stop 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 msgr2=0x7fa974198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 0x7fa974198800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa96400d8d0 tx=0x7fa96400dd90 comp rx=0 tx=0).stop 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 shutdown_connections 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa95c07af90 0x7fa95c07d450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa974106800 0x7fa974198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 --2- 172.21.15.7:0/825673881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa974100800 0x7fa974198800 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 >> 172.21.15.7:0/825673881 conn(0x7fa9740fc520 msgr2=0x7fa9740fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 shutdown_connections 2024-09-21T04:43:32.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.566+0000 7fa978b3e640 1 -- 172.21.15.7:0/825673881 wait complete. 2024-09-21T04:43:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.650+0000 7f1aba8b8640 1 -- 172.21.15.7:0/903292089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 msgr2=0x7f1ab4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.650+0000 7f1aba8b8640 1 --2- 172.21.15.7:0/903292089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 0x7f1ab4106bd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009880 tx=0x7f1aa402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.650+0000 7f1aba8b8640 1 -- 172.21.15.7:0/903292089 shutdown_connections 2024-09-21T04:43:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.650+0000 7f1aba8b8640 1 --2- 172.21.15.7:0/903292089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab41007f0 0x7f1ab4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.650+0000 7f1aba8b8640 1 --2- 172.21.15.7:0/903292089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 0x7f1ab4106bd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.650+0000 7f1aba8b8640 1 -- 172.21.15.7:0/903292089 >> 172.21.15.7:0/903292089 conn(0x7f1ab40fc550 msgr2=0x7f1ab40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.650+0000 7f1aba8b8640 1 -- 172.21.15.7:0/903292089 shutdown_connections 2024-09-21T04:43:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.651+0000 7f1aba8b8640 1 -- 172.21.15.7:0/903292089 wait complete. 2024-09-21T04:43:32.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.651+0000 7f1aba8b8640 1 Processor -- start 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.651+0000 7f1aba8b8640 1 -- start start 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1aba8b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab41007f0 0x7f1ab419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1aba8b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 0x7f1ab419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1aba8b8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab410dca0 con 0x7f1ab41067f0 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1aba8b8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab410db20 con 0x7f1ab41007f0 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 0x7f1ab419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 0x7f1ab419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39882/0 (socket says 172.21.15.7:39882) 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab37fe640 1 -- 172.21.15.7:0/2982890027 learned_addr learned my addr 172.21.15.7:0/2982890027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab3fff640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab41007f0 0x7f1ab419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab37fe640 1 -- 172.21.15.7:0/2982890027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab41007f0 msgr2=0x7f1ab419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab37fe640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab41007f0 0x7f1ab419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab37fe640 1 -- 172.21.15.7:0/2982890027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab4197750 con 0x7f1ab41067f0 2024-09-21T04:43:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab37fe640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 0x7f1ab419dbc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d8d0 tx=0x7f1a9c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab17fa640 1 -- 172.21.15.7:0/2982890027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1a9c017030 con 0x7f1ab41067f0 2024-09-21T04:43:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab4197a40 con 0x7f1ab41067f0 2024-09-21T04:43:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab17fa640 1 -- 172.21.15.7:0/2982890027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a9c004020 con 0x7f1ab41067f0 2024-09-21T04:43:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1ab17fa640 1 -- 172.21.15.7:0/2982890027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1a9c0047a0 con 0x7f1ab41067f0 2024-09-21T04:43:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.652+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab4197fa0 con 0x7f1ab41067f0 2024-09-21T04:43:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.653+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab4101f30 con 0x7f1ab41067f0 2024-09-21T04:43:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.654+0000 7f1ab17fa640 1 -- 172.21.15.7:0/2982890027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1a9c0041d0 con 0x7f1ab41067f0 2024-09-21T04:43:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.655+0000 7f1ab17fa640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1aa007af90 0x7f1aa007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.655+0000 7f1ab3fff640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1aa007af90 0x7f1aa007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.655+0000 7f1ab17fa640 1 -- 172.21.15.7:0/2982890027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6242+0+0 (secure 0 0 0) 0x7f1a9c018070 con 0x7f1ab41067f0 2024-09-21T04:43:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.655+0000 7f1ab3fff640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1aa007af90 0x7f1aa007d450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40096c0 tx=0x7f1aa4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:32.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.659+0000 7f1ab17fa640 1 -- 172.21.15.7:0/2982890027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a9c069370 con 0x7f1ab41067f0 2024-09-21T04:43:32.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.885+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1ab41982b0 con 0x7f1ab41067f0 2024-09-21T04:43:32.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:32 smithi007 ceph-mon[108238]: from='client.34326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:32.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:32 smithi007 ceph-mon[108238]: pgmap v261: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:32.888 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:32 smithi007 ceph-mon[108238]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:32.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.885+0000 7f1ab17fa640 1 -- 172.21.15.7:0/2982890027 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+1815 (secure 0 0 0) 0x7f1a9c06e220 con 0x7f1ab41067f0 2024-09-21T04:43:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:e23 2024-09-21T04:43:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:43:29:471342+0000 2024-09-21T04:43:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:43:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:43:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:43:32.890 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:43:32.890 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:43:32.890 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:43:32.890 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:43:32.890 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:43:32.890 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:43:32.890 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:43:32.890 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:43:32.891 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:43:32.892 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:43:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34322} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/840879872,v1:172.21.15.7:6829/840879872] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:43:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:43:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1786424362,v1:172.21.15.53:6825/1786424362] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:43:32.894 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-09-21T04:43:32.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.888+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1aa007af90 msgr2=0x7f1aa007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.888+0000 7f1aba8b8640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1aa007af90 0x7f1aa007d450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40096c0 tx=0x7f1aa4031400 comp rx=0 tx=0).stop 2024-09-21T04:43:32.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 msgr2=0x7f1ab419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 0x7f1ab419dbc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d8d0 tx=0x7f1a9c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:43:32.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 shutdown_connections 2024-09-21T04:43:32.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1aa007af90 0x7f1aa007d450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ab41067f0 0x7f1ab419dbc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 --2- 172.21.15.7:0/2982890027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ab41007f0 0x7f1ab419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 >> 172.21.15.7:0/2982890027 conn(0x7f1ab40fc550 msgr2=0x7f1ab4069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 shutdown_connections 2024-09-21T04:43:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.889+0000 7f1aba8b8640 1 -- 172.21.15.7:0/2982890027 wait complete. 2024-09-21T04:43:32.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:32 smithi053 ceph-mon[80176]: from='client.34326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:32.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:32 smithi053 ceph-mon[80176]: pgmap v261: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:32.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:32 smithi053 ceph-mon[80176]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:32.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 -- 172.21.15.7:0/367852242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0103480 msgr2=0x7f54f010b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 --2- 172.21.15.7:0/367852242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0103480 0x7f54f010b970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009880 tx=0x7f54d802a530 comp rx=0 tx=0).stop 2024-09-21T04:43:32.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 -- 172.21.15.7:0/367852242 shutdown_connections 2024-09-21T04:43:32.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 --2- 172.21.15.7:0/367852242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0103480 0x7f54f010b970 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 --2- 172.21.15.7:0/367852242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f0102b60 0x7f54f0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 -- 172.21.15.7:0/367852242 >> 172.21.15.7:0/367852242 conn(0x7f54f00fc520 msgr2=0x7f54f00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:32.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 -- 172.21.15.7:0/367852242 shutdown_connections 2024-09-21T04:43:32.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 -- 172.21.15.7:0/367852242 wait complete. 2024-09-21T04:43:32.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.975+0000 7f54f7098640 1 Processor -- start 2024-09-21T04:43:32.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f7098640 1 -- start start 2024-09-21T04:43:32.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f7098640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0102b60 0x7f54f01007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f7098640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f0103480 0x7f54f0100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f7098640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010edd0 con 0x7f54f0102b60 2024-09-21T04:43:32.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f7098640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010ec50 con 0x7f54f0103480 2024-09-21T04:43:32.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f4e34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0102b60 0x7f54f01007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f4e34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0102b60 0x7f54f01007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39904/0 (socket says 172.21.15.7:39904) 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f4e34640 1 -- 172.21.15.7:0/29433737 learned_addr learned my addr 172.21.15.7:0/29433737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54effff640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f0103480 0x7f54f0100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f4e34640 1 -- 172.21.15.7:0/29433737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f0103480 msgr2=0x7f54f0100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f4e34640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f0103480 0x7f54f0100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f4e34640 1 -- 172.21.15.7:0/29433737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f0101240 con 0x7f54f0102b60 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.976+0000 7f54f4e34640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0102b60 0x7f54f01007c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c8a0 tx=0x7f54e000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.977+0000 7f54edffb640 1 -- 172.21.15.7:0/29433737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f54e00079b0 con 0x7f54f0102b60 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.977+0000 7f54edffb640 1 -- 172.21.15.7:0/29433737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f54e0007b50 con 0x7f54f0102b60 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.977+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f0101530 con 0x7f54f0102b60 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.977+0000 7f54edffb640 1 -- 172.21.15.7:0/29433737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f54e0018700 con 0x7f54f0102b60 2024-09-21T04:43:32.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.977+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01a3e20 con 0x7f54f0102b60 2024-09-21T04:43:32.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.978+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f006a9f0 con 0x7f54f0102b60 2024-09-21T04:43:32.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.979+0000 7f54edffb640 1 -- 172.21.15.7:0/29433737 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f54e0007d00 con 0x7f54f0102b60 2024-09-21T04:43:32.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.979+0000 7f54edffb640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f54dc07af40 0x7f54dc07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:32.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.979+0000 7f54edffb640 1 -- 172.21.15.7:0/29433737 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6242+0+0 (secure 0 0 0) 0x7f54e009e2c0 con 0x7f54f0102b60 2024-09-21T04:43:32.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.979+0000 7f54effff640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f54dc07af40 0x7f54dc07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:32.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.980+0000 7f54effff640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f54dc07af40 0x7f54dc07d400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f54f0102250 tx=0x7f54d8004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:32.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:32.983+0000 7f54edffb640 1 -- 172.21.15.7:0/29433737 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54e0014030 con 0x7f54f0102b60 2024-09-21T04:43:33.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.190+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f54f0062630 con 0x7f54dc07af40 2024-09-21T04:43:33.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.191+0000 7f54edffb640 1 -- 172.21.15.7:0/29433737 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f54f0062630 con 0x7f54dc07af40 2024-09-21T04:43:33.194 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:43:33.194 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:43:33.194 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:43:33.194 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:43:33.194 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "12/23 daemons upgraded", 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:43:33.195 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:43:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.193+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f54dc07af40 msgr2=0x7f54dc07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:33.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.193+0000 7f54f7098640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f54dc07af40 0x7f54dc07d400 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f54f0102250 tx=0x7f54d8004f30 comp rx=0 tx=0).stop 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.193+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0102b60 msgr2=0x7f54f01007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.193+0000 7f54f7098640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0102b60 0x7f54f01007c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c8a0 tx=0x7f54e000cd60 comp rx=0 tx=0).stop 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.194+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 shutdown_connections 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.194+0000 7f54f7098640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f54dc07af40 0x7f54dc07d400 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.194+0000 7f54f7098640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f0103480 0x7f54f0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.194+0000 7f54f7098640 1 --2- 172.21.15.7:0/29433737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f0102b60 0x7f54f01007c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.194+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 >> 172.21.15.7:0/29433737 conn(0x7f54f00fc520 msgr2=0x7f54f0103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.194+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 shutdown_connections 2024-09-21T04:43:33.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.194+0000 7f54f7098640 1 -- 172.21.15.7:0/29433737 wait complete. 2024-09-21T04:43:33.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.280+0000 7f6d2a562640 1 -- 172.21.15.7:0/1091597570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d241033e0 msgr2=0x7f6d2410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.280+0000 7f6d2a562640 1 --2- 172.21.15.7:0/1091597570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d241033e0 0x7f6d2410b8d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009880 tx=0x7f6d1802a530 comp rx=0 tx=0).stop 2024-09-21T04:43:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.280+0000 7f6d2a562640 1 -- 172.21.15.7:0/1091597570 shutdown_connections 2024-09-21T04:43:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.280+0000 7f6d2a562640 1 --2- 172.21.15.7:0/1091597570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d241033e0 0x7f6d2410b8d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.280+0000 7f6d2a562640 1 --2- 172.21.15.7:0/1091597570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d24102ac0 0x7f6d24102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.280+0000 7f6d2a562640 1 -- 172.21.15.7:0/1091597570 >> 172.21.15.7:0/1091597570 conn(0x7f6d240fc520 msgr2=0x7f6d240fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.280+0000 7f6d2a562640 1 -- 172.21.15.7:0/1091597570 shutdown_connections 2024-09-21T04:43:33.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.280+0000 7f6d2a562640 1 -- 172.21.15.7:0/1091597570 wait complete. 2024-09-21T04:43:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.281+0000 7f6d2a562640 1 Processor -- start 2024-09-21T04:43:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.281+0000 7f6d2a562640 1 -- start start 2024-09-21T04:43:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.281+0000 7f6d2a562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24102ac0 0x7f6d2419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d2a562640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d241033e0 0x7f6d2419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d2a562640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2410edf0 con 0x7f6d24102ac0 2024-09-21T04:43:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d2a562640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2410ec70 con 0x7f6d241033e0 2024-09-21T04:43:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d23fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24102ac0 0x7f6d2419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d23fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24102ac0 0x7f6d2419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39922/0 (socket says 172.21.15.7:39922) 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d23fff640 1 -- 172.21.15.7:0/2214296420 learned_addr learned my addr 172.21.15.7:0/2214296420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d237fe640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d241033e0 0x7f6d2419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d23fff640 1 -- 172.21.15.7:0/2214296420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d241033e0 msgr2=0x7f6d2419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d23fff640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d241033e0 0x7f6d2419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d23fff640 1 -- 172.21.15.7:0/2214296420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d241977e0 con 0x7f6d24102ac0 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d23fff640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24102ac0 0x7f6d2419d660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400d6f0 tx=0x7f6d1400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d217fa640 1 -- 172.21.15.7:0/2214296420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d14017070 con 0x7f6d24102ac0 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.282+0000 7f6d217fa640 1 -- 172.21.15.7:0/2214296420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6d1400de00 con 0x7f6d24102ac0 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.283+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d24197ad0 con 0x7f6d24102ac0 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.283+0000 7f6d217fa640 1 -- 172.21.15.7:0/2214296420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d14004bb0 con 0x7f6d24102ac0 2024-09-21T04:43:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.283+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d24198030 con 0x7f6d24102ac0 2024-09-21T04:43:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.284+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2406aa90 con 0x7f6d24102ac0 2024-09-21T04:43:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.284+0000 7f6d217fa640 1 -- 172.21.15.7:0/2214296420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6d14009a30 con 0x7f6d24102ac0 2024-09-21T04:43:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.285+0000 7f6d217fa640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6d0c07af80 0x7f6d0c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:43:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.285+0000 7f6d217fa640 1 -- 172.21.15.7:0/2214296420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6242+0+0 (secure 0 0 0) 0x7f6d1409db20 con 0x7f6d24102ac0 2024-09-21T04:43:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.285+0000 7f6d237fe640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6d0c07af80 0x7f6d0c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:43:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.285+0000 7f6d237fe640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6d0c07af80 0x7f6d0c07d440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d24198b30 tx=0x7f6d18031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:43:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.289+0000 7f6d217fa640 1 -- 172.21.15.7:0/2214296420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6d14069140 con 0x7f6d24102ac0 2024-09-21T04:43:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.552+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6d24105f60 con 0x7f6d24102ac0 2024-09-21T04:43:33.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.553+0000 7f6d217fa640 1 -- 172.21.15.7:0/2214296420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f6d1406dff0 con 0x7f6d24102ac0 2024-09-21T04:43:33.556 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:43:33.556 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:43:33.556 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:43:33.556 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:43:33.556 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:43:33.556 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:43:33.556 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:43:33.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6d0c07af80 msgr2=0x7f6d0c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:33.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6d0c07af80 0x7f6d0c07d440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d24198b30 tx=0x7f6d18031400 comp rx=0 tx=0).stop 2024-09-21T04:43:33.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24102ac0 msgr2=0x7f6d2419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:43:33.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24102ac0 0x7f6d2419d660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400d6f0 tx=0x7f6d1400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 shutdown_connections 2024-09-21T04:43:33.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6d0c07af80 0x7f6d0c07d440 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d241033e0 0x7f6d2419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 --2- 172.21.15.7:0/2214296420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24102ac0 0x7f6d2419d660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:43:33.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 >> 172.21.15.7:0/2214296420 conn(0x7f6d240fc520 msgr2=0x7f6d24108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:43:33.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 shutdown_connections 2024-09-21T04:43:33.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:43:33.555+0000 7f6d2a562640 1 -- 172.21.15.7:0/2214296420 wait complete. 2024-09-21T04:43:33.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:33 smithi007 ceph-mon[108238]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:33.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:33 smithi007 ceph-mon[108238]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/333 objects degraded (1.502%), 2 pgs degraded) 2024-09-21T04:43:33.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:33 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/825673881' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:33.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:33 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2982890027' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:43:33.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:33 smithi053 ceph-mon[80176]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:33.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:33 smithi053 ceph-mon[80176]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/333 objects degraded (1.502%), 2 pgs degraded) 2024-09-21T04:43:33.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:33 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/825673881' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:33.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:33 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2982890027' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:43:34.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:34 smithi007 ceph-mon[108238]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:34.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:34 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2214296420' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:43:34.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:34 smithi007 ceph-mon[108238]: pgmap v262: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:34.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:34 smithi053 ceph-mon[80176]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:43:34.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:34 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2214296420' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:43:34.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:34 smithi053 ceph-mon[80176]: pgmap v262: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:36.226 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:36 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:36.226 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:36 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:36.226 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:36 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:36.226 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:36 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:boot 2024-09-21T04:43:36.226 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:36 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:43:36.227 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:36 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"} : dispatch 2024-09-21T04:43:36.227 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:36 smithi007 ceph-mon[108238]: pgmap v263: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:36 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:36 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:36 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:36.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:36 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:boot 2024-09-21T04:43:36.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:36 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:43:36.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:36 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.oxcnqk"} : dispatch 2024-09-21T04:43:36.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:36 smithi053 ceph-mon[80176]: pgmap v263: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:38 smithi053 ceph-mon[80176]: pgmap v264: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:39.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:38 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:38 smithi007 ceph-mon[108238]: pgmap v264: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:38 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:40 smithi053 ceph-mon[80176]: pgmap v265: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:40 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:40 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:40 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:41.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:40 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:40 smithi007 ceph-mon[108238]: pgmap v265: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:40 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:40 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:40 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:40 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:43.190 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:42 smithi007 ceph-mon[108238]: pgmap v266: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:43.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:42 smithi053 ceph-mon[80176]: pgmap v266: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: pgmap v267: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:44.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:44.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:44.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: pgmap v267: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:44.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:44.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:43:46.485 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:46 smithi007 ceph-mon[108238]: Upgrade: Updating mds.cephfs.smithi007.eyygpg 2024-09-21T04:43:46.485 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:46 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:46.485 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:46 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.eyygpg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:43:46.485 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:46 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:46.485 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:46 smithi007 ceph-mon[108238]: Deploying daemon mds.cephfs.smithi007.eyygpg on smithi007 2024-09-21T04:43:46.485 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:46 smithi007 ceph-mon[108238]: pgmap v268: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:46 smithi053 ceph-mon[80176]: Upgrade: Updating mds.cephfs.smithi007.eyygpg 2024-09-21T04:43:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:46 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:46 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.eyygpg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:43:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:46 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:43:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:46 smithi053 ceph-mon[80176]: Deploying daemon mds.cephfs.smithi007.eyygpg on smithi007 2024-09-21T04:43:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:46 smithi053 ceph-mon[80176]: pgmap v268: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:48.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:48 smithi053 ceph-mon[80176]: osdmap e78: 6 total, 6 up, 6 in 2024-09-21T04:43:48.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:48 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-21T04:43:48.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:48 smithi053 ceph-mon[80176]: pgmap v270: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:48.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:48 smithi007 ceph-mon[108238]: osdmap e78: 6 total, 6 up, 6 in 2024-09-21T04:43:48.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:48 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-21T04:43:48.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:48 smithi007 ceph-mon[108238]: pgmap v270: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:51.165 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:50 smithi007 ceph-mon[108238]: pgmap v271: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:51.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:50 smithi053 ceph-mon[80176]: pgmap v271: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:52.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:52 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:52.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:52 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:52.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:52 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:52.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:52 smithi053 ceph-mon[80176]: pgmap v272: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:52.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:52 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:boot 2024-09-21T04:43:52.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:52 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:43:52.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:52 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"} : dispatch 2024-09-21T04:43:52.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:52 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:52.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:52 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:52.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:52 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:43:52.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:52 smithi007 ceph-mon[108238]: pgmap v272: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:52.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:52 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:boot 2024-09-21T04:43:52.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:52 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:43:52.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:52 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi007.eyygpg"} : dispatch 2024-09-21T04:43:54.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:54 smithi007 ceph-mon[108238]: pgmap v273: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:54 smithi053 ceph-mon[80176]: pgmap v273: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:56 smithi053 ceph-mon[80176]: pgmap v274: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:56 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:57.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:56 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:56 smithi007 ceph-mon[108238]: pgmap v274: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:56 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:57.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:56 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:58.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:57 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:58.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:57 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:58.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:57 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:58.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:57 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:43:59.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:58 smithi007 ceph-mon[108238]: pgmap v275: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:59.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:43:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:43:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:58 smithi053 ceph-mon[80176]: pgmap v275: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:43:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:43:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:44:01.041 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: pgmap v276: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:01.041 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:01.041 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:01.041 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:44:01.041 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:44:01.041 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:01.041 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:44:01.042 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:01.042 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:01.042 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:01.042 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:00 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: pgmap v276: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:44:01.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:44:01.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:01.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:44:01.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:01.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:01.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:01.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:00 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:03.006 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:02 smithi053 ceph-mon[80176]: Upgrade: Updating mds.cephfs.smithi053.wdnfat 2024-09-21T04:44:03.006 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:02 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:03.006 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:02 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.wdnfat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:44:03.006 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:02 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:44:03.006 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:02 smithi053 ceph-mon[80176]: Deploying daemon mds.cephfs.smithi053.wdnfat on smithi053 2024-09-21T04:44:03.006 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:02 smithi053 ceph-mon[80176]: pgmap v277: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:03.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:02 smithi007 ceph-mon[108238]: Upgrade: Updating mds.cephfs.smithi053.wdnfat 2024-09-21T04:44:03.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:02 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:03.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:02 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.wdnfat", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:44:03.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:02 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:44:03.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:02 smithi007 ceph-mon[108238]: Deploying daemon mds.cephfs.smithi053.wdnfat on smithi053 2024-09-21T04:44:03.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:02 smithi007 ceph-mon[108238]: pgmap v277: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.655+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/1226295775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 msgr2=0x7f1bd41123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.655+0000 7f1bdb2a7640 1 --2- 172.21.15.7:0/1226295775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 0x7f1bd41123d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009880 tx=0x7f1bc802a530 comp rx=0 tx=0).stop 2024-09-21T04:44:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.655+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/1226295775 shutdown_connections 2024-09-21T04:44:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.655+0000 7f1bdb2a7640 1 --2- 172.21.15.7:0/1226295775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 0x7f1bd41123d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.655+0000 7f1bdb2a7640 1 --2- 172.21.15.7:0/1226295775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bd4074590 0x7f1bd4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.655+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/1226295775 >> 172.21.15.7:0/1226295775 conn(0x7f1bd40fd600 msgr2=0x7f1bd40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:03.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.655+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/1226295775 shutdown_connections 2024-09-21T04:44:03.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.655+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/1226295775 wait complete. 2024-09-21T04:44:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bdb2a7640 1 Processor -- start 2024-09-21T04:44:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bdb2a7640 1 -- start start 2024-09-21T04:44:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bdb2a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bd4074590 0x7f1bd419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bdb2a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 0x7f1bd419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bdb2a7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd41153c0 con 0x7f1bd4074eb0 2024-09-21T04:44:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bdb2a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4115240 con 0x7f1bd4074590 2024-09-21T04:44:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bd8842640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 0x7f1bd419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bd8842640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 0x7f1bd419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46210/0 (socket says 172.21.15.7:46210) 2024-09-21T04:44:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bd8842640 1 -- 172.21.15.7:0/3525418776 learned_addr learned my addr 172.21.15.7:0/3525418776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bd9043640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bd4074590 0x7f1bd419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bd8842640 1 -- 172.21.15.7:0/3525418776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bd4074590 msgr2=0x7f1bd419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bd8842640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bd4074590 0x7f1bd419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.656+0000 7f1bd8842640 1 -- 172.21.15.7:0/3525418776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd41a2890 con 0x7f1bd4074eb0 2024-09-21T04:44:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.657+0000 7f1bd8842640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 0x7f1bd419e500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8031460 tx=0x7f1bc80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.657+0000 7f1bc67fc640 1 -- 172.21.15.7:0/3525418776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1bc803f070 con 0x7f1bd4074eb0 2024-09-21T04:44:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.657+0000 7f1bc67fc640 1 -- 172.21.15.7:0/3525418776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1bc80035f0 con 0x7f1bd4074eb0 2024-09-21T04:44:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.657+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd41a2b20 con 0x7f1bd4074eb0 2024-09-21T04:44:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.657+0000 7f1bc67fc640 1 -- 172.21.15.7:0/3525418776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1bc8035030 con 0x7f1bd4074eb0 2024-09-21T04:44:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.657+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd41a3050 con 0x7f1bd4074eb0 2024-09-21T04:44:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.658+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd4075cd0 con 0x7f1bd4074eb0 2024-09-21T04:44:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.659+0000 7f1bc67fc640 1 -- 172.21.15.7:0/3525418776 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1bc8042070 con 0x7f1bd4074eb0 2024-09-21T04:44:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.659+0000 7f1bc67fc640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1bc007af80 0x7f1bc007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.659+0000 7f1bc67fc640 1 -- 172.21.15.7:0/3525418776 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f1bc80be8c0 con 0x7f1bd4074eb0 2024-09-21T04:44:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.659+0000 7f1bd9043640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1bc007af80 0x7f1bc007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.660+0000 7f1bd9043640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1bc007af80 0x7f1bc007d440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0005d00 tx=0x7f1bd000b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.664+0000 7f1bc67fc640 1 -- 172.21.15.7:0/3525418776 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1bc8089ee0 con 0x7f1bd4074eb0 2024-09-21T04:44:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.875+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1bd419f320 con 0x7f1bc007af80 2024-09-21T04:44:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.876+0000 7f1bc67fc640 1 -- 172.21.15.7:0/3525418776 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f1bd419f320 con 0x7f1bc007af80 2024-09-21T04:44:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1bc007af80 msgr2=0x7f1bc007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1bc007af80 0x7f1bc007d440 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0005d00 tx=0x7f1bd000b410 comp rx=0 tx=0).stop 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 msgr2=0x7f1bd419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 0x7f1bd419e500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8031460 tx=0x7f1bc80045e0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 shutdown_connections 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1bc007af80 0x7f1bc007d440 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd4074eb0 0x7f1bd419e500 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 --2- 172.21.15.7:0/3525418776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bd4074590 0x7f1bd419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.878+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 >> 172.21.15.7:0/3525418776 conn(0x7f1bd40fd600 msgr2=0x7f1bd4104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.879+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 shutdown_connections 2024-09-21T04:44:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.879+0000 7f1bdb2a7640 1 -- 172.21.15.7:0/3525418776 wait complete. 2024-09-21T04:44:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:44:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.966+0000 7f25730aa640 1 -- 172.21.15.7:0/3889007324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 msgr2=0x7f256c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.966+0000 7f25730aa640 1 --2- 172.21.15.7:0/3889007324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 0x7f256c10b8d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2558009880 tx=0x7f255802a530 comp rx=0 tx=0).stop 2024-09-21T04:44:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.966+0000 7f25730aa640 1 -- 172.21.15.7:0/3889007324 shutdown_connections 2024-09-21T04:44:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.966+0000 7f25730aa640 1 --2- 172.21.15.7:0/3889007324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 0x7f256c10b8d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.966+0000 7f25730aa640 1 --2- 172.21.15.7:0/3889007324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f256c102ac0 0x7f256c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.966+0000 7f25730aa640 1 -- 172.21.15.7:0/3889007324 >> 172.21.15.7:0/3889007324 conn(0x7f256c0fc520 msgr2=0x7f256c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.966+0000 7f25730aa640 1 -- 172.21.15.7:0/3889007324 shutdown_connections 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.966+0000 7f25730aa640 1 -- 172.21.15.7:0/3889007324 wait complete. 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f25730aa640 1 Processor -- start 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f25730aa640 1 -- start start 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f25730aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f256c102ac0 0x7f256c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f25730aa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 0x7f256c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f25730aa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f256c10edf0 con 0x7f256c1033e0 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f25730aa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f256c10ec70 con 0x7f256c102ac0 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f256bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 0x7f256c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f256bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 0x7f256c19dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46228/0 (socket says 172.21.15.7:46228) 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f256bfff640 1 -- 172.21.15.7:0/1055250853 learned_addr learned my addr 172.21.15.7:0/1055250853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.967+0000 7f2570e46640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f256c102ac0 0x7f256c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f256bfff640 1 -- 172.21.15.7:0/1055250853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f256c102ac0 msgr2=0x7f256c19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f256bfff640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f256c102ac0 0x7f256c19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f256bfff640 1 -- 172.21.15.7:0/1055250853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c197820 con 0x7f256c1033e0 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f256bfff640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 0x7f256c19dbe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2558031460 tx=0x7f2558004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f2569ffb640 1 -- 172.21.15.7:0/1055250853 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f255803f070 con 0x7f256c1033e0 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f2569ffb640 1 -- 172.21.15.7:0/1055250853 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25580035f0 con 0x7f256c1033e0 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c197ab0 con 0x7f256c1033e0 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f2569ffb640 1 -- 172.21.15.7:0/1055250853 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2558035030 con 0x7f256c1033e0 2024-09-21T04:44:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.968+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c197fe0 con 0x7f256c1033e0 2024-09-21T04:44:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.969+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f256c06aa90 con 0x7f256c1033e0 2024-09-21T04:44:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.970+0000 7f2569ffb640 1 -- 172.21.15.7:0/1055250853 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2558042070 con 0x7f256c1033e0 2024-09-21T04:44:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.971+0000 7f2569ffb640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f255c07af30 0x7f255c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.971+0000 7f2569ffb640 1 -- 172.21.15.7:0/1055250853 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f25580bea00 con 0x7f256c1033e0 2024-09-21T04:44:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.971+0000 7f2570e46640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f255c07af30 0x7f255c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.971+0000 7f2570e46640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f255c07af30 0x7f255c07d3f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2560005fd0 tx=0x7f256000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:03.975+0000 7f2569ffb640 1 -- 172.21.15.7:0/1055250853 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2558089f00 con 0x7f256c1033e0 2024-09-21T04:44:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.206+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f256c1982f0 con 0x7f255c07af30 2024-09-21T04:44:04.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.207+0000 7f2569ffb640 1 -- 172.21.15.7:0/1055250853 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f256c1982f0 con 0x7f255c07af30 2024-09-21T04:44:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f255c07af30 msgr2=0x7f255c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f255c07af30 0x7f255c07d3f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2560005fd0 tx=0x7f256000b390 comp rx=0 tx=0).stop 2024-09-21T04:44:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 msgr2=0x7f256c19dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 0x7f256c19dbe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2558031460 tx=0x7f2558004530 comp rx=0 tx=0).stop 2024-09-21T04:44:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 shutdown_connections 2024-09-21T04:44:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f255c07af30 0x7f255c07d3f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c1033e0 0x7f256c19dbe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 --2- 172.21.15.7:0/1055250853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f256c102ac0 0x7f256c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 >> 172.21.15.7:0/1055250853 conn(0x7f256c0fc520 msgr2=0x7f256c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 shutdown_connections 2024-09-21T04:44:04.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.209+0000 7f25730aa640 1 -- 172.21.15.7:0/1055250853 wait complete. 2024-09-21T04:44:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.296+0000 7f6191a38640 1 -- 172.21.15.7:0/2724210618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 msgr2=0x7f618c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.296+0000 7f6191a38640 1 --2- 172.21.15.7:0/2724210618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 0x7f618c106bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f617c009880 tx=0x7f617c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.296+0000 7f6191a38640 1 -- 172.21.15.7:0/2724210618 shutdown_connections 2024-09-21T04:44:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.296+0000 7f6191a38640 1 --2- 172.21.15.7:0/2724210618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f618c1007f0 0x7f618c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.296+0000 7f6191a38640 1 --2- 172.21.15.7:0/2724210618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 0x7f618c106bd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.296+0000 7f6191a38640 1 -- 172.21.15.7:0/2724210618 >> 172.21.15.7:0/2724210618 conn(0x7f618c0fc550 msgr2=0x7f618c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.296+0000 7f6191a38640 1 -- 172.21.15.7:0/2724210618 shutdown_connections 2024-09-21T04:44:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.297+0000 7f6191a38640 1 -- 172.21.15.7:0/2724210618 wait complete. 2024-09-21T04:44:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.297+0000 7f6191a38640 1 Processor -- start 2024-09-21T04:44:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.297+0000 7f6191a38640 1 -- start start 2024-09-21T04:44:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f6191a38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f618c1007f0 0x7f618c107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f6191a38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 0x7f618c107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f6191a38640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f618c10dca0 con 0x7f618c1067f0 2024-09-21T04:44:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f6191a38640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f618c10db20 con 0x7f618c1007f0 2024-09-21T04:44:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f618a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 0x7f618c107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f618a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 0x7f618c107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46246/0 (socket says 172.21.15.7:46246) 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f618affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f618c1007f0 0x7f618c107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f618a7fc640 1 -- 172.21.15.7:0/922047962 learned_addr learned my addr 172.21.15.7:0/922047962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f618a7fc640 1 -- 172.21.15.7:0/922047962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f618c1007f0 msgr2=0x7f618c107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f618a7fc640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f618c1007f0 0x7f618c107720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f618a7fc640 1 -- 172.21.15.7:0/922047962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c1081a0 con 0x7f618c1067f0 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.298+0000 7f618a7fc640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 0x7f618c107c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f618000c8a0 tx=0x7f618000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.299+0000 7f6190a36640 1 -- 172.21.15.7:0/922047962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f61800079b0 con 0x7f618c1067f0 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.299+0000 7f6190a36640 1 -- 172.21.15.7:0/922047962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6180007b50 con 0x7f618c1067f0 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.299+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c108490 con 0x7f618c1067f0 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.299+0000 7f6190a36640 1 -- 172.21.15.7:0/922047962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6180018700 con 0x7f618c1067f0 2024-09-21T04:44:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.299+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c1a3d70 con 0x7f618c1067f0 2024-09-21T04:44:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.300+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c101f30 con 0x7f618c1067f0 2024-09-21T04:44:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.301+0000 7f6190a36640 1 -- 172.21.15.7:0/922047962 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6180007d00 con 0x7f618c1067f0 2024-09-21T04:44:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.302+0000 7f6190a36640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f617407af30 0x7f617407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.302+0000 7f6190a36640 1 -- 172.21.15.7:0/922047962 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f618009e270 con 0x7f618c1067f0 2024-09-21T04:44:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.302+0000 7f618affd640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f617407af30 0x7f617407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.302+0000 7f618affd640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f617407af30 0x7f617407d3f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f617c000c00 tx=0x7f617c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.306+0000 7f6190a36640 1 -- 172.21.15.7:0/922047962 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6180014030 con 0x7f618c1067f0 2024-09-21T04:44:04.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.503+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f618c06a520 con 0x7f617407af30 2024-09-21T04:44:04.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.514+0000 7f6190a36640 1 -- 172.21.15.7:0/922047962 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f618c06a520 con 0x7f617407af30 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (6m) 8s ago 16m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (9m) 8s ago 17m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (8m) 43s ago 16m 9042k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (4m) 8s ago 17m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (4m) 43s ago 16m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (5m) 8s ago 16m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (13s) 8s ago 11m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (29s) 8s ago 11m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (11m) 43s ago 11m 26.1M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:44:04.517 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (11m) 43s ago 11m 31.6M - 19.1.1-373-g4e45e6b4 164465d2334a 595e1f98ed6d 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (8m) 8s ago 18m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (7m) 43s ago 16m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (4m) 8s ago 18m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (4m) 43s ago 16m 37.1M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (6m) 8s ago 17m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (6m) 43s ago 16m 10.4M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (3m) 8s ago 15m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (3m) 8s ago 14m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (2m) 8s ago 14m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (118s) 43s ago 13m 95.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:44:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (82s) 43s ago 13m 85.5M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:44:04.519 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (47s) 43s ago 12m 12.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:44:04.519 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (6m) 8s ago 16m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:44:04.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f617407af30 msgr2=0x7f617407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f617407af30 0x7f617407d3f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f617c000c00 tx=0x7f617c031400 comp rx=0 tx=0).stop 2024-09-21T04:44:04.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 msgr2=0x7f618c107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 0x7f618c107c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f618000c8a0 tx=0x7f618000cd60 comp rx=0 tx=0).stop 2024-09-21T04:44:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 shutdown_connections 2024-09-21T04:44:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f617407af30 0x7f617407d3f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c1067f0 0x7f618c107c60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 --2- 172.21.15.7:0/922047962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f618c1007f0 0x7f618c107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 >> 172.21.15.7:0/922047962 conn(0x7f618c0fc550 msgr2=0x7f618c069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 shutdown_connections 2024-09-21T04:44:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.516+0000 7f6191a38640 1 -- 172.21.15.7:0/922047962 wait complete. 2024-09-21T04:44:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 -- 172.21.15.7:0/3841282867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc101900 msgr2=0x7f5cbc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 --2- 172.21.15.7:0/3841282867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc101900 0x7f5cbc101d60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009880 tx=0x7f5ca402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 -- 172.21.15.7:0/3841282867 shutdown_connections 2024-09-21T04:44:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 --2- 172.21.15.7:0/3841282867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc101900 0x7f5cbc101d60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 --2- 172.21.15.7:0/3841282867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cbc107900 0x7f5cbc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 -- 172.21.15.7:0/3841282867 >> 172.21.15.7:0/3841282867 conn(0x7f5cbc0fd600 msgr2=0x7f5cbc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 -- 172.21.15.7:0/3841282867 shutdown_connections 2024-09-21T04:44:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 -- 172.21.15.7:0/3841282867 wait complete. 2024-09-21T04:44:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.599+0000 7f5cc19a0640 1 Processor -- start 2024-09-21T04:44:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cc19a0640 1 -- start start 2024-09-21T04:44:04.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cc19a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cbc101900 0x7f5cbc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cc19a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc107900 0x7f5cbc19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cc19a0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc10edd0 con 0x7f5cbc107900 2024-09-21T04:44:04.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cc19a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc10ec50 con 0x7f5cbc101900 2024-09-21T04:44:04.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cba7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc107900 0x7f5cbc19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cba7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc107900 0x7f5cbc19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46264/0 (socket says 172.21.15.7:46264) 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cba7fc640 1 -- 172.21.15.7:0/2731837179 learned_addr learned my addr 172.21.15.7:0/2731837179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cbaffd640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cbc101900 0x7f5cbc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cba7fc640 1 -- 172.21.15.7:0/2731837179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cbc101900 msgr2=0x7f5cbc19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cba7fc640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cbc101900 0x7f5cbc19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.600+0000 7f5cba7fc640 1 -- 172.21.15.7:0/2731837179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc199a10 con 0x7f5cbc107900 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.601+0000 7f5cba7fc640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc107900 0x7f5cbc19fd90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4031460 tx=0x7f5ca40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.601+0000 7f5cc099e640 1 -- 172.21.15.7:0/2731837179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5ca403f070 con 0x7f5cbc107900 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.601+0000 7f5cc099e640 1 -- 172.21.15.7:0/2731837179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5ca40035f0 con 0x7f5cbc107900 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.601+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cbc199ca0 con 0x7f5cbc107900 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.601+0000 7f5cc099e640 1 -- 172.21.15.7:0/2731837179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5ca4035030 con 0x7f5cbc107900 2024-09-21T04:44:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.601+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cbc19a1d0 con 0x7f5cbc107900 2024-09-21T04:44:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.602+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cbc103040 con 0x7f5cbc107900 2024-09-21T04:44:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.603+0000 7f5cc099e640 1 -- 172.21.15.7:0/2731837179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5ca4042070 con 0x7f5cbc107900 2024-09-21T04:44:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.603+0000 7f5cc099e640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ca807af80 0x7f5ca807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.604+0000 7f5cc099e640 1 -- 172.21.15.7:0/2731837179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5ca40be8c0 con 0x7f5cbc107900 2024-09-21T04:44:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.604+0000 7f5cbaffd640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ca807af80 0x7f5ca807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.604+0000 7f5cbaffd640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ca807af80 0x7f5ca807d440 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc19ab50 tx=0x7f5cac00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.608+0000 7f5cc099e640 1 -- 172.21.15.7:0/2731837179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ca4089ee0 con 0x7f5cbc107900 2024-09-21T04:44:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.879+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f5cbc19b010 con 0x7f5cbc107900 2024-09-21T04:44:04.882 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:04 smithi007 ceph-mon[108238]: osdmap e79: 6 total, 6 up, 6 in 2024-09-21T04:44:04.882 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:04 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-21T04:44:04.883 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:04 smithi007 ceph-mon[108238]: pgmap v279: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:04.883 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:04 smithi007 ceph-mon[108238]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:04.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.880+0000 7f5cc099e640 1 -- 172.21.15.7:0/2731837179 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f5ca408c400 con 0x7f5cbc107900 2024-09-21T04:44:04.883 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:44:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:44:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:44:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:44:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 12 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:44:04.884 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:44:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ca807af80 msgr2=0x7f5ca807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ca807af80 0x7f5ca807d440 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc19ab50 tx=0x7f5cac00b390 comp rx=0 tx=0).stop 2024-09-21T04:44:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc107900 msgr2=0x7f5cbc19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc107900 0x7f5cbc19fd90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4031460 tx=0x7f5ca40045e0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 shutdown_connections 2024-09-21T04:44:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ca807af80 0x7f5ca807d440 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cbc107900 0x7f5cbc19fd90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 --2- 172.21.15.7:0/2731837179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cbc101900 0x7f5cbc19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 >> 172.21.15.7:0/2731837179 conn(0x7f5cbc0fd600 msgr2=0x7f5cbc074480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 shutdown_connections 2024-09-21T04:44:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.882+0000 7f5cc19a0640 1 -- 172.21.15.7:0/2731837179 wait complete. 2024-09-21T04:44:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.970+0000 7f0d0438e640 1 -- 172.21.15.7:0/959706339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc1033e0 msgr2=0x7f0cfc10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.970+0000 7f0d0438e640 1 --2- 172.21.15.7:0/959706339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc1033e0 0x7f0cfc10b8d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009880 tx=0x7f0cec02a530 comp rx=0 tx=0).stop 2024-09-21T04:44:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.971+0000 7f0d0438e640 1 -- 172.21.15.7:0/959706339 shutdown_connections 2024-09-21T04:44:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.971+0000 7f0d0438e640 1 --2- 172.21.15.7:0/959706339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc1033e0 0x7f0cfc10b8d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.971+0000 7f0d0438e640 1 --2- 172.21.15.7:0/959706339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cfc102ac0 0x7f0cfc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.971+0000 7f0d0438e640 1 -- 172.21.15.7:0/959706339 >> 172.21.15.7:0/959706339 conn(0x7f0cfc0fc520 msgr2=0x7f0cfc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.971+0000 7f0d0438e640 1 -- 172.21.15.7:0/959706339 shutdown_connections 2024-09-21T04:44:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.971+0000 7f0d0438e640 1 -- 172.21.15.7:0/959706339 wait complete. 2024-09-21T04:44:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.971+0000 7f0d0438e640 1 Processor -- start 2024-09-21T04:44:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.971+0000 7f0d0438e640 1 -- start start 2024-09-21T04:44:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0438e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc102ac0 0x7f0cfc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0438e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cfc1033e0 0x7f0cfc19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0438e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc10edf0 con 0x7f0cfc102ac0 2024-09-21T04:44:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0438e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc10ec70 con 0x7f0cfc1033e0 2024-09-21T04:44:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0212a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc102ac0 0x7f0cfc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0212a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc102ac0 0x7f0cfc19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46278/0 (socket says 172.21.15.7:46278) 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0212a640 1 -- 172.21.15.7:0/485105438 learned_addr learned my addr 172.21.15.7:0/485105438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d01929640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cfc1033e0 0x7f0cfc19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0212a640 1 -- 172.21.15.7:0/485105438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cfc1033e0 msgr2=0x7f0cfc19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0212a640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cfc1033e0 0x7f0cfc19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0212a640 1 -- 172.21.15.7:0/485105438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc197800 con 0x7f0cfc102ac0 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d01929640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cfc1033e0 0x7f0cfc19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0d0212a640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc102ac0 0x7f0cfc19d680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800cc60 tx=0x7f0cf800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.972+0000 7f0cf37fe640 1 -- 172.21.15.7:0/485105438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0cf800ad30 con 0x7f0cfc102ac0 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.973+0000 7f0cf37fe640 1 -- 172.21.15.7:0/485105438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0cf8005020 con 0x7f0cfc102ac0 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.973+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc197af0 con 0x7f0cfc102ac0 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.973+0000 7f0cf37fe640 1 -- 172.21.15.7:0/485105438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0cf800f640 con 0x7f0cfc102ac0 2024-09-21T04:44:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.973+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc198080 con 0x7f0cfc102ac0 2024-09-21T04:44:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.974+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cfc06aa90 con 0x7f0cfc102ac0 2024-09-21T04:44:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.974+0000 7f0cf37fe640 1 -- 172.21.15.7:0/485105438 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0cf80051d0 con 0x7f0cfc102ac0 2024-09-21T04:44:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.975+0000 7f0cf37fe640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ce807af80 0x7f0ce807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.975+0000 7f0cf37fe640 1 -- 172.21.15.7:0/485105438 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0cf809e210 con 0x7f0cfc102ac0 2024-09-21T04:44:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.975+0000 7f0d01929640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ce807af80 0x7f0ce807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:04.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.976+0000 7f0d01929640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ce807af80 0x7f0ce807d440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0cec031460 tx=0x7f0cec0095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:04.979+0000 7f0cf37fe640 1 -- 172.21.15.7:0/485105438 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0cf8014030 con 0x7f0cfc102ac0 2024-09-21T04:44:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:04 smithi053 ceph-mon[80176]: osdmap e79: 6 total, 6 up, 6 in 2024-09-21T04:44:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:04 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-21T04:44:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:04 smithi053 ceph-mon[80176]: pgmap v279: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:05.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:04 smithi053 ceph-mon[80176]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.207+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0cfc105f60 con 0x7f0cfc102ac0 2024-09-21T04:44:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.207+0000 7f0cf37fe640 1 -- 172.21.15.7:0/485105438 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1815 (secure 0 0 0) 0x7f0cf8069830 con 0x7f0cfc102ac0 2024-09-21T04:44:05.211 INFO:teuthology.orchestra.run.smithi007.stdout:e27 2024-09-21T04:44:05.211 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:03:751154+0000 2024-09-21T04:44:05.211 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:44:05.211 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:44:05.211 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:44:05.211 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:44:05.211 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:44:05.211 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:44:05.212 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:44:05.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:44:05.214 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:44:05.214 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:44:05.214 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:44:05.214 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:44:05.214 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:44:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-09-21T04:44:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ce807af80 msgr2=0x7f0ce807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ce807af80 0x7f0ce807d440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0cec031460 tx=0x7f0cec0095f0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc102ac0 msgr2=0x7f0cfc19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc102ac0 0x7f0cfc19d680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800cc60 tx=0x7f0cf800a620 comp rx=0 tx=0).stop 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 shutdown_connections 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ce807af80 0x7f0ce807d440 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cfc1033e0 0x7f0cfc19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 --2- 172.21.15.7:0/485105438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc102ac0 0x7f0cfc19d680 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 >> 172.21.15.7:0/485105438 conn(0x7f0cfc0fc520 msgr2=0x7f0cfc108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 shutdown_connections 2024-09-21T04:44:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.210+0000 7f0d0438e640 1 -- 172.21.15.7:0/485105438 wait complete. 2024-09-21T04:44:05.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.300+0000 7f6085fa9640 1 -- 172.21.15.7:0/1740587660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 msgr2=0x7f608010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.300+0000 7f6085fa9640 1 --2- 172.21.15.7:0/1740587660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 0x7f608010b930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f60740098b0 tx=0x7f607402a560 comp rx=0 tx=0).stop 2024-09-21T04:44:05.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.300+0000 7f6085fa9640 1 -- 172.21.15.7:0/1740587660 shutdown_connections 2024-09-21T04:44:05.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.300+0000 7f6085fa9640 1 --2- 172.21.15.7:0/1740587660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 0x7f608010b930 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.300+0000 7f6085fa9640 1 --2- 172.21.15.7:0/1740587660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6080102b20 0x7f6080102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.300+0000 7f6085fa9640 1 -- 172.21.15.7:0/1740587660 >> 172.21.15.7:0/1740587660 conn(0x7f60800fc520 msgr2=0x7f60800fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:05.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.300+0000 7f6085fa9640 1 -- 172.21.15.7:0/1740587660 shutdown_connections 2024-09-21T04:44:05.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.300+0000 7f6085fa9640 1 -- 172.21.15.7:0/1740587660 wait complete. 2024-09-21T04:44:05.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f6085fa9640 1 Processor -- start 2024-09-21T04:44:05.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f6085fa9640 1 -- start start 2024-09-21T04:44:05.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f6085fa9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6080102b20 0x7f608019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:05.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f6085fa9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 0x7f608019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f6085fa9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f608010ee50 con 0x7f6080103440 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f6085fa9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f608010ecd0 con 0x7f6080102b20 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f607effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 0x7f608019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f607effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 0x7f608019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46296/0 (socket says 172.21.15.7:46296) 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.301+0000 7f607effd640 1 -- 172.21.15.7:0/2859031758 learned_addr learned my addr 172.21.15.7:0/2859031758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f607f7fe640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6080102b20 0x7f608019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f607effd640 1 -- 172.21.15.7:0/2859031758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6080102b20 msgr2=0x7f608019d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f607effd640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6080102b20 0x7f608019d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f607effd640 1 -- 172.21.15.7:0/2859031758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6080197900 con 0x7f6080103440 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f607effd640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 0x7f608019dc80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6074031460 tx=0x7f60740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f607cff9640 1 -- 172.21.15.7:0/2859031758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f607403f070 con 0x7f6080103440 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6080197b90 con 0x7f6080103440 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f607cff9640 1 -- 172.21.15.7:0/2859031758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f60740044d0 con 0x7f6080103440 2024-09-21T04:44:05.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f607cff9640 1 -- 172.21.15.7:0/2859031758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6074035030 con 0x7f6080103440 2024-09-21T04:44:05.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.302+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60801980c0 con 0x7f6080103440 2024-09-21T04:44:05.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.303+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608006aa30 con 0x7f6080103440 2024-09-21T04:44:05.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.304+0000 7f607cff9640 1 -- 172.21.15.7:0/2859031758 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6074042070 con 0x7f6080103440 2024-09-21T04:44:05.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.304+0000 7f607cff9640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f606807af30 0x7f606807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:05.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.305+0000 7f607cff9640 1 -- 172.21.15.7:0/2859031758 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f60740bea90 con 0x7f6080103440 2024-09-21T04:44:05.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.305+0000 7f607f7fe640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f606807af30 0x7f606807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:05.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.305+0000 7f607f7fe640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f606807af30 0x7f606807d3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6080198a40 tx=0x7f6070009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.309+0000 7f607cff9640 1 -- 172.21.15.7:0/2859031758 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f607408a030 con 0x7f6080103440 2024-09-21T04:44:05.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.516+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6080106020 con 0x7f606807af30 2024-09-21T04:44:05.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.517+0000 7f607cff9640 1 -- 172.21.15.7:0/2859031758 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f60700095e0 con 0x7f606807af30 2024-09-21T04:44:05.520 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:44:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:44:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "14/23 daemons upgraded", 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:44:05.521 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:44:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.519+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f606807af30 msgr2=0x7f606807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.519+0000 7f6085fa9640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f606807af30 0x7f606807d3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6080198a40 tx=0x7f6070009110 comp rx=0 tx=0).stop 2024-09-21T04:44:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.519+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 msgr2=0x7f608019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.519+0000 7f6085fa9640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 0x7f608019dc80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6074031460 tx=0x7f60740048f0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.519+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 shutdown_connections 2024-09-21T04:44:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.519+0000 7f6085fa9640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f606807af30 0x7f606807d3f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.519+0000 7f6085fa9640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6080103440 0x7f608019dc80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.520+0000 7f6085fa9640 1 --2- 172.21.15.7:0/2859031758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6080102b20 0x7f608019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.520+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 >> 172.21.15.7:0/2859031758 conn(0x7f60800fc520 msgr2=0x7f6080103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.520+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 shutdown_connections 2024-09-21T04:44:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.520+0000 7f6085fa9640 1 -- 172.21.15.7:0/2859031758 wait complete. 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.605+0000 7f041fe94640 1 -- 172.21.15.7:0/851252778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418101900 msgr2=0x7f0418101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.605+0000 7f041fe94640 1 --2- 172.21.15.7:0/851252778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418101900 0x7f0418101d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0408009880 tx=0x7f040802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.605+0000 7f041fe94640 1 -- 172.21.15.7:0/851252778 shutdown_connections 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.605+0000 7f041fe94640 1 --2- 172.21.15.7:0/851252778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418101900 0x7f0418101d60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.605+0000 7f041fe94640 1 --2- 172.21.15.7:0/851252778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0418107900 0x7f0418107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.605+0000 7f041fe94640 1 -- 172.21.15.7:0/851252778 >> 172.21.15.7:0/851252778 conn(0x7f04180fd600 msgr2=0x7f04180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.605+0000 7f041fe94640 1 -- 172.21.15.7:0/851252778 shutdown_connections 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.605+0000 7f041fe94640 1 -- 172.21.15.7:0/851252778 wait complete. 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041fe94640 1 Processor -- start 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041fe94640 1 -- start start 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041fe94640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0418101900 0x7f041819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041fe94640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418107900 0x7f041819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041fe94640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f041810edd0 con 0x7f0418107900 2024-09-21T04:44:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041fe94640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f041810ec50 con 0x7f0418101900 2024-09-21T04:44:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041d42f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418107900 0x7f041819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041d42f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418107900 0x7f041819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46316/0 (socket says 172.21.15.7:46316) 2024-09-21T04:44:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041d42f640 1 -- 172.21.15.7:0/892823324 learned_addr learned my addr 172.21.15.7:0/892823324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041d42f640 1 -- 172.21.15.7:0/892823324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0418101900 msgr2=0x7f041819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041d42f640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0418101900 0x7f041819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041d42f640 1 -- 172.21.15.7:0/892823324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418197800 con 0x7f0418107900 2024-09-21T04:44:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.606+0000 7f041d42f640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418107900 0x7f041819dbc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0408031460 tx=0x7f04080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.607+0000 7f040effd640 1 -- 172.21.15.7:0/892823324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f040803f070 con 0x7f0418107900 2024-09-21T04:44:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.607+0000 7f040effd640 1 -- 172.21.15.7:0/892823324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0408004cf0 con 0x7f0418107900 2024-09-21T04:44:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.607+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0418197a90 con 0x7f0418107900 2024-09-21T04:44:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.607+0000 7f040effd640 1 -- 172.21.15.7:0/892823324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0408035030 con 0x7f0418107900 2024-09-21T04:44:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.607+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0418197fc0 con 0x7f0418107900 2024-09-21T04:44:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.608+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0418103040 con 0x7f0418107900 2024-09-21T04:44:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.608+0000 7f040effd640 1 -- 172.21.15.7:0/892823324 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0408042070 con 0x7f0418107900 2024-09-21T04:44:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.609+0000 7f040effd640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f040407af80 0x7f040407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.609+0000 7f040effd640 1 -- 172.21.15.7:0/892823324 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f04080bea40 con 0x7f0418107900 2024-09-21T04:44:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.609+0000 7f041dc30640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f040407af80 0x7f040407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.610+0000 7f041dc30640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f040407af80 0x7f040407d440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f041400a8f0 tx=0x7f0414005bf0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.613+0000 7f040effd640 1 -- 172.21.15.7:0/892823324 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f040808a060 con 0x7f0418107900 2024-09-21T04:44:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.875+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f0418109e70 con 0x7f0418107900 2024-09-21T04:44:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.875+0000 7f040effd640 1 -- 172.21.15.7:0/892823324 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f040808ef10 con 0x7f0418107900 2024-09-21T04:44:05.878 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:44:05.878 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:44:05.878 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:44:05.878 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:44:05.879 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:44:05.879 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:44:05.879 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:44:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f040407af80 msgr2=0x7f040407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f040407af80 0x7f040407d440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f041400a8f0 tx=0x7f0414005bf0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418107900 msgr2=0x7f041819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418107900 0x7f041819dbc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0408031460 tx=0x7f04080048f0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 shutdown_connections 2024-09-21T04:44:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f040407af80 0x7f040407d440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418107900 0x7f041819dbc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 --2- 172.21.15.7:0/892823324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0418101900 0x7f041819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:05.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.877+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 >> 172.21.15.7:0/892823324 conn(0x7f04180fd600 msgr2=0x7f04180fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:05.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.878+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 shutdown_connections 2024-09-21T04:44:05.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:05.878+0000 7f041fe94640 1 -- 172.21.15.7:0/892823324 wait complete. 2024-09-21T04:44:06.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:05 smithi053 ceph-mon[80176]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:06.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:05 smithi053 ceph-mon[80176]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:06.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:05 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2731837179' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:06.056 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:05 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/485105438' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:44:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:05 smithi007 ceph-mon[108238]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:05 smithi007 ceph-mon[108238]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:05 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2731837179' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:06.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:05 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/485105438' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:44:06.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:06 smithi053 ceph-mon[80176]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:06.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:06 smithi053 ceph-mon[80176]: pgmap v280: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:06.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:06 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/892823324' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:44:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:06 smithi007 ceph-mon[108238]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:06 smithi007 ceph-mon[108238]: pgmap v280: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:06 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/892823324' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:44:08.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:08.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:08.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:44:08.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:08 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] up:boot 2024-09-21T04:44:08.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:08 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:44:08.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"} : dispatch 2024-09-21T04:44:08.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:08 smithi053 ceph-mon[80176]: pgmap v281: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:08.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:08.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:08.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:44:08.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:08 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] up:boot 2024-09-21T04:44:08.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:08 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:44:08.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi053.wdnfat"} : dispatch 2024-09-21T04:44:08.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:08 smithi007 ceph-mon[108238]: pgmap v281: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:10 smithi053 ceph-mon[80176]: pgmap v282: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:10 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:10 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:10 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:11.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:10 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:10 smithi007 ceph-mon[108238]: pgmap v282: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:10 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:10 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:10 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:10 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:44:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:12 smithi053 ceph-mon[80176]: pgmap v283: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:12 smithi007 ceph-mon[108238]: pgmap v283: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:14.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:13 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:44:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:13 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:44:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:14 smithi053 ceph-mon[80176]: pgmap v284: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:15.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:14 smithi007 ceph-mon[108238]: pgmap v284: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:17.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:16 smithi053 ceph-mon[80176]: pgmap v285: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:16 smithi007 ceph-mon[108238]: pgmap v285: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:19.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:18 smithi053 ceph-mon[80176]: pgmap v286: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:18 smithi007 ceph-mon[108238]: pgmap v286: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:21.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:20 smithi053 ceph-mon[80176]: pgmap v287: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:20 smithi007 ceph-mon[108238]: pgmap v287: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:22 smithi053 ceph-mon[80176]: pgmap v288: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:22 smithi007 ceph-mon[108238]: pgmap v288: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:25.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:24 smithi053 ceph-mon[80176]: pgmap v289: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:24 smithi007 ceph-mon[108238]: pgmap v289: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:27.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:26 smithi053 ceph-mon[80176]: pgmap v290: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:27.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:26 smithi007 ceph-mon[108238]: pgmap v290: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:28 smithi053 ceph-mon[80176]: pgmap v291: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:44:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:28 smithi007 ceph-mon[108238]: pgmap v291: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:44:31.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:30 smithi053 ceph-mon[80176]: pgmap v292: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:31.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:30 smithi007 ceph-mon[108238]: pgmap v292: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:33.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:32 smithi053 ceph-mon[80176]: pgmap v293: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:32 smithi007 ceph-mon[108238]: pgmap v293: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:35.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:34 smithi053 ceph-mon[80176]: pgmap v294: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:34 smithi007 ceph-mon[108238]: pgmap v294: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.976+0000 7fb351df9640 1 -- 172.21.15.7:0/2029500899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 msgr2=0x7fb34c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.976+0000 7fb351df9640 1 --2- 172.21.15.7:0/2029500899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 0x7fb34c074a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009880 tx=0x7fb33c02a530 comp rx=0 tx=0).stop 2024-09-21T04:44:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.976+0000 7fb351df9640 1 -- 172.21.15.7:0/2029500899 shutdown_connections 2024-09-21T04:44:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.976+0000 7fb351df9640 1 --2- 172.21.15.7:0/2029500899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c074f40 0x7fb34c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.976+0000 7fb351df9640 1 --2- 172.21.15.7:0/2029500899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 0x7fb34c074a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.976+0000 7fb351df9640 1 -- 172.21.15.7:0/2029500899 >> 172.21.15.7:0/2029500899 conn(0x7fb34c0fd600 msgr2=0x7fb34c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.976+0000 7fb351df9640 1 -- 172.21.15.7:0/2029500899 shutdown_connections 2024-09-21T04:44:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.976+0000 7fb351df9640 1 -- 172.21.15.7:0/2029500899 wait complete. 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.977+0000 7fb351df9640 1 Processor -- start 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.977+0000 7fb351df9640 1 -- start start 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb351df9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 0x7fb34c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb351df9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c074f40 0x7fb34c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb351df9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c110ff0 con 0x7fb34c074620 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb351df9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c110e70 con 0x7fb34c074f40 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 0x7fb34c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 0x7fb34c19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38166/0 (socket says 172.21.15.7:38166) 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34b7fe640 1 -- 172.21.15.7:0/2374107777 learned_addr learned my addr 172.21.15.7:0/2374107777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34affd640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c074f40 0x7fb34c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34b7fe640 1 -- 172.21.15.7:0/2374107777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c074f40 msgr2=0x7fb34c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34b7fe640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c074f40 0x7fb34c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34b7fe640 1 -- 172.21.15.7:0/2374107777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c199a00 con 0x7fb34c074620 2024-09-21T04:44:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34affd640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c074f40 0x7fb34c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:44:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.978+0000 7fb34b7fe640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 0x7fb34c19f840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb33c031460 tx=0x7fb33c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.979+0000 7fb348ff9640 1 -- 172.21.15.7:0/2374107777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb33c03f070 con 0x7fb34c074620 2024-09-21T04:44:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.979+0000 7fb348ff9640 1 -- 172.21.15.7:0/2374107777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb33c004cf0 con 0x7fb34c074620 2024-09-21T04:44:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.979+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34c199c90 con 0x7fb34c074620 2024-09-21T04:44:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.979+0000 7fb348ff9640 1 -- 172.21.15.7:0/2374107777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb33c035030 con 0x7fb34c074620 2024-09-21T04:44:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.979+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34c19a1c0 con 0x7fb34c074620 2024-09-21T04:44:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.980+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34c075cd0 con 0x7fb34c074620 2024-09-21T04:44:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.981+0000 7fb348ff9640 1 -- 172.21.15.7:0/2374107777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb33c042070 con 0x7fb34c074620 2024-09-21T04:44:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.981+0000 7fb348ff9640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb33407af30 0x7fb33407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.982+0000 7fb34affd640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb33407af30 0x7fb33407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.982+0000 7fb348ff9640 1 -- 172.21.15.7:0/2374107777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb33c0beb20 con 0x7fb34c074620 2024-09-21T04:44:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.982+0000 7fb34affd640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb33407af30 0x7fb33407d3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb34c10ac30 tx=0x7fb3400082f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:35.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:35.986+0000 7fb348ff9640 1 -- 172.21.15.7:0/2374107777 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb33c08a0c0 con 0x7fb34c074620 2024-09-21T04:44:36.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.194+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb34c19af30 con 0x7fb33407af30 2024-09-21T04:44:36.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.195+0000 7fb348ff9640 1 -- 172.21.15.7:0/2374107777 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fb34c19af30 con 0x7fb33407af30 2024-09-21T04:44:36.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.197+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb33407af30 msgr2=0x7fb33407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb33407af30 0x7fb33407d3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb34c10ac30 tx=0x7fb3400082f0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 msgr2=0x7fb34c19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 0x7fb34c19f840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb33c031460 tx=0x7fb33c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 shutdown_connections 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb33407af30 0x7fb33407d3f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c074f40 0x7fb34c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 --2- 172.21.15.7:0/2374107777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c074620 0x7fb34c19f840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 >> 172.21.15.7:0/2374107777 conn(0x7fb34c0fd600 msgr2=0x7fb34c10b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 shutdown_connections 2024-09-21T04:44:36.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.198+0000 7fb351df9640 1 -- 172.21.15.7:0/2374107777 wait complete. 2024-09-21T04:44:36.210 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:44:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.284+0000 7f08c19a2640 1 -- 172.21.15.7:0/689701839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1067f0 msgr2=0x7f08bc106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.284+0000 7f08c19a2640 1 --2- 172.21.15.7:0/689701839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1067f0 0x7f08bc106bd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009880 tx=0x7f08ac02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.284+0000 7f08c19a2640 1 -- 172.21.15.7:0/689701839 shutdown_connections 2024-09-21T04:44:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.284+0000 7f08c19a2640 1 --2- 172.21.15.7:0/689701839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1007f0 0x7f08bc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.284+0000 7f08c19a2640 1 --2- 172.21.15.7:0/689701839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1067f0 0x7f08bc106bd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.284+0000 7f08c19a2640 1 -- 172.21.15.7:0/689701839 >> 172.21.15.7:0/689701839 conn(0x7f08bc0fc550 msgr2=0x7f08bc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.284+0000 7f08c19a2640 1 -- 172.21.15.7:0/689701839 shutdown_connections 2024-09-21T04:44:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.284+0000 7f08c19a2640 1 -- 172.21.15.7:0/689701839 wait complete. 2024-09-21T04:44:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08c19a2640 1 Processor -- start 2024-09-21T04:44:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08c19a2640 1 -- start start 2024-09-21T04:44:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08c19a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1007f0 0x7f08bc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08c19a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1067f0 0x7f08bc19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08c19a2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc10dca0 con 0x7f08bc1007f0 2024-09-21T04:44:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08c19a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc10db20 con 0x7f08bc1067f0 2024-09-21T04:44:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08baffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1007f0 0x7f08bc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08baffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1007f0 0x7f08bc19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38198/0 (socket says 172.21.15.7:38198) 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08baffd640 1 -- 172.21.15.7:0/1642591542 learned_addr learned my addr 172.21.15.7:0/1642591542 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08b3fff640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1067f0 0x7f08bc19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08baffd640 1 -- 172.21.15.7:0/1642591542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1067f0 msgr2=0x7f08bc19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08baffd640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1067f0 0x7f08bc19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.285+0000 7f08baffd640 1 -- 172.21.15.7:0/1642591542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08bc197750 con 0x7f08bc1007f0 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.286+0000 7f08baffd640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1007f0 0x7f08bc19d680 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f08ac031460 tx=0x7f08ac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.286+0000 7f08c09a0640 1 -- 172.21.15.7:0/1642591542 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f08ac03f070 con 0x7f08bc1007f0 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.286+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08bc1979e0 con 0x7f08bc1007f0 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.286+0000 7f08c09a0640 1 -- 172.21.15.7:0/1642591542 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08ac0035f0 con 0x7f08bc1007f0 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.286+0000 7f08c09a0640 1 -- 172.21.15.7:0/1642591542 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f08ac035030 con 0x7f08bc1007f0 2024-09-21T04:44:36.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.286+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08bc197ee0 con 0x7f08bc1007f0 2024-09-21T04:44:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.287+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08bc101f30 con 0x7f08bc1007f0 2024-09-21T04:44:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.288+0000 7f08c09a0640 1 -- 172.21.15.7:0/1642591542 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f08ac042070 con 0x7f08bc1007f0 2024-09-21T04:44:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.288+0000 7f08c09a0640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f08a807af80 0x7f08a807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.289+0000 7f08c09a0640 1 -- 172.21.15.7:0/1642591542 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f08ac0be8c0 con 0x7f08bc1007f0 2024-09-21T04:44:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.289+0000 7f08b3fff640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f08a807af80 0x7f08a807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.289+0000 7f08b3fff640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f08a807af80 0x7f08a807d440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08bc198fa0 tx=0x7f08a400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.293+0000 7f08c09a0640 1 -- 172.21.15.7:0/1642591542 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f08ac089ee0 con 0x7f08bc1007f0 2024-09-21T04:44:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.499+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f08bc198190 con 0x7f08a807af80 2024-09-21T04:44:36.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.500+0000 7f08c09a0640 1 -- 172.21.15.7:0/1642591542 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f08bc198190 con 0x7f08a807af80 2024-09-21T04:44:36.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f08a807af80 msgr2=0x7f08a807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f08a807af80 0x7f08a807d440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08bc198fa0 tx=0x7f08a400b390 comp rx=0 tx=0).stop 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1007f0 msgr2=0x7f08bc19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1007f0 0x7f08bc19d680 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f08ac031460 tx=0x7f08ac0045e0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 shutdown_connections 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f08a807af80 0x7f08a807d440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc1067f0 0x7f08bc19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 --2- 172.21.15.7:0/1642591542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc1007f0 0x7f08bc19d680 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.502+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 >> 172.21.15.7:0/1642591542 conn(0x7f08bc0fc550 msgr2=0x7f08bc069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.503+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 shutdown_connections 2024-09-21T04:44:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.503+0000 7f08c19a2640 1 -- 172.21.15.7:0/1642591542 wait complete. 2024-09-21T04:44:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.587+0000 7f206e19f640 1 -- 172.21.15.7:0/3730283907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 msgr2=0x7f2068107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.587+0000 7f206e19f640 1 --2- 172.21.15.7:0/3730283907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 0x7f2068107ce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2058009880 tx=0x7f205802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.587+0000 7f206e19f640 1 -- 172.21.15.7:0/3730283907 shutdown_connections 2024-09-21T04:44:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.587+0000 7f206e19f640 1 --2- 172.21.15.7:0/3730283907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2068101900 0x7f2068101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.587+0000 7f206e19f640 1 --2- 172.21.15.7:0/3730283907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 0x7f2068107ce0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.587+0000 7f206e19f640 1 -- 172.21.15.7:0/3730283907 >> 172.21.15.7:0/3730283907 conn(0x7f20680fd600 msgr2=0x7f20680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.587+0000 7f206e19f640 1 -- 172.21.15.7:0/3730283907 shutdown_connections 2024-09-21T04:44:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.587+0000 7f206e19f640 1 -- 172.21.15.7:0/3730283907 wait complete. 2024-09-21T04:44:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.588+0000 7f206e19f640 1 Processor -- start 2024-09-21T04:44:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.588+0000 7f206e19f640 1 -- start start 2024-09-21T04:44:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.588+0000 7f206e19f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2068101900 0x7f206819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.588+0000 7f206e19f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 0x7f206819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.588+0000 7f206e19f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f206810ee30 con 0x7f2068107900 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.588+0000 7f206e19f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f206810ecb0 con 0x7f2068101900 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 0x7f206819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2066ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 0x7f206819dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38208/0 (socket says 172.21.15.7:38208) 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2066ffd640 1 -- 172.21.15.7:0/3017775190 learned_addr learned my addr 172.21.15.7:0/3017775190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f20677fe640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2068101900 0x7f206819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2066ffd640 1 -- 172.21.15.7:0/3017775190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2068101900 msgr2=0x7f206819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2066ffd640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2068101900 0x7f206819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2066ffd640 1 -- 172.21.15.7:0/3017775190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2068197860 con 0x7f2068107900 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2066ffd640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 0x7f206819dbe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b420 tx=0x7f205c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2064ff9640 1 -- 172.21.15.7:0/3017775190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f205c011630 con 0x7f2068107900 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f2064ff9640 1 -- 172.21.15.7:0/3017775190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f205c016070 con 0x7f2068107900 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.589+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2068197b50 con 0x7f2068107900 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.590+0000 7f2064ff9640 1 -- 172.21.15.7:0/3017775190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f205c010d40 con 0x7f2068107900 2024-09-21T04:44:36.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.590+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20681980e0 con 0x7f2068107900 2024-09-21T04:44:36.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.591+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2068103040 con 0x7f2068107900 2024-09-21T04:44:36.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.591+0000 7f2064ff9640 1 -- 172.21.15.7:0/3017775190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f205c01b480 con 0x7f2068107900 2024-09-21T04:44:36.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.592+0000 7f2064ff9640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f205007af30 0x7f205007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.592+0000 7f20677fe640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f205007af30 0x7f205007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.592+0000 7f2064ff9640 1 -- 172.21.15.7:0/3017775190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f205c09d6f0 con 0x7f2068107900 2024-09-21T04:44:36.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.593+0000 7f20677fe640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f205007af30 0x7f205007d3f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2058000c00 tx=0x7f2058031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:36.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.596+0000 7f2064ff9640 1 -- 172.21.15.7:0/3017775190 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f205c068c90 con 0x7f2068107900 2024-09-21T04:44:36.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.792+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2068198d90 con 0x7f205007af30 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.802+0000 7f2064ff9640 1 -- 172.21.15.7:0/3017775190 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f2068198d90 con 0x7f205007af30 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (6m) 40s ago 17m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (9m) 40s ago 17m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (9m) 26s ago 16m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (4m) 40s ago 17m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (4m) 26s ago 16m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (6m) 40s ago 17m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:44:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (45s) 40s ago 12m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (62s) 40s ago 12m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (12m) 26s ago 12m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (29s) 26s ago 12m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (8m) 40s ago 18m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (7m) 26s ago 16m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (5m) 40s ago 19m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (5m) 26s ago 16m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (7m) 40s ago 17m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (7m) 26s ago 16m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (4m) 40s ago 15m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (3m) 40s ago 15m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (3m) 40s ago 14m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (2m) 26s ago 14m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (115s) 26s ago 13m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (79s) 26s ago 13m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:44:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (6m) 40s ago 17m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:44:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f205007af30 msgr2=0x7f205007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f205007af30 0x7f205007d3f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2058000c00 tx=0x7f2058031400 comp rx=0 tx=0).stop 2024-09-21T04:44:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 msgr2=0x7f206819dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 0x7f206819dbe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b420 tx=0x7f205c00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 shutdown_connections 2024-09-21T04:44:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f205007af30 0x7f205007d3f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2068107900 0x7f206819dbe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 --2- 172.21.15.7:0/3017775190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2068101900 0x7f206819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 >> 172.21.15.7:0/3017775190 conn(0x7f20680fd600 msgr2=0x7f20680fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:36.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 shutdown_connections 2024-09-21T04:44:36.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.804+0000 7f206e19f640 1 -- 172.21.15.7:0/3017775190 wait complete. 2024-09-21T04:44:36.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.888+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/279927265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb981033e0 msgr2=0x7fcb9810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.888+0000 7fcb9cdb9640 1 --2- 172.21.15.7:0/279927265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb981033e0 0x7fcb9810b8d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009880 tx=0x7fcb8c02a530 comp rx=0 tx=0).stop 2024-09-21T04:44:36.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.888+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/279927265 shutdown_connections 2024-09-21T04:44:36.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.888+0000 7fcb9cdb9640 1 --2- 172.21.15.7:0/279927265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb981033e0 0x7fcb9810b8d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.888+0000 7fcb9cdb9640 1 --2- 172.21.15.7:0/279927265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb98102ac0 0x7fcb98102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.888+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/279927265 >> 172.21.15.7:0/279927265 conn(0x7fcb980fc520 msgr2=0x7fcb980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:36.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.889+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/279927265 shutdown_connections 2024-09-21T04:44:36.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.889+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/279927265 wait complete. 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.889+0000 7fcb9cdb9640 1 Processor -- start 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.889+0000 7fcb9cdb9640 1 -- start start 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9cdb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb98102ac0 0x7fcb9819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9cdb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb981033e0 0x7fcb9819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9cdb9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9810edd0 con 0x7fcb98102ac0 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9cdb9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9810ec50 con 0x7fcb981033e0 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb98102ac0 0x7fcb9819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb98102ac0 0x7fcb9819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38224/0 (socket says 172.21.15.7:38224) 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9659c640 1 -- 172.21.15.7:0/1563083595 learned_addr learned my addr 172.21.15.7:0/1563083595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb95d9b640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb981033e0 0x7fcb9819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9659c640 1 -- 172.21.15.7:0/1563083595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb981033e0 msgr2=0x7fcb9819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:36.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9659c640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb981033e0 0x7fcb9819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:36.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9659c640 1 -- 172.21.15.7:0/1563083595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb981977e0 con 0x7fcb98102ac0 2024-09-21T04:44:36.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.890+0000 7fcb9659c640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb98102ac0 0x7fcb9819d660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800d8d0 tx=0x7fcb8800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:36.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.891+0000 7fcb877fe640 1 -- 172.21.15.7:0/1563083595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcb88017030 con 0x7fcb98102ac0 2024-09-21T04:44:36.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.891+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb98197ad0 con 0x7fcb98102ac0 2024-09-21T04:44:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.891+0000 7fcb877fe640 1 -- 172.21.15.7:0/1563083595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb88004020 con 0x7fcb98102ac0 2024-09-21T04:44:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.891+0000 7fcb877fe640 1 -- 172.21.15.7:0/1563083595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcb880047a0 con 0x7fcb98102ac0 2024-09-21T04:44:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.891+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb98198030 con 0x7fcb98102ac0 2024-09-21T04:44:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.892+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9806aa90 con 0x7fcb98102ac0 2024-09-21T04:44:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.892+0000 7fcb877fe640 1 -- 172.21.15.7:0/1563083595 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fcb880041d0 con 0x7fcb98102ac0 2024-09-21T04:44:36.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.893+0000 7fcb877fe640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcb8007af30 0x7fcb8007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:36.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.893+0000 7fcb877fe640 1 -- 172.21.15.7:0/1563083595 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcb88018070 con 0x7fcb98102ac0 2024-09-21T04:44:36.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.893+0000 7fcb95d9b640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcb8007af30 0x7fcb8007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:36.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.894+0000 7fcb95d9b640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcb8007af30 0x7fcb8007d3f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb98198b30 tx=0x7fcb8c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:36.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:36.897+0000 7fcb877fe640 1 -- 172.21.15.7:0/1563083595 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb880a4080 con 0x7fcb98102ac0 2024-09-21T04:44:37.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.162+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fcb98105f60 con 0x7fcb98102ac0 2024-09-21T04:44:37.166 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:36 smithi007 ceph-mon[108238]: pgmap v295: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:37.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.163+0000 7fcb877fe640 1 -- 172.21.15.7:0/1563083595 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fcb88069350 con 0x7fcb98102ac0 2024-09-21T04:44:37.166 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:44:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:44:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:44:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:44:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:44:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:44:37.168 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:44:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcb8007af30 msgr2=0x7fcb8007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcb8007af30 0x7fcb8007d3f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb98198b30 tx=0x7fcb8c031400 comp rx=0 tx=0).stop 2024-09-21T04:44:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb98102ac0 msgr2=0x7fcb9819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb98102ac0 0x7fcb9819d660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800d8d0 tx=0x7fcb8800dd90 comp rx=0 tx=0).stop 2024-09-21T04:44:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 shutdown_connections 2024-09-21T04:44:37.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcb8007af30 0x7fcb8007d3f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb981033e0 0x7fcb9819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 --2- 172.21.15.7:0/1563083595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb98102ac0 0x7fcb9819d660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 >> 172.21.15.7:0/1563083595 conn(0x7fcb980fc520 msgr2=0x7fcb98108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:37.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 shutdown_connections 2024-09-21T04:44:37.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.165+0000 7fcb9cdb9640 1 -- 172.21.15.7:0/1563083595 wait complete. 2024-09-21T04:44:37.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:36 smithi053 ceph-mon[80176]: pgmap v295: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.249+0000 7f1d86c74640 1 -- 172.21.15.7:0/304393534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80107900 msgr2=0x7f1d80107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.249+0000 7f1d86c74640 1 --2- 172.21.15.7:0/304393534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80107900 0x7f1d80107ce0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009c20 tx=0x7f1d7802a970 comp rx=0 tx=0).stop 2024-09-21T04:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 -- 172.21.15.7:0/304393534 shutdown_connections 2024-09-21T04:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 --2- 172.21.15.7:0/304393534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d80101900 0x7f1d80101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 --2- 172.21.15.7:0/304393534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80107900 0x7f1d80107ce0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 -- 172.21.15.7:0/304393534 >> 172.21.15.7:0/304393534 conn(0x7f1d800fd600 msgr2=0x7f1d800ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 -- 172.21.15.7:0/304393534 shutdown_connections 2024-09-21T04:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 -- 172.21.15.7:0/304393534 wait complete. 2024-09-21T04:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 Processor -- start 2024-09-21T04:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 -- start start 2024-09-21T04:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.250+0000 7f1d86c74640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80101900 0x7f1d8019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d86c74640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d80107900 0x7f1d8019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d86c74640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8010edd0 con 0x7f1d80101900 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d86c74640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8010ec50 con 0x7f1d80107900 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d84a10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80101900 0x7f1d8019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d84a10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80101900 0x7f1d8019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38246/0 (socket says 172.21.15.7:38246) 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d84a10640 1 -- 172.21.15.7:0/3814149104 learned_addr learned my addr 172.21.15.7:0/3814149104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d7ffff640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d80107900 0x7f1d8019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d84a10640 1 -- 172.21.15.7:0/3814149104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d80107900 msgr2=0x7f1d8019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d84a10640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d80107900 0x7f1d8019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d84a10640 1 -- 172.21.15.7:0/3814149104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d801977a0 con 0x7f1d80101900 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d84a10640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80101900 0x7f1d8019d620 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d7802ae90 tx=0x7f1d7802fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d7dffb640 1 -- 172.21.15.7:0/3814149104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1d78047070 con 0x7f1d80101900 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d80197a30 con 0x7f1d80101900 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d7dffb640 1 -- 172.21.15.7:0/3814149104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d78003e30 con 0x7f1d80101900 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.251+0000 7f1d7dffb640 1 -- 172.21.15.7:0/3814149104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1d78034070 con 0x7f1d80101900 2024-09-21T04:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.252+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d80197f60 con 0x7f1d80101900 2024-09-21T04:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.253+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d80103040 con 0x7f1d80101900 2024-09-21T04:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.253+0000 7f1d7dffb640 1 -- 172.21.15.7:0/3814149104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1d7804a070 con 0x7f1d80101900 2024-09-21T04:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.254+0000 7f1d7dffb640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d6c07af30 0x7f1d6c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.254+0000 7f1d7dffb640 1 -- 172.21.15.7:0/3814149104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f1d780bec50 con 0x7f1d80101900 2024-09-21T04:44:37.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.254+0000 7f1d7ffff640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d6c07af30 0x7f1d6c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.254+0000 7f1d7ffff640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d6c07af30 0x7f1d6c07d3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d800ffc20 tx=0x7f1d70005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.258+0000 7f1d7dffb640 1 -- 172.21.15.7:0/3814149104 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d7803f070 con 0x7f1d80101900 2024-09-21T04:44:37.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.485+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1d80109e70 con 0x7f1d80101900 2024-09-21T04:44:37.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.486+0000 7f1d7dffb640 1 -- 172.21.15.7:0/3814149104 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f1d7808a1f0 con 0x7f1d80101900 2024-09-21T04:44:37.489 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:44:37.489 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:44:37.489 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:44:37.489 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:44:37.489 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:44:37.489 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:44:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:44:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:44:37.492 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:44:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:44:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d6c07af30 msgr2=0x7f1d6c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d6c07af30 0x7f1d6c07d3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d800ffc20 tx=0x7f1d70005f90 comp rx=0 tx=0).stop 2024-09-21T04:44:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80101900 msgr2=0x7f1d8019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80101900 0x7f1d8019d620 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d7802ae90 tx=0x7f1d7802fd60 comp rx=0 tx=0).stop 2024-09-21T04:44:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 shutdown_connections 2024-09-21T04:44:37.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d6c07af30 0x7f1d6c07d3f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d80107900 0x7f1d8019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 --2- 172.21.15.7:0/3814149104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d80101900 0x7f1d8019d620 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 >> 172.21.15.7:0/3814149104 conn(0x7f1d800fd600 msgr2=0x7f1d800fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:37.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 shutdown_connections 2024-09-21T04:44:37.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.488+0000 7f1d86c74640 1 -- 172.21.15.7:0/3814149104 wait complete. 2024-09-21T04:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 -- 172.21.15.7:0/1162631244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08101900 msgr2=0x7f7d08101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 --2- 172.21.15.7:0/1162631244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08101900 0x7f7d08101d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009880 tx=0x7f7cf802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 -- 172.21.15.7:0/1162631244 shutdown_connections 2024-09-21T04:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 --2- 172.21.15.7:0/1162631244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08101900 0x7f7d08101d60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 --2- 172.21.15.7:0/1162631244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d08107900 0x7f7d08107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 -- 172.21.15.7:0/1162631244 >> 172.21.15.7:0/1162631244 conn(0x7f7d080fd600 msgr2=0x7f7d080ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 -- 172.21.15.7:0/1162631244 shutdown_connections 2024-09-21T04:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 -- 172.21.15.7:0/1162631244 wait complete. 2024-09-21T04:44:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 Processor -- start 2024-09-21T04:44:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.574+0000 7f7d0f686640 1 -- start start 2024-09-21T04:44:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0f686640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d08101900 0x7f7d0819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0f686640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08107900 0x7f7d0819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0f686640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810edd0 con 0x7f7d08107900 2024-09-21T04:44:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0f686640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0810ec50 con 0x7f7d08101900 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0cc21640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08107900 0x7f7d0819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0cc21640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08107900 0x7f7d0819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38266/0 (socket says 172.21.15.7:38266) 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0cc21640 1 -- 172.21.15.7:0/3065307513 learned_addr learned my addr 172.21.15.7:0/3065307513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0d422640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d08101900 0x7f7d0819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0cc21640 1 -- 172.21.15.7:0/3065307513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d08101900 msgr2=0x7f7d0819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0cc21640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d08101900 0x7f7d0819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0cc21640 1 -- 172.21.15.7:0/3065307513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d081977a0 con 0x7f7d08107900 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0cc21640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08107900 0x7f7d0819db60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8031460 tx=0x7f7cf8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7cfe7fc640 1 -- 172.21.15.7:0/3065307513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cf803f070 con 0x7f7d08107900 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7cfe7fc640 1 -- 172.21.15.7:0/3065307513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cf80035f0 con 0x7f7d08107900 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.575+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d08197a30 con 0x7f7d08107900 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.576+0000 7f7cfe7fc640 1 -- 172.21.15.7:0/3065307513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cf8035030 con 0x7f7d08107900 2024-09-21T04:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.576+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d08197f60 con 0x7f7d08107900 2024-09-21T04:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.576+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d08103040 con 0x7f7d08107900 2024-09-21T04:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.577+0000 7f7cfe7fc640 1 -- 172.21.15.7:0/3065307513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7cf8042070 con 0x7f7d08107900 2024-09-21T04:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.578+0000 7f7cfe7fc640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf407af80 0x7f7cf407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.578+0000 7f7d0d422640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf407af80 0x7f7cf407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.578+0000 7f7cfe7fc640 1 -- 172.21.15.7:0/3065307513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7cf80be890 con 0x7f7d08107900 2024-09-21T04:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.578+0000 7f7d0d422640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf407af80 0x7f7cf407d440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009bd0 tx=0x7f7d04005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.582+0000 7f7cfe7fc640 1 -- 172.21.15.7:0/3065307513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7cf8089eb0 con 0x7f7d08107900 2024-09-21T04:44:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.791+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7d08198210 con 0x7f7cf407af80 2024-09-21T04:44:37.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.792+0000 7f7cfe7fc640 1 -- 172.21.15.7:0/3065307513 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f7d08198210 con 0x7f7cf407af80 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:44:37.795 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:44:37.796 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:44:37.796 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:44:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf407af80 msgr2=0x7f7cf407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf407af80 0x7f7cf407d440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009bd0 tx=0x7f7d04005f90 comp rx=0 tx=0).stop 2024-09-21T04:44:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08107900 msgr2=0x7f7d0819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08107900 0x7f7d0819db60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8031460 tx=0x7f7cf8004530 comp rx=0 tx=0).stop 2024-09-21T04:44:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 shutdown_connections 2024-09-21T04:44:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf407af80 0x7f7cf407d440 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d08107900 0x7f7d0819db60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 --2- 172.21.15.7:0/3065307513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d08101900 0x7f7d0819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 >> 172.21.15.7:0/3065307513 conn(0x7f7d080fd600 msgr2=0x7f7d080fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 shutdown_connections 2024-09-21T04:44:37.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.794+0000 7f7d0f686640 1 -- 172.21.15.7:0/3065307513 wait complete. 2024-09-21T04:44:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.879+0000 7f34a5dd6640 1 -- 172.21.15.7:0/629778354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0102b20 msgr2=0x7f34a0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.879+0000 7f34a5dd6640 1 --2- 172.21.15.7:0/629778354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0102b20 0x7f34a0102f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f34900098b0 tx=0x7f349002a560 comp rx=0 tx=0).stop 2024-09-21T04:44:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.879+0000 7f34a5dd6640 1 -- 172.21.15.7:0/629778354 shutdown_connections 2024-09-21T04:44:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.879+0000 7f34a5dd6640 1 --2- 172.21.15.7:0/629778354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a0103440 0x7f34a010b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.879+0000 7f34a5dd6640 1 --2- 172.21.15.7:0/629778354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0102b20 0x7f34a0102f00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.879+0000 7f34a5dd6640 1 -- 172.21.15.7:0/629778354 >> 172.21.15.7:0/629778354 conn(0x7f34a00fc520 msgr2=0x7f34a00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:37.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.879+0000 7f34a5dd6640 1 -- 172.21.15.7:0/629778354 shutdown_connections 2024-09-21T04:44:37.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.879+0000 7f34a5dd6640 1 -- 172.21.15.7:0/629778354 wait complete. 2024-09-21T04:44:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f34a5dd6640 1 Processor -- start 2024-09-21T04:44:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f34a5dd6640 1 -- start start 2024-09-21T04:44:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f34a5dd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a0102b20 0x7f34a0100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f34a5dd6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0103440 0x7f34a0100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f34a5dd6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010edd0 con 0x7f34a0103440 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f34a5dd6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f34a010ec50 con 0x7f34a0102b20 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f349effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0103440 0x7f34a0100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f349effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0103440 0x7f34a0100cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38282/0 (socket says 172.21.15.7:38282) 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f349effd640 1 -- 172.21.15.7:0/1233551225 learned_addr learned my addr 172.21.15.7:0/1233551225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.880+0000 7f349f7fe640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a0102b20 0x7f34a0100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f349effd640 1 -- 172.21.15.7:0/1233551225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a0102b20 msgr2=0x7f34a0100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f349effd640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a0102b20 0x7f34a0100780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f349effd640 1 -- 172.21.15.7:0/1233551225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a0101200 con 0x7f34a0103440 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f349effd640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0103440 0x7f34a0100cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f349400c8a0 tx=0x7f349400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f349cff9640 1 -- 172.21.15.7:0/1233551225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f34940079b0 con 0x7f34a0103440 2024-09-21T04:44:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a01014f0 con 0x7f34a0103440 2024-09-21T04:44:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f349cff9640 1 -- 172.21.15.7:0/1233551225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3494007b50 con 0x7f34a0103440 2024-09-21T04:44:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f349cff9640 1 -- 172.21.15.7:0/1233551225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3494018700 con 0x7f34a0103440 2024-09-21T04:44:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.881+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a3e20 con 0x7f34a0103440 2024-09-21T04:44:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.882+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a006aa30 con 0x7f34a0103440 2024-09-21T04:44:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.883+0000 7f349cff9640 1 -- 172.21.15.7:0/1233551225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3494007d00 con 0x7f34a0103440 2024-09-21T04:44:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.884+0000 7f349cff9640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f348807af30 0x7f348807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:44:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.884+0000 7f349f7fe640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f348807af30 0x7f348807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:44:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.884+0000 7f349cff9640 1 -- 172.21.15.7:0/1233551225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f349409e2d0 con 0x7f34a0103440 2024-09-21T04:44:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.884+0000 7f349f7fe640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f348807af30 0x7f348807d3f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3490000c00 tx=0x7f3490031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:44:37.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:37.888+0000 7f349cff9640 1 -- 172.21.15.7:0/1233551225 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3494014030 con 0x7f34a0103440 2024-09-21T04:44:38.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.151+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f34a002d7c0 con 0x7f34a0103440 2024-09-21T04:44:38.154 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:37 smithi007 ceph-mon[108238]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:38.154 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:37 smithi007 ceph-mon[108238]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:38.154 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:37 smithi007 ceph-mon[108238]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:38.155 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:37 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1563083595' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:38.155 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:37 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3814149104' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:44:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.151+0000 7f349cff9640 1 -- 172.21.15.7:0/1233551225 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f3494069810 con 0x7f34a0103440 2024-09-21T04:44:38.155 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:44:38.155 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:44:38.155 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:44:38.155 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:44:38.155 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:44:38.155 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:44:38.156 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:44:38.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f348807af30 msgr2=0x7f348807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:38.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f348807af30 0x7f348807d3f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3490000c00 tx=0x7f3490031400 comp rx=0 tx=0).stop 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0103440 msgr2=0x7f34a0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0103440 0x7f34a0100cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f349400c8a0 tx=0x7f349400cd60 comp rx=0 tx=0).stop 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 shutdown_connections 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f348807af30 0x7f348807d3f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34a0103440 0x7f34a0100cc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 --2- 172.21.15.7:0/1233551225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34a0102b20 0x7f34a0100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.153+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 >> 172.21.15.7:0/1233551225 conn(0x7f34a00fc520 msgr2=0x7f34a0103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.154+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 shutdown_connections 2024-09-21T04:44:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:44:38.154+0000 7f34a5dd6640 1 -- 172.21.15.7:0/1233551225 wait complete. 2024-09-21T04:44:38.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:37 smithi053 ceph-mon[80176]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:37 smithi053 ceph-mon[80176]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:37 smithi053 ceph-mon[80176]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:37 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1563083595' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:44:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:37 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3814149104' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:44:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:38 smithi053 ceph-mon[80176]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:38 smithi053 ceph-mon[80176]: pgmap v296: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:38 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1233551225' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:44:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:38 smithi007 ceph-mon[108238]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:44:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:38 smithi007 ceph-mon[108238]: pgmap v296: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:38 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1233551225' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:44:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:40 smithi007 ceph-mon[108238]: pgmap v297: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:41.394 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:40 smithi053 ceph-mon[80176]: pgmap v297: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:42 smithi053 ceph-mon[80176]: pgmap v298: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:42 smithi007 ceph-mon[108238]: pgmap v298: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:44:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:44:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:44 smithi007 ceph-mon[108238]: pgmap v299: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:44 smithi053 ceph-mon[80176]: pgmap v299: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:46 smithi007 ceph-mon[108238]: pgmap v300: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:46 smithi053 ceph-mon[80176]: pgmap v300: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:49.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:48 smithi053 ceph-mon[80176]: pgmap v301: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:48 smithi007 ceph-mon[108238]: pgmap v301: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:50 smithi053 ceph-mon[80176]: pgmap v302: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:50 smithi007 ceph-mon[108238]: pgmap v302: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:52 smithi007 ceph-mon[108238]: pgmap v303: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:52 smithi053 ceph-mon[80176]: pgmap v303: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:54 smithi007 ceph-mon[108238]: pgmap v304: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:54 smithi053 ceph-mon[80176]: pgmap v304: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:56 smithi053 ceph-mon[80176]: pgmap v305: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:56 smithi007 ceph-mon[108238]: pgmap v305: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:58 smithi007 ceph-mon[108238]: pgmap v306: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:44:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:44:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:58 smithi053 ceph-mon[80176]: pgmap v306: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:44:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:44:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:45:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:00 smithi053 ceph-mon[80176]: pgmap v307: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:00 smithi007 ceph-mon[108238]: pgmap v307: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:02 smithi053 ceph-mon[80176]: pgmap v308: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:02 smithi007 ceph-mon[108238]: pgmap v308: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:04 smithi007 ceph-mon[108238]: pgmap v309: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:04 smithi053 ceph-mon[80176]: pgmap v309: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:07.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:06 smithi053 ceph-mon[80176]: pgmap v310: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:06 smithi007 ceph-mon[108238]: pgmap v310: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.251+0000 7fd16ca47640 1 -- 172.21.15.7:0/2669434083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 msgr2=0x7fd168106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.251+0000 7fd16ca47640 1 --2- 172.21.15.7:0/2669434083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 0x7fd168106be0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd158009880 tx=0x7fd15802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.251+0000 7fd16ca47640 1 -- 172.21.15.7:0/2669434083 shutdown_connections 2024-09-21T04:45:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.251+0000 7fd16ca47640 1 --2- 172.21.15.7:0/2669434083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168100800 0x7fd168100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.251+0000 7fd16ca47640 1 --2- 172.21.15.7:0/2669434083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 0x7fd168106be0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.251+0000 7fd16ca47640 1 -- 172.21.15.7:0/2669434083 >> 172.21.15.7:0/2669434083 conn(0x7fd1680fc520 msgr2=0x7fd1680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.251+0000 7fd16ca47640 1 -- 172.21.15.7:0/2669434083 shutdown_connections 2024-09-21T04:45:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.251+0000 7fd16ca47640 1 -- 172.21.15.7:0/2669434083 wait complete. 2024-09-21T04:45:08.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.252+0000 7fd16ca47640 1 Processor -- start 2024-09-21T04:45:08.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.252+0000 7fd16ca47640 1 -- start start 2024-09-21T04:45:08.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd16ca47640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168100800 0x7fd1681987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd16ca47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 0x7fd168198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd16ca47640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810dcd0 con 0x7fd168106800 2024-09-21T04:45:08.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd16ca47640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810db50 con 0x7fd168100800 2024-09-21T04:45:08.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd165d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 0x7fd168198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd16659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168100800 0x7fd1681987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd165d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 0x7fd168198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47132/0 (socket says 172.21.15.7:47132) 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd165d9b640 1 -- 172.21.15.7:0/2133219258 learned_addr learned my addr 172.21.15.7:0/2133219258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd165d9b640 1 -- 172.21.15.7:0/2133219258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168100800 msgr2=0x7fd1681987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd165d9b640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168100800 0x7fd1681987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd165d9b640 1 -- 172.21.15.7:0/2133219258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd168199320 con 0x7fd168106800 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.253+0000 7fd165d9b640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 0x7fd168198d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00d6f0 tx=0x7fd15c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.254+0000 7fd1577fe640 1 -- 172.21.15.7:0/2133219258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd15c017070 con 0x7fd168106800 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.254+0000 7fd1577fe640 1 -- 172.21.15.7:0/2133219258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd15c00de00 con 0x7fd168106800 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.254+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd168199610 con 0x7fd168106800 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.254+0000 7fd1577fe640 1 -- 172.21.15.7:0/2133219258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd15c004bb0 con 0x7fd168106800 2024-09-21T04:45:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.254+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a1c10 con 0x7fd168106800 2024-09-21T04:45:08.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.255+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd168101f40 con 0x7fd168106800 2024-09-21T04:45:08.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.256+0000 7fd1577fe640 1 -- 172.21.15.7:0/2133219258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd15c009a30 con 0x7fd168106800 2024-09-21T04:45:08.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.256+0000 7fd1577fe640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd15007af90 0x7fd15007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.257+0000 7fd16659c640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd15007af90 0x7fd15007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.257+0000 7fd1577fe640 1 -- 172.21.15.7:0/2133219258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd15c09db20 con 0x7fd168106800 2024-09-21T04:45:08.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.257+0000 7fd16659c640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd15007af90 0x7fd15007d450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1580096c0 tx=0x7fd158031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:08.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.261+0000 7fd1577fe640 1 -- 172.21.15.7:0/2133219258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd168199610 con 0x7fd168106800 2024-09-21T04:45:08.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.472+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd168062630 con 0x7fd15007af90 2024-09-21T04:45:08.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.473+0000 7fd1577fe640 1 -- 172.21.15.7:0/2133219258 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd168062630 con 0x7fd15007af90 2024-09-21T04:45:08.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.475+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd15007af90 msgr2=0x7fd15007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.475+0000 7fd16ca47640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd15007af90 0x7fd15007d450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd1580096c0 tx=0x7fd158031400 comp rx=0 tx=0).stop 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.475+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 msgr2=0x7fd168198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.475+0000 7fd16ca47640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 0x7fd168198d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00d6f0 tx=0x7fd15c00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.476+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 shutdown_connections 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.476+0000 7fd16ca47640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd15007af90 0x7fd15007d450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.476+0000 7fd16ca47640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd168106800 0x7fd168198d20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.476+0000 7fd16ca47640 1 --2- 172.21.15.7:0/2133219258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168100800 0x7fd1681987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.476+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 >> 172.21.15.7:0/2133219258 conn(0x7fd1680fc520 msgr2=0x7fd1680fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.476+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 shutdown_connections 2024-09-21T04:45:08.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.476+0000 7fd16ca47640 1 -- 172.21.15.7:0/2133219258 wait complete. 2024-09-21T04:45:08.487 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:45:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.561+0000 7f71a274b640 1 -- 172.21.15.7:0/3853663586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c107900 msgr2=0x7f719c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.561+0000 7f71a274b640 1 --2- 172.21.15.7:0/3853663586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c107900 0x7f719c107ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f718c009880 tx=0x7f718c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.561+0000 7f71a274b640 1 -- 172.21.15.7:0/3853663586 shutdown_connections 2024-09-21T04:45:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.561+0000 7f71a274b640 1 --2- 172.21.15.7:0/3853663586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c101900 0x7f719c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.561+0000 7f71a274b640 1 --2- 172.21.15.7:0/3853663586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c107900 0x7f719c107ce0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.561+0000 7f71a274b640 1 -- 172.21.15.7:0/3853663586 >> 172.21.15.7:0/3853663586 conn(0x7f719c0fd600 msgr2=0x7f719c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.561+0000 7f71a274b640 1 -- 172.21.15.7:0/3853663586 shutdown_connections 2024-09-21T04:45:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.561+0000 7f71a274b640 1 -- 172.21.15.7:0/3853663586 wait complete. 2024-09-21T04:45:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.562+0000 7f71a274b640 1 Processor -- start 2024-09-21T04:45:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.562+0000 7f71a274b640 1 -- start start 2024-09-21T04:45:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f71a274b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c101900 0x7f719c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f71a274b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c107900 0x7f719c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f71a274b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10edb0 con 0x7f719c101900 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f71a274b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10ec30 con 0x7f719c107900 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f719bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c101900 0x7f719c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f719bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c101900 0x7f719c19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47148/0 (socket says 172.21.15.7:47148) 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f719bfff640 1 -- 172.21.15.7:0/566895249 learned_addr learned my addr 172.21.15.7:0/566895249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f719bfff640 1 -- 172.21.15.7:0/566895249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c107900 msgr2=0x7f719c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f719b7fe640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c107900 0x7f719c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f719bfff640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c107900 0x7f719c19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f719bfff640 1 -- 172.21.15.7:0/566895249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c1977e0 con 0x7f719c101900 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f719bfff640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c101900 0x7f719c19d660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f718c031460 tx=0x7f718c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.563+0000 7f71997fa640 1 -- 172.21.15.7:0/566895249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f718c03f070 con 0x7f719c101900 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.564+0000 7f71997fa640 1 -- 172.21.15.7:0/566895249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f718c004cf0 con 0x7f719c101900 2024-09-21T04:45:08.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.564+0000 7f71997fa640 1 -- 172.21.15.7:0/566895249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f718c035030 con 0x7f719c101900 2024-09-21T04:45:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.564+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c197a70 con 0x7f719c101900 2024-09-21T04:45:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.564+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c197fa0 con 0x7f719c101900 2024-09-21T04:45:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.565+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c103040 con 0x7f719c101900 2024-09-21T04:45:08.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.565+0000 7f71997fa640 1 -- 172.21.15.7:0/566895249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f718c042070 con 0x7f719c101900 2024-09-21T04:45:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.566+0000 7f71997fa640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f718807af30 0x7f718807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.566+0000 7f719b7fe640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f718807af30 0x7f718807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.566+0000 7f71997fa640 1 -- 172.21.15.7:0/566895249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f718c0beb20 con 0x7f719c101900 2024-09-21T04:45:08.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.567+0000 7f719b7fe640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f718807af30 0x7f718807d3f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f719c100d10 tx=0x7f7184005cc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:08.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.570+0000 7f71997fa640 1 -- 172.21.15.7:0/566895249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f718c08a0c0 con 0x7f719c101900 2024-09-21T04:45:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.776+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f719c198d10 con 0x7f718807af30 2024-09-21T04:45:08.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.777+0000 7f71997fa640 1 -- 172.21.15.7:0/566895249 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f719c198d10 con 0x7f718807af30 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.779+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f718807af30 msgr2=0x7f718807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f718807af30 0x7f718807d3f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f719c100d10 tx=0x7f7184005cc0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c101900 msgr2=0x7f719c19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c101900 0x7f719c19d660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f718c031460 tx=0x7f718c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 shutdown_connections 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f718807af30 0x7f718807d3f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c107900 0x7f719c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 --2- 172.21.15.7:0/566895249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f719c101900 0x7f719c19d660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 >> 172.21.15.7:0/566895249 conn(0x7f719c0fd600 msgr2=0x7f719c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 shutdown_connections 2024-09-21T04:45:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.780+0000 7f71a274b640 1 -- 172.21.15.7:0/566895249 wait complete. 2024-09-21T04:45:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.864+0000 7f0d4ee69640 1 -- 172.21.15.7:0/3265742992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48102b50 msgr2=0x7f0d48102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.864+0000 7f0d4ee69640 1 --2- 172.21.15.7:0/3265742992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48102b50 0x7f0d48102f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009880 tx=0x7f0d3802a530 comp rx=0 tx=0).stop 2024-09-21T04:45:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.865+0000 7f0d4ee69640 1 -- 172.21.15.7:0/3265742992 shutdown_connections 2024-09-21T04:45:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.865+0000 7f0d4ee69640 1 --2- 172.21.15.7:0/3265742992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d48103470 0x7f0d4810b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.865+0000 7f0d4ee69640 1 --2- 172.21.15.7:0/3265742992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48102b50 0x7f0d48102f30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.865+0000 7f0d4ee69640 1 -- 172.21.15.7:0/3265742992 >> 172.21.15.7:0/3265742992 conn(0x7f0d480fc550 msgr2=0x7f0d480fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.865+0000 7f0d4ee69640 1 -- 172.21.15.7:0/3265742992 shutdown_connections 2024-09-21T04:45:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.865+0000 7f0d4ee69640 1 -- 172.21.15.7:0/3265742992 wait complete. 2024-09-21T04:45:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.865+0000 7f0d4ee69640 1 Processor -- start 2024-09-21T04:45:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d4ee69640 1 -- start start 2024-09-21T04:45:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d4ee69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d48102b50 0x7f0d481007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d4ee69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48103470 0x7f0d48100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d4ee69640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4810edc0 con 0x7f0d48103470 2024-09-21T04:45:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d4ee69640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4810ec40 con 0x7f0d48102b50 2024-09-21T04:45:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48103470 0x7f0d48100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48103470 0x7f0d48100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47156/0 (socket says 172.21.15.7:47156) 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d47fff640 1 -- 172.21.15.7:0/2357630618 learned_addr learned my addr 172.21.15.7:0/2357630618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d4cc05640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d48102b50 0x7f0d481007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d47fff640 1 -- 172.21.15.7:0/2357630618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d48102b50 msgr2=0x7f0d481007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d47fff640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d48102b50 0x7f0d481007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.866+0000 7f0d47fff640 1 -- 172.21.15.7:0/2357630618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d48101230 con 0x7f0d48103470 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.867+0000 7f0d47fff640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48103470 0x7f0d48100cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d8d0 tx=0x7f0d3c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.867+0000 7f0d45ffb640 1 -- 172.21.15.7:0/2357630618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d3c017070 con 0x7f0d48103470 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.867+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d48101520 con 0x7f0d48103470 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.867+0000 7f0d45ffb640 1 -- 172.21.15.7:0/2357630618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d3c010460 con 0x7f0d48103470 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.867+0000 7f0d45ffb640 1 -- 172.21.15.7:0/2357630618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d3c00f5f0 con 0x7f0d48103470 2024-09-21T04:45:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.867+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481a3e30 con 0x7f0d48103470 2024-09-21T04:45:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.868+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4806a4e0 con 0x7f0d48103470 2024-09-21T04:45:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.869+0000 7f0d45ffb640 1 -- 172.21.15.7:0/2357630618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0d3c009a30 con 0x7f0d48103470 2024-09-21T04:45:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.869+0000 7f0d45ffb640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d3007af90 0x7f0d3007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.870+0000 7f0d4cc05640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d3007af90 0x7f0d3007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.870+0000 7f0d45ffb640 1 -- 172.21.15.7:0/2357630618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0d3c09ddb0 con 0x7f0d48103470 2024-09-21T04:45:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.870+0000 7f0d4cc05640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d3007af90 0x7f0d3007d450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d380096c0 tx=0x7f0d38004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:08.874+0000 7f0d45ffb640 1 -- 172.21.15.7:0/2357630618 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0d3c0693d0 con 0x7f0d48103470 2024-09-21T04:45:09.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.070+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0d48062630 con 0x7f0d3007af90 2024-09-21T04:45:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.079+0000 7f0d45ffb640 1 -- 172.21.15.7:0/2357630618 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f0d48062630 con 0x7f0d3007af90 2024-09-21T04:45:09.082 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (7m) 72s ago 18m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (10m) 72s ago 18m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (9m) 58s ago 17m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (5m) 72s ago 18m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (5m) 58s ago 17m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (6m) 72s ago 17m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (78s) 72s ago 12m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (94s) 72s ago 12m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (12m) 58s ago 12m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (62s) 58s ago 12m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (9m) 72s ago 19m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (8m) 58s ago 17m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:45:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (5m) 72s ago 19m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (5m) 58s ago 17m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (8m) 72s ago 18m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (7m) 58s ago 17m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (4m) 72s ago 16m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (4m) 72s ago 15m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (3m) 72s ago 15m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (3m) 58s ago 14m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (2m) 58s ago 14m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (111s) 58s ago 13m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:45:09.084 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (7m) 72s ago 17m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:45:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d3007af90 msgr2=0x7f0d3007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d3007af90 0x7f0d3007d450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d380096c0 tx=0x7f0d38004f30 comp rx=0 tx=0).stop 2024-09-21T04:45:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48103470 msgr2=0x7f0d48100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48103470 0x7f0d48100cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d8d0 tx=0x7f0d3c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:45:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 shutdown_connections 2024-09-21T04:45:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d3007af90 0x7f0d3007d450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d48103470 0x7f0d48100cf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 --2- 172.21.15.7:0/2357630618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d48102b50 0x7f0d481007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 >> 172.21.15.7:0/2357630618 conn(0x7f0d480fc550 msgr2=0x7f0d48108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 shutdown_connections 2024-09-21T04:45:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.082+0000 7f0d4ee69640 1 -- 172.21.15.7:0/2357630618 wait complete. 2024-09-21T04:45:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.165+0000 7f138d4de640 1 -- 172.21.15.7:0/4031596245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 msgr2=0x7f1388106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.165+0000 7f138d4de640 1 --2- 172.21.15.7:0/4031596245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 0x7f1388106be0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1378009880 tx=0x7f137802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.165+0000 7f138d4de640 1 -- 172.21.15.7:0/4031596245 shutdown_connections 2024-09-21T04:45:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.165+0000 7f138d4de640 1 --2- 172.21.15.7:0/4031596245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1388100800 0x7f1388100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.165+0000 7f138d4de640 1 --2- 172.21.15.7:0/4031596245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 0x7f1388106be0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.165+0000 7f138d4de640 1 -- 172.21.15.7:0/4031596245 >> 172.21.15.7:0/4031596245 conn(0x7f13880fc520 msgr2=0x7f13880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.166+0000 7f138d4de640 1 -- 172.21.15.7:0/4031596245 shutdown_connections 2024-09-21T04:45:09.169 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:08 smithi007 ceph-mon[108238]: pgmap v311: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.166+0000 7f138d4de640 1 -- 172.21.15.7:0/4031596245 wait complete. 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.166+0000 7f138d4de640 1 Processor -- start 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f138d4de640 1 -- start start 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f138d4de640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1388100800 0x7f138819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f138d4de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 0x7f138819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f138d4de640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f138810dcb0 con 0x7f1388106800 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f138d4de640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f138810db30 con 0x7f1388100800 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f13867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 0x7f138819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f13867fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 0x7f138819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47160/0 (socket says 172.21.15.7:47160) 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f13867fc640 1 -- 172.21.15.7:0/327062786 learned_addr learned my addr 172.21.15.7:0/327062786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f1386ffd640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1388100800 0x7f138819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f13867fc640 1 -- 172.21.15.7:0/327062786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1388100800 msgr2=0x7f138819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f13867fc640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1388100800 0x7f138819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.167+0000 7f13867fc640 1 -- 172.21.15.7:0/327062786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1388197840 con 0x7f1388106800 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.168+0000 7f13867fc640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 0x7f138819dc00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f137c00d8d0 tx=0x7f137c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.168+0000 7f136ffff640 1 -- 172.21.15.7:0/327062786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f137c017070 con 0x7f1388106800 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.168+0000 7f136ffff640 1 -- 172.21.15.7:0/327062786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f137c010460 con 0x7f1388106800 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.168+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1388197b30 con 0x7f1388106800 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.168+0000 7f136ffff640 1 -- 172.21.15.7:0/327062786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f137c00f5f0 con 0x7f1388106800 2024-09-21T04:45:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.168+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13881980c0 con 0x7f1388106800 2024-09-21T04:45:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.169+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1388101f40 con 0x7f1388106800 2024-09-21T04:45:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.170+0000 7f136ffff640 1 -- 172.21.15.7:0/327062786 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f137c009a30 con 0x7f1388106800 2024-09-21T04:45:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.170+0000 7f136ffff640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137007af40 0x7f137007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.170+0000 7f136ffff640 1 -- 172.21.15.7:0/327062786 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f137c09de90 con 0x7f1388106800 2024-09-21T04:45:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.170+0000 7f1386ffd640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137007af40 0x7f137007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.171+0000 7f1386ffd640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137007af40 0x7f137007d400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13780096c0 tx=0x7f1378031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.176+0000 7f136ffff640 1 -- 172.21.15.7:0/327062786 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f137c069430 con 0x7f1388106800 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.439+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f1388198da0 con 0x7f1388106800 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.440+0000 7f136ffff640 1 -- 172.21.15.7:0/327062786 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f1388198da0 con 0x7f1388106800 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:45:09.443 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:45:09.444 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:45:09.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:08 smithi053 ceph-mon[80176]: pgmap v311: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137007af40 msgr2=0x7f137007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137007af40 0x7f137007d400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13780096c0 tx=0x7f1378031400 comp rx=0 tx=0).stop 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 msgr2=0x7f138819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 0x7f138819dc00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f137c00d8d0 tx=0x7f137c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 shutdown_connections 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137007af40 0x7f137007d400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388106800 0x7f138819dc00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 --2- 172.21.15.7:0/327062786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1388100800 0x7f138819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 >> 172.21.15.7:0/327062786 conn(0x7f13880fc520 msgr2=0x7f138806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 shutdown_connections 2024-09-21T04:45:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.442+0000 7f138d4de640 1 -- 172.21.15.7:0/327062786 wait complete. 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.526+0000 7f2f7dd62640 1 -- 172.21.15.7:0/3111544336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78101900 msgr2=0x7f2f78101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.526+0000 7f2f7dd62640 1 --2- 172.21.15.7:0/3111544336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78101900 0x7f2f78101d60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009880 tx=0x7f2f6c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.526+0000 7f2f7dd62640 1 -- 172.21.15.7:0/3111544336 shutdown_connections 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.526+0000 7f2f7dd62640 1 --2- 172.21.15.7:0/3111544336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78101900 0x7f2f78101d60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.526+0000 7f2f7dd62640 1 --2- 172.21.15.7:0/3111544336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f78107900 0x7f2f78107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.526+0000 7f2f7dd62640 1 -- 172.21.15.7:0/3111544336 >> 172.21.15.7:0/3111544336 conn(0x7f2f780fd600 msgr2=0x7f2f780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.526+0000 7f2f7dd62640 1 -- 172.21.15.7:0/3111544336 shutdown_connections 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.527+0000 7f2f7dd62640 1 -- 172.21.15.7:0/3111544336 wait complete. 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.527+0000 7f2f7dd62640 1 Processor -- start 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.527+0000 7f2f7dd62640 1 -- start start 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.527+0000 7f2f7dd62640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f78101900 0x7f2f7819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.527+0000 7f2f7dd62640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78107900 0x7f2f7819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.527+0000 7f2f7dd62640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7810edb0 con 0x7f2f78107900 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f7dd62640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7810ec30 con 0x7f2f78101900 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f76ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78107900 0x7f2f7819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f76ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78107900 0x7f2f7819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47180/0 (socket says 172.21.15.7:47180) 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f76ffd640 1 -- 172.21.15.7:0/2182273063 learned_addr learned my addr 172.21.15.7:0/2182273063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f777fe640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f78101900 0x7f2f7819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f76ffd640 1 -- 172.21.15.7:0/2182273063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f78101900 msgr2=0x7f2f7819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f76ffd640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f78101900 0x7f2f7819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f76ffd640 1 -- 172.21.15.7:0/2182273063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f78197800 con 0x7f2f78107900 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f777fe640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f78101900 0x7f2f7819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:45:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f76ffd640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78107900 0x7f2f7819dbc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c031460 tx=0x7f2f6c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f74ff9640 1 -- 172.21.15.7:0/2182273063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c03f070 con 0x7f2f78107900 2024-09-21T04:45:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f74ff9640 1 -- 172.21.15.7:0/2182273063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2f6c0035f0 con 0x7f2f78107900 2024-09-21T04:45:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f78197a90 con 0x7f2f78107900 2024-09-21T04:45:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f74ff9640 1 -- 172.21.15.7:0/2182273063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c035030 con 0x7f2f78107900 2024-09-21T04:45:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.528+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f78197fc0 con 0x7f2f78107900 2024-09-21T04:45:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.529+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f78103040 con 0x7f2f78107900 2024-09-21T04:45:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.530+0000 7f2f74ff9640 1 -- 172.21.15.7:0/2182273063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2f6c042070 con 0x7f2f78107900 2024-09-21T04:45:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.531+0000 7f2f74ff9640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f6007af30 0x7f2f6007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.531+0000 7f2f74ff9640 1 -- 172.21.15.7:0/2182273063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2f6c0bea80 con 0x7f2f78107900 2024-09-21T04:45:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.531+0000 7f2f777fe640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f6007af30 0x7f2f6007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.531+0000 7f2f777fe640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f6007af30 0x7f2f6007d3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f68005d00 tx=0x7f2f68009550 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.535+0000 7f2f74ff9640 1 -- 172.21.15.7:0/2182273063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f6c08a020 con 0x7f2f78107900 2024-09-21T04:45:09.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.763+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2f78109e90 con 0x7f2f78107900 2024-09-21T04:45:09.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.764+0000 7f2f74ff9640 1 -- 172.21.15.7:0/2182273063 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f2f6c0be7e0 con 0x7f2f78107900 2024-09-21T04:45:09.767 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:45:09.767 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:45:09.767 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:45:09.767 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:45:09.768 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:45:09.769 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:45:09.770 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:45:09.771 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:45:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:45:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f6007af30 msgr2=0x7f2f6007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f6007af30 0x7f2f6007d3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f68005d00 tx=0x7f2f68009550 comp rx=0 tx=0).stop 2024-09-21T04:45:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78107900 msgr2=0x7f2f7819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78107900 0x7f2f7819dbc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c031460 tx=0x7f2f6c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 shutdown_connections 2024-09-21T04:45:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2f6007af30 0x7f2f6007d3f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f78107900 0x7f2f7819dbc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 --2- 172.21.15.7:0/2182273063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f78101900 0x7f2f7819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.766+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 >> 172.21.15.7:0/2182273063 conn(0x7f2f780fd600 msgr2=0x7f2f780fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.767+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 shutdown_connections 2024-09-21T04:45:09.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.767+0000 7f2f7dd62640 1 -- 172.21.15.7:0/2182273063 wait complete. 2024-09-21T04:45:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 -- 172.21.15.7:0/3572992331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1067f0 msgr2=0x7f6fec106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 --2- 172.21.15.7:0/3572992331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1067f0 0x7f6fec106bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009880 tx=0x7f6fe802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 -- 172.21.15.7:0/3572992331 shutdown_connections 2024-09-21T04:45:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 --2- 172.21.15.7:0/3572992331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fec1007f0 0x7f6fec100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 --2- 172.21.15.7:0/3572992331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1067f0 0x7f6fec106bd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 -- 172.21.15.7:0/3572992331 >> 172.21.15.7:0/3572992331 conn(0x7f6fec0fc550 msgr2=0x7f6fec0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 -- 172.21.15.7:0/3572992331 shutdown_connections 2024-09-21T04:45:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 -- 172.21.15.7:0/3572992331 wait complete. 2024-09-21T04:45:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 Processor -- start 2024-09-21T04:45:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.852+0000 7f6ff3d22640 1 -- start start 2024-09-21T04:45:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff3d22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1007f0 0x7f6fec107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff3d22640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fec1067f0 0x7f6fec107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff3d22640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec10dca0 con 0x7f6fec1007f0 2024-09-21T04:45:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff3d22640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec10db20 con 0x7f6fec1067f0 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff1abe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1007f0 0x7f6fec107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff1abe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1007f0 0x7f6fec107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47196/0 (socket says 172.21.15.7:47196) 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff1abe640 1 -- 172.21.15.7:0/282518600 learned_addr learned my addr 172.21.15.7:0/282518600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff12bd640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fec1067f0 0x7f6fec107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff1abe640 1 -- 172.21.15.7:0/282518600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fec1067f0 msgr2=0x7f6fec107c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff1abe640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fec1067f0 0x7f6fec107c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff1abe640 1 -- 172.21.15.7:0/282518600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec1081a0 con 0x7f6fec1007f0 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff12bd640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fec1067f0 0x7f6fec107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.853+0000 7f6ff1abe640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1007f0 0x7f6fec107720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8031460 tx=0x7f6fe80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.854+0000 7f6fe2ffd640 1 -- 172.21.15.7:0/282518600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6fe803f070 con 0x7f6fec1007f0 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.854+0000 7f6fe2ffd640 1 -- 172.21.15.7:0/282518600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fe80044d0 con 0x7f6fec1007f0 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.854+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec108430 con 0x7f6fec1007f0 2024-09-21T04:45:09.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.854+0000 7f6fe2ffd640 1 -- 172.21.15.7:0/282518600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6fe8035030 con 0x7f6fec1007f0 2024-09-21T04:45:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.854+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec1a3cc0 con 0x7f6fec1007f0 2024-09-21T04:45:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.855+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec101f30 con 0x7f6fec1007f0 2024-09-21T04:45:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.856+0000 7f6fe2ffd640 1 -- 172.21.15.7:0/282518600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6fe8042070 con 0x7f6fec1007f0 2024-09-21T04:45:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.856+0000 7f6fe2ffd640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fd807af90 0x7f6fd807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.856+0000 7f6fe2ffd640 1 -- 172.21.15.7:0/282518600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6fe80bea90 con 0x7f6fec1007f0 2024-09-21T04:45:09.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.856+0000 7f6ff12bd640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fd807af90 0x7f6fd807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:09.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.857+0000 7f6ff12bd640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fd807af90 0x7f6fd807d450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6fec1091b0 tx=0x7f6fdc00a5c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:09.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:09.861+0000 7f6fe2ffd640 1 -- 172.21.15.7:0/282518600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6fe8004cf0 con 0x7f6fec1007f0 2024-09-21T04:45:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.064+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6fec06a520 con 0x7f6fd807af90 2024-09-21T04:45:10.067 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:09 smithi007 ceph-mon[108238]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:10.067 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:09 smithi007 ceph-mon[108238]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:10.068 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:09 smithi007 ceph-mon[108238]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:10.068 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:09 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/327062786' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:45:10.068 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:09 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2182273063' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:45:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.065+0000 7f6fe2ffd640 1 -- 172.21.15.7:0/282518600 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f6fec06a520 con 0x7f6fd807af90 2024-09-21T04:45:10.068 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:45:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:45:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:45:10.069 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:45:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fd807af90 msgr2=0x7f6fd807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fd807af90 0x7f6fd807d450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6fec1091b0 tx=0x7f6fdc00a5c0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1007f0 msgr2=0x7f6fec107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1007f0 0x7f6fec107720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8031460 tx=0x7f6fe80048f0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 shutdown_connections 2024-09-21T04:45:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fd807af90 0x7f6fd807d450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6fec1067f0 0x7f6fec107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 --2- 172.21.15.7:0/282518600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fec1007f0 0x7f6fec107720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 >> 172.21.15.7:0/282518600 conn(0x7f6fec0fc550 msgr2=0x7f6fec069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 shutdown_connections 2024-09-21T04:45:10.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.068+0000 7f6ff3d22640 1 -- 172.21.15.7:0/282518600 wait complete. 2024-09-21T04:45:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.154+0000 7f8eac980640 1 -- 172.21.15.7:0/2580907963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8102b30 msgr2=0x7f8ea8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.154+0000 7f8eac980640 1 --2- 172.21.15.7:0/2580907963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8102b30 0x7f8ea8102f10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009c20 tx=0x7f8e9802aa00 comp rx=0 tx=0).stop 2024-09-21T04:45:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.154+0000 7f8eac980640 1 -- 172.21.15.7:0/2580907963 shutdown_connections 2024-09-21T04:45:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.154+0000 7f8eac980640 1 --2- 172.21.15.7:0/2580907963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ea8103450 0x7f8ea810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.154+0000 7f8eac980640 1 --2- 172.21.15.7:0/2580907963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8102b30 0x7f8ea8102f10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.154+0000 7f8eac980640 1 -- 172.21.15.7:0/2580907963 >> 172.21.15.7:0/2580907963 conn(0x7f8ea80fc550 msgr2=0x7f8ea80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.154+0000 7f8eac980640 1 -- 172.21.15.7:0/2580907963 shutdown_connections 2024-09-21T04:45:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.154+0000 7f8eac980640 1 -- 172.21.15.7:0/2580907963 wait complete. 2024-09-21T04:45:10.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8eac980640 1 Processor -- start 2024-09-21T04:45:10.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8eac980640 1 -- start start 2024-09-21T04:45:10.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8eac980640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ea8102b30 0x7f8ea8100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:10.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8eac980640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8103450 0x7f8ea8100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:10.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8eac980640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea810ede0 con 0x7f8ea8103450 2024-09-21T04:45:10.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8eac980640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea810ec60 con 0x7f8ea8102b30 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8ea5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8103450 0x7f8ea8100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8ea5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8103450 0x7f8ea8100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47214/0 (socket says 172.21.15.7:47214) 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8ea5d9b640 1 -- 172.21.15.7:0/1918775184 learned_addr learned my addr 172.21.15.7:0/1918775184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.155+0000 7f8ea659c640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ea8102b30 0x7f8ea8100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8ea5d9b640 1 -- 172.21.15.7:0/1918775184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ea8102b30 msgr2=0x7f8ea8100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8ea5d9b640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ea8102b30 0x7f8ea8100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8ea5d9b640 1 -- 172.21.15.7:0/1918775184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea8101210 con 0x7f8ea8103450 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8ea5d9b640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8103450 0x7f8ea8100cd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00d6f0 tx=0x7f8e9c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8e977fe640 1 -- 172.21.15.7:0/1918775184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8e9c017070 con 0x7f8ea8103450 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea8101500 con 0x7f8ea8103450 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8e977fe640 1 -- 172.21.15.7:0/1918775184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e9c00de00 con 0x7f8ea8103450 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8e977fe640 1 -- 172.21.15.7:0/1918775184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8e9c004bb0 con 0x7f8ea8103450 2024-09-21T04:45:10.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.156+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea81a3e50 con 0x7f8ea8103450 2024-09-21T04:45:10.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.157+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea806a500 con 0x7f8ea8103450 2024-09-21T04:45:10.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.158+0000 7f8e977fe640 1 -- 172.21.15.7:0/1918775184 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8e9c004e60 con 0x7f8ea8103450 2024-09-21T04:45:10.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.159+0000 7f8e977fe640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8e9007af40 0x7f8e9007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:10.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.159+0000 7f8e977fe640 1 -- 172.21.15.7:0/1918775184 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8e9c09dca0 con 0x7f8ea8103450 2024-09-21T04:45:10.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.159+0000 7f8ea659c640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8e9007af40 0x7f8e9007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:10.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.159+0000 7f8ea659c640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8e9007af40 0x7f8e9007d400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009430 tx=0x7f8e9803a040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:10.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.166+0000 7f8e977fe640 1 -- 172.21.15.7:0/1918775184 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e9c0691a0 con 0x7f8ea8103450 2024-09-21T04:45:10.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.431+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f8ea8101cc0 con 0x7f8ea8103450 2024-09-21T04:45:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.432+0000 7f8e977fe640 1 -- 172.21.15.7:0/1918775184 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f8e9c06e050 con 0x7f8ea8103450 2024-09-21T04:45:10.435 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:45:10.435 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:45:10.435 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:45:10.435 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:45:10.435 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:45:10.435 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:45:10.435 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8e9007af40 msgr2=0x7f8e9007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8e9007af40 0x7f8e9007d400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009430 tx=0x7f8e9803a040 comp rx=0 tx=0).stop 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8103450 msgr2=0x7f8ea8100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8103450 0x7f8ea8100cd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00d6f0 tx=0x7f8e9c00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 shutdown_connections 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8e9007af40 0x7f8e9007d400 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ea8103450 0x7f8ea8100cd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 --2- 172.21.15.7:0/1918775184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ea8102b30 0x7f8ea8100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 >> 172.21.15.7:0/1918775184 conn(0x7f8ea80fc550 msgr2=0x7f8ea8108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 shutdown_connections 2024-09-21T04:45:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:10.434+0000 7f8eac980640 1 -- 172.21.15.7:0/1918775184 wait complete. 2024-09-21T04:45:10.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:09 smithi053 ceph-mon[80176]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:09 smithi053 ceph-mon[80176]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:09 smithi053 ceph-mon[80176]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:09 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/327062786' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:45:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:09 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2182273063' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:45:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:11 smithi007 ceph-mon[108238]: pgmap v312: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:11 smithi007 ceph-mon[108238]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:11 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1918775184' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:45:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:11 smithi053 ceph-mon[80176]: pgmap v312: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:11 smithi053 ceph-mon[80176]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:11 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1918775184' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:45:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:13 smithi053 ceph-mon[80176]: pgmap v313: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:13 smithi007 ceph-mon[108238]: pgmap v313: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:45:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:45:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:15 smithi053 ceph-mon[80176]: pgmap v314: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:15 smithi007 ceph-mon[108238]: pgmap v314: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:17.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:17 smithi053 ceph-mon[80176]: pgmap v315: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:17 smithi007 ceph-mon[108238]: pgmap v315: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:19 smithi053 ceph-mon[80176]: pgmap v316: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:19 smithi007 ceph-mon[108238]: pgmap v316: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:21.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:21 smithi053 ceph-mon[80176]: pgmap v317: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:21 smithi007 ceph-mon[108238]: pgmap v317: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:23 smithi007 ceph-mon[108238]: pgmap v318: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:23.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:23 smithi053 ceph-mon[80176]: pgmap v318: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:25.377 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:25 smithi007 ceph-mon[108238]: pgmap v319: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:25 smithi053 ceph-mon[80176]: pgmap v319: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:27.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:27 smithi053 ceph-mon[80176]: pgmap v320: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:27 smithi007 ceph-mon[108238]: pgmap v320: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:29 smithi007 ceph-mon[108238]: pgmap v321: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:45:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:29 smithi053 ceph-mon[80176]: pgmap v321: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:45:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:31 smithi053 ceph-mon[80176]: pgmap v322: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:31 smithi007 ceph-mon[108238]: pgmap v322: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:33 smithi053 ceph-mon[80176]: pgmap v323: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:33 smithi007 ceph-mon[108238]: pgmap v323: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:35.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:34 smithi053 ceph-mon[80176]: pgmap v324: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:34 smithi007 ceph-mon[108238]: pgmap v324: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:37.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:36 smithi053 ceph-mon[80176]: pgmap v325: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:36 smithi007 ceph-mon[108238]: pgmap v325: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:39.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:38 smithi053 ceph-mon[80176]: pgmap v326: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:39.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:38 smithi007 ceph-mon[108238]: pgmap v326: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 -- 172.21.15.7:0/880109894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410103440 msgr2=0x7fd41010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 --2- 172.21.15.7:0/880109894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410103440 0x7fd41010b930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80098b0 tx=0x7fd3f802a560 comp rx=0 tx=0).stop 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 -- 172.21.15.7:0/880109894 shutdown_connections 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 --2- 172.21.15.7:0/880109894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410103440 0x7fd41010b930 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 --2- 172.21.15.7:0/880109894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd410102b20 0x7fd410102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 -- 172.21.15.7:0/880109894 >> 172.21.15.7:0/880109894 conn(0x7fd4100fc520 msgr2=0x7fd4100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 -- 172.21.15.7:0/880109894 shutdown_connections 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 -- 172.21.15.7:0/880109894 wait complete. 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 Processor -- start 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.533+0000 7fd41686a640 1 -- start start 2024-09-21T04:45:40.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd41686a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410102b20 0x7fd410100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd41686a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd410103440 0x7fd410100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd41686a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd41010edd0 con 0x7fd410102b20 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd41686a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd41010ec50 con 0x7fd410103440 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd40ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410102b20 0x7fd410100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd40ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410102b20 0x7fd410100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49128/0 (socket says 172.21.15.7:49128) 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd40ffff640 1 -- 172.21.15.7:0/1623692151 learned_addr learned my addr 172.21.15.7:0/1623692151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd407fff640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd410103440 0x7fd410100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd40ffff640 1 -- 172.21.15.7:0/1623692151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd410103440 msgr2=0x7fd410100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd40ffff640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd410103440 0x7fd410100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd40ffff640 1 -- 172.21.15.7:0/1623692151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd410101200 con 0x7fd410102b20 2024-09-21T04:45:40.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.534+0000 7fd40ffff640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410102b20 0x7fd410100780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d8d0 tx=0x7fd40000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:40.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.535+0000 7fd40dffb640 1 -- 172.21.15.7:0/1623692151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd400017070 con 0x7fd410102b20 2024-09-21T04:45:40.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.535+0000 7fd40dffb640 1 -- 172.21.15.7:0/1623692151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd400010460 con 0x7fd410102b20 2024-09-21T04:45:40.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.535+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4101014f0 con 0x7fd410102b20 2024-09-21T04:45:40.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.535+0000 7fd40dffb640 1 -- 172.21.15.7:0/1623692151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd40000f5f0 con 0x7fd410102b20 2024-09-21T04:45:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.535+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4101a3ea0 con 0x7fd410102b20 2024-09-21T04:45:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.536+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41006aa30 con 0x7fd410102b20 2024-09-21T04:45:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.536+0000 7fd40dffb640 1 -- 172.21.15.7:0/1623692151 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd400009a30 con 0x7fd410102b20 2024-09-21T04:45:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.537+0000 7fd40dffb640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd3fc07af80 0x7fd3fc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.537+0000 7fd407fff640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd3fc07af80 0x7fd3fc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:40.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.537+0000 7fd40dffb640 1 -- 172.21.15.7:0/1623692151 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd40009de70 con 0x7fd410102b20 2024-09-21T04:45:40.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.538+0000 7fd407fff640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd3fc07af80 0x7fd3fc07d440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8000c00 tx=0x7fd3f8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:40.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.542+0000 7fd40dffb640 1 -- 172.21.15.7:0/1623692151 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd400069490 con 0x7fd410102b20 2024-09-21T04:45:40.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.760+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd410062630 con 0x7fd3fc07af80 2024-09-21T04:45:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.761+0000 7fd40dffb640 1 -- 172.21.15.7:0/1623692151 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd410062630 con 0x7fd3fc07af80 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd3fc07af80 msgr2=0x7fd3fc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd3fc07af80 0x7fd3fc07d440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8000c00 tx=0x7fd3f8031400 comp rx=0 tx=0).stop 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410102b20 msgr2=0x7fd410100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410102b20 0x7fd410100780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d8d0 tx=0x7fd40000dd90 comp rx=0 tx=0).stop 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 shutdown_connections 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd3fc07af80 0x7fd3fc07d440 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd410103440 0x7fd410100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 --2- 172.21.15.7:0/1623692151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd410102b20 0x7fd410100780 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 >> 172.21.15.7:0/1623692151 conn(0x7fd4100fc520 msgr2=0x7fd410103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 shutdown_connections 2024-09-21T04:45:40.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.764+0000 7fd41686a640 1 -- 172.21.15.7:0/1623692151 wait complete. 2024-09-21T04:45:40.776 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:45:40.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.849+0000 7f66477f9640 1 -- 172.21.15.7:0/3716023473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 msgr2=0x7f6640107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:40.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.849+0000 7f66477f9640 1 --2- 172.21.15.7:0/3716023473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 0x7f6640107cf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f663c009880 tx=0x7f663c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.849+0000 7f66477f9640 1 -- 172.21.15.7:0/3716023473 shutdown_connections 2024-09-21T04:45:40.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.849+0000 7f66477f9640 1 --2- 172.21.15.7:0/3716023473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6640101910 0x7f6640101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.849+0000 7f66477f9640 1 --2- 172.21.15.7:0/3716023473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 0x7f6640107cf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.849+0000 7f66477f9640 1 -- 172.21.15.7:0/3716023473 >> 172.21.15.7:0/3716023473 conn(0x7f66400fd630 msgr2=0x7f66400ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:40.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.849+0000 7f66477f9640 1 -- 172.21.15.7:0/3716023473 shutdown_connections 2024-09-21T04:45:40.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.850+0000 7f66477f9640 1 -- 172.21.15.7:0/3716023473 wait complete. 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.850+0000 7f66477f9640 1 Processor -- start 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.850+0000 7f66477f9640 1 -- start start 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f66477f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6640101910 0x7f66400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f66477f9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 0x7f6640074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f66477f9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f664010ede0 con 0x7f6640107910 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f66477f9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f664010ec60 con 0x7f6640101910 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f6644d94640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 0x7f6640074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f6644d94640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 0x7f6640074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49160/0 (socket says 172.21.15.7:49160) 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f6644d94640 1 -- 172.21.15.7:0/2864003142 learned_addr learned my addr 172.21.15.7:0/2864003142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f6645595640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6640101910 0x7f66400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f6644d94640 1 -- 172.21.15.7:0/2864003142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6640101910 msgr2=0x7f66400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f6644d94640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6640101910 0x7f66400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f6644d94640 1 -- 172.21.15.7:0/2864003142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6640074f70 con 0x7f6640107910 2024-09-21T04:45:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f6644d94640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 0x7f6640074a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f663000c8f0 tx=0x7f663000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f66367fc640 1 -- 172.21.15.7:0/2864003142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66300079b0 con 0x7f6640107910 2024-09-21T04:45:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6640075260 con 0x7f6640107910 2024-09-21T04:45:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.851+0000 7f66367fc640 1 -- 172.21.15.7:0/2864003142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66300054e0 con 0x7f6640107910 2024-09-21T04:45:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.852+0000 7f66367fc640 1 -- 172.21.15.7:0/2864003142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6630004dd0 con 0x7f6640107910 2024-09-21T04:45:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.852+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66401a3e20 con 0x7f6640107910 2024-09-21T04:45:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.853+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6640103050 con 0x7f6640107910 2024-09-21T04:45:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.853+0000 7f66367fc640 1 -- 172.21.15.7:0/2864003142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6630005020 con 0x7f6640107910 2024-09-21T04:45:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.854+0000 7f66367fc640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f662c07aed0 0x7f662c07d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.854+0000 7f66367fc640 1 -- 172.21.15.7:0/2864003142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f663009dfb0 con 0x7f6640107910 2024-09-21T04:45:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.854+0000 7f6645595640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f662c07aed0 0x7f662c07d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.854+0000 7f6645595640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f662c07aed0 0x7f662c07d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f663c000c00 tx=0x7f663c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:40.858+0000 7f66367fc640 1 -- 172.21.15.7:0/2864003142 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6630014030 con 0x7f6640107910 2024-09-21T04:45:41.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.065+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6640062630 con 0x7f662c07aed0 2024-09-21T04:45:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.066+0000 7f66367fc640 1 -- 172.21.15.7:0/2864003142 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f6640062630 con 0x7f662c07aed0 2024-09-21T04:45:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f662c07aed0 msgr2=0x7f662c07d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f662c07aed0 0x7f662c07d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f663c000c00 tx=0x7f663c031400 comp rx=0 tx=0).stop 2024-09-21T04:45:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 msgr2=0x7f6640074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 0x7f6640074a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f663000c8f0 tx=0x7f663000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 shutdown_connections 2024-09-21T04:45:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f662c07aed0 0x7f662c07d390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6640107910 0x7f6640074a30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 --2- 172.21.15.7:0/2864003142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6640101910 0x7f66400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 >> 172.21.15.7:0/2864003142 conn(0x7f66400fd630 msgr2=0x7f66400fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 shutdown_connections 2024-09-21T04:45:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.069+0000 7f66477f9640 1 -- 172.21.15.7:0/2864003142 wait complete. 2024-09-21T04:45:41.157 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:40 smithi007 ceph-mon[108238]: pgmap v327: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 -- 172.21.15.7:0/909320856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 msgr2=0x7f9158102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 --2- 172.21.15.7:0/909320856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 0x7f9158102ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9148009880 tx=0x7f914802a530 comp rx=0 tx=0).stop 2024-09-21T04:45:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 -- 172.21.15.7:0/909320856 shutdown_connections 2024-09-21T04:45:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 --2- 172.21.15.7:0/909320856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91581033e0 0x7f915810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 --2- 172.21.15.7:0/909320856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 0x7f9158102ea0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 -- 172.21.15.7:0/909320856 >> 172.21.15.7:0/909320856 conn(0x7f91580fc520 msgr2=0x7f91580fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:41.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 -- 172.21.15.7:0/909320856 shutdown_connections 2024-09-21T04:45:41.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 -- 172.21.15.7:0/909320856 wait complete. 2024-09-21T04:45:41.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.154+0000 7f915e251640 1 Processor -- start 2024-09-21T04:45:41.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f915e251640 1 -- start start 2024-09-21T04:45:41.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f915e251640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 0x7f915819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f915e251640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91581033e0 0x7f915819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f915e251640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f915810edd0 con 0x7f9158102ac0 2024-09-21T04:45:41.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f915e251640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f915810ec50 con 0x7f91581033e0 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f91577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 0x7f915819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f91577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 0x7f915819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49178/0 (socket says 172.21.15.7:49178) 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f91577fe640 1 -- 172.21.15.7:0/1333454410 learned_addr learned my addr 172.21.15.7:0/1333454410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f9156ffd640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91581033e0 0x7f915819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f91577fe640 1 -- 172.21.15.7:0/1333454410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91581033e0 msgr2=0x7f915819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f91577fe640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91581033e0 0x7f915819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.155+0000 7f91577fe640 1 -- 172.21.15.7:0/1333454410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9158199a10 con 0x7f9158102ac0 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.156+0000 7f91577fe640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 0x7f915819f800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9148031460 tx=0x7f91480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:41.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.156+0000 7f9154ff9640 1 -- 172.21.15.7:0/1333454410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f914803f070 con 0x7f9158102ac0 2024-09-21T04:45:41.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.156+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9158199ca0 con 0x7f9158102ac0 2024-09-21T04:45:41.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.156+0000 7f9154ff9640 1 -- 172.21.15.7:0/1333454410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91480035f0 con 0x7f9158102ac0 2024-09-21T04:45:41.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.156+0000 7f9154ff9640 1 -- 172.21.15.7:0/1333454410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9148035030 con 0x7f9158102ac0 2024-09-21T04:45:41.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.156+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915819a1d0 con 0x7f9158102ac0 2024-09-21T04:45:41.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.157+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915806aa90 con 0x7f9158102ac0 2024-09-21T04:45:41.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.158+0000 7f9154ff9640 1 -- 172.21.15.7:0/1333454410 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9148042070 con 0x7f9158102ac0 2024-09-21T04:45:41.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.158+0000 7f9154ff9640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f914007af80 0x7f914007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.158+0000 7f9154ff9640 1 -- 172.21.15.7:0/1333454410 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f91480be960 con 0x7f9158102ac0 2024-09-21T04:45:41.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.158+0000 7f9156ffd640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f914007af80 0x7f914007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.159+0000 7f9156ffd640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f914007af80 0x7f914007d440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91580fee80 tx=0x7f914c00b2e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:41.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.163+0000 7f9154ff9640 1 -- 172.21.15.7:0/1333454410 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9148089ee0 con 0x7f9158102ac0 2024-09-21T04:45:41.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:40 smithi053 ceph-mon[80176]: pgmap v327: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.358+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f915819aef0 con 0x7f914007af80 2024-09-21T04:45:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.368+0000 7f9154ff9640 1 -- 172.21.15.7:0/1333454410 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f915819aef0 con 0x7f914007af80 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (7m) 105s ago 18m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (10m) 105s ago 18m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (10m) 91s ago 17m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (5m) 105s ago 18m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (5m) 91s ago 17m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (7m) 105s ago 18m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (110s) 105s ago 13m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (2m) 105s ago 13m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (13m) 91s ago 13m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:45:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (94s) 91s ago 13m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (9m) 105s ago 20m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (8m) 91s ago 17m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (6m) 105s ago 20m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (6m) 91s ago 17m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (8m) 105s ago 18m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (8m) 91s ago 17m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (5m) 105s ago 16m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (4m) 105s ago 16m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (4m) 105s ago 15m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (3m) 91s ago 15m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (2m) 91s ago 14m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (2m) 91s ago 14m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:45:41.372 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (8m) 105s ago 18m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:45:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.370+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f914007af80 msgr2=0x7f914007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.370+0000 7f915e251640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f914007af80 0x7f914007d440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91580fee80 tx=0x7f914c00b2e0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.370+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 msgr2=0x7f915819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.370+0000 7f915e251640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 0x7f915819f800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9148031460 tx=0x7f91480045e0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.370+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 shutdown_connections 2024-09-21T04:45:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.370+0000 7f915e251640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f914007af80 0x7f914007d440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.370+0000 7f915e251640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91581033e0 0x7f915819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.371+0000 7f915e251640 1 --2- 172.21.15.7:0/1333454410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9158102ac0 0x7f915819f800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.371+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 >> 172.21.15.7:0/1333454410 conn(0x7f91580fc520 msgr2=0x7f9158100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.371+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 shutdown_connections 2024-09-21T04:45:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.371+0000 7f915e251640 1 -- 172.21.15.7:0/1333454410 wait complete. 2024-09-21T04:45:41.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.453+0000 7f14cd4e3640 1 -- 172.21.15.7:0/3999815968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8106800 msgr2=0x7f14c8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.453+0000 7f14cd4e3640 1 --2- 172.21.15.7:0/3999815968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8106800 0x7f14c8106be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009880 tx=0x7f14b802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.453+0000 7f14cd4e3640 1 -- 172.21.15.7:0/3999815968 shutdown_connections 2024-09-21T04:45:41.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.453+0000 7f14cd4e3640 1 --2- 172.21.15.7:0/3999815968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c8100800 0x7f14c8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.453+0000 7f14cd4e3640 1 --2- 172.21.15.7:0/3999815968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8106800 0x7f14c8106be0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.453+0000 7f14cd4e3640 1 -- 172.21.15.7:0/3999815968 >> 172.21.15.7:0/3999815968 conn(0x7f14c80fc520 msgr2=0x7f14c80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:41.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.453+0000 7f14cd4e3640 1 -- 172.21.15.7:0/3999815968 shutdown_connections 2024-09-21T04:45:41.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.454+0000 7f14cd4e3640 1 -- 172.21.15.7:0/3999815968 wait complete. 2024-09-21T04:45:41.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.454+0000 7f14cd4e3640 1 Processor -- start 2024-09-21T04:45:41.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.454+0000 7f14cd4e3640 1 -- start start 2024-09-21T04:45:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.454+0000 7f14cd4e3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8100800 0x7f14c819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.454+0000 7f14cd4e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c8106800 0x7f14c819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14cd4e3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14c810dcb0 con 0x7f14c8100800 2024-09-21T04:45:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14cd4e3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14c810db30 con 0x7f14c8106800 2024-09-21T04:45:41.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8100800 0x7f14c819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8100800 0x7f14c819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49188/0 (socket says 172.21.15.7:49188) 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c6ffd640 1 -- 172.21.15.7:0/4229898818 learned_addr learned my addr 172.21.15.7:0/4229898818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c67fc640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c8106800 0x7f14c819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c6ffd640 1 -- 172.21.15.7:0/4229898818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c8106800 msgr2=0x7f14c819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c6ffd640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c8106800 0x7f14c819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c6ffd640 1 -- 172.21.15.7:0/4229898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c81977b0 con 0x7f14c8100800 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c67fc640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c8106800 0x7f14c819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14c6ffd640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8100800 0x7f14c819d630 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14b8031460 tx=0x7f14b80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14affff640 1 -- 172.21.15.7:0/4229898818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14b803f070 con 0x7f14c8100800 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c8197a40 con 0x7f14c8100800 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14affff640 1 -- 172.21.15.7:0/4229898818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14b80035f0 con 0x7f14c8100800 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14affff640 1 -- 172.21.15.7:0/4229898818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14b8035030 con 0x7f14c8100800 2024-09-21T04:45:41.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.455+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c8197f70 con 0x7f14c8100800 2024-09-21T04:45:41.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.457+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c8101f40 con 0x7f14c8100800 2024-09-21T04:45:41.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.457+0000 7f14affff640 1 -- 172.21.15.7:0/4229898818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f14b8042070 con 0x7f14c8100800 2024-09-21T04:45:41.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.458+0000 7f14affff640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b007af80 0x7f14b007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.458+0000 7f14affff640 1 -- 172.21.15.7:0/4229898818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f14b80be970 con 0x7f14c8100800 2024-09-21T04:45:41.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.458+0000 7f14c67fc640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b007af80 0x7f14b007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.458+0000 7f14c67fc640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b007af80 0x7f14b007d440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14c80ffb90 tx=0x7f14bc005e50 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:41.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.462+0000 7f14affff640 1 -- 172.21.15.7:0/4229898818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14b80c0e20 con 0x7f14c8100800 2024-09-21T04:45:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.721+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f14c8198220 con 0x7f14c8100800 2024-09-21T04:45:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.721+0000 7f14affff640 1 -- 172.21.15.7:0/4229898818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f14b8089f90 con 0x7f14c8100800 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:45:41.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:45:41.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:45:41.726 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:45:41.726 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:45:41.726 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:45:41.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:45:41.726 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:45:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b007af80 msgr2=0x7f14b007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b007af80 0x7f14b007d440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14c80ffb90 tx=0x7f14bc005e50 comp rx=0 tx=0).stop 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8100800 msgr2=0x7f14c819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8100800 0x7f14c819d630 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14b8031460 tx=0x7f14b80045e0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 shutdown_connections 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b007af80 0x7f14b007d440 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14c8106800 0x7f14c819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 --2- 172.21.15.7:0/4229898818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14c8100800 0x7f14c819d630 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 >> 172.21.15.7:0/4229898818 conn(0x7f14c80fc520 msgr2=0x7f14c806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 shutdown_connections 2024-09-21T04:45:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.724+0000 7f14cd4e3640 1 -- 172.21.15.7:0/4229898818 wait complete. 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.807+0000 7f84f6431640 1 -- 172.21.15.7:0/1854155636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0101900 msgr2=0x7f84f0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.807+0000 7f84f6431640 1 --2- 172.21.15.7:0/1854155636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0101900 0x7f84f0101d60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009880 tx=0x7f84e402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.807+0000 7f84f6431640 1 -- 172.21.15.7:0/1854155636 shutdown_connections 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.807+0000 7f84f6431640 1 --2- 172.21.15.7:0/1854155636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0101900 0x7f84f0101d60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.807+0000 7f84f6431640 1 --2- 172.21.15.7:0/1854155636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f0107900 0x7f84f0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.807+0000 7f84f6431640 1 -- 172.21.15.7:0/1854155636 >> 172.21.15.7:0/1854155636 conn(0x7f84f00fd600 msgr2=0x7f84f00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.807+0000 7f84f6431640 1 -- 172.21.15.7:0/1854155636 shutdown_connections 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.807+0000 7f84f6431640 1 -- 172.21.15.7:0/1854155636 wait complete. 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.808+0000 7f84f6431640 1 Processor -- start 2024-09-21T04:45:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.808+0000 7f84f6431640 1 -- start start 2024-09-21T04:45:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84f6431640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f0101900 0x7f84f019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84f6431640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0107900 0x7f84f019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84f6431640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84f010edd0 con 0x7f84f0107900 2024-09-21T04:45:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84f6431640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84f010ec50 con 0x7f84f0101900 2024-09-21T04:45:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0107900 0x7f84f019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0107900 0x7f84f019dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49212/0 (socket says 172.21.15.7:49212) 2024-09-21T04:45:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ef7fe640 1 -- 172.21.15.7:0/3066302153 learned_addr learned my addr 172.21.15.7:0/3066302153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84effff640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f0101900 0x7f84f019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ef7fe640 1 -- 172.21.15.7:0/3066302153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f0101900 msgr2=0x7f84f019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ef7fe640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f0101900 0x7f84f019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ef7fe640 1 -- 172.21.15.7:0/3066302153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f0197800 con 0x7f84f0107900 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ef7fe640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0107900 0x7f84f019dbc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84e4031460 tx=0x7f84e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ed7fa640 1 -- 172.21.15.7:0/3066302153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f84e403f070 con 0x7f84f0107900 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ed7fa640 1 -- 172.21.15.7:0/3066302153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f84e40035f0 con 0x7f84f0107900 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f0197a90 con 0x7f84f0107900 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.809+0000 7f84ed7fa640 1 -- 172.21.15.7:0/3066302153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f84e4035030 con 0x7f84f0107900 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.810+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f0197fc0 con 0x7f84f0107900 2024-09-21T04:45:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.810+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f0103040 con 0x7f84f0107900 2024-09-21T04:45:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.811+0000 7f84ed7fa640 1 -- 172.21.15.7:0/3066302153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f84e4042070 con 0x7f84f0107900 2024-09-21T04:45:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.812+0000 7f84ed7fa640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f84d807af80 0x7f84d807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.812+0000 7f84ed7fa640 1 -- 172.21.15.7:0/3066302153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f84e40be8c0 con 0x7f84f0107900 2024-09-21T04:45:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.812+0000 7f84effff640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f84d807af80 0x7f84d807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.812+0000 7f84effff640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f84d807af80 0x7f84d807d440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84e0005fd0 tx=0x7f84e000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:41.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:41.816+0000 7f84ed7fa640 1 -- 172.21.15.7:0/3066302153 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84e4089ee0 con 0x7f84f0107900 2024-09-21T04:45:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.041+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f84f0109e70 con 0x7f84f0107900 2024-09-21T04:45:42.044 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:41 smithi007 ceph-mon[108238]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:42.044 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:41 smithi007 ceph-mon[108238]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:42.044 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:41 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4229898818' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.041+0000 7f84ed7fa640 1 -- 172.21.15.7:0/3066302153 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f84e408ed90 con 0x7f84f0107900 2024-09-21T04:45:42.045 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:45:42.045 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:45:42.045 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:45:42.045 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:45:42.045 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:45:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:45:42.045 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:45:42.046 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:45:42.047 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:45:42.048 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.043+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f84d807af80 msgr2=0x7f84d807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.043+0000 7f84f6431640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f84d807af80 0x7f84d807d440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84e0005fd0 tx=0x7f84e000b390 comp rx=0 tx=0).stop 2024-09-21T04:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.043+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0107900 msgr2=0x7f84f019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.043+0000 7f84f6431640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0107900 0x7f84f019dbc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84e4031460 tx=0x7f84e40045e0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.044+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 shutdown_connections 2024-09-21T04:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.044+0000 7f84f6431640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f84d807af80 0x7f84d807d440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.044+0000 7f84f6431640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f0107900 0x7f84f019dbc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.044+0000 7f84f6431640 1 --2- 172.21.15.7:0/3066302153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f0101900 0x7f84f019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.044+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 >> 172.21.15.7:0/3066302153 conn(0x7f84f00fd600 msgr2=0x7f84f00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.044+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 shutdown_connections 2024-09-21T04:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.044+0000 7f84f6431640 1 -- 172.21.15.7:0/3066302153 wait complete. 2024-09-21T04:45:42.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.129+0000 7fd27a268640 1 -- 172.21.15.7:0/3246032611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 msgr2=0x7fd274101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.129+0000 7fd27a268640 1 --2- 172.21.15.7:0/3246032611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 0x7fd274101d60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd268009880 tx=0x7fd26802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.129+0000 7fd27a268640 1 -- 172.21.15.7:0/3246032611 shutdown_connections 2024-09-21T04:45:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.129+0000 7fd27a268640 1 --2- 172.21.15.7:0/3246032611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 0x7fd274101d60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.129+0000 7fd27a268640 1 --2- 172.21.15.7:0/3246032611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd274107900 0x7fd274107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.129+0000 7fd27a268640 1 -- 172.21.15.7:0/3246032611 >> 172.21.15.7:0/3246032611 conn(0x7fd2740fd600 msgr2=0x7fd2740ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.129+0000 7fd27a268640 1 -- 172.21.15.7:0/3246032611 shutdown_connections 2024-09-21T04:45:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.129+0000 7fd27a268640 1 -- 172.21.15.7:0/3246032611 wait complete. 2024-09-21T04:45:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd27a268640 1 Processor -- start 2024-09-21T04:45:42.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd27a268640 1 -- start start 2024-09-21T04:45:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd27a268640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 0x7fd27419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd27a268640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd274107900 0x7fd27419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd27a268640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd27410edd0 con 0x7fd274101900 2024-09-21T04:45:42.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd27a268640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd27410ec50 con 0x7fd274107900 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd273fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 0x7fd27419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd273fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 0x7fd27419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49240/0 (socket says 172.21.15.7:49240) 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd273fff640 1 -- 172.21.15.7:0/3695329685 learned_addr learned my addr 172.21.15.7:0/3695329685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.130+0000 7fd2737fe640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd274107900 0x7fd27419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd273fff640 1 -- 172.21.15.7:0/3695329685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd274107900 msgr2=0x7fd27419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd273fff640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd274107900 0x7fd27419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd273fff640 1 -- 172.21.15.7:0/3695329685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2741999d0 con 0x7fd274101900 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd273fff640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 0x7fd27419f810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd26400d6f0 tx=0x7fd26400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd2717fa640 1 -- 172.21.15.7:0/3695329685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd264017070 con 0x7fd274101900 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd2717fa640 1 -- 172.21.15.7:0/3695329685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd26400de00 con 0x7fd274101900 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd274199cc0 con 0x7fd274101900 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd2717fa640 1 -- 172.21.15.7:0/3695329685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd264004bb0 con 0x7fd274101900 2024-09-21T04:45:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.131+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27419a220 con 0x7fd274101900 2024-09-21T04:45:42.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.132+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd274103040 con 0x7fd274101900 2024-09-21T04:45:42.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.133+0000 7fd2717fa640 1 -- 172.21.15.7:0/3695329685 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd264009a30 con 0x7fd274101900 2024-09-21T04:45:42.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.133+0000 7fd2717fa640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd25c07af90 0x7fd25c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:42.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.133+0000 7fd2717fa640 1 -- 172.21.15.7:0/3695329685 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd26409db20 con 0x7fd274101900 2024-09-21T04:45:42.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.133+0000 7fd2737fe640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd25c07af90 0x7fd25c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:42.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.134+0000 7fd2737fe640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd25c07af90 0x7fd25c07d450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd27419b1c0 tx=0x7fd268031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:42.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.138+0000 7fd2717fa640 1 -- 172.21.15.7:0/3695329685 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd264069140 con 0x7fd274101900 2024-09-21T04:45:42.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:41 smithi053 ceph-mon[80176]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:42.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:41 smithi053 ceph-mon[80176]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:42.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:41 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4229898818' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:45:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.343+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd27419a4d0 con 0x7fd25c07af90 2024-09-21T04:45:42.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.345+0000 7fd2717fa640 1 -- 172.21.15.7:0/3695329685 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd27419a4d0 con 0x7fd25c07af90 2024-09-21T04:45:42.348 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:45:42.348 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:45:42.348 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:45:42.348 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:45:42.348 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:45:42.348 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:45:42.348 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:45:42.349 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:45:42.349 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:45:42.349 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:45:42.349 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:45:42.349 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:45:42.349 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:45:42.349 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:45:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd25c07af90 msgr2=0x7fd25c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd25c07af90 0x7fd25c07d450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd27419b1c0 tx=0x7fd268031400 comp rx=0 tx=0).stop 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 msgr2=0x7fd27419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 0x7fd27419f810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd26400d6f0 tx=0x7fd26400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 shutdown_connections 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd25c07af90 0x7fd25c07d450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd274107900 0x7fd27419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 --2- 172.21.15.7:0/3695329685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd274101900 0x7fd27419f810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 >> 172.21.15.7:0/3695329685 conn(0x7fd2740fd600 msgr2=0x7fd274074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.347+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 shutdown_connections 2024-09-21T04:45:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.348+0000 7fd27a268640 1 -- 172.21.15.7:0/3695329685 wait complete. 2024-09-21T04:45:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.431+0000 7f8a0a306640 1 -- 172.21.15.7:0/3416351063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04106800 msgr2=0x7f8a04106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.431+0000 7f8a0a306640 1 --2- 172.21.15.7:0/3416351063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04106800 0x7f8a04106be0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009880 tx=0x7f89f402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.432+0000 7f8a0a306640 1 -- 172.21.15.7:0/3416351063 shutdown_connections 2024-09-21T04:45:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.432+0000 7f8a0a306640 1 --2- 172.21.15.7:0/3416351063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04100800 0x7f8a04100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.432+0000 7f8a0a306640 1 --2- 172.21.15.7:0/3416351063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04106800 0x7f8a04106be0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.432+0000 7f8a0a306640 1 -- 172.21.15.7:0/3416351063 >> 172.21.15.7:0/3416351063 conn(0x7f8a040fc520 msgr2=0x7f8a040fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.432+0000 7f8a0a306640 1 -- 172.21.15.7:0/3416351063 shutdown_connections 2024-09-21T04:45:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.432+0000 7f8a0a306640 1 -- 172.21.15.7:0/3416351063 wait complete. 2024-09-21T04:45:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.432+0000 7f8a0a306640 1 Processor -- start 2024-09-21T04:45:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.432+0000 7f8a0a306640 1 -- start start 2024-09-21T04:45:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a0a306640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04100800 0x7f8a040fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a0a306640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04106800 0x7f8a040fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a0a306640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410dcd0 con 0x7f8a04100800 2024-09-21T04:45:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a0a306640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410db50 con 0x7f8a04106800 2024-09-21T04:45:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04100800 0x7f8a040fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04100800 0x7f8a040fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49256/0 (socket says 172.21.15.7:49256) 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a03fff640 1 -- 172.21.15.7:0/2453157662 learned_addr learned my addr 172.21.15.7:0/2453157662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a037fe640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04106800 0x7f8a040fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a03fff640 1 -- 172.21.15.7:0/2453157662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04106800 msgr2=0x7f8a040fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a03fff640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04106800 0x7f8a040fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a03fff640 1 -- 172.21.15.7:0/2453157662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a040fd650 con 0x7f8a04100800 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a03fff640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04100800 0x7f8a040fcbd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89f4031460 tx=0x7f89f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a017fa640 1 -- 172.21.15.7:0/2453157662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f89f403f070 con 0x7f8a04100800 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.433+0000 7f8a017fa640 1 -- 172.21.15.7:0/2453157662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f89f4004b10 con 0x7f8a04100800 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.434+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a040fd8e0 con 0x7f8a04100800 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.434+0000 7f8a017fa640 1 -- 172.21.15.7:0/2453157662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f89f4035030 con 0x7f8a04100800 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.434+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041a3d80 con 0x7f8a04100800 2024-09-21T04:45:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.435+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a04101f40 con 0x7f8a04100800 2024-09-21T04:45:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.435+0000 7f8a017fa640 1 -- 172.21.15.7:0/2453157662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f89f4042070 con 0x7f8a04100800 2024-09-21T04:45:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.436+0000 7f8a017fa640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f89ec07af90 0x7f89ec07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:45:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.436+0000 7f8a017fa640 1 -- 172.21.15.7:0/2453157662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f89f40bea20 con 0x7f8a04100800 2024-09-21T04:45:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.436+0000 7f8a037fe640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f89ec07af90 0x7f89ec07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:45:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.436+0000 7f8a037fe640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f89ec07af90 0x7f89ec07d450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a040fe660 tx=0x7f89f800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:45:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.440+0000 7f8a017fa640 1 -- 172.21.15.7:0/2453157662 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89f408a040 con 0x7f8a04100800 2024-09-21T04:45:42.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.701+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f8a04109d80 con 0x7f8a04100800 2024-09-21T04:45:42.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.702+0000 7f8a017fa640 1 -- 172.21.15.7:0/2453157662 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f89f408eef0 con 0x7f8a04100800 2024-09-21T04:45:42.705 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:45:42.705 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:45:42.705 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:45:42.705 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:45:42.705 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:45:42.705 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:45:42.705 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f89ec07af90 msgr2=0x7f89ec07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f89ec07af90 0x7f89ec07d450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a040fe660 tx=0x7f89f800b390 comp rx=0 tx=0).stop 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04100800 msgr2=0x7f8a040fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04100800 0x7f8a040fcbd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89f4031460 tx=0x7f89f40045e0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 shutdown_connections 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f89ec07af90 0x7f89ec07d450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04106800 0x7f8a040fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 --2- 172.21.15.7:0/2453157662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a04100800 0x7f8a040fcbd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 >> 172.21.15.7:0/2453157662 conn(0x7f8a040fc520 msgr2=0x7f8a0406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 shutdown_connections 2024-09-21T04:45:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:45:42.704+0000 7f8a0a306640 1 -- 172.21.15.7:0/2453157662 wait complete. 2024-09-21T04:45:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:42 smithi053 ceph-mon[80176]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:42 smithi053 ceph-mon[80176]: pgmap v328: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:42 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3066302153' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:45:43.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:42 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2453157662' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:45:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:42 smithi007 ceph-mon[108238]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:42 smithi007 ceph-mon[108238]: pgmap v328: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:42 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3066302153' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:45:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:42 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2453157662' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:45:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:43 smithi053 ceph-mon[80176]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:44.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:45:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:43 smithi007 ceph-mon[108238]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:45:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:45:45.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:44 smithi053 ceph-mon[80176]: pgmap v329: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:45.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:44 smithi007 ceph-mon[108238]: pgmap v329: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:47.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:46 smithi053 ceph-mon[80176]: pgmap v330: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:46 smithi007 ceph-mon[108238]: pgmap v330: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:48 smithi053 ceph-mon[80176]: pgmap v331: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:48 smithi007 ceph-mon[108238]: pgmap v331: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:51.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:50 smithi007 ceph-mon[108238]: pgmap v332: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:50 smithi053 ceph-mon[80176]: pgmap v332: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:52 smithi007 ceph-mon[108238]: pgmap v333: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:53.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:52 smithi053 ceph-mon[80176]: pgmap v333: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:54 smithi053 ceph-mon[80176]: pgmap v334: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:54 smithi007 ceph-mon[108238]: pgmap v334: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:56 smithi053 ceph-mon[80176]: pgmap v335: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:56 smithi007 ceph-mon[108238]: pgmap v335: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:58 smithi007 ceph-mon[108238]: pgmap v336: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:45:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:45:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:58 smithi053 ceph-mon[80176]: pgmap v336: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:45:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:45:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:46:01.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:00 smithi007 ceph-mon[108238]: pgmap v337: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:00 smithi053 ceph-mon[80176]: pgmap v337: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:02 smithi053 ceph-mon[80176]: pgmap v338: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:02 smithi007 ceph-mon[108238]: pgmap v338: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:04 smithi007 ceph-mon[108238]: pgmap v339: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:04 smithi053 ceph-mon[80176]: pgmap v339: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:06 smithi053 ceph-mon[80176]: pgmap v340: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:06 smithi007 ceph-mon[108238]: pgmap v340: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:08 smithi053 ceph-mon[80176]: pgmap v341: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:08 smithi007 ceph-mon[108238]: pgmap v341: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:10 smithi007 ceph-mon[108238]: pgmap v342: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:10 smithi053 ceph-mon[80176]: pgmap v342: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.802+0000 7f3cf83bf640 1 -- 172.21.15.7:0/1500847712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf01033e0 msgr2=0x7f3cf010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.802+0000 7f3cf83bf640 1 --2- 172.21.15.7:0/1500847712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf01033e0 0x7f3cf010b8d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009880 tx=0x7f3ce002a530 comp rx=0 tx=0).stop 2024-09-21T04:46:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.802+0000 7f3cf83bf640 1 -- 172.21.15.7:0/1500847712 shutdown_connections 2024-09-21T04:46:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.802+0000 7f3cf83bf640 1 --2- 172.21.15.7:0/1500847712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf01033e0 0x7f3cf010b8d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.802+0000 7f3cf83bf640 1 --2- 172.21.15.7:0/1500847712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf0102ac0 0x7f3cf0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.802+0000 7f3cf83bf640 1 -- 172.21.15.7:0/1500847712 >> 172.21.15.7:0/1500847712 conn(0x7f3cf00fc520 msgr2=0x7f3cf00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.802+0000 7f3cf83bf640 1 -- 172.21.15.7:0/1500847712 shutdown_connections 2024-09-21T04:46:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.802+0000 7f3cf83bf640 1 -- 172.21.15.7:0/1500847712 wait complete. 2024-09-21T04:46:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf83bf640 1 Processor -- start 2024-09-21T04:46:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf83bf640 1 -- start start 2024-09-21T04:46:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf83bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf0102ac0 0x7f3cf019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf83bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf01033e0 0x7f3cf019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf83bf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf010edf0 con 0x7f3cf0102ac0 2024-09-21T04:46:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf83bf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf010ec70 con 0x7f3cf01033e0 2024-09-21T04:46:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf615b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf0102ac0 0x7f3cf019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf615b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf0102ac0 0x7f3cf019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55010/0 (socket says 172.21.15.7:55010) 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf615b640 1 -- 172.21.15.7:0/3607299862 learned_addr learned my addr 172.21.15.7:0/3607299862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf595a640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf01033e0 0x7f3cf019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf615b640 1 -- 172.21.15.7:0/3607299862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf01033e0 msgr2=0x7f3cf019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf615b640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf01033e0 0x7f3cf019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.803+0000 7f3cf615b640 1 -- 172.21.15.7:0/3607299862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf0197820 con 0x7f3cf0102ac0 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.804+0000 7f3cf615b640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf0102ac0 0x7f3cf019d6a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00e970 tx=0x7f3cec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.804+0000 7f3ce77fe640 1 -- 172.21.15.7:0/3607299862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3cec00cac0 con 0x7f3cf0102ac0 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.804+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf0197b10 con 0x7f3cf0102ac0 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.804+0000 7f3ce77fe640 1 -- 172.21.15.7:0/3607299862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3cec0054e0 con 0x7f3cf0102ac0 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.804+0000 7f3ce77fe640 1 -- 172.21.15.7:0/3607299862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3cec010430 con 0x7f3cf0102ac0 2024-09-21T04:46:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.804+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf0198070 con 0x7f3cf0102ac0 2024-09-21T04:46:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.805+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf006aa90 con 0x7f3cf0102ac0 2024-09-21T04:46:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.806+0000 7f3ce77fe640 1 -- 172.21.15.7:0/3607299862 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3cec004f00 con 0x7f3cf0102ac0 2024-09-21T04:46:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.806+0000 7f3ce77fe640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cdc07afe0 0x7f3cdc07d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.807+0000 7f3ce77fe640 1 -- 172.21.15.7:0/3607299862 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f3cec09e6a0 con 0x7f3cf0102ac0 2024-09-21T04:46:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.807+0000 7f3cf595a640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cdc07afe0 0x7f3cdc07d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.807+0000 7f3cf595a640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cdc07afe0 0x7f3cdc07d4a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0198b70 tx=0x7f3ce0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:12.811+0000 7f3ce77fe640 1 -- 172.21.15.7:0/3607299862 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3cec069c20 con 0x7f3cf0102ac0 2024-09-21T04:46:13.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.016+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3cf0198380 con 0x7f3cdc07afe0 2024-09-21T04:46:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.017+0000 7f3ce77fe640 1 -- 172.21.15.7:0/3607299862 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f3cf0198380 con 0x7f3cdc07afe0 2024-09-21T04:46:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cdc07afe0 msgr2=0x7f3cdc07d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cdc07afe0 0x7f3cdc07d4a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0198b70 tx=0x7f3ce0031400 comp rx=0 tx=0).stop 2024-09-21T04:46:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf0102ac0 msgr2=0x7f3cf019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf0102ac0 0x7f3cf019d6a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00e970 tx=0x7f3cec00ed70 comp rx=0 tx=0).stop 2024-09-21T04:46:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 shutdown_connections 2024-09-21T04:46:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cdc07afe0 0x7f3cdc07d4a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf01033e0 0x7f3cf019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 --2- 172.21.15.7:0/3607299862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cf0102ac0 0x7f3cf019d6a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 >> 172.21.15.7:0/3607299862 conn(0x7f3cf00fc520 msgr2=0x7f3cf0108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 shutdown_connections 2024-09-21T04:46:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.019+0000 7f3cf83bf640 1 -- 172.21.15.7:0/3607299862 wait complete. 2024-09-21T04:46:13.031 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:46:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.105+0000 7f49c745f640 1 -- 172.21.15.7:0/3010776448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074620 msgr2=0x7f49c0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.105+0000 7f49c745f640 1 --2- 172.21.15.7:0/3010776448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074620 0x7f49c0074a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009880 tx=0x7f49bc02a530 comp rx=0 tx=0).stop 2024-09-21T04:46:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.105+0000 7f49c745f640 1 -- 172.21.15.7:0/3010776448 shutdown_connections 2024-09-21T04:46:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.105+0000 7f49c745f640 1 --2- 172.21.15.7:0/3010776448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0074f40 0x7f49c010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.105+0000 7f49c745f640 1 --2- 172.21.15.7:0/3010776448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074620 0x7f49c0074a00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.105+0000 7f49c745f640 1 -- 172.21.15.7:0/3010776448 >> 172.21.15.7:0/3010776448 conn(0x7f49c00fd630 msgr2=0x7f49c00ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:13.108 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:12 smithi007 ceph-mon[108238]: pgmap v343: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.105+0000 7f49c745f640 1 -- 172.21.15.7:0/3010776448 shutdown_connections 2024-09-21T04:46:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.105+0000 7f49c745f640 1 -- 172.21.15.7:0/3010776448 wait complete. 2024-09-21T04:46:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c745f640 1 Processor -- start 2024-09-21T04:46:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c745f640 1 -- start start 2024-09-21T04:46:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c745f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0074620 0x7f49c019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c745f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074f40 0x7f49c019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c745f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0111000 con 0x7f49c0074f40 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c745f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0110e80 con 0x7f49c0074620 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c49fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074f40 0x7f49c019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c49fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074f40 0x7f49c019ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55032/0 (socket says 172.21.15.7:55032) 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c49fa640 1 -- 172.21.15.7:0/1533685310 learned_addr learned my addr 172.21.15.7:0/1533685310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.106+0000 7f49c51fb640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0074620 0x7f49c019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49c49fa640 1 -- 172.21.15.7:0/1533685310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0074620 msgr2=0x7f49c019fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49c49fa640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0074620 0x7f49c019fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49c49fa640 1 -- 172.21.15.7:0/1533685310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c0199b10 con 0x7f49c0074f40 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49c49fa640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074f40 0x7f49c019ff80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f49b000d6f0 tx=0x7f49b000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49b67fc640 1 -- 172.21.15.7:0/1533685310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49b0017070 con 0x7f49c0074f40 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c0199e00 con 0x7f49c0074f40 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49b67fc640 1 -- 172.21.15.7:0/1533685310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49b000de00 con 0x7f49c0074f40 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49b67fc640 1 -- 172.21.15.7:0/1533685310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49b0004bb0 con 0x7f49c0074f40 2024-09-21T04:46:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.107+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c019a360 con 0x7f49c0074f40 2024-09-21T04:46:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.108+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c0075cd0 con 0x7f49c0074f40 2024-09-21T04:46:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.109+0000 7f49b67fc640 1 -- 172.21.15.7:0/1533685310 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f49b0004e60 con 0x7f49c0074f40 2024-09-21T04:46:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.110+0000 7f49b67fc640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49ac07af30 0x7f49ac07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.110+0000 7f49b67fc640 1 -- 172.21.15.7:0/1533685310 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f49b009e340 con 0x7f49c0074f40 2024-09-21T04:46:13.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.110+0000 7f49c51fb640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49ac07af30 0x7f49ac07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.110+0000 7f49c51fb640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49ac07af30 0x7f49ac07d3f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0096c0 tx=0x7f49bc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:13.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.113+0000 7f49b67fc640 1 -- 172.21.15.7:0/1533685310 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49b0020030 con 0x7f49c0074f40 2024-09-21T04:46:13.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:12 smithi053 ceph-mon[80176]: pgmap v343: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.318+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f49c019a6d0 con 0x7f49ac07af30 2024-09-21T04:46:13.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.319+0000 7f49b67fc640 1 -- 172.21.15.7:0/1533685310 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f49c019a6d0 con 0x7f49ac07af30 2024-09-21T04:46:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.321+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49ac07af30 msgr2=0x7f49ac07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49ac07af30 0x7f49ac07d3f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0096c0 tx=0x7f49bc031400 comp rx=0 tx=0).stop 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074f40 msgr2=0x7f49c019ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074f40 0x7f49c019ff80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f49b000d6f0 tx=0x7f49b000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 shutdown_connections 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49ac07af30 0x7f49ac07d3f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0074f40 0x7f49c019ff80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 --2- 172.21.15.7:0/1533685310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c0074620 0x7f49c019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 >> 172.21.15.7:0/1533685310 conn(0x7f49c00fd630 msgr2=0x7f49c010b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 shutdown_connections 2024-09-21T04:46:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.322+0000 7f49c745f640 1 -- 172.21.15.7:0/1533685310 wait complete. 2024-09-21T04:46:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.405+0000 7fc9bcbb7640 1 -- 172.21.15.7:0/1872356062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81067f0 msgr2=0x7fc9b8106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.405+0000 7fc9bcbb7640 1 --2- 172.21.15.7:0/1872356062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81067f0 0x7fc9b8106bd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009880 tx=0x7fc9a802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.405+0000 7fc9bcbb7640 1 -- 172.21.15.7:0/1872356062 shutdown_connections 2024-09-21T04:46:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.405+0000 7fc9bcbb7640 1 --2- 172.21.15.7:0/1872356062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9b81007f0 0x7fc9b8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.405+0000 7fc9bcbb7640 1 --2- 172.21.15.7:0/1872356062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81067f0 0x7fc9b8106bd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.405+0000 7fc9bcbb7640 1 -- 172.21.15.7:0/1872356062 >> 172.21.15.7:0/1872356062 conn(0x7fc9b80fc550 msgr2=0x7fc9b80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.406+0000 7fc9bcbb7640 1 -- 172.21.15.7:0/1872356062 shutdown_connections 2024-09-21T04:46:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.406+0000 7fc9bcbb7640 1 -- 172.21.15.7:0/1872356062 wait complete. 2024-09-21T04:46:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.406+0000 7fc9bcbb7640 1 Processor -- start 2024-09-21T04:46:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9bcbb7640 1 -- start start 2024-09-21T04:46:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9bcbb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81007f0 0x7fc9b8198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9bcbb7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9b81067f0 0x7fc9b8198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9bcbb7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b810dcc0 con 0x7fc9b81007f0 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9bcbb7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b810db40 con 0x7fc9b81067f0 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81007f0 0x7fc9b8198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81007f0 0x7fc9b8198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55048/0 (socket says 172.21.15.7:55048) 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9b659c640 1 -- 172.21.15.7:0/874165025 learned_addr learned my addr 172.21.15.7:0/874165025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9add9b640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9b81067f0 0x7fc9b8198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9b659c640 1 -- 172.21.15.7:0/874165025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9b81067f0 msgr2=0x7fc9b8198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9b659c640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9b81067f0 0x7fc9b8198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.407+0000 7fc9b659c640 1 -- 172.21.15.7:0/874165025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b8199350 con 0x7fc9b81007f0 2024-09-21T04:46:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.408+0000 7fc9b659c640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81007f0 0x7fc9b8198810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8031460 tx=0x7fc9a80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:13.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.408+0000 7fc9affff640 1 -- 172.21.15.7:0/874165025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc9a803f070 con 0x7fc9b81007f0 2024-09-21T04:46:13.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.408+0000 7fc9bcbb7640 1 -- 172.21.15.7:0/874165025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b81995e0 con 0x7fc9b81007f0 2024-09-21T04:46:13.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.408+0000 7fc9affff640 1 -- 172.21.15.7:0/874165025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc9a8004cf0 con 0x7fc9b81007f0 2024-09-21T04:46:13.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.408+0000 7fc9affff640 1 -- 172.21.15.7:0/874165025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc9a8035030 con 0x7fc9b81007f0 2024-09-21T04:46:13.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.408+0000 7fc9bcbb7640 1 -- 172.21.15.7:0/874165025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b81a1b90 con 0x7fc9b81007f0 2024-09-21T04:46:13.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.409+0000 7fc9ad59a640 1 -- 172.21.15.7:0/874165025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b8101f30 con 0x7fc9b81007f0 2024-09-21T04:46:13.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.411+0000 7fc9affff640 1 -- 172.21.15.7:0/874165025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc9a8042070 con 0x7fc9b81007f0 2024-09-21T04:46:13.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.412+0000 7fc9affff640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9a407af30 0x7fc9a407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.412+0000 7fc9affff640 1 -- 172.21.15.7:0/874165025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc9a80bf1d0 con 0x7fc9b81007f0 2024-09-21T04:46:13.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.412+0000 7fc9add9b640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9a407af30 0x7fc9a407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.412+0000 7fc9add9b640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9a407af30 0x7fc9a407d3f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005fd0 tx=0x7fc9a0005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.415+0000 7fc9affff640 1 -- 172.21.15.7:0/874165025 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9a808a750 con 0x7fc9b81007f0 2024-09-21T04:46:13.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.627+0000 7fc9ad59a640 1 -- 172.21.15.7:0/874165025 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc9b8100c50 con 0x7fc9a407af30 2024-09-21T04:46:13.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.637+0000 7fc9affff640 1 -- 172.21.15.7:0/874165025 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fc9b8100c50 con 0x7fc9a407af30 2024-09-21T04:46:13.640 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:46:13.640 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (8m) 2m ago 19m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:46:13.640 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (11m) 2m ago 19m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:46:13.640 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (10m) 2m ago 18m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:46:13.640 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (6m) 2m ago 19m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:46:13.640 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (6m) 2m ago 18m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (7m) 2m ago 18m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (2m) 2m ago 13m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (2m) 2m ago 13m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (13m) 2m ago 13m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (2m) 2m ago 13m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (10m) 2m ago 20m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (9m) 2m ago 18m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (6m) 2m ago 20m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (6m) 2m ago 18m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (9m) 2m ago 19m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (9m) 2m ago 18m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (6m) 2m ago 17m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:46:13.641 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (5m) 2m ago 16m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:46:13.642 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (4m) 2m ago 16m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:46:13.642 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (4m) 2m ago 15m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:46:13.642 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (3m) 2m ago 15m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:46:13.642 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (2m) 2m ago 14m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:46:13.642 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (8m) 2m ago 18m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:46:13.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.639+0000 7fc9ad59a640 1 -- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9a407af30 msgr2=0x7fc9a407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.639+0000 7fc9ad59a640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9a407af30 0x7fc9a407d3f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005fd0 tx=0x7fc9a0005e20 comp rx=0 tx=0).stop 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.639+0000 7fc9ad59a640 1 -- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81007f0 msgr2=0x7fc9b8198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.639+0000 7fc9ad59a640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81007f0 0x7fc9b8198810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8031460 tx=0x7fc9a80048f0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.639+0000 7fc9ad59a640 1 -- 172.21.15.7:0/874165025 shutdown_connections 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.639+0000 7fc9ad59a640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9a407af30 0x7fc9a407d3f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.639+0000 7fc9ad59a640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9b81067f0 0x7fc9b8198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.639+0000 7fc9ad59a640 1 --2- 172.21.15.7:0/874165025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9b81007f0 0x7fc9b8198810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.640+0000 7fc9ad59a640 1 -- 172.21.15.7:0/874165025 >> 172.21.15.7:0/874165025 conn(0x7fc9b80fc550 msgr2=0x7fc9b80fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.640+0000 7fc9ad59a640 1 -- 172.21.15.7:0/874165025 shutdown_connections 2024-09-21T04:46:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.640+0000 7fc9ad59a640 1 -- 172.21.15.7:0/874165025 wait complete. 2024-09-21T04:46:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.725+0000 7f90ecbc6640 1 -- 172.21.15.7:0/2160740087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 msgr2=0x7f90e8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.725+0000 7f90ecbc6640 1 --2- 172.21.15.7:0/2160740087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 0x7f90e8100c60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009880 tx=0x7f90dc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.725+0000 7f90ecbc6640 1 -- 172.21.15.7:0/2160740087 shutdown_connections 2024-09-21T04:46:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.725+0000 7f90ecbc6640 1 --2- 172.21.15.7:0/2160740087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 0x7f90e8100c60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.725+0000 7f90ecbc6640 1 --2- 172.21.15.7:0/2160740087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e8106800 0x7f90e8106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.725+0000 7f90ecbc6640 1 -- 172.21.15.7:0/2160740087 >> 172.21.15.7:0/2160740087 conn(0x7f90e80fc520 msgr2=0x7f90e80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.726+0000 7f90ecbc6640 1 -- 172.21.15.7:0/2160740087 shutdown_connections 2024-09-21T04:46:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.726+0000 7f90ecbc6640 1 -- 172.21.15.7:0/2160740087 wait complete. 2024-09-21T04:46:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.726+0000 7f90ecbc6640 1 Processor -- start 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.726+0000 7f90ecbc6640 1 -- start start 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90ecbc6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 0x7f90e819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90ecbc6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e8106800 0x7f90e819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90ecbc6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90e810dcd0 con 0x7f90e8100800 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90ecbc6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f90e810db50 con 0x7f90e8106800 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90e659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 0x7f90e819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90e659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 0x7f90e819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45812/0 (socket says 172.21.15.7:45812) 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90e659c640 1 -- 172.21.15.7:0/179802791 learned_addr learned my addr 172.21.15.7:0/179802791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90e5d9b640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e8106800 0x7f90e819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90e659c640 1 -- 172.21.15.7:0/179802791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e8106800 msgr2=0x7f90e819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90e659c640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e8106800 0x7f90e819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90e659c640 1 -- 172.21.15.7:0/179802791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e81999e0 con 0x7f90e8100800 2024-09-21T04:46:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90e659c640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 0x7f90e819f820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f90d800d6f0 tx=0x7f90d800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.727+0000 7f90d77fe640 1 -- 172.21.15.7:0/179802791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f90d8017070 con 0x7f90e8100800 2024-09-21T04:46:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.728+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e8199cd0 con 0x7f90e8100800 2024-09-21T04:46:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.728+0000 7f90d77fe640 1 -- 172.21.15.7:0/179802791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90d800de00 con 0x7f90e8100800 2024-09-21T04:46:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.728+0000 7f90d77fe640 1 -- 172.21.15.7:0/179802791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f90d8004bb0 con 0x7f90e8100800 2024-09-21T04:46:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.728+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e819a230 con 0x7f90e8100800 2024-09-21T04:46:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.729+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e8101f40 con 0x7f90e8100800 2024-09-21T04:46:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.729+0000 7f90d77fe640 1 -- 172.21.15.7:0/179802791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f90d8004e60 con 0x7f90e8100800 2024-09-21T04:46:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.730+0000 7f90d77fe640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f90d007af40 0x7f90d007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.730+0000 7f90e5d9b640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f90d007af40 0x7f90d007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.730+0000 7f90d77fe640 1 -- 172.21.15.7:0/179802791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f90d809dc00 con 0x7f90e8100800 2024-09-21T04:46:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.730+0000 7f90e5d9b640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f90d007af40 0x7f90d007d400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90e819b1d0 tx=0x7f90dc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.734+0000 7f90d77fe640 1 -- 172.21.15.7:0/179802791 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f90d80a3020 con 0x7f90e8100800 2024-09-21T04:46:14.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.998+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f90e819af10 con 0x7f90e8100800 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:13.999+0000 7f90d77fe640 1 -- 172.21.15.7:0/179802791 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f90d80691a0 con 0x7f90e8100800 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:46:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:46:14.003 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f90d007af40 msgr2=0x7f90d007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f90d007af40 0x7f90d007d400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90e819b1d0 tx=0x7f90dc031400 comp rx=0 tx=0).stop 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 msgr2=0x7f90e819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 0x7f90e819f820 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f90d800d6f0 tx=0x7f90d800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 shutdown_connections 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f90d007af40 0x7f90d007d400 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90e8106800 0x7f90e819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 --2- 172.21.15.7:0/179802791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90e8100800 0x7f90e819f820 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 >> 172.21.15.7:0/179802791 conn(0x7f90e80fc520 msgr2=0x7f90e80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 shutdown_connections 2024-09-21T04:46:14.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.001+0000 7f90ecbc6640 1 -- 172.21.15.7:0/179802791 wait complete. 2024-09-21T04:46:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.085+0000 7f0a1e272640 1 -- 172.21.15.7:0/2006443141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a181033e0 msgr2=0x7f0a1810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.085+0000 7f0a1e272640 1 --2- 172.21.15.7:0/2006443141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a181033e0 0x7f0a1810b8d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009880 tx=0x7f0a0c02a530 comp rx=0 tx=0).stop 2024-09-21T04:46:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.085+0000 7f0a1e272640 1 -- 172.21.15.7:0/2006443141 shutdown_connections 2024-09-21T04:46:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.085+0000 7f0a1e272640 1 --2- 172.21.15.7:0/2006443141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a181033e0 0x7f0a1810b8d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.085+0000 7f0a1e272640 1 --2- 172.21.15.7:0/2006443141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a18102ac0 0x7f0a18102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.085+0000 7f0a1e272640 1 -- 172.21.15.7:0/2006443141 >> 172.21.15.7:0/2006443141 conn(0x7f0a180fc520 msgr2=0x7f0a180fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.085+0000 7f0a1e272640 1 -- 172.21.15.7:0/2006443141 shutdown_connections 2024-09-21T04:46:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.086+0000 7f0a1e272640 1 -- 172.21.15.7:0/2006443141 wait complete. 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.086+0000 7f0a1e272640 1 Processor -- start 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.086+0000 7f0a1e272640 1 -- start start 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a1e272640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18102ac0 0x7f0a1819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a1e272640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a181033e0 0x7f0a1819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a1e272640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810edf0 con 0x7f0a18102ac0 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a1e272640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810ec70 con 0x7f0a181033e0 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18102ac0 0x7f0a1819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18102ac0 0x7f0a1819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45830/0 (socket says 172.21.15.7:45830) 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a17fff640 1 -- 172.21.15.7:0/1572134618 learned_addr learned my addr 172.21.15.7:0/1572134618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a177fe640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a181033e0 0x7f0a1819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a17fff640 1 -- 172.21.15.7:0/1572134618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a181033e0 msgr2=0x7f0a1819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a17fff640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a181033e0 0x7f0a1819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a17fff640 1 -- 172.21.15.7:0/1572134618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a18197800 con 0x7f0a18102ac0 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a177fe640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a181033e0 0x7f0a1819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a17fff640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18102ac0 0x7f0a1819d680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800d6f0 tx=0x7f0a0800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a157fa640 1 -- 172.21.15.7:0/1572134618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0a08017070 con 0x7f0a18102ac0 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a157fa640 1 -- 172.21.15.7:0/1572134618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a0800de00 con 0x7f0a18102ac0 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a18197af0 con 0x7f0a18102ac0 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.087+0000 7f0a157fa640 1 -- 172.21.15.7:0/1572134618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0a08004bb0 con 0x7f0a18102ac0 2024-09-21T04:46:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.088+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a18198080 con 0x7f0a18102ac0 2024-09-21T04:46:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.089+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a1806aa90 con 0x7f0a18102ac0 2024-09-21T04:46:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.089+0000 7f0a157fa640 1 -- 172.21.15.7:0/1572134618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0a08009a30 con 0x7f0a18102ac0 2024-09-21T04:46:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.090+0000 7f0a157fa640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0007af80 0x7f0a0007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.090+0000 7f0a157fa640 1 -- 172.21.15.7:0/1572134618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0a0809dbd0 con 0x7f0a18102ac0 2024-09-21T04:46:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.090+0000 7f0a177fe640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0007af80 0x7f0a0007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.090+0000 7f0a177fe640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0007af80 0x7f0a0007d440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c031460 tx=0x7f0a0c0095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.094+0000 7f0a157fa640 1 -- 172.21.15.7:0/1572134618 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a080691f0 con 0x7f0a18102ac0 2024-09-21T04:46:14.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.322+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0a18105f60 con 0x7f0a18102ac0 2024-09-21T04:46:14.325 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:13 smithi007 ceph-mon[108238]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:14.325 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:13 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.322+0000 7f0a157fa640 1 -- 172.21.15.7:0/1572134618 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f0a0806e0a0 con 0x7f0a18102ac0 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:46:14.326 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:46:14.327 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:46:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:46:14.329 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:46:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:46:14.329 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:46:14.329 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:46:14.329 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:46:14.329 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0007af80 msgr2=0x7f0a0007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0007af80 0x7f0a0007d440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c031460 tx=0x7f0a0c0095f0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18102ac0 msgr2=0x7f0a1819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18102ac0 0x7f0a1819d680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800d6f0 tx=0x7f0a0800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 shutdown_connections 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0007af80 0x7f0a0007d440 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a181033e0 0x7f0a1819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 --2- 172.21.15.7:0/1572134618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18102ac0 0x7f0a1819d680 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 >> 172.21.15.7:0/1572134618 conn(0x7f0a180fc520 msgr2=0x7f0a18108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:14.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 shutdown_connections 2024-09-21T04:46:14.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.326+0000 7f0a1e272640 1 -- 172.21.15.7:0/1572134618 wait complete. 2024-09-21T04:46:14.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.412+0000 7f167b9e7640 1 -- 172.21.15.7:0/3287709025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 msgr2=0x7f16741123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.412+0000 7f167b9e7640 1 --2- 172.21.15.7:0/3287709025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 0x7f16741123d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1664009880 tx=0x7f166402a530 comp rx=0 tx=0).stop 2024-09-21T04:46:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.412+0000 7f167b9e7640 1 -- 172.21.15.7:0/3287709025 shutdown_connections 2024-09-21T04:46:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.412+0000 7f167b9e7640 1 --2- 172.21.15.7:0/3287709025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 0x7f16741123d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.412+0000 7f167b9e7640 1 --2- 172.21.15.7:0/3287709025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1674074590 0x7f1674074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.412+0000 7f167b9e7640 1 -- 172.21.15.7:0/3287709025 >> 172.21.15.7:0/3287709025 conn(0x7f16740fd600 msgr2=0x7f16740ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.412+0000 7f167b9e7640 1 -- 172.21.15.7:0/3287709025 shutdown_connections 2024-09-21T04:46:14.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.412+0000 7f167b9e7640 1 -- 172.21.15.7:0/3287709025 wait complete. 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f167b9e7640 1 Processor -- start 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f167b9e7640 1 -- start start 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f167b9e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1674074590 0x7f167419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f167b9e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 0x7f167419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f167b9e7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16741153c0 con 0x7f1674074eb0 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f167b9e7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1674115240 con 0x7f1674074590 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f1678f82640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 0x7f167419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f1678f82640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 0x7f167419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45850/0 (socket says 172.21.15.7:45850) 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f1678f82640 1 -- 172.21.15.7:0/3928620115 learned_addr learned my addr 172.21.15.7:0/3928620115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.413+0000 7f1679783640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1674074590 0x7f167419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f1678f82640 1 -- 172.21.15.7:0/3928620115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1674074590 msgr2=0x7f167419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f1678f82640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1674074590 0x7f167419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f1678f82640 1 -- 172.21.15.7:0/3928620115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16741a2890 con 0x7f1674074eb0 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f1679783640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1674074590 0x7f167419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f1678f82640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 0x7f167419e500 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1664031460 tx=0x7f16640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f166a7fc640 1 -- 172.21.15.7:0/3928620115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f166403f070 con 0x7f1674074eb0 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16741a2b20 con 0x7f1674074eb0 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f166a7fc640 1 -- 172.21.15.7:0/3928620115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1664004cf0 con 0x7f1674074eb0 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f166a7fc640 1 -- 172.21.15.7:0/3928620115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1664035030 con 0x7f1674074eb0 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.414+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16741a3050 con 0x7f1674074eb0 2024-09-21T04:46:14.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.415+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1674075cd0 con 0x7f1674074eb0 2024-09-21T04:46:14.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.416+0000 7f166a7fc640 1 -- 172.21.15.7:0/3928620115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1664042070 con 0x7f1674074eb0 2024-09-21T04:46:14.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.416+0000 7f166a7fc640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f166007af30 0x7f166007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.417+0000 7f166a7fc640 1 -- 172.21.15.7:0/3928620115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f16640bebc0 con 0x7f1674074eb0 2024-09-21T04:46:14.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.417+0000 7f1679783640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f166007af30 0x7f166007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.417+0000 7f1679783640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f166007af30 0x7f166007d3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16700058c0 tx=0x7f1670005ec0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:14.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.421+0000 7f166a7fc640 1 -- 172.21.15.7:0/3928620115 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f166408a0c0 con 0x7f1674074eb0 2024-09-21T04:46:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:13 smithi053 ceph-mon[80176]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:13 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:46:14.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.624+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f167419f320 con 0x7f166007af30 2024-09-21T04:46:14.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.625+0000 7f166a7fc640 1 -- 172.21.15.7:0/3928620115 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f167419f320 con 0x7f166007af30 2024-09-21T04:46:14.628 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:46:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:46:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:46:14.628 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:46:14.629 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f166007af30 msgr2=0x7f166007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f166007af30 0x7f166007d3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16700058c0 tx=0x7f1670005ec0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 msgr2=0x7f167419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 0x7f167419e500 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1664031460 tx=0x7f16640048f0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 shutdown_connections 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f166007af30 0x7f166007d3f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1674074eb0 0x7f167419e500 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 --2- 172.21.15.7:0/3928620115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1674074590 0x7f167419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 >> 172.21.15.7:0/3928620115 conn(0x7f16740fd600 msgr2=0x7f1674104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 shutdown_connections 2024-09-21T04:46:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.628+0000 7f167b9e7640 1 -- 172.21.15.7:0/3928620115 wait complete. 2024-09-21T04:46:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.712+0000 7fe6c7a66640 1 -- 172.21.15.7:0/3100930468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0107900 msgr2=0x7fe6c0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.712+0000 7fe6c7a66640 1 --2- 172.21.15.7:0/3100930468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0107900 0x7fe6c0107ce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009c20 tx=0x7fe6bc02a970 comp rx=0 tx=0).stop 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.713+0000 7fe6c7a66640 1 -- 172.21.15.7:0/3100930468 shutdown_connections 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.713+0000 7fe6c7a66640 1 --2- 172.21.15.7:0/3100930468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6c0101900 0x7fe6c0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.713+0000 7fe6c7a66640 1 --2- 172.21.15.7:0/3100930468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0107900 0x7fe6c0107ce0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.713+0000 7fe6c7a66640 1 -- 172.21.15.7:0/3100930468 >> 172.21.15.7:0/3100930468 conn(0x7fe6c00fd600 msgr2=0x7fe6c00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.713+0000 7fe6c7a66640 1 -- 172.21.15.7:0/3100930468 shutdown_connections 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.713+0000 7fe6c7a66640 1 -- 172.21.15.7:0/3100930468 wait complete. 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.713+0000 7fe6c7a66640 1 Processor -- start 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.713+0000 7fe6c7a66640 1 -- start start 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c7a66640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0101900 0x7fe6c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c7a66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6c0107900 0x7fe6c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c7a66640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c010edd0 con 0x7fe6c0101900 2024-09-21T04:46:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c7a66640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c010ec50 con 0x7fe6c0107900 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c5802640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0101900 0x7fe6c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c5802640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0101900 0x7fe6c00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45862/0 (socket says 172.21.15.7:45862) 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c5802640 1 -- 172.21.15.7:0/714451917 learned_addr learned my addr 172.21.15.7:0/714451917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c5001640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6c0107900 0x7fe6c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c5802640 1 -- 172.21.15.7:0/714451917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6c0107900 msgr2=0x7fe6c0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c5802640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6c0107900 0x7fe6c0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c5802640 1 -- 172.21.15.7:0/714451917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c0074f70 con 0x7fe6c0101900 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6c5802640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0101900 0x7fe6c00744f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc02af60 tx=0x7fe6bc02fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.714+0000 7fe6b6ffd640 1 -- 172.21.15.7:0/714451917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe6bc047070 con 0x7fe6c0101900 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.715+0000 7fe6b6ffd640 1 -- 172.21.15.7:0/714451917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe6bc003e30 con 0x7fe6c0101900 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.715+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c0075200 con 0x7fe6c0101900 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.715+0000 7fe6b6ffd640 1 -- 172.21.15.7:0/714451917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe6bc034070 con 0x7fe6c0101900 2024-09-21T04:46:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.715+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a3d60 con 0x7fe6c0101900 2024-09-21T04:46:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.716+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c0103040 con 0x7fe6c0101900 2024-09-21T04:46:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.716+0000 7fe6b6ffd640 1 -- 172.21.15.7:0/714451917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe6bc04a070 con 0x7fe6c0101900 2024-09-21T04:46:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.717+0000 7fe6b6ffd640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe6ac07af90 0x7fe6ac07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.717+0000 7fe6b6ffd640 1 -- 172.21.15.7:0/714451917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe6bc0bed30 con 0x7fe6c0101900 2024-09-21T04:46:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.717+0000 7fe6c5001640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe6ac07af90 0x7fe6ac07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.717+0000 7fe6c5001640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe6ac07af90 0x7fe6ac07d450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0075f80 tx=0x7fe6b000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.722+0000 7fe6b6ffd640 1 -- 172.21.15.7:0/714451917 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6bc03f070 con 0x7fe6c0101900 2024-09-21T04:46:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.985+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe6c0101d60 con 0x7fe6c0101900 2024-09-21T04:46:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.987+0000 7fe6b6ffd640 1 -- 172.21.15.7:0/714451917 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fe6bc08a2d0 con 0x7fe6c0101900 2024-09-21T04:46:14.990 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:46:14.990 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:46:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:46:14.991 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:46:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:46:14.991 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:46:14.991 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe6ac07af90 msgr2=0x7fe6ac07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe6ac07af90 0x7fe6ac07d450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0075f80 tx=0x7fe6b000b390 comp rx=0 tx=0).stop 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0101900 msgr2=0x7fe6c00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0101900 0x7fe6c00744f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc02af60 tx=0x7fe6bc02fdd0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 shutdown_connections 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe6ac07af90 0x7fe6ac07d450 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6c0107900 0x7fe6c0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 --2- 172.21.15.7:0/714451917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c0101900 0x7fe6c00744f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 >> 172.21.15.7:0/714451917 conn(0x7fe6c00fd600 msgr2=0x7fe6c00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 shutdown_connections 2024-09-21T04:46:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:14.990+0000 7fe6c7a66640 1 -- 172.21.15.7:0/714451917 wait complete. 2024-09-21T04:46:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:14 smithi053 ceph-mon[80176]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:14 smithi053 ceph-mon[80176]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:14 smithi053 ceph-mon[80176]: pgmap v344: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:14 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/179802791' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:46:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:14 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1572134618' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:46:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:14 smithi007 ceph-mon[108238]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:14 smithi007 ceph-mon[108238]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:14 smithi007 ceph-mon[108238]: pgmap v344: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:14 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/179802791' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:46:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:14 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1572134618' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:46:16.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:15 smithi053 ceph-mon[80176]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:16.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:15 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/714451917' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:46:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:15 smithi007 ceph-mon[108238]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:15 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/714451917' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:46:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:16 smithi007 ceph-mon[108238]: pgmap v345: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:16 smithi053 ceph-mon[80176]: pgmap v345: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:19.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:18 smithi053 ceph-mon[80176]: pgmap v346: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:18 smithi007 ceph-mon[108238]: pgmap v346: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:20 smithi053 ceph-mon[80176]: pgmap v347: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:20 smithi007 ceph-mon[108238]: pgmap v347: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:23.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:23 smithi053 ceph-mon[80176]: pgmap v348: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:23 smithi007 ceph-mon[108238]: pgmap v348: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:25 smithi053 ceph-mon[80176]: pgmap v349: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:25 smithi007 ceph-mon[108238]: pgmap v349: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:27.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:27 smithi053 ceph-mon[80176]: pgmap v350: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:27 smithi007 ceph-mon[108238]: pgmap v350: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:29 smithi007 ceph-mon[108238]: pgmap v351: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:46:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:29 smithi053 ceph-mon[80176]: pgmap v351: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:46:31.364 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:31 smithi007 ceph-mon[108238]: pgmap v352: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:31 smithi053 ceph-mon[80176]: pgmap v352: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:33 smithi053 ceph-mon[80176]: pgmap v353: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:33 smithi007 ceph-mon[108238]: pgmap v353: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:35 smithi007 ceph-mon[108238]: pgmap v354: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:35 smithi053 ceph-mon[80176]: pgmap v354: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:37 smithi053 ceph-mon[80176]: pgmap v355: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:37 smithi007 ceph-mon[108238]: pgmap v355: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:39 smithi053 ceph-mon[80176]: pgmap v356: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:39 smithi007 ceph-mon[108238]: pgmap v356: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:41 smithi007 ceph-mon[108238]: pgmap v357: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:41 smithi053 ceph-mon[80176]: pgmap v357: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:43 smithi053 ceph-mon[80176]: pgmap v358: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:43 smithi007 ceph-mon[108238]: pgmap v358: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:46:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:46:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.088+0000 7f8fafb80640 1 -- 172.21.15.7:0/411410658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81007f0 msgr2=0x7f8fa8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.088+0000 7f8fafb80640 1 --2- 172.21.15.7:0/411410658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81007f0 0x7f8fa8100c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009880 tx=0x7f8f9802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.088+0000 7f8fafb80640 1 -- 172.21.15.7:0/411410658 shutdown_connections 2024-09-21T04:46:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.088+0000 7f8fafb80640 1 --2- 172.21.15.7:0/411410658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81007f0 0x7f8fa8100c50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.088+0000 7f8fafb80640 1 --2- 172.21.15.7:0/411410658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa81067f0 0x7f8fa8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.088+0000 7f8fafb80640 1 -- 172.21.15.7:0/411410658 >> 172.21.15.7:0/411410658 conn(0x7f8fa80fc550 msgr2=0x7f8fa80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.088+0000 7f8fafb80640 1 -- 172.21.15.7:0/411410658 shutdown_connections 2024-09-21T04:46:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.088+0000 7f8fafb80640 1 -- 172.21.15.7:0/411410658 wait complete. 2024-09-21T04:46:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fafb80640 1 Processor -- start 2024-09-21T04:46:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fafb80640 1 -- start start 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fafb80640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa81007f0 0x7f8fa819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fafb80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81067f0 0x7f8fa819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fafb80640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa810dca0 con 0x7f8fa81067f0 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fafb80640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa810db20 con 0x7f8fa81007f0 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fad11b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81067f0 0x7f8fa819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fad11b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81067f0 0x7f8fa819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53130/0 (socket says 172.21.15.7:53130) 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fad11b640 1 -- 172.21.15.7:0/31211565 learned_addr learned my addr 172.21.15.7:0/31211565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.089+0000 7f8fad91c640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa81007f0 0x7f8fa819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8fad11b640 1 -- 172.21.15.7:0/31211565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa81007f0 msgr2=0x7f8fa819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8fad11b640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa81007f0 0x7f8fa819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8fad11b640 1 -- 172.21.15.7:0/31211565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa8197780 con 0x7f8fa81067f0 2024-09-21T04:46:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8fad11b640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81067f0 0x7f8fa819db40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f98031460 tx=0x7f8f980045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8f9effd640 1 -- 172.21.15.7:0/31211565 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f9803f070 con 0x7f8fa81067f0 2024-09-21T04:46:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa8197a10 con 0x7f8fa81067f0 2024-09-21T04:46:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8f9effd640 1 -- 172.21.15.7:0/31211565 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f980035f0 con 0x7f8fa81067f0 2024-09-21T04:46:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8f9effd640 1 -- 172.21.15.7:0/31211565 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f98035030 con 0x7f8fa81067f0 2024-09-21T04:46:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.090+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa8197f40 con 0x7f8fa81067f0 2024-09-21T04:46:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.091+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa8101f30 con 0x7f8fa81067f0 2024-09-21T04:46:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.092+0000 7f8f9effd640 1 -- 172.21.15.7:0/31211565 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8f98042070 con 0x7f8fa81067f0 2024-09-21T04:46:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.093+0000 7f8f9effd640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8f9407af30 0x7f8f9407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.093+0000 7f8f9effd640 1 -- 172.21.15.7:0/31211565 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8f980be610 con 0x7f8fa81067f0 2024-09-21T04:46:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.093+0000 7f8fad91c640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8f9407af30 0x7f8f9407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.094+0000 7f8fad91c640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8f9407af30 0x7f8f9407d3f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4005d00 tx=0x7f8fa400b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.098+0000 7f8f9effd640 1 -- 172.21.15.7:0/31211565 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f98089bb0 con 0x7f8fa81067f0 2024-09-21T04:46:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.303+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8fa81981f0 con 0x7f8f9407af30 2024-09-21T04:46:45.306 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:45 smithi007 ceph-mon[108238]: pgmap v359: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:45.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.304+0000 7f8f9effd640 1 -- 172.21.15.7:0/31211565 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f8fa81981f0 con 0x7f8f9407af30 2024-09-21T04:46:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8f9407af30 msgr2=0x7f8f9407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8f9407af30 0x7f8f9407d3f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4005d00 tx=0x7f8fa400b410 comp rx=0 tx=0).stop 2024-09-21T04:46:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81067f0 msgr2=0x7f8fa819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81067f0 0x7f8fa819db40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f98031460 tx=0x7f8f980045e0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 shutdown_connections 2024-09-21T04:46:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8f9407af30 0x7f8f9407d3f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fa81067f0 0x7f8fa819db40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 --2- 172.21.15.7:0/31211565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa81007f0 0x7f8fa819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 >> 172.21.15.7:0/31211565 conn(0x7f8fa80fc550 msgr2=0x7f8fa8069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 shutdown_connections 2024-09-21T04:46:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.306+0000 7f8fafb80640 1 -- 172.21.15.7:0/31211565 wait complete. 2024-09-21T04:46:45.318 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.392+0000 7f659e21b640 1 -- 172.21.15.7:0/968556570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 msgr2=0x7f6598102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.392+0000 7f659e21b640 1 --2- 172.21.15.7:0/968556570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 0x7f6598102f20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6588009880 tx=0x7f658802a530 comp rx=0 tx=0).stop 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.392+0000 7f659e21b640 1 -- 172.21.15.7:0/968556570 shutdown_connections 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.392+0000 7f659e21b640 1 --2- 172.21.15.7:0/968556570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6598103460 0x7f659810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.392+0000 7f659e21b640 1 --2- 172.21.15.7:0/968556570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 0x7f6598102f20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.392+0000 7f659e21b640 1 -- 172.21.15.7:0/968556570 >> 172.21.15.7:0/968556570 conn(0x7f65980fc520 msgr2=0x7f65980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.392+0000 7f659e21b640 1 -- 172.21.15.7:0/968556570 shutdown_connections 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.393+0000 7f659e21b640 1 -- 172.21.15.7:0/968556570 wait complete. 2024-09-21T04:46:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.393+0000 7f659e21b640 1 Processor -- start 2024-09-21T04:46:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.393+0000 7f659e21b640 1 -- start start 2024-09-21T04:46:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f659e21b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 0x7f659819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f659e21b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6598103460 0x7f659819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f659e21b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f659810edd0 con 0x7f6598102b40 2024-09-21T04:46:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f659e21b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f659810ec50 con 0x7f6598103460 2024-09-21T04:46:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f65977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 0x7f659819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f65977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 0x7f659819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53146/0 (socket says 172.21.15.7:53146) 2024-09-21T04:46:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f65977fe640 1 -- 172.21.15.7:0/3351130502 learned_addr learned my addr 172.21.15.7:0/3351130502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f6596ffd640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6598103460 0x7f659819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f65977fe640 1 -- 172.21.15.7:0/3351130502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6598103460 msgr2=0x7f659819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f65977fe640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6598103460 0x7f659819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f65977fe640 1 -- 172.21.15.7:0/3351130502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65981977c0 con 0x7f6598102b40 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f65977fe640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 0x7f659819d640 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6588004f90 tx=0x7f65880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.394+0000 7f6594ff9640 1 -- 172.21.15.7:0/3351130502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f658803f070 con 0x7f6598102b40 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.395+0000 7f6594ff9640 1 -- 172.21.15.7:0/3351130502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6588004c70 con 0x7f6598102b40 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.395+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6598197a50 con 0x7f6598102b40 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.395+0000 7f6594ff9640 1 -- 172.21.15.7:0/3351130502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6588035030 con 0x7f6598102b40 2024-09-21T04:46:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.395+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6598197f80 con 0x7f6598102b40 2024-09-21T04:46:45.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.396+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659806aa10 con 0x7f6598102b40 2024-09-21T04:46:45.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.396+0000 7f6594ff9640 1 -- 172.21.15.7:0/3351130502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6588042070 con 0x7f6598102b40 2024-09-21T04:46:45.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.397+0000 7f6594ff9640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f658007af80 0x7f658007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.397+0000 7f6594ff9640 1 -- 172.21.15.7:0/3351130502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f65880be8f0 con 0x7f6598102b40 2024-09-21T04:46:45.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.397+0000 7f6596ffd640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f658007af80 0x7f658007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.397+0000 7f6596ffd640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f658007af80 0x7f658007d440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6598108910 tx=0x7f658c00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:45.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.401+0000 7f6594ff9640 1 -- 172.21.15.7:0/3351130502 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6588089f10 con 0x7f6598102b40 2024-09-21T04:46:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:45 smithi053 ceph-mon[80176]: pgmap v359: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:45.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.608+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f65981982f0 con 0x7f658007af80 2024-09-21T04:46:45.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.609+0000 7f6594ff9640 1 -- 172.21.15.7:0/3351130502 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f65981982f0 con 0x7f658007af80 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f658007af80 msgr2=0x7f658007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f658007af80 0x7f658007d440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6598108910 tx=0x7f658c00b390 comp rx=0 tx=0).stop 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 msgr2=0x7f659819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 0x7f659819d640 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6588004f90 tx=0x7f65880048f0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 shutdown_connections 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f658007af80 0x7f658007d440 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6598103460 0x7f659819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 --2- 172.21.15.7:0/3351130502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598102b40 0x7f659819d640 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 >> 172.21.15.7:0/3351130502 conn(0x7f65980fc520 msgr2=0x7f6598108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 shutdown_connections 2024-09-21T04:46:45.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.611+0000 7f659e21b640 1 -- 172.21.15.7:0/3351130502 wait complete. 2024-09-21T04:46:45.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.696+0000 7ff6262e6640 1 -- 172.21.15.7:0/141307026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620101910 msgr2=0x7ff620101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.696+0000 7ff6262e6640 1 --2- 172.21.15.7:0/141307026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620101910 0x7ff620101d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff614009880 tx=0x7ff61402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.696+0000 7ff6262e6640 1 -- 172.21.15.7:0/141307026 shutdown_connections 2024-09-21T04:46:45.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.696+0000 7ff6262e6640 1 --2- 172.21.15.7:0/141307026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620101910 0x7ff620101d70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.696+0000 7ff6262e6640 1 --2- 172.21.15.7:0/141307026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff620107910 0x7ff620107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.696+0000 7ff6262e6640 1 -- 172.21.15.7:0/141307026 >> 172.21.15.7:0/141307026 conn(0x7ff6200fd630 msgr2=0x7ff6200ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:45.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.696+0000 7ff6262e6640 1 -- 172.21.15.7:0/141307026 shutdown_connections 2024-09-21T04:46:45.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.696+0000 7ff6262e6640 1 -- 172.21.15.7:0/141307026 wait complete. 2024-09-21T04:46:45.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff6262e6640 1 Processor -- start 2024-09-21T04:46:45.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff6262e6640 1 -- start start 2024-09-21T04:46:45.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff6262e6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff620101910 0x7ff62019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff6262e6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620107910 0x7ff62019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff6262e6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff62010ede0 con 0x7ff620107910 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff6262e6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff62010ec60 con 0x7ff620101910 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620107910 0x7ff62019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620107910 0x7ff62019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53176/0 (socket says 172.21.15.7:53176) 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff61f7fe640 1 -- 172.21.15.7:0/2380923796 learned_addr learned my addr 172.21.15.7:0/2380923796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff61ffff640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff620101910 0x7ff62019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff61f7fe640 1 -- 172.21.15.7:0/2380923796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff620101910 msgr2=0x7ff62019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff61f7fe640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff620101910 0x7ff62019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.697+0000 7ff61f7fe640 1 -- 172.21.15.7:0/2380923796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6201977b0 con 0x7ff620107910 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.698+0000 7ff61f7fe640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620107910 0x7ff62019db70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff614031460 tx=0x7ff614004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.698+0000 7ff61d7fa640 1 -- 172.21.15.7:0/2380923796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff61403f070 con 0x7ff620107910 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.698+0000 7ff61d7fa640 1 -- 172.21.15.7:0/2380923796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff6140035f0 con 0x7ff620107910 2024-09-21T04:46:45.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.698+0000 7ff61d7fa640 1 -- 172.21.15.7:0/2380923796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff614035030 con 0x7ff620107910 2024-09-21T04:46:45.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.698+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff620197a40 con 0x7ff620107910 2024-09-21T04:46:45.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.698+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff620197f70 con 0x7ff620107910 2024-09-21T04:46:45.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.699+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff620103050 con 0x7ff620107910 2024-09-21T04:46:45.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.700+0000 7ff61d7fa640 1 -- 172.21.15.7:0/2380923796 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff614042070 con 0x7ff620107910 2024-09-21T04:46:45.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.700+0000 7ff61d7fa640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff60807af80 0x7ff60807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:45.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.700+0000 7ff61d7fa640 1 -- 172.21.15.7:0/2380923796 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff6140be930 con 0x7ff620107910 2024-09-21T04:46:45.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.700+0000 7ff61ffff640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff60807af80 0x7ff60807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:45.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.701+0000 7ff61ffff640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff60807af80 0x7ff60807d440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff610009bd0 tx=0x7ff610005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:45.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.705+0000 7ff61d7fa640 1 -- 172.21.15.7:0/2380923796 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff614089eb0 con 0x7ff620107910 2024-09-21T04:46:45.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.901+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff6201982e0 con 0x7ff60807af80 2024-09-21T04:46:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.911+0000 7ff61d7fa640 1 -- 172.21.15.7:0/2380923796 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7ff6201982e0 con 0x7ff60807af80 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (8m) 2m ago 19m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (11m) 2m ago 19m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (11m) 2m ago 19m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (7m) 2m ago 19m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (6m) 2m ago 19m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (8m) 2m ago 19m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (2m) 2m ago 14m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (3m) 2m ago 14m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (14m) 2m ago 14m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (2m) 2m ago 14m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (11m) 2m ago 21m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (10m) 2m ago 18m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:46:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (7m) 2m ago 21m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (7m) 2m ago 18m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (9m) 2m ago 19m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (9m) 2m ago 18m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (6m) 2m ago 17m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (5m) 2m ago 17m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (5m) 2m ago 16m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (4m) 2m ago 16m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (4m) 2m ago 15m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (3m) 2m ago 15m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:46:45.915 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (9m) 2m ago 19m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:46:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff60807af80 msgr2=0x7ff60807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff60807af80 0x7ff60807d440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff610009bd0 tx=0x7ff610005f90 comp rx=0 tx=0).stop 2024-09-21T04:46:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620107910 msgr2=0x7ff62019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620107910 0x7ff62019db70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff614031460 tx=0x7ff614004530 comp rx=0 tx=0).stop 2024-09-21T04:46:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 shutdown_connections 2024-09-21T04:46:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff60807af80 0x7ff60807d440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff620107910 0x7ff62019db70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 --2- 172.21.15.7:0/2380923796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff620101910 0x7ff62019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 >> 172.21.15.7:0/2380923796 conn(0x7ff6200fd630 msgr2=0x7ff6200fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 shutdown_connections 2024-09-21T04:46:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.913+0000 7ff6262e6640 1 -- 172.21.15.7:0/2380923796 wait complete. 2024-09-21T04:46:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 -- 172.21.15.7:0/548458283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 msgr2=0x7fb430100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 --2- 172.21.15.7:0/548458283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 0x7fb430100c60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb424009880 tx=0x7fb42402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 -- 172.21.15.7:0/548458283 shutdown_connections 2024-09-21T04:46:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 --2- 172.21.15.7:0/548458283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 0x7fb430100c60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 --2- 172.21.15.7:0/548458283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430106800 0x7fb430106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 -- 172.21.15.7:0/548458283 >> 172.21.15.7:0/548458283 conn(0x7fb4300fc520 msgr2=0x7fb4300fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:46.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 -- 172.21.15.7:0/548458283 shutdown_connections 2024-09-21T04:46:46.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 -- 172.21.15.7:0/548458283 wait complete. 2024-09-21T04:46:46.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.996+0000 7fb4370d6640 1 Processor -- start 2024-09-21T04:46:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb4370d6640 1 -- start start 2024-09-21T04:46:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb4370d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 0x7fb4300fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb4370d6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430106800 0x7fb4300fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb4370d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb43010dcd0 con 0x7fb430100800 2024-09-21T04:46:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb4370d6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb43010db50 con 0x7fb430106800 2024-09-21T04:46:46.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb434e72640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 0x7fb4300fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb434e72640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 0x7fb4300fcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53182/0 (socket says 172.21.15.7:53182) 2024-09-21T04:46:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb434e72640 1 -- 172.21.15.7:0/438510190 learned_addr learned my addr 172.21.15.7:0/438510190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb42ffff640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430106800 0x7fb4300fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb434e72640 1 -- 172.21.15.7:0/438510190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430106800 msgr2=0x7fb4300fd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb434e72640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430106800 0x7fb4300fd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb434e72640 1 -- 172.21.15.7:0/438510190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4300fd6f0 con 0x7fb430100800 2024-09-21T04:46:46.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.997+0000 7fb42ffff640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430106800 0x7fb4300fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:46:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.998+0000 7fb434e72640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 0x7fb4300fcc70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb42000c8f0 tx=0x7fb42000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.998+0000 7fb42dffb640 1 -- 172.21.15.7:0/438510190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4200079b0 con 0x7fb430100800 2024-09-21T04:46:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.998+0000 7fb42dffb640 1 -- 172.21.15.7:0/438510190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4200054e0 con 0x7fb430100800 2024-09-21T04:46:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.998+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4300fd9e0 con 0x7fb430100800 2024-09-21T04:46:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.998+0000 7fb42dffb640 1 -- 172.21.15.7:0/438510190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb420004dd0 con 0x7fb430100800 2024-09-21T04:46:46.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.998+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4301a3e70 con 0x7fb430100800 2024-09-21T04:46:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:45.999+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb430101f40 con 0x7fb430100800 2024-09-21T04:46:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.000+0000 7fb42dffb640 1 -- 172.21.15.7:0/438510190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb420005020 con 0x7fb430100800 2024-09-21T04:46:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.000+0000 7fb42dffb640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb41807aed0 0x7fb41807d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.000+0000 7fb42dffb640 1 -- 172.21.15.7:0/438510190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb42009e020 con 0x7fb430100800 2024-09-21T04:46:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.000+0000 7fb42ffff640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb41807aed0 0x7fb41807d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.001+0000 7fb42ffff640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb41807aed0 0x7fb41807d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb424006010 tx=0x7fb4240095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:46.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.005+0000 7fb42dffb640 1 -- 172.21.15.7:0/438510190 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb420014030 con 0x7fb430100800 2024-09-21T04:46:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.266+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fb430109d80 con 0x7fb430100800 2024-09-21T04:46:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.267+0000 7fb42dffb640 1 -- 172.21.15.7:0/438510190 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fb420007de0 con 0x7fb430100800 2024-09-21T04:46:46.270 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:46:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:46:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:46:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:46:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:46:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:46:46.271 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:46:46.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb41807aed0 msgr2=0x7fb41807d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb41807aed0 0x7fb41807d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb424006010 tx=0x7fb4240095f0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 msgr2=0x7fb4300fcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 0x7fb4300fcc70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb42000c8f0 tx=0x7fb42000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 shutdown_connections 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb41807aed0 0x7fb41807d390 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430106800 0x7fb4300fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 --2- 172.21.15.7:0/438510190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb430100800 0x7fb4300fcc70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 >> 172.21.15.7:0/438510190 conn(0x7fb4300fc520 msgr2=0x7fb43006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 shutdown_connections 2024-09-21T04:46:46.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.270+0000 7fb4370d6640 1 -- 172.21.15.7:0/438510190 wait complete. 2024-09-21T04:46:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.354+0000 7fd006ae0640 1 -- 172.21.15.7:0/2408879081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 msgr2=0x7fd000107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.354+0000 7fd006ae0640 1 --2- 172.21.15.7:0/2408879081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 0x7fd000107ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009880 tx=0x7fcffc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.354+0000 7fd006ae0640 1 -- 172.21.15.7:0/2408879081 shutdown_connections 2024-09-21T04:46:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.354+0000 7fd006ae0640 1 --2- 172.21.15.7:0/2408879081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd000101900 0x7fd000101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.354+0000 7fd006ae0640 1 --2- 172.21.15.7:0/2408879081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 0x7fd000107ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.354+0000 7fd006ae0640 1 -- 172.21.15.7:0/2408879081 >> 172.21.15.7:0/2408879081 conn(0x7fd0000fd600 msgr2=0x7fd0000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.354+0000 7fd006ae0640 1 -- 172.21.15.7:0/2408879081 shutdown_connections 2024-09-21T04:46:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.354+0000 7fd006ae0640 1 -- 172.21.15.7:0/2408879081 wait complete. 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fd006ae0640 1 Processor -- start 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fd006ae0640 1 -- start start 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fd006ae0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd000101900 0x7fd0000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fd006ae0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 0x7fd000074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fd006ae0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd00010edd0 con 0x7fd000107900 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fd006ae0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd00010ec50 con 0x7fd000101900 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fcff7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 0x7fd000074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fcff7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 0x7fd000074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53192/0 (socket says 172.21.15.7:53192) 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fcff7fff640 1 -- 172.21.15.7:0/3900307046 learned_addr learned my addr 172.21.15.7:0/3900307046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.355+0000 7fd00487c640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd000101900 0x7fd0000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fcff7fff640 1 -- 172.21.15.7:0/3900307046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd000101900 msgr2=0x7fd0000744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fcff7fff640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd000101900 0x7fd0000744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fcff7fff640 1 -- 172.21.15.7:0/3900307046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd000074f70 con 0x7fd000107900 2024-09-21T04:46:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fcff7fff640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 0x7fd000074a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcff000e970 tx=0x7fcff000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fcff5ffb640 1 -- 172.21.15.7:0/3900307046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcff000cac0 con 0x7fd000107900 2024-09-21T04:46:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fcff5ffb640 1 -- 172.21.15.7:0/3900307046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcff00054e0 con 0x7fd000107900 2024-09-21T04:46:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd000075260 con 0x7fd000107900 2024-09-21T04:46:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fcff5ffb640 1 -- 172.21.15.7:0/3900307046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcff0010430 con 0x7fd000107900 2024-09-21T04:46:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.356+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0001a3e20 con 0x7fd000107900 2024-09-21T04:46:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.357+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd000103040 con 0x7fd000107900 2024-09-21T04:46:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.358+0000 7fcff5ffb640 1 -- 172.21.15.7:0/3900307046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fcff0004f00 con 0x7fd000107900 2024-09-21T04:46:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.358+0000 7fcff5ffb640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfe807af30 0x7fcfe807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.358+0000 7fcff5ffb640 1 -- 172.21.15.7:0/3900307046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcff009e7e0 con 0x7fd000107900 2024-09-21T04:46:46.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.358+0000 7fd00487c640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfe807af30 0x7fcfe807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.359+0000 7fd00487c640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfe807af30 0x7fcfe807d3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0096c0 tx=0x7fcffc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.363+0000 7fcff5ffb640 1 -- 172.21.15.7:0/3900307046 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcff0069ce0 con 0x7fd000107900 2024-09-21T04:46:46.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.587+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd000109e70 con 0x7fd000107900 2024-09-21T04:46:46.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.588+0000 7fcff5ffb640 1 -- 172.21.15.7:0/3900307046 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7fcff006eb90 con 0x7fd000107900 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:46:46.591 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:46:46.592 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:46:46.593 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfe807af30 msgr2=0x7fcfe807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfe807af30 0x7fcfe807d3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0096c0 tx=0x7fcffc031400 comp rx=0 tx=0).stop 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 msgr2=0x7fd000074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 0x7fd000074a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcff000e970 tx=0x7fcff000ed70 comp rx=0 tx=0).stop 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 shutdown_connections 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfe807af30 0x7fcfe807d3f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd000107900 0x7fd000074a30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 --2- 172.21.15.7:0/3900307046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd000101900 0x7fd0000744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 >> 172.21.15.7:0/3900307046 conn(0x7fd0000fd600 msgr2=0x7fd0000fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:46.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 shutdown_connections 2024-09-21T04:46:46.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.590+0000 7fd006ae0640 1 -- 172.21.15.7:0/3900307046 wait complete. 2024-09-21T04:46:46.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 -- 172.21.15.7:0/1076178199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8103430 msgr2=0x7f94c810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 --2- 172.21.15.7:0/1076178199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8103430 0x7f94c810b920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0098b0 tx=0x7f94bc02a560 comp rx=0 tx=0).stop 2024-09-21T04:46:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 -- 172.21.15.7:0/1076178199 shutdown_connections 2024-09-21T04:46:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 --2- 172.21.15.7:0/1076178199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8103430 0x7f94c810b920 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 --2- 172.21.15.7:0/1076178199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c8102b10 0x7f94c8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 -- 172.21.15.7:0/1076178199 >> 172.21.15.7:0/1076178199 conn(0x7f94c80fc550 msgr2=0x7f94c80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 -- 172.21.15.7:0/1076178199 shutdown_connections 2024-09-21T04:46:46.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 -- 172.21.15.7:0/1076178199 wait complete. 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.679+0000 7f94cde7f640 1 Processor -- start 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94cde7f640 1 -- start start 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94cde7f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8102b10 0x7f94c8100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94cde7f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c8103430 0x7f94c8100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94cde7f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810edc0 con 0x7f94c8102b10 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94cde7f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f94c810ec40 con 0x7f94c8103430 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94c77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8102b10 0x7f94c8100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94c77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8102b10 0x7f94c8100770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53208/0 (socket says 172.21.15.7:53208) 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94c77fe640 1 -- 172.21.15.7:0/2183215575 learned_addr learned my addr 172.21.15.7:0/2183215575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:46.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94c6ffd640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c8103430 0x7f94c8100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94c77fe640 1 -- 172.21.15.7:0/2183215575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c8103430 msgr2=0x7f94c8100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94c77fe640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c8103430 0x7f94c8100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.680+0000 7f94c77fe640 1 -- 172.21.15.7:0/2183215575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c81011f0 con 0x7f94c8102b10 2024-09-21T04:46:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.681+0000 7f94c77fe640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8102b10 0x7f94c8100770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f94b800cc60 tx=0x7f94b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.681+0000 7f94c4ff9640 1 -- 172.21.15.7:0/2183215575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f94b800ad30 con 0x7f94c8102b10 2024-09-21T04:46:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.681+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c81014e0 con 0x7f94c8102b10 2024-09-21T04:46:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.681+0000 7f94c4ff9640 1 -- 172.21.15.7:0/2183215575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94b8005020 con 0x7f94c8102b10 2024-09-21T04:46:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.681+0000 7f94c4ff9640 1 -- 172.21.15.7:0/2183215575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f94b800f660 con 0x7f94c8102b10 2024-09-21T04:46:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.681+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81a3e70 con 0x7f94c8102b10 2024-09-21T04:46:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.682+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c806a520 con 0x7f94c8102b10 2024-09-21T04:46:46.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.682+0000 7f94c4ff9640 1 -- 172.21.15.7:0/2183215575 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f94b80051d0 con 0x7f94c8102b10 2024-09-21T04:46:46.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.683+0000 7f94c4ff9640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f94b007af80 0x7f94b007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.683+0000 7f94c4ff9640 1 -- 172.21.15.7:0/2183215575 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f94b809e570 con 0x7f94c8102b10 2024-09-21T04:46:46.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.683+0000 7f94c6ffd640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f94b007af80 0x7f94b007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.684+0000 7f94c6ffd640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f94b007af80 0x7f94b007d440 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94bc000c00 tx=0x7f94bc031420 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.687+0000 7f94c4ff9640 1 -- 172.21.15.7:0/2183215575 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94b8014030 con 0x7f94c8102b10 2024-09-21T04:46:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.894+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f94c8106010 con 0x7f94b007af80 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.895+0000 7f94c4ff9640 1 -- 172.21.15.7:0/2183215575 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f94bc0042c0 con 0x7f94b007af80 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:46:46.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:46:46.899 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:46:46.899 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:46:46.899 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:46:46.899 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:46:46.899 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:46:46.899 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:46:46.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f94b007af80 msgr2=0x7f94b007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f94b007af80 0x7f94b007d440 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94bc000c00 tx=0x7f94bc031420 comp rx=0 tx=0).stop 2024-09-21T04:46:46.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8102b10 msgr2=0x7f94c8100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8102b10 0x7f94c8100770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f94b800cc60 tx=0x7f94b800a620 comp rx=0 tx=0).stop 2024-09-21T04:46:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 shutdown_connections 2024-09-21T04:46:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f94b007af80 0x7f94b007d440 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94c8103430 0x7f94c8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 --2- 172.21.15.7:0/2183215575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c8102b10 0x7f94c8100770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 >> 172.21.15.7:0/2183215575 conn(0x7f94c80fc550 msgr2=0x7f94c8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 shutdown_connections 2024-09-21T04:46:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.897+0000 7f94cde7f640 1 -- 172.21.15.7:0/2183215575 wait complete. 2024-09-21T04:46:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.983+0000 7f8899a5f640 1 -- 172.21.15.7:0/2425611381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 msgr2=0x7f8894107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.983+0000 7f8899a5f640 1 --2- 172.21.15.7:0/2425611381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 0x7f8894107cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8884009880 tx=0x7f888402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.983+0000 7f8899a5f640 1 -- 172.21.15.7:0/2425611381 shutdown_connections 2024-09-21T04:46:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.983+0000 7f8899a5f640 1 --2- 172.21.15.7:0/2425611381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894101910 0x7f8894101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.983+0000 7f8899a5f640 1 --2- 172.21.15.7:0/2425611381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 0x7f8894107cf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.983+0000 7f8899a5f640 1 -- 172.21.15.7:0/2425611381 >> 172.21.15.7:0/2425611381 conn(0x7f88940fd630 msgr2=0x7f88940ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.983+0000 7f8899a5f640 1 -- 172.21.15.7:0/2425611381 shutdown_connections 2024-09-21T04:46:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.983+0000 7f8899a5f640 1 -- 172.21.15.7:0/2425611381 wait complete. 2024-09-21T04:46:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f8899a5f640 1 Processor -- start 2024-09-21T04:46:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f8899a5f640 1 -- start start 2024-09-21T04:46:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f8899a5f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894101910 0x7f889419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f8899a5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 0x7f889419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f8899a5f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f889410edc0 con 0x7f8894107910 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f8899a5f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f889410ec40 con 0x7f8894101910 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f888bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 0x7f889419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f888bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 0x7f889419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53228/0 (socket says 172.21.15.7:53228) 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f888bfff640 1 -- 172.21.15.7:0/1608005605 learned_addr learned my addr 172.21.15.7:0/1608005605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f8892ffd640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894101910 0x7f889419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f888bfff640 1 -- 172.21.15.7:0/1608005605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894101910 msgr2=0x7f889419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f888bfff640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894101910 0x7f889419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.984+0000 7f888bfff640 1 -- 172.21.15.7:0/1608005605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8894197810 con 0x7f8894107910 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.985+0000 7f888bfff640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 0x7f889419dbd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c8a0 tx=0x7f887c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.985+0000 7f8898a5d640 1 -- 172.21.15.7:0/1608005605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f887c0079b0 con 0x7f8894107910 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.985+0000 7f8898a5d640 1 -- 172.21.15.7:0/1608005605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f887c007b50 con 0x7f8894107910 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.985+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8894197b00 con 0x7f8894107910 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.985+0000 7f8898a5d640 1 -- 172.21.15.7:0/1608005605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f887c018700 con 0x7f8894107910 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.985+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8894198090 con 0x7f8894107910 2024-09-21T04:46:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.986+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8894103050 con 0x7f8894107910 2024-09-21T04:46:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.987+0000 7f8898a5d640 1 -- 172.21.15.7:0/1608005605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f887c007d00 con 0x7f8894107910 2024-09-21T04:46:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.987+0000 7f8898a5d640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888007af30 0x7f888007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:46:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.987+0000 7f8898a5d640 1 -- 172.21.15.7:0/1608005605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f887c09e280 con 0x7f8894107910 2024-09-21T04:46:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.987+0000 7f8892ffd640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888007af30 0x7f888007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:46:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.988+0000 7f8892ffd640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888007af30 0x7f888007d3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:46:46.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:46.992+0000 7f8898a5d640 1 -- 172.21.15.7:0/1608005605 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f887c014030 con 0x7f8894107910 2024-09-21T04:46:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:47 smithi007 ceph-mon[108238]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:47 smithi007 ceph-mon[108238]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:47 smithi007 ceph-mon[108238]: pgmap v360: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:47 smithi007 ceph-mon[108238]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:47 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/438510190' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:46:47.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:47 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3900307046' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:46:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.255+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f8894198d40 con 0x7f8894107910 2024-09-21T04:46:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.255+0000 7f8898a5d640 1 -- 172.21.15.7:0/1608005605 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f887c069820 con 0x7f8894107910 2024-09-21T04:46:47.258 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:46:47.258 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:46:47.258 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:46:47.258 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:46:47.259 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:46:47.259 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:46:47.259 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:46:47.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888007af30 msgr2=0x7f888007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:47.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888007af30 0x7f888007d3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884031400 comp rx=0 tx=0).stop 2024-09-21T04:46:47.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 msgr2=0x7f889419dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:46:47.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 0x7f889419dbd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c8a0 tx=0x7f887c00cd60 comp rx=0 tx=0).stop 2024-09-21T04:46:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 shutdown_connections 2024-09-21T04:46:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888007af30 0x7f888007d3f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8894107910 0x7f889419dbd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 --2- 172.21.15.7:0/1608005605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8894101910 0x7f889419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:46:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 >> 172.21.15.7:0/1608005605 conn(0x7f88940fd630 msgr2=0x7f88940ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:46:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 shutdown_connections 2024-09-21T04:46:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:46:47.257+0000 7f8899a5f640 1 -- 172.21.15.7:0/1608005605 wait complete. 2024-09-21T04:46:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:47 smithi053 ceph-mon[80176]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:47 smithi053 ceph-mon[80176]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:47 smithi053 ceph-mon[80176]: pgmap v360: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:47 smithi053 ceph-mon[80176]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:47 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/438510190' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:46:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:47 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3900307046' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:46:48.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:48 smithi053 ceph-mon[80176]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:48.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:48 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1608005605' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:46:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:48 smithi007 ceph-mon[108238]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:46:48.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:48 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1608005605' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:46:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:49 smithi053 ceph-mon[80176]: pgmap v361: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:49 smithi007 ceph-mon[108238]: pgmap v361: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:51.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:51 smithi053 ceph-mon[80176]: pgmap v362: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:51 smithi007 ceph-mon[108238]: pgmap v362: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:53 smithi007 ceph-mon[108238]: pgmap v363: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:53.359 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:53 smithi053 ceph-mon[80176]: pgmap v363: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:55 smithi053 ceph-mon[80176]: pgmap v364: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:55 smithi007 ceph-mon[108238]: pgmap v364: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:57 smithi053 ceph-mon[80176]: pgmap v365: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:57 smithi007 ceph-mon[108238]: pgmap v365: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:59 smithi007 ceph-mon[108238]: pgmap v366: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:46:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:46:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:59 smithi053 ceph-mon[80176]: pgmap v366: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:46:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:46:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:47:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:01 smithi053 ceph-mon[80176]: pgmap v367: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:01 smithi007 ceph-mon[108238]: pgmap v367: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:03 smithi053 ceph-mon[80176]: pgmap v368: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:03 smithi007 ceph-mon[108238]: pgmap v368: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:05 smithi007 ceph-mon[108238]: pgmap v369: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:05 smithi053 ceph-mon[80176]: pgmap v369: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:07 smithi007 ceph-mon[108238]: pgmap v370: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:07 smithi053 ceph-mon[80176]: pgmap v370: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:09 smithi053 ceph-mon[80176]: pgmap v371: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:09 smithi007 ceph-mon[108238]: pgmap v371: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:11 smithi007 ceph-mon[108238]: pgmap v372: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:11 smithi053 ceph-mon[80176]: pgmap v372: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:13.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:13 smithi053 ceph-mon[80176]: pgmap v373: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:13 smithi007 ceph-mon[108238]: pgmap v373: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:47:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:47:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:15 smithi053 ceph-mon[80176]: pgmap v374: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:15 smithi007 ceph-mon[108238]: pgmap v374: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:17 smithi007 ceph-mon[108238]: pgmap v375: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.352+0000 7f7b3e058640 1 -- 172.21.15.7:0/1908933186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38107900 msgr2=0x7f7b38107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.352+0000 7f7b3e058640 1 --2- 172.21.15.7:0/1908933186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38107900 0x7f7b38107ce0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009880 tx=0x7f7b2802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.352+0000 7f7b3e058640 1 -- 172.21.15.7:0/1908933186 shutdown_connections 2024-09-21T04:47:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.352+0000 7f7b3e058640 1 --2- 172.21.15.7:0/1908933186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b38101900 0x7f7b38101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.352+0000 7f7b3e058640 1 --2- 172.21.15.7:0/1908933186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38107900 0x7f7b38107ce0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.352+0000 7f7b3e058640 1 -- 172.21.15.7:0/1908933186 >> 172.21.15.7:0/1908933186 conn(0x7f7b380fd600 msgr2=0x7f7b380ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.352+0000 7f7b3e058640 1 -- 172.21.15.7:0/1908933186 shutdown_connections 2024-09-21T04:47:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.352+0000 7f7b3e058640 1 -- 172.21.15.7:0/1908933186 wait complete. 2024-09-21T04:47:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b3e058640 1 Processor -- start 2024-09-21T04:47:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b3e058640 1 -- start start 2024-09-21T04:47:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b3e058640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38101900 0x7f7b3819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b3e058640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b38107900 0x7f7b3819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b3e058640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3810edd0 con 0x7f7b38101900 2024-09-21T04:47:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b3e058640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3810ec50 con 0x7f7b38107900 2024-09-21T04:47:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38101900 0x7f7b3819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38101900 0x7f7b3819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42564/0 (socket says 172.21.15.7:42564) 2024-09-21T04:47:17.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b377fe640 1 -- 172.21.15.7:0/1122466856 learned_addr learned my addr 172.21.15.7:0/1122466856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.353+0000 7f7b2effd640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b38107900 0x7f7b3819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b377fe640 1 -- 172.21.15.7:0/1122466856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b38107900 msgr2=0x7f7b3819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b377fe640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b38107900 0x7f7b3819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b377fe640 1 -- 172.21.15.7:0/1122466856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b381977c0 con 0x7f7b38101900 2024-09-21T04:47:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b377fe640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38101900 0x7f7b3819d640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b28031460 tx=0x7f7b280045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b357fa640 1 -- 172.21.15.7:0/1122466856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7b2803f070 con 0x7f7b38101900 2024-09-21T04:47:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b38197a50 con 0x7f7b38101900 2024-09-21T04:47:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b357fa640 1 -- 172.21.15.7:0/1122466856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7b280035f0 con 0x7f7b38101900 2024-09-21T04:47:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b357fa640 1 -- 172.21.15.7:0/1122466856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7b28035030 con 0x7f7b38101900 2024-09-21T04:47:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.354+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b38197f80 con 0x7f7b38101900 2024-09-21T04:47:17.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.355+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b38103040 con 0x7f7b38101900 2024-09-21T04:47:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.356+0000 7f7b357fa640 1 -- 172.21.15.7:0/1122466856 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7b28042070 con 0x7f7b38101900 2024-09-21T04:47:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.357+0000 7f7b357fa640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7b2407af30 0x7f7b2407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.357+0000 7f7b357fa640 1 -- 172.21.15.7:0/1122466856 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7b280be9d0 con 0x7f7b38101900 2024-09-21T04:47:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.357+0000 7f7b2effd640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7b2407af30 0x7f7b2407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.358+0000 7f7b2effd640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7b2407af30 0x7f7b2407d3f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7b380ffbf0 tx=0x7f7b2000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:17.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.362+0000 7f7b357fa640 1 -- 172.21.15.7:0/1122466856 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b28089f70 con 0x7f7b38101900 2024-09-21T04:47:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:17 smithi053 ceph-mon[80176]: pgmap v375: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.566+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7b38198230 con 0x7f7b2407af30 2024-09-21T04:47:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.567+0000 7f7b357fa640 1 -- 172.21.15.7:0/1122466856 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f7b38198230 con 0x7f7b2407af30 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7b2407af30 msgr2=0x7f7b2407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7b2407af30 0x7f7b2407d3f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7b380ffbf0 tx=0x7f7b2000b390 comp rx=0 tx=0).stop 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38101900 msgr2=0x7f7b3819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38101900 0x7f7b3819d640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7b28031460 tx=0x7f7b280045e0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 shutdown_connections 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7b2407af30 0x7f7b2407d3f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b38107900 0x7f7b3819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 --2- 172.21.15.7:0/1122466856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38101900 0x7f7b3819d640 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 >> 172.21.15.7:0/1122466856 conn(0x7f7b380fd600 msgr2=0x7f7b380fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 shutdown_connections 2024-09-21T04:47:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.569+0000 7f7b3e058640 1 -- 172.21.15.7:0/1122466856 wait complete. 2024-09-21T04:47:17.581 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.654+0000 7f0f288f5640 1 -- 172.21.15.7:0/779813062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 msgr2=0x7f0f24102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.654+0000 7f0f288f5640 1 --2- 172.21.15.7:0/779813062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 0x7f0f24102ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009c50 tx=0x7f0f1402aa30 comp rx=0 tx=0).stop 2024-09-21T04:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.655+0000 7f0f288f5640 1 -- 172.21.15.7:0/779813062 shutdown_connections 2024-09-21T04:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.655+0000 7f0f288f5640 1 --2- 172.21.15.7:0/779813062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f24103430 0x7f0f2410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.655+0000 7f0f288f5640 1 --2- 172.21.15.7:0/779813062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 0x7f0f24102ef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.655+0000 7f0f288f5640 1 -- 172.21.15.7:0/779813062 >> 172.21.15.7:0/779813062 conn(0x7f0f240fc550 msgr2=0x7f0f240fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.655+0000 7f0f288f5640 1 -- 172.21.15.7:0/779813062 shutdown_connections 2024-09-21T04:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.655+0000 7f0f288f5640 1 -- 172.21.15.7:0/779813062 wait complete. 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f288f5640 1 Processor -- start 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f288f5640 1 -- start start 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f288f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 0x7f0f2419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f288f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f24103430 0x7f0f2419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f288f5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2410ee40 con 0x7f0f24102b10 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f288f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2410ecc0 con 0x7f0f24103430 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f2259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 0x7f0f2419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f2259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 0x7f0f2419d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42580/0 (socket says 172.21.15.7:42580) 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f2259c640 1 -- 172.21.15.7:0/973228388 learned_addr learned my addr 172.21.15.7:0/973228388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f21d9b640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f24103430 0x7f0f2419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f2259c640 1 -- 172.21.15.7:0/973228388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f24103430 msgr2=0x7f0f2419dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.656+0000 7f0f2259c640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f24103430 0x7f0f2419dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.657+0000 7f0f2259c640 1 -- 172.21.15.7:0/973228388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f241978f0 con 0x7f0f24102b10 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.657+0000 7f0f21d9b640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f24103430 0x7f0f2419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.657+0000 7f0f2259c640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 0x7f0f2419d730 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f14006010 tx=0x7f0f1402ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.657+0000 7f0f137fe640 1 -- 172.21.15.7:0/973228388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f14047070 con 0x7f0f24102b10 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.657+0000 7f0f137fe640 1 -- 172.21.15.7:0/973228388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f14003e30 con 0x7f0f24102b10 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.657+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f24197b80 con 0x7f0f24102b10 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.657+0000 7f0f137fe640 1 -- 172.21.15.7:0/973228388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f14033050 con 0x7f0f24102b10 2024-09-21T04:47:17.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.657+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f241980b0 con 0x7f0f24102b10 2024-09-21T04:47:17.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.659+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2406a520 con 0x7f0f24102b10 2024-09-21T04:47:17.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.659+0000 7f0f137fe640 1 -- 172.21.15.7:0/973228388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0f1404a070 con 0x7f0f24102b10 2024-09-21T04:47:17.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.660+0000 7f0f137fe640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f0c07af80 0x7f0f0c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.660+0000 7f0f21d9b640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f0c07af80 0x7f0f0c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.660+0000 7f0f137fe640 1 -- 172.21.15.7:0/973228388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0f1403e030 con 0x7f0f24102b10 2024-09-21T04:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.660+0000 7f0f21d9b640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f0c07af80 0x7f0f0c07d440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.664+0000 7f0f137fe640 1 -- 172.21.15.7:0/973228388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f1408a750 con 0x7f0f24102b10 2024-09-21T04:47:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.866+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0f24198e20 con 0x7f0f0c07af80 2024-09-21T04:47:17.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.868+0000 7f0f137fe640 1 -- 172.21.15.7:0/973228388 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0f24198e20 con 0x7f0f0c07af80 2024-09-21T04:47:17.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.869+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f0c07af80 msgr2=0x7f0f0c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.869+0000 7f0f288f5640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f0c07af80 0x7f0f0c07d440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18008040 comp rx=0 tx=0).stop 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.869+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 msgr2=0x7f0f2419d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.869+0000 7f0f288f5640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 0x7f0f2419d730 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f14006010 tx=0x7f0f1402ed90 comp rx=0 tx=0).stop 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.870+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 shutdown_connections 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.870+0000 7f0f288f5640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f0c07af80 0x7f0f0c07d440 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.870+0000 7f0f288f5640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f24103430 0x7f0f2419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.870+0000 7f0f288f5640 1 --2- 172.21.15.7:0/973228388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f24102b10 0x7f0f2419d730 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.870+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 >> 172.21.15.7:0/973228388 conn(0x7f0f240fc550 msgr2=0x7f0f24103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.870+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 shutdown_connections 2024-09-21T04:47:17.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.870+0000 7f0f288f5640 1 -- 172.21.15.7:0/973228388 wait complete. 2024-09-21T04:47:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 -- 172.21.15.7:0/2648611584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 msgr2=0x7fd584100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 --2- 172.21.15.7:0/2648611584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 0x7fd584100c50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009880 tx=0x7fd56c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 -- 172.21.15.7:0/2648611584 shutdown_connections 2024-09-21T04:47:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 --2- 172.21.15.7:0/2648611584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 0x7fd584100c50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 --2- 172.21.15.7:0/2648611584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841067f0 0x7fd584106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 -- 172.21.15.7:0/2648611584 >> 172.21.15.7:0/2648611584 conn(0x7fd5840fc550 msgr2=0x7fd5840fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 -- 172.21.15.7:0/2648611584 shutdown_connections 2024-09-21T04:47:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 -- 172.21.15.7:0/2648611584 wait complete. 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.954+0000 7fd588aae640 1 Processor -- start 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd588aae640 1 -- start start 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd588aae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 0x7fd58419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd588aae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841067f0 0x7fd58419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd588aae640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd58410dcc0 con 0x7fd5841007f0 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd588aae640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd58410db40 con 0x7fd5841067f0 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd58259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 0x7fd58419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd58259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 0x7fd58419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42596/0 (socket says 172.21.15.7:42596) 2024-09-21T04:47:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd58259c640 1 -- 172.21.15.7:0/3264207588 learned_addr learned my addr 172.21.15.7:0/3264207588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd579d9b640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841067f0 0x7fd58419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd58259c640 1 -- 172.21.15.7:0/3264207588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841067f0 msgr2=0x7fd58419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd58259c640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841067f0 0x7fd58419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.955+0000 7fd58259c640 1 -- 172.21.15.7:0/3264207588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5841999b0 con 0x7fd5841007f0 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.956+0000 7fd58259c640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 0x7fd58419f7f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd57400e970 tx=0x7fd57400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.956+0000 7fd57bfff640 1 -- 172.21.15.7:0/3264207588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd57400cac0 con 0x7fd5841007f0 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.956+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd584199ca0 con 0x7fd5841007f0 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.956+0000 7fd57bfff640 1 -- 172.21.15.7:0/3264207588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5740054e0 con 0x7fd5841007f0 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.956+0000 7fd57bfff640 1 -- 172.21.15.7:0/3264207588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd574010430 con 0x7fd5841007f0 2024-09-21T04:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.956+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58419a200 con 0x7fd5841007f0 2024-09-21T04:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.957+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd584101f30 con 0x7fd5841007f0 2024-09-21T04:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.957+0000 7fd57bfff640 1 -- 172.21.15.7:0/3264207588 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd574004e80 con 0x7fd5841007f0 2024-09-21T04:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.958+0000 7fd57bfff640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd57007af30 0x7fd57007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.958+0000 7fd57bfff640 1 -- 172.21.15.7:0/3264207588 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd57409e730 con 0x7fd5841007f0 2024-09-21T04:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.958+0000 7fd579d9b640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd57007af30 0x7fd57007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.959+0000 7fd579d9b640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd57007af30 0x7fd57007d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd58419b1a0 tx=0x7fd56c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:17.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:17.962+0000 7fd57bfff640 1 -- 172.21.15.7:0/3264207588 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd574069c30 con 0x7fd5841007f0 2024-09-21T04:47:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.163+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd58419a510 con 0x7fd57007af30 2024-09-21T04:47:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.172+0000 7fd57bfff640 1 -- 172.21.15.7:0/3264207588 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fd58419a510 con 0x7fd57007af30 2024-09-21T04:47:18.175 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:47:18.175 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (9m) 3m ago 20m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (12m) 3m ago 20m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (12m) 3m ago 19m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (7m) 3m ago 20m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (7m) 3m ago 19m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (8m) 3m ago 20m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (3m) 3m ago 14m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (3m) 3m ago 14m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (14m) 3m ago 14m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (3m) 3m ago 14m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (11m) 3m ago 21m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (10m) 3m ago 19m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (7m) 3m ago 21m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (7m) 3m ago 19m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:47:18.176 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (10m) 3m ago 20m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:47:18.177 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (10m) 3m ago 19m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:47:18.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 3m ago 18m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:47:18.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (6m) 3m ago 17m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:47:18.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (5m) 3m ago 17m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:47:18.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (5m) 3m ago 16m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:47:18.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (4m) 3m ago 16m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:47:18.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (4m) 3m ago 15m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:47:18.177 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (9m) 3m ago 19m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:47:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd57007af30 msgr2=0x7fd57007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd57007af30 0x7fd57007d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd58419b1a0 tx=0x7fd56c031400 comp rx=0 tx=0).stop 2024-09-21T04:47:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 msgr2=0x7fd58419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 0x7fd58419f7f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd57400e970 tx=0x7fd57400ed70 comp rx=0 tx=0).stop 2024-09-21T04:47:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 shutdown_connections 2024-09-21T04:47:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd57007af30 0x7fd57007d3f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5841067f0 0x7fd58419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 --2- 172.21.15.7:0/3264207588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5841007f0 0x7fd58419f7f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 >> 172.21.15.7:0/3264207588 conn(0x7fd5840fc550 msgr2=0x7fd584069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 shutdown_connections 2024-09-21T04:47:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.175+0000 7fd588aae640 1 -- 172.21.15.7:0/3264207588 wait complete. 2024-09-21T04:47:18.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.258+0000 7f7f3d36e640 1 -- 172.21.15.7:0/765264486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38102ab0 msgr2=0x7f7f38102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.258+0000 7f7f3d36e640 1 --2- 172.21.15.7:0/765264486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38102ab0 0x7f7f38102e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009880 tx=0x7f7f2802a530 comp rx=0 tx=0).stop 2024-09-21T04:47:18.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.258+0000 7f7f3d36e640 1 -- 172.21.15.7:0/765264486 shutdown_connections 2024-09-21T04:47:18.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.258+0000 7f7f3d36e640 1 --2- 172.21.15.7:0/765264486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f381033d0 0x7f7f3810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.258+0000 7f7f3d36e640 1 --2- 172.21.15.7:0/765264486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f38102ab0 0x7f7f38102e90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.258+0000 7f7f3d36e640 1 -- 172.21.15.7:0/765264486 >> 172.21.15.7:0/765264486 conn(0x7f7f380fc550 msgr2=0x7f7f380fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:18.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.258+0000 7f7f3d36e640 1 -- 172.21.15.7:0/765264486 shutdown_connections 2024-09-21T04:47:18.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.258+0000 7f7f3d36e640 1 -- 172.21.15.7:0/765264486 wait complete. 2024-09-21T04:47:18.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f3d36e640 1 Processor -- start 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f3d36e640 1 -- start start 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f3d36e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f38102ab0 0x7f7f38100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f3d36e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f381033d0 0x7f7f38100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f3d36e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810ede0 con 0x7f7f381033d0 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f3d36e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810ec60 con 0x7f7f38102ab0 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f367fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f381033d0 0x7f7f38100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f367fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f381033d0 0x7f7f38100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42622/0 (socket says 172.21.15.7:42622) 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f367fc640 1 -- 172.21.15.7:0/4144583282 learned_addr learned my addr 172.21.15.7:0/4144583282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.259+0000 7f7f36ffd640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f38102ab0 0x7f7f38100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f367fc640 1 -- 172.21.15.7:0/4144583282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f38102ab0 msgr2=0x7f7f38100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f367fc640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f38102ab0 0x7f7f38100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f367fc640 1 -- 172.21.15.7:0/4144583282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f38101190 con 0x7f7f381033d0 2024-09-21T04:47:18.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f367fc640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f381033d0 0x7f7f38100c50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00d6d0 tx=0x7f7f2c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:18.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f1ffff640 1 -- 172.21.15.7:0/4144583282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7f2c017070 con 0x7f7f381033d0 2024-09-21T04:47:18.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f38101480 con 0x7f7f381033d0 2024-09-21T04:47:18.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f381a3e30 con 0x7f7f381033d0 2024-09-21T04:47:18.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f1ffff640 1 -- 172.21.15.7:0/4144583282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f2c0054e0 con 0x7f7f381033d0 2024-09-21T04:47:18.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.260+0000 7f7f1ffff640 1 -- 172.21.15.7:0/4144583282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7f2c004ae0 con 0x7f7f381033d0 2024-09-21T04:47:18.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.261+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24005290 con 0x7f7f381033d0 2024-09-21T04:47:18.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.262+0000 7f7f1ffff640 1 -- 172.21.15.7:0/4144583282 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7f2c009820 con 0x7f7f381033d0 2024-09-21T04:47:18.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.263+0000 7f7f1ffff640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7f2007af80 0x7f7f2007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.263+0000 7f7f1ffff640 1 -- 172.21.15.7:0/4144583282 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7f2c09da30 con 0x7f7f381033d0 2024-09-21T04:47:18.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.263+0000 7f7f36ffd640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7f2007af80 0x7f7f2007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.263+0000 7f7f36ffd640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7f2007af80 0x7f7f2007d440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f280096c0 tx=0x7f7f28031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:18.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.267+0000 7f7f1ffff640 1 -- 172.21.15.7:0/4144583282 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7f2c069050 con 0x7f7f381033d0 2024-09-21T04:47:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.534+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7f24005d90 con 0x7f7f381033d0 2024-09-21T04:47:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.534+0000 7f7f1ffff640 1 -- 172.21.15.7:0/4144583282 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f7f2c06df00 con 0x7f7f381033d0 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:47:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:47:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:47:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:47:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:47:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:47:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:47:18.539 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7f2007af80 msgr2=0x7f7f2007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7f2007af80 0x7f7f2007d440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f280096c0 tx=0x7f7f28031400 comp rx=0 tx=0).stop 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f381033d0 msgr2=0x7f7f38100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f381033d0 0x7f7f38100c50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00d6d0 tx=0x7f7f2c00db90 comp rx=0 tx=0).stop 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 shutdown_connections 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7f2007af80 0x7f7f2007d440 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f381033d0 0x7f7f38100c50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 --2- 172.21.15.7:0/4144583282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f38102ab0 0x7f7f38100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.536+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 >> 172.21.15.7:0/4144583282 conn(0x7f7f380fc550 msgr2=0x7f7f3810aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.537+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 shutdown_connections 2024-09-21T04:47:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.537+0000 7f7f3d36e640 1 -- 172.21.15.7:0/4144583282 wait complete. 2024-09-21T04:47:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.621+0000 7f618a473640 1 -- 172.21.15.7:0/631273813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184100800 msgr2=0x7f6184100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.621+0000 7f618a473640 1 --2- 172.21.15.7:0/631273813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184100800 0x7f6184100c60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6178009880 tx=0x7f617802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.621+0000 7f618a473640 1 -- 172.21.15.7:0/631273813 shutdown_connections 2024-09-21T04:47:18.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.621+0000 7f618a473640 1 --2- 172.21.15.7:0/631273813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184100800 0x7f6184100c60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.621+0000 7f618a473640 1 --2- 172.21.15.7:0/631273813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6184106800 0x7f6184106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.621+0000 7f618a473640 1 -- 172.21.15.7:0/631273813 >> 172.21.15.7:0/631273813 conn(0x7f61840fc520 msgr2=0x7f61840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:18.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.621+0000 7f618a473640 1 -- 172.21.15.7:0/631273813 shutdown_connections 2024-09-21T04:47:18.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.621+0000 7f618a473640 1 -- 172.21.15.7:0/631273813 wait complete. 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.622+0000 7f618a473640 1 Processor -- start 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.622+0000 7f618a473640 1 -- start start 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f618a473640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6184100800 0x7f618419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f618a473640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184106800 0x7f618419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f618a473640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f618410dcb0 con 0x7f6184106800 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f618a473640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f618410db30 con 0x7f6184100800 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184106800 0x7f618419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184106800 0x7f618419dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42638/0 (socket says 172.21.15.7:42638) 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61837fe640 1 -- 172.21.15.7:0/1023667304 learned_addr learned my addr 172.21.15.7:0/1023667304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f6183fff640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6184100800 0x7f618419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61837fe640 1 -- 172.21.15.7:0/1023667304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6184100800 msgr2=0x7f618419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61837fe640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6184100800 0x7f618419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61837fe640 1 -- 172.21.15.7:0/1023667304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61841977f0 con 0x7f6184106800 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61837fe640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184106800 0x7f618419dbb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6178031460 tx=0x7f61780045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61817fa640 1 -- 172.21.15.7:0/1023667304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f617803f070 con 0x7f6184106800 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61817fa640 1 -- 172.21.15.7:0/1023667304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61780035f0 con 0x7f6184106800 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f61817fa640 1 -- 172.21.15.7:0/1023667304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6178035030 con 0x7f6184106800 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.623+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6184197a80 con 0x7f6184106800 2024-09-21T04:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.624+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6184197fb0 con 0x7f6184106800 2024-09-21T04:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.625+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6184101f40 con 0x7f6184106800 2024-09-21T04:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.625+0000 7f61817fa640 1 -- 172.21.15.7:0/1023667304 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6178042070 con 0x7f6184106800 2024-09-21T04:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.626+0000 7f61817fa640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f616c07af30 0x7f616c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.626+0000 7f61817fa640 1 -- 172.21.15.7:0/1023667304 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f61780bea70 con 0x7f6184106800 2024-09-21T04:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.626+0000 7f6183fff640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f616c07af30 0x7f616c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.627+0000 7f6183fff640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f616c07af30 0x7f616c07d3f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f617400bbe0 tx=0x7f617400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.631+0000 7f61817fa640 1 -- 172.21.15.7:0/1023667304 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6178089f70 con 0x7f6184106800 2024-09-21T04:47:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.856+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6184109da0 con 0x7f6184106800 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.857+0000 7f61817fa640 1 -- 172.21.15.7:0/1023667304 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f617808ee20 con 0x7f6184106800 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:47:18.861 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:47:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:47:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:47:18.864 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.859+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f616c07af30 msgr2=0x7f616c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.859+0000 7f618a473640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f616c07af30 0x7f616c07d3f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f617400bbe0 tx=0x7f617400b390 comp rx=0 tx=0).stop 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.859+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184106800 msgr2=0x7f618419dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.859+0000 7f618a473640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184106800 0x7f618419dbb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6178031460 tx=0x7f61780045e0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.860+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 shutdown_connections 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.860+0000 7f618a473640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f616c07af30 0x7f616c07d3f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.860+0000 7f618a473640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6184106800 0x7f618419dbb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.860+0000 7f618a473640 1 --2- 172.21.15.7:0/1023667304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6184100800 0x7f618419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.860+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 >> 172.21.15.7:0/1023667304 conn(0x7f61840fc520 msgr2=0x7f618406a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.860+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 shutdown_connections 2024-09-21T04:47:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.860+0000 7f618a473640 1 -- 172.21.15.7:0/1023667304 wait complete. 2024-09-21T04:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.946+0000 7f7754f95640 1 -- 172.21.15.7:0/221113124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750101900 msgr2=0x7f7750101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.946+0000 7f7754f95640 1 --2- 172.21.15.7:0/221113124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750101900 0x7f7750101d60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7738009880 tx=0x7f773802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.946+0000 7f7754f95640 1 -- 172.21.15.7:0/221113124 shutdown_connections 2024-09-21T04:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.946+0000 7f7754f95640 1 --2- 172.21.15.7:0/221113124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750101900 0x7f7750101d60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.946+0000 7f7754f95640 1 --2- 172.21.15.7:0/221113124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750107900 0x7f7750107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.946+0000 7f7754f95640 1 -- 172.21.15.7:0/221113124 >> 172.21.15.7:0/221113124 conn(0x7f77500fd600 msgr2=0x7f77500ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.946+0000 7f7754f95640 1 -- 172.21.15.7:0/221113124 shutdown_connections 2024-09-21T04:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.946+0000 7f7754f95640 1 -- 172.21.15.7:0/221113124 wait complete. 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7754f95640 1 Processor -- start 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7754f95640 1 -- start start 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7754f95640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750101900 0x7f77500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7754f95640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750107900 0x7f7750074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7754f95640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f775010edd0 con 0x7f7750107900 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7754f95640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f775010ec50 con 0x7f7750101900 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7745d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750107900 0x7f7750074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7745d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750107900 0x7f7750074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42646/0 (socket says 172.21.15.7:42646) 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f774e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750101900 0x7f77500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.947+0000 7f7745d9b640 1 -- 172.21.15.7:0/1124301594 learned_addr learned my addr 172.21.15.7:0/1124301594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f774e59c640 1 -- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750107900 msgr2=0x7f7750074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f774e59c640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750107900 0x7f7750074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f774e59c640 1 -- 172.21.15.7:0/1124301594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7750074f70 con 0x7f7750101900 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f774e59c640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750101900 0x7f77500744f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f774000e970 tx=0x7f774000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f7747fff640 1 -- 172.21.15.7:0/1124301594 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f774000cac0 con 0x7f7750101900 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7750075260 con 0x7f7750101900 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f7747fff640 1 -- 172.21.15.7:0/1124301594 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f774000cc60 con 0x7f7750101900 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f7747fff640 1 -- 172.21.15.7:0/1124301594 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f77400047a0 con 0x7f7750101900 2024-09-21T04:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.948+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77501a3e20 con 0x7f7750101900 2024-09-21T04:47:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.949+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7750103040 con 0x7f7750101900 2024-09-21T04:47:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.950+0000 7f7747fff640 1 -- 172.21.15.7:0/1124301594 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7740004a50 con 0x7f7750101900 2024-09-21T04:47:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.950+0000 7f7747fff640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f773c07af40 0x7f773c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.950+0000 7f7747fff640 1 -- 172.21.15.7:0/1124301594 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f774009e540 con 0x7f7750101900 2024-09-21T04:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.950+0000 7f7745d9b640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f773c07af40 0x7f773c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.951+0000 7f7745d9b640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f773c07af40 0x7f773c07d400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7750075f80 tx=0x7f7738005a20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:18.955+0000 7f7747fff640 1 -- 172.21.15.7:0/1124301594 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7740069a80 con 0x7f7750101900 2024-09-21T04:47:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.160+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7750062630 con 0x7f773c07af40 2024-09-21T04:47:19.164 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:19 smithi007 ceph-mon[108238]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:19.164 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:19 smithi007 ceph-mon[108238]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:19.164 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:19 smithi007 ceph-mon[108238]: pgmap v376: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:19.164 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:19 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4144583282' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:47:19.164 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:19 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1023667304' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.162+0000 7f7747fff640 1 -- 172.21.15.7:0/1124301594 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f7750062630 con 0x7f773c07af40 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:47:19.165 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:47:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:47:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:47:19.166 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:47:19.166 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:47:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f773c07af40 msgr2=0x7f773c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f773c07af40 0x7f773c07d400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7750075f80 tx=0x7f7738005a20 comp rx=0 tx=0).stop 2024-09-21T04:47:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750101900 msgr2=0x7f77500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750101900 0x7f77500744f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f774000e970 tx=0x7f774000ee30 comp rx=0 tx=0).stop 2024-09-21T04:47:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 shutdown_connections 2024-09-21T04:47:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f773c07af40 0x7f773c07d400 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7750107900 0x7f7750074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 --2- 172.21.15.7:0/1124301594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7750101900 0x7f77500744f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 >> 172.21.15.7:0/1124301594 conn(0x7f77500fd600 msgr2=0x7f77500fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 shutdown_connections 2024-09-21T04:47:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.165+0000 7f7754f95640 1 -- 172.21.15.7:0/1124301594 wait complete. 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.249+0000 7f7d113d6640 1 -- 172.21.15.7:0/3130975627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 msgr2=0x7f7d0c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.249+0000 7f7d113d6640 1 --2- 172.21.15.7:0/3130975627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 0x7f7d0c100c60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009880 tx=0x7f7cf402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.250+0000 7f7d113d6640 1 -- 172.21.15.7:0/3130975627 shutdown_connections 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.250+0000 7f7d113d6640 1 --2- 172.21.15.7:0/3130975627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 0x7f7d0c100c60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.250+0000 7f7d113d6640 1 --2- 172.21.15.7:0/3130975627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d0c106800 0x7f7d0c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.250+0000 7f7d113d6640 1 -- 172.21.15.7:0/3130975627 >> 172.21.15.7:0/3130975627 conn(0x7f7d0c0fc520 msgr2=0x7f7d0c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.250+0000 7f7d113d6640 1 -- 172.21.15.7:0/3130975627 shutdown_connections 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.250+0000 7f7d113d6640 1 -- 172.21.15.7:0/3130975627 wait complete. 2024-09-21T04:47:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.250+0000 7f7d113d6640 1 Processor -- start 2024-09-21T04:47:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.250+0000 7f7d113d6640 1 -- start start 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d113d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 0x7f7d0c0fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d113d6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d0c106800 0x7f7d0c0fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d113d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c10dcd0 con 0x7f7d0c100800 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d113d6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c10db50 con 0x7f7d0c106800 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 0x7f7d0c0fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 0x7f7d0c0fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42658/0 (socket says 172.21.15.7:42658) 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d0affd640 1 -- 172.21.15.7:0/732688731 learned_addr learned my addr 172.21.15.7:0/732688731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d0affd640 1 -- 172.21.15.7:0/732688731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d0c106800 msgr2=0x7f7d0c0fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d03fff640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d0c106800 0x7f7d0c0fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d0affd640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d0c106800 0x7f7d0c0fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d0affd640 1 -- 172.21.15.7:0/732688731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c0fd670 con 0x7f7d0c100800 2024-09-21T04:47:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d0affd640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 0x7f7d0c0fcbf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00e970 tx=0x7f7cfc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d08ff9640 1 -- 172.21.15.7:0/732688731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cfc00cac0 con 0x7f7d0c100800 2024-09-21T04:47:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d08ff9640 1 -- 172.21.15.7:0/732688731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cfc0054e0 con 0x7f7d0c100800 2024-09-21T04:47:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.251+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c0fd960 con 0x7f7d0c100800 2024-09-21T04:47:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.252+0000 7f7d08ff9640 1 -- 172.21.15.7:0/732688731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cfc010430 con 0x7f7d0c100800 2024-09-21T04:47:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.252+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c1a3e70 con 0x7f7d0c100800 2024-09-21T04:47:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.253+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0c101f40 con 0x7f7d0c100800 2024-09-21T04:47:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.253+0000 7f7d08ff9640 1 -- 172.21.15.7:0/732688731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7cfc004ee0 con 0x7f7d0c100800 2024-09-21T04:47:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.254+0000 7f7d08ff9640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf807af30 0x7f7cf807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.254+0000 7f7d03fff640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf807af30 0x7f7cf807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.254+0000 7f7d08ff9640 1 -- 172.21.15.7:0/732688731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7cfc09e7c0 con 0x7f7d0c100800 2024-09-21T04:47:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.254+0000 7f7d03fff640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf807af30 0x7f7cf807d3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4031460 tx=0x7f7cf4009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.258+0000 7f7d08ff9640 1 -- 172.21.15.7:0/732688731 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7cfc069d60 con 0x7f7d0c100800 2024-09-21T04:47:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:19 smithi053 ceph-mon[80176]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:19 smithi053 ceph-mon[80176]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:19 smithi053 ceph-mon[80176]: pgmap v376: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:19 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4144583282' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:47:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:19 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1023667304' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.522+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f7d0c100c60 con 0x7f7d0c100800 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.522+0000 7f7d08ff9640 1 -- 172.21.15.7:0/732688731 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f7cfc06ec10 con 0x7f7d0c100800 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:47:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:47:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf807af30 msgr2=0x7f7cf807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf807af30 0x7f7cf807d3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4031460 tx=0x7f7cf4009560 comp rx=0 tx=0).stop 2024-09-21T04:47:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 msgr2=0x7f7d0c0fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 0x7f7d0c0fcbf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00e970 tx=0x7f7cfc00ed70 comp rx=0 tx=0).stop 2024-09-21T04:47:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 shutdown_connections 2024-09-21T04:47:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cf807af30 0x7f7cf807d3f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d0c106800 0x7f7d0c0fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 --2- 172.21.15.7:0/732688731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d0c100800 0x7f7d0c0fcbf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 >> 172.21.15.7:0/732688731 conn(0x7f7d0c0fc520 msgr2=0x7f7d0c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 shutdown_connections 2024-09-21T04:47:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:19.524+0000 7f7d113d6640 1 -- 172.21.15.7:0/732688731 wait complete. 2024-09-21T04:47:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:20 smithi007 ceph-mon[108238]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:20 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/732688731' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:47:20.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:20 smithi053 ceph-mon[80176]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:20.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:20 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/732688731' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:47:21.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:21 smithi053 ceph-mon[80176]: from='client.44453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:21 smithi053 ceph-mon[80176]: pgmap v377: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:21 smithi007 ceph-mon[108238]: from='client.44453 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:21 smithi007 ceph-mon[108238]: pgmap v377: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:23 smithi007 ceph-mon[108238]: pgmap v378: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:23.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:23 smithi053 ceph-mon[80176]: pgmap v378: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:25 smithi053 ceph-mon[80176]: pgmap v379: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:25 smithi007 ceph-mon[108238]: pgmap v379: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:27 smithi053 ceph-mon[80176]: pgmap v380: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:27 smithi007 ceph-mon[108238]: pgmap v380: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:29 smithi053 ceph-mon[80176]: pgmap v381: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:29.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:47:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:29 smithi007 ceph-mon[108238]: pgmap v381: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:47:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:31 smithi053 ceph-mon[80176]: pgmap v382: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:31 smithi007 ceph-mon[108238]: pgmap v382: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:33 smithi053 ceph-mon[80176]: pgmap v383: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:33 smithi007 ceph-mon[108238]: pgmap v383: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:34 smithi053 ceph-mon[80176]: pgmap v384: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:34 smithi007 ceph-mon[108238]: pgmap v384: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:37.349 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:36 smithi007 ceph-mon[108238]: pgmap v385: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:36 smithi053 ceph-mon[80176]: pgmap v385: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:38 smithi053 ceph-mon[80176]: pgmap v386: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:38 smithi007 ceph-mon[108238]: pgmap v386: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:40 smithi007 ceph-mon[108238]: pgmap v387: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:40 smithi053 ceph-mon[80176]: pgmap v387: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:43.245 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:42 smithi053 ceph-mon[80176]: pgmap v388: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:42 smithi007 ceph-mon[108238]: pgmap v388: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:47:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:47:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:44 smithi053 ceph-mon[80176]: pgmap v389: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:44 smithi007 ceph-mon[108238]: pgmap v389: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:46 smithi007 ceph-mon[108238]: pgmap v390: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:46 smithi053 ceph-mon[80176]: pgmap v390: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:48 smithi053 ceph-mon[80176]: pgmap v391: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:48 smithi007 ceph-mon[108238]: pgmap v391: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:49.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.622+0000 7f8014412640 1 -- 172.21.15.7:0/2501126700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 msgr2=0x7f800c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.622+0000 7f8014412640 1 --2- 172.21.15.7:0/2501126700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 0x7f800c102ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8008009880 tx=0x7f800802a530 comp rx=0 tx=0).stop 2024-09-21T04:47:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.622+0000 7f8014412640 1 -- 172.21.15.7:0/2501126700 shutdown_connections 2024-09-21T04:47:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.623+0000 7f8014412640 1 --2- 172.21.15.7:0/2501126700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f800c1033e0 0x7f800c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.623+0000 7f8014412640 1 --2- 172.21.15.7:0/2501126700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 0x7f800c102ea0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.623+0000 7f8014412640 1 -- 172.21.15.7:0/2501126700 >> 172.21.15.7:0/2501126700 conn(0x7f800c0fc520 msgr2=0x7f800c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.623+0000 7f8014412640 1 -- 172.21.15.7:0/2501126700 shutdown_connections 2024-09-21T04:47:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.623+0000 7f8014412640 1 -- 172.21.15.7:0/2501126700 wait complete. 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.623+0000 7f8014412640 1 Processor -- start 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.623+0000 7f8014412640 1 -- start start 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f8014412640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 0x7f800c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f8014412640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f800c1033e0 0x7f800c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f8014412640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10edd0 con 0x7f800c102ac0 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f8014412640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10ec50 con 0x7f800c1033e0 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80121ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 0x7f800c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80121ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 0x7f800c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41218/0 (socket says 172.21.15.7:41218) 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80121ae640 1 -- 172.21.15.7:0/3205955939 learned_addr learned my addr 172.21.15.7:0/3205955939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80119ad640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f800c1033e0 0x7f800c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80121ae640 1 -- 172.21.15.7:0/3205955939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f800c1033e0 msgr2=0x7f800c19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80121ae640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f800c1033e0 0x7f800c19dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80121ae640 1 -- 172.21.15.7:0/3205955939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c19e310 con 0x7f800c102ac0 2024-09-21T04:47:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80121ae640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 0x7f800c19d720 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8008031460 tx=0x7f80080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80037fe640 1 -- 172.21.15.7:0/3205955939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f800803f070 con 0x7f800c102ac0 2024-09-21T04:47:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.624+0000 7f80037fe640 1 -- 172.21.15.7:0/3205955939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80080044d0 con 0x7f800c102ac0 2024-09-21T04:47:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.625+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c1979a0 con 0x7f800c102ac0 2024-09-21T04:47:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.625+0000 7f80037fe640 1 -- 172.21.15.7:0/3205955939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8008035030 con 0x7f800c102ac0 2024-09-21T04:47:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.625+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c197ea0 con 0x7f800c102ac0 2024-09-21T04:47:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.626+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c06aa90 con 0x7f800c102ac0 2024-09-21T04:47:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.626+0000 7f80037fe640 1 -- 172.21.15.7:0/3205955939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8008042070 con 0x7f800c102ac0 2024-09-21T04:47:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.627+0000 7f80037fe640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ff807af90 0x7f7ff807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.627+0000 7f80037fe640 1 -- 172.21.15.7:0/3205955939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f80080bea90 con 0x7f800c102ac0 2024-09-21T04:47:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.627+0000 7f80119ad640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ff807af90 0x7f7ff807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.628+0000 7f80119ad640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ff807af90 0x7f7ff807d450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f800c0feec0 tx=0x7f7ffc00a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.632+0000 7f80037fe640 1 -- 172.21.15.7:0/3205955939 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8008004cf0 con 0x7f800c102ac0 2024-09-21T04:47:49.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.838+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f800c198e10 con 0x7f7ff807af90 2024-09-21T04:47:49.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.839+0000 7f80037fe640 1 -- 172.21.15.7:0/3205955939 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f800c198e10 con 0x7f7ff807af90 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ff807af90 msgr2=0x7f7ff807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ff807af90 0x7f7ff807d450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f800c0feec0 tx=0x7f7ffc00a250 comp rx=0 tx=0).stop 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 msgr2=0x7f800c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 0x7f800c19d720 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8008031460 tx=0x7f80080048f0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 shutdown_connections 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ff807af90 0x7f7ff807d450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f800c1033e0 0x7f800c19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 --2- 172.21.15.7:0/3205955939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c102ac0 0x7f800c19d720 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 >> 172.21.15.7:0/3205955939 conn(0x7f800c0fc520 msgr2=0x7f800c108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 shutdown_connections 2024-09-21T04:47:49.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.842+0000 7f8014412640 1 -- 172.21.15.7:0/3205955939 wait complete. 2024-09-21T04:47:49.854 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:47:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.928+0000 7f67cd759640 1 -- 172.21.15.7:0/371127627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 msgr2=0x7f67c8106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.928+0000 7f67cd759640 1 --2- 172.21.15.7:0/371127627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 0x7f67c8106bd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009880 tx=0x7f67b002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.928+0000 7f67cd759640 1 -- 172.21.15.7:0/371127627 shutdown_connections 2024-09-21T04:47:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.928+0000 7f67cd759640 1 --2- 172.21.15.7:0/371127627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81007f0 0x7f67c8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.928+0000 7f67cd759640 1 --2- 172.21.15.7:0/371127627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 0x7f67c8106bd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.928+0000 7f67cd759640 1 -- 172.21.15.7:0/371127627 >> 172.21.15.7:0/371127627 conn(0x7f67c80fc550 msgr2=0x7f67c80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.928+0000 7f67cd759640 1 -- 172.21.15.7:0/371127627 shutdown_connections 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.928+0000 7f67cd759640 1 -- 172.21.15.7:0/371127627 wait complete. 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.929+0000 7f67cd759640 1 Processor -- start 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.929+0000 7f67cd759640 1 -- start start 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.929+0000 7f67cd759640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81007f0 0x7f67c81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.929+0000 7f67cd759640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 0x7f67c8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.929+0000 7f67cd759640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67c810dca0 con 0x7f67c81067f0 2024-09-21T04:47:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.929+0000 7f67cd759640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67c810db20 con 0x7f67c81007f0 2024-09-21T04:47:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.930+0000 7f67c67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 0x7f67c8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.930+0000 7f67c67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 0x7f67c8198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41242/0 (socket says 172.21.15.7:41242) 2024-09-21T04:47:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.930+0000 7f67c67fc640 1 -- 172.21.15.7:0/3047725532 learned_addr learned my addr 172.21.15.7:0/3047725532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.930+0000 7f67c6ffd640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81007f0 0x7f67c81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.930+0000 7f67c67fc640 1 -- 172.21.15.7:0/3047725532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81007f0 msgr2=0x7f67c81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.930+0000 7f67c67fc640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81007f0 0x7f67c81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.930+0000 7f67c67fc640 1 -- 172.21.15.7:0/3047725532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c8199410 con 0x7f67c81067f0 2024-09-21T04:47:49.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.933+0000 7f67c67fc640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 0x7f67c8198e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f67b800d8d0 tx=0x7f67b800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:49.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.934+0000 7f67bffff640 1 -- 172.21.15.7:0/3047725532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f67b8017030 con 0x7f67c81067f0 2024-09-21T04:47:49.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.934+0000 7f67bffff640 1 -- 172.21.15.7:0/3047725532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f67b8004020 con 0x7f67c81067f0 2024-09-21T04:47:49.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.934+0000 7f67bffff640 1 -- 172.21.15.7:0/3047725532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f67b80047a0 con 0x7f67c81067f0 2024-09-21T04:47:49.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.934+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c8199700 con 0x7f67c81067f0 2024-09-21T04:47:49.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.935+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c81a1d00 con 0x7f67c81067f0 2024-09-21T04:47:49.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.936+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c8101f30 con 0x7f67c81067f0 2024-09-21T04:47:49.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.937+0000 7f67bffff640 1 -- 172.21.15.7:0/3047725532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f67b80041d0 con 0x7f67c81067f0 2024-09-21T04:47:49.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.937+0000 7f67bffff640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f67b407af80 0x7f67b407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:49.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.937+0000 7f67bffff640 1 -- 172.21.15.7:0/3047725532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f67b8018070 con 0x7f67c81067f0 2024-09-21T04:47:49.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.937+0000 7f67c6ffd640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f67b407af80 0x7f67b407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:49.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.938+0000 7f67c6ffd640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f67b407af80 0x7f67b407d440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67b00096c0 tx=0x7f67b0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:49.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:49.942+0000 7f67bffff640 1 -- 172.21.15.7:0/3047725532 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f67c8199700 con 0x7f67c81067f0 2024-09-21T04:47:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.148+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f67c8062630 con 0x7f67b407af80 2024-09-21T04:47:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.149+0000 7f67bffff640 1 -- 172.21.15.7:0/3047725532 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f67c8062630 con 0x7f67b407af80 2024-09-21T04:47:50.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f67b407af80 msgr2=0x7f67b407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f67b407af80 0x7f67b407d440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67b00096c0 tx=0x7f67b0031400 comp rx=0 tx=0).stop 2024-09-21T04:47:50.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 msgr2=0x7f67c8198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 0x7f67c8198e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f67b800d8d0 tx=0x7f67b800dd90 comp rx=0 tx=0).stop 2024-09-21T04:47:50.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 shutdown_connections 2024-09-21T04:47:50.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f67b407af80 0x7f67b407d440 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c81067f0 0x7f67c8198e10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 --2- 172.21.15.7:0/3047725532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81007f0 0x7f67c81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.151+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 >> 172.21.15.7:0/3047725532 conn(0x7f67c80fc550 msgr2=0x7f67c80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:50.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.152+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 shutdown_connections 2024-09-21T04:47:50.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.152+0000 7f67cd759640 1 -- 172.21.15.7:0/3047725532 wait complete. 2024-09-21T04:47:50.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.236+0000 7fbb60a17640 1 -- 172.21.15.7:0/62813222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 msgr2=0x7fbb5c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.236+0000 7fbb60a17640 1 --2- 172.21.15.7:0/62813222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 0x7fbb5c101d60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009880 tx=0x7fbb5002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.236+0000 7fbb60a17640 1 -- 172.21.15.7:0/62813222 shutdown_connections 2024-09-21T04:47:50.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.236+0000 7fbb60a17640 1 --2- 172.21.15.7:0/62813222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 0x7fbb5c101d60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.236+0000 7fbb60a17640 1 --2- 172.21.15.7:0/62813222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb5c107900 0x7fbb5c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.236+0000 7fbb60a17640 1 -- 172.21.15.7:0/62813222 >> 172.21.15.7:0/62813222 conn(0x7fbb5c0fd600 msgr2=0x7fbb5c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:50.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.236+0000 7fbb60a17640 1 -- 172.21.15.7:0/62813222 shutdown_connections 2024-09-21T04:47:50.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.236+0000 7fbb60a17640 1 -- 172.21.15.7:0/62813222 wait complete. 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb60a17640 1 Processor -- start 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb60a17640 1 -- start start 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb60a17640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 0x7fbb5c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb60a17640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb5c107900 0x7fbb5c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb60a17640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c10edb0 con 0x7fbb5c101900 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb60a17640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c10ec30 con 0x7fbb5c107900 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb5a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 0x7fbb5c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb5a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 0x7fbb5c19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41256/0 (socket says 172.21.15.7:41256) 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.237+0000 7fbb5a59c640 1 -- 172.21.15.7:0/149017199 learned_addr learned my addr 172.21.15.7:0/149017199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb59d9b640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb5c107900 0x7fbb5c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb5a59c640 1 -- 172.21.15.7:0/149017199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb5c107900 msgr2=0x7fbb5c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb5a59c640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb5c107900 0x7fbb5c19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb5a59c640 1 -- 172.21.15.7:0/149017199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c197800 con 0x7fbb5c101900 2024-09-21T04:47:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb5a59c640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 0x7fbb5c19d680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c8a0 tx=0x7fbb4c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb4b7fe640 1 -- 172.21.15.7:0/149017199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb4c0079b0 con 0x7fbb5c101900 2024-09-21T04:47:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb4b7fe640 1 -- 172.21.15.7:0/149017199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb4c007b50 con 0x7fbb5c101900 2024-09-21T04:47:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c197af0 con 0x7fbb5c101900 2024-09-21T04:47:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb4b7fe640 1 -- 172.21.15.7:0/149017199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb4c018700 con 0x7fbb5c101900 2024-09-21T04:47:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.238+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c198050 con 0x7fbb5c101900 2024-09-21T04:47:50.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.239+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c103040 con 0x7fbb5c101900 2024-09-21T04:47:50.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.240+0000 7fbb4b7fe640 1 -- 172.21.15.7:0/149017199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbb4c007d00 con 0x7fbb5c101900 2024-09-21T04:47:50.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.241+0000 7fbb4b7fe640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb4407af80 0x7fbb4407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.241+0000 7fbb59d9b640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb4407af80 0x7fbb4407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.241+0000 7fbb4b7fe640 1 -- 172.21.15.7:0/149017199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fbb4c09e350 con 0x7fbb5c101900 2024-09-21T04:47:50.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.241+0000 7fbb59d9b640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb4407af80 0x7fbb4407d440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c198b50 tx=0x7fbb50031420 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.245+0000 7fbb4b7fe640 1 -- 172.21.15.7:0/149017199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbb4c014030 con 0x7fbb5c101900 2024-09-21T04:47:50.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.441+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbb5c198360 con 0x7fbb4407af80 2024-09-21T04:47:50.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.451+0000 7fbb4b7fe640 1 -- 172.21.15.7:0/149017199 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fbb5c198360 con 0x7fbb4407af80 2024-09-21T04:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (9m) 3m ago 20m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (12m) 3m ago 21m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (12m) 3m ago 20m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (8m) 3m ago 20m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (7m) 3m ago 20m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (9m) 3m ago 20m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (3m) 3m ago 15m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (4m) 3m ago 15m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (15m) 3m ago 15m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (3m) 3m ago 15m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (12m) 3m ago 22m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (11m) 3m ago 19m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (8m) 3m ago 22m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (8m) 3m ago 19m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (10m) 3m ago 20m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (10m) 3m ago 20m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 3m ago 18m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (6m) 3m ago 18m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (6m) 3m ago 17m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (5m) 3m ago 17m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (5m) 3m ago 16m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (4m) 3m ago 16m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (10m) 3m ago 20m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb4407af80 msgr2=0x7fbb4407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb4407af80 0x7fbb4407d440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c198b50 tx=0x7fbb50031420 comp rx=0 tx=0).stop 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 msgr2=0x7fbb5c19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 0x7fbb5c19d680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c8a0 tx=0x7fbb4c00cd60 comp rx=0 tx=0).stop 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 shutdown_connections 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb4407af80 0x7fbb4407d440 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb5c107900 0x7fbb5c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 --2- 172.21.15.7:0/149017199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5c101900 0x7fbb5c19d680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 >> 172.21.15.7:0/149017199 conn(0x7fbb5c0fd600 msgr2=0x7fbb5c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 shutdown_connections 2024-09-21T04:47:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.453+0000 7fbb60a17640 1 -- 172.21.15.7:0/149017199 wait complete. 2024-09-21T04:47:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.537+0000 7f493bd20640 1 -- 172.21.15.7:0/2598258581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 msgr2=0x7f4934101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.537+0000 7f493bd20640 1 --2- 172.21.15.7:0/2598258581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 0x7f4934101d60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4924009880 tx=0x7f492402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.537+0000 7f493bd20640 1 -- 172.21.15.7:0/2598258581 shutdown_connections 2024-09-21T04:47:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.537+0000 7f493bd20640 1 --2- 172.21.15.7:0/2598258581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 0x7f4934101d60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.537+0000 7f493bd20640 1 --2- 172.21.15.7:0/2598258581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4934107900 0x7f4934107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.537+0000 7f493bd20640 1 -- 172.21.15.7:0/2598258581 >> 172.21.15.7:0/2598258581 conn(0x7f49340fd600 msgr2=0x7f49340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:50.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.538+0000 7f493bd20640 1 -- 172.21.15.7:0/2598258581 shutdown_connections 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.538+0000 7f493bd20640 1 -- 172.21.15.7:0/2598258581 wait complete. 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.538+0000 7f493bd20640 1 Processor -- start 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.538+0000 7f493bd20640 1 -- start start 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f493bd20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 0x7f493419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f493bd20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4934107900 0x7f493419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f493bd20640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f493410edb0 con 0x7f4934101900 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f493bd20640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f493410ec30 con 0x7f4934107900 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f4939abc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 0x7f493419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f4939abc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 0x7f493419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41276/0 (socket says 172.21.15.7:41276) 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f4939abc640 1 -- 172.21.15.7:0/1522314447 learned_addr learned my addr 172.21.15.7:0/1522314447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f49392bb640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4934107900 0x7f493419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f4939abc640 1 -- 172.21.15.7:0/1522314447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4934107900 msgr2=0x7f493419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f4939abc640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4934107900 0x7f493419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f4939abc640 1 -- 172.21.15.7:0/1522314447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4934197800 con 0x7f4934101900 2024-09-21T04:47:50.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.539+0000 7f4939abc640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 0x7f493419d680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f493000d8d0 tx=0x7f493000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:50.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.540+0000 7f492affd640 1 -- 172.21.15.7:0/1522314447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4930017070 con 0x7f4934101900 2024-09-21T04:47:50.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.540+0000 7f492affd640 1 -- 172.21.15.7:0/1522314447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4930010460 con 0x7f4934101900 2024-09-21T04:47:50.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.540+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4934197af0 con 0x7f4934101900 2024-09-21T04:47:50.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.540+0000 7f492affd640 1 -- 172.21.15.7:0/1522314447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f493000f5f0 con 0x7f4934101900 2024-09-21T04:47:50.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.540+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4934198050 con 0x7f4934101900 2024-09-21T04:47:50.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.541+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4934103040 con 0x7f4934101900 2024-09-21T04:47:50.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.542+0000 7f492affd640 1 -- 172.21.15.7:0/1522314447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4930009a30 con 0x7f4934101900 2024-09-21T04:47:50.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.542+0000 7f492affd640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f492007af80 0x7f492007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.542+0000 7f492affd640 1 -- 172.21.15.7:0/1522314447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f493009dda0 con 0x7f4934101900 2024-09-21T04:47:50.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.542+0000 7f49392bb640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f492007af80 0x7f492007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.543+0000 7f49392bb640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f492007af80 0x7f492007d440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4934198b50 tx=0x7f4924031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:50.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.547+0000 7f492affd640 1 -- 172.21.15.7:0/1522314447 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49300693c0 con 0x7f4934101900 2024-09-21T04:47:50.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.813+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f4934109e90 con 0x7f4934101900 2024-09-21T04:47:50.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.813+0000 7f492affd640 1 -- 172.21.15.7:0/1522314447 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f493006e270 con 0x7f4934101900 2024-09-21T04:47:50.816 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:47:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:47:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:47:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:47:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:47:50.817 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:47:50.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f492007af80 msgr2=0x7f492007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f492007af80 0x7f492007d440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4934198b50 tx=0x7f4924031400 comp rx=0 tx=0).stop 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 msgr2=0x7f493419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 0x7f493419d680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f493000d8d0 tx=0x7f493000dd90 comp rx=0 tx=0).stop 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 shutdown_connections 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f492007af80 0x7f492007d440 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4934107900 0x7f493419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 --2- 172.21.15.7:0/1522314447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4934101900 0x7f493419d680 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 >> 172.21.15.7:0/1522314447 conn(0x7f49340fd600 msgr2=0x7f49340fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 shutdown_connections 2024-09-21T04:47:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.816+0000 7f493bd20640 1 -- 172.21.15.7:0/1522314447 wait complete. 2024-09-21T04:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 -- 172.21.15.7:0/3967151403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 msgr2=0x7f558c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 --2- 172.21.15.7:0/3967151403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 0x7f558c102f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f557c009880 tx=0x7f557c02a530 comp rx=0 tx=0).stop 2024-09-21T04:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 -- 172.21.15.7:0/3967151403 shutdown_connections 2024-09-21T04:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 --2- 172.21.15.7:0/3967151403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c103490 0x7f558c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 --2- 172.21.15.7:0/3967151403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 0x7f558c102f50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 -- 172.21.15.7:0/3967151403 >> 172.21.15.7:0/3967151403 conn(0x7f558c0fc550 msgr2=0x7f558c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 -- 172.21.15.7:0/3967151403 shutdown_connections 2024-09-21T04:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 -- 172.21.15.7:0/3967151403 wait complete. 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 Processor -- start 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.901+0000 7f55923bf640 1 -- start start 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f55923bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 0x7f558c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f55923bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c103490 0x7f558c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f55923bf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10ede0 con 0x7f558c102b70 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f55923bf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10ec60 con 0x7f558c103490 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f558bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 0x7f558c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f558bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 0x7f558c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41300/0 (socket says 172.21.15.7:41300) 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f558bfff640 1 -- 172.21.15.7:0/3809248380 learned_addr learned my addr 172.21.15.7:0/3809248380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f558b7fe640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c103490 0x7f558c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f558bfff640 1 -- 172.21.15.7:0/3809248380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c103490 msgr2=0x7f558c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f558bfff640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c103490 0x7f558c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f558bfff640 1 -- 172.21.15.7:0/3809248380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c1999e0 con 0x7f558c102b70 2024-09-21T04:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.902+0000 7f558bfff640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 0x7f558c19f820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f557c004f90 tx=0x7f557c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.903+0000 7f55897fa640 1 -- 172.21.15.7:0/3809248380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f557c03f070 con 0x7f558c102b70 2024-09-21T04:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.903+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c199c70 con 0x7f558c102b70 2024-09-21T04:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.903+0000 7f55897fa640 1 -- 172.21.15.7:0/3809248380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f557c004cf0 con 0x7f558c102b70 2024-09-21T04:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.903+0000 7f55897fa640 1 -- 172.21.15.7:0/3809248380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f557c035030 con 0x7f558c102b70 2024-09-21T04:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.903+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c19a1a0 con 0x7f558c102b70 2024-09-21T04:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.904+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c06a4c0 con 0x7f558c102b70 2024-09-21T04:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.904+0000 7f55897fa640 1 -- 172.21.15.7:0/3809248380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f557c042070 con 0x7f558c102b70 2024-09-21T04:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.905+0000 7f55897fa640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f557407af40 0x7f557407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.905+0000 7f55897fa640 1 -- 172.21.15.7:0/3809248380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f557c0bebc0 con 0x7f558c102b70 2024-09-21T04:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.905+0000 7f558b7fe640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f557407af40 0x7f557407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.905+0000 7f558b7fe640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f557407af40 0x7f557407d400 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5580005fd0 tx=0x7f5580005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:50.909+0000 7f55897fa640 1 -- 172.21.15.7:0/3809248380 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f557c08a160 con 0x7f558c102b70 2024-09-21T04:47:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.135+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f558c10b3e0 con 0x7f558c102b70 2024-09-21T04:47:51.138 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:50 smithi007 ceph-mon[108238]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:51.138 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:50 smithi007 ceph-mon[108238]: pgmap v392: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:51.138 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:50 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1522314447' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.135+0000 7f55897fa640 1 -- 172.21.15.7:0/3809248380 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f557c08f010 con 0x7f558c102b70 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:47:51.139 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:47:51.140 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:47:51.141 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:47:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:47:51.142 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:47:51.142 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:47:51.142 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:47:51.142 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:47:51.142 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f557407af40 msgr2=0x7f557407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f557407af40 0x7f557407d400 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5580005fd0 tx=0x7f5580005e20 comp rx=0 tx=0).stop 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 msgr2=0x7f558c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 0x7f558c19f820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f557c004f90 tx=0x7f557c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 shutdown_connections 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f557407af40 0x7f557407d400 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f558c103490 0x7f558c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 --2- 172.21.15.7:0/3809248380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c102b70 0x7f558c19f820 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 >> 172.21.15.7:0/3809248380 conn(0x7f558c0fc550 msgr2=0x7f558c100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 shutdown_connections 2024-09-21T04:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.138+0000 7f55923bf640 1 -- 172.21.15.7:0/3809248380 wait complete. 2024-09-21T04:47:51.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.223+0000 7fccb4064640 1 -- 172.21.15.7:0/1979158730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 msgr2=0x7fccac100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.223+0000 7fccb4064640 1 --2- 172.21.15.7:0/1979158730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 0x7fccac100c60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009880 tx=0x7fcc9c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 -- 172.21.15.7:0/1979158730 shutdown_connections 2024-09-21T04:47:51.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 --2- 172.21.15.7:0/1979158730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 0x7fccac100c60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 --2- 172.21.15.7:0/1979158730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccac106800 0x7fccac106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 -- 172.21.15.7:0/1979158730 >> 172.21.15.7:0/1979158730 conn(0x7fccac0fc520 msgr2=0x7fccac0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:51.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 -- 172.21.15.7:0/1979158730 shutdown_connections 2024-09-21T04:47:51.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 -- 172.21.15.7:0/1979158730 wait complete. 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 Processor -- start 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 -- start start 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 0x7fccac19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccac106800 0x7fccac19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fccac10dcd0 con 0x7fccac100800 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.224+0000 7fccb4064640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fccac10db50 con 0x7fccac106800 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 0x7fccac19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 0x7fccac19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41310/0 (socket says 172.21.15.7:41310) 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb1e00640 1 -- 172.21.15.7:0/3414384780 learned_addr learned my addr 172.21.15.7:0/3414384780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb15ff640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccac106800 0x7fccac19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb1e00640 1 -- 172.21.15.7:0/3414384780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccac106800 msgr2=0x7fccac19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb1e00640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccac106800 0x7fccac19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb1e00640 1 -- 172.21.15.7:0/3414384780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac1977f0 con 0x7fccac100800 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb1e00640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 0x7fccac19d670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcca800d6f0 tx=0x7fcca800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fcca2ffd640 1 -- 172.21.15.7:0/3414384780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcca8017070 con 0x7fccac100800 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac197ae0 con 0x7fccac100800 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fcca2ffd640 1 -- 172.21.15.7:0/3414384780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcca800de00 con 0x7fccac100800 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fcca2ffd640 1 -- 172.21.15.7:0/3414384780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcca8004bb0 con 0x7fccac100800 2024-09-21T04:47:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.225+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac198040 con 0x7fccac100800 2024-09-21T04:47:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.226+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccac101f40 con 0x7fccac100800 2024-09-21T04:47:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.227+0000 7fcca2ffd640 1 -- 172.21.15.7:0/3414384780 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fcca8004e60 con 0x7fccac100800 2024-09-21T04:47:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.228+0000 7fcca2ffd640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcc9807f340 0x7fcc98081800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.228+0000 7fcca2ffd640 1 -- 172.21.15.7:0/3414384780 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcca809dbf0 con 0x7fccac100800 2024-09-21T04:47:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.228+0000 7fccb15ff640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcc9807f340 0x7fcc98081800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.228+0000 7fccb15ff640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcc9807f340 0x7fcc98081800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fccac198b40 tx=0x7fcc9c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:51.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.232+0000 7fcca2ffd640 1 -- 172.21.15.7:0/3414384780 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcca8069190 con 0x7fccac100800 2024-09-21T04:47:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:50 smithi053 ceph-mon[80176]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:50 smithi053 ceph-mon[80176]: pgmap v392: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:50 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1522314447' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:47:51.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.440+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fccac1982f0 con 0x7fcc9807f340 2024-09-21T04:47:51.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.442+0000 7fcca2ffd640 1 -- 172.21.15.7:0/3414384780 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fccac1982f0 con 0x7fcc9807f340 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:47:51.445 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:47:51.446 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:47:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.444+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcc9807f340 msgr2=0x7fcc98081800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.444+0000 7fccb4064640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcc9807f340 0x7fcc98081800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fccac198b40 tx=0x7fcc9c031400 comp rx=0 tx=0).stop 2024-09-21T04:47:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.444+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 msgr2=0x7fccac19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.444+0000 7fccb4064640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 0x7fccac19d670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcca800d6f0 tx=0x7fcca800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.444+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 shutdown_connections 2024-09-21T04:47:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.444+0000 7fccb4064640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcc9807f340 0x7fcc98081800 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.444+0000 7fccb4064640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccac106800 0x7fccac19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.444+0000 7fccb4064640 1 --2- 172.21.15.7:0/3414384780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccac100800 0x7fccac19d670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.445+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 >> 172.21.15.7:0/3414384780 conn(0x7fccac0fc520 msgr2=0x7fccac06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.445+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 shutdown_connections 2024-09-21T04:47:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.445+0000 7fccb4064640 1 -- 172.21.15.7:0/3414384780 wait complete. 2024-09-21T04:47:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.529+0000 7f7e3a155640 1 -- 172.21.15.7:0/1254195951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e341033e0 msgr2=0x7f7e3410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.529+0000 7f7e3a155640 1 --2- 172.21.15.7:0/1254195951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e341033e0 0x7f7e3410b8d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009880 tx=0x7f7e2802a530 comp rx=0 tx=0).stop 2024-09-21T04:47:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.529+0000 7f7e3a155640 1 -- 172.21.15.7:0/1254195951 shutdown_connections 2024-09-21T04:47:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.529+0000 7f7e3a155640 1 --2- 172.21.15.7:0/1254195951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e341033e0 0x7f7e3410b8d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.529+0000 7f7e3a155640 1 --2- 172.21.15.7:0/1254195951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e34102ac0 0x7f7e34102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.529+0000 7f7e3a155640 1 -- 172.21.15.7:0/1254195951 >> 172.21.15.7:0/1254195951 conn(0x7f7e340fc520 msgr2=0x7f7e340fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.529+0000 7f7e3a155640 1 -- 172.21.15.7:0/1254195951 shutdown_connections 2024-09-21T04:47:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.529+0000 7f7e3a155640 1 -- 172.21.15.7:0/1254195951 wait complete. 2024-09-21T04:47:51.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.530+0000 7f7e3a155640 1 Processor -- start 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.530+0000 7f7e3a155640 1 -- start start 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.530+0000 7f7e3a155640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e34102ac0 0x7f7e3419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.530+0000 7f7e3a155640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e341033e0 0x7f7e3419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.530+0000 7f7e3a155640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3410edf0 con 0x7f7e34102ac0 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.530+0000 7f7e3a155640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3410ec70 con 0x7f7e341033e0 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e34102ac0 0x7f7e3419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e34102ac0 0x7f7e3419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41318/0 (socket says 172.21.15.7:41318) 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e337fe640 1 -- 172.21.15.7:0/2399660881 learned_addr learned my addr 172.21.15.7:0/2399660881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e32ffd640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e341033e0 0x7f7e3419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e337fe640 1 -- 172.21.15.7:0/2399660881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e341033e0 msgr2=0x7f7e3419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e337fe640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e341033e0 0x7f7e3419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e337fe640 1 -- 172.21.15.7:0/2399660881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e34197800 con 0x7f7e34102ac0 2024-09-21T04:47:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e337fe640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e34102ac0 0x7f7e3419d680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400d6f0 tx=0x7f7e2400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e30ff9640 1 -- 172.21.15.7:0/2399660881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e24017070 con 0x7f7e34102ac0 2024-09-21T04:47:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e30ff9640 1 -- 172.21.15.7:0/2399660881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e2400de00 con 0x7f7e34102ac0 2024-09-21T04:47:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e34197af0 con 0x7f7e34102ac0 2024-09-21T04:47:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e30ff9640 1 -- 172.21.15.7:0/2399660881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e24004bb0 con 0x7f7e34102ac0 2024-09-21T04:47:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.531+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e34198050 con 0x7f7e34102ac0 2024-09-21T04:47:51.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.532+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3406aa90 con 0x7f7e34102ac0 2024-09-21T04:47:51.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.533+0000 7f7e30ff9640 1 -- 172.21.15.7:0/2399660881 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7e24009a30 con 0x7f7e34102ac0 2024-09-21T04:47:51.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.534+0000 7f7e30ff9640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7e1c07af90 0x7f7e1c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:47:51.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.534+0000 7f7e30ff9640 1 -- 172.21.15.7:0/2399660881 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7e2409dbc0 con 0x7f7e34102ac0 2024-09-21T04:47:51.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.534+0000 7f7e32ffd640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7e1c07af90 0x7f7e1c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:47:51.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.534+0000 7f7e32ffd640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7e1c07af90 0x7f7e1c07d450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e34198b50 tx=0x7f7e28031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:47:51.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.538+0000 7f7e30ff9640 1 -- 172.21.15.7:0/2399660881 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7e24069140 con 0x7f7e34102ac0 2024-09-21T04:47:51.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.796+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f7e34105f60 con 0x7f7e34102ac0 2024-09-21T04:47:51.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.796+0000 7f7e30ff9640 1 -- 172.21.15.7:0/2399660881 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f7e2406dff0 con 0x7f7e34102ac0 2024-09-21T04:47:51.799 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:47:51.799 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:47:51.800 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:47:51.800 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:47:51.800 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:47:51.800 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:47:51.800 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7e1c07af90 msgr2=0x7f7e1c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7e1c07af90 0x7f7e1c07d450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7e34198b50 tx=0x7f7e28031400 comp rx=0 tx=0).stop 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e34102ac0 msgr2=0x7f7e3419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e34102ac0 0x7f7e3419d680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400d6f0 tx=0x7f7e2400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 shutdown_connections 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7e1c07af90 0x7f7e1c07d450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e341033e0 0x7f7e3419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 --2- 172.21.15.7:0/2399660881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e34102ac0 0x7f7e3419d680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 >> 172.21.15.7:0/2399660881 conn(0x7f7e340fc520 msgr2=0x7f7e34108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 shutdown_connections 2024-09-21T04:47:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:47:51.799+0000 7f7e3a155640 1 -- 172.21.15.7:0/2399660881 wait complete. 2024-09-21T04:47:52.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:51 smithi007 ceph-mon[108238]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:52.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:51 smithi007 ceph-mon[108238]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:52.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:51 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3809248380' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:47:52.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:51 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2399660881' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:47:52.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:51 smithi053 ceph-mon[80176]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:51 smithi053 ceph-mon[80176]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:51 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3809248380' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:47:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:51 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2399660881' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:47:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:52 smithi007 ceph-mon[108238]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:52 smithi007 ceph-mon[108238]: pgmap v393: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:53.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:52 smithi053 ceph-mon[80176]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:47:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:52 smithi053 ceph-mon[80176]: pgmap v393: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:54 smithi053 ceph-mon[80176]: pgmap v394: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:54 smithi007 ceph-mon[108238]: pgmap v394: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:56 smithi053 ceph-mon[80176]: pgmap v395: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:56 smithi007 ceph-mon[108238]: pgmap v395: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:58 smithi007 ceph-mon[108238]: pgmap v396: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:47:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:47:59.341 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:58 smithi053 ceph-mon[80176]: pgmap v396: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:47:59.342 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:47:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:48:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:00 smithi053 ceph-mon[80176]: pgmap v397: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:00 smithi007 ceph-mon[108238]: pgmap v397: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:02 smithi053 ceph-mon[80176]: pgmap v398: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:02 smithi007 ceph-mon[108238]: pgmap v398: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:05 smithi007 ceph-mon[108238]: pgmap v399: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:05 smithi053 ceph-mon[80176]: pgmap v399: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:07 smithi053 ceph-mon[80176]: pgmap v400: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:07 smithi007 ceph-mon[108238]: pgmap v400: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:09 smithi053 ceph-mon[80176]: pgmap v401: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:09 smithi007 ceph-mon[108238]: pgmap v401: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:11 smithi007 ceph-mon[108238]: pgmap v402: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:11 smithi053 ceph-mon[80176]: pgmap v402: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:13.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:13 smithi007 ceph-mon[108238]: pgmap v403: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:13 smithi053 ceph-mon[80176]: pgmap v403: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:48:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:48:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:15 smithi053 ceph-mon[80176]: pgmap v404: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:15 smithi007 ceph-mon[108238]: pgmap v404: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:17 smithi007 ceph-mon[108238]: pgmap v405: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:17 smithi053 ceph-mon[80176]: pgmap v405: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:19 smithi053 ceph-mon[80176]: pgmap v406: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:19 smithi007 ceph-mon[108238]: pgmap v406: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:21 smithi053 ceph-mon[80176]: pgmap v407: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:21 smithi007 ceph-mon[108238]: pgmap v407: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.898+0000 7f8d543f1640 1 -- 172.21.15.7:0/1265146154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c101900 msgr2=0x7f8d4c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.898+0000 7f8d543f1640 1 --2- 172.21.15.7:0/1265146154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c101900 0x7f8d4c101d60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009880 tx=0x7f8d3c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.899+0000 7f8d543f1640 1 -- 172.21.15.7:0/1265146154 shutdown_connections 2024-09-21T04:48:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.899+0000 7f8d543f1640 1 --2- 172.21.15.7:0/1265146154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c101900 0x7f8d4c101d60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:21.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.899+0000 7f8d543f1640 1 --2- 172.21.15.7:0/1265146154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c107900 0x7f8d4c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.899+0000 7f8d543f1640 1 -- 172.21.15.7:0/1265146154 >> 172.21.15.7:0/1265146154 conn(0x7f8d4c0fd600 msgr2=0x7f8d4c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.899+0000 7f8d543f1640 1 -- 172.21.15.7:0/1265146154 shutdown_connections 2024-09-21T04:48:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.899+0000 7f8d543f1640 1 -- 172.21.15.7:0/1265146154 wait complete. 2024-09-21T04:48:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.899+0000 7f8d543f1640 1 Processor -- start 2024-09-21T04:48:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d543f1640 1 -- start start 2024-09-21T04:48:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d543f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c101900 0x7f8d4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d543f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c107900 0x7f8d4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d543f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c10edd0 con 0x7f8d4c107900 2024-09-21T04:48:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d543f1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c10ec50 con 0x7f8d4c101900 2024-09-21T04:48:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d5198c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c107900 0x7f8d4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d5198c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c107900 0x7f8d4c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54358/0 (socket says 172.21.15.7:54358) 2024-09-21T04:48:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d5198c640 1 -- 172.21.15.7:0/3829704652 learned_addr learned my addr 172.21.15.7:0/3829704652 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d5218d640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c101900 0x7f8d4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d5198c640 1 -- 172.21.15.7:0/3829704652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c101900 msgr2=0x7f8d4c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d5198c640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c101900 0x7f8d4c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.900+0000 7f8d5198c640 1 -- 172.21.15.7:0/3829704652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c074f70 con 0x7f8d4c107900 2024-09-21T04:48:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.901+0000 7f8d5198c640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c107900 0x7f8d4c074a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c031460 tx=0x7f8d3c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.901+0000 7f8d437fe640 1 -- 172.21.15.7:0/3829704652 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8d3c03f070 con 0x7f8d4c107900 2024-09-21T04:48:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.901+0000 7f8d437fe640 1 -- 172.21.15.7:0/3829704652 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d3c0035f0 con 0x7f8d4c107900 2024-09-21T04:48:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.901+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c075200 con 0x7f8d4c107900 2024-09-21T04:48:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.901+0000 7f8d437fe640 1 -- 172.21.15.7:0/3829704652 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8d3c035030 con 0x7f8d4c107900 2024-09-21T04:48:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.901+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4c1a3d60 con 0x7f8d4c107900 2024-09-21T04:48:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.902+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4c103040 con 0x7f8d4c107900 2024-09-21T04:48:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.903+0000 7f8d437fe640 1 -- 172.21.15.7:0/3829704652 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8d3c042070 con 0x7f8d4c107900 2024-09-21T04:48:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.903+0000 7f8d437fe640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d3807af30 0x7f8d3807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:21.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.904+0000 7f8d437fe640 1 -- 172.21.15.7:0/3829704652 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8d3c0beb20 con 0x7f8d4c107900 2024-09-21T04:48:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.904+0000 7f8d5218d640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d3807af30 0x7f8d3807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.904+0000 7f8d5218d640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d3807af30 0x7f8d3807d3f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d48005fd0 tx=0x7f8d48009870 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:21.908+0000 7f8d437fe640 1 -- 172.21.15.7:0/3829704652 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d3c08a020 con 0x7f8d4c107900 2024-09-21T04:48:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.116+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8d4c062630 con 0x7f8d3807af30 2024-09-21T04:48:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.117+0000 7f8d437fe640 1 -- 172.21.15.7:0/3829704652 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f8d4c062630 con 0x7f8d3807af30 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d3807af30 msgr2=0x7f8d3807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d3807af30 0x7f8d3807d3f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d48005fd0 tx=0x7f8d48009870 comp rx=0 tx=0).stop 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c107900 msgr2=0x7f8d4c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c107900 0x7f8d4c074a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c031460 tx=0x7f8d3c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 shutdown_connections 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8d3807af30 0x7f8d3807d3f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c107900 0x7f8d4c074a30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 --2- 172.21.15.7:0/3829704652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c101900 0x7f8d4c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 >> 172.21.15.7:0/3829704652 conn(0x7f8d4c0fd600 msgr2=0x7f8d4c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 shutdown_connections 2024-09-21T04:48:22.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.120+0000 7f8d543f1640 1 -- 172.21.15.7:0/3829704652 wait complete. 2024-09-21T04:48:22.132 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:48:22.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.205+0000 7fae3d277640 1 -- 172.21.15.7:0/3906754825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 msgr2=0x7fae38102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.205+0000 7fae3d277640 1 --2- 172.21.15.7:0/3906754825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 0x7fae38102f40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae28009880 tx=0x7fae2802a530 comp rx=0 tx=0).stop 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.205+0000 7fae3d277640 1 -- 172.21.15.7:0/3906754825 shutdown_connections 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.205+0000 7fae3d277640 1 --2- 172.21.15.7:0/3906754825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38103480 0x7fae3810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.205+0000 7fae3d277640 1 --2- 172.21.15.7:0/3906754825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 0x7fae38102f40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.205+0000 7fae3d277640 1 -- 172.21.15.7:0/3906754825 >> 172.21.15.7:0/3906754825 conn(0x7fae380fc520 msgr2=0x7fae380fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.205+0000 7fae3d277640 1 -- 172.21.15.7:0/3906754825 shutdown_connections 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.205+0000 7fae3d277640 1 -- 172.21.15.7:0/3906754825 wait complete. 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae3d277640 1 Processor -- start 2024-09-21T04:48:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae3d277640 1 -- start start 2024-09-21T04:48:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae3d277640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 0x7fae3819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae3d277640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38103480 0x7fae3819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae3d277640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810edd0 con 0x7fae38102b60 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae3d277640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810ec50 con 0x7fae38103480 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae36ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 0x7fae3819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae36ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 0x7fae3819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54378/0 (socket says 172.21.15.7:54378) 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.206+0000 7fae36ffd640 1 -- 172.21.15.7:0/30729827 learned_addr learned my addr 172.21.15.7:0/30729827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae367fc640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38103480 0x7fae3819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae36ffd640 1 -- 172.21.15.7:0/30729827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38103480 msgr2=0x7fae3819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae36ffd640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38103480 0x7fae3819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae36ffd640 1 -- 172.21.15.7:0/30729827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae381977c0 con 0x7fae38102b60 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae36ffd640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 0x7fae3819d640 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae28004f90 tx=0x7fae280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae1ffff640 1 -- 172.21.15.7:0/30729827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fae2803f070 con 0x7fae38102b60 2024-09-21T04:48:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae38197a50 con 0x7fae38102b60 2024-09-21T04:48:22.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae1ffff640 1 -- 172.21.15.7:0/30729827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae28004cf0 con 0x7fae38102b60 2024-09-21T04:48:22.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae1ffff640 1 -- 172.21.15.7:0/30729827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fae28035030 con 0x7fae38102b60 2024-09-21T04:48:22.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.207+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae38197f80 con 0x7fae38102b60 2024-09-21T04:48:22.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.208+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3806a9f0 con 0x7fae38102b60 2024-09-21T04:48:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.209+0000 7fae1ffff640 1 -- 172.21.15.7:0/30729827 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fae28042070 con 0x7fae38102b60 2024-09-21T04:48:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.209+0000 7fae1ffff640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fae2007af30 0x7fae2007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.209+0000 7fae1ffff640 1 -- 172.21.15.7:0/30729827 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fae280beaa0 con 0x7fae38102b60 2024-09-21T04:48:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.209+0000 7fae367fc640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fae2007af30 0x7fae2007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.210+0000 7fae367fc640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fae2007af30 0x7fae2007d3f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fae381089a0 tx=0x7fae2c00a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:22.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.214+0000 7fae1ffff640 1 -- 172.21.15.7:0/30729827 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fae2808a040 con 0x7fae38102b60 2024-09-21T04:48:22.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.422+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fae38198230 con 0x7fae2007af30 2024-09-21T04:48:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.424+0000 7fae1ffff640 1 -- 172.21.15.7:0/30729827 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fae38198230 con 0x7fae2007af30 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fae2007af30 msgr2=0x7fae2007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fae2007af30 0x7fae2007d3f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fae381089a0 tx=0x7fae2c00a2d0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 msgr2=0x7fae3819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 0x7fae3819d640 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae28004f90 tx=0x7fae280048f0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 shutdown_connections 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fae2007af30 0x7fae2007d3f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae38103480 0x7fae3819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 --2- 172.21.15.7:0/30729827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae38102b60 0x7fae3819d640 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 >> 172.21.15.7:0/30729827 conn(0x7fae380fc520 msgr2=0x7fae38103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 shutdown_connections 2024-09-21T04:48:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.426+0000 7fae3d277640 1 -- 172.21.15.7:0/30729827 wait complete. 2024-09-21T04:48:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.510+0000 7f73f9188640 1 -- 172.21.15.7:0/3353611781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41067f0 msgr2=0x7f73f4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.510+0000 7f73f9188640 1 --2- 172.21.15.7:0/3353611781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41067f0 0x7f73f4106bd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009880 tx=0x7f73e402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.511+0000 7f73f9188640 1 -- 172.21.15.7:0/3353611781 shutdown_connections 2024-09-21T04:48:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.511+0000 7f73f9188640 1 --2- 172.21.15.7:0/3353611781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f41007f0 0x7f73f4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.511+0000 7f73f9188640 1 --2- 172.21.15.7:0/3353611781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41067f0 0x7f73f4106bd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.511+0000 7f73f9188640 1 -- 172.21.15.7:0/3353611781 >> 172.21.15.7:0/3353611781 conn(0x7f73f40fc550 msgr2=0x7f73f40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.511+0000 7f73f9188640 1 -- 172.21.15.7:0/3353611781 shutdown_connections 2024-09-21T04:48:22.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.511+0000 7f73f9188640 1 -- 172.21.15.7:0/3353611781 wait complete. 2024-09-21T04:48:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f9188640 1 Processor -- start 2024-09-21T04:48:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f9188640 1 -- start start 2024-09-21T04:48:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f9188640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41007f0 0x7f73f4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f9188640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f41067f0 0x7f73f4198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f9188640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73f410dcc0 con 0x7f73f41007f0 2024-09-21T04:48:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f9188640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73f410db40 con 0x7f73f41067f0 2024-09-21T04:48:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41007f0 0x7f73f4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41007f0 0x7f73f4198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54396/0 (socket says 172.21.15.7:54396) 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.512+0000 7f73f2d9d640 1 -- 172.21.15.7:0/2742643547 learned_addr learned my addr 172.21.15.7:0/2742643547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73f259c640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f41067f0 0x7f73f4198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73f2d9d640 1 -- 172.21.15.7:0/2742643547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f41067f0 msgr2=0x7f73f4198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73f2d9d640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f41067f0 0x7f73f4198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73f2d9d640 1 -- 172.21.15.7:0/2742643547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f4199350 con 0x7f73f41007f0 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73f259c640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f41067f0 0x7f73f4198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73f2d9d640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41007f0 0x7f73f4198810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73e4031460 tx=0x7f73e40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73dbfff640 1 -- 172.21.15.7:0/2742643547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f73e403f070 con 0x7f73f41007f0 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f41995e0 con 0x7f73f41007f0 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73dbfff640 1 -- 172.21.15.7:0/2742643547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f73e4004cf0 con 0x7f73f41007f0 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73dbfff640 1 -- 172.21.15.7:0/2742643547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f73e4035030 con 0x7f73f41007f0 2024-09-21T04:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.513+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f41a1b90 con 0x7f73f41007f0 2024-09-21T04:48:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.515+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f4101f30 con 0x7f73f41007f0 2024-09-21T04:48:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.515+0000 7f73dbfff640 1 -- 172.21.15.7:0/2742643547 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f73e4042070 con 0x7f73f41007f0 2024-09-21T04:48:22.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.515+0000 7f73dbfff640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73dc07af30 0x7f73dc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.516+0000 7f73dbfff640 1 -- 172.21.15.7:0/2742643547 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f73e40beb20 con 0x7f73f41007f0 2024-09-21T04:48:22.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.516+0000 7f73f259c640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73dc07af30 0x7f73dc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.516+0000 7f73f259c640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73dc07af30 0x7f73dc07d3f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005fd0 tx=0x7f73e8005ea0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:22.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.521+0000 7f73dbfff640 1 -- 172.21.15.7:0/2742643547 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f73e408a0c0 con 0x7f73f41007f0 2024-09-21T04:48:22.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.719+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f73f4062630 con 0x7f73dc07af30 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.730+0000 7f73dbfff640 1 -- 172.21.15.7:0/2742643547 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f73f4062630 con 0x7f73dc07af30 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (10m) 4m ago 21m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (13m) 4m ago 21m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (13m) 4m ago 20m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (8m) 4m ago 21m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (8m) 4m ago 20m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (10m) 4m ago 21m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:48:22.733 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (4m) 4m ago 15m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (4m) 4m ago 16m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (15m) 4m ago 15m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (4m) 4m ago 16m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (12m) 4m ago 22m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (11m) 4m ago 20m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (9m) 4m ago 22m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (8m) 4m ago 20m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (11m) 4m ago 21m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (11m) 4m ago 20m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (8m) 4m ago 19m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (7m) 4m ago 18m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (6m) 4m ago 18m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:48:22.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (6m) 4m ago 17m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:48:22.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (5m) 4m ago 17m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:48:22.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (5m) 4m ago 17m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:48:22.735 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (10m) 4m ago 20m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.732+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73dc07af30 msgr2=0x7f73dc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73dc07af30 0x7f73dc07d3f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005fd0 tx=0x7f73e8005ea0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41007f0 msgr2=0x7f73f4198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41007f0 0x7f73f4198810 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f73e4031460 tx=0x7f73e40048f0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 shutdown_connections 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73dc07af30 0x7f73dc07d3f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f41067f0 0x7f73f4198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 --2- 172.21.15.7:0/2742643547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73f41007f0 0x7f73f4198810 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 >> 172.21.15.7:0/2742643547 conn(0x7f73f40fc550 msgr2=0x7f73f40fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 shutdown_connections 2024-09-21T04:48:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.733+0000 7f73f9188640 1 -- 172.21.15.7:0/2742643547 wait complete. 2024-09-21T04:48:22.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.816+0000 7f796d485640 1 -- 172.21.15.7:0/2709827438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968105860 msgr2=0x7f7968105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.816+0000 7f796d485640 1 --2- 172.21.15.7:0/2709827438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968105860 0x7f7968105c40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7958009880 tx=0x7f795802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.816+0000 7f796d485640 1 -- 172.21.15.7:0/2709827438 shutdown_connections 2024-09-21T04:48:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.816+0000 7f796d485640 1 --2- 172.21.15.7:0/2709827438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7968101010 0x7f7968101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.816+0000 7f796d485640 1 --2- 172.21.15.7:0/2709827438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968105860 0x7f7968105c40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.816+0000 7f796d485640 1 -- 172.21.15.7:0/2709827438 >> 172.21.15.7:0/2709827438 conn(0x7f79680fbc90 msgr2=0x7f79680fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.816+0000 7f796d485640 1 -- 172.21.15.7:0/2709827438 shutdown_connections 2024-09-21T04:48:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.816+0000 7f796d485640 1 -- 172.21.15.7:0/2709827438 wait complete. 2024-09-21T04:48:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f796d485640 1 Processor -- start 2024-09-21T04:48:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f796d485640 1 -- start start 2024-09-21T04:48:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f796d485640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968101010 0x7f796819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f796d485640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7968105860 0x7f796819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f796d485640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f796810dea0 con 0x7f7968101010 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f796d485640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f796810dd20 con 0x7f7968105860 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f7966ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968101010 0x7f796819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f7966ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968101010 0x7f796819d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54426/0 (socket says 172.21.15.7:54426) 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.817+0000 7f7966ffd640 1 -- 172.21.15.7:0/1775707424 learned_addr learned my addr 172.21.15.7:0/1775707424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.818+0000 7f79667fc640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7968105860 0x7f796819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.818+0000 7f7966ffd640 1 -- 172.21.15.7:0/1775707424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7968105860 msgr2=0x7f796819dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.818+0000 7f7966ffd640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7968105860 0x7f796819dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.818+0000 7f7966ffd640 1 -- 172.21.15.7:0/1775707424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7968197900 con 0x7f7968101010 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.818+0000 7f7966ffd640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968101010 0x7f796819d740 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7958031460 tx=0x7f79580048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.818+0000 7f794ffff640 1 -- 172.21.15.7:0/1775707424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f795803f070 con 0x7f7968101010 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.818+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7968197b90 con 0x7f7968101010 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.818+0000 7f794ffff640 1 -- 172.21.15.7:0/1775707424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7958004cf0 con 0x7f7968101010 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.819+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7968198000 con 0x7f7968101010 2024-09-21T04:48:22.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.819+0000 7f794ffff640 1 -- 172.21.15.7:0/1775707424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7958049060 con 0x7f7968101010 2024-09-21T04:48:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.820+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7968102020 con 0x7f7968101010 2024-09-21T04:48:22.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.820+0000 7f794ffff640 1 -- 172.21.15.7:0/1775707424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7958042070 con 0x7f7968101010 2024-09-21T04:48:22.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.821+0000 7f794ffff640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f795007af30 0x7f795007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:22.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.821+0000 7f794ffff640 1 -- 172.21.15.7:0/1775707424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f79580be940 con 0x7f7968101010 2024-09-21T04:48:22.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.821+0000 7f79667fc640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f795007af30 0x7f795007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:22.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.822+0000 7f79667fc640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f795007af30 0x7f795007d3f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79681051a0 tx=0x7f795c009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:22.825+0000 7f794ffff640 1 -- 172.21.15.7:0/1775707424 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7958089ee0 con 0x7f7968101010 2024-09-21T04:48:23.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.120+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7968109dc0 con 0x7f7968101010 2024-09-21T04:48:23.124 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:23 smithi007 ceph-mon[108238]: pgmap v408: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:23.124 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:23 smithi007 ceph-mon[108238]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:23.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.121+0000 7f794ffff640 1 -- 172.21.15.7:0/1775707424 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f795808ed90 con 0x7f7968101010 2024-09-21T04:48:23.124 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:48:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:48:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:48:23.126 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:48:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f795007af30 msgr2=0x7f795007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f795007af30 0x7f795007d3f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79681051a0 tx=0x7f795c009110 comp rx=0 tx=0).stop 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968101010 msgr2=0x7f796819d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968101010 0x7f796819d740 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7958031460 tx=0x7f79580048f0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 shutdown_connections 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f795007af30 0x7f795007d3f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7968105860 0x7f796819dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 --2- 172.21.15.7:0/1775707424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7968101010 0x7f796819d740 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.123+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 >> 172.21.15.7:0/1775707424 conn(0x7f79680fbc90 msgr2=0x7f7968100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.124+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 shutdown_connections 2024-09-21T04:48:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.124+0000 7f796d485640 1 -- 172.21.15.7:0/1775707424 wait complete. 2024-09-21T04:48:23.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 -- 172.21.15.7:0/3083092080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941067f0 msgr2=0x7fe794106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 --2- 172.21.15.7:0/3083092080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941067f0 0x7fe794106bd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe784009880 tx=0x7fe78402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 -- 172.21.15.7:0/3083092080 shutdown_connections 2024-09-21T04:48:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 --2- 172.21.15.7:0/3083092080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7941007f0 0x7fe794100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 --2- 172.21.15.7:0/3083092080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941067f0 0x7fe794106bd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 -- 172.21.15.7:0/3083092080 >> 172.21.15.7:0/3083092080 conn(0x7fe7940fc550 msgr2=0x7fe7940fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 -- 172.21.15.7:0/3083092080 shutdown_connections 2024-09-21T04:48:23.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 -- 172.21.15.7:0/3083092080 wait complete. 2024-09-21T04:48:23.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 Processor -- start 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 -- start start 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.208+0000 7fe79abcb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941007f0 0x7fe794196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe79abcb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7941067f0 0x7fe794196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe79abcb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410dcc0 con 0x7fe7941007f0 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe79abcb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410db40 con 0x7fe7941067f0 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe798967640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941007f0 0x7fe794196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe798967640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941007f0 0x7fe794196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54450/0 (socket says 172.21.15.7:54450) 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe798967640 1 -- 172.21.15.7:0/3488936855 learned_addr learned my addr 172.21.15.7:0/3488936855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe793fff640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7941067f0 0x7fe794196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe798967640 1 -- 172.21.15.7:0/3488936855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7941067f0 msgr2=0x7fe794196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe798967640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7941067f0 0x7fe794196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe798967640 1 -- 172.21.15.7:0/3488936855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe794197080 con 0x7fe7941007f0 2024-09-21T04:48:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe798967640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941007f0 0x7fe794196600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe784031460 tx=0x7fe7840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe791ffb640 1 -- 172.21.15.7:0/3488936855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe78403f070 con 0x7fe7941007f0 2024-09-21T04:48:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe794197310 con 0x7fe7941007f0 2024-09-21T04:48:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe791ffb640 1 -- 172.21.15.7:0/3488936855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7840035f0 con 0x7fe7941007f0 2024-09-21T04:48:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.209+0000 7fe791ffb640 1 -- 172.21.15.7:0/3488936855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe784035030 con 0x7fe7941007f0 2024-09-21T04:48:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.210+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7941a3e40 con 0x7fe7941007f0 2024-09-21T04:48:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.211+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe794101f30 con 0x7fe7941007f0 2024-09-21T04:48:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.211+0000 7fe791ffb640 1 -- 172.21.15.7:0/3488936855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe784042070 con 0x7fe7941007f0 2024-09-21T04:48:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.212+0000 7fe791ffb640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe77c07af80 0x7fe77c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.212+0000 7fe791ffb640 1 -- 172.21.15.7:0/3488936855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe7840be9d0 con 0x7fe7941007f0 2024-09-21T04:48:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.212+0000 7fe793fff640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe77c07af80 0x7fe77c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.212+0000 7fe793fff640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe77c07af80 0x7fe77c07d440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe788005fd0 tx=0x7fe788005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:23.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.216+0000 7fe791ffb640 1 -- 172.21.15.7:0/3488936855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe784089f90 con 0x7fe7941007f0 2024-09-21T04:48:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:23 smithi053 ceph-mon[80176]: pgmap v408: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:23 smithi053 ceph-mon[80176]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:23.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.444+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe794108db0 con 0x7fe7941007f0 2024-09-21T04:48:23.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.444+0000 7fe791ffb640 1 -- 172.21.15.7:0/3488936855 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7fe794108db0 con 0x7fe7941007f0 2024-09-21T04:48:23.447 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:48:23.447 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:48:23.447 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:48:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:48:23.449 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:48:23.450 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:48:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:48:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe77c07af80 msgr2=0x7fe77c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe77c07af80 0x7fe77c07d440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe788005fd0 tx=0x7fe788005850 comp rx=0 tx=0).stop 2024-09-21T04:48:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941007f0 msgr2=0x7fe794196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941007f0 0x7fe794196600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe784031460 tx=0x7fe7840045e0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 shutdown_connections 2024-09-21T04:48:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe77c07af80 0x7fe77c07d440 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7941067f0 0x7fe794196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 --2- 172.21.15.7:0/3488936855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7941007f0 0x7fe794196600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 >> 172.21.15.7:0/3488936855 conn(0x7fe7940fc550 msgr2=0x7fe7940fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 shutdown_connections 2024-09-21T04:48:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.447+0000 7fe79abcb640 1 -- 172.21.15.7:0/3488936855 wait complete. 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.533+0000 7f067d9d3640 1 -- 172.21.15.7:0/2213054720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678101900 msgr2=0x7f0678101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.533+0000 7f067d9d3640 1 --2- 172.21.15.7:0/2213054720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678101900 0x7f0678101d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f066c009880 tx=0x7f066c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.533+0000 7f067d9d3640 1 -- 172.21.15.7:0/2213054720 shutdown_connections 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.533+0000 7f067d9d3640 1 --2- 172.21.15.7:0/2213054720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678101900 0x7f0678101d60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.533+0000 7f067d9d3640 1 --2- 172.21.15.7:0/2213054720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678107900 0x7f0678107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.533+0000 7f067d9d3640 1 -- 172.21.15.7:0/2213054720 >> 172.21.15.7:0/2213054720 conn(0x7f06780fd600 msgr2=0x7f06780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.533+0000 7f067d9d3640 1 -- 172.21.15.7:0/2213054720 shutdown_connections 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.533+0000 7f067d9d3640 1 -- 172.21.15.7:0/2213054720 wait complete. 2024-09-21T04:48:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f067d9d3640 1 Processor -- start 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f067d9d3640 1 -- start start 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f067d9d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678101900 0x7f067819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f067d9d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678107900 0x7f067819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f067d9d3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f067810ee30 con 0x7f0678107900 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f067d9d3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f067810ecb0 con 0x7f0678101900 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f06767fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678107900 0x7f067819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f06767fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678107900 0x7f067819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40428/0 (socket says 172.21.15.7:40428) 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f06767fc640 1 -- 172.21.15.7:0/1017956820 learned_addr learned my addr 172.21.15.7:0/1017956820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.534+0000 7f0676ffd640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678101900 0x7f067819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f06767fc640 1 -- 172.21.15.7:0/1017956820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678101900 msgr2=0x7f067819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f06767fc640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678101900 0x7f067819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f06767fc640 1 -- 172.21.15.7:0/1017956820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0678197840 con 0x7f0678107900 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f06767fc640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678107900 0x7f067819dc00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f066c031460 tx=0x7f066c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f067c9d1640 1 -- 172.21.15.7:0/1017956820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f066c03f070 con 0x7f0678107900 2024-09-21T04:48:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0678197ad0 con 0x7f0678107900 2024-09-21T04:48:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f067c9d1640 1 -- 172.21.15.7:0/1017956820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f066c004cf0 con 0x7f0678107900 2024-09-21T04:48:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f067c9d1640 1 -- 172.21.15.7:0/1017956820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f066c035030 con 0x7f0678107900 2024-09-21T04:48:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.535+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0678198000 con 0x7f0678107900 2024-09-21T04:48:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.536+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0678103040 con 0x7f0678107900 2024-09-21T04:48:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.537+0000 7f067c9d1640 1 -- 172.21.15.7:0/1017956820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f066c042070 con 0x7f0678107900 2024-09-21T04:48:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.537+0000 7f067c9d1640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f066007af30 0x7f066007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.538+0000 7f067c9d1640 1 -- 172.21.15.7:0/1017956820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f066c0beb60 con 0x7f0678107900 2024-09-21T04:48:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.538+0000 7f0676ffd640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f066007af30 0x7f066007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.538+0000 7f0676ffd640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f066007af30 0x7f066007d3f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0668009890 tx=0x7f0668008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.542+0000 7f067c9d1640 1 -- 172.21.15.7:0/1017956820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f066c08a100 con 0x7f0678107900 2024-09-21T04:48:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.746+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0678198370 con 0x7f066007af30 2024-09-21T04:48:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.747+0000 7f067c9d1640 1 -- 172.21.15.7:0/1017956820 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0678198370 con 0x7f066007af30 2024-09-21T04:48:23.750 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:48:23.750 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:48:23.750 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:48:23.750 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:48:23.750 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:48:23.751 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f066007af30 msgr2=0x7f066007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f066007af30 0x7f066007d3f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0668009890 tx=0x7f0668008040 comp rx=0 tx=0).stop 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678107900 msgr2=0x7f067819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678107900 0x7f067819dc00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f066c031460 tx=0x7f066c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 shutdown_connections 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f066007af30 0x7f066007d3f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0678107900 0x7f067819dc00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 --2- 172.21.15.7:0/1017956820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0678101900 0x7f067819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 >> 172.21.15.7:0/1017956820 conn(0x7f06780fd600 msgr2=0x7f06780fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 shutdown_connections 2024-09-21T04:48:23.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.750+0000 7f067d9d3640 1 -- 172.21.15.7:0/1017956820 wait complete. 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 -- 172.21.15.7:0/96917870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81007f0 msgr2=0x7f74a8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 --2- 172.21.15.7:0/96917870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81007f0 0x7f74a8100c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f749c009880 tx=0x7f749c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 -- 172.21.15.7:0/96917870 shutdown_connections 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 --2- 172.21.15.7:0/96917870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81007f0 0x7f74a8100c50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 --2- 172.21.15.7:0/96917870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74a81067f0 0x7f74a8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 -- 172.21.15.7:0/96917870 >> 172.21.15.7:0/96917870 conn(0x7f74a80fc550 msgr2=0x7f74a80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 -- 172.21.15.7:0/96917870 shutdown_connections 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 -- 172.21.15.7:0/96917870 wait complete. 2024-09-21T04:48:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 Processor -- start 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.834+0000 7f74b062d640 1 -- start start 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74b062d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74a81007f0 0x7f74a819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74b062d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81067f0 0x7f74a819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74b062d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74a810dcc0 con 0x7f74a81067f0 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74b062d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74a810db40 con 0x7f74a81007f0 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74adbc8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81067f0 0x7f74a819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74adbc8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81067f0 0x7f74a819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40444/0 (socket says 172.21.15.7:40444) 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74adbc8640 1 -- 172.21.15.7:0/211675517 learned_addr learned my addr 172.21.15.7:0/211675517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74ae3c9640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74a81007f0 0x7f74a819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74adbc8640 1 -- 172.21.15.7:0/211675517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74a81007f0 msgr2=0x7f74a819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74adbc8640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74a81007f0 0x7f74a819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74adbc8640 1 -- 172.21.15.7:0/211675517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a81966b0 con 0x7f74a81067f0 2024-09-21T04:48:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74adbc8640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81067f0 0x7f74a819cb20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f749c031460 tx=0x7f749c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:23.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74a77fe640 1 -- 172.21.15.7:0/211675517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f749c03f070 con 0x7f74a81067f0 2024-09-21T04:48:23.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74a77fe640 1 -- 172.21.15.7:0/211675517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f749c004cf0 con 0x7f74a81067f0 2024-09-21T04:48:23.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a8196940 con 0x7f74a81067f0 2024-09-21T04:48:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.835+0000 7f74a77fe640 1 -- 172.21.15.7:0/211675517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f749c035030 con 0x7f74a81067f0 2024-09-21T04:48:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.836+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a8196e40 con 0x7f74a81067f0 2024-09-21T04:48:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.836+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a8101f30 con 0x7f74a81067f0 2024-09-21T04:48:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.837+0000 7f74a77fe640 1 -- 172.21.15.7:0/211675517 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f749c042070 con 0x7f74a81067f0 2024-09-21T04:48:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.838+0000 7f74a77fe640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f749007af30 0x7f749007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.838+0000 7f74a77fe640 1 -- 172.21.15.7:0/211675517 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f749c0beb20 con 0x7f74a81067f0 2024-09-21T04:48:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.838+0000 7f74ae3c9640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f749007af30 0x7f749007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.838+0000 7f74ae3c9640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f749007af30 0x7f749007d3f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7498005c40 tx=0x7f749800a390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:23.842+0000 7f74a77fe640 1 -- 172.21.15.7:0/211675517 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f749c08a0c0 con 0x7f74a81067f0 2024-09-21T04:48:24.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.106+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f74a8197be0 con 0x7f74a81067f0 2024-09-21T04:48:24.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.106+0000 7f74a77fe640 1 -- 172.21.15.7:0/211675517 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f749c08ef70 con 0x7f74a81067f0 2024-09-21T04:48:24.109 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:48:24.109 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:48:24.109 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:48:24.109 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:48:24.109 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:48:24.110 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:48:24.110 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:48:24.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f749007af30 msgr2=0x7f749007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:24.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f749007af30 0x7f749007d3f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7498005c40 tx=0x7f749800a390 comp rx=0 tx=0).stop 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81067f0 msgr2=0x7f74a819cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81067f0 0x7f74a819cb20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f749c031460 tx=0x7f749c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 shutdown_connections 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f749007af30 0x7f749007d3f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a81067f0 0x7f74a819cb20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 --2- 172.21.15.7:0/211675517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74a81007f0 0x7f74a819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.108+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 >> 172.21.15.7:0/211675517 conn(0x7f74a80fc550 msgr2=0x7f74a80fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.109+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 shutdown_connections 2024-09-21T04:48:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:24.109+0000 7f74b062d640 1 -- 172.21.15.7:0/211675517 wait complete. 2024-09-21T04:48:24.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:24 smithi053 ceph-mon[80176]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:24 smithi053 ceph-mon[80176]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:24 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1775707424' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:48:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:24 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3488936855' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:48:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:24 smithi007 ceph-mon[108238]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:24 smithi007 ceph-mon[108238]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:24 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1775707424' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:48:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:24 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3488936855' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:48:25.347 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:25 smithi007 ceph-mon[108238]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:25.347 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:25 smithi007 ceph-mon[108238]: pgmap v409: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:25.347 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:25 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/211675517' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:48:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:25 smithi053 ceph-mon[80176]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:25 smithi053 ceph-mon[80176]: pgmap v409: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:25 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/211675517' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:48:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:27 smithi053 ceph-mon[80176]: pgmap v410: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:27 smithi007 ceph-mon[108238]: pgmap v410: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:29 smithi007 ceph-mon[108238]: pgmap v411: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:48:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:29 smithi053 ceph-mon[80176]: pgmap v411: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:48:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:31 smithi053 ceph-mon[80176]: pgmap v412: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:31 smithi007 ceph-mon[108238]: pgmap v412: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:33 smithi053 ceph-mon[80176]: pgmap v413: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:33 smithi007 ceph-mon[108238]: pgmap v413: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:35 smithi053 ceph-mon[80176]: pgmap v414: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:35 smithi007 ceph-mon[108238]: pgmap v414: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:37 smithi053 ceph-mon[80176]: pgmap v415: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:37 smithi007 ceph-mon[108238]: pgmap v415: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:39 smithi053 ceph-mon[80176]: pgmap v416: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:39 smithi007 ceph-mon[108238]: pgmap v416: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:41 smithi007 ceph-mon[108238]: pgmap v417: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:41 smithi053 ceph-mon[80176]: pgmap v417: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:43.358 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:43 smithi007 ceph-mon[108238]: pgmap v418: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:43 smithi053 ceph-mon[80176]: pgmap v418: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:48:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:48:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:45 smithi053 ceph-mon[80176]: pgmap v419: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:45 smithi007 ceph-mon[108238]: pgmap v419: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:47 smithi053 ceph-mon[80176]: pgmap v420: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:47 smithi007 ceph-mon[108238]: pgmap v420: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:49 smithi053 ceph-mon[80176]: pgmap v421: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:49 smithi007 ceph-mon[108238]: pgmap v421: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:51.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:51 smithi053 ceph-mon[80176]: pgmap v422: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:51 smithi007 ceph-mon[108238]: pgmap v422: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:53 smithi007 ceph-mon[108238]: pgmap v423: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:53 smithi053 ceph-mon[80176]: pgmap v423: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.206+0000 7f889f6c4640 1 -- 172.21.15.7:0/218746552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100250 msgr2=0x7f8898100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.206+0000 7f889f6c4640 1 --2- 172.21.15.7:0/218746552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100250 0x7f8898100630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8894009880 tx=0x7f889402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.206+0000 7f889f6c4640 1 -- 172.21.15.7:0/218746552 shutdown_connections 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.206+0000 7f889f6c4640 1 --2- 172.21.15.7:0/218746552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100c00 0x7f8898104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.206+0000 7f889f6c4640 1 --2- 172.21.15.7:0/218746552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100250 0x7f8898100630 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.206+0000 7f889f6c4640 1 -- 172.21.15.7:0/218746552 >> 172.21.15.7:0/218746552 conn(0x7f88980fbc90 msgr2=0x7f88980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.206+0000 7f889f6c4640 1 -- 172.21.15.7:0/218746552 shutdown_connections 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.206+0000 7f889f6c4640 1 -- 172.21.15.7:0/218746552 wait complete. 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.207+0000 7f889f6c4640 1 Processor -- start 2024-09-21T04:48:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.207+0000 7f889f6c4640 1 -- start start 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889f6c4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100250 0x7f8898196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889f6c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100c00 0x7f8898196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889f6c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f889810dd50 con 0x7f8898100c00 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889f6c4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f889810dbd0 con 0x7f8898100250 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889cc5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100c00 0x7f8898196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889cc5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100c00 0x7f8898196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33622/0 (socket says 172.21.15.7:33622) 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889cc5f640 1 -- 172.21.15.7:0/172106645 learned_addr learned my addr 172.21.15.7:0/172106645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889d460640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100250 0x7f8898196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889cc5f640 1 -- 172.21.15.7:0/172106645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100250 msgr2=0x7f8898196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889cc5f640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100250 0x7f8898196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889cc5f640 1 -- 172.21.15.7:0/172106645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8898197110 con 0x7f8898100c00 2024-09-21T04:48:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.208+0000 7f889cc5f640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100c00 0x7f8898196bd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f888800d6f0 tx=0x7f888800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.209+0000 7f888e7fc640 1 -- 172.21.15.7:0/172106645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8888017070 con 0x7f8898100c00 2024-09-21T04:48:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.209+0000 7f888e7fc640 1 -- 172.21.15.7:0/172106645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f888800de00 con 0x7f8898100c00 2024-09-21T04:48:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.209+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8898197400 con 0x7f8898100c00 2024-09-21T04:48:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.209+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88981a3ed0 con 0x7f8898100c00 2024-09-21T04:48:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.209+0000 7f888e7fc640 1 -- 172.21.15.7:0/172106645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8888004bb0 con 0x7f8898100c00 2024-09-21T04:48:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.210+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8898101730 con 0x7f8898100c00 2024-09-21T04:48:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.210+0000 7f888e7fc640 1 -- 172.21.15.7:0/172106645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8888004e60 con 0x7f8898100c00 2024-09-21T04:48:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.211+0000 7f888e7fc640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888407af30 0x7f888407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.211+0000 7f889d460640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888407af30 0x7f888407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.211+0000 7f888e7fc640 1 -- 172.21.15.7:0/172106645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f888809dc10 con 0x7f8898100c00 2024-09-21T04:48:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.212+0000 7f889d460640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888407af30 0x7f888407d3f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f88940096c0 tx=0x7f8894031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.216+0000 7f888e7fc640 1 -- 172.21.15.7:0/172106645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88880691b0 con 0x7f8898100c00 2024-09-21T04:48:54.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.419+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8898062630 con 0x7f888407af30 2024-09-21T04:48:54.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.421+0000 7f888e7fc640 1 -- 172.21.15.7:0/172106645 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f8898062630 con 0x7f888407af30 2024-09-21T04:48:54.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888407af30 msgr2=0x7f888407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888407af30 0x7f888407d3f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f88940096c0 tx=0x7f8894031400 comp rx=0 tx=0).stop 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100c00 msgr2=0x7f8898196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100c00 0x7f8898196bd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f888800d6f0 tx=0x7f888800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 shutdown_connections 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f888407af30 0x7f888407d3f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8898100c00 0x7f8898196bd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 --2- 172.21.15.7:0/172106645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8898100250 0x7f8898196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 >> 172.21.15.7:0/172106645 conn(0x7f88980fbc90 msgr2=0x7f88980fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 shutdown_connections 2024-09-21T04:48:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.423+0000 7f889f6c4640 1 -- 172.21.15.7:0/172106645 wait complete. 2024-09-21T04:48:54.435 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:48:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.508+0000 7f639be14640 1 -- 172.21.15.7:0/2232869019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63941033e0 msgr2=0x7f639410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.508+0000 7f639be14640 1 --2- 172.21.15.7:0/2232869019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63941033e0 0x7f639410b8d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6384009880 tx=0x7f638402a530 comp rx=0 tx=0).stop 2024-09-21T04:48:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.508+0000 7f639be14640 1 -- 172.21.15.7:0/2232869019 shutdown_connections 2024-09-21T04:48:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.508+0000 7f639be14640 1 --2- 172.21.15.7:0/2232869019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63941033e0 0x7f639410b8d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.508+0000 7f639be14640 1 --2- 172.21.15.7:0/2232869019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6394102ac0 0x7f6394102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.508+0000 7f639be14640 1 -- 172.21.15.7:0/2232869019 >> 172.21.15.7:0/2232869019 conn(0x7f63940fc520 msgr2=0x7f63940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:54.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.508+0000 7f639be14640 1 -- 172.21.15.7:0/2232869019 shutdown_connections 2024-09-21T04:48:54.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.508+0000 7f639be14640 1 -- 172.21.15.7:0/2232869019 wait complete. 2024-09-21T04:48:54.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.509+0000 7f639be14640 1 Processor -- start 2024-09-21T04:48:54.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.509+0000 7f639be14640 1 -- start start 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f639be14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6394102ac0 0x7f639419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f639be14640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63941033e0 0x7f639419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f639be14640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f639410edf0 con 0x7f6394102ac0 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f639be14640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f639410ec70 con 0x7f63941033e0 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f6399bb0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6394102ac0 0x7f639419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f6399bb0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6394102ac0 0x7f639419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33634/0 (socket says 172.21.15.7:33634) 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f6399bb0640 1 -- 172.21.15.7:0/2596617460 learned_addr learned my addr 172.21.15.7:0/2596617460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f63993af640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63941033e0 0x7f639419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f6399bb0640 1 -- 172.21.15.7:0/2596617460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63941033e0 msgr2=0x7f639419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f6399bb0640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63941033e0 0x7f639419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f6399bb0640 1 -- 172.21.15.7:0/2596617460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6394197820 con 0x7f6394102ac0 2024-09-21T04:48:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f6399bb0640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6394102ac0 0x7f639419d6a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f639000c8f0 tx=0x7f639000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f638affd640 1 -- 172.21.15.7:0/2596617460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f63900079b0 con 0x7f6394102ac0 2024-09-21T04:48:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.510+0000 7f638affd640 1 -- 172.21.15.7:0/2596617460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63900054e0 con 0x7f6394102ac0 2024-09-21T04:48:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.511+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6394197b10 con 0x7f6394102ac0 2024-09-21T04:48:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.511+0000 7f638affd640 1 -- 172.21.15.7:0/2596617460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6390004dd0 con 0x7f6394102ac0 2024-09-21T04:48:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.511+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6394198070 con 0x7f6394102ac0 2024-09-21T04:48:54.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.512+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639406aa90 con 0x7f6394102ac0 2024-09-21T04:48:54.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.512+0000 7f638affd640 1 -- 172.21.15.7:0/2596617460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6390005020 con 0x7f6394102ac0 2024-09-21T04:48:54.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.513+0000 7f638affd640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f638007aed0 0x7f638007d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.513+0000 7f638affd640 1 -- 172.21.15.7:0/2596617460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f639009dfd0 con 0x7f6394102ac0 2024-09-21T04:48:54.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.513+0000 7f63993af640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f638007aed0 0x7f638007d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.514+0000 7f63993af640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f638007aed0 0x7f638007d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6394198b70 tx=0x7f6384031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:54.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.518+0000 7f638affd640 1 -- 172.21.15.7:0/2596617460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6390014030 con 0x7f6394102ac0 2024-09-21T04:48:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.722+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6394198380 con 0x7f638007aed0 2024-09-21T04:48:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.723+0000 7f638affd640 1 -- 172.21.15.7:0/2596617460 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f6394198380 con 0x7f638007aed0 2024-09-21T04:48:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.725+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f638007aed0 msgr2=0x7f638007d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.725+0000 7f639be14640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f638007aed0 0x7f638007d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6394198b70 tx=0x7f6384031400 comp rx=0 tx=0).stop 2024-09-21T04:48:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.725+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6394102ac0 msgr2=0x7f639419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.725+0000 7f639be14640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6394102ac0 0x7f639419d6a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f639000c8f0 tx=0x7f639000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.725+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 shutdown_connections 2024-09-21T04:48:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.726+0000 7f639be14640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f638007aed0 0x7f638007d390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.726+0000 7f639be14640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63941033e0 0x7f639419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.726+0000 7f639be14640 1 --2- 172.21.15.7:0/2596617460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6394102ac0 0x7f639419d6a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.726+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 >> 172.21.15.7:0/2596617460 conn(0x7f63940fc520 msgr2=0x7f6394108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.726+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 shutdown_connections 2024-09-21T04:48:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.726+0000 7f639be14640 1 -- 172.21.15.7:0/2596617460 wait complete. 2024-09-21T04:48:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.811+0000 7f1a11e42640 1 -- 172.21.15.7:0/599566123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 msgr2=0x7f1a0c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.811+0000 7f1a11e42640 1 --2- 172.21.15.7:0/599566123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 0x7f1a0c100c60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009880 tx=0x7f1a0002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.811+0000 7f1a11e42640 1 -- 172.21.15.7:0/599566123 shutdown_connections 2024-09-21T04:48:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.811+0000 7f1a11e42640 1 --2- 172.21.15.7:0/599566123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 0x7f1a0c100c60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.811+0000 7f1a11e42640 1 --2- 172.21.15.7:0/599566123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a0c106800 0x7f1a0c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.811+0000 7f1a11e42640 1 -- 172.21.15.7:0/599566123 >> 172.21.15.7:0/599566123 conn(0x7f1a0c0fc520 msgr2=0x7f1a0c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.811+0000 7f1a11e42640 1 -- 172.21.15.7:0/599566123 shutdown_connections 2024-09-21T04:48:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.811+0000 7f1a11e42640 1 -- 172.21.15.7:0/599566123 wait complete. 2024-09-21T04:48:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a11e42640 1 Processor -- start 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a11e42640 1 -- start start 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a11e42640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 0x7f1a0c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a11e42640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a0c106800 0x7f1a0c19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a11e42640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c10dcb0 con 0x7f1a0c100800 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a11e42640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c10db30 con 0x7f1a0c106800 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 0x7f1a0c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 0x7f1a0c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33650/0 (socket says 172.21.15.7:33650) 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a0b7fe640 1 -- 172.21.15.7:0/1650588822 learned_addr learned my addr 172.21.15.7:0/1650588822 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a0affd640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a0c106800 0x7f1a0c19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a0b7fe640 1 -- 172.21.15.7:0/1650588822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a0c106800 msgr2=0x7f1a0c19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a0b7fe640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a0c106800 0x7f1a0c19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.812+0000 7f1a0b7fe640 1 -- 172.21.15.7:0/1650588822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c1977f0 con 0x7f1a0c100800 2024-09-21T04:48:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.813+0000 7f1a0b7fe640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 0x7f1a0c19d670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00d8d0 tx=0x7f19fc00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.813+0000 7f1a08ff9640 1 -- 172.21.15.7:0/1650588822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f19fc017030 con 0x7f1a0c100800 2024-09-21T04:48:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.813+0000 7f1a08ff9640 1 -- 172.21.15.7:0/1650588822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f19fc004020 con 0x7f1a0c100800 2024-09-21T04:48:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.813+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c197ae0 con 0x7f1a0c100800 2024-09-21T04:48:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.813+0000 7f1a08ff9640 1 -- 172.21.15.7:0/1650588822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f19fc0047a0 con 0x7f1a0c100800 2024-09-21T04:48:54.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.813+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c198040 con 0x7f1a0c100800 2024-09-21T04:48:54.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.814+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0c101f40 con 0x7f1a0c100800 2024-09-21T04:48:54.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.815+0000 7f1a08ff9640 1 -- 172.21.15.7:0/1650588822 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f19fc0041d0 con 0x7f1a0c100800 2024-09-21T04:48:54.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.815+0000 7f1a08ff9640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f19f407f340 0x7f19f4081800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:54.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.815+0000 7f1a08ff9640 1 -- 172.21.15.7:0/1650588822 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f19fc018070 con 0x7f1a0c100800 2024-09-21T04:48:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.816+0000 7f1a0affd640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f19f407f340 0x7f19f4081800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.816+0000 7f1a0affd640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f19f407f340 0x7f19f4081800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c198b40 tx=0x7f1a00031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:54.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:54.820+0000 7f1a08ff9640 1 -- 172.21.15.7:0/1650588822 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19fc069420 con 0x7f1a0c100800 2024-09-21T04:48:55.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.016+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1a0c198350 con 0x7f19f407f340 2024-09-21T04:48:55.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.026+0000 7f1a08ff9640 1 -- 172.21.15.7:0/1650588822 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f1a0c198350 con 0x7f19f407f340 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (10m) 4m ago 21m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (14m) 4m ago 22m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (13m) 4m ago 21m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (9m) 4m ago 22m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (9m) 4m ago 21m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (10m) 4m ago 21m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (5m) 4m ago 16m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (5m) 4m ago 16m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (16m) 4m ago 16m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (4m) 4m ago 16m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (13m) 4m ago 23m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:48:55.029 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (12m) 4m ago 21m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (9m) 4m ago 23m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (9m) 4m ago 20m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (11m) 4m ago 21m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (11m) 4m ago 21m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (8m) 4m ago 19m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (8m) 4m ago 19m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (7m) 4m ago 18m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (6m) 4m ago 18m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (6m) 4m ago 18m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (5m) 4m ago 17m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:48:55.030 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (11m) 4m ago 21m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:48:55.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f19f407f340 msgr2=0x7f19f4081800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f19f407f340 0x7f19f4081800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c198b40 tx=0x7f1a00031400 comp rx=0 tx=0).stop 2024-09-21T04:48:55.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 msgr2=0x7f1a0c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 0x7f1a0c19d670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00d8d0 tx=0x7f19fc00dd90 comp rx=0 tx=0).stop 2024-09-21T04:48:55.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 shutdown_connections 2024-09-21T04:48:55.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f19f407f340 0x7f19f4081800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a0c106800 0x7f1a0c19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 --2- 172.21.15.7:0/1650588822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a0c100800 0x7f1a0c19d670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 >> 172.21.15.7:0/1650588822 conn(0x7f1a0c0fc520 msgr2=0x7f1a0c06a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 shutdown_connections 2024-09-21T04:48:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.028+0000 7f1a11e42640 1 -- 172.21.15.7:0/1650588822 wait complete. 2024-09-21T04:48:55.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.112+0000 7f3c10bcc640 1 -- 172.21.15.7:0/224878689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 msgr2=0x7f3c0c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.112+0000 7f3c10bcc640 1 --2- 172.21.15.7:0/224878689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 0x7f3c0c10b8c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009880 tx=0x7f3bf402a530 comp rx=0 tx=0).stop 2024-09-21T04:48:55.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.113+0000 7f3c10bcc640 1 -- 172.21.15.7:0/224878689 shutdown_connections 2024-09-21T04:48:55.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.113+0000 7f3c10bcc640 1 --2- 172.21.15.7:0/224878689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 0x7f3c0c10b8c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.113+0000 7f3c10bcc640 1 --2- 172.21.15.7:0/224878689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c0c102ab0 0x7f3c0c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.113+0000 7f3c10bcc640 1 -- 172.21.15.7:0/224878689 >> 172.21.15.7:0/224878689 conn(0x7f3c0c0fc550 msgr2=0x7f3c0c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:55.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.113+0000 7f3c10bcc640 1 -- 172.21.15.7:0/224878689 shutdown_connections 2024-09-21T04:48:55.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.113+0000 7f3c10bcc640 1 -- 172.21.15.7:0/224878689 wait complete. 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.113+0000 7f3c10bcc640 1 Processor -- start 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.113+0000 7f3c10bcc640 1 -- start start 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c10bcc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c0c102ab0 0x7f3c0c100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c10bcc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 0x7f3c0c100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c10bcc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c10ede0 con 0x7f3c0c1033d0 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c10bcc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c10ec60 con 0x7f3c0c102ab0 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c09d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 0x7f3c0c100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c09d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 0x7f3c0c100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33654/0 (socket says 172.21.15.7:33654) 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c09d9b640 1 -- 172.21.15.7:0/931519330 learned_addr learned my addr 172.21.15.7:0/931519330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c0a59c640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c0c102ab0 0x7f3c0c100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c09d9b640 1 -- 172.21.15.7:0/931519330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c0c102ab0 msgr2=0x7f3c0c100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c09d9b640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c0c102ab0 0x7f3c0c100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c09d9b640 1 -- 172.21.15.7:0/931519330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c101190 con 0x7f3c0c1033d0 2024-09-21T04:48:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.114+0000 7f3c09d9b640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 0x7f3c0c100c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4031460 tx=0x7f3bf4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.115+0000 7f3c037fe640 1 -- 172.21.15.7:0/931519330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3bf403f070 con 0x7f3c0c1033d0 2024-09-21T04:48:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.115+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c101420 con 0x7f3c0c1033d0 2024-09-21T04:48:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.115+0000 7f3c037fe640 1 -- 172.21.15.7:0/931519330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3bf40035f0 con 0x7f3c0c1033d0 2024-09-21T04:48:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.115+0000 7f3c037fe640 1 -- 172.21.15.7:0/931519330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3bf4035030 con 0x7f3c0c1033d0 2024-09-21T04:48:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.115+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c1a3dd0 con 0x7f3c0c1033d0 2024-09-21T04:48:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.116+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c06a580 con 0x7f3c0c1033d0 2024-09-21T04:48:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.116+0000 7f3c037fe640 1 -- 172.21.15.7:0/931519330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3bf4042070 con 0x7f3c0c1033d0 2024-09-21T04:48:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.117+0000 7f3c037fe640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bf807af30 0x7f3bf807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.117+0000 7f3c0a59c640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bf807af30 0x7f3bf807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.118+0000 7f3c037fe640 1 -- 172.21.15.7:0/931519330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f3bf40be960 con 0x7f3c0c1033d0 2024-09-21T04:48:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.118+0000 7f3c0a59c640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bf807af30 0x7f3bf807d3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005fd0 tx=0x7f3bfc00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:55.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.122+0000 7f3c037fe640 1 -- 172.21.15.7:0/931519330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3bf4089f00 con 0x7f3c0c1033d0 2024-09-21T04:48:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.385+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f3c0c105f50 con 0x7f3c0c1033d0 2024-09-21T04:48:55.388 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:55 smithi007 ceph-mon[108238]: pgmap v424: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.385+0000 7f3c037fe640 1 -- 172.21.15.7:0/931519330 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f3bf408c420 con 0x7f3c0c1033d0 2024-09-21T04:48:55.388 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:48:55.389 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:48:55.390 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:48:55.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:48:55.390 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:48:55.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bf807af30 msgr2=0x7f3bf807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bf807af30 0x7f3bf807d3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005fd0 tx=0x7f3bfc00b390 comp rx=0 tx=0).stop 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 msgr2=0x7f3c0c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 0x7f3c0c100c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4031460 tx=0x7f3bf4004530 comp rx=0 tx=0).stop 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 shutdown_connections 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bf807af30 0x7f3bf807d3f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c1033d0 0x7f3c0c100c50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 --2- 172.21.15.7:0/931519330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3c0c102ab0 0x7f3c0c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 >> 172.21.15.7:0/931519330 conn(0x7f3c0c0fc550 msgr2=0x7f3c0c108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.388+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 shutdown_connections 2024-09-21T04:48:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.389+0000 7f3c10bcc640 1 -- 172.21.15.7:0/931519330 wait complete. 2024-09-21T04:48:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:55 smithi053 ceph-mon[80176]: pgmap v424: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.473+0000 7f67182f6640 1 -- 172.21.15.7:0/776213570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 msgr2=0x7f6710100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.473+0000 7f67182f6640 1 --2- 172.21.15.7:0/776213570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 0x7f6710100c60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6700009880 tx=0x7f670002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.473+0000 7f67182f6640 1 -- 172.21.15.7:0/776213570 shutdown_connections 2024-09-21T04:48:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.473+0000 7f67182f6640 1 --2- 172.21.15.7:0/776213570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 0x7f6710100c60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.473+0000 7f67182f6640 1 --2- 172.21.15.7:0/776213570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6710106800 0x7f6710106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.473+0000 7f67182f6640 1 -- 172.21.15.7:0/776213570 >> 172.21.15.7:0/776213570 conn(0x7f67100fc520 msgr2=0x7f67100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.473+0000 7f67182f6640 1 -- 172.21.15.7:0/776213570 shutdown_connections 2024-09-21T04:48:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.473+0000 7f67182f6640 1 -- 172.21.15.7:0/776213570 wait complete. 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f67182f6640 1 Processor -- start 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f67182f6640 1 -- start start 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f67182f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 0x7f67100fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f67182f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6710106800 0x7f67100fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f67182f6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f671010dd30 con 0x7f6710100800 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f67182f6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f671010dbb0 con 0x7f6710106800 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f6716092640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 0x7f67100fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f6716092640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 0x7f67100fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33666/0 (socket says 172.21.15.7:33666) 2024-09-21T04:48:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f6716092640 1 -- 172.21.15.7:0/2154065812 learned_addr learned my addr 172.21.15.7:0/2154065812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f6716092640 1 -- 172.21.15.7:0/2154065812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6710106800 msgr2=0x7f67100fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f6716092640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6710106800 0x7f67100fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.474+0000 7f6716092640 1 -- 172.21.15.7:0/2154065812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67100fd650 con 0x7f6710100800 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.475+0000 7f6716092640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 0x7f67100fcbd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f670c00b420 tx=0x7f670c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.475+0000 7f67077fe640 1 -- 172.21.15.7:0/2154065812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f670c011630 con 0x7f6710100800 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.475+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67100fd940 con 0x7f6710100800 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.475+0000 7f67077fe640 1 -- 172.21.15.7:0/2154065812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f670c016070 con 0x7f6710100800 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.475+0000 7f67077fe640 1 -- 172.21.15.7:0/2154065812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f670c010d40 con 0x7f6710100800 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.475+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67101a3ef0 con 0x7f6710100800 2024-09-21T04:48:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.476+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6710101f40 con 0x7f6710100800 2024-09-21T04:48:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.477+0000 7f67077fe640 1 -- 172.21.15.7:0/2154065812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f670c01b460 con 0x7f6710100800 2024-09-21T04:48:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.477+0000 7f67077fe640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66fc07af30 0x7f66fc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.477+0000 7f67077fe640 1 -- 172.21.15.7:0/2154065812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f670c09d810 con 0x7f6710100800 2024-09-21T04:48:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.477+0000 7f6715891640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66fc07af30 0x7f66fc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.478+0000 7f6715891640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66fc07af30 0x7f66fc07d3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f67000096c0 tx=0x7f6700000bc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.482+0000 7f67077fe640 1 -- 172.21.15.7:0/2154065812 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f670c021250 con 0x7f6710100800 2024-09-21T04:48:55.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.707+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6710109da0 con 0x7f6710100800 2024-09-21T04:48:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.708+0000 7f67077fe640 1 -- 172.21.15.7:0/2154065812 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f670c068db0 con 0x7f6710100800 2024-09-21T04:48:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:48:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:48:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:48:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:48:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:48:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:48:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:48:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:48:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:48:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66fc07af30 msgr2=0x7f66fc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66fc07af30 0x7f66fc07d3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f67000096c0 tx=0x7f6700000bc0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 msgr2=0x7f67100fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 0x7f67100fcbd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f670c00b420 tx=0x7f670c00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 shutdown_connections 2024-09-21T04:48:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66fc07af30 0x7f66fc07d3f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6710106800 0x7f67100fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 --2- 172.21.15.7:0/2154065812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6710100800 0x7f67100fcbd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 >> 172.21.15.7:0/2154065812 conn(0x7f67100fc520 msgr2=0x7f671006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 shutdown_connections 2024-09-21T04:48:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.710+0000 7f67182f6640 1 -- 172.21.15.7:0/2154065812 wait complete. 2024-09-21T04:48:55.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.795+0000 7f7ce90d9640 1 -- 172.21.15.7:0/593289577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 msgr2=0x7f7ce410dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.795+0000 7f7ce90d9640 1 --2- 172.21.15.7:0/593289577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 0x7f7ce410dcc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009880 tx=0x7f7ccc02a490 comp rx=0 tx=0).stop 2024-09-21T04:48:55.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.795+0000 7f7ce90d9640 1 -- 172.21.15.7:0/593289577 shutdown_connections 2024-09-21T04:48:55.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.795+0000 7f7ce90d9640 1 --2- 172.21.15.7:0/593289577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 0x7f7ce410dcc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.795+0000 7f7ce90d9640 1 --2- 172.21.15.7:0/593289577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce4100650 0x7f7ce4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.795+0000 7f7ce90d9640 1 -- 172.21.15.7:0/593289577 >> 172.21.15.7:0/593289577 conn(0x7f7ce40fc520 msgr2=0x7f7ce40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:55.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.795+0000 7f7ce90d9640 1 -- 172.21.15.7:0/593289577 shutdown_connections 2024-09-21T04:48:55.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.795+0000 7f7ce90d9640 1 -- 172.21.15.7:0/593289577 wait complete. 2024-09-21T04:48:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.796+0000 7f7ce90d9640 1 Processor -- start 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.796+0000 7f7ce90d9640 1 -- start start 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.796+0000 7f7ce90d9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce4100650 0x7f7ce419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.796+0000 7f7ce90d9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 0x7f7ce419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.796+0000 7f7ce90d9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4110ff0 con 0x7f7ce4101000 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.796+0000 7f7ce90d9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce4110e70 con 0x7f7ce4100650 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 0x7f7ce419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 0x7f7ce419fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33684/0 (socket says 172.21.15.7:33684) 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce259c640 1 -- 172.21.15.7:0/1015393269 learned_addr learned my addr 172.21.15.7:0/1015393269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce2d9d640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce4100650 0x7f7ce419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce259c640 1 -- 172.21.15.7:0/1015393269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce4100650 msgr2=0x7f7ce419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce259c640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce4100650 0x7f7ce419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce259c640 1 -- 172.21.15.7:0/1015393269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce4199a80 con 0x7f7ce4101000 2024-09-21T04:48:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce2d9d640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce4100650 0x7f7ce419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:48:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce259c640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 0x7f7ce419fe00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc031460 tx=0x7f7ccc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7cdbfff640 1 -- 172.21.15.7:0/1015393269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ccc03f070 con 0x7f7ce4101000 2024-09-21T04:48:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce4199d10 con 0x7f7ce4101000 2024-09-21T04:48:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7cdbfff640 1 -- 172.21.15.7:0/1015393269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ccc004cf0 con 0x7f7ce4101000 2024-09-21T04:48:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.797+0000 7f7cdbfff640 1 -- 172.21.15.7:0/1015393269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ccc035030 con 0x7f7ce4101000 2024-09-21T04:48:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.798+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce419a2a0 con 0x7f7ce4101000 2024-09-21T04:48:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.798+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce406aa90 con 0x7f7ce4101000 2024-09-21T04:48:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.799+0000 7f7cdbfff640 1 -- 172.21.15.7:0/1015393269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7ccc042070 con 0x7f7ce4101000 2024-09-21T04:48:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.800+0000 7f7cdbfff640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cd007af30 0x7f7cd007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.800+0000 7f7cdbfff640 1 -- 172.21.15.7:0/1015393269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7ccc0beb60 con 0x7f7ce4101000 2024-09-21T04:48:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.800+0000 7f7ce2d9d640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cd007af30 0x7f7cd007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.800+0000 7f7ce2d9d640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cd007af30 0x7f7cd007d3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ce419abc0 tx=0x7f7cd4008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:55.804+0000 7f7cdbfff640 1 -- 172.21.15.7:0/1015393269 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ccc08a100 con 0x7f7ce4101000 2024-09-21T04:48:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.023+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7ce410b410 con 0x7f7cd007af30 2024-09-21T04:48:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.024+0000 7f7cdbfff640 1 -- 172.21.15.7:0/1015393269 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f7cd400d7e0 con 0x7f7cd007af30 2024-09-21T04:48:56.027 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:48:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:48:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:48:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:48:56.029 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cd007af30 msgr2=0x7f7cd007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cd007af30 0x7f7cd007d3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ce419abc0 tx=0x7f7cd4008040 comp rx=0 tx=0).stop 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 msgr2=0x7f7ce419fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 0x7f7ce419fe00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc031460 tx=0x7f7ccc0048f0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 shutdown_connections 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7cd007af30 0x7f7cd007d3f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce4101000 0x7f7ce419fe00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 --2- 172.21.15.7:0/1015393269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ce4100650 0x7f7ce419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 >> 172.21.15.7:0/1015393269 conn(0x7f7ce40fc520 msgr2=0x7f7ce40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 shutdown_connections 2024-09-21T04:48:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.027+0000 7f7ce90d9640 1 -- 172.21.15.7:0/1015393269 wait complete. 2024-09-21T04:48:56.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.111+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1649004146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 msgr2=0x7f7e04101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:56.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.111+0000 7f7e0b1d1640 1 --2- 172.21.15.7:0/1649004146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 0x7f7e04101d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009c20 tx=0x7f7df402a970 comp rx=0 tx=0).stop 2024-09-21T04:48:56.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.111+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1649004146 shutdown_connections 2024-09-21T04:48:56.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.111+0000 7f7e0b1d1640 1 --2- 172.21.15.7:0/1649004146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 0x7f7e04101d60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.111+0000 7f7e0b1d1640 1 --2- 172.21.15.7:0/1649004146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04107900 0x7f7e04107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.111+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1649004146 >> 172.21.15.7:0/1649004146 conn(0x7f7e040fd600 msgr2=0x7f7e040ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:56.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.111+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1649004146 shutdown_connections 2024-09-21T04:48:56.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.111+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1649004146 wait complete. 2024-09-21T04:48:56.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e0b1d1640 1 Processor -- start 2024-09-21T04:48:56.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e0b1d1640 1 -- start start 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e0b1d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 0x7f7e0419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e0b1d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04107900 0x7f7e0419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e0b1d1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0410edd0 con 0x7f7e04101900 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e0b1d1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0410ec50 con 0x7f7e04107900 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e08f6d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 0x7f7e0419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e08f6d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 0x7f7e0419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33704/0 (socket says 172.21.15.7:33704) 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e08f6d640 1 -- 172.21.15.7:0/1562545768 learned_addr learned my addr 172.21.15.7:0/1562545768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.112+0000 7f7e03fff640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04107900 0x7f7e0419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e08f6d640 1 -- 172.21.15.7:0/1562545768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04107900 msgr2=0x7f7e0419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e08f6d640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04107900 0x7f7e0419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e08f6d640 1 -- 172.21.15.7:0/1562545768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e04197800 con 0x7f7e04101900 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e08f6d640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 0x7f7e0419d680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00d6f0 tx=0x7f7dec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:56.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e01ffb640 1 -- 172.21.15.7:0/1562545768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7dec017070 con 0x7f7e04101900 2024-09-21T04:48:56.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e01ffb640 1 -- 172.21.15.7:0/1562545768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7dec00de00 con 0x7f7e04101900 2024-09-21T04:48:56.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e04197af0 con 0x7f7e04101900 2024-09-21T04:48:56.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e01ffb640 1 -- 172.21.15.7:0/1562545768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7dec004bb0 con 0x7f7e04101900 2024-09-21T04:48:56.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.113+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e04198080 con 0x7f7e04101900 2024-09-21T04:48:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.114+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e04103040 con 0x7f7e04101900 2024-09-21T04:48:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.115+0000 7f7e01ffb640 1 -- 172.21.15.7:0/1562545768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7dec009a30 con 0x7f7e04101900 2024-09-21T04:48:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.115+0000 7f7e01ffb640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7df007af90 0x7f7df007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:48:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.116+0000 7f7e03fff640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7df007af90 0x7f7df007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:48:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.116+0000 7f7e01ffb640 1 -- 172.21.15.7:0/1562545768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7dec09db20 con 0x7f7e04101900 2024-09-21T04:48:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.116+0000 7f7e03fff640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7df007af90 0x7f7df007d450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e040ffc10 tx=0x7f7df403c040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:48:56.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.120+0000 7f7e01ffb640 1 -- 172.21.15.7:0/1562545768 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7dec069140 con 0x7f7e04101900 2024-09-21T04:48:56.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.382+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f7e04109e70 con 0x7f7e04101900 2024-09-21T04:48:56.385 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:56 smithi007 ceph-mon[108238]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:56.385 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:56 smithi007 ceph-mon[108238]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:56.385 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:56 smithi007 ceph-mon[108238]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:56.385 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:56 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/931519330' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:48:56.386 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:56 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2154065812' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:48:56.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.382+0000 7f7e01ffb640 1 -- 172.21.15.7:0/1562545768 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f7dec06dff0 con 0x7f7e04101900 2024-09-21T04:48:56.386 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:48:56.386 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:48:56.386 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:48:56.386 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:48:56.386 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:48:56.386 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:48:56.387 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:48:56.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.384+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7df007af90 msgr2=0x7f7df007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.384+0000 7f7e0b1d1640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7df007af90 0x7f7df007d450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e040ffc10 tx=0x7f7df403c040 comp rx=0 tx=0).stop 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.384+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 msgr2=0x7f7e0419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.384+0000 7f7e0b1d1640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 0x7f7e0419d680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00d6f0 tx=0x7f7dec00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.385+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 shutdown_connections 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.385+0000 7f7e0b1d1640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7df007af90 0x7f7df007d450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.385+0000 7f7e0b1d1640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e04107900 0x7f7e0419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.385+0000 7f7e0b1d1640 1 --2- 172.21.15.7:0/1562545768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e04101900 0x7f7e0419d680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.385+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 >> 172.21.15.7:0/1562545768 conn(0x7f7e040fd600 msgr2=0x7f7e040fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.385+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 shutdown_connections 2024-09-21T04:48:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:48:56.385+0000 7f7e0b1d1640 1 -- 172.21.15.7:0/1562545768 wait complete. 2024-09-21T04:48:56.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:56 smithi053 ceph-mon[80176]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:56.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:56 smithi053 ceph-mon[80176]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:56 smithi053 ceph-mon[80176]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:56 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/931519330' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:48:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:56 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2154065812' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:48:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:57 smithi053 ceph-mon[80176]: pgmap v425: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:57 smithi053 ceph-mon[80176]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:57 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1562545768' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:48:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:57 smithi007 ceph-mon[108238]: pgmap v425: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:57 smithi007 ceph-mon[108238]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:48:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:57 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1562545768' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:48:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:59 smithi007 ceph-mon[108238]: pgmap v426: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:48:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:48:59.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:59 smithi053 ceph-mon[80176]: pgmap v426: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:48:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:48:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:01 smithi053 ceph-mon[80176]: pgmap v427: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:01 smithi007 ceph-mon[108238]: pgmap v427: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:03 smithi053 ceph-mon[80176]: pgmap v428: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:03 smithi007 ceph-mon[108238]: pgmap v428: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:05 smithi053 ceph-mon[80176]: pgmap v429: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:05 smithi007 ceph-mon[108238]: pgmap v429: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:07 smithi053 ceph-mon[80176]: pgmap v430: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:07 smithi007 ceph-mon[108238]: pgmap v430: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:09.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:09 smithi053 ceph-mon[80176]: pgmap v431: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:09 smithi007 ceph-mon[108238]: pgmap v431: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:11 smithi007 ceph-mon[108238]: pgmap v432: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:11 smithi053 ceph-mon[80176]: pgmap v432: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:13 smithi053 ceph-mon[80176]: pgmap v433: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:13 smithi007 ceph-mon[108238]: pgmap v433: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:15 smithi053 ceph-mon[80176]: pgmap v434: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:15 smithi007 ceph-mon[108238]: pgmap v434: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:17 smithi007 ceph-mon[108238]: pgmap v435: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:17 smithi053 ceph-mon[80176]: pgmap v435: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:19.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:19 smithi007 ceph-mon[108238]: pgmap v436: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:19.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:19 smithi053 ceph-mon[80176]: pgmap v436: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:21 smithi053 ceph-mon[80176]: pgmap v437: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:21 smithi007 ceph-mon[108238]: pgmap v437: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:23 smithi007 ceph-mon[108238]: pgmap v438: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:23 smithi053 ceph-mon[80176]: pgmap v438: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:25 smithi053 ceph-mon[80176]: pgmap v439: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:25 smithi007 ceph-mon[108238]: pgmap v439: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.481+0000 7f3a97c93640 1 -- 172.21.15.7:0/2594518637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90107900 msgr2=0x7f3a90107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.481+0000 7f3a97c93640 1 --2- 172.21.15.7:0/2594518637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90107900 0x7f3a90107ce0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009880 tx=0x7f3a8c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.481+0000 7f3a97c93640 1 -- 172.21.15.7:0/2594518637 shutdown_connections 2024-09-21T04:49:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.481+0000 7f3a97c93640 1 --2- 172.21.15.7:0/2594518637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90101900 0x7f3a90101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.481+0000 7f3a97c93640 1 --2- 172.21.15.7:0/2594518637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90107900 0x7f3a90107ce0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.481+0000 7f3a97c93640 1 -- 172.21.15.7:0/2594518637 >> 172.21.15.7:0/2594518637 conn(0x7f3a900fd600 msgr2=0x7f3a900ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.482+0000 7f3a97c93640 1 -- 172.21.15.7:0/2594518637 shutdown_connections 2024-09-21T04:49:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.482+0000 7f3a97c93640 1 -- 172.21.15.7:0/2594518637 wait complete. 2024-09-21T04:49:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.482+0000 7f3a97c93640 1 Processor -- start 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.482+0000 7f3a97c93640 1 -- start start 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.482+0000 7f3a97c93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90101900 0x7f3a900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a97c93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90107900 0x7f3a90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a97c93640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9010edb0 con 0x7f3a90101900 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a97c93640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9010ec30 con 0x7f3a90107900 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a95a2f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90101900 0x7f3a900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a95a2f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90101900 0x7f3a900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35024/0 (socket says 172.21.15.7:35024) 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a95a2f640 1 -- 172.21.15.7:0/2877080307 learned_addr learned my addr 172.21.15.7:0/2877080307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a9522e640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90107900 0x7f3a90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a95a2f640 1 -- 172.21.15.7:0/2877080307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90107900 msgr2=0x7f3a90074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a95a2f640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90107900 0x7f3a90074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a95a2f640 1 -- 172.21.15.7:0/2877080307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a90074f70 con 0x7f3a90101900 2024-09-21T04:49:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a95a2f640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90101900 0x7f3a900744f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c031460 tx=0x7f3a8c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.483+0000 7f3a86ffd640 1 -- 172.21.15.7:0/2877080307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3a8c03f070 con 0x7f3a90101900 2024-09-21T04:49:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.484+0000 7f3a86ffd640 1 -- 172.21.15.7:0/2877080307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a8c004cf0 con 0x7f3a90101900 2024-09-21T04:49:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.484+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a90075200 con 0x7f3a90101900 2024-09-21T04:49:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.484+0000 7f3a86ffd640 1 -- 172.21.15.7:0/2877080307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3a8c035030 con 0x7f3a90101900 2024-09-21T04:49:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.484+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a901a3d50 con 0x7f3a90101900 2024-09-21T04:49:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.485+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a90103040 con 0x7f3a90101900 2024-09-21T04:49:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.486+0000 7f3a86ffd640 1 -- 172.21.15.7:0/2877080307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3a8c042070 con 0x7f3a90101900 2024-09-21T04:49:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.486+0000 7f3a86ffd640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a7c07af80 0x7f3a7c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.486+0000 7f3a9522e640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a7c07af80 0x7f3a7c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.486+0000 7f3a86ffd640 1 -- 172.21.15.7:0/2877080307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f3a8c0be990 con 0x7f3a90101900 2024-09-21T04:49:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.487+0000 7f3a9522e640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a7c07af80 0x7f3a7c07d440 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a90075f80 tx=0x7f3a8000a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.491+0000 7f3a86ffd640 1 -- 172.21.15.7:0/2877080307 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a8c089fb0 con 0x7f3a90101900 2024-09-21T04:49:26.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.694+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3a90104b60 con 0x7f3a7c07af80 2024-09-21T04:49:26.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.695+0000 7f3a86ffd640 1 -- 172.21.15.7:0/2877080307 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f3a90104b60 con 0x7f3a7c07af80 2024-09-21T04:49:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a7c07af80 msgr2=0x7f3a7c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a7c07af80 0x7f3a7c07d440 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a90075f80 tx=0x7f3a8000a2d0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90101900 msgr2=0x7f3a900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:26.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90101900 0x7f3a900744f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c031460 tx=0x7f3a8c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 shutdown_connections 2024-09-21T04:49:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a7c07af80 0x7f3a7c07d440 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90107900 0x7f3a90074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 --2- 172.21.15.7:0/2877080307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a90101900 0x7f3a900744f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 >> 172.21.15.7:0/2877080307 conn(0x7f3a900fd600 msgr2=0x7f3a900fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 shutdown_connections 2024-09-21T04:49:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.697+0000 7f3a97c93640 1 -- 172.21.15.7:0/2877080307 wait complete. 2024-09-21T04:49:26.710 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:49:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.784+0000 7f7293f4a640 1 -- 172.21.15.7:0/3138024702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 msgr2=0x7f728c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.784+0000 7f7293f4a640 1 --2- 172.21.15.7:0/3138024702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 0x7f728c100c60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7280009880 tx=0x7f728002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.784+0000 7f7293f4a640 1 -- 172.21.15.7:0/3138024702 shutdown_connections 2024-09-21T04:49:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.784+0000 7f7293f4a640 1 --2- 172.21.15.7:0/3138024702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 0x7f728c100c60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.784+0000 7f7293f4a640 1 --2- 172.21.15.7:0/3138024702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f728c106800 0x7f728c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.784+0000 7f7293f4a640 1 -- 172.21.15.7:0/3138024702 >> 172.21.15.7:0/3138024702 conn(0x7f728c0fc520 msgr2=0x7f728c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.784+0000 7f7293f4a640 1 -- 172.21.15.7:0/3138024702 shutdown_connections 2024-09-21T04:49:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.784+0000 7f7293f4a640 1 -- 172.21.15.7:0/3138024702 wait complete. 2024-09-21T04:49:26.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.785+0000 7f7293f4a640 1 Processor -- start 2024-09-21T04:49:26.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.785+0000 7f7293f4a640 1 -- start start 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.785+0000 7f7293f4a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 0x7f728c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.785+0000 7f7293f4a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f728c106800 0x7f728c102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.785+0000 7f7293f4a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f728c10dcb0 con 0x7f728c100800 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.785+0000 7f7293f4a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f728c10db30 con 0x7f728c106800 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7291ce6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 0x7f728c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7291ce6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 0x7f728c1023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35044/0 (socket says 172.21.15.7:35044) 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7291ce6640 1 -- 172.21.15.7:0/2122276142 learned_addr learned my addr 172.21.15.7:0/2122276142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f72914e5640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f728c106800 0x7f728c102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:26.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7291ce6640 1 -- 172.21.15.7:0/2122276142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f728c106800 msgr2=0x7f728c102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7291ce6640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f728c106800 0x7f728c102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7291ce6640 1 -- 172.21.15.7:0/2122276142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c102e50 con 0x7f728c100800 2024-09-21T04:49:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7291ce6640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 0x7f728c1023d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f728800c810 tx=0x7f728800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f727effd640 1 -- 172.21.15.7:0/2122276142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7288004020 con 0x7f728c100800 2024-09-21T04:49:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f727effd640 1 -- 172.21.15.7:0/2122276142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f72880041c0 con 0x7f728c100800 2024-09-21T04:49:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c103140 con 0x7f728c100800 2024-09-21T04:49:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f727effd640 1 -- 172.21.15.7:0/2122276142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f72880047a0 con 0x7f728c100800 2024-09-21T04:49:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.786+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c1a3df0 con 0x7f728c100800 2024-09-21T04:49:26.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.788+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c101f40 con 0x7f728c100800 2024-09-21T04:49:26.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.788+0000 7f727effd640 1 -- 172.21.15.7:0/2122276142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f72880079b0 con 0x7f728c100800 2024-09-21T04:49:26.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.789+0000 7f727effd640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f727807af30 0x7f727807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:26.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.789+0000 7f727effd640 1 -- 172.21.15.7:0/2122276142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f728809df90 con 0x7f728c100800 2024-09-21T04:49:26.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.789+0000 7f72914e5640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f727807af30 0x7f727807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:26.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.790+0000 7f72914e5640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f727807af30 0x7f727807d3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f728c103e60 tx=0x7f7280031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:26.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.794+0000 7f727effd640 1 -- 172.21.15.7:0/2122276142 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7288014030 con 0x7f728c100800 2024-09-21T04:49:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.996+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f728c06a1b0 con 0x7f727807af30 2024-09-21T04:49:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:26.998+0000 7f727effd640 1 -- 172.21.15.7:0/2122276142 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f728c06a1b0 con 0x7f727807af30 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f727807af30 msgr2=0x7f727807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f727807af30 0x7f727807d3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f728c103e60 tx=0x7f7280031400 comp rx=0 tx=0).stop 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 msgr2=0x7f728c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 0x7f728c1023d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f728800c810 tx=0x7f728800ccd0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 shutdown_connections 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f727807af30 0x7f727807d3f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f728c106800 0x7f728c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 --2- 172.21.15.7:0/2122276142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c100800 0x7f728c1023d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 >> 172.21.15.7:0/2122276142 conn(0x7f728c0fc520 msgr2=0x7f728c1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 shutdown_connections 2024-09-21T04:49:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.000+0000 7f7293f4a640 1 -- 172.21.15.7:0/2122276142 wait complete. 2024-09-21T04:49:27.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.084+0000 7f0b2ef99640 1 -- 172.21.15.7:0/2915472242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b28102ac0 msgr2=0x7f0b28102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.084+0000 7f0b2ef99640 1 --2- 172.21.15.7:0/2915472242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b28102ac0 0x7f0b28102ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009880 tx=0x7f0b1c02a530 comp rx=0 tx=0).stop 2024-09-21T04:49:27.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.084+0000 7f0b2ef99640 1 -- 172.21.15.7:0/2915472242 shutdown_connections 2024-09-21T04:49:27.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.084+0000 7f0b2ef99640 1 --2- 172.21.15.7:0/2915472242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b281033e0 0x7f0b2810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.084+0000 7f0b2ef99640 1 --2- 172.21.15.7:0/2915472242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b28102ac0 0x7f0b28102ea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.084+0000 7f0b2ef99640 1 -- 172.21.15.7:0/2915472242 >> 172.21.15.7:0/2915472242 conn(0x7f0b280fc520 msgr2=0x7f0b280fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:27.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.084+0000 7f0b2ef99640 1 -- 172.21.15.7:0/2915472242 shutdown_connections 2024-09-21T04:49:27.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.085+0000 7f0b2ef99640 1 -- 172.21.15.7:0/2915472242 wait complete. 2024-09-21T04:49:27.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.085+0000 7f0b2ef99640 1 Processor -- start 2024-09-21T04:49:27.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.085+0000 7f0b2ef99640 1 -- start start 2024-09-21T04:49:27.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b2ef99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b28102ac0 0x7f0b28100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b2ef99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b281033e0 0x7f0b28100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b2ef99640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2810edd0 con 0x7f0b281033e0 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b2ef99640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2810ec50 con 0x7f0b28102ac0 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b281033e0 0x7f0b28100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b281033e0 0x7f0b28100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35052/0 (socket says 172.21.15.7:35052) 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b27fff640 1 -- 172.21.15.7:0/3173655778 learned_addr learned my addr 172.21.15.7:0/3173655778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b2cd35640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b28102ac0 0x7f0b28100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b27fff640 1 -- 172.21.15.7:0/3173655778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b28102ac0 msgr2=0x7f0b28100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b27fff640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b28102ac0 0x7f0b28100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b27fff640 1 -- 172.21.15.7:0/3173655778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b281011a0 con 0x7f0b281033e0 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b2cd35640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b28102ac0 0x7f0b28100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.086+0000 7f0b27fff640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b281033e0 0x7f0b28100c60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400d8d0 tx=0x7f0b1400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.087+0000 7f0b25ffb640 1 -- 172.21.15.7:0/3173655778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0b14017030 con 0x7f0b281033e0 2024-09-21T04:49:27.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.087+0000 7f0b25ffb640 1 -- 172.21.15.7:0/3173655778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b1400be10 con 0x7f0b281033e0 2024-09-21T04:49:27.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.087+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b28101490 con 0x7f0b281033e0 2024-09-21T04:49:27.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.087+0000 7f0b25ffb640 1 -- 172.21.15.7:0/3173655778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0b14005580 con 0x7f0b281033e0 2024-09-21T04:49:27.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.087+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b281a3e40 con 0x7f0b281033e0 2024-09-21T04:49:27.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.088+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2806aa90 con 0x7f0b281033e0 2024-09-21T04:49:27.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.089+0000 7f0b25ffb640 1 -- 172.21.15.7:0/3173655778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0b14004020 con 0x7f0b281033e0 2024-09-21T04:49:27.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.090+0000 7f0b25ffb640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b1807af30 0x7f0b1807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.090+0000 7f0b25ffb640 1 -- 172.21.15.7:0/3173655778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0b14018070 con 0x7f0b281033e0 2024-09-21T04:49:27.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.090+0000 7f0b2cd35640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b1807af30 0x7f0b1807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.090+0000 7f0b2cd35640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b1807af30 0x7f0b1807d3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000c00 tx=0x7f0b1c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:27.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.095+0000 7f0b25ffb640 1 -- 172.21.15.7:0/3173655778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b14069370 con 0x7f0b281033e0 2024-09-21T04:49:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.293+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0b28062630 con 0x7f0b1807af30 2024-09-21T04:49:27.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.302+0000 7f0b25ffb640 1 -- 172.21.15.7:0/3173655778 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f0b28062630 con 0x7f0b1807af30 2024-09-21T04:49:27.306 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:49:27.306 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (11m) 5m ago 22m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (14m) 5m ago 22m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (14m) 5m ago 21m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (9m) 5m ago 22m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (9m) 5m ago 21m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (11m) 5m ago 22m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (5m) 5m ago 17m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (5m) 5m ago 17m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (16m) 5m ago 16m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (5m) 5m ago 17m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (13m) 5m ago 23m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (12m) 5m ago 21m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (10m) 5m ago 23m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (9m) 5m ago 21m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (12m) 5m ago 22m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:49:27.307 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (12m) 5m ago 21m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:49:27.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (9m) 5m ago 20m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:49:27.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (8m) 5m ago 19m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:49:27.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (7m) 5m ago 19m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:49:27.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (7m) 5m ago 19m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:49:27.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (6m) 5m ago 18m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:49:27.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (6m) 5m ago 18m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:49:27.308 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (11m) 5m ago 22m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b1807af30 msgr2=0x7f0b1807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b1807af30 0x7f0b1807d3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000c00 tx=0x7f0b1c031400 comp rx=0 tx=0).stop 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b281033e0 msgr2=0x7f0b28100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b281033e0 0x7f0b28100c60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400d8d0 tx=0x7f0b1400dd90 comp rx=0 tx=0).stop 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 shutdown_connections 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b1807af30 0x7f0b1807d3f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b281033e0 0x7f0b28100c60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 --2- 172.21.15.7:0/3173655778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b28102ac0 0x7f0b28100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.304+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 >> 172.21.15.7:0/3173655778 conn(0x7f0b280fc520 msgr2=0x7f0b28108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.305+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 shutdown_connections 2024-09-21T04:49:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.305+0000 7f0b2ef99640 1 -- 172.21.15.7:0/3173655778 wait complete. 2024-09-21T04:49:27.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.388+0000 7fccd175b640 1 -- 172.21.15.7:0/4221726476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 msgr2=0x7fcccc106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.388+0000 7fccd175b640 1 --2- 172.21.15.7:0/4221726476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 0x7fcccc106bd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009880 tx=0x7fccbc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 -- 172.21.15.7:0/4221726476 shutdown_connections 2024-09-21T04:49:27.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 --2- 172.21.15.7:0/4221726476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcccc1007f0 0x7fcccc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 --2- 172.21.15.7:0/4221726476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 0x7fcccc106bd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 -- 172.21.15.7:0/4221726476 >> 172.21.15.7:0/4221726476 conn(0x7fcccc0fc550 msgr2=0x7fcccc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:27.392 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:27 smithi007 ceph-mon[108238]: pgmap v440: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 -- 172.21.15.7:0/4221726476 shutdown_connections 2024-09-21T04:49:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 -- 172.21.15.7:0/4221726476 wait complete. 2024-09-21T04:49:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 Processor -- start 2024-09-21T04:49:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 -- start start 2024-09-21T04:49:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcccc1007f0 0x7fcccc107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 0x7fcccc107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc10dca0 con 0x7fcccc1067f0 2024-09-21T04:49:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccd175b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc10db20 con 0x7fcccc1007f0 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccca7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 0x7fcccc107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccca7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 0x7fcccc107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35070/0 (socket says 172.21.15.7:35070) 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccca7fc640 1 -- 172.21.15.7:0/1157080739 learned_addr learned my addr 172.21.15.7:0/1157080739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fcccaffd640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcccc1007f0 0x7fcccc107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccca7fc640 1 -- 172.21.15.7:0/1157080739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcccc1007f0 msgr2=0x7fcccc107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccca7fc640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcccc1007f0 0x7fcccc107720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.389+0000 7fccca7fc640 1 -- 172.21.15.7:0/1157080739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc1081a0 con 0x7fcccc1067f0 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.390+0000 7fcccaffd640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcccc1007f0 0x7fcccc107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.390+0000 7fccca7fc640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 0x7fcccc107c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c8a0 tx=0x7fccb400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.390+0000 7fccc3fff640 1 -- 172.21.15.7:0/1157080739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fccb40079b0 con 0x7fcccc1067f0 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.390+0000 7fccc3fff640 1 -- 172.21.15.7:0/1157080739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fccb4007b50 con 0x7fcccc1067f0 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.390+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcccc108490 con 0x7fcccc1067f0 2024-09-21T04:49:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.390+0000 7fccc3fff640 1 -- 172.21.15.7:0/1157080739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fccb4018700 con 0x7fcccc1067f0 2024-09-21T04:49:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.390+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcccc1a3d70 con 0x7fcccc1067f0 2024-09-21T04:49:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.392+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcccc101f30 con 0x7fcccc1067f0 2024-09-21T04:49:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.393+0000 7fccc3fff640 1 -- 172.21.15.7:0/1157080739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fccb4007d00 con 0x7fcccc1067f0 2024-09-21T04:49:27.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.394+0000 7fccc3fff640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fccb807af90 0x7fccb807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.394+0000 7fcccaffd640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fccb807af90 0x7fccb807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.394+0000 7fccc3fff640 1 -- 172.21.15.7:0/1157080739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fccb409e240 con 0x7fcccc1067f0 2024-09-21T04:49:27.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.394+0000 7fcccaffd640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fccb807af90 0x7fccb807d450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fccbc031460 tx=0x7fccbc009580 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.398+0000 7fccc3fff640 1 -- 172.21.15.7:0/1157080739 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fccb4014030 con 0x7fcccc1067f0 2024-09-21T04:49:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:27 smithi053 ceph-mon[80176]: pgmap v440: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:27.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.659+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fcccc1a4050 con 0x7fcccc1067f0 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.660+0000 7fccc3fff640 1 -- 172.21.15.7:0/1157080739 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fccb4069900 con 0x7fcccc1067f0 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:49:27.663 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:49:27.664 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:49:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fccb807af90 msgr2=0x7fccb807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fccb807af90 0x7fccb807d450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fccbc031460 tx=0x7fccbc009580 comp rx=0 tx=0).stop 2024-09-21T04:49:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 msgr2=0x7fcccc107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 0x7fcccc107c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fccb400c8a0 tx=0x7fccb400cd60 comp rx=0 tx=0).stop 2024-09-21T04:49:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 shutdown_connections 2024-09-21T04:49:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fccb807af90 0x7fccb807d450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcccc1067f0 0x7fcccc107c60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 --2- 172.21.15.7:0/1157080739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcccc1007f0 0x7fcccc107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.662+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 >> 172.21.15.7:0/1157080739 conn(0x7fcccc0fc550 msgr2=0x7fcccc069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.663+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 shutdown_connections 2024-09-21T04:49:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.663+0000 7fccd175b640 1 -- 172.21.15.7:0/1157080739 wait complete. 2024-09-21T04:49:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 -- 172.21.15.7:0/3418423656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 msgr2=0x7efdbc106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 --2- 172.21.15.7:0/3418423656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 0x7efdbc106bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efdac009c20 tx=0x7efdac02a970 comp rx=0 tx=0).stop 2024-09-21T04:49:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 -- 172.21.15.7:0/3418423656 shutdown_connections 2024-09-21T04:49:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 --2- 172.21.15.7:0/3418423656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdbc1007f0 0x7efdbc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 --2- 172.21.15.7:0/3418423656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 0x7efdbc106bd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 -- 172.21.15.7:0/3418423656 >> 172.21.15.7:0/3418423656 conn(0x7efdbc0fc550 msgr2=0x7efdbc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:27.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 -- 172.21.15.7:0/3418423656 shutdown_connections 2024-09-21T04:49:27.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 -- 172.21.15.7:0/3418423656 wait complete. 2024-09-21T04:49:27.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.747+0000 7efdc47da640 1 Processor -- start 2024-09-21T04:49:27.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc47da640 1 -- start start 2024-09-21T04:49:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc47da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdbc1007f0 0x7efdbc19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc47da640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 0x7efdbc19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc47da640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc10dcc0 con 0x7efdbc1067f0 2024-09-21T04:49:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc47da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc10db40 con 0x7efdbc1007f0 2024-09-21T04:49:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc1d75640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 0x7efdbc19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc1d75640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 0x7efdbc19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35090/0 (socket says 172.21.15.7:35090) 2024-09-21T04:49:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc1d75640 1 -- 172.21.15.7:0/1410732684 learned_addr learned my addr 172.21.15.7:0/1410732684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc2576640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdbc1007f0 0x7efdbc19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc1d75640 1 -- 172.21.15.7:0/1410732684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdbc1007f0 msgr2=0x7efdbc19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc1d75640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdbc1007f0 0x7efdbc19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc1d75640 1 -- 172.21.15.7:0/1410732684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc1977e0 con 0x7efdbc1067f0 2024-09-21T04:49:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.748+0000 7efdc1d75640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 0x7efdbc19dba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efdb000e970 tx=0x7efdb000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:27.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.749+0000 7efdbb7fe640 1 -- 172.21.15.7:0/1410732684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efdb000cac0 con 0x7efdbc1067f0 2024-09-21T04:49:27.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.749+0000 7efdbb7fe640 1 -- 172.21.15.7:0/1410732684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efdb00054e0 con 0x7efdbc1067f0 2024-09-21T04:49:27.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.749+0000 7efdbb7fe640 1 -- 172.21.15.7:0/1410732684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efdb0010430 con 0x7efdbc1067f0 2024-09-21T04:49:27.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.749+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc197ad0 con 0x7efdbc1067f0 2024-09-21T04:49:27.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.749+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc198060 con 0x7efdbc1067f0 2024-09-21T04:49:27.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.750+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdbc101f30 con 0x7efdbc1067f0 2024-09-21T04:49:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.750+0000 7efdbb7fe640 1 -- 172.21.15.7:0/1410732684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7efdb0004f00 con 0x7efdbc1067f0 2024-09-21T04:49:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.751+0000 7efdbb7fe640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efda407af80 0x7efda407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.751+0000 7efdbb7fe640 1 -- 172.21.15.7:0/1410732684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7efdb009e660 con 0x7efdbc1067f0 2024-09-21T04:49:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.751+0000 7efdc2576640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efda407af80 0x7efda407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:27.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.751+0000 7efdc2576640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efda407af80 0x7efda407d440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efdac009430 tx=0x7efdac03c040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.755+0000 7efdbb7fe640 1 -- 172.21.15.7:0/1410732684 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efdb0069c20 con 0x7efdbc1067f0 2024-09-21T04:49:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.983+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7efdbc109d70 con 0x7efdbc1067f0 2024-09-21T04:49:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.983+0000 7efdbb7fe640 1 -- 172.21.15.7:0/1410732684 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7efdb006ead0 con 0x7efdbc1067f0 2024-09-21T04:49:27.986 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:49:27.986 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:49:27.986 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:49:27.986 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:49:27.986 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:49:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:49:27.987 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:49:27.988 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:49:27.989 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:49:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:49:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efda407af80 msgr2=0x7efda407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efda407af80 0x7efda407d440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efdac009430 tx=0x7efdac03c040 comp rx=0 tx=0).stop 2024-09-21T04:49:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 msgr2=0x7efdbc19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 0x7efdbc19dba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efdb000e970 tx=0x7efdb000ed70 comp rx=0 tx=0).stop 2024-09-21T04:49:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 shutdown_connections 2024-09-21T04:49:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efda407af80 0x7efda407d440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdbc1067f0 0x7efdbc19dba0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 --2- 172.21.15.7:0/1410732684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdbc1007f0 0x7efdbc19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 >> 172.21.15.7:0/1410732684 conn(0x7efdbc0fc550 msgr2=0x7efdbc069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 shutdown_connections 2024-09-21T04:49:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:27.985+0000 7efdc47da640 1 -- 172.21.15.7:0/1410732684 wait complete. 2024-09-21T04:49:28.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.070+0000 7f26c7e88640 1 -- 172.21.15.7:0/630692217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 msgr2=0x7f26c0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:28.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.070+0000 7f26c7e88640 1 --2- 172.21.15.7:0/630692217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0102ea0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009880 tx=0x7f26bc02a530 comp rx=0 tx=0).stop 2024-09-21T04:49:28.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.070+0000 7f26c7e88640 1 -- 172.21.15.7:0/630692217 shutdown_connections 2024-09-21T04:49:28.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.070+0000 7f26c7e88640 1 --2- 172.21.15.7:0/630692217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.070+0000 7f26c7e88640 1 --2- 172.21.15.7:0/630692217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0102ea0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.070+0000 7f26c7e88640 1 -- 172.21.15.7:0/630692217 >> 172.21.15.7:0/630692217 conn(0x7f26c00fc520 msgr2=0x7f26c00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:28.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.071+0000 7f26c7e88640 1 -- 172.21.15.7:0/630692217 shutdown_connections 2024-09-21T04:49:28.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.071+0000 7f26c7e88640 1 -- 172.21.15.7:0/630692217 wait complete. 2024-09-21T04:49:28.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.071+0000 7f26c7e88640 1 Processor -- start 2024-09-21T04:49:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.071+0000 7f26c7e88640 1 -- start start 2024-09-21T04:49:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.071+0000 7f26c7e88640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.071+0000 7f26c7e88640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c0100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.071+0000 7f26c7e88640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010edd0 con 0x7f26c0102ac0 2024-09-21T04:49:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.071+0000 7f26c7e88640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010ec50 con 0x7f26c01033e0 2024-09-21T04:49:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c5c24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c5c24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35104/0 (socket says 172.21.15.7:35104) 2024-09-21T04:49:28.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c5c24640 1 -- 172.21.15.7:0/2263787514 learned_addr learned my addr 172.21.15.7:0/2263787514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c5423640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c0100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c5c24640 1 -- 172.21.15.7:0/2263787514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 msgr2=0x7f26c0100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c5c24640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c0100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c5c24640 1 -- 172.21.15.7:0/2263787514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c01011a0 con 0x7f26c0102ac0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c5c24640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0100720 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f26bc031460 tx=0x7f26bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26b6ffd640 1 -- 172.21.15.7:0/2263787514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f26bc03f070 con 0x7f26c0102ac0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26b6ffd640 1 -- 172.21.15.7:0/2263787514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26bc0035f0 con 0x7f26c0102ac0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c0101430 con 0x7f26c0102ac0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.072+0000 7f26b6ffd640 1 -- 172.21.15.7:0/2263787514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f26bc035030 con 0x7f26c0102ac0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.073+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c01a3ea0 con 0x7f26c0102ac0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.074+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c006aa90 con 0x7f26c0102ac0 2024-09-21T04:49:28.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.074+0000 7f26b6ffd640 1 -- 172.21.15.7:0/2263787514 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f26bc042070 con 0x7f26c0102ac0 2024-09-21T04:49:28.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.074+0000 7f26b6ffd640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26ac07af80 0x7f26ac07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:28.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.075+0000 7f26b6ffd640 1 -- 172.21.15.7:0/2263787514 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f26bc0be4b0 con 0x7f26c0102ac0 2024-09-21T04:49:28.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.075+0000 7f26c5423640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26ac07af80 0x7f26ac07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:28.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.075+0000 7f26c5423640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26ac07af80 0x7f26ac07d440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26c01021b0 tx=0x7f26b000b2e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.079+0000 7f26b6ffd640 1 -- 172.21.15.7:0/2263787514 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26bc089ad0 con 0x7f26c0102ac0 2024-09-21T04:49:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.289+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f26c0062630 con 0x7f26ac07af80 2024-09-21T04:49:28.292 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:28 smithi007 ceph-mon[108238]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:49:28.292 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:28 smithi007 ceph-mon[108238]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:49:28.292 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1157080739' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:49:28.292 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1410732684' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:49:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.290+0000 7f26b6ffd640 1 -- 172.21.15.7:0/2263787514 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f26c0062630 con 0x7f26ac07af80 2024-09-21T04:49:28.293 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:49:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:49:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:49:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:49:28.293 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:49:28.294 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.292+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26ac07af80 msgr2=0x7f26ac07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.292+0000 7f26c7e88640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26ac07af80 0x7f26ac07d440 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26c01021b0 tx=0x7f26b000b2e0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.292+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 msgr2=0x7f26c0100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.292+0000 7f26c7e88640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0100720 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f26bc031460 tx=0x7f26bc0045e0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.292+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 shutdown_connections 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.293+0000 7f26c7e88640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26ac07af80 0x7f26ac07d440 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.293+0000 7f26c7e88640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c0100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.293+0000 7f26c7e88640 1 --2- 172.21.15.7:0/2263787514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0100720 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.293+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 >> 172.21.15.7:0/2263787514 conn(0x7f26c00fc520 msgr2=0x7f26c0108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.293+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 shutdown_connections 2024-09-21T04:49:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.293+0000 7f26c7e88640 1 -- 172.21.15.7:0/2263787514 wait complete. 2024-09-21T04:49:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.377+0000 7f51d280f640 1 -- 172.21.15.7:0/299491968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc1033f0 msgr2=0x7f51cc10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.377+0000 7f51d280f640 1 --2- 172.21.15.7:0/299491968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc1033f0 0x7f51cc10b8e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009880 tx=0x7f51c002a530 comp rx=0 tx=0).stop 2024-09-21T04:49:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.377+0000 7f51d280f640 1 -- 172.21.15.7:0/299491968 shutdown_connections 2024-09-21T04:49:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.377+0000 7f51d280f640 1 --2- 172.21.15.7:0/299491968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc1033f0 0x7f51cc10b8e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.377+0000 7f51d280f640 1 --2- 172.21.15.7:0/299491968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc102ad0 0x7f51cc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.377+0000 7f51d280f640 1 -- 172.21.15.7:0/299491968 >> 172.21.15.7:0/299491968 conn(0x7f51cc0fc510 msgr2=0x7f51cc0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.378+0000 7f51d280f640 1 -- 172.21.15.7:0/299491968 shutdown_connections 2024-09-21T04:49:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.378+0000 7f51d280f640 1 -- 172.21.15.7:0/299491968 wait complete. 2024-09-21T04:49:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.378+0000 7f51d280f640 1 Processor -- start 2024-09-21T04:49:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.378+0000 7f51d280f640 1 -- start start 2024-09-21T04:49:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51d280f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc102ad0 0x7f51cc19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51d280f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc1033f0 0x7f51cc19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51d280f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc10ee60 con 0x7f51cc102ad0 2024-09-21T04:49:28.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51d280f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc10ece0 con 0x7f51cc1033f0 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc102ad0 0x7f51cc19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc102ad0 0x7f51cc19f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35124/0 (socket says 172.21.15.7:35124) 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51cbfff640 1 -- 172.21.15.7:0/3096312048 learned_addr learned my addr 172.21.15.7:0/3096312048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51cb7fe640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc1033f0 0x7f51cc19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51cbfff640 1 -- 172.21.15.7:0/3096312048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc1033f0 msgr2=0x7f51cc19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51cbfff640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc1033f0 0x7f51cc19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51cbfff640 1 -- 172.21.15.7:0/3096312048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc199a70 con 0x7f51cc102ad0 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51cbfff640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc102ad0 0x7f51cc19f8b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00b420 tx=0x7f51bc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51c97fa640 1 -- 172.21.15.7:0/3096312048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f51bc011630 con 0x7f51cc102ad0 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51c97fa640 1 -- 172.21.15.7:0/3096312048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f51bc016070 con 0x7f51cc102ad0 2024-09-21T04:49:28.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51cc199d60 con 0x7f51cc102ad0 2024-09-21T04:49:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51c97fa640 1 -- 172.21.15.7:0/3096312048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f51bc010d40 con 0x7f51cc102ad0 2024-09-21T04:49:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.379+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51cc19a2c0 con 0x7f51cc102ad0 2024-09-21T04:49:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.381+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51cc06ada0 con 0x7f51cc102ad0 2024-09-21T04:49:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.381+0000 7f51c97fa640 1 -- 172.21.15.7:0/3096312048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f51bc0117d0 con 0x7f51cc102ad0 2024-09-21T04:49:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.382+0000 7f51c97fa640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51b407af80 0x7f51b407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.382+0000 7f51cb7fe640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51b407af80 0x7f51b407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.382+0000 7f51c97fa640 1 -- 172.21.15.7:0/3096312048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f51bc09d690 con 0x7f51cc102ad0 2024-09-21T04:49:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.382+0000 7f51cb7fe640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51b407af80 0x7f51b407d440 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51cc19b260 tx=0x7f51c0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.386+0000 7f51c97fa640 1 -- 172.21.15.7:0/3096312048 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51bc068cb0 con 0x7f51cc102ad0 2024-09-21T04:49:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:28 smithi053 ceph-mon[80176]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:49:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:28 smithi053 ceph-mon[80176]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:49:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1157080739' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:49:28.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1410732684' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:49:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.650+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f51cc19afa0 con 0x7f51cc102ad0 2024-09-21T04:49:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.650+0000 7f51c97fa640 1 -- 172.21.15.7:0/3096312048 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f51bc06db60 con 0x7f51cc102ad0 2024-09-21T04:49:28.653 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:49:28.654 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:49:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:49:28.654 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:49:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:49:28.654 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:49:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51b407af80 msgr2=0x7f51b407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51b407af80 0x7f51b407d440 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51cc19b260 tx=0x7f51c0031400 comp rx=0 tx=0).stop 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc102ad0 msgr2=0x7f51cc19f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc102ad0 0x7f51cc19f8b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00b420 tx=0x7f51bc00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 shutdown_connections 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51b407af80 0x7f51b407d440 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc1033f0 0x7f51cc19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 --2- 172.21.15.7:0/3096312048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51cc102ad0 0x7f51cc19f8b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 >> 172.21.15.7:0/3096312048 conn(0x7f51cc0fc510 msgr2=0x7f51cc100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 shutdown_connections 2024-09-21T04:49:28.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:28.654+0000 7f51d280f640 1 -- 172.21.15.7:0/3096312048 wait complete. 2024-09-21T04:49:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:29 smithi007 ceph-mon[108238]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:49:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:29 smithi007 ceph-mon[108238]: pgmap v441: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:29.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:29 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3096312048' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:49:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:29 smithi053 ceph-mon[80176]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:49:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:29 smithi053 ceph-mon[80176]: pgmap v441: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:29 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3096312048' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:49:30.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:30 smithi053 ceph-mon[80176]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:49:30.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:30 smithi007 ceph-mon[108238]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:49:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:31 smithi053 ceph-mon[80176]: pgmap v442: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:31 smithi007 ceph-mon[108238]: pgmap v442: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:33 smithi053 ceph-mon[80176]: pgmap v443: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:33 smithi007 ceph-mon[108238]: pgmap v443: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:34 smithi053 ceph-mon[80176]: pgmap v444: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:34 smithi007 ceph-mon[108238]: pgmap v444: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:36 smithi053 ceph-mon[80176]: pgmap v445: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:36 smithi007 ceph-mon[108238]: pgmap v445: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:38 smithi053 ceph-mon[80176]: pgmap v446: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:38 smithi007 ceph-mon[108238]: pgmap v446: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:41.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:40 smithi053 ceph-mon[80176]: pgmap v447: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:40 smithi007 ceph-mon[108238]: pgmap v447: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:42 smithi053 ceph-mon[80176]: pgmap v448: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:42 smithi007 ceph-mon[108238]: pgmap v448: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:45 smithi053 ceph-mon[80176]: pgmap v449: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:45 smithi007 ceph-mon[108238]: pgmap v449: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:47 smithi007 ceph-mon[108238]: pgmap v450: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:47 smithi053 ceph-mon[80176]: pgmap v450: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:49.341 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:49 smithi007 ceph-mon[108238]: pgmap v451: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:49.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:49 smithi053 ceph-mon[80176]: pgmap v451: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:51 smithi053 ceph-mon[80176]: pgmap v452: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:51 smithi007 ceph-mon[108238]: pgmap v452: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:53 smithi007 ceph-mon[108238]: pgmap v453: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:53.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:53 smithi053 ceph-mon[80176]: pgmap v453: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:55 smithi053 ceph-mon[80176]: pgmap v454: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:55 smithi007 ceph-mon[108238]: pgmap v454: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:57 smithi053 ceph-mon[80176]: pgmap v455: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:57 smithi007 ceph-mon[108238]: pgmap v455: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:58.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.753+0000 7efec9c1e640 1 -- 172.21.15.7:0/4242460925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 msgr2=0x7efec4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:58.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.753+0000 7efec9c1e640 1 --2- 172.21.15.7:0/4242460925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 0x7efec4100c60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009880 tx=0x7efeb802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:49:58.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.753+0000 7efec9c1e640 1 -- 172.21.15.7:0/4242460925 shutdown_connections 2024-09-21T04:49:58.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.753+0000 7efec9c1e640 1 --2- 172.21.15.7:0/4242460925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 0x7efec4100c60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:58.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.753+0000 7efec9c1e640 1 --2- 172.21.15.7:0/4242460925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4106800 0x7efec4106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:58.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.753+0000 7efec9c1e640 1 -- 172.21.15.7:0/4242460925 >> 172.21.15.7:0/4242460925 conn(0x7efec40fc520 msgr2=0x7efec40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:58.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.753+0000 7efec9c1e640 1 -- 172.21.15.7:0/4242460925 shutdown_connections 2024-09-21T04:49:58.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.753+0000 7efec9c1e640 1 -- 172.21.15.7:0/4242460925 wait complete. 2024-09-21T04:49:58.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.754+0000 7efec9c1e640 1 Processor -- start 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.754+0000 7efec9c1e640 1 -- start start 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.754+0000 7efec9c1e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 0x7efec419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.754+0000 7efec9c1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4106800 0x7efec419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.754+0000 7efec9c1e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efec410dcb0 con 0x7efec4100800 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.754+0000 7efec9c1e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efec410db30 con 0x7efec4106800 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.755+0000 7efec37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 0x7efec419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.755+0000 7efec37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 0x7efec419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39676/0 (socket says 172.21.15.7:39676) 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.755+0000 7efec37fe640 1 -- 172.21.15.7:0/1766960618 learned_addr learned my addr 172.21.15.7:0/1766960618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.755+0000 7efec2ffd640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4106800 0x7efec419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.755+0000 7efec37fe640 1 -- 172.21.15.7:0/1766960618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4106800 msgr2=0x7efec419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.755+0000 7efec37fe640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4106800 0x7efec419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.755+0000 7efec37fe640 1 -- 172.21.15.7:0/1766960618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41977a0 con 0x7efec4100800 2024-09-21T04:49:58.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.755+0000 7efec37fe640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 0x7efec419d620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efeb400d8d0 tx=0x7efeb400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:58.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.756+0000 7efec0ff9640 1 -- 172.21.15.7:0/1766960618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efeb4017070 con 0x7efec4100800 2024-09-21T04:49:58.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.756+0000 7efec0ff9640 1 -- 172.21.15.7:0/1766960618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efeb4010460 con 0x7efec4100800 2024-09-21T04:49:58.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.756+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec4197a90 con 0x7efec4100800 2024-09-21T04:49:58.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.756+0000 7efec0ff9640 1 -- 172.21.15.7:0/1766960618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efeb400f5f0 con 0x7efec4100800 2024-09-21T04:49:58.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.756+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec4197ff0 con 0x7efec4100800 2024-09-21T04:49:58.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.757+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec4101f40 con 0x7efec4100800 2024-09-21T04:49:58.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.758+0000 7efec0ff9640 1 -- 172.21.15.7:0/1766960618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7efeb4009a30 con 0x7efec4100800 2024-09-21T04:49:58.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.758+0000 7efec0ff9640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeac07af90 0x7efeac07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:58.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.758+0000 7efec0ff9640 1 -- 172.21.15.7:0/1766960618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7efeb409dda0 con 0x7efec4100800 2024-09-21T04:49:58.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.758+0000 7efec2ffd640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeac07af90 0x7efeac07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:58.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.759+0000 7efec2ffd640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeac07af90 0x7efeac07d450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efec4198af0 tx=0x7efeb8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:58.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.763+0000 7efec0ff9640 1 -- 172.21.15.7:0/1766960618 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efeb40693c0 con 0x7efec4100800 2024-09-21T04:49:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.975+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7efec41982a0 con 0x7efeac07af90 2024-09-21T04:49:58.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.977+0000 7efec0ff9640 1 -- 172.21.15.7:0/1766960618 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7efec41982a0 con 0x7efeac07af90 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.979+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeac07af90 msgr2=0x7efeac07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.979+0000 7efec9c1e640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeac07af90 0x7efeac07d450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efec4198af0 tx=0x7efeb8031400 comp rx=0 tx=0).stop 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.979+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 msgr2=0x7efec419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.979+0000 7efec9c1e640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 0x7efec419d620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efeb400d8d0 tx=0x7efeb400dd90 comp rx=0 tx=0).stop 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.979+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 shutdown_connections 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.979+0000 7efec9c1e640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeac07af90 0x7efeac07d450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.980+0000 7efec9c1e640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec4106800 0x7efec419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.980+0000 7efec9c1e640 1 --2- 172.21.15.7:0/1766960618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec4100800 0x7efec419d620 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.980+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 >> 172.21.15.7:0/1766960618 conn(0x7efec40fc520 msgr2=0x7efec406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.980+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 shutdown_connections 2024-09-21T04:49:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:58.980+0000 7efec9c1e640 1 -- 172.21.15.7:0/1766960618 wait complete. 2024-09-21T04:49:58.991 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:49:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.065+0000 7f6863eb5640 1 -- 172.21.15.7:0/901172658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c107900 msgr2=0x7f685c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.065+0000 7f6863eb5640 1 --2- 172.21.15.7:0/901172658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c107900 0x7f685c107ce0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6858009c20 tx=0x7f685802a970 comp rx=0 tx=0).stop 2024-09-21T04:49:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.065+0000 7f6863eb5640 1 -- 172.21.15.7:0/901172658 shutdown_connections 2024-09-21T04:49:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.065+0000 7f6863eb5640 1 --2- 172.21.15.7:0/901172658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c101900 0x7f685c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.065+0000 7f6863eb5640 1 --2- 172.21.15.7:0/901172658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c107900 0x7f685c107ce0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.065+0000 7f6863eb5640 1 -- 172.21.15.7:0/901172658 >> 172.21.15.7:0/901172658 conn(0x7f685c0fd600 msgr2=0x7f685c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:59.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.066+0000 7f6863eb5640 1 -- 172.21.15.7:0/901172658 shutdown_connections 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.066+0000 7f6863eb5640 1 -- 172.21.15.7:0/901172658 wait complete. 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.066+0000 7f6863eb5640 1 Processor -- start 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.066+0000 7f6863eb5640 1 -- start start 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.066+0000 7f6863eb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c101900 0x7f685c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.066+0000 7f6863eb5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c107900 0x7f685c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.066+0000 7f6863eb5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f685c10edd0 con 0x7f685c101900 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.066+0000 7f6863eb5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f685c10ec50 con 0x7f685c107900 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861c51640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c101900 0x7f685c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861c51640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c101900 0x7f685c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39696/0 (socket says 172.21.15.7:39696) 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861c51640 1 -- 172.21.15.7:0/193775716 learned_addr learned my addr 172.21.15.7:0/193775716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861450640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c107900 0x7f685c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861c51640 1 -- 172.21.15.7:0/193775716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c107900 msgr2=0x7f685c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861c51640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c107900 0x7f685c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861c51640 1 -- 172.21.15.7:0/193775716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c1977c0 con 0x7f685c101900 2024-09-21T04:49:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861450640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c107900 0x7f685c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:49:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6861c51640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c101900 0x7f685c19d640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6858005cc0 tx=0x7f685802f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f684effd640 1 -- 172.21.15.7:0/193775716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6858047070 con 0x7f685c101900 2024-09-21T04:49:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c197a50 con 0x7f685c101900 2024-09-21T04:49:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f684effd640 1 -- 172.21.15.7:0/193775716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f685802f7e0 con 0x7f685c101900 2024-09-21T04:49:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f684effd640 1 -- 172.21.15.7:0/193775716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6858034070 con 0x7f685c101900 2024-09-21T04:49:59.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.067+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c197f80 con 0x7f685c101900 2024-09-21T04:49:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.068+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c103040 con 0x7f685c101900 2024-09-21T04:49:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.069+0000 7f684effd640 1 -- 172.21.15.7:0/193775716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f685804a070 con 0x7f685c101900 2024-09-21T04:49:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.069+0000 7f684effd640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f684807af30 0x7f684807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.070+0000 7f6861450640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f684807af30 0x7f684807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.070+0000 7f684effd640 1 -- 172.21.15.7:0/193775716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f68580bef40 con 0x7f685c101900 2024-09-21T04:49:59.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.070+0000 7f6861450640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f684807af30 0x7f684807d3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6850005fd0 tx=0x7f6850005ea0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:59.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.074+0000 7f684effd640 1 -- 172.21.15.7:0/193775716 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f685803f070 con 0x7f685c101900 2024-09-21T04:49:59.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.277+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f685c1982f0 con 0x7f684807af30 2024-09-21T04:49:59.280 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:59 smithi007 ceph-mon[108238]: pgmap v456: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:59.280 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:49:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:59.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.278+0000 7f684effd640 1 -- 172.21.15.7:0/193775716 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f685c1982f0 con 0x7f684807af30 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.280+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f684807af30 msgr2=0x7f684807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.280+0000 7f6863eb5640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f684807af30 0x7f684807d3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6850005fd0 tx=0x7f6850005ea0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.280+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c101900 msgr2=0x7f685c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.280+0000 7f6863eb5640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c101900 0x7f685c19d640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6858005cc0 tx=0x7f685802f5f0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.280+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 shutdown_connections 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.280+0000 7f6863eb5640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f684807af30 0x7f684807d3f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.280+0000 7f6863eb5640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c107900 0x7f685c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.280+0000 7f6863eb5640 1 --2- 172.21.15.7:0/193775716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c101900 0x7f685c19d640 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.281+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 >> 172.21.15.7:0/193775716 conn(0x7f685c0fd600 msgr2=0x7f685c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.281+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 shutdown_connections 2024-09-21T04:49:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.281+0000 7f6863eb5640 1 -- 172.21.15.7:0/193775716 wait complete. 2024-09-21T04:49:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.364+0000 7f109d1cb640 1 -- 172.21.15.7:0/4230971777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 msgr2=0x7f109810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.364+0000 7f109d1cb640 1 --2- 172.21.15.7:0/4230971777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 0x7f109810b8d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f108c009880 tx=0x7f108c02a530 comp rx=0 tx=0).stop 2024-09-21T04:49:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.364+0000 7f109d1cb640 1 -- 172.21.15.7:0/4230971777 shutdown_connections 2024-09-21T04:49:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.364+0000 7f109d1cb640 1 --2- 172.21.15.7:0/4230971777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 0x7f109810b8d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.364+0000 7f109d1cb640 1 --2- 172.21.15.7:0/4230971777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1098102ac0 0x7f1098102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.364+0000 7f109d1cb640 1 -- 172.21.15.7:0/4230971777 >> 172.21.15.7:0/4230971777 conn(0x7f10980fc520 msgr2=0x7f10980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.364+0000 7f109d1cb640 1 -- 172.21.15.7:0/4230971777 shutdown_connections 2024-09-21T04:49:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.364+0000 7f109d1cb640 1 -- 172.21.15.7:0/4230971777 wait complete. 2024-09-21T04:49:59.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.365+0000 7f109d1cb640 1 Processor -- start 2024-09-21T04:49:59.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.365+0000 7f109d1cb640 1 -- start start 2024-09-21T04:49:59.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.365+0000 7f109d1cb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1098102ac0 0x7f109819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.365+0000 7f109d1cb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 0x7f109819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.365+0000 7f109d1cb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f109810edd0 con 0x7f10981033e0 2024-09-21T04:49:59.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.365+0000 7f109d1cb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f109810ec50 con 0x7f1098102ac0 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f109659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 0x7f109819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f1096d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1098102ac0 0x7f109819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f109659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 0x7f109819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39714/0 (socket says 172.21.15.7:39714) 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f109659c640 1 -- 172.21.15.7:0/3442082961 learned_addr learned my addr 172.21.15.7:0/3442082961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f1096d9d640 1 -- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 msgr2=0x7f109819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f1096d9d640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 0x7f109819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f1096d9d640 1 -- 172.21.15.7:0/3442082961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981977e0 con 0x7f1098102ac0 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f109659c640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 0x7f109819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f1096d9d640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1098102ac0 0x7f109819d660 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1088005fd0 tx=0x7f108800dd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.366+0000 7f107ffff640 1 -- 172.21.15.7:0/3442082961 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f108800be30 con 0x7f1098102ac0 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.367+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1098197ad0 con 0x7f1098102ac0 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.367+0000 7f107ffff640 1 -- 172.21.15.7:0/3442082961 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1088004030 con 0x7f1098102ac0 2024-09-21T04:49:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.367+0000 7f107ffff640 1 -- 172.21.15.7:0/3442082961 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1088003cd0 con 0x7f1098102ac0 2024-09-21T04:49:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.367+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1098198030 con 0x7f1098102ac0 2024-09-21T04:49:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.368+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f109806aa90 con 0x7f1098102ac0 2024-09-21T04:49:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.368+0000 7f107ffff640 1 -- 172.21.15.7:0/3442082961 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f10880041e0 con 0x7f1098102ac0 2024-09-21T04:49:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.369+0000 7f107ffff640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f108007af30 0x7f108007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.369+0000 7f109659c640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f108007af30 0x7f108007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.369+0000 7f107ffff640 1 -- 172.21.15.7:0/3442082961 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f108809da50 con 0x7f1098102ac0 2024-09-21T04:49:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.370+0000 7f109659c640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f108007af30 0x7f108007d3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1098198b30 tx=0x7f108c0058d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.373+0000 7f107ffff640 1 -- 172.21.15.7:0/3442082961 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1088017020 con 0x7f1098102ac0 2024-09-21T04:49:59.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:59 smithi053 ceph-mon[80176]: pgmap v456: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:49:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:49:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:49:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.575+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f10981982e0 con 0x7f108007af30 2024-09-21T04:49:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.585+0000 7f107ffff640 1 -- 172.21.15.7:0/3442082961 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f10981982e0 con 0x7f108007af30 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (12m) 6m ago 22m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (15m) 6m ago 23m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (14m) 5m ago 22m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (10m) 6m ago 23m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (10m) 5m ago 22m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (11m) 6m ago 22m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (6m) 6m ago 17m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (6m) 6m ago 17m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (17m) 5m ago 17m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:49:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (5m) 5m ago 17m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (14m) 6m ago 24m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (13m) 5m ago 22m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (10m) 6m ago 24m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (10m) 5m ago 22m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (12m) 6m ago 23m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (12m) 5m ago 22m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (9m) 6m ago 20m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (9m) 6m ago 20m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (8m) 6m ago 19m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (7m) 5m ago 19m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (7m) 5m ago 19m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:49:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (6m) 5m ago 18m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:49:59.590 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (12m) 6m ago 22m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:49:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f108007af30 msgr2=0x7f108007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f108007af30 0x7f108007d3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1098198b30 tx=0x7f108c0058d0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1098102ac0 msgr2=0x7f109819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1098102ac0 0x7f109819d660 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1088005fd0 tx=0x7f108800dd60 comp rx=0 tx=0).stop 2024-09-21T04:49:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 shutdown_connections 2024-09-21T04:49:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f108007af30 0x7f108007d3f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10981033e0 0x7f109819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 --2- 172.21.15.7:0/3442082961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1098102ac0 0x7f109819d660 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 >> 172.21.15.7:0/3442082961 conn(0x7f10980fc520 msgr2=0x7f1098108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 shutdown_connections 2024-09-21T04:49:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.587+0000 7f109d1cb640 1 -- 172.21.15.7:0/3442082961 wait complete. 2024-09-21T04:49:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.669+0000 7f4d3ec87640 1 -- 172.21.15.7:0/1761230293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381007f0 msgr2=0x7f4d38100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.669+0000 7f4d3ec87640 1 --2- 172.21.15.7:0/1761230293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381007f0 0x7f4d38100c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009c20 tx=0x7f4d2802a970 comp rx=0 tx=0).stop 2024-09-21T04:49:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 -- 172.21.15.7:0/1761230293 shutdown_connections 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 --2- 172.21.15.7:0/1761230293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381007f0 0x7f4d38100c50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 --2- 172.21.15.7:0/1761230293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d381067f0 0x7f4d38106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 -- 172.21.15.7:0/1761230293 >> 172.21.15.7:0/1761230293 conn(0x7f4d380fc550 msgr2=0x7f4d380fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 -- 172.21.15.7:0/1761230293 shutdown_connections 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 -- 172.21.15.7:0/1761230293 wait complete. 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 Processor -- start 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 -- start start 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d381007f0 0x7f4d380fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381067f0 0x7f4d380fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810dcc0 con 0x7f4d381067f0 2024-09-21T04:49:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.670+0000 7f4d3ec87640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810db40 con 0x7f4d381007f0 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381067f0 0x7f4d380fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381067f0 0x7f4d380fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39734/0 (socket says 172.21.15.7:39734) 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d37fff640 1 -- 172.21.15.7:0/3612112977 learned_addr learned my addr 172.21.15.7:0/3612112977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d3ca23640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d381007f0 0x7f4d380fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d37fff640 1 -- 172.21.15.7:0/3612112977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d381007f0 msgr2=0x7f4d380fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d37fff640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d381007f0 0x7f4d380fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d37fff640 1 -- 172.21.15.7:0/3612112977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d380fd750 con 0x7f4d381067f0 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d3ca23640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d381007f0 0x7f4d380fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d37fff640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381067f0 0x7f4d380fd210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d2802ae90 tx=0x7f4d2802f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d35ffb640 1 -- 172.21.15.7:0/3612112977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4d28047070 con 0x7f4d381067f0 2024-09-21T04:49:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d380fd9e0 con 0x7f4d381067f0 2024-09-21T04:49:59.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d35ffb640 1 -- 172.21.15.7:0/3612112977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d2802f7e0 con 0x7f4d381067f0 2024-09-21T04:49:59.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.671+0000 7f4d35ffb640 1 -- 172.21.15.7:0/3612112977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4d28034070 con 0x7f4d381067f0 2024-09-21T04:49:59.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.672+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d381a3da0 con 0x7f4d381067f0 2024-09-21T04:49:59.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.673+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d38101f30 con 0x7f4d381067f0 2024-09-21T04:49:59.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.673+0000 7f4d35ffb640 1 -- 172.21.15.7:0/3612112977 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4d2804a070 con 0x7f4d381067f0 2024-09-21T04:49:59.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.674+0000 7f4d35ffb640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d2407af30 0x7f4d2407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:49:59.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.674+0000 7f4d35ffb640 1 -- 172.21.15.7:0/3612112977 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4d280bee00 con 0x7f4d381067f0 2024-09-21T04:49:59.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.674+0000 7f4d3ca23640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d2407af30 0x7f4d2407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:49:59.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.674+0000 7f4d3ca23640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d2407af30 0x7f4d2407d3f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4d200058c0 tx=0x7f4d20005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:49:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.678+0000 7f4d35ffb640 1 -- 172.21.15.7:0/3612112977 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d2803f070 con 0x7f4d381067f0 2024-09-21T04:49:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.944+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f4d38109d70 con 0x7f4d381067f0 2024-09-21T04:49:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.945+0000 7f4d35ffb640 1 -- 172.21.15.7:0/3612112977 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f4d2808a3a0 con 0x7f4d381067f0 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:49:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:49:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:49:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:49:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:49:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:49:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:49:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:49:59.949 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:49:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d2407af30 msgr2=0x7f4d2407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d2407af30 0x7f4d2407d3f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4d200058c0 tx=0x7f4d20005850 comp rx=0 tx=0).stop 2024-09-21T04:49:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381067f0 msgr2=0x7f4d380fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:49:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381067f0 0x7f4d380fd210 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d2802ae90 tx=0x7f4d2802f5f0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 shutdown_connections 2024-09-21T04:49:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d2407af30 0x7f4d2407d3f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d381067f0 0x7f4d380fd210 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 --2- 172.21.15.7:0/3612112977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d381007f0 0x7f4d380fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:49:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 >> 172.21.15.7:0/3612112977 conn(0x7f4d380fc550 msgr2=0x7f4d38069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:49:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 shutdown_connections 2024-09-21T04:49:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:49:59.947+0000 7f4d3ec87640 1 -- 172.21.15.7:0/3612112977 wait complete. 2024-09-21T04:50:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 -- 172.21.15.7:0/359260129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 msgr2=0x7f85f0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 --2- 172.21.15.7:0/359260129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 0x7f85f0107ce0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009880 tx=0x7f85e002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 -- 172.21.15.7:0/359260129 shutdown_connections 2024-09-21T04:50:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 --2- 172.21.15.7:0/359260129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85f0101900 0x7f85f0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 --2- 172.21.15.7:0/359260129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 0x7f85f0107ce0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 -- 172.21.15.7:0/359260129 >> 172.21.15.7:0/359260129 conn(0x7f85f00fd600 msgr2=0x7f85f00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 -- 172.21.15.7:0/359260129 shutdown_connections 2024-09-21T04:50:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 -- 172.21.15.7:0/359260129 wait complete. 2024-09-21T04:50:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.034+0000 7f85f69d6640 1 Processor -- start 2024-09-21T04:50:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85f69d6640 1 -- start start 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85f69d6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85f0101900 0x7f85f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85f69d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 0x7f85f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85f69d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010edd0 con 0x7f85f0107900 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85f69d6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010ec50 con 0x7f85f0101900 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 0x7f85f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 0x7f85f0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39740/0 (socket says 172.21.15.7:39740) 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85ef7fe640 1 -- 172.21.15.7:0/2106527314 learned_addr learned my addr 172.21.15.7:0/2106527314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85effff640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85f0101900 0x7f85f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85ef7fe640 1 -- 172.21.15.7:0/2106527314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85f0101900 msgr2=0x7f85f00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85ef7fe640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85f0101900 0x7f85f00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85ef7fe640 1 -- 172.21.15.7:0/2106527314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f0074f70 con 0x7f85f0107900 2024-09-21T04:50:00.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.035+0000 7f85effff640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85f0101900 0x7f85f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:50:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.036+0000 7f85ef7fe640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 0x7f85f0074a30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85e400d6f0 tx=0x7f85e400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.036+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2106527314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f85e4017070 con 0x7f85f0107900 2024-09-21T04:50:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.036+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2106527314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85e400de00 con 0x7f85f0107900 2024-09-21T04:50:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.036+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f0075260 con 0x7f85f0107900 2024-09-21T04:50:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.036+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2106527314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f85e4004bb0 con 0x7f85f0107900 2024-09-21T04:50:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.036+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f01a3e20 con 0x7f85f0107900 2024-09-21T04:50:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.037+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f0103040 con 0x7f85f0107900 2024-09-21T04:50:00.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.038+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2106527314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f85e4004e60 con 0x7f85f0107900 2024-09-21T04:50:00.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.038+0000 7f85ed7fa640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f85d807af30 0x7f85d807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.039+0000 7f85effff640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f85d807af30 0x7f85d807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.039+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2106527314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f85e409dc90 con 0x7f85f0107900 2024-09-21T04:50:00.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.039+0000 7f85effff640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f85d807af30 0x7f85d807d3f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:00.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.043+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2106527314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f85e4069230 con 0x7f85f0107900 2024-09-21T04:50:00.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.268+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f85f0109e70 con 0x7f85f0107900 2024-09-21T04:50:00.271 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:49:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:50:00.271 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:49:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:50:00.271 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:49:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-09-21T04:50:00.271 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:49:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:50:00.271 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:49:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:50:00.271 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:49:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:49:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3612112977' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: fs cephfs is degraded 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:50:00.272 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:00 smithi007 ceph-mon[108238]: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:50:00.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.268+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2106527314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f85e406e0e0 con 0x7f85f0107900 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:50:00.273 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:50:00.274 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:50:00.275 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:50:00.276 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:50:00.276 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:50:00.276 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:50:00.276 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:50:00.276 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.270+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f85d807af30 msgr2=0x7f85d807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.270+0000 7f85f69d6640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f85d807af30 0x7f85d807d3f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e0031400 comp rx=0 tx=0).stop 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.270+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 msgr2=0x7f85f0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.270+0000 7f85f69d6640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 0x7f85f0074a30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85e400d6f0 tx=0x7f85e400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.271+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 shutdown_connections 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.271+0000 7f85f69d6640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f85d807af30 0x7f85d807d3f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.271+0000 7f85f69d6640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85f0107900 0x7f85f0074a30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.271+0000 7f85f69d6640 1 --2- 172.21.15.7:0/2106527314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85f0101900 0x7f85f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.271+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 >> 172.21.15.7:0/2106527314 conn(0x7f85f00fd600 msgr2=0x7f85f00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.271+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 shutdown_connections 2024-09-21T04:50:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.271+0000 7f85f69d6640 1 -- 172.21.15.7:0/2106527314 wait complete. 2024-09-21T04:50:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.357+0000 7fa6ed990640 1 -- 172.21.15.7:0/200601657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 msgr2=0x7fa6e810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.357+0000 7fa6ed990640 1 --2- 172.21.15.7:0/200601657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 0x7fa6e810b8d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009880 tx=0x7fa6dc02a530 comp rx=0 tx=0).stop 2024-09-21T04:50:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.357+0000 7fa6ed990640 1 -- 172.21.15.7:0/200601657 shutdown_connections 2024-09-21T04:50:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.357+0000 7fa6ed990640 1 --2- 172.21.15.7:0/200601657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 0x7fa6e810b8d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.357+0000 7fa6ed990640 1 --2- 172.21.15.7:0/200601657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8102ac0 0x7fa6e8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.357+0000 7fa6ed990640 1 -- 172.21.15.7:0/200601657 >> 172.21.15.7:0/200601657 conn(0x7fa6e80fc520 msgr2=0x7fa6e80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.357+0000 7fa6ed990640 1 -- 172.21.15.7:0/200601657 shutdown_connections 2024-09-21T04:50:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.357+0000 7fa6ed990640 1 -- 172.21.15.7:0/200601657 wait complete. 2024-09-21T04:50:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.358+0000 7fa6ed990640 1 Processor -- start 2024-09-21T04:50:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.358+0000 7fa6ed990640 1 -- start start 2024-09-21T04:50:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.358+0000 7fa6ed990640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8102ac0 0x7fa6e8100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.358+0000 7fa6ed990640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 0x7fa6e8100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.358+0000 7fa6ed990640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810edf0 con 0x7fa6e81033e0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.358+0000 7fa6ed990640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e810ec70 con 0x7fa6e8102ac0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 0x7fa6e8100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6e67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 0x7fa6e8100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39766/0 (socket says 172.21.15.7:39766) 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6e67fc640 1 -- 172.21.15.7:0/4111866884 learned_addr learned my addr 172.21.15.7:0/4111866884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6e6ffd640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8102ac0 0x7fa6e8100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6e67fc640 1 -- 172.21.15.7:0/4111866884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8102ac0 msgr2=0x7fa6e8100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6e67fc640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8102ac0 0x7fa6e8100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6e67fc640 1 -- 172.21.15.7:0/4111866884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e81011a0 con 0x7fa6e81033e0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6e67fc640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 0x7fa6e8100c60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc031460 tx=0x7fa6dc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6ec98e640 1 -- 172.21.15.7:0/4111866884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa6dc03f070 con 0x7fa6e81033e0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6ec98e640 1 -- 172.21.15.7:0/4111866884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6dc0035f0 con 0x7fa6e81033e0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e8101430 con 0x7fa6e81033e0 2024-09-21T04:50:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6ec98e640 1 -- 172.21.15.7:0/4111866884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa6dc035030 con 0x7fa6e81033e0 2024-09-21T04:50:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.359+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e81a3de0 con 0x7fa6e81033e0 2024-09-21T04:50:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.360+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e806aa90 con 0x7fa6e81033e0 2024-09-21T04:50:00.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.361+0000 7fa6ec98e640 1 -- 172.21.15.7:0/4111866884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa6dc042070 con 0x7fa6e81033e0 2024-09-21T04:50:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.362+0000 7fa6ec98e640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6d007af30 0x7fa6d007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.362+0000 7fa6ec98e640 1 -- 172.21.15.7:0/4111866884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa6dc0bea00 con 0x7fa6e81033e0 2024-09-21T04:50:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.362+0000 7fa6e6ffd640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6d007af30 0x7fa6d007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.362+0000 7fa6e6ffd640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6d007af30 0x7fa6d007d3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005fd0 tx=0x7fa6d800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:00.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.366+0000 7fa6ec98e640 1 -- 172.21.15.7:0/4111866884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa6dc089f00 con 0x7fa6e81033e0 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3612112977' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: fs cephfs is degraded 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:50:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:00 smithi053 ceph-mon[80176]: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:50:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.570+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa6e8062630 con 0x7fa6d007af30 2024-09-21T04:50:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.572+0000 7fa6ec98e640 1 -- 172.21.15.7:0/4111866884 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fa6e8062630 con 0x7fa6d007af30 2024-09-21T04:50:00.574 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:50:00.575 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:50:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6d007af30 msgr2=0x7fa6d007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6d007af30 0x7fa6d007d3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005fd0 tx=0x7fa6d800b390 comp rx=0 tx=0).stop 2024-09-21T04:50:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 msgr2=0x7fa6e8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 0x7fa6e8100c60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc031460 tx=0x7fa6dc004530 comp rx=0 tx=0).stop 2024-09-21T04:50:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 shutdown_connections 2024-09-21T04:50:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa6d007af30 0x7fa6d007d3f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e81033e0 0x7fa6e8100c60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 --2- 172.21.15.7:0/4111866884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e8102ac0 0x7fa6e8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 >> 172.21.15.7:0/4111866884 conn(0x7fa6e80fc520 msgr2=0x7fa6e8108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 shutdown_connections 2024-09-21T04:50:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.574+0000 7fa6ed990640 1 -- 172.21.15.7:0/4111866884 wait complete. 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 -- 172.21.15.7:0/3044140895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41067f0 msgr2=0x7fcea4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 --2- 172.21.15.7:0/3044140895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41067f0 0x7fcea4106bd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fce94009880 tx=0x7fce9402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 -- 172.21.15.7:0/3044140895 shutdown_connections 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 --2- 172.21.15.7:0/3044140895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea41007f0 0x7fcea4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 --2- 172.21.15.7:0/3044140895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41067f0 0x7fcea4106bd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 -- 172.21.15.7:0/3044140895 >> 172.21.15.7:0/3044140895 conn(0x7fcea40fc550 msgr2=0x7fcea40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 -- 172.21.15.7:0/3044140895 shutdown_connections 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 -- 172.21.15.7:0/3044140895 wait complete. 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 Processor -- start 2024-09-21T04:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.660+0000 7fceab1ab640 1 -- start start 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fceab1ab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41007f0 0x7fcea419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fceab1ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea41067f0 0x7fcea419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fceab1ab640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcea410dcc0 con 0x7fcea41007f0 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fceab1ab640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcea410db40 con 0x7fcea41067f0 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea8f47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41007f0 0x7fcea419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea8f47640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41007f0 0x7fcea419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39774/0 (socket says 172.21.15.7:39774) 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea8f47640 1 -- 172.21.15.7:0/1564891168 learned_addr learned my addr 172.21.15.7:0/1564891168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea3fff640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea41067f0 0x7fcea419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea8f47640 1 -- 172.21.15.7:0/1564891168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea41067f0 msgr2=0x7fcea419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea8f47640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea41067f0 0x7fcea419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea8f47640 1 -- 172.21.15.7:0/1564891168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea41977b0 con 0x7fcea41007f0 2024-09-21T04:50:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea8f47640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41007f0 0x7fcea419d630 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fce94031460 tx=0x7fce94004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.661+0000 7fcea1ffb640 1 -- 172.21.15.7:0/1564891168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fce9403f070 con 0x7fcea41007f0 2024-09-21T04:50:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.662+0000 7fcea1ffb640 1 -- 172.21.15.7:0/1564891168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce94004a90 con 0x7fcea41007f0 2024-09-21T04:50:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.662+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea4197a40 con 0x7fcea41007f0 2024-09-21T04:50:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.662+0000 7fcea1ffb640 1 -- 172.21.15.7:0/1564891168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fce94035030 con 0x7fcea41007f0 2024-09-21T04:50:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.662+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea4197f70 con 0x7fcea41007f0 2024-09-21T04:50:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.663+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea4101f30 con 0x7fcea41007f0 2024-09-21T04:50:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.663+0000 7fcea1ffb640 1 -- 172.21.15.7:0/1564891168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fce94042070 con 0x7fcea41007f0 2024-09-21T04:50:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.664+0000 7fcea1ffb640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fce8c07af90 0x7fce8c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.664+0000 7fcea1ffb640 1 -- 172.21.15.7:0/1564891168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fce940bea30 con 0x7fcea41007f0 2024-09-21T04:50:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.664+0000 7fcea3fff640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fce8c07af90 0x7fce8c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.665+0000 7fcea3fff640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fce8c07af90 0x7fce8c07d450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcea4104fd0 tx=0x7fce98005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.668+0000 7fcea1ffb640 1 -- 172.21.15.7:0/1564891168 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fce9408a050 con 0x7fcea41007f0 2024-09-21T04:50:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.929+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fcea4109d70 con 0x7fcea41007f0 2024-09-21T04:50:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.930+0000 7fcea1ffb640 1 -- 172.21.15.7:0/1564891168 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fce9408ef00 con 0x7fcea41007f0 2024-09-21T04:50:00.933 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:50:00.933 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:50:00.933 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:50:00.933 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:50:00.933 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:50:00.933 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:50:00.933 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fce8c07af90 msgr2=0x7fce8c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fce8c07af90 0x7fce8c07d450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcea4104fd0 tx=0x7fce98005f90 comp rx=0 tx=0).stop 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41007f0 msgr2=0x7fcea419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41007f0 0x7fcea419d630 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fce94031460 tx=0x7fce94004530 comp rx=0 tx=0).stop 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 shutdown_connections 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fce8c07af90 0x7fce8c07d450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea41067f0 0x7fcea419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 --2- 172.21.15.7:0/1564891168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea41007f0 0x7fcea419d630 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 >> 172.21.15.7:0/1564891168 conn(0x7fcea40fc550 msgr2=0x7fcea4069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 shutdown_connections 2024-09-21T04:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:00.932+0000 7fceab1ab640 1 -- 172.21.15.7:0/1564891168 wait complete. 2024-09-21T04:50:01.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:01 smithi007 ceph-mon[108238]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:01.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:01 smithi007 ceph-mon[108238]: from='client.44545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:01.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:01 smithi007 ceph-mon[108238]: pgmap v457: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:01.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:01 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2106527314' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:50:01.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:01 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1564891168' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:50:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:01 smithi053 ceph-mon[80176]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:01 smithi053 ceph-mon[80176]: from='client.44545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:01 smithi053 ceph-mon[80176]: pgmap v457: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:01 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2106527314' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:50:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:01 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1564891168' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:50:02.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:02 smithi007 ceph-mon[108238]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:02.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:02 smithi053 ceph-mon[80176]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:03 smithi053 ceph-mon[80176]: pgmap v458: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:03 smithi007 ceph-mon[108238]: pgmap v458: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:05 smithi007 ceph-mon[108238]: pgmap v459: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:05 smithi053 ceph-mon[80176]: pgmap v459: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:07 smithi053 ceph-mon[80176]: pgmap v460: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:07 smithi007 ceph-mon[108238]: pgmap v460: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:09 smithi053 ceph-mon[80176]: pgmap v461: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:09 smithi007 ceph-mon[108238]: pgmap v461: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:11 smithi007 ceph-mon[108238]: pgmap v462: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:11.307 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:11 smithi053 ceph-mon[80176]: pgmap v462: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:13.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:13 smithi053 ceph-mon[80176]: pgmap v463: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:13 smithi007 ceph-mon[108238]: pgmap v463: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:50:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:50:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:15 smithi053 ceph-mon[80176]: pgmap v464: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:15 smithi007 ceph-mon[108238]: pgmap v464: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:17 smithi007 ceph-mon[108238]: pgmap v465: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:17 smithi053 ceph-mon[80176]: pgmap v465: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:19.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:19 smithi053 ceph-mon[80176]: pgmap v466: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:19 smithi007 ceph-mon[108238]: pgmap v466: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:21 smithi053 ceph-mon[80176]: pgmap v467: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:21 smithi007 ceph-mon[108238]: pgmap v467: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:23 smithi007 ceph-mon[108238]: pgmap v468: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:23.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:23 smithi053 ceph-mon[80176]: pgmap v468: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:25 smithi007 ceph-mon[108238]: pgmap v469: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:25 smithi053 ceph-mon[80176]: pgmap v469: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:27 smithi053 ceph-mon[80176]: pgmap v470: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:27 smithi007 ceph-mon[108238]: pgmap v470: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:29 smithi007 ceph-mon[108238]: pgmap v471: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:29.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:50:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:29 smithi053 ceph-mon[80176]: pgmap v471: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.029+0000 7f48d9ccb640 1 -- 172.21.15.7:0/2092184904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 msgr2=0x7f48d4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.029+0000 7f48d9ccb640 1 --2- 172.21.15.7:0/2092184904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 0x7f48d4107ce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009880 tx=0x7f48c402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.029+0000 7f48d9ccb640 1 -- 172.21.15.7:0/2092184904 shutdown_connections 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.029+0000 7f48d9ccb640 1 --2- 172.21.15.7:0/2092184904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4101900 0x7f48d4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.029+0000 7f48d9ccb640 1 --2- 172.21.15.7:0/2092184904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 0x7f48d4107ce0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.029+0000 7f48d9ccb640 1 -- 172.21.15.7:0/2092184904 >> 172.21.15.7:0/2092184904 conn(0x7f48d40fd600 msgr2=0x7f48d40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.029+0000 7f48d9ccb640 1 -- 172.21.15.7:0/2092184904 shutdown_connections 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.029+0000 7f48d9ccb640 1 -- 172.21.15.7:0/2092184904 wait complete. 2024-09-21T04:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.030+0000 7f48d9ccb640 1 Processor -- start 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.030+0000 7f48d9ccb640 1 -- start start 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.030+0000 7f48d9ccb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4101900 0x7f48d419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.030+0000 7f48d9ccb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 0x7f48d419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.030+0000 7f48d9ccb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48d410ee30 con 0x7f48d4107900 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.030+0000 7f48d9ccb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f48d410ecb0 con 0x7f48d4101900 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 0x7f48d419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 0x7f48d419fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54810/0 (socket says 172.21.15.7:54810) 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d2ffd640 1 -- 172.21.15.7:0/4128433045 learned_addr learned my addr 172.21.15.7:0/4128433045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d37fe640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4101900 0x7f48d419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d2ffd640 1 -- 172.21.15.7:0/4128433045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4101900 msgr2=0x7f48d419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d2ffd640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4101900 0x7f48d419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d2ffd640 1 -- 172.21.15.7:0/4128433045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d4199a10 con 0x7f48d4107900 2024-09-21T04:50:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d2ffd640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 0x7f48d419fd90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f48c800b420 tx=0x7f48c800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d0ff9640 1 -- 172.21.15.7:0/4128433045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f48c8011630 con 0x7f48d4107900 2024-09-21T04:50:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.031+0000 7f48d0ff9640 1 -- 172.21.15.7:0/4128433045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f48c8016070 con 0x7f48d4107900 2024-09-21T04:50:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.032+0000 7f48d0ff9640 1 -- 172.21.15.7:0/4128433045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f48c8010d40 con 0x7f48d4107900 2024-09-21T04:50:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.032+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d4199d00 con 0x7f48d4107900 2024-09-21T04:50:31.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.032+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d419a290 con 0x7f48d4107900 2024-09-21T04:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.033+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d4103040 con 0x7f48d4107900 2024-09-21T04:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.033+0000 7f48d0ff9640 1 -- 172.21.15.7:0/4128433045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f48c801b460 con 0x7f48d4107900 2024-09-21T04:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.034+0000 7f48d0ff9640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f48bc07af40 0x7f48bc07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.034+0000 7f48d0ff9640 1 -- 172.21.15.7:0/4128433045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f48c809d720 con 0x7f48d4107900 2024-09-21T04:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.034+0000 7f48d37fe640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f48bc07af40 0x7f48bc07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.035+0000 7f48d37fe640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f48bc07af40 0x7f48bc07d400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:31.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.039+0000 7f48d0ff9640 1 -- 172.21.15.7:0/4128433045 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f48c8068cc0 con 0x7f48d4107900 2024-09-21T04:50:31.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.247+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f48d419a5a0 con 0x7f48bc07af40 2024-09-21T04:50:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.248+0000 7f48d0ff9640 1 -- 172.21.15.7:0/4128433045 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f48d419a5a0 con 0x7f48bc07af40 2024-09-21T04:50:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f48bc07af40 msgr2=0x7f48bc07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f48bc07af40 0x7f48bc07d400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c4031400 comp rx=0 tx=0).stop 2024-09-21T04:50:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 msgr2=0x7f48d419fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 0x7f48d419fd90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f48c800b420 tx=0x7f48c800b8e0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 shutdown_connections 2024-09-21T04:50:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f48bc07af40 0x7f48bc07d400 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d4107900 0x7f48d419fd90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 --2- 172.21.15.7:0/4128433045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48d4101900 0x7f48d419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 >> 172.21.15.7:0/4128433045 conn(0x7f48d40fd600 msgr2=0x7f48d4074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 shutdown_connections 2024-09-21T04:50:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.250+0000 7f48d9ccb640 1 -- 172.21.15.7:0/4128433045 wait complete. 2024-09-21T04:50:31.263 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:50:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.336+0000 7f8b898ca640 1 -- 172.21.15.7:0/2327007373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 msgr2=0x7f8b8410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.336+0000 7f8b898ca640 1 --2- 172.21.15.7:0/2327007373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 0x7f8b8410b8d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009880 tx=0x7f8b6c02a530 comp rx=0 tx=0).stop 2024-09-21T04:50:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.336+0000 7f8b898ca640 1 -- 172.21.15.7:0/2327007373 shutdown_connections 2024-09-21T04:50:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.336+0000 7f8b898ca640 1 --2- 172.21.15.7:0/2327007373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 0x7f8b8410b8d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.336+0000 7f8b898ca640 1 --2- 172.21.15.7:0/2327007373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b84102ac0 0x7f8b84102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.336+0000 7f8b898ca640 1 -- 172.21.15.7:0/2327007373 >> 172.21.15.7:0/2327007373 conn(0x7f8b840fc520 msgr2=0x7f8b840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:31.341 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:31 smithi007 ceph-mon[108238]: pgmap v472: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.336+0000 7f8b898ca640 1 -- 172.21.15.7:0/2327007373 shutdown_connections 2024-09-21T04:50:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.336+0000 7f8b898ca640 1 -- 172.21.15.7:0/2327007373 wait complete. 2024-09-21T04:50:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.337+0000 7f8b898ca640 1 Processor -- start 2024-09-21T04:50:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.337+0000 7f8b898ca640 1 -- start start 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.337+0000 7f8b898ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b84102ac0 0x7f8b84100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.337+0000 7f8b898ca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 0x7f8b84100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.337+0000 7f8b898ca640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8410edf0 con 0x7f8b841033e0 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.337+0000 7f8b898ca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8410ec70 con 0x7f8b84102ac0 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b827fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 0x7f8b84100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b827fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 0x7f8b84100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54818/0 (socket says 172.21.15.7:54818) 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b827fc640 1 -- 172.21.15.7:0/1200587024 learned_addr learned my addr 172.21.15.7:0/1200587024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b82ffd640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b84102ac0 0x7f8b84100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b827fc640 1 -- 172.21.15.7:0/1200587024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b84102ac0 msgr2=0x7f8b84100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b827fc640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b84102ac0 0x7f8b84100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b827fc640 1 -- 172.21.15.7:0/1200587024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b841011a0 con 0x7f8b841033e0 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b827fc640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 0x7f8b84100c60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c031460 tx=0x7f8b6c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b888c8640 1 -- 172.21.15.7:0/1200587024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8b6c03f070 con 0x7f8b841033e0 2024-09-21T04:50:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b84101430 con 0x7f8b841033e0 2024-09-21T04:50:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b888c8640 1 -- 172.21.15.7:0/1200587024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b6c0035f0 con 0x7f8b841033e0 2024-09-21T04:50:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b841a3de0 con 0x7f8b841033e0 2024-09-21T04:50:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.338+0000 7f8b888c8640 1 -- 172.21.15.7:0/1200587024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8b6c035030 con 0x7f8b841033e0 2024-09-21T04:50:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.340+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8406aa90 con 0x7f8b841033e0 2024-09-21T04:50:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.340+0000 7f8b888c8640 1 -- 172.21.15.7:0/1200587024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8b6c042070 con 0x7f8b841033e0 2024-09-21T04:50:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.341+0000 7f8b888c8640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8b7007af80 0x7f8b7007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.341+0000 7f8b888c8640 1 -- 172.21.15.7:0/1200587024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8b6c0be890 con 0x7f8b841033e0 2024-09-21T04:50:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.341+0000 7f8b82ffd640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8b7007af80 0x7f8b7007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.342+0000 7f8b82ffd640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8b7007af80 0x7f8b7007d440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8b74005fd0 tx=0x7f8b7400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:31.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.346+0000 7f8b888c8640 1 -- 172.21.15.7:0/1200587024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b6c089eb0 con 0x7f8b841033e0 2024-09-21T04:50:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:31 smithi053 ceph-mon[80176]: pgmap v472: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:31.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.549+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8b84062630 con 0x7f8b7007af80 2024-09-21T04:50:31.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.550+0000 7f8b888c8640 1 -- 172.21.15.7:0/1200587024 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f8b84062630 con 0x7f8b7007af80 2024-09-21T04:50:31.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8b7007af80 msgr2=0x7f8b7007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8b7007af80 0x7f8b7007d440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8b74005fd0 tx=0x7f8b7400b390 comp rx=0 tx=0).stop 2024-09-21T04:50:31.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 msgr2=0x7f8b84100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 0x7f8b84100c60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c031460 tx=0x7f8b6c004530 comp rx=0 tx=0).stop 2024-09-21T04:50:31.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 shutdown_connections 2024-09-21T04:50:31.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8b7007af80 0x7f8b7007d440 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b841033e0 0x7f8b84100c60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 --2- 172.21.15.7:0/1200587024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b84102ac0 0x7f8b84100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.552+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 >> 172.21.15.7:0/1200587024 conn(0x7f8b840fc520 msgr2=0x7f8b84108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:31.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.553+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 shutdown_connections 2024-09-21T04:50:31.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.553+0000 7f8b898ca640 1 -- 172.21.15.7:0/1200587024 wait complete. 2024-09-21T04:50:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 -- 172.21.15.7:0/1699357762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81067f0 msgr2=0x7f09f8106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 --2- 172.21.15.7:0/1699357762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81067f0 0x7f09f8106bd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009880 tx=0x7f09e802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 -- 172.21.15.7:0/1699357762 shutdown_connections 2024-09-21T04:50:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 --2- 172.21.15.7:0/1699357762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f81007f0 0x7f09f8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 --2- 172.21.15.7:0/1699357762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81067f0 0x7f09f8106bd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 -- 172.21.15.7:0/1699357762 >> 172.21.15.7:0/1699357762 conn(0x7f09f80fc550 msgr2=0x7f09f80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 -- 172.21.15.7:0/1699357762 shutdown_connections 2024-09-21T04:50:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 -- 172.21.15.7:0/1699357762 wait complete. 2024-09-21T04:50:31.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.637+0000 7f09fdb27640 1 Processor -- start 2024-09-21T04:50:31.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09fdb27640 1 -- start start 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09fdb27640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81007f0 0x7f09f819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09fdb27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f81067f0 0x7f09f819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09fdb27640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810dcc0 con 0x7f09f81007f0 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09fdb27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810db40 con 0x7f09f81067f0 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81007f0 0x7f09f819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81007f0 0x7f09f819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54830/0 (socket says 172.21.15.7:54830) 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09f77fe640 1 -- 172.21.15.7:0/1368730599 learned_addr learned my addr 172.21.15.7:0/1368730599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09f6ffd640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f81067f0 0x7f09f819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09f77fe640 1 -- 172.21.15.7:0/1368730599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f81067f0 msgr2=0x7f09f819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09f77fe640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f81067f0 0x7f09f819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.638+0000 7f09f77fe640 1 -- 172.21.15.7:0/1368730599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f8199970 con 0x7f09f81007f0 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.639+0000 7f09f77fe640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81007f0 0x7f09f819f7b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f09e8031460 tx=0x7f09e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:31.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.639+0000 7f09f4ff9640 1 -- 172.21.15.7:0/1368730599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f09e803f070 con 0x7f09f81007f0 2024-09-21T04:50:31.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.639+0000 7f09f4ff9640 1 -- 172.21.15.7:0/1368730599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f09e80035f0 con 0x7f09f81007f0 2024-09-21T04:50:31.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.639+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f8199c00 con 0x7f09f81007f0 2024-09-21T04:50:31.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.639+0000 7f09f4ff9640 1 -- 172.21.15.7:0/1368730599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f09e8035030 con 0x7f09f81007f0 2024-09-21T04:50:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.639+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f819a130 con 0x7f09f81007f0 2024-09-21T04:50:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.640+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f8101f30 con 0x7f09f81007f0 2024-09-21T04:50:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.641+0000 7f09f4ff9640 1 -- 172.21.15.7:0/1368730599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f09e8042070 con 0x7f09f81007f0 2024-09-21T04:50:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.641+0000 7f09f4ff9640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f09e007af30 0x7f09e007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.641+0000 7f09f4ff9640 1 -- 172.21.15.7:0/1368730599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f09e80be9d0 con 0x7f09f81007f0 2024-09-21T04:50:31.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.641+0000 7f09f6ffd640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f09e007af30 0x7f09e007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.642+0000 7f09f6ffd640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f09e007af30 0x7f09e007d3f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09f80fea20 tx=0x7f09ec00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:31.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.646+0000 7f09f4ff9640 1 -- 172.21.15.7:0/1368730599 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09e8089f70 con 0x7f09f81007f0 2024-09-21T04:50:31.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.844+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f09f819a3e0 con 0x7f09e007af30 2024-09-21T04:50:31.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.854+0000 7f09f4ff9640 1 -- 172.21.15.7:0/1368730599 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f09f819a3e0 con 0x7f09e007af30 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (12m) 6m ago 23m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (15m) 6m ago 23m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (15m) 6m ago 22m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (10m) 6m ago 23m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (10m) 6m ago 22m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (12m) 6m ago 23m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (6m) 6m ago 18m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (6m) 6m ago 18m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (18m) 6m ago 18m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:50:31.858 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (6m) 6m ago 18m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (14m) 6m ago 24m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (13m) 6m ago 22m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (11m) 6m ago 24m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (10m) 6m ago 22m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (13m) 6m ago 23m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (13m) 6m ago 22m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 6m ago 21m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (9m) 6m ago 21m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (9m) 6m ago 20m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (8m) 6m ago 20m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (7m) 6m ago 19m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:50:31.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (7m) 6m ago 19m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:50:31.860 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (12m) 6m ago 23m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:50:31.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f09e007af30 msgr2=0x7f09e007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f09e007af30 0x7f09e007d3f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09f80fea20 tx=0x7f09ec00b390 comp rx=0 tx=0).stop 2024-09-21T04:50:31.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81007f0 msgr2=0x7f09f819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81007f0 0x7f09f819f7b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f09e8031460 tx=0x7f09e80045e0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 shutdown_connections 2024-09-21T04:50:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f09e007af30 0x7f09e007d3f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09f81067f0 0x7f09f819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 --2- 172.21.15.7:0/1368730599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09f81007f0 0x7f09f819f7b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 >> 172.21.15.7:0/1368730599 conn(0x7f09f80fc550 msgr2=0x7f09f81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 shutdown_connections 2024-09-21T04:50:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.856+0000 7f09fdb27640 1 -- 172.21.15.7:0/1368730599 wait complete. 2024-09-21T04:50:31.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.940+0000 7f5a05a9d640 1 -- 172.21.15.7:0/3784523781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00101900 msgr2=0x7f5a00101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.940+0000 7f5a05a9d640 1 --2- 172.21.15.7:0/3784523781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00101900 0x7f5a00101d60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009880 tx=0x7f59f402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.940+0000 7f5a05a9d640 1 -- 172.21.15.7:0/3784523781 shutdown_connections 2024-09-21T04:50:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.940+0000 7f5a05a9d640 1 --2- 172.21.15.7:0/3784523781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00101900 0x7f5a00101d60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.940+0000 7f5a05a9d640 1 --2- 172.21.15.7:0/3784523781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a00107900 0x7f5a00107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.940+0000 7f5a05a9d640 1 -- 172.21.15.7:0/3784523781 >> 172.21.15.7:0/3784523781 conn(0x7f5a000fd600 msgr2=0x7f5a000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.940+0000 7f5a05a9d640 1 -- 172.21.15.7:0/3784523781 shutdown_connections 2024-09-21T04:50:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.940+0000 7f5a05a9d640 1 -- 172.21.15.7:0/3784523781 wait complete. 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.941+0000 7f5a05a9d640 1 Processor -- start 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.941+0000 7f5a05a9d640 1 -- start start 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.941+0000 7f5a05a9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a00101900 0x7f5a000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.941+0000 7f5a05a9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00107900 0x7f5a00074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.941+0000 7f5a05a9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0010edd0 con 0x7f5a00107900 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.941+0000 7f5a05a9d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0010ec50 con 0x7f5a00101900 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59feffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00107900 0x7f5a00074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59feffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00107900 0x7f5a00074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54842/0 (socket says 172.21.15.7:54842) 2024-09-21T04:50:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59feffd640 1 -- 172.21.15.7:0/1257591819 learned_addr learned my addr 172.21.15.7:0/1257591819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59ff7fe640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a00101900 0x7f5a000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59feffd640 1 -- 172.21.15.7:0/1257591819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a00101900 msgr2=0x7f5a000744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59feffd640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a00101900 0x7f5a000744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59feffd640 1 -- 172.21.15.7:0/1257591819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a00074f70 con 0x7f5a00107900 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59feffd640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00107900 0x7f5a00074a30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f59f4031460 tx=0x7f59f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59fcff9640 1 -- 172.21.15.7:0/1257591819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f59f403f070 con 0x7f5a00107900 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a00075200 con 0x7f5a00107900 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59fcff9640 1 -- 172.21.15.7:0/1257591819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59f40035f0 con 0x7f5a00107900 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.942+0000 7f59fcff9640 1 -- 172.21.15.7:0/1257591819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f59f4035030 con 0x7f5a00107900 2024-09-21T04:50:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.943+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a001a3dc0 con 0x7f5a00107900 2024-09-21T04:50:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.944+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a00103040 con 0x7f5a00107900 2024-09-21T04:50:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.944+0000 7f59fcff9640 1 -- 172.21.15.7:0/1257591819 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f59f4042070 con 0x7f5a00107900 2024-09-21T04:50:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.945+0000 7f59fcff9640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f59e807af30 0x7f59e807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.945+0000 7f59fcff9640 1 -- 172.21.15.7:0/1257591819 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f59f40be9b0 con 0x7f5a00107900 2024-09-21T04:50:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.945+0000 7f59ff7fe640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f59e807af30 0x7f59e807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.945+0000 7f59ff7fe640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f59e807af30 0x7f59e807d3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59f0005fd0 tx=0x7f59f000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:31.949+0000 7f59fcff9640 1 -- 172.21.15.7:0/1257591819 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f59f4089f50 con 0x7f5a00107900 2024-09-21T04:50:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.213+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f5a00109e70 con 0x7f5a00107900 2024-09-21T04:50:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.214+0000 7f59fcff9640 1 -- 172.21.15.7:0/1257591819 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f59f408c470 con 0x7f5a00107900 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:50:32.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:50:32.219 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:50:32.219 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:50:32.219 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:50:32.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:50:32.219 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:50:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f59e807af30 msgr2=0x7f59e807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f59e807af30 0x7f59e807d3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59f0005fd0 tx=0x7f59f000b390 comp rx=0 tx=0).stop 2024-09-21T04:50:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00107900 msgr2=0x7f5a00074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00107900 0x7f5a00074a30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f59f4031460 tx=0x7f59f40045e0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 shutdown_connections 2024-09-21T04:50:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f59e807af30 0x7f59e807d3f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a00107900 0x7f5a00074a30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 --2- 172.21.15.7:0/1257591819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a00101900 0x7f5a000744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 >> 172.21.15.7:0/1257591819 conn(0x7f5a000fd600 msgr2=0x7f5a000fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 shutdown_connections 2024-09-21T04:50:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.216+0000 7f5a05a9d640 1 -- 172.21.15.7:0/1257591819 wait complete. 2024-09-21T04:50:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.301+0000 7faa0a399640 1 -- 172.21.15.7:0/3937407118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 msgr2=0x7faa04107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.301+0000 7faa0a399640 1 --2- 172.21.15.7:0/3937407118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 0x7faa04107ce0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009880 tx=0x7fa9f402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.301+0000 7faa0a399640 1 -- 172.21.15.7:0/3937407118 shutdown_connections 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.301+0000 7faa0a399640 1 --2- 172.21.15.7:0/3937407118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa04101900 0x7faa04101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.301+0000 7faa0a399640 1 --2- 172.21.15.7:0/3937407118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 0x7faa04107ce0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.301+0000 7faa0a399640 1 -- 172.21.15.7:0/3937407118 >> 172.21.15.7:0/3937407118 conn(0x7faa040fd620 msgr2=0x7faa040ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.301+0000 7faa0a399640 1 -- 172.21.15.7:0/3937407118 shutdown_connections 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.301+0000 7faa0a399640 1 -- 172.21.15.7:0/3937407118 wait complete. 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7faa0a399640 1 Processor -- start 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7faa0a399640 1 -- start start 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7faa0a399640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa04101900 0x7faa0419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7faa0a399640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 0x7faa0419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7faa0a399640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410edb0 con 0x7faa04107900 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7faa0a399640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410ec30 con 0x7faa04101900 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7fa9fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 0x7faa0419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7fa9fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 0x7faa0419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54870/0 (socket says 172.21.15.7:54870) 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7fa9fbfff640 1 -- 172.21.15.7:0/3072904584 learned_addr learned my addr 172.21.15.7:0/3072904584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7faa03fff640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa04101900 0x7faa0419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7fa9fbfff640 1 -- 172.21.15.7:0/3072904584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa04101900 msgr2=0x7faa0419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7fa9fbfff640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa04101900 0x7faa0419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.302+0000 7fa9fbfff640 1 -- 172.21.15.7:0/3072904584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa04197760 con 0x7faa04107900 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.303+0000 7fa9fbfff640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 0x7faa0419dbd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c8a0 tx=0x7fa9ec00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.303+0000 7faa01ffb640 1 -- 172.21.15.7:0/3072904584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa9ec0079b0 con 0x7faa04107900 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.303+0000 7faa01ffb640 1 -- 172.21.15.7:0/3072904584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9ec007b50 con 0x7faa04107900 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.303+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa04197a50 con 0x7faa04107900 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.303+0000 7faa01ffb640 1 -- 172.21.15.7:0/3072904584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa9ec018700 con 0x7faa04107900 2024-09-21T04:50:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.303+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa04197fe0 con 0x7faa04107900 2024-09-21T04:50:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.304+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa04103040 con 0x7faa04107900 2024-09-21T04:50:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.305+0000 7faa01ffb640 1 -- 172.21.15.7:0/3072904584 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa9ec007d00 con 0x7faa04107900 2024-09-21T04:50:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.305+0000 7faa01ffb640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa9f007af30 0x7fa9f007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.305+0000 7faa01ffb640 1 -- 172.21.15.7:0/3072904584 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa9ec09e2e0 con 0x7faa04107900 2024-09-21T04:50:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.305+0000 7faa03fff640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa9f007af30 0x7fa9f007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.306+0000 7faa03fff640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa9f007af30 0x7fa9f007d3f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4000c00 tx=0x7fa9f4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.310+0000 7faa01ffb640 1 -- 172.21.15.7:0/3072904584 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9ec014030 con 0x7faa04107900 2024-09-21T04:50:32.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.539+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7faa04104b10 con 0x7faa04107900 2024-09-21T04:50:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.540+0000 7faa01ffb640 1 -- 172.21.15.7:0/3072904584 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7fa9ec069820 con 0x7faa04107900 2024-09-21T04:50:32.544 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:50:32.544 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:50:32.544 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:50:32.544 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:50:32.544 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:50:32.545 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:50:32.546 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:50:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:50:32.548 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:50:32.548 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:50:32.548 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:50:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa9f007af30 msgr2=0x7fa9f007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa9f007af30 0x7fa9f007d3f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4000c00 tx=0x7fa9f4031400 comp rx=0 tx=0).stop 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 msgr2=0x7faa0419dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 0x7faa0419dbd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00c8a0 tx=0x7fa9ec00cd60 comp rx=0 tx=0).stop 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 shutdown_connections 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa9f007af30 0x7fa9f007d3f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa04107900 0x7faa0419dbd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 --2- 172.21.15.7:0/3072904584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa04101900 0x7faa0419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 >> 172.21.15.7:0/3072904584 conn(0x7faa040fd620 msgr2=0x7faa040feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 shutdown_connections 2024-09-21T04:50:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.543+0000 7faa0a399640 1 -- 172.21.15.7:0/3072904584 wait complete. 2024-09-21T04:50:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.629+0000 7fd942314640 1 -- 172.21.15.7:0/2971042708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c101900 msgr2=0x7fd93c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.629+0000 7fd942314640 1 --2- 172.21.15.7:0/2971042708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c101900 0x7fd93c101d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd930009880 tx=0x7fd93002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.630+0000 7fd942314640 1 -- 172.21.15.7:0/2971042708 shutdown_connections 2024-09-21T04:50:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.630+0000 7fd942314640 1 --2- 172.21.15.7:0/2971042708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c101900 0x7fd93c101d60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.630+0000 7fd942314640 1 --2- 172.21.15.7:0/2971042708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd93c107900 0x7fd93c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.630+0000 7fd942314640 1 -- 172.21.15.7:0/2971042708 >> 172.21.15.7:0/2971042708 conn(0x7fd93c0fd600 msgr2=0x7fd93c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.630+0000 7fd942314640 1 -- 172.21.15.7:0/2971042708 shutdown_connections 2024-09-21T04:50:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.630+0000 7fd942314640 1 -- 172.21.15.7:0/2971042708 wait complete. 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.630+0000 7fd942314640 1 Processor -- start 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd942314640 1 -- start start 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd942314640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd93c101900 0x7fd93c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd942314640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c107900 0x7fd93c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd942314640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c10edd0 con 0x7fd93c107900 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd942314640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c10ec50 con 0x7fd93c101900 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c107900 0x7fd93c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd93b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c107900 0x7fd93c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54884/0 (socket says 172.21.15.7:54884) 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd93b7fe640 1 -- 172.21.15.7:0/2790578102 learned_addr learned my addr 172.21.15.7:0/2790578102 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd93bfff640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd93c101900 0x7fd93c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd93b7fe640 1 -- 172.21.15.7:0/2790578102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd93c101900 msgr2=0x7fd93c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd93b7fe640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd93c101900 0x7fd93c19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.631+0000 7fd93b7fe640 1 -- 172.21.15.7:0/2790578102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c1977a0 con 0x7fd93c107900 2024-09-21T04:50:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.632+0000 7fd93b7fe640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c107900 0x7fd93c19db60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd930031460 tx=0x7fd9300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.632+0000 7fd9397fa640 1 -- 172.21.15.7:0/2790578102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd93003f070 con 0x7fd93c107900 2024-09-21T04:50:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.632+0000 7fd9397fa640 1 -- 172.21.15.7:0/2790578102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd930004cf0 con 0x7fd93c107900 2024-09-21T04:50:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.632+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93c197a30 con 0x7fd93c107900 2024-09-21T04:50:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.632+0000 7fd9397fa640 1 -- 172.21.15.7:0/2790578102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd930035030 con 0x7fd93c107900 2024-09-21T04:50:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.632+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93c197f60 con 0x7fd93c107900 2024-09-21T04:50:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.633+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93c103040 con 0x7fd93c107900 2024-09-21T04:50:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.634+0000 7fd9397fa640 1 -- 172.21.15.7:0/2790578102 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd930042070 con 0x7fd93c107900 2024-09-21T04:50:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.634+0000 7fd9397fa640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd92407af30 0x7fd92407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.635+0000 7fd9397fa640 1 -- 172.21.15.7:0/2790578102 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd9300bea70 con 0x7fd93c107900 2024-09-21T04:50:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.635+0000 7fd93bfff640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd92407af30 0x7fd92407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.635+0000 7fd93bfff640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd92407af30 0x7fd92407d3f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0058c0 tx=0x7fd92c00a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:32.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.639+0000 7fd9397fa640 1 -- 172.21.15.7:0/2790578102 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd93008a010 con 0x7fd93c107900 2024-09-21T04:50:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.843+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd93c1982d0 con 0x7fd92407af30 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.844+0000 7fd9397fa640 1 -- 172.21.15.7:0/2790578102 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd93c1982d0 con 0x7fd92407af30 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:50:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:50:32.849 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:50:32.849 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:50:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd92407af30 msgr2=0x7fd92407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd92407af30 0x7fd92407d3f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0058c0 tx=0x7fd92c00a250 comp rx=0 tx=0).stop 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c107900 msgr2=0x7fd93c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c107900 0x7fd93c19db60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd930031460 tx=0x7fd9300048f0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 shutdown_connections 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd92407af30 0x7fd92407d3f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd93c107900 0x7fd93c19db60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 --2- 172.21.15.7:0/2790578102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd93c101900 0x7fd93c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 >> 172.21.15.7:0/2790578102 conn(0x7fd93c0fd600 msgr2=0x7fd93c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 shutdown_connections 2024-09-21T04:50:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.846+0000 7fd942314640 1 -- 172.21.15.7:0/2790578102 wait complete. 2024-09-21T04:50:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.930+0000 7f7398dde640 1 -- 172.21.15.7:0/3305598757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 msgr2=0x7f7394106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.930+0000 7f7398dde640 1 --2- 172.21.15.7:0/3305598757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 0x7f7394106be0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7384009880 tx=0x7f738402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.930+0000 7f7398dde640 1 -- 172.21.15.7:0/3305598757 shutdown_connections 2024-09-21T04:50:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.930+0000 7f7398dde640 1 --2- 172.21.15.7:0/3305598757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7394100800 0x7f7394100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.930+0000 7f7398dde640 1 --2- 172.21.15.7:0/3305598757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 0x7f7394106be0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.930+0000 7f7398dde640 1 -- 172.21.15.7:0/3305598757 >> 172.21.15.7:0/3305598757 conn(0x7f73940fc520 msgr2=0x7f73940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.930+0000 7f7398dde640 1 -- 172.21.15.7:0/3305598757 shutdown_connections 2024-09-21T04:50:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.930+0000 7f7398dde640 1 -- 172.21.15.7:0/3305598757 wait complete. 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7398dde640 1 Processor -- start 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7398dde640 1 -- start start 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7398dde640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7394100800 0x7f73940fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7398dde640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 0x7f73940fd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7398dde640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f739410dcd0 con 0x7f7394106800 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7398dde640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f739410db50 con 0x7f7394100800 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7389d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 0x7f73940fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7389d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 0x7f73940fd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54902/0 (socket says 172.21.15.7:54902) 2024-09-21T04:50:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f7389d9b640 1 -- 172.21.15.7:0/2833123000 learned_addr learned my addr 172.21.15.7:0/2833123000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.931+0000 7f739259c640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7394100800 0x7f73940fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f7389d9b640 1 -- 172.21.15.7:0/2833123000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7394100800 msgr2=0x7f73940fcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f7389d9b640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7394100800 0x7f73940fcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f7389d9b640 1 -- 172.21.15.7:0/2833123000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73940fd6b0 con 0x7f7394106800 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f7389d9b640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 0x7f73940fd170 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f737c00d6f0 tx=0x7f737c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f738bfff640 1 -- 172.21.15.7:0/2833123000 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f737c017070 con 0x7f7394106800 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73940fd9a0 con 0x7f7394106800 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f738bfff640 1 -- 172.21.15.7:0/2833123000 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f737c00de00 con 0x7f7394106800 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f738bfff640 1 -- 172.21.15.7:0/2833123000 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f737c004bb0 con 0x7f7394106800 2024-09-21T04:50:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.932+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941a3e40 con 0x7f7394106800 2024-09-21T04:50:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.933+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7394101f40 con 0x7f7394106800 2024-09-21T04:50:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.934+0000 7f738bfff640 1 -- 172.21.15.7:0/2833123000 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f737c004e60 con 0x7f7394106800 2024-09-21T04:50:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.934+0000 7f738bfff640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f738007af30 0x7f738007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:50:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.935+0000 7f738bfff640 1 -- 172.21.15.7:0/2833123000 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f737c09dbe0 con 0x7f7394106800 2024-09-21T04:50:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.935+0000 7f739259c640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f738007af30 0x7f738007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:50:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.935+0000 7f739259c640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f738007af30 0x7f738007d3f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f73840096c0 tx=0x7f7384031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:50:32.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:32.939+0000 7f738bfff640 1 -- 172.21.15.7:0/2833123000 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f737c069180 con 0x7f7394106800 2024-09-21T04:50:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:33 smithi007 ceph-mon[108238]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:33 smithi007 ceph-mon[108238]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:33 smithi007 ceph-mon[108238]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:33 smithi007 ceph-mon[108238]: pgmap v473: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:33 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1257591819' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:50:33.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:33 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3072904584' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:50:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.204+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f7394100c60 con 0x7f7394106800 2024-09-21T04:50:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.205+0000 7f738bfff640 1 -- 172.21.15.7:0/2833123000 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f737c06e030 con 0x7f7394106800 2024-09-21T04:50:33.209 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:50:33.209 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:50:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:50:33.209 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:50:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:50:33.209 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:50:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:50:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f738007af30 msgr2=0x7f738007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f738007af30 0x7f738007d3f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f73840096c0 tx=0x7f7384031400 comp rx=0 tx=0).stop 2024-09-21T04:50:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 msgr2=0x7f73940fd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:50:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 0x7f73940fd170 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f737c00d6f0 tx=0x7f737c00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:50:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 shutdown_connections 2024-09-21T04:50:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f738007af30 0x7f738007d3f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7394106800 0x7f73940fd170 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 --2- 172.21.15.7:0/2833123000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7394100800 0x7f73940fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:50:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.207+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 >> 172.21.15.7:0/2833123000 conn(0x7f73940fc520 msgr2=0x7f739406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:50:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.208+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 shutdown_connections 2024-09-21T04:50:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:50:33.208+0000 7f7398dde640 1 -- 172.21.15.7:0/2833123000 wait complete. 2024-09-21T04:50:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:33 smithi053 ceph-mon[80176]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:33 smithi053 ceph-mon[80176]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:33 smithi053 ceph-mon[80176]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:33 smithi053 ceph-mon[80176]: pgmap v473: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:33 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1257591819' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:50:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:33 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3072904584' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:50:34.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:34 smithi053 ceph-mon[80176]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:34.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:34 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2833123000' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:50:34.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:34 smithi007 ceph-mon[108238]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:50:34.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:34 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2833123000' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:50:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:35 smithi007 ceph-mon[108238]: pgmap v474: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:35 smithi053 ceph-mon[80176]: pgmap v474: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:37 smithi053 ceph-mon[80176]: pgmap v475: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:37 smithi007 ceph-mon[108238]: pgmap v475: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:39 smithi053 ceph-mon[80176]: pgmap v476: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:39 smithi007 ceph-mon[108238]: pgmap v476: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:41 smithi007 ceph-mon[108238]: pgmap v477: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:41 smithi053 ceph-mon[80176]: pgmap v477: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:43 smithi053 ceph-mon[80176]: pgmap v478: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:43 smithi007 ceph-mon[108238]: pgmap v478: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:50:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:50:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:45 smithi053 ceph-mon[80176]: pgmap v479: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:45 smithi007 ceph-mon[108238]: pgmap v479: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:47 smithi007 ceph-mon[108238]: pgmap v480: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:47 smithi053 ceph-mon[80176]: pgmap v480: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:49 smithi053 ceph-mon[80176]: pgmap v481: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:49 smithi007 ceph-mon[108238]: pgmap v481: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:51 smithi053 ceph-mon[80176]: pgmap v482: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:51 smithi007 ceph-mon[108238]: pgmap v482: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:53 smithi007 ceph-mon[108238]: pgmap v483: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:53.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:53 smithi053 ceph-mon[80176]: pgmap v483: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:55 smithi053 ceph-mon[80176]: pgmap v484: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:55 smithi007 ceph-mon[108238]: pgmap v484: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:57 smithi053 ceph-mon[80176]: pgmap v485: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:57 smithi007 ceph-mon[108238]: pgmap v485: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:59 smithi007 ceph-mon[108238]: pgmap v486: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:50:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:50:59.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:59 smithi053 ceph-mon[80176]: pgmap v486: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:50:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:50:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:51:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:01 smithi053 ceph-mon[80176]: pgmap v487: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:01 smithi007 ceph-mon[108238]: pgmap v487: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:03.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.306+0000 7f727267e640 1 -- 172.21.15.7:0/1667658741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c100800 msgr2=0x7f726c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.306+0000 7f727267e640 1 --2- 172.21.15.7:0/1667658741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c100800 0x7f726c100c60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7254009880 tx=0x7f725402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.306+0000 7f727267e640 1 -- 172.21.15.7:0/1667658741 shutdown_connections 2024-09-21T04:51:03.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.306+0000 7f727267e640 1 --2- 172.21.15.7:0/1667658741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c100800 0x7f726c100c60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.306+0000 7f727267e640 1 --2- 172.21.15.7:0/1667658741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f726c106800 0x7f726c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.306+0000 7f727267e640 1 -- 172.21.15.7:0/1667658741 >> 172.21.15.7:0/1667658741 conn(0x7f726c0fc520 msgr2=0x7f726c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.306+0000 7f727267e640 1 -- 172.21.15.7:0/1667658741 shutdown_connections 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.306+0000 7f727267e640 1 -- 172.21.15.7:0/1667658741 wait complete. 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f727267e640 1 Processor -- start 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f727267e640 1 -- start start 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f727267e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f726c100800 0x7f726c0fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f727267e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c106800 0x7f726c0fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f727267e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f726c10dcd0 con 0x7f726c106800 2024-09-21T04:51:03.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f727267e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f726c10db50 con 0x7f726c100800 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f7263fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c106800 0x7f726c0fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f7263fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c106800 0x7f726c0fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42906/0 (socket says 172.21.15.7:42906) 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.307+0000 7f7263fff640 1 -- 172.21.15.7:0/3840674639 learned_addr learned my addr 172.21.15.7:0/3840674639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f726bfff640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f726c100800 0x7f726c0fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f7263fff640 1 -- 172.21.15.7:0/3840674639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f726c100800 msgr2=0x7f726c0fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f7263fff640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f726c100800 0x7f726c0fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f7263fff640 1 -- 172.21.15.7:0/3840674639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c0fd670 con 0x7f726c106800 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f7263fff640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c106800 0x7f726c0fd130 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7254031460 tx=0x7f72540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:03.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f7269ffb640 1 -- 172.21.15.7:0/3840674639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f725403f070 con 0x7f726c106800 2024-09-21T04:51:03.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726c0fd900 con 0x7f726c106800 2024-09-21T04:51:03.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f7269ffb640 1 -- 172.21.15.7:0/3840674639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7254004b10 con 0x7f726c106800 2024-09-21T04:51:03.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f7269ffb640 1 -- 172.21.15.7:0/3840674639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7254035030 con 0x7f726c106800 2024-09-21T04:51:03.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.308+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726c1a3db0 con 0x7f726c106800 2024-09-21T04:51:03.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.309+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f726c101f40 con 0x7f726c106800 2024-09-21T04:51:03.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.310+0000 7f7269ffb640 1 -- 172.21.15.7:0/3840674639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7254042070 con 0x7f726c106800 2024-09-21T04:51:03.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.311+0000 7f7269ffb640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f725807af40 0x7f725807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.311+0000 7f7269ffb640 1 -- 172.21.15.7:0/3840674639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f72540beba0 con 0x7f726c106800 2024-09-21T04:51:03.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.311+0000 7f726bfff640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f725807af40 0x7f725807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.311+0000 7f726bfff640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f725807af40 0x7f725807d400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f725c005fd0 tx=0x7f725c00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.315+0000 7f7269ffb640 1 -- 172.21.15.7:0/3840674639 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f725408a0a0 con 0x7f726c106800 2024-09-21T04:51:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:03 smithi053 ceph-mon[80176]: pgmap v488: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:03 smithi007 ceph-mon[108238]: pgmap v488: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.522+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f726c062630 con 0x7f725807af40 2024-09-21T04:51:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.523+0000 7f7269ffb640 1 -- 172.21.15.7:0/3840674639 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f726c062630 con 0x7f725807af40 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f725807af40 msgr2=0x7f725807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f725807af40 0x7f725807d400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f725c005fd0 tx=0x7f725c00b390 comp rx=0 tx=0).stop 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c106800 msgr2=0x7f726c0fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c106800 0x7f726c0fd130 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7254031460 tx=0x7f72540045e0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 shutdown_connections 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f725807af40 0x7f725807d400 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f726c106800 0x7f726c0fd130 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 --2- 172.21.15.7:0/3840674639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f726c100800 0x7f726c0fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 >> 172.21.15.7:0/3840674639 conn(0x7f726c0fc520 msgr2=0x7f726c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 shutdown_connections 2024-09-21T04:51:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.526+0000 7f727267e640 1 -- 172.21.15.7:0/3840674639 wait complete. 2024-09-21T04:51:03.538 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:51:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.612+0000 7f6651877640 1 -- 172.21.15.7:0/4031159272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c1033e0 msgr2=0x7f664c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.612+0000 7f6651877640 1 --2- 172.21.15.7:0/4031159272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c1033e0 0x7f664c10b8d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6634009880 tx=0x7f663402a530 comp rx=0 tx=0).stop 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.612+0000 7f6651877640 1 -- 172.21.15.7:0/4031159272 shutdown_connections 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.612+0000 7f6651877640 1 --2- 172.21.15.7:0/4031159272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c1033e0 0x7f664c10b8d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.612+0000 7f6651877640 1 --2- 172.21.15.7:0/4031159272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c102ac0 0x7f664c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.612+0000 7f6651877640 1 -- 172.21.15.7:0/4031159272 >> 172.21.15.7:0/4031159272 conn(0x7f664c0fc520 msgr2=0x7f664c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.612+0000 7f6651877640 1 -- 172.21.15.7:0/4031159272 shutdown_connections 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.612+0000 7f6651877640 1 -- 172.21.15.7:0/4031159272 wait complete. 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f6651877640 1 Processor -- start 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f6651877640 1 -- start start 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f6651877640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c102ac0 0x7f664c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f6651877640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c1033e0 0x7f664c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f6651877640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f664c10edf0 con 0x7f664c102ac0 2024-09-21T04:51:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f6651877640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f664c10ec70 con 0x7f664c1033e0 2024-09-21T04:51:03.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f664affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c102ac0 0x7f664c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f664affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c102ac0 0x7f664c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40724/0 (socket says 172.21.15.7:40724) 2024-09-21T04:51:03.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f664affd640 1 -- 172.21.15.7:0/2732113792 learned_addr learned my addr 172.21.15.7:0/2732113792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:03.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f6643fff640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c1033e0 0x7f664c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f664affd640 1 -- 172.21.15.7:0/2732113792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c1033e0 msgr2=0x7f664c19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f664affd640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c1033e0 0x7f664c19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.613+0000 7f664affd640 1 -- 172.21.15.7:0/2732113792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664c197820 con 0x7f664c102ac0 2024-09-21T04:51:03.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.614+0000 7f664affd640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c102ac0 0x7f664c19d6a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f663c00e970 tx=0x7f663c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.614+0000 7f6650875640 1 -- 172.21.15.7:0/2732113792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f663c00cac0 con 0x7f664c102ac0 2024-09-21T04:51:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.614+0000 7f6650875640 1 -- 172.21.15.7:0/2732113792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f663c0054e0 con 0x7f664c102ac0 2024-09-21T04:51:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.614+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664c197b10 con 0x7f664c102ac0 2024-09-21T04:51:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.614+0000 7f6650875640 1 -- 172.21.15.7:0/2732113792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f663c010430 con 0x7f664c102ac0 2024-09-21T04:51:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.614+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664c198070 con 0x7f664c102ac0 2024-09-21T04:51:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.615+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664c06aa90 con 0x7f664c102ac0 2024-09-21T04:51:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.615+0000 7f6650875640 1 -- 172.21.15.7:0/2732113792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f663c004f00 con 0x7f664c102ac0 2024-09-21T04:51:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.616+0000 7f6650875640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f663807af90 0x7f663807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.616+0000 7f6643fff640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f663807af90 0x7f663807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.616+0000 7f6650875640 1 -- 172.21.15.7:0/2732113792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f663c09e600 con 0x7f664c102ac0 2024-09-21T04:51:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.617+0000 7f6643fff640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f663807af90 0x7f663807d450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f664c198b70 tx=0x7f6634031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.620+0000 7f6650875640 1 -- 172.21.15.7:0/2732113792 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f663c069c20 con 0x7f664c102ac0 2024-09-21T04:51:03.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.826+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f664c1983e0 con 0x7f663807af90 2024-09-21T04:51:03.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.827+0000 7f6650875640 1 -- 172.21.15.7:0/2732113792 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f664c1983e0 con 0x7f663807af90 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f663807af90 msgr2=0x7f663807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f663807af90 0x7f663807d450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f664c198b70 tx=0x7f6634031400 comp rx=0 tx=0).stop 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c102ac0 msgr2=0x7f664c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c102ac0 0x7f664c19d6a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f663c00e970 tx=0x7f663c00ed70 comp rx=0 tx=0).stop 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 shutdown_connections 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f663807af90 0x7f663807d450 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f664c1033e0 0x7f664c19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 --2- 172.21.15.7:0/2732113792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f664c102ac0 0x7f664c19d6a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 >> 172.21.15.7:0/2732113792 conn(0x7f664c0fc520 msgr2=0x7f664c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 shutdown_connections 2024-09-21T04:51:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.829+0000 7f6651877640 1 -- 172.21.15.7:0/2732113792 wait complete. 2024-09-21T04:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.914+0000 7fda15f52640 1 -- 172.21.15.7:0/1038479373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 msgr2=0x7fda10100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.914+0000 7fda15f52640 1 --2- 172.21.15.7:0/1038479373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 0x7fda10100c60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fda04009880 tx=0x7fda0402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.914+0000 7fda15f52640 1 -- 172.21.15.7:0/1038479373 shutdown_connections 2024-09-21T04:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.914+0000 7fda15f52640 1 --2- 172.21.15.7:0/1038479373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 0x7fda10100c60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.914+0000 7fda15f52640 1 --2- 172.21.15.7:0/1038479373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda10106800 0x7fda10106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.914+0000 7fda15f52640 1 -- 172.21.15.7:0/1038479373 >> 172.21.15.7:0/1038479373 conn(0x7fda100fc520 msgr2=0x7fda100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.914+0000 7fda15f52640 1 -- 172.21.15.7:0/1038479373 shutdown_connections 2024-09-21T04:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.914+0000 7fda15f52640 1 -- 172.21.15.7:0/1038479373 wait complete. 2024-09-21T04:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda15f52640 1 Processor -- start 2024-09-21T04:51:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda15f52640 1 -- start start 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda15f52640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 0x7fda100fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda15f52640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda10106800 0x7fda100fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda15f52640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda1010dcd0 con 0x7fda10100800 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda15f52640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda1010db50 con 0x7fda10106800 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda0f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 0x7fda100fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda0f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 0x7fda100fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40748/0 (socket says 172.21.15.7:40748) 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.915+0000 7fda0f7fe640 1 -- 172.21.15.7:0/1540498352 learned_addr learned my addr 172.21.15.7:0/1540498352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda0effd640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda10106800 0x7fda100fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda0f7fe640 1 -- 172.21.15.7:0/1540498352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda10106800 msgr2=0x7fda100fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda0f7fe640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda10106800 0x7fda100fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda0f7fe640 1 -- 172.21.15.7:0/1540498352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda100fd670 con 0x7fda10100800 2024-09-21T04:51:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda0f7fe640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 0x7fda100fcbf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d6f0 tx=0x7fda0000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda0cff9640 1 -- 172.21.15.7:0/1540498352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda00017070 con 0x7fda10100800 2024-09-21T04:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda0cff9640 1 -- 172.21.15.7:0/1540498352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fda0000de00 con 0x7fda10100800 2024-09-21T04:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda100fd960 con 0x7fda10100800 2024-09-21T04:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda0cff9640 1 -- 172.21.15.7:0/1540498352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda00004bb0 con 0x7fda10100800 2024-09-21T04:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.916+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda101a3e70 con 0x7fda10100800 2024-09-21T04:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.917+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda10101f40 con 0x7fda10100800 2024-09-21T04:51:03.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.918+0000 7fda0cff9640 1 -- 172.21.15.7:0/1540498352 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fda00004e60 con 0x7fda10100800 2024-09-21T04:51:03.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.918+0000 7fda0cff9640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd9f807af30 0x7fd9f807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:03.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.918+0000 7fda0cff9640 1 -- 172.21.15.7:0/1540498352 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fda0009dd30 con 0x7fda10100800 2024-09-21T04:51:03.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.918+0000 7fda0effd640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd9f807af30 0x7fd9f807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:03.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.919+0000 7fda0effd640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd9f807af30 0x7fd9f807d3f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda04031460 tx=0x7fda04009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:03.923+0000 7fda0cff9640 1 -- 172.21.15.7:0/1540498352 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda00069230 con 0x7fda10100800 2024-09-21T04:51:04.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.116+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fda10062630 con 0x7fd9f807af30 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.127+0000 7fda0cff9640 1 -- 172.21.15.7:0/1540498352 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fda10062630 con 0x7fd9f807af30 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (13m) 7m ago 23m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (16m) 7m ago 24m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (15m) 6m ago 23m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (11m) 7m ago 24m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (11m) 6m ago 23m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (12m) 7m ago 23m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (7m) 7m ago 18m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (7m) 7m ago 18m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:51:04.130 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (18m) 6m ago 18m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (6m) 6m ago 18m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (15m) 7m ago 25m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (14m) 6m ago 23m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (11m) 7m ago 25m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (11m) 6m ago 23m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (13m) 7m ago 24m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (13m) 6m ago 23m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 7m ago 22m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (10m) 7m ago 21m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (9m) 7m ago 21m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (8m) 6m ago 20m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (8m) 6m ago 20m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (7m) 6m ago 19m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:51:04.131 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (13m) 7m ago 23m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:51:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd9f807af30 msgr2=0x7fd9f807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd9f807af30 0x7fd9f807d3f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda04031460 tx=0x7fda04009560 comp rx=0 tx=0).stop 2024-09-21T04:51:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 msgr2=0x7fda100fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 0x7fda100fcbf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d6f0 tx=0x7fda0000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 shutdown_connections 2024-09-21T04:51:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd9f807af30 0x7fd9f807d3f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda10106800 0x7fda100fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 --2- 172.21.15.7:0/1540498352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda10100800 0x7fda100fcbf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.129+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 >> 172.21.15.7:0/1540498352 conn(0x7fda100fc520 msgr2=0x7fda1006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.130+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 shutdown_connections 2024-09-21T04:51:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.130+0000 7fda15f52640 1 -- 172.21.15.7:0/1540498352 wait complete. 2024-09-21T04:51:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.213+0000 7f1fd321a640 1 -- 172.21.15.7:0/2780366827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 msgr2=0x7f1fcc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.213+0000 7f1fd321a640 1 --2- 172.21.15.7:0/2780366827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 0x7f1fcc101d60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009880 tx=0x7f1fbc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.213+0000 7f1fd321a640 1 -- 172.21.15.7:0/2780366827 shutdown_connections 2024-09-21T04:51:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.213+0000 7f1fd321a640 1 --2- 172.21.15.7:0/2780366827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 0x7f1fcc101d60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.213+0000 7f1fd321a640 1 --2- 172.21.15.7:0/2780366827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fcc107900 0x7f1fcc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.213+0000 7f1fd321a640 1 -- 172.21.15.7:0/2780366827 >> 172.21.15.7:0/2780366827 conn(0x7f1fcc0fd600 msgr2=0x7f1fcc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.213+0000 7f1fd321a640 1 -- 172.21.15.7:0/2780366827 shutdown_connections 2024-09-21T04:51:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.213+0000 7f1fd321a640 1 -- 172.21.15.7:0/2780366827 wait complete. 2024-09-21T04:51:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd321a640 1 Processor -- start 2024-09-21T04:51:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd321a640 1 -- start start 2024-09-21T04:51:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd321a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 0x7f1fcc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd321a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fcc107900 0x7f1fcc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd321a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10ee30 con 0x7f1fcc101900 2024-09-21T04:51:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd321a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc10ecb0 con 0x7f1fcc107900 2024-09-21T04:51:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd0fb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 0x7f1fcc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd0fb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 0x7f1fcc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40766/0 (socket says 172.21.15.7:40766) 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd0fb6640 1 -- 172.21.15.7:0/1865951224 learned_addr learned my addr 172.21.15.7:0/1865951224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd0fb6640 1 -- 172.21.15.7:0/1865951224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fcc107900 msgr2=0x7f1fcc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd0fb6640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fcc107900 0x7f1fcc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.214+0000 7f1fd0fb6640 1 -- 172.21.15.7:0/1865951224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc074f70 con 0x7f1fcc101900 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.215+0000 7f1fd0fb6640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 0x7f1fcc0744f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400b420 tx=0x7f1fb400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.215+0000 7f1fc9ffb640 1 -- 172.21.15.7:0/1865951224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1fb4011630 con 0x7f1fcc101900 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.215+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc075260 con 0x7f1fcc101900 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.215+0000 7f1fc9ffb640 1 -- 172.21.15.7:0/1865951224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1fb4016070 con 0x7f1fcc101900 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.215+0000 7f1fc9ffb640 1 -- 172.21.15.7:0/1865951224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1fb4010d40 con 0x7f1fcc101900 2024-09-21T04:51:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.215+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc1a3ec0 con 0x7f1fcc101900 2024-09-21T04:51:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.216+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc103040 con 0x7f1fcc101900 2024-09-21T04:51:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.217+0000 7f1fc9ffb640 1 -- 172.21.15.7:0/1865951224 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1fb401b460 con 0x7f1fcc101900 2024-09-21T04:51:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.217+0000 7f1fc9ffb640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1fb807af30 0x7f1fb807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.217+0000 7f1fc9ffb640 1 -- 172.21.15.7:0/1865951224 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f1fb409d810 con 0x7f1fcc101900 2024-09-21T04:51:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.217+0000 7f1fcbfff640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1fb807af30 0x7f1fb807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.218+0000 7f1fcbfff640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1fb807af30 0x7f1fb807d3f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0096c0 tx=0x7f1fbc000bc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.222+0000 7f1fc9ffb640 1 -- 172.21.15.7:0/1865951224 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1fb4021250 con 0x7f1fcc101900 2024-09-21T04:51:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.485+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f1fcc109e90 con 0x7f1fcc101900 2024-09-21T04:51:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.485+0000 7f1fc9ffb640 1 -- 172.21.15.7:0/1865951224 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f1fb4068db0 con 0x7f1fcc101900 2024-09-21T04:51:04.488 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:51:04.489 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:51:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:51:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:51:04.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:51:04.490 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1fb807af30 msgr2=0x7f1fb807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1fb807af30 0x7f1fb807d3f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0096c0 tx=0x7f1fbc000bc0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 msgr2=0x7f1fcc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 0x7f1fcc0744f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400b420 tx=0x7f1fb400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 shutdown_connections 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1fb807af30 0x7f1fb807d3f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fcc107900 0x7f1fcc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 --2- 172.21.15.7:0/1865951224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101900 0x7f1fcc0744f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 >> 172.21.15.7:0/1865951224 conn(0x7f1fcc0fd600 msgr2=0x7f1fcc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 shutdown_connections 2024-09-21T04:51:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.488+0000 7f1fd321a640 1 -- 172.21.15.7:0/1865951224 wait complete. 2024-09-21T04:51:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.572+0000 7fdebb4e8640 1 -- 172.21.15.7:0/3454773272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4107900 msgr2=0x7fdeb4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.572+0000 7fdebb4e8640 1 --2- 172.21.15.7:0/3454773272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4107900 0x7fdeb4107ce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009880 tx=0x7fdeb002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.572+0000 7fdebb4e8640 1 -- 172.21.15.7:0/3454773272 shutdown_connections 2024-09-21T04:51:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.572+0000 7fdebb4e8640 1 --2- 172.21.15.7:0/3454773272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4101900 0x7fdeb4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.572+0000 7fdebb4e8640 1 --2- 172.21.15.7:0/3454773272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4107900 0x7fdeb4107ce0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.572+0000 7fdebb4e8640 1 -- 172.21.15.7:0/3454773272 >> 172.21.15.7:0/3454773272 conn(0x7fdeb40fd600 msgr2=0x7fdeb40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.572+0000 7fdebb4e8640 1 -- 172.21.15.7:0/3454773272 shutdown_connections 2024-09-21T04:51:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.572+0000 7fdebb4e8640 1 -- 172.21.15.7:0/3454773272 wait complete. 2024-09-21T04:51:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdebb4e8640 1 Processor -- start 2024-09-21T04:51:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdebb4e8640 1 -- start start 2024-09-21T04:51:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdebb4e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4101900 0x7fdeb40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdebb4e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4107900 0x7fdeb4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdebb4e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410ee30 con 0x7fdeb4101900 2024-09-21T04:51:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdebb4e8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410ecb0 con 0x7fdeb4107900 2024-09-21T04:51:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdeb9284640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4101900 0x7fdeb40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdeb9284640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4101900 0x7fdeb40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40788/0 (socket says 172.21.15.7:40788) 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdeb9284640 1 -- 172.21.15.7:0/2648030516 learned_addr learned my addr 172.21.15.7:0/2648030516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdeb8a83640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4107900 0x7fdeb4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdeb9284640 1 -- 172.21.15.7:0/2648030516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4107900 msgr2=0x7fdeb4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdeb9284640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4107900 0x7fdeb4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.573+0000 7fdeb9284640 1 -- 172.21.15.7:0/2648030516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb4074f70 con 0x7fdeb4101900 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.574+0000 7fdeb9284640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4101900 0x7fdeb40744f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0031460 tx=0x7fdeb00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.574+0000 7fdeaa7fc640 1 -- 172.21.15.7:0/2648030516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdeb003f070 con 0x7fdeb4101900 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.574+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb4075200 con 0x7fdeb4101900 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.574+0000 7fdeaa7fc640 1 -- 172.21.15.7:0/2648030516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdeb0004cf0 con 0x7fdeb4101900 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.574+0000 7fdeaa7fc640 1 -- 172.21.15.7:0/2648030516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdeb0035030 con 0x7fdeb4101900 2024-09-21T04:51:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.574+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb41a3e20 con 0x7fdeb4101900 2024-09-21T04:51:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.575+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb4103040 con 0x7fdeb4101900 2024-09-21T04:51:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.576+0000 7fdeaa7fc640 1 -- 172.21.15.7:0/2648030516 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdeb0042070 con 0x7fdeb4101900 2024-09-21T04:51:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.576+0000 7fdeaa7fc640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdea007af30 0x7fdea007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.576+0000 7fdeaa7fc640 1 -- 172.21.15.7:0/2648030516 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fdeb00beb60 con 0x7fdeb4101900 2024-09-21T04:51:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.576+0000 7fdeb8a83640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdea007af30 0x7fdea007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.577+0000 7fdeb8a83640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdea007af30 0x7fdea007d3f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4075f80 tx=0x7fdea4008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.581+0000 7fdeaa7fc640 1 -- 172.21.15.7:0/2648030516 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdeb008a100 con 0x7fdeb4101900 2024-09-21T04:51:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.814+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fdeb4109e70 con 0x7fdeb4101900 2024-09-21T04:51:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.814+0000 7fdeaa7fc640 1 -- 172.21.15.7:0/2648030516 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7fdeb008efb0 con 0x7fdeb4101900 2024-09-21T04:51:04.817 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:51:04.817 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:51:04.817 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:51:04.818 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:51:04.819 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:51:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:51:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:51:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.816+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdea007af30 msgr2=0x7fdea007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.816+0000 7fdebb4e8640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdea007af30 0x7fdea007d3f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4075f80 tx=0x7fdea4008040 comp rx=0 tx=0).stop 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.816+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4101900 msgr2=0x7fdeb40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.816+0000 7fdebb4e8640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4101900 0x7fdeb40744f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0031460 tx=0x7fdeb00048f0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.817+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 shutdown_connections 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.817+0000 7fdebb4e8640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdea007af30 0x7fdea007d3f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.817+0000 7fdebb4e8640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeb4107900 0x7fdeb4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.817+0000 7fdebb4e8640 1 --2- 172.21.15.7:0/2648030516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdeb4101900 0x7fdeb40744f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.817+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 >> 172.21.15.7:0/2648030516 conn(0x7fdeb40fd600 msgr2=0x7fdeb40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.817+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 shutdown_connections 2024-09-21T04:51:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.817+0000 7fdebb4e8640 1 -- 172.21.15.7:0/2648030516 wait complete. 2024-09-21T04:51:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.902+0000 7fdab9803640 1 -- 172.21.15.7:0/2154649415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 msgr2=0x7fdab4100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.902+0000 7fdab9803640 1 --2- 172.21.15.7:0/2154649415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 0x7fdab4100590 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009880 tx=0x7fdaa802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.903+0000 7fdab9803640 1 -- 172.21.15.7:0/2154649415 shutdown_connections 2024-09-21T04:51:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.903+0000 7fdab9803640 1 --2- 172.21.15.7:0/2154649415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 0x7fdab4100590 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.903+0000 7fdab9803640 1 --2- 172.21.15.7:0/2154649415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab4069920 0x7fdab4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.903+0000 7fdab9803640 1 -- 172.21.15.7:0/2154649415 >> 172.21.15.7:0/2154649415 conn(0x7fdab40fbaf0 msgr2=0x7fdab40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.903+0000 7fdab9803640 1 -- 172.21.15.7:0/2154649415 shutdown_connections 2024-09-21T04:51:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.903+0000 7fdab9803640 1 -- 172.21.15.7:0/2154649415 wait complete. 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.903+0000 7fdab9803640 1 Processor -- start 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.903+0000 7fdab9803640 1 -- start start 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab9803640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab4069920 0x7fdab4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab9803640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 0x7fdab4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab9803640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410db50 con 0x7fdab4069d90 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab9803640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410d9d0 con 0x7fdab4069920 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 0x7fdab4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 0x7fdab4198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40794/0 (socket says 172.21.15.7:40794) 2024-09-21T04:51:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab27fc640 1 -- 172.21.15.7:0/1846707838 learned_addr learned my addr 172.21.15.7:0/1846707838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab2ffd640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab4069920 0x7fdab4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab27fc640 1 -- 172.21.15.7:0/1846707838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab4069920 msgr2=0x7fdab4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab27fc640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab4069920 0x7fdab4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab27fc640 1 -- 172.21.15.7:0/1846707838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab41991a0 con 0x7fdab4069d90 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab2ffd640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab4069920 0x7fdab4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab27fc640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 0x7fdab4198ba0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8031460 tx=0x7fdaa8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.904+0000 7fdab8801640 1 -- 172.21.15.7:0/1846707838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdaa803f070 con 0x7fdab4069d90 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.905+0000 7fdab8801640 1 -- 172.21.15.7:0/1846707838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdaa80035f0 con 0x7fdab4069d90 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.905+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab4199430 con 0x7fdab4069d90 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.905+0000 7fdab8801640 1 -- 172.21.15.7:0/1846707838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdaa8035030 con 0x7fdab4069d90 2024-09-21T04:51:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.905+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab41a19e0 con 0x7fdab4069d90 2024-09-21T04:51:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.906+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab4101530 con 0x7fdab4069d90 2024-09-21T04:51:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.906+0000 7fdab8801640 1 -- 172.21.15.7:0/1846707838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdaa8042070 con 0x7fdab4069d90 2024-09-21T04:51:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.907+0000 7fdab8801640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda9c07af80 0x7fda9c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.907+0000 7fdab8801640 1 -- 172.21.15.7:0/1846707838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fdaa80be9e0 con 0x7fdab4069d90 2024-09-21T04:51:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.907+0000 7fdab2ffd640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda9c07af80 0x7fda9c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.907+0000 7fdab2ffd640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda9c07af80 0x7fda9c07d440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009c20 tx=0x7fdaa4005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:04.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:04.911+0000 7fdab8801640 1 -- 172.21.15.7:0/1846707838 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdaa8089f60 con 0x7fdab4069d90 2024-09-21T04:51:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.119+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdab40fc9b0 con 0x7fda9c07af80 2024-09-21T04:51:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.121+0000 7fdab8801640 1 -- 172.21.15.7:0/1846707838 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fdab40fc9b0 con 0x7fda9c07af80 2024-09-21T04:51:05.123 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:51:05.124 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:51:05.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda9c07af80 msgr2=0x7fda9c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda9c07af80 0x7fda9c07d440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009c20 tx=0x7fdaa4005f90 comp rx=0 tx=0).stop 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 msgr2=0x7fdab4198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 0x7fdab4198ba0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8031460 tx=0x7fdaa8004530 comp rx=0 tx=0).stop 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 shutdown_connections 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda9c07af80 0x7fda9c07d440 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdab4069d90 0x7fdab4198ba0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 --2- 172.21.15.7:0/1846707838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdab4069920 0x7fdab4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.123+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 >> 172.21.15.7:0/1846707838 conn(0x7fdab40fbaf0 msgr2=0x7fdab40fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.124+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 shutdown_connections 2024-09-21T04:51:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.124+0000 7fdab9803640 1 -- 172.21.15.7:0/1846707838 wait complete. 2024-09-21T04:51:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 -- 172.21.15.7:0/1792772820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424100800 msgr2=0x7fd424100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 --2- 172.21.15.7:0/1792772820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424100800 0x7fd424100c60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd418009880 tx=0x7fd41802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 -- 172.21.15.7:0/1792772820 shutdown_connections 2024-09-21T04:51:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 --2- 172.21.15.7:0/1792772820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424100800 0x7fd424100c60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 --2- 172.21.15.7:0/1792772820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd424106800 0x7fd424106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 -- 172.21.15.7:0/1792772820 >> 172.21.15.7:0/1792772820 conn(0x7fd4240fc520 msgr2=0x7fd4240fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 -- 172.21.15.7:0/1792772820 shutdown_connections 2024-09-21T04:51:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 -- 172.21.15.7:0/1792772820 wait complete. 2024-09-21T04:51:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.209+0000 7fd429277640 1 Processor -- start 2024-09-21T04:51:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd429277640 1 -- start start 2024-09-21T04:51:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd429277640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd424100800 0x7fd42419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd429277640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424106800 0x7fd42419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd429277640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd42410dcb0 con 0x7fd424106800 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd429277640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd42410db30 con 0x7fd424100800 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd4227fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424106800 0x7fd42419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd4227fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424106800 0x7fd42419dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40810/0 (socket says 172.21.15.7:40810) 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd4227fc640 1 -- 172.21.15.7:0/4223970982 learned_addr learned my addr 172.21.15.7:0/4223970982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd422ffd640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd424100800 0x7fd42419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd4227fc640 1 -- 172.21.15.7:0/4223970982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd424100800 msgr2=0x7fd42419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd4227fc640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd424100800 0x7fd42419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.210+0000 7fd4227fc640 1 -- 172.21.15.7:0/4223970982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4241977f0 con 0x7fd424106800 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.211+0000 7fd4227fc640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424106800 0x7fd42419dbb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd418031460 tx=0x7fd4180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.211+0000 7fd40bfff640 1 -- 172.21.15.7:0/4223970982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd41803f070 con 0x7fd424106800 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.211+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd424197a80 con 0x7fd424106800 2024-09-21T04:51:05.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.211+0000 7fd40bfff640 1 -- 172.21.15.7:0/4223970982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd418004cf0 con 0x7fd424106800 2024-09-21T04:51:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.211+0000 7fd40bfff640 1 -- 172.21.15.7:0/4223970982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd418035030 con 0x7fd424106800 2024-09-21T04:51:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.211+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd424197fb0 con 0x7fd424106800 2024-09-21T04:51:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.213+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd424101f40 con 0x7fd424106800 2024-09-21T04:51:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.213+0000 7fd40bfff640 1 -- 172.21.15.7:0/4223970982 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd418042070 con 0x7fd424106800 2024-09-21T04:51:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.214+0000 7fd40bfff640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd40c07af30 0x7fd40c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.214+0000 7fd40bfff640 1 -- 172.21.15.7:0/4223970982 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd4180beb20 con 0x7fd424106800 2024-09-21T04:51:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.214+0000 7fd422ffd640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd40c07af30 0x7fd40c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.214+0000 7fd422ffd640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd40c07af30 0x7fd40c07d3f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd414005e10 tx=0x7fd414005da0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:05.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.218+0000 7fd40bfff640 1 -- 172.21.15.7:0/4223970982 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd41808a0c0 con 0x7fd424106800 2024-09-21T04:51:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:05 smithi053 ceph-mon[80176]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:05 smithi053 ceph-mon[80176]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:05 smithi053 ceph-mon[80176]: pgmap v489: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:05 smithi053 ceph-mon[80176]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:05 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1865951224' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:51:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:05 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2648030516' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:51:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:05 smithi007 ceph-mon[108238]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:05 smithi007 ceph-mon[108238]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:05 smithi007 ceph-mon[108238]: pgmap v489: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:05 smithi007 ceph-mon[108238]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:05 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1865951224' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:51:05.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:05 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2648030516' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:51:05.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.478+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fd424198d20 con 0x7fd424106800 2024-09-21T04:51:05.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.478+0000 7fd40bfff640 1 -- 172.21.15.7:0/4223970982 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fd41808ef70 con 0x7fd424106800 2024-09-21T04:51:05.481 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:51:05.481 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:51:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:51:05.482 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:51:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:51:05.482 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:51:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:51:05.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.480+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd40c07af30 msgr2=0x7fd40c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.480+0000 7fd429277640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd40c07af30 0x7fd40c07d3f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd414005e10 tx=0x7fd414005da0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.480+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424106800 msgr2=0x7fd42419dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.480+0000 7fd429277640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424106800 0x7fd42419dbb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd418031460 tx=0x7fd4180048f0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.481+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 shutdown_connections 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.481+0000 7fd429277640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd40c07af30 0x7fd40c07d3f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.481+0000 7fd429277640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd424106800 0x7fd42419dbb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.481+0000 7fd429277640 1 --2- 172.21.15.7:0/4223970982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd424100800 0x7fd42419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.481+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 >> 172.21.15.7:0/4223970982 conn(0x7fd4240fc520 msgr2=0x7fd42406a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.481+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 shutdown_connections 2024-09-21T04:51:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:05.481+0000 7fd429277640 1 -- 172.21.15.7:0/4223970982 wait complete. 2024-09-21T04:51:06.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:06 smithi053 ceph-mon[80176]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:06.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:06 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4223970982' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:51:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:06 smithi007 ceph-mon[108238]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:06 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4223970982' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:51:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:07 smithi053 ceph-mon[80176]: pgmap v490: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:07 smithi007 ceph-mon[108238]: pgmap v490: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:09.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:09 smithi053 ceph-mon[80176]: pgmap v491: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:09 smithi007 ceph-mon[108238]: pgmap v491: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:11 smithi007 ceph-mon[108238]: pgmap v492: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:11 smithi053 ceph-mon[80176]: pgmap v492: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:13 smithi053 ceph-mon[80176]: pgmap v493: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:13 smithi007 ceph-mon[108238]: pgmap v493: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:51:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:51:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:15 smithi053 ceph-mon[80176]: pgmap v494: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:15 smithi007 ceph-mon[108238]: pgmap v494: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:17 smithi007 ceph-mon[108238]: pgmap v495: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:17 smithi053 ceph-mon[80176]: pgmap v495: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:19.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:19 smithi053 ceph-mon[80176]: pgmap v496: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:19 smithi007 ceph-mon[108238]: pgmap v496: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:21 smithi053 ceph-mon[80176]: pgmap v497: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:21 smithi007 ceph-mon[108238]: pgmap v497: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:23 smithi007 ceph-mon[108238]: pgmap v498: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:23.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:23 smithi053 ceph-mon[80176]: pgmap v498: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:25 smithi053 ceph-mon[80176]: pgmap v499: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:25 smithi007 ceph-mon[108238]: pgmap v499: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:27.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:27 smithi053 ceph-mon[80176]: pgmap v500: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:27 smithi007 ceph-mon[108238]: pgmap v500: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:29 smithi007 ceph-mon[108238]: pgmap v501: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:51:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:29 smithi053 ceph-mon[80176]: pgmap v501: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:51:30.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:30 smithi053 ceph-mon[80176]: pgmap v502: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:30.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:30 smithi007 ceph-mon[108238]: pgmap v502: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:33 smithi053 ceph-mon[80176]: pgmap v503: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:33 smithi007 ceph-mon[108238]: pgmap v503: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:35 smithi007 ceph-mon[108238]: pgmap v504: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:35.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:35 smithi053 ceph-mon[80176]: pgmap v504: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:35.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.578+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/1376231601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 msgr2=0x7fd4ac100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:35.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.578+0000 7fd4b2bbf640 1 --2- 172.21.15.7:0/1376231601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 0x7fd4ac100c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009880 tx=0x7fd49c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.578+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/1376231601 shutdown_connections 2024-09-21T04:51:35.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.578+0000 7fd4b2bbf640 1 --2- 172.21.15.7:0/1376231601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 0x7fd4ac100c50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.578+0000 7fd4b2bbf640 1 --2- 172.21.15.7:0/1376231601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ac1067f0 0x7fd4ac106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.578+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/1376231601 >> 172.21.15.7:0/1376231601 conn(0x7fd4ac0fc550 msgr2=0x7fd4ac0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:35.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.578+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/1376231601 shutdown_connections 2024-09-21T04:51:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.578+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/1376231601 wait complete. 2024-09-21T04:51:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b2bbf640 1 Processor -- start 2024-09-21T04:51:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b2bbf640 1 -- start start 2024-09-21T04:51:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b2bbf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 0x7fd4ac1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b2bbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ac1067f0 0x7fd4ac198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b2bbf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac10dcc0 con 0x7fd4ac1007f0 2024-09-21T04:51:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b2bbf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac10db40 con 0x7fd4ac1067f0 2024-09-21T04:51:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b095b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 0x7fd4ac1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b095b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 0x7fd4ac1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49752/0 (socket says 172.21.15.7:49752) 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.579+0000 7fd4b095b640 1 -- 172.21.15.7:0/3720597443 learned_addr learned my addr 172.21.15.7:0/3720597443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4a3fff640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ac1067f0 0x7fd4ac198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4b095b640 1 -- 172.21.15.7:0/3720597443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ac1067f0 msgr2=0x7fd4ac198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4b095b640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ac1067f0 0x7fd4ac198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4b095b640 1 -- 172.21.15.7:0/3720597443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac199330 con 0x7fd4ac1007f0 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4b095b640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 0x7fd4ac1987f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800d6f0 tx=0x7fd4a800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4a1ffb640 1 -- 172.21.15.7:0/3720597443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd4a8017070 con 0x7fd4ac1007f0 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4a1ffb640 1 -- 172.21.15.7:0/3720597443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd4a800de00 con 0x7fd4ac1007f0 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ac199620 con 0x7fd4ac1007f0 2024-09-21T04:51:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.580+0000 7fd4a1ffb640 1 -- 172.21.15.7:0/3720597443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd4a8004bb0 con 0x7fd4ac1007f0 2024-09-21T04:51:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.581+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ac1a1c20 con 0x7fd4ac1007f0 2024-09-21T04:51:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.582+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ac101f30 con 0x7fd4ac1007f0 2024-09-21T04:51:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.582+0000 7fd4a1ffb640 1 -- 172.21.15.7:0/3720597443 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd4a8009a30 con 0x7fd4ac1007f0 2024-09-21T04:51:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.583+0000 7fd4a1ffb640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd49407af80 0x7fd49407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:35.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.583+0000 7fd4a1ffb640 1 -- 172.21.15.7:0/3720597443 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd4a809dbc0 con 0x7fd4ac1007f0 2024-09-21T04:51:35.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.583+0000 7fd4a3fff640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd49407af80 0x7fd49407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:35.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.583+0000 7fd4a3fff640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd49407af80 0x7fd49407d440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac06a7f0 tx=0x7fd49c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.587+0000 7fd4a1ffb640 1 -- 172.21.15.7:0/3720597443 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4ac199620 con 0x7fd4ac1007f0 2024-09-21T04:51:35.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.794+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd4ac062630 con 0x7fd49407af80 2024-09-21T04:51:35.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.795+0000 7fd4a1ffb640 1 -- 172.21.15.7:0/3720597443 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd4ac062630 con 0x7fd49407af80 2024-09-21T04:51:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.797+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd49407af80 msgr2=0x7fd49407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.797+0000 7fd4b2bbf640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd49407af80 0x7fd49407d440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac06a7f0 tx=0x7fd49c031400 comp rx=0 tx=0).stop 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.797+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 msgr2=0x7fd4ac1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.797+0000 7fd4b2bbf640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 0x7fd4ac1987f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800d6f0 tx=0x7fd4a800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.798+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 shutdown_connections 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.798+0000 7fd4b2bbf640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd49407af80 0x7fd49407d440 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.798+0000 7fd4b2bbf640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4ac1067f0 0x7fd4ac198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.798+0000 7fd4b2bbf640 1 --2- 172.21.15.7:0/3720597443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ac1007f0 0x7fd4ac1987f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.798+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 >> 172.21.15.7:0/3720597443 conn(0x7fd4ac0fc550 msgr2=0x7fd4ac0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.798+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 shutdown_connections 2024-09-21T04:51:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.798+0000 7fd4b2bbf640 1 -- 172.21.15.7:0/3720597443 wait complete. 2024-09-21T04:51:35.810 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:51:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.883+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3565043460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 msgr2=0x7fc224101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.883+0000 7fc22aa5a640 1 --2- 172.21.15.7:0/3565043460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 0x7fc224101d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc218009880 tx=0x7fc21802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.883+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3565043460 shutdown_connections 2024-09-21T04:51:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.883+0000 7fc22aa5a640 1 --2- 172.21.15.7:0/3565043460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 0x7fc224101d70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.883+0000 7fc22aa5a640 1 --2- 172.21.15.7:0/3565043460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc224107910 0x7fc224107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.883+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3565043460 >> 172.21.15.7:0/3565043460 conn(0x7fc2240fd630 msgr2=0x7fc2240ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.884+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3565043460 shutdown_connections 2024-09-21T04:51:35.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.884+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3565043460 wait complete. 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.884+0000 7fc22aa5a640 1 Processor -- start 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc22aa5a640 1 -- start start 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc22aa5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 0x7fc22419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc22aa5a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc224107910 0x7fc22419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc22aa5a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc22410ede0 con 0x7fc224101910 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc22aa5a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc22410ec60 con 0x7fc224107910 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc223fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 0x7fc22419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc223fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 0x7fc22419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49776/0 (socket says 172.21.15.7:49776) 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc223fff640 1 -- 172.21.15.7:0/3670372855 learned_addr learned my addr 172.21.15.7:0/3670372855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc2237fe640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc224107910 0x7fc22419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:35.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc223fff640 1 -- 172.21.15.7:0/3670372855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc224107910 msgr2=0x7fc22419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc223fff640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc224107910 0x7fc22419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.885+0000 7fc223fff640 1 -- 172.21.15.7:0/3670372855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc224197780 con 0x7fc224101910 2024-09-21T04:51:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.886+0000 7fc223fff640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 0x7fc22419d600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc21400d6f0 tx=0x7fc21400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.886+0000 7fc2217fa640 1 -- 172.21.15.7:0/3670372855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc214017070 con 0x7fc224101910 2024-09-21T04:51:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.886+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc224197a70 con 0x7fc224101910 2024-09-21T04:51:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.886+0000 7fc2217fa640 1 -- 172.21.15.7:0/3670372855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc21400de00 con 0x7fc224101910 2024-09-21T04:51:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.886+0000 7fc2217fa640 1 -- 172.21.15.7:0/3670372855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc214004bb0 con 0x7fc224101910 2024-09-21T04:51:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.886+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc224197fd0 con 0x7fc224101910 2024-09-21T04:51:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.887+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc224103050 con 0x7fc224101910 2024-09-21T04:51:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.888+0000 7fc2217fa640 1 -- 172.21.15.7:0/3670372855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc214004e60 con 0x7fc224101910 2024-09-21T04:51:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.888+0000 7fc2217fa640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc20c07af30 0x7fc20c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.888+0000 7fc2217fa640 1 -- 172.21.15.7:0/3670372855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc21409dc00 con 0x7fc224101910 2024-09-21T04:51:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.888+0000 7fc2237fe640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc20c07af30 0x7fc20c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:35.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.889+0000 7fc2237fe640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc20c07af30 0x7fc20c07d3f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc224198ad0 tx=0x7fc218031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:35.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:35.893+0000 7fc2217fa640 1 -- 172.21.15.7:0/3670372855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2140a3020 con 0x7fc224101910 2024-09-21T04:51:36.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.099+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc224198280 con 0x7fc20c07af30 2024-09-21T04:51:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.100+0000 7fc2217fa640 1 -- 172.21.15.7:0/3670372855 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fc224198280 con 0x7fc20c07af30 2024-09-21T04:51:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc20c07af30 msgr2=0x7fc20c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc20c07af30 0x7fc20c07d3f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc224198ad0 tx=0x7fc218031400 comp rx=0 tx=0).stop 2024-09-21T04:51:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 msgr2=0x7fc22419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 0x7fc22419d600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc21400d6f0 tx=0x7fc21400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 shutdown_connections 2024-09-21T04:51:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc20c07af30 0x7fc20c07d3f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc224107910 0x7fc22419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 --2- 172.21.15.7:0/3670372855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc224101910 0x7fc22419d600 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 >> 172.21.15.7:0/3670372855 conn(0x7fc2240fd630 msgr2=0x7fc2240fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 shutdown_connections 2024-09-21T04:51:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.102+0000 7fc22aa5a640 1 -- 172.21.15.7:0/3670372855 wait complete. 2024-09-21T04:51:36.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.186+0000 7f31ed921640 1 -- 172.21.15.7:0/74439123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 msgr2=0x7f31e8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.186+0000 7f31ed921640 1 --2- 172.21.15.7:0/74439123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 0x7f31e8102e90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009c20 tx=0x7f31d802aa00 comp rx=0 tx=0).stop 2024-09-21T04:51:36.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.187+0000 7f31ed921640 1 -- 172.21.15.7:0/74439123 shutdown_connections 2024-09-21T04:51:36.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.187+0000 7f31ed921640 1 --2- 172.21.15.7:0/74439123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31e81033d0 0x7f31e810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.187+0000 7f31ed921640 1 --2- 172.21.15.7:0/74439123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 0x7f31e8102e90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.187+0000 7f31ed921640 1 -- 172.21.15.7:0/74439123 >> 172.21.15.7:0/74439123 conn(0x7f31e80fc550 msgr2=0x7f31e80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:36.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.187+0000 7f31ed921640 1 -- 172.21.15.7:0/74439123 shutdown_connections 2024-09-21T04:51:36.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.187+0000 7f31ed921640 1 -- 172.21.15.7:0/74439123 wait complete. 2024-09-21T04:51:36.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.187+0000 7f31ed921640 1 Processor -- start 2024-09-21T04:51:36.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31ed921640 1 -- start start 2024-09-21T04:51:36.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31ed921640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 0x7f31e819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31ed921640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31e81033d0 0x7f31e819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31ed921640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31e810edc0 con 0x7f31e8102ab0 2024-09-21T04:51:36.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31ed921640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f31e810ec40 con 0x7f31e81033d0 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 0x7f31e819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 0x7f31e819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49786/0 (socket says 172.21.15.7:49786) 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31e6ffd640 1 -- 172.21.15.7:0/4213958829 learned_addr learned my addr 172.21.15.7:0/4213958829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31e67fc640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31e81033d0 0x7f31e819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31e6ffd640 1 -- 172.21.15.7:0/4213958829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31e81033d0 msgr2=0x7f31e819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31e6ffd640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31e81033d0 0x7f31e819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.188+0000 7f31e6ffd640 1 -- 172.21.15.7:0/4213958829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e81977b0 con 0x7f31e8102ab0 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.189+0000 7f31e6ffd640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 0x7f31e819d630 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f31d802af20 tx=0x7f31d802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.189+0000 7f31ec91f640 1 -- 172.21.15.7:0/4213958829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f31d8047070 con 0x7f31e8102ab0 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.189+0000 7f31ec91f640 1 -- 172.21.15.7:0/4213958829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f31d8003e30 con 0x7f31e8102ab0 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.189+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e8197a40 con 0x7f31e8102ab0 2024-09-21T04:51:36.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.189+0000 7f31ec91f640 1 -- 172.21.15.7:0/4213958829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f31d8033050 con 0x7f31e8102ab0 2024-09-21T04:51:36.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.189+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e8197f70 con 0x7f31e8102ab0 2024-09-21T04:51:36.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.190+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e806a580 con 0x7f31e8102ab0 2024-09-21T04:51:36.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.190+0000 7f31ec91f640 1 -- 172.21.15.7:0/4213958829 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f31d804a070 con 0x7f31e8102ab0 2024-09-21T04:51:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.191+0000 7f31ec91f640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d007af90 0x7f31d007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.191+0000 7f31e67fc640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d007af90 0x7f31d007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.191+0000 7f31ec91f640 1 -- 172.21.15.7:0/4213958829 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f31d803e030 con 0x7f31e8102ab0 2024-09-21T04:51:36.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.192+0000 7f31e67fc640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d007af90 0x7f31d007d450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31dc004630 tx=0x7f31dc005c50 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:36.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.196+0000 7f31ec91f640 1 -- 172.21.15.7:0/4213958829 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31d808a710 con 0x7f31e8102ab0 2024-09-21T04:51:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.396+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f31e81982e0 con 0x7f31d007af90 2024-09-21T04:51:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.406+0000 7f31ec91f640 1 -- 172.21.15.7:0/4213958829 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f31e81982e0 con 0x7f31d007af90 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (13m) 7m ago 24m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (16m) 7m ago 24m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (16m) 7m ago 23m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (11m) 7m ago 24m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (11m) 7m ago 23m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (13m) 7m ago 24m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (7m) 7m ago 19m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:51:36.410 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (8m) 7m ago 19m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (19m) 7m ago 19m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (7m) 7m ago 19m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (15m) 7m ago 25m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (14m) 7m ago 23m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (12m) 7m ago 26m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (12m) 7m ago 23m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (14m) 7m ago 24m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (14m) 7m ago 23m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (11m) 7m ago 22m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (10m) 7m ago 22m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (10m) 7m ago 21m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:51:36.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (9m) 7m ago 21m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:51:36.412 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (8m) 7m ago 20m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:51:36.412 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (8m) 7m ago 20m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:51:36.412 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (13m) 7m ago 24m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:51:36.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d007af90 msgr2=0x7f31d007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d007af90 0x7f31d007d450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31dc004630 tx=0x7f31dc005c50 comp rx=0 tx=0).stop 2024-09-21T04:51:36.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 msgr2=0x7f31e819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 0x7f31e819d630 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f31d802af20 tx=0x7f31d802ee60 comp rx=0 tx=0).stop 2024-09-21T04:51:36.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 shutdown_connections 2024-09-21T04:51:36.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f31d007af90 0x7f31d007d450 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31e81033d0 0x7f31e819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 --2- 172.21.15.7:0/4213958829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31e8102ab0 0x7f31e819d630 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 >> 172.21.15.7:0/4213958829 conn(0x7f31e80fc550 msgr2=0x7f31e8108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:36.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 shutdown_connections 2024-09-21T04:51:36.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.408+0000 7f31ed921640 1 -- 172.21.15.7:0/4213958829 wait complete. 2024-09-21T04:51:36.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.493+0000 7fe82a061640 1 -- 172.21.15.7:0/516383873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824101900 msgr2=0x7fe824101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.493+0000 7fe82a061640 1 --2- 172.21.15.7:0/516383873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824101900 0x7fe824101d60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009880 tx=0x7fe80c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 -- 172.21.15.7:0/516383873 shutdown_connections 2024-09-21T04:51:36.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 --2- 172.21.15.7:0/516383873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824101900 0x7fe824101d60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 --2- 172.21.15.7:0/516383873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe824107900 0x7fe824107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 -- 172.21.15.7:0/516383873 >> 172.21.15.7:0/516383873 conn(0x7fe8240fd600 msgr2=0x7fe8240ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:36.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 -- 172.21.15.7:0/516383873 shutdown_connections 2024-09-21T04:51:36.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 -- 172.21.15.7:0/516383873 wait complete. 2024-09-21T04:51:36.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 Processor -- start 2024-09-21T04:51:36.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 -- start start 2024-09-21T04:51:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe824101900 0x7fe82419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824107900 0x7fe82419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410edd0 con 0x7fe824107900 2024-09-21T04:51:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.494+0000 7fe82a061640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410ec50 con 0x7fe824101900 2024-09-21T04:51:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe81bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824107900 0x7fe82419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe81bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824107900 0x7fe82419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49804/0 (socket says 172.21.15.7:49804) 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe81bfff640 1 -- 172.21.15.7:0/335028096 learned_addr learned my addr 172.21.15.7:0/335028096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe8237fe640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe824101900 0x7fe82419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe81bfff640 1 -- 172.21.15.7:0/335028096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe824101900 msgr2=0x7fe82419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe81bfff640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe824101900 0x7fe82419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe81bfff640 1 -- 172.21.15.7:0/335028096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe824197800 con 0x7fe824107900 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe8237fe640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe824101900 0x7fe82419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe81bfff640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824107900 0x7fe82419dbc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe80c031460 tx=0x7fe80c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe8217fa640 1 -- 172.21.15.7:0/335028096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe80c03f070 con 0x7fe824107900 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe824197a90 con 0x7fe824107900 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe8217fa640 1 -- 172.21.15.7:0/335028096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe80c0035f0 con 0x7fe824107900 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe8217fa640 1 -- 172.21.15.7:0/335028096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe80c035030 con 0x7fe824107900 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.495+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe824197fc0 con 0x7fe824107900 2024-09-21T04:51:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.496+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe824103040 con 0x7fe824107900 2024-09-21T04:51:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.497+0000 7fe8217fa640 1 -- 172.21.15.7:0/335028096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe80c042070 con 0x7fe824107900 2024-09-21T04:51:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.498+0000 7fe8217fa640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81007af30 0x7fe81007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.498+0000 7fe8217fa640 1 -- 172.21.15.7:0/335028096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe80c0bea80 con 0x7fe824107900 2024-09-21T04:51:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.498+0000 7fe8237fe640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81007af30 0x7fe81007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.498+0000 7fe8237fe640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81007af30 0x7fe81007d3f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe8140058c0 tx=0x7fe814005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:36.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.502+0000 7fe8217fa640 1 -- 172.21.15.7:0/335028096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe80c08a020 con 0x7fe824107900 2024-09-21T04:51:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.766+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fe824109e70 con 0x7fe824107900 2024-09-21T04:51:36.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.767+0000 7fe8217fa640 1 -- 172.21.15.7:0/335028096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fe80c08c070 con 0x7fe824107900 2024-09-21T04:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:51:36.772 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:51:36.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81007af30 msgr2=0x7fe81007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81007af30 0x7fe81007d3f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe8140058c0 tx=0x7fe814005850 comp rx=0 tx=0).stop 2024-09-21T04:51:36.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824107900 msgr2=0x7fe82419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824107900 0x7fe82419dbc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe80c031460 tx=0x7fe80c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 shutdown_connections 2024-09-21T04:51:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81007af30 0x7fe81007d3f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe824107900 0x7fe82419dbc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 --2- 172.21.15.7:0/335028096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe824101900 0x7fe82419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 >> 172.21.15.7:0/335028096 conn(0x7fe8240fd600 msgr2=0x7fe8240fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 shutdown_connections 2024-09-21T04:51:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.769+0000 7fe82a061640 1 -- 172.21.15.7:0/335028096 wait complete. 2024-09-21T04:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.854+0000 7f4fdca7e640 1 -- 172.21.15.7:0/2848003153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100c00 msgr2=0x7f4fd8104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.854+0000 7f4fdca7e640 1 --2- 172.21.15.7:0/2848003153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100c00 0x7f4fd8104c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009880 tx=0x7f4fcc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.854+0000 7f4fdca7e640 1 -- 172.21.15.7:0/2848003153 shutdown_connections 2024-09-21T04:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.854+0000 7f4fdca7e640 1 --2- 172.21.15.7:0/2848003153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100c00 0x7f4fd8104c50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.854+0000 7f4fdca7e640 1 --2- 172.21.15.7:0/2848003153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd8100250 0x7f4fd8100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.854+0000 7f4fdca7e640 1 -- 172.21.15.7:0/2848003153 >> 172.21.15.7:0/2848003153 conn(0x7f4fd80fbc90 msgr2=0x7f4fd80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:36.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.854+0000 7f4fdca7e640 1 -- 172.21.15.7:0/2848003153 shutdown_connections 2024-09-21T04:51:36.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.854+0000 7f4fdca7e640 1 -- 172.21.15.7:0/2848003153 wait complete. 2024-09-21T04:51:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fdca7e640 1 Processor -- start 2024-09-21T04:51:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fdca7e640 1 -- start start 2024-09-21T04:51:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fdca7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100250 0x7f4fd8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fdca7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd8100c00 0x7f4fd8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fdca7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810d4d0 con 0x7f4fd8100250 2024-09-21T04:51:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fdca7e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd810d350 con 0x7f4fd8100c00 2024-09-21T04:51:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100250 0x7f4fd8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fd659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100250 0x7f4fd8198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49820/0 (socket says 172.21.15.7:49820) 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fd659c640 1 -- 172.21.15.7:0/3749073221 learned_addr learned my addr 172.21.15.7:0/3749073221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fd5d9b640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd8100c00 0x7f4fd8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fd659c640 1 -- 172.21.15.7:0/3749073221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd8100c00 msgr2=0x7f4fd8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fd659c640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd8100c00 0x7f4fd8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.855+0000 7f4fd659c640 1 -- 172.21.15.7:0/3749073221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd8199400 con 0x7f4fd8100250 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.856+0000 7f4fd659c640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100250 0x7f4fd8198870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800eb10 tx=0x7f4fc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.856+0000 7f4fc77fe640 1 -- 172.21.15.7:0/3749073221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4fc800cce0 con 0x7f4fd8100250 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.856+0000 7f4fc77fe640 1 -- 172.21.15.7:0/3749073221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4fc8005020 con 0x7f4fd8100250 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.856+0000 7f4fc77fe640 1 -- 172.21.15.7:0/3749073221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4fc8010660 con 0x7f4fd8100250 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.856+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd81a18e0 con 0x7f4fd8100250 2024-09-21T04:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.856+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a1ce0 con 0x7f4fd8100250 2024-09-21T04:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.857+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd8101730 con 0x7f4fd8100250 2024-09-21T04:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.858+0000 7f4fc77fe640 1 -- 172.21.15.7:0/3749073221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4fc80051d0 con 0x7f4fd8100250 2024-09-21T04:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.858+0000 7f4fc77fe640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4fc007af80 0x7f4fc007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.858+0000 7f4fc77fe640 1 -- 172.21.15.7:0/3749073221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4fc809e780 con 0x7f4fd8100250 2024-09-21T04:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.858+0000 7f4fd5d9b640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4fc007af80 0x7f4fc007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.859+0000 7f4fd5d9b640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4fc007af80 0x7f4fc007d440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fd810e840 tx=0x7f4fcc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:36.862+0000 7f4fc77fe640 1 -- 172.21.15.7:0/3749073221 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4fc8069da0 con 0x7f4fd8100250 2024-09-21T04:51:37.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.087+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4fd81a2040 con 0x7f4fd8100250 2024-09-21T04:51:37.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.087+0000 7f4fc77fe640 1 -- 172.21.15.7:0/3749073221 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f4fc806ec50 con 0x7f4fd8100250 2024-09-21T04:51:37.091 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:51:37.091 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:51:37.091 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:51:37.091 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:51:37.091 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:51:37.092 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:51:37.093 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:51:37.094 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:51:37.095 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:51:37.095 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:51:37.095 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4fc007af80 msgr2=0x7f4fc007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4fc007af80 0x7f4fc007d440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4fd810e840 tx=0x7f4fcc031400 comp rx=0 tx=0).stop 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100250 msgr2=0x7f4fd8198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100250 0x7f4fd8198870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800eb10 tx=0x7f4fc800c2d0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 shutdown_connections 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4fc007af80 0x7f4fc007d440 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fd8100c00 0x7f4fd8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 --2- 172.21.15.7:0/3749073221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd8100250 0x7f4fd8198870 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 >> 172.21.15.7:0/3749073221 conn(0x7f4fd80fbc90 msgr2=0x7f4fd80fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 shutdown_connections 2024-09-21T04:51:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.090+0000 7f4fdca7e640 1 -- 172.21.15.7:0/3749073221 wait complete. 2024-09-21T04:51:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 -- 172.21.15.7:0/2482270613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 msgr2=0x7f51e810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 --2- 172.21.15.7:0/2482270613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 0x7f51e810b920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f51d80098b0 tx=0x7f51d802a560 comp rx=0 tx=0).stop 2024-09-21T04:51:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 -- 172.21.15.7:0/2482270613 shutdown_connections 2024-09-21T04:51:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 --2- 172.21.15.7:0/2482270613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 0x7f51e810b920 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 --2- 172.21.15.7:0/2482270613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e8102b10 0x7f51e8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 -- 172.21.15.7:0/2482270613 >> 172.21.15.7:0/2482270613 conn(0x7f51e80fc550 msgr2=0x7f51e80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 -- 172.21.15.7:0/2482270613 shutdown_connections 2024-09-21T04:51:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 -- 172.21.15.7:0/2482270613 wait complete. 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 Processor -- start 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.176+0000 7f51eeb51640 1 -- start start 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51eeb51640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e8102b10 0x7f51e819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51eeb51640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 0x7f51e819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51eeb51640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51e810edc0 con 0x7f51e8103430 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51eeb51640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51e810ec40 con 0x7f51e8102b10 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 0x7f51e819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 0x7f51e819dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49838/0 (socket says 172.21.15.7:49838) 2024-09-21T04:51:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51e7fff640 1 -- 172.21.15.7:0/619992161 learned_addr learned my addr 172.21.15.7:0/619992161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51ec8ed640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e8102b10 0x7f51e819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51e7fff640 1 -- 172.21.15.7:0/619992161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e8102b10 msgr2=0x7f51e819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51e7fff640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e8102b10 0x7f51e819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51e7fff640 1 -- 172.21.15.7:0/619992161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e81977f0 con 0x7f51e8103430 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.177+0000 7f51e7fff640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 0x7f51e819dbb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f51d8031460 tx=0x7f51d80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.178+0000 7f51e5ffb640 1 -- 172.21.15.7:0/619992161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f51d803f070 con 0x7f51e8103430 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.178+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e8197a80 con 0x7f51e8103430 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.178+0000 7f51e5ffb640 1 -- 172.21.15.7:0/619992161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f51d80044d0 con 0x7f51e8103430 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.178+0000 7f51e5ffb640 1 -- 172.21.15.7:0/619992161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f51d8035030 con 0x7f51e8103430 2024-09-21T04:51:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.178+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e8197fb0 con 0x7f51e8103430 2024-09-21T04:51:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.179+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e806a520 con 0x7f51e8103430 2024-09-21T04:51:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.179+0000 7f51e5ffb640 1 -- 172.21.15.7:0/619992161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f51d8042070 con 0x7f51e8103430 2024-09-21T04:51:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.180+0000 7f51e5ffb640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51d407af30 0x7f51d407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.180+0000 7f51e5ffb640 1 -- 172.21.15.7:0/619992161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f51d80bea90 con 0x7f51e8103430 2024-09-21T04:51:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.180+0000 7f51ec8ed640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51d407af30 0x7f51d407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.181+0000 7f51ec8ed640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51d407af30 0x7f51d407d3f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51d0005e10 tx=0x7f51d0005da0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:37.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.184+0000 7f51e5ffb640 1 -- 172.21.15.7:0/619992161 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51d8004cf0 con 0x7f51e8103430 2024-09-21T04:51:37.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.388+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f51e8198260 con 0x7f51d407af30 2024-09-21T04:51:37.393 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:37 smithi007 ceph-mon[108238]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:37.393 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:37 smithi007 ceph-mon[108238]: pgmap v505: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:37.393 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:37 smithi007 ceph-mon[108238]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:37.393 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:37 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/335028096' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:51:37.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.390+0000 7f51e5ffb640 1 -- 172.21.15.7:0/619992161 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f51e8198260 con 0x7f51d407af30 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:51:37.394 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:51:37.395 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51d407af30 msgr2=0x7f51d407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51d407af30 0x7f51d407d3f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51d0005e10 tx=0x7f51d0005da0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 msgr2=0x7f51e819dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 0x7f51e819dbb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f51d8031460 tx=0x7f51d80048f0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 shutdown_connections 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f51d407af30 0x7f51d407d3f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e8103430 0x7f51e819dbb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 --2- 172.21.15.7:0/619992161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e8102b10 0x7f51e819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 >> 172.21.15.7:0/619992161 conn(0x7f51e80fc550 msgr2=0x7f51e8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:37.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 shutdown_connections 2024-09-21T04:51:37.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.392+0000 7f51eeb51640 1 -- 172.21.15.7:0/619992161 wait complete. 2024-09-21T04:51:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:37 smithi053 ceph-mon[80176]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:37 smithi053 ceph-mon[80176]: pgmap v505: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:37 smithi053 ceph-mon[80176]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:37 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/335028096' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:51:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.475+0000 7f1c6efce640 1 -- 172.21.15.7:0/1258563311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 msgr2=0x7f1c68106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.475+0000 7f1c6efce640 1 --2- 172.21.15.7:0/1258563311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 0x7f1c68106be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009880 tx=0x7f1c5802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.475+0000 7f1c6efce640 1 -- 172.21.15.7:0/1258563311 shutdown_connections 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.475+0000 7f1c6efce640 1 --2- 172.21.15.7:0/1258563311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c68100800 0x7f1c68100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.475+0000 7f1c6efce640 1 --2- 172.21.15.7:0/1258563311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 0x7f1c68106be0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.475+0000 7f1c6efce640 1 -- 172.21.15.7:0/1258563311 >> 172.21.15.7:0/1258563311 conn(0x7f1c680fc520 msgr2=0x7f1c680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.475+0000 7f1c6efce640 1 -- 172.21.15.7:0/1258563311 shutdown_connections 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.475+0000 7f1c6efce640 1 -- 172.21.15.7:0/1258563311 wait complete. 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c6efce640 1 Processor -- start 2024-09-21T04:51:37.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c6efce640 1 -- start start 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c6efce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c68100800 0x7f1c6819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c6efce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 0x7f1c6819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c6efce640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6810dcb0 con 0x7f1c68106800 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c6efce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6810db30 con 0x7f1c68100800 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 0x7f1c6819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 0x7f1c6819dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49852/0 (socket says 172.21.15.7:49852) 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c67fff640 1 -- 172.21.15.7:0/3387378166 learned_addr learned my addr 172.21.15.7:0/3387378166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.476+0000 7f1c6cd6a640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c68100800 0x7f1c6819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c67fff640 1 -- 172.21.15.7:0/3387378166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c68100800 msgr2=0x7f1c6819d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c67fff640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c68100800 0x7f1c6819d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c67fff640 1 -- 172.21.15.7:0/3387378166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c68197850 con 0x7f1c68106800 2024-09-21T04:51:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c67fff640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 0x7f1c6819dc10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c8a0 tx=0x7f1c5c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c65ffb640 1 -- 172.21.15.7:0/3387378166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1c5c0079b0 con 0x7f1c68106800 2024-09-21T04:51:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c65ffb640 1 -- 172.21.15.7:0/3387378166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c5c007b50 con 0x7f1c68106800 2024-09-21T04:51:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c68197b40 con 0x7f1c68106800 2024-09-21T04:51:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c65ffb640 1 -- 172.21.15.7:0/3387378166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1c5c018700 con 0x7f1c68106800 2024-09-21T04:51:37.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.477+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c681980d0 con 0x7f1c68106800 2024-09-21T04:51:37.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.478+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c68101f40 con 0x7f1c68106800 2024-09-21T04:51:37.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.479+0000 7f1c65ffb640 1 -- 172.21.15.7:0/3387378166 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1c5c007d00 con 0x7f1c68106800 2024-09-21T04:51:37.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.480+0000 7f1c65ffb640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c5007af30 0x7f1c5007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:51:37.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.480+0000 7f1c65ffb640 1 -- 172.21.15.7:0/3387378166 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f1c5c09e280 con 0x7f1c68106800 2024-09-21T04:51:37.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.480+0000 7f1c6cd6a640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c5007af30 0x7f1c5007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:51:37.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.480+0000 7f1c6cd6a640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c5007af30 0x7f1c5007d3f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1c58000c00 tx=0x7f1c58031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:51:37.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.484+0000 7f1c65ffb640 1 -- 172.21.15.7:0/3387378166 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1c5c014030 con 0x7f1c68106800 2024-09-21T04:51:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.743+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f1c68198d80 con 0x7f1c68106800 2024-09-21T04:51:37.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.744+0000 7f1c65ffb640 1 -- 172.21.15.7:0/3387378166 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f1c5c069820 con 0x7f1c68106800 2024-09-21T04:51:37.747 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:51:37.748 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:51:37.748 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:51:37.748 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:51:37.748 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:51:37.748 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:51:37.748 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:51:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c5007af30 msgr2=0x7f1c5007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c5007af30 0x7f1c5007d3f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1c58000c00 tx=0x7f1c58031400 comp rx=0 tx=0).stop 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 msgr2=0x7f1c6819dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 0x7f1c6819dc10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c8a0 tx=0x7f1c5c00cd60 comp rx=0 tx=0).stop 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 shutdown_connections 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c5007af30 0x7f1c5007d3f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c68106800 0x7f1c6819dc10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 --2- 172.21.15.7:0/3387378166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c68100800 0x7f1c6819d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 >> 172.21.15.7:0/3387378166 conn(0x7f1c680fc520 msgr2=0x7f1c6806a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 shutdown_connections 2024-09-21T04:51:37.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:51:37.746+0000 7f1c6efce640 1 -- 172.21.15.7:0/3387378166 wait complete. 2024-09-21T04:51:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:38 smithi053 ceph-mon[80176]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:38 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3749073221' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:51:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:38 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3387378166' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:51:38.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:38 smithi007 ceph-mon[108238]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:38.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:38 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3749073221' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:51:38.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:38 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3387378166' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:51:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:39 smithi053 ceph-mon[80176]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:39 smithi053 ceph-mon[80176]: pgmap v506: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:39 smithi007 ceph-mon[108238]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:51:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:39 smithi007 ceph-mon[108238]: pgmap v506: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:41 smithi007 ceph-mon[108238]: pgmap v507: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:41 smithi053 ceph-mon[80176]: pgmap v507: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:43 smithi053 ceph-mon[80176]: pgmap v508: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:43 smithi007 ceph-mon[108238]: pgmap v508: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:51:44.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:51:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:45 smithi053 ceph-mon[80176]: pgmap v509: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:45 smithi007 ceph-mon[108238]: pgmap v509: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:47 smithi007 ceph-mon[108238]: pgmap v510: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:47 smithi053 ceph-mon[80176]: pgmap v510: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:49 smithi053 ceph-mon[80176]: pgmap v511: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:49 smithi007 ceph-mon[108238]: pgmap v511: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:51 smithi053 ceph-mon[80176]: pgmap v512: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:51 smithi007 ceph-mon[108238]: pgmap v512: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:53.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:53 smithi053 ceph-mon[80176]: pgmap v513: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:53 smithi007 ceph-mon[108238]: pgmap v513: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:55 smithi053 ceph-mon[80176]: pgmap v514: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:55 smithi007 ceph-mon[108238]: pgmap v514: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:57 smithi053 ceph-mon[80176]: pgmap v515: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:57 smithi007 ceph-mon[108238]: pgmap v515: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:59 smithi007 ceph-mon[108238]: pgmap v516: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:51:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:51:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:59 smithi053 ceph-mon[80176]: pgmap v516: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:51:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:51:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:52:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:01 smithi053 ceph-mon[80176]: pgmap v517: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:01 smithi007 ceph-mon[108238]: pgmap v517: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:03 smithi053 ceph-mon[80176]: pgmap v518: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:03 smithi007 ceph-mon[108238]: pgmap v518: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:05 smithi007 ceph-mon[108238]: pgmap v519: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:05 smithi053 ceph-mon[80176]: pgmap v519: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:07 smithi053 ceph-mon[80176]: pgmap v520: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:07 smithi007 ceph-mon[108238]: pgmap v520: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:07.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.845+0000 7fd83b088640 1 -- 172.21.15.7:0/1256268839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 msgr2=0x7fd8341004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:07.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.845+0000 7fd83b088640 1 --2- 172.21.15.7:0/1256268839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 0x7fd8341004c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd824009880 tx=0x7fd82402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:52:07.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.845+0000 7fd83b088640 1 -- 172.21.15.7:0/1256268839 shutdown_connections 2024-09-21T04:52:07.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.845+0000 7fd83b088640 1 --2- 172.21.15.7:0/1256268839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd834100a90 0x7fd8341065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:07.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.845+0000 7fd83b088640 1 --2- 172.21.15.7:0/1256268839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 0x7fd8341004c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:07.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.845+0000 7fd83b088640 1 -- 172.21.15.7:0/1256268839 >> 172.21.15.7:0/1256268839 conn(0x7fd8340fbc90 msgr2=0x7fd8340fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:07.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.845+0000 7fd83b088640 1 -- 172.21.15.7:0/1256268839 shutdown_connections 2024-09-21T04:52:07.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.845+0000 7fd83b088640 1 -- 172.21.15.7:0/1256268839 wait complete. 2024-09-21T04:52:07.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.846+0000 7fd83b088640 1 Processor -- start 2024-09-21T04:52:07.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.846+0000 7fd83b088640 1 -- start start 2024-09-21T04:52:07.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.846+0000 7fd83b088640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 0x7fd834101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.846+0000 7fd83b088640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd834100a90 0x7fd8341015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.846+0000 7fd83b088640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410ee30 con 0x7fd8341000e0 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.846+0000 7fd83b088640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410ecb0 con 0x7fd834100a90 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.846+0000 7fd838e24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 0x7fd834101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd838e24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 0x7fd834101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52996/0 (socket says 172.21.15.7:52996) 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd838e24640 1 -- 172.21.15.7:0/740806022 learned_addr learned my addr 172.21.15.7:0/740806022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd833fff640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd834100a90 0x7fd8341015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd838e24640 1 -- 172.21.15.7:0/740806022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd834100a90 msgr2=0x7fd8341015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd838e24640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd834100a90 0x7fd8341015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd838e24640 1 -- 172.21.15.7:0/740806022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd834101ae0 con 0x7fd8341000e0 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd838e24640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 0x7fd834101060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd824031460 tx=0x7fd8240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd831ffb640 1 -- 172.21.15.7:0/740806022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd82403f070 con 0x7fd8341000e0 2024-09-21T04:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd831ffb640 1 -- 172.21.15.7:0/740806022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd8240044d0 con 0x7fd8341000e0 2024-09-21T04:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd834101d70 con 0x7fd8341000e0 2024-09-21T04:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.847+0000 7fd831ffb640 1 -- 172.21.15.7:0/740806022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd824035030 con 0x7fd8341000e0 2024-09-21T04:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.848+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8341a3eb0 con 0x7fd8341000e0 2024-09-21T04:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.849+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8341004c0 con 0x7fd8341000e0 2024-09-21T04:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.849+0000 7fd831ffb640 1 -- 172.21.15.7:0/740806022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd824042070 con 0x7fd8341000e0 2024-09-21T04:52:07.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.850+0000 7fd831ffb640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd82007af90 0x7fd82007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:07.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.850+0000 7fd831ffb640 1 -- 172.21.15.7:0/740806022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd8240bea90 con 0x7fd8341000e0 2024-09-21T04:52:07.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.850+0000 7fd833fff640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd82007af90 0x7fd82007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:07.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.851+0000 7fd833fff640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd82007af90 0x7fd82007d450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd81c005e10 tx=0x7fd81c005da0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:07.855+0000 7fd831ffb640 1 -- 172.21.15.7:0/740806022 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd824004cf0 con 0x7fd8341000e0 2024-09-21T04:52:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.063+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd8341030e0 con 0x7fd82007af90 2024-09-21T04:52:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.064+0000 7fd831ffb640 1 -- 172.21.15.7:0/740806022 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd8341030e0 con 0x7fd82007af90 2024-09-21T04:52:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd82007af90 msgr2=0x7fd82007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd82007af90 0x7fd82007d450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd81c005e10 tx=0x7fd81c005da0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 msgr2=0x7fd834101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 0x7fd834101060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd824031460 tx=0x7fd8240048f0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 shutdown_connections 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd82007af90 0x7fd82007d450 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd834100a90 0x7fd8341015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 --2- 172.21.15.7:0/740806022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8341000e0 0x7fd834101060 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.066+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 >> 172.21.15.7:0/740806022 conn(0x7fd8340fbc90 msgr2=0x7fd8340fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.067+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 shutdown_connections 2024-09-21T04:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.067+0000 7fd83b088640 1 -- 172.21.15.7:0/740806022 wait complete. 2024-09-21T04:52:08.079 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:52:08.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.152+0000 7f74ff606640 1 -- 172.21.15.7:0/1600463556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8107900 msgr2=0x7f74f8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.152+0000 7f74ff606640 1 --2- 172.21.15.7:0/1600463556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8107900 0x7f74f8107ce0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009880 tx=0x7f74f402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.152+0000 7f74ff606640 1 -- 172.21.15.7:0/1600463556 shutdown_connections 2024-09-21T04:52:08.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.152+0000 7f74ff606640 1 --2- 172.21.15.7:0/1600463556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f8101900 0x7f74f8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.152+0000 7f74ff606640 1 --2- 172.21.15.7:0/1600463556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8107900 0x7f74f8107ce0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.152+0000 7f74ff606640 1 -- 172.21.15.7:0/1600463556 >> 172.21.15.7:0/1600463556 conn(0x7f74f80fd600 msgr2=0x7f74f80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:08.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.152+0000 7f74ff606640 1 -- 172.21.15.7:0/1600463556 shutdown_connections 2024-09-21T04:52:08.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.152+0000 7f74ff606640 1 -- 172.21.15.7:0/1600463556 wait complete. 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.153+0000 7f74ff606640 1 Processor -- start 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.153+0000 7f74ff606640 1 -- start start 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74ff606640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8101900 0x7f74f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74ff606640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f8107900 0x7f74f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74ff606640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74f810edd0 con 0x7f74f8101900 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74ff606640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74f810ec50 con 0x7f74f8107900 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74fd3a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8101900 0x7f74f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74fd3a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8101900 0x7f74f80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53028/0 (socket says 172.21.15.7:53028) 2024-09-21T04:52:08.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74fd3a2640 1 -- 172.21.15.7:0/674984378 learned_addr learned my addr 172.21.15.7:0/674984378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:08.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74fcba1640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f8107900 0x7f74f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74fd3a2640 1 -- 172.21.15.7:0/674984378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f8107900 msgr2=0x7f74f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74fd3a2640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f8107900 0x7f74f8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74fd3a2640 1 -- 172.21.15.7:0/674984378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f8074f70 con 0x7f74f8101900 2024-09-21T04:52:08.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74fd3a2640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8101900 0x7f74f80744f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f74f4031460 tx=0x7f74f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:08.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.154+0000 7f74ee7fc640 1 -- 172.21.15.7:0/674984378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74f403f070 con 0x7f74f8101900 2024-09-21T04:52:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.155+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f8075200 con 0x7f74f8101900 2024-09-21T04:52:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.155+0000 7f74ee7fc640 1 -- 172.21.15.7:0/674984378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f74f4004b10 con 0x7f74f8101900 2024-09-21T04:52:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.155+0000 7f74ee7fc640 1 -- 172.21.15.7:0/674984378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74f4035030 con 0x7f74f8101900 2024-09-21T04:52:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.155+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f81a3d80 con 0x7f74f8101900 2024-09-21T04:52:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.156+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f8103040 con 0x7f74f8101900 2024-09-21T04:52:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.156+0000 7f74ee7fc640 1 -- 172.21.15.7:0/674984378 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f74f4042070 con 0x7f74f8101900 2024-09-21T04:52:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.157+0000 7f74ee7fc640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74e407af40 0x7f74e407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.157+0000 7f74ee7fc640 1 -- 172.21.15.7:0/674984378 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f74f40beae0 con 0x7f74f8101900 2024-09-21T04:52:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.157+0000 7f74fcba1640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74e407af40 0x7f74e407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.158+0000 7f74fcba1640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74e407af40 0x7f74e407d400 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74f8075f80 tx=0x7f74e800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.161+0000 7f74ee7fc640 1 -- 172.21.15.7:0/674984378 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74f408a080 con 0x7f74f8101900 2024-09-21T04:52:08.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.369+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f74f8062630 con 0x7f74e407af40 2024-09-21T04:52:08.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.370+0000 7f74ee7fc640 1 -- 172.21.15.7:0/674984378 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f74f8062630 con 0x7f74e407af40 2024-09-21T04:52:08.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.372+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74e407af40 msgr2=0x7f74e407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.372+0000 7f74ff606640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74e407af40 0x7f74e407d400 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f74f8075f80 tx=0x7f74e800b390 comp rx=0 tx=0).stop 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.372+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8101900 msgr2=0x7f74f80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.372+0000 7f74ff606640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8101900 0x7f74f80744f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f74f4031460 tx=0x7f74f40045e0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.373+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 shutdown_connections 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.373+0000 7f74ff606640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f74e407af40 0x7f74e407d400 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.373+0000 7f74ff606640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f8107900 0x7f74f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.373+0000 7f74ff606640 1 --2- 172.21.15.7:0/674984378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74f8101900 0x7f74f80744f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.373+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 >> 172.21.15.7:0/674984378 conn(0x7f74f80fd600 msgr2=0x7f74f80fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.373+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 shutdown_connections 2024-09-21T04:52:08.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.373+0000 7f74ff606640 1 -- 172.21.15.7:0/674984378 wait complete. 2024-09-21T04:52:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 -- 172.21.15.7:0/2054434966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41007f0 msgr2=0x7f7ad4100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 --2- 172.21.15.7:0/2054434966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41007f0 0x7f7ad4100c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009880 tx=0x7f7ac402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 -- 172.21.15.7:0/2054434966 shutdown_connections 2024-09-21T04:52:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 --2- 172.21.15.7:0/2054434966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41007f0 0x7f7ad4100c50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 --2- 172.21.15.7:0/2054434966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad41067f0 0x7f7ad4106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 -- 172.21.15.7:0/2054434966 >> 172.21.15.7:0/2054434966 conn(0x7f7ad40fc550 msgr2=0x7f7ad40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 -- 172.21.15.7:0/2054434966 shutdown_connections 2024-09-21T04:52:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 -- 172.21.15.7:0/2054434966 wait complete. 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.457+0000 7f7adc216640 1 Processor -- start 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7adc216640 1 -- start start 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7adc216640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad41007f0 0x7f7ad41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7adc216640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41067f0 0x7f7ad4198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7adc216640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad410dcc0 con 0x7f7ad41067f0 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7adc216640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad410db40 con 0x7f7ad41007f0 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7ad97b1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41067f0 0x7f7ad4198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7ad97b1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41067f0 0x7f7ad4198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53036/0 (socket says 172.21.15.7:53036) 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7ad97b1640 1 -- 172.21.15.7:0/1840648855 learned_addr learned my addr 172.21.15.7:0/1840648855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7ad9fb2640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad41007f0 0x7f7ad41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7ad97b1640 1 -- 172.21.15.7:0/1840648855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad41007f0 msgr2=0x7f7ad41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7ad97b1640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad41007f0 0x7f7ad41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7ad97b1640 1 -- 172.21.15.7:0/1840648855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad4199330 con 0x7f7ad41067f0 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7ad97b1640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41067f0 0x7f7ad4198d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4031460 tx=0x7f7ac4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.458+0000 7f7acaffd640 1 -- 172.21.15.7:0/1840648855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ac403f070 con 0x7f7ad41067f0 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.459+0000 7f7acaffd640 1 -- 172.21.15.7:0/1840648855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ac4004a90 con 0x7f7ad41067f0 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.459+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad41995c0 con 0x7f7ad41067f0 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.459+0000 7f7acaffd640 1 -- 172.21.15.7:0/1840648855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ac4035030 con 0x7f7ad41067f0 2024-09-21T04:52:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.459+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad41a1b70 con 0x7f7ad41067f0 2024-09-21T04:52:08.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.460+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad4101f30 con 0x7f7ad41067f0 2024-09-21T04:52:08.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.460+0000 7f7acaffd640 1 -- 172.21.15.7:0/1840648855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7ac4042070 con 0x7f7ad41067f0 2024-09-21T04:52:08.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.461+0000 7f7acaffd640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ac007af90 0x7f7ac007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.461+0000 7f7acaffd640 1 -- 172.21.15.7:0/1840648855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7ac40beae0 con 0x7f7ad41067f0 2024-09-21T04:52:08.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.461+0000 7f7ad9fb2640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ac007af90 0x7f7ac007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.461+0000 7f7ad9fb2640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ac007af90 0x7f7ac007d450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009b60 tx=0x7f7ad0005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.465+0000 7f7acaffd640 1 -- 172.21.15.7:0/1840648855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ac408a100 con 0x7f7ad41067f0 2024-09-21T04:52:08.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.660+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7ad4062630 con 0x7f7ac007af90 2024-09-21T04:52:08.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.670+0000 7f7acaffd640 1 -- 172.21.15.7:0/1840648855 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f7ad4062630 con 0x7f7ac007af90 2024-09-21T04:52:08.673 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:52:08.673 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (14m) 8m ago 25m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (17m) 8m ago 25m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (16m) 7m ago 24m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (12m) 8m ago 25m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (12m) 7m ago 24m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (13m) 8m ago 24m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (8m) 8m ago 19m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (8m) 8m ago 19m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (19m) 7m ago 19m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (8m) 7m ago 19m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (16m) 8m ago 26m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (15m) 7m ago 24m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (12m) 8m ago 26m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:52:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (12m) 7m ago 24m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (15m) 8m ago 25m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (14m) 7m ago 24m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (11m) 8m ago 23m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (11m) 8m ago 22m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (10m) 8m ago 22m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (10m) 7m ago 21m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (9m) 7m ago 21m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (8m) 7m ago 20m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:52:08.675 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (14m) 8m ago 24m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ac007af90 msgr2=0x7f7ac007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ac007af90 0x7f7ac007d450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009b60 tx=0x7f7ad0005f90 comp rx=0 tx=0).stop 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41067f0 msgr2=0x7f7ad4198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41067f0 0x7f7ad4198d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4031460 tx=0x7f7ac4004530 comp rx=0 tx=0).stop 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 shutdown_connections 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ac007af90 0x7f7ac007d450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ad41067f0 0x7f7ad4198d30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 --2- 172.21.15.7:0/1840648855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad41007f0 0x7f7ad41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 >> 172.21.15.7:0/1840648855 conn(0x7f7ad40fc550 msgr2=0x7f7ad40fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 shutdown_connections 2024-09-21T04:52:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.673+0000 7f7adc216640 1 -- 172.21.15.7:0/1840648855 wait complete. 2024-09-21T04:52:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.757+0000 7f78e78e5640 1 -- 172.21.15.7:0/66555793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 msgr2=0x7f78e010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.757+0000 7f78e78e5640 1 --2- 172.21.15.7:0/66555793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 0x7f78e010b8d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009880 tx=0x7f78dc02a530 comp rx=0 tx=0).stop 2024-09-21T04:52:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.757+0000 7f78e78e5640 1 -- 172.21.15.7:0/66555793 shutdown_connections 2024-09-21T04:52:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.757+0000 7f78e78e5640 1 --2- 172.21.15.7:0/66555793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 0x7f78e010b8d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.757+0000 7f78e78e5640 1 --2- 172.21.15.7:0/66555793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e0102ac0 0x7f78e0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.757+0000 7f78e78e5640 1 -- 172.21.15.7:0/66555793 >> 172.21.15.7:0/66555793 conn(0x7f78e00fc520 msgr2=0x7f78e00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.757+0000 7f78e78e5640 1 -- 172.21.15.7:0/66555793 shutdown_connections 2024-09-21T04:52:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.757+0000 7f78e78e5640 1 -- 172.21.15.7:0/66555793 wait complete. 2024-09-21T04:52:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e78e5640 1 Processor -- start 2024-09-21T04:52:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e78e5640 1 -- start start 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e78e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e0102ac0 0x7f78e019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e78e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 0x7f78e019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e78e5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010edd0 con 0x7f78e01033e0 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e78e5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f78e010ec50 con 0x7f78e0102ac0 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e4e80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 0x7f78e019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e4e80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 0x7f78e019fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53064/0 (socket says 172.21.15.7:53064) 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.758+0000 7f78e4e80640 1 -- 172.21.15.7:0/3087263199 learned_addr learned my addr 172.21.15.7:0/3087263199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78e5681640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e0102ac0 0x7f78e019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78e4e80640 1 -- 172.21.15.7:0/3087263199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e0102ac0 msgr2=0x7f78e019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78e4e80640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e0102ac0 0x7f78e019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78e4e80640 1 -- 172.21.15.7:0/3087263199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e01999f0 con 0x7f78e01033e0 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78e4e80640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 0x7f78e019fd70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f78dc031460 tx=0x7f78dc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78d27fc640 1 -- 172.21.15.7:0/3087263199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f78dc03f070 con 0x7f78e01033e0 2024-09-21T04:52:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78d27fc640 1 -- 172.21.15.7:0/3087263199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f78dc0035f0 con 0x7f78e01033e0 2024-09-21T04:52:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e0199c80 con 0x7f78e01033e0 2024-09-21T04:52:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78d27fc640 1 -- 172.21.15.7:0/3087263199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f78dc035030 con 0x7f78e01033e0 2024-09-21T04:52:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.759+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e019a1b0 con 0x7f78e01033e0 2024-09-21T04:52:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.761+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e006aa90 con 0x7f78e01033e0 2024-09-21T04:52:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.761+0000 7f78d27fc640 1 -- 172.21.15.7:0/3087263199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f78dc042070 con 0x7f78e01033e0 2024-09-21T04:52:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.762+0000 7f78d27fc640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f78cc07af30 0x7f78cc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.762+0000 7f78e5681640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f78cc07af30 0x7f78cc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.762+0000 7f78d27fc640 1 -- 172.21.15.7:0/3087263199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f78dc0be960 con 0x7f78e01033e0 2024-09-21T04:52:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.763+0000 7f78e5681640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f78cc07af30 0x7f78cc07d3f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78e019ab30 tx=0x7f78d400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:08.766+0000 7f78d27fc640 1 -- 172.21.15.7:0/3087263199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78dc089f00 con 0x7f78e01033e0 2024-09-21T04:52:09.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.028+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f78e010b3a0 con 0x7f78e01033e0 2024-09-21T04:52:09.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.028+0000 7f78d27fc640 1 -- 172.21.15.7:0/3087263199 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f78dc08c420 con 0x7f78e01033e0 2024-09-21T04:52:09.031 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:52:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:52:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:52:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:52:09.031 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:52:09.032 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:52:09.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f78cc07af30 msgr2=0x7f78cc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f78cc07af30 0x7f78cc07d3f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78e019ab30 tx=0x7f78d400b390 comp rx=0 tx=0).stop 2024-09-21T04:52:09.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 msgr2=0x7f78e019fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 0x7f78e019fd70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f78dc031460 tx=0x7f78dc004530 comp rx=0 tx=0).stop 2024-09-21T04:52:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 shutdown_connections 2024-09-21T04:52:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f78cc07af30 0x7f78cc07d3f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e01033e0 0x7f78e019fd70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 --2- 172.21.15.7:0/3087263199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e0102ac0 0x7f78e019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.030+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 >> 172.21.15.7:0/3087263199 conn(0x7f78e00fc520 msgr2=0x7f78e0100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.031+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 shutdown_connections 2024-09-21T04:52:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.031+0000 7f78e78e5640 1 -- 172.21.15.7:0/3087263199 wait complete. 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.114+0000 7f801dd11640 1 -- 172.21.15.7:0/4112401299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 msgr2=0x7f8018101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.114+0000 7f801dd11640 1 --2- 172.21.15.7:0/4112401299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 0x7f8018101d60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8008009880 tx=0x7f800802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.115+0000 7f801dd11640 1 -- 172.21.15.7:0/4112401299 shutdown_connections 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.115+0000 7f801dd11640 1 --2- 172.21.15.7:0/4112401299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 0x7f8018101d60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.115+0000 7f801dd11640 1 --2- 172.21.15.7:0/4112401299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018107900 0x7f8018107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.115+0000 7f801dd11640 1 -- 172.21.15.7:0/4112401299 >> 172.21.15.7:0/4112401299 conn(0x7f80180fd600 msgr2=0x7f80180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.115+0000 7f801dd11640 1 -- 172.21.15.7:0/4112401299 shutdown_connections 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.115+0000 7f801dd11640 1 -- 172.21.15.7:0/4112401299 wait complete. 2024-09-21T04:52:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.115+0000 7f801dd11640 1 Processor -- start 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.115+0000 7f801dd11640 1 -- start start 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f801dd11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 0x7f80180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f801dd11640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018107900 0x7f8018074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f801dd11640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f801810ee30 con 0x7f8018101900 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f801dd11640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f801810ecb0 con 0x7f8018107900 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f80177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 0x7f80180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f80177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 0x7f80180744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53072/0 (socket says 172.21.15.7:53072) 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f80177fe640 1 -- 172.21.15.7:0/725863423 learned_addr learned my addr 172.21.15.7:0/725863423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f8016ffd640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018107900 0x7f8018074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f80177fe640 1 -- 172.21.15.7:0/725863423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018107900 msgr2=0x7f8018074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f80177fe640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018107900 0x7f8018074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f80177fe640 1 -- 172.21.15.7:0/725863423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8018074f70 con 0x7f8018101900 2024-09-21T04:52:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.116+0000 7f80177fe640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 0x7f80180744f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f800000b420 tx=0x7f800000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.117+0000 7f8014ff9640 1 -- 172.21.15.7:0/725863423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8000011630 con 0x7f8018101900 2024-09-21T04:52:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.117+0000 7f8014ff9640 1 -- 172.21.15.7:0/725863423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8000016070 con 0x7f8018101900 2024-09-21T04:52:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.117+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8018075260 con 0x7f8018101900 2024-09-21T04:52:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.117+0000 7f8014ff9640 1 -- 172.21.15.7:0/725863423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8000010d40 con 0x7f8018101900 2024-09-21T04:52:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.117+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181a3ec0 con 0x7f8018101900 2024-09-21T04:52:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.118+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8018103040 con 0x7f8018101900 2024-09-21T04:52:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.118+0000 7f8014ff9640 1 -- 172.21.15.7:0/725863423 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f800001b460 con 0x7f8018101900 2024-09-21T04:52:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.119+0000 7f8014ff9640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f800407af30 0x7f800407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.119+0000 7f8014ff9640 1 -- 172.21.15.7:0/725863423 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f800009d7a0 con 0x7f8018101900 2024-09-21T04:52:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.119+0000 7f8016ffd640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f800407af30 0x7f800407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.120+0000 7f8016ffd640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f800407af30 0x7f800407d3f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8018075f80 tx=0x7f8008031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:09.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.124+0000 7f8014ff9640 1 -- 172.21.15.7:0/725863423 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8000021250 con 0x7f8018101900 2024-09-21T04:52:09.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.348+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8018109e90 con 0x7f8018101900 2024-09-21T04:52:09.352 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:09 smithi007 ceph-mon[108238]: pgmap v521: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:09.352 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:09 smithi007 ceph-mon[108238]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:09.352 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:09 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3087263199' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:52:09.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.349+0000 7f8014ff9640 1 -- 172.21.15.7:0/725863423 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f8000068d40 con 0x7f8018101900 2024-09-21T04:52:09.352 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:52:09.352 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:52:09.353 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:52:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:52:09.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:52:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:52:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.352+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f800407af30 msgr2=0x7f800407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.352+0000 7f801dd11640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f800407af30 0x7f800407d3f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8018075f80 tx=0x7f8008031400 comp rx=0 tx=0).stop 2024-09-21T04:52:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.352+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 msgr2=0x7f80180744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.352+0000 7f801dd11640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 0x7f80180744f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f800000b420 tx=0x7f800000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.353+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 shutdown_connections 2024-09-21T04:52:09.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.353+0000 7f801dd11640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f800407af30 0x7f800407d3f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.353+0000 7f801dd11640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8018107900 0x7f8018074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.353+0000 7f801dd11640 1 --2- 172.21.15.7:0/725863423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8018101900 0x7f80180744f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.353+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 >> 172.21.15.7:0/725863423 conn(0x7f80180fd600 msgr2=0x7f80180fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:09.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.353+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 shutdown_connections 2024-09-21T04:52:09.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.353+0000 7f801dd11640 1 -- 172.21.15.7:0/725863423 wait complete. 2024-09-21T04:52:09.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.438+0000 7f44636a2640 1 -- 172.21.15.7:0/2366534102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 msgr2=0x7f445c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.438+0000 7f44636a2640 1 --2- 172.21.15.7:0/2366534102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 0x7f445c104c50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f444c00a990 tx=0x7f444c02b7d0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.438+0000 7f44636a2640 1 -- 172.21.15.7:0/2366534102 shutdown_connections 2024-09-21T04:52:09.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.438+0000 7f44636a2640 1 --2- 172.21.15.7:0/2366534102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 0x7f445c104c50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.438+0000 7f44636a2640 1 --2- 172.21.15.7:0/2366534102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c100250 0x7f445c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.438+0000 7f44636a2640 1 -- 172.21.15.7:0/2366534102 >> 172.21.15.7:0/2366534102 conn(0x7f445c0fbc90 msgr2=0x7f445c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.438+0000 7f44636a2640 1 -- 172.21.15.7:0/2366534102 shutdown_connections 2024-09-21T04:52:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.438+0000 7f44636a2640 1 -- 172.21.15.7:0/2366534102 wait complete. 2024-09-21T04:52:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f44636a2640 1 Processor -- start 2024-09-21T04:52:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f44636a2640 1 -- start start 2024-09-21T04:52:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f44636a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c100250 0x7f445c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f44636a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 0x7f445c102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f44636a2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f445c10cbd0 con 0x7f445c100c00 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f44636a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f445c10ca50 con 0x7f445c100250 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f4460c3d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 0x7f445c102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f4460c3d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 0x7f445c102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53090/0 (socket says 172.21.15.7:53090) 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f4460c3d640 1 -- 172.21.15.7:0/249178947 learned_addr learned my addr 172.21.15.7:0/249178947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.439+0000 7f446143e640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c100250 0x7f445c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f4460c3d640 1 -- 172.21.15.7:0/249178947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c100250 msgr2=0x7f445c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f4460c3d640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c100250 0x7f445c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f4460c3d640 1 -- 172.21.15.7:0/249178947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c102770 con 0x7f445c100c00 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f4460c3d640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 0x7f445c102230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f444c006010 tx=0x7f444c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f44527fc640 1 -- 172.21.15.7:0/249178947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f444c008e90 con 0x7f445c100c00 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f44527fc640 1 -- 172.21.15.7:0/249178947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f444c045070 con 0x7f445c100c00 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445c102a00 con 0x7f445c100c00 2024-09-21T04:52:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f44527fc640 1 -- 172.21.15.7:0/249178947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f444c0035f0 con 0x7f445c100c00 2024-09-21T04:52:09.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.440+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445c1a3d90 con 0x7f445c100c00 2024-09-21T04:52:09.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:09 smithi053 ceph-mon[80176]: pgmap v521: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:09.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:09 smithi053 ceph-mon[80176]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:09.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:09 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3087263199' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:52:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.441+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445c101730 con 0x7f445c100c00 2024-09-21T04:52:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.442+0000 7f44527fc640 1 -- 172.21.15.7:0/249178947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f444c032070 con 0x7f445c100c00 2024-09-21T04:52:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.443+0000 7f44527fc640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f444807af80 0x7f444807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.443+0000 7f446143e640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f444807af80 0x7f444807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.443+0000 7f44527fc640 1 -- 172.21.15.7:0/249178947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f444c008030 con 0x7f445c100c00 2024-09-21T04:52:09.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.443+0000 7f446143e640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f444807af80 0x7f444807d440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4458005fd0 tx=0x7f445800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:09.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.447+0000 7f44527fc640 1 -- 172.21.15.7:0/249178947 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f444c08a3f0 con 0x7f445c100c00 2024-09-21T04:52:09.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.651+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f445c062630 con 0x7f444807af80 2024-09-21T04:52:09.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.653+0000 7f44527fc640 1 -- 172.21.15.7:0/249178947 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f445c062630 con 0x7f444807af80 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:52:09.656 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:52:09.657 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:52:09.657 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:52:09.657 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:52:09.657 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:52:09.657 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:52:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f444807af80 msgr2=0x7f444807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f444807af80 0x7f444807d440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4458005fd0 tx=0x7f445800b390 comp rx=0 tx=0).stop 2024-09-21T04:52:09.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 msgr2=0x7f445c102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 0x7f445c102230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f444c006010 tx=0x7f444c004270 comp rx=0 tx=0).stop 2024-09-21T04:52:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 shutdown_connections 2024-09-21T04:52:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f444807af80 0x7f444807d440 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c100c00 0x7f445c102230 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 --2- 172.21.15.7:0/249178947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c100250 0x7f445c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 >> 172.21.15.7:0/249178947 conn(0x7f445c0fbc90 msgr2=0x7f445c0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 shutdown_connections 2024-09-21T04:52:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.655+0000 7f44636a2640 1 -- 172.21.15.7:0/249178947 wait complete. 2024-09-21T04:52:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.739+0000 7fca364b9640 1 -- 172.21.15.7:0/2633535466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 msgr2=0x7fca30106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.739+0000 7fca364b9640 1 --2- 172.21.15.7:0/2633535466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 0x7fca30106d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca20009880 tx=0x7fca2002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.739+0000 7fca364b9640 1 -- 172.21.15.7:0/2633535466 shutdown_connections 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.739+0000 7fca364b9640 1 --2- 172.21.15.7:0/2633535466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca300ffdc0 0x7fca30100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.739+0000 7fca364b9640 1 --2- 172.21.15.7:0/2633535466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 0x7fca30106d80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.739+0000 7fca364b9640 1 -- 172.21.15.7:0/2633535466 >> 172.21.15.7:0/2633535466 conn(0x7fca300fbc90 msgr2=0x7fca300fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.739+0000 7fca364b9640 1 -- 172.21.15.7:0/2633535466 shutdown_connections 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.739+0000 7fca364b9640 1 -- 172.21.15.7:0/2633535466 wait complete. 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca364b9640 1 Processor -- start 2024-09-21T04:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca364b9640 1 -- start start 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca364b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca300ffdc0 0x7fca3019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca364b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 0x7fca3019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca364b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca3010ef40 con 0x7fca301069a0 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca364b9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca3010edc0 con 0x7fca300ffdc0 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 0x7fca3019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 0x7fca3019fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53108/0 (socket says 172.21.15.7:53108) 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.740+0000 7fca2f7fe640 1 -- 172.21.15.7:0/1244549746 learned_addr learned my addr 172.21.15.7:0/1244549746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2ffff640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca300ffdc0 0x7fca3019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2f7fe640 1 -- 172.21.15.7:0/1244549746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca300ffdc0 msgr2=0x7fca3019f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2f7fe640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca300ffdc0 0x7fca3019f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2f7fe640 1 -- 172.21.15.7:0/1244549746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca30199b00 con 0x7fca301069a0 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2ffff640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca300ffdc0 0x7fca3019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2f7fe640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 0x7fca3019fe80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca2400d6f0 tx=0x7fca2400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2d7fa640 1 -- 172.21.15.7:0/1244549746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fca24017070 con 0x7fca301069a0 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca30199df0 con 0x7fca301069a0 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2d7fa640 1 -- 172.21.15.7:0/1244549746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca2400de00 con 0x7fca301069a0 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca2d7fa640 1 -- 172.21.15.7:0/1244549746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fca24004bb0 con 0x7fca301069a0 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.741+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3019a380 con 0x7fca301069a0 2024-09-21T04:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.742+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca301030e0 con 0x7fca301069a0 2024-09-21T04:52:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.743+0000 7fca2d7fa640 1 -- 172.21.15.7:0/1244549746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fca24004e60 con 0x7fca301069a0 2024-09-21T04:52:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.743+0000 7fca2d7fa640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af30 0x7fca1807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.744+0000 7fca2ffff640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af30 0x7fca1807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.744+0000 7fca2d7fa640 1 -- 172.21.15.7:0/1244549746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fca2409dcb0 con 0x7fca301069a0 2024-09-21T04:52:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.744+0000 7fca2ffff640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af30 0x7fca1807d3f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fca20000c00 tx=0x7fca20031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:09.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:09.748+0000 7fca2d7fa640 1 -- 172.21.15.7:0/1244549746 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fca24069250 con 0x7fca301069a0 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.009+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fca3019aff0 con 0x7fca301069a0 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.010+0000 7fca2d7fa640 1 -- 172.21.15.7:0/1244549746 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fca2406e100 con 0x7fca301069a0 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:52:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:52:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af30 msgr2=0x7fca1807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af30 0x7fca1807d3f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fca20000c00 tx=0x7fca20031400 comp rx=0 tx=0).stop 2024-09-21T04:52:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 msgr2=0x7fca3019fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 0x7fca3019fe80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca2400d6f0 tx=0x7fca2400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:52:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 shutdown_connections 2024-09-21T04:52:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af30 0x7fca1807d3f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca301069a0 0x7fca3019fe80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:10.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 --2- 172.21.15.7:0/1244549746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca300ffdc0 0x7fca3019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:10.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 >> 172.21.15.7:0/1244549746 conn(0x7fca300fbc90 msgr2=0x7fca30100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:10.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 shutdown_connections 2024-09-21T04:52:10.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:10.012+0000 7fca364b9640 1 -- 172.21.15.7:0/1244549746 wait complete. 2024-09-21T04:52:10.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:10 smithi053 ceph-mon[80176]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:10 smithi053 ceph-mon[80176]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:10 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/725863423' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:52:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:10 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1244549746' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:52:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:10 smithi007 ceph-mon[108238]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:10 smithi007 ceph-mon[108238]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:10 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/725863423' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:52:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:10 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1244549746' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:52:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:11 smithi007 ceph-mon[108238]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:11 smithi007 ceph-mon[108238]: pgmap v522: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:11 smithi053 ceph-mon[80176]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:11 smithi053 ceph-mon[80176]: pgmap v522: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:13.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:13 smithi053 ceph-mon[80176]: pgmap v523: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:13 smithi007 ceph-mon[108238]: pgmap v523: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:52:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:52:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:15 smithi053 ceph-mon[80176]: pgmap v524: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:15 smithi007 ceph-mon[108238]: pgmap v524: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:17 smithi007 ceph-mon[108238]: pgmap v525: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:17 smithi053 ceph-mon[80176]: pgmap v525: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:19.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:19 smithi053 ceph-mon[80176]: pgmap v526: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:19 smithi007 ceph-mon[108238]: pgmap v526: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:21 smithi053 ceph-mon[80176]: pgmap v527: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:21 smithi007 ceph-mon[108238]: pgmap v527: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:23 smithi007 ceph-mon[108238]: pgmap v528: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:23.388 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:23 smithi053 ceph-mon[80176]: pgmap v528: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:24 smithi053 ceph-mon[80176]: pgmap v529: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:25.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:24 smithi007 ceph-mon[108238]: pgmap v529: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:27.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:27 smithi053 ceph-mon[80176]: pgmap v530: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:27 smithi007 ceph-mon[108238]: pgmap v530: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:29 smithi007 ceph-mon[108238]: pgmap v531: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:52:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:29 smithi053 ceph-mon[80176]: pgmap v531: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:52:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:31 smithi053 ceph-mon[80176]: pgmap v532: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:31 smithi007 ceph-mon[108238]: pgmap v532: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:33.431 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:33 smithi053 ceph-mon[80176]: pgmap v533: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:33 smithi007 ceph-mon[108238]: pgmap v533: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:35 smithi007 ceph-mon[108238]: pgmap v534: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:35.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:35 smithi053 ceph-mon[80176]: pgmap v534: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:37 smithi007 ceph-mon[108238]: pgmap v535: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:37 smithi053 ceph-mon[80176]: pgmap v535: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:39 smithi053 ceph-mon[80176]: pgmap v536: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:39 smithi007 ceph-mon[108238]: pgmap v536: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.110+0000 7f37ef75e640 1 -- 172.21.15.7:0/2860299751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8107900 msgr2=0x7f37e8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.110+0000 7f37ef75e640 1 --2- 172.21.15.7:0/2860299751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8107900 0x7f37e8107ce0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009c20 tx=0x7f37e402a970 comp rx=0 tx=0).stop 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.110+0000 7f37ef75e640 1 -- 172.21.15.7:0/2860299751 shutdown_connections 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.110+0000 7f37ef75e640 1 --2- 172.21.15.7:0/2860299751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e8101900 0x7f37e8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.110+0000 7f37ef75e640 1 --2- 172.21.15.7:0/2860299751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8107900 0x7f37e8107ce0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.110+0000 7f37ef75e640 1 -- 172.21.15.7:0/2860299751 >> 172.21.15.7:0/2860299751 conn(0x7f37e80fd600 msgr2=0x7f37e80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.110+0000 7f37ef75e640 1 -- 172.21.15.7:0/2860299751 shutdown_connections 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.110+0000 7f37ef75e640 1 -- 172.21.15.7:0/2860299751 wait complete. 2024-09-21T04:52:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.111+0000 7f37ef75e640 1 Processor -- start 2024-09-21T04:52:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.111+0000 7f37ef75e640 1 -- start start 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.111+0000 7f37ef75e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8101900 0x7f37e819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.111+0000 7f37ef75e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e8107900 0x7f37e819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.111+0000 7f37ef75e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37e810edd0 con 0x7f37e8101900 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.111+0000 7f37ef75e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f37e810ec50 con 0x7f37e8107900 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.111+0000 7f37ed4fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8101900 0x7f37e819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37ed4fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8101900 0x7f37e819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49890/0 (socket says 172.21.15.7:49890) 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37ed4fa640 1 -- 172.21.15.7:0/752164323 learned_addr learned my addr 172.21.15.7:0/752164323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37eccf9640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e8107900 0x7f37e819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37ed4fa640 1 -- 172.21.15.7:0/752164323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e8107900 msgr2=0x7f37e819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37ed4fa640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e8107900 0x7f37e819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37ed4fa640 1 -- 172.21.15.7:0/752164323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e81977c0 con 0x7f37e8101900 2024-09-21T04:52:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37ed4fa640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8101900 0x7f37e819d640 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37e402ae90 tx=0x7f37e402f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37de7fc640 1 -- 172.21.15.7:0/752164323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f37e4047070 con 0x7f37e8101900 2024-09-21T04:52:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37de7fc640 1 -- 172.21.15.7:0/752164323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f37e402f7e0 con 0x7f37e8101900 2024-09-21T04:52:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e8197a50 con 0x7f37e8101900 2024-09-21T04:52:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37de7fc640 1 -- 172.21.15.7:0/752164323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f37e4034070 con 0x7f37e8101900 2024-09-21T04:52:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.112+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e8197f80 con 0x7f37e8101900 2024-09-21T04:52:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.113+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e8103040 con 0x7f37e8101900 2024-09-21T04:52:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.114+0000 7f37de7fc640 1 -- 172.21.15.7:0/752164323 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f37e404a070 con 0x7f37e8101900 2024-09-21T04:52:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.115+0000 7f37de7fc640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f37d407af80 0x7f37d407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.115+0000 7f37de7fc640 1 -- 172.21.15.7:0/752164323 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f37e40bec60 con 0x7f37e8101900 2024-09-21T04:52:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.115+0000 7f37eccf9640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f37d407af80 0x7f37d407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.115+0000 7f37eccf9640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f37d407af80 0x7f37d407d440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f37e80ffbf0 tx=0x7f37d800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.119+0000 7f37de7fc640 1 -- 172.21.15.7:0/752164323 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37e403f070 con 0x7f37e8101900 2024-09-21T04:52:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.326+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f37e81982f0 con 0x7f37d407af80 2024-09-21T04:52:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.327+0000 7f37de7fc640 1 -- 172.21.15.7:0/752164323 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f37e81982f0 con 0x7f37d407af80 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f37d407af80 msgr2=0x7f37d407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f37d407af80 0x7f37d407d440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f37e80ffbf0 tx=0x7f37d800b390 comp rx=0 tx=0).stop 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8101900 msgr2=0x7f37e819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8101900 0x7f37e819d640 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37e402ae90 tx=0x7f37e402f5f0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 shutdown_connections 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f37d407af80 0x7f37d407d440 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e8107900 0x7f37e819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 --2- 172.21.15.7:0/752164323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37e8101900 0x7f37e819d640 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 >> 172.21.15.7:0/752164323 conn(0x7f37e80fd600 msgr2=0x7f37e80fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 shutdown_connections 2024-09-21T04:52:40.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.330+0000 7f37ef75e640 1 -- 172.21.15.7:0/752164323 wait complete. 2024-09-21T04:52:40.342 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:52:40.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.415+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2055308994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f01033d0 msgr2=0x7fb7f010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.415+0000 7fb7f53ed640 1 --2- 172.21.15.7:0/2055308994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f01033d0 0x7fb7f010b8c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009880 tx=0x7fb7e402a530 comp rx=0 tx=0).stop 2024-09-21T04:52:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.416+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2055308994 shutdown_connections 2024-09-21T04:52:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.416+0000 7fb7f53ed640 1 --2- 172.21.15.7:0/2055308994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f01033d0 0x7fb7f010b8c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.416+0000 7fb7f53ed640 1 --2- 172.21.15.7:0/2055308994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f0102ab0 0x7fb7f0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.416+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2055308994 >> 172.21.15.7:0/2055308994 conn(0x7fb7f00fc550 msgr2=0x7fb7f00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.416+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2055308994 shutdown_connections 2024-09-21T04:52:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.416+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2055308994 wait complete. 2024-09-21T04:52:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.416+0000 7fb7f53ed640 1 Processor -- start 2024-09-21T04:52:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7f53ed640 1 -- start start 2024-09-21T04:52:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7f53ed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f0102ab0 0x7fb7f019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7f53ed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f01033d0 0x7fb7f019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7f53ed640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010ee40 con 0x7fb7f0102ab0 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7f53ed640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010ecc0 con 0x7fb7f01033d0 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7eeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f0102ab0 0x7fb7f019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7eeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f0102ab0 0x7fb7f019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49898/0 (socket says 172.21.15.7:49898) 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7eeffd640 1 -- 172.21.15.7:0/2765185375 learned_addr learned my addr 172.21.15.7:0/2765185375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7eeffd640 1 -- 172.21.15.7:0/2765185375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f01033d0 msgr2=0x7fb7f019dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7ee7fc640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f01033d0 0x7fb7f019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7eeffd640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f01033d0 0x7fb7f019dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7eeffd640 1 -- 172.21.15.7:0/2765185375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f0197830 con 0x7fb7f0102ab0 2024-09-21T04:52:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.417+0000 7fb7eeffd640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f0102ab0 0x7fb7f019d6b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ba50 tx=0x7fb7e000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:40.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.418+0000 7fb7d7fff640 1 -- 172.21.15.7:0/2765185375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb7e0019070 con 0x7fb7f0102ab0 2024-09-21T04:52:40.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.418+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f0197b20 con 0x7fb7f0102ab0 2024-09-21T04:52:40.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.418+0000 7fb7d7fff640 1 -- 172.21.15.7:0/2765185375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb7e000ce20 con 0x7fb7f0102ab0 2024-09-21T04:52:40.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.418+0000 7fb7d7fff640 1 -- 172.21.15.7:0/2765185375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb7e00075a0 con 0x7fb7f0102ab0 2024-09-21T04:52:40.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.418+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f01980b0 con 0x7fb7f0102ab0 2024-09-21T04:52:40.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.419+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f006a580 con 0x7fb7f0102ab0 2024-09-21T04:52:40.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.419+0000 7fb7d7fff640 1 -- 172.21.15.7:0/2765185375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb7e001c070 con 0x7fb7f0102ab0 2024-09-21T04:52:40.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.420+0000 7fb7d7fff640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb7d807af30 0x7fb7d807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.420+0000 7fb7d7fff640 1 -- 172.21.15.7:0/2765185375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb7e009d950 con 0x7fb7f0102ab0 2024-09-21T04:52:40.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.420+0000 7fb7ee7fc640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb7d807af30 0x7fb7d807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.420+0000 7fb7ee7fc640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb7d807af30 0x7fb7d807d3f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4031460 tx=0x7fb7e4009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:40.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.424+0000 7fb7d7fff640 1 -- 172.21.15.7:0/2765185375 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb7e0068ef0 con 0x7fb7f0102ab0 2024-09-21T04:52:40.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.628+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb7f01983c0 con 0x7fb7d807af30 2024-09-21T04:52:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.629+0000 7fb7d7fff640 1 -- 172.21.15.7:0/2765185375 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fb7f01983c0 con 0x7fb7d807af30 2024-09-21T04:52:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb7d807af30 msgr2=0x7fb7d807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb7d807af30 0x7fb7d807d3f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4031460 tx=0x7fb7e4009560 comp rx=0 tx=0).stop 2024-09-21T04:52:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f0102ab0 msgr2=0x7fb7f019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f0102ab0 0x7fb7f019d6b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ba50 tx=0x7fb7e000bf10 comp rx=0 tx=0).stop 2024-09-21T04:52:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 shutdown_connections 2024-09-21T04:52:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb7d807af30 0x7fb7d807d3f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7f01033d0 0x7fb7f019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 --2- 172.21.15.7:0/2765185375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f0102ab0 0x7fb7f019d6b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.631+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 >> 172.21.15.7:0/2765185375 conn(0x7fb7f00fc550 msgr2=0x7fb7f0108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.632+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 shutdown_connections 2024-09-21T04:52:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.632+0000 7fb7f53ed640 1 -- 172.21.15.7:0/2765185375 wait complete. 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.717+0000 7f70474be640 1 -- 172.21.15.7:0/3437395807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074620 msgr2=0x7f7040074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.717+0000 7f70474be640 1 --2- 172.21.15.7:0/3437395807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074620 0x7f7040074a00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f703c009880 tx=0x7f703c02a530 comp rx=0 tx=0).stop 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.717+0000 7f70474be640 1 -- 172.21.15.7:0/3437395807 shutdown_connections 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.717+0000 7f70474be640 1 --2- 172.21.15.7:0/3437395807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040074f40 0x7f704010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.717+0000 7f70474be640 1 --2- 172.21.15.7:0/3437395807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074620 0x7f7040074a00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.717+0000 7f70474be640 1 -- 172.21.15.7:0/3437395807 >> 172.21.15.7:0/3437395807 conn(0x7f70400fd600 msgr2=0x7f70400ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.717+0000 7f70474be640 1 -- 172.21.15.7:0/3437395807 shutdown_connections 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.717+0000 7f70474be640 1 -- 172.21.15.7:0/3437395807 wait complete. 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f70474be640 1 Processor -- start 2024-09-21T04:52:40.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f70474be640 1 -- start start 2024-09-21T04:52:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f70474be640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040074620 0x7f704019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f70474be640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074f40 0x7f704019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f70474be640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7040110ff0 con 0x7f7040074f40 2024-09-21T04:52:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f70474be640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7040110e70 con 0x7f7040074620 2024-09-21T04:52:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f7044a59640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074f40 0x7f704019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f7044a59640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074f40 0x7f704019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49914/0 (socket says 172.21.15.7:49914) 2024-09-21T04:52:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.718+0000 7f7044a59640 1 -- 172.21.15.7:0/2609726657 learned_addr learned my addr 172.21.15.7:0/2609726657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:40.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f704525a640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040074620 0x7f704019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f7044a59640 1 -- 172.21.15.7:0/2609726657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040074620 msgr2=0x7f704019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f7044a59640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040074620 0x7f704019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f7044a59640 1 -- 172.21.15.7:0/2609726657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7040199a00 con 0x7f7040074f40 2024-09-21T04:52:40.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f7044a59640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074f40 0x7f704019fd80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f703000c8f0 tx=0x7f703000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:40.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f70367fc640 1 -- 172.21.15.7:0/2609726657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f70300079b0 con 0x7f7040074f40 2024-09-21T04:52:40.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f70367fc640 1 -- 172.21.15.7:0/2609726657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f70300054e0 con 0x7f7040074f40 2024-09-21T04:52:40.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7040199cf0 con 0x7f7040074f40 2024-09-21T04:52:40.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f70367fc640 1 -- 172.21.15.7:0/2609726657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7030004dd0 con 0x7f7040074f40 2024-09-21T04:52:40.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.719+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704019a280 con 0x7f7040074f40 2024-09-21T04:52:40.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.720+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7040075cd0 con 0x7f7040074f40 2024-09-21T04:52:40.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.721+0000 7f70367fc640 1 -- 172.21.15.7:0/2609726657 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7030005020 con 0x7f7040074f40 2024-09-21T04:52:40.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.722+0000 7f70367fc640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f702c07aed0 0x7f702c07d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.722+0000 7f704525a640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f702c07aed0 0x7f702c07d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.722+0000 7f70367fc640 1 -- 172.21.15.7:0/2609726657 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f703009e0e0 con 0x7f7040074f40 2024-09-21T04:52:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.722+0000 7f704525a640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f702c07aed0 0x7f702c07d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f703c000c00 tx=0x7f703c004120 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:40.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.726+0000 7f70367fc640 1 -- 172.21.15.7:0/2609726657 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7030014030 con 0x7f7040074f40 2024-09-21T04:52:40.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.921+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f704019af30 con 0x7f702c07aed0 2024-09-21T04:52:40.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.931+0000 7f70367fc640 1 -- 172.21.15.7:0/2609726657 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f704019af30 con 0x7f702c07aed0 2024-09-21T04:52:40.935 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:52:40.936 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (14m) 8m ago 25m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:52:40.936 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (17m) 8m ago 25m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:52:40.936 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (17m) 8m ago 24m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:52:40.936 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (12m) 8m ago 25m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:52:40.936 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (12m) 8m ago 24m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:52:40.936 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (14m) 8m ago 25m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:52:40.937 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (8m) 8m ago 20m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:52:40.937 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (9m) 8m ago 20m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:52:40.937 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (20m) 8m ago 20m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:52:40.937 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (8m) 8m ago 20m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:52:40.937 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (16m) 8m ago 27m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:52:40.937 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (15m) 8m ago 24m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:52:40.938 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (13m) 8m ago 27m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:52:40.938 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (13m) 8m ago 24m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:52:40.938 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (15m) 8m ago 25m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:52:40.938 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (15m) 8m ago 24m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:52:40.938 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (12m) 8m ago 23m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:52:40.938 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (11m) 8m ago 23m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:52:40.939 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (11m) 8m ago 22m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:52:40.939 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (10m) 8m ago 22m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:52:40.939 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (9m) 8m ago 21m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:52:40.939 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (9m) 8m ago 21m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:52:40.939 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (15m) 8m ago 25m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:52:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f702c07aed0 msgr2=0x7f702c07d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f702c07aed0 0x7f702c07d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f703c000c00 tx=0x7f703c004120 comp rx=0 tx=0).stop 2024-09-21T04:52:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074f40 msgr2=0x7f704019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074f40 0x7f704019fd80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f703000c8f0 tx=0x7f703000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 shutdown_connections 2024-09-21T04:52:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f702c07aed0 0x7f702c07d390 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7040074f40 0x7f704019fd80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 --2- 172.21.15.7:0/2609726657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040074620 0x7f704019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.934+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 >> 172.21.15.7:0/2609726657 conn(0x7f70400fd600 msgr2=0x7f704010b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.935+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 shutdown_connections 2024-09-21T04:52:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:40.935+0000 7f70474be640 1 -- 172.21.15.7:0/2609726657 wait complete. 2024-09-21T04:52:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.018+0000 7f7740a19640 1 -- 172.21.15.7:0/2905581751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 msgr2=0x7f773c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.018+0000 7f7740a19640 1 --2- 172.21.15.7:0/2905581751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 0x7f773c10b8c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7730009880 tx=0x7f773002a530 comp rx=0 tx=0).stop 2024-09-21T04:52:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.018+0000 7f7740a19640 1 -- 172.21.15.7:0/2905581751 shutdown_connections 2024-09-21T04:52:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.018+0000 7f7740a19640 1 --2- 172.21.15.7:0/2905581751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 0x7f773c10b8c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.018+0000 7f7740a19640 1 --2- 172.21.15.7:0/2905581751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c102ab0 0x7f773c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.018+0000 7f7740a19640 1 -- 172.21.15.7:0/2905581751 >> 172.21.15.7:0/2905581751 conn(0x7f773c0fc550 msgr2=0x7f773c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.018+0000 7f7740a19640 1 -- 172.21.15.7:0/2905581751 shutdown_connections 2024-09-21T04:52:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.018+0000 7f7740a19640 1 -- 172.21.15.7:0/2905581751 wait complete. 2024-09-21T04:52:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.019+0000 7f7740a19640 1 Processor -- start 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.019+0000 7f7740a19640 1 -- start start 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.019+0000 7f7740a19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c102ab0 0x7f773c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.019+0000 7f7740a19640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 0x7f773c19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.019+0000 7f7740a19640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f773c10ede0 con 0x7f773c1033d0 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.019+0000 7f7740a19640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f773c10ec60 con 0x7f773c102ab0 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.019+0000 7f7739d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 0x7f773c19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f7739d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 0x7f773c19dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49932/0 (socket says 172.21.15.7:49932) 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f7739d9b640 1 -- 172.21.15.7:0/1229027626 learned_addr learned my addr 172.21.15.7:0/1229027626 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f773a59c640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c102ab0 0x7f773c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f7739d9b640 1 -- 172.21.15.7:0/1229027626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c102ab0 msgr2=0x7f773c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f7739d9b640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c102ab0 0x7f773c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f7739d9b640 1 -- 172.21.15.7:0/1229027626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c197810 con 0x7f773c1033d0 2024-09-21T04:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f7739d9b640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 0x7f773c19dbd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7730031460 tx=0x7f77300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f772b7fe640 1 -- 172.21.15.7:0/1229027626 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f773003f070 con 0x7f773c1033d0 2024-09-21T04:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f772b7fe640 1 -- 172.21.15.7:0/1229027626 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7730004ac0 con 0x7f773c1033d0 2024-09-21T04:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c197aa0 con 0x7f773c1033d0 2024-09-21T04:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f772b7fe640 1 -- 172.21.15.7:0/1229027626 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7730035030 con 0x7f773c1033d0 2024-09-21T04:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.020+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c197fd0 con 0x7f773c1033d0 2024-09-21T04:52:41.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.022+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773c06a580 con 0x7f773c1033d0 2024-09-21T04:52:41.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.023+0000 7f772b7fe640 1 -- 172.21.15.7:0/1229027626 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7730042070 con 0x7f773c1033d0 2024-09-21T04:52:41.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.023+0000 7f772b7fe640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f772407af80 0x7f772407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.023+0000 7f772b7fe640 1 -- 172.21.15.7:0/1229027626 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f77300be970 con 0x7f773c1033d0 2024-09-21T04:52:41.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.023+0000 7f773a59c640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f772407af80 0x7f772407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.024+0000 7f773a59c640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f772407af80 0x7f772407d440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f772c0058c0 tx=0x7f772c00a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.028+0000 7f772b7fe640 1 -- 172.21.15.7:0/1229027626 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7730089f30 con 0x7f773c1033d0 2024-09-21T04:52:41.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.291+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f773c105f50 con 0x7f773c1033d0 2024-09-21T04:52:41.295 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:41 smithi007 ceph-mon[108238]: pgmap v537: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:41.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.291+0000 7f772b7fe640 1 -- 172.21.15.7:0/1229027626 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f773008c450 con 0x7f773c1033d0 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:52:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:52:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:52:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:52:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:52:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:52:41.297 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:52:41.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.293+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f772407af80 msgr2=0x7f772407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.293+0000 7f7740a19640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f772407af80 0x7f772407d440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f772c0058c0 tx=0x7f772c00a250 comp rx=0 tx=0).stop 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.293+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 msgr2=0x7f773c19dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.293+0000 7f7740a19640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 0x7f773c19dbd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7730031460 tx=0x7f77300046c0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.294+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 shutdown_connections 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.294+0000 7f7740a19640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f772407af80 0x7f772407d440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.294+0000 7f7740a19640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f773c1033d0 0x7f773c19dbd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.294+0000 7f7740a19640 1 --2- 172.21.15.7:0/1229027626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c102ab0 0x7f773c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.294+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 >> 172.21.15.7:0/1229027626 conn(0x7f773c0fc550 msgr2=0x7f773c108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.294+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 shutdown_connections 2024-09-21T04:52:41.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.294+0000 7f7740a19640 1 -- 172.21.15.7:0/1229027626 wait complete. 2024-09-21T04:52:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.378+0000 7f1240a84640 1 -- 172.21.15.7:0/915143887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 msgr2=0x7f123c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.378+0000 7f1240a84640 1 --2- 172.21.15.7:0/915143887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 0x7f123c100c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1224009880 tx=0x7f122402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.378+0000 7f1240a84640 1 -- 172.21.15.7:0/915143887 shutdown_connections 2024-09-21T04:52:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.378+0000 7f1240a84640 1 --2- 172.21.15.7:0/915143887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 0x7f123c100c50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.378+0000 7f1240a84640 1 --2- 172.21.15.7:0/915143887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f123c1067f0 0x7f123c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.378+0000 7f1240a84640 1 -- 172.21.15.7:0/915143887 >> 172.21.15.7:0/915143887 conn(0x7f123c0fc550 msgr2=0x7f123c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.378+0000 7f1240a84640 1 -- 172.21.15.7:0/915143887 shutdown_connections 2024-09-21T04:52:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.378+0000 7f1240a84640 1 -- 172.21.15.7:0/915143887 wait complete. 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f1240a84640 1 Processor -- start 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f1240a84640 1 -- start start 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f1240a84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 0x7f123c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f1240a84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f123c1067f0 0x7f123c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f1240a84640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10dd20 con 0x7f123c1007f0 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f1240a84640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10dba0 con 0x7f123c1067f0 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f123a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 0x7f123c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f123a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 0x7f123c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49950/0 (socket says 172.21.15.7:49950) 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f123a59c640 1 -- 172.21.15.7:0/3797998549 learned_addr learned my addr 172.21.15.7:0/3797998549 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f123a59c640 1 -- 172.21.15.7:0/3797998549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f123c1067f0 msgr2=0x7f123c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f123a59c640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f123c1067f0 0x7f123c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.379+0000 7f123a59c640 1 -- 172.21.15.7:0/3797998549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c1994b0 con 0x7f123c1007f0 2024-09-21T04:52:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.380+0000 7f123a59c640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 0x7f123c198970 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b420 tx=0x7f122c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.380+0000 7f12337fe640 1 -- 172.21.15.7:0/3797998549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f122c011630 con 0x7f123c1007f0 2024-09-21T04:52:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.380+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123c1997a0 con 0x7f123c1007f0 2024-09-21T04:52:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.380+0000 7f12337fe640 1 -- 172.21.15.7:0/3797998549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f122c016070 con 0x7f123c1007f0 2024-09-21T04:52:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.380+0000 7f12337fe640 1 -- 172.21.15.7:0/3797998549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f122c010d40 con 0x7f123c1007f0 2024-09-21T04:52:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.380+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123c1a1da0 con 0x7f123c1007f0 2024-09-21T04:52:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.381+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123c101f30 con 0x7f123c1007f0 2024-09-21T04:52:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.382+0000 7f12337fe640 1 -- 172.21.15.7:0/3797998549 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f122c01b460 con 0x7f123c1007f0 2024-09-21T04:52:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.382+0000 7f12337fe640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f122807af30 0x7f122807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.382+0000 7f12337fe640 1 -- 172.21.15.7:0/3797998549 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f122c09d7a0 con 0x7f123c1007f0 2024-09-21T04:52:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.382+0000 7f1233fff640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f122807af30 0x7f122807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.383+0000 7f1233fff640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f122807af30 0x7f122807d3f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12240096c0 tx=0x7f1224000bc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:41.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.387+0000 7f12337fe640 1 -- 172.21.15.7:0/3797998549 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f123c1997a0 con 0x7f123c1007f0 2024-09-21T04:52:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:41 smithi053 ceph-mon[80176]: pgmap v537: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:41.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.612+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f123c100c50 con 0x7f123c1007f0 2024-09-21T04:52:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.612+0000 7f12337fe640 1 -- 172.21.15.7:0/3797998549 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f122c021250 con 0x7f123c1007f0 2024-09-21T04:52:41.616 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:52:41.616 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:52:41.616 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:52:41.617 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:52:41.618 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:52:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:52:41.620 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:52:41.620 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:52:41.620 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:52:41.620 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:52:41.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f122807af30 msgr2=0x7f122807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f122807af30 0x7f122807d3f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12240096c0 tx=0x7f1224000bc0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 msgr2=0x7f123c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 0x7f123c198970 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b420 tx=0x7f122c00b8e0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 shutdown_connections 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f122807af30 0x7f122807d3f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f123c1067f0 0x7f123c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 --2- 172.21.15.7:0/3797998549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c1007f0 0x7f123c198970 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 >> 172.21.15.7:0/3797998549 conn(0x7f123c0fc550 msgr2=0x7f123c0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 shutdown_connections 2024-09-21T04:52:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.616+0000 7f1240a84640 1 -- 172.21.15.7:0/3797998549 wait complete. 2024-09-21T04:52:41.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.703+0000 7f23342d8640 1 -- 172.21.15.7:0/699129568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c101900 msgr2=0x7f232c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.703+0000 7f23342d8640 1 --2- 172.21.15.7:0/699129568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c101900 0x7f232c101d60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2320009880 tx=0x7f232002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.703+0000 7f23342d8640 1 -- 172.21.15.7:0/699129568 shutdown_connections 2024-09-21T04:52:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.703+0000 7f23342d8640 1 --2- 172.21.15.7:0/699129568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c101900 0x7f232c101d60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.703+0000 7f23342d8640 1 --2- 172.21.15.7:0/699129568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f232c107900 0x7f232c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.703+0000 7f23342d8640 1 -- 172.21.15.7:0/699129568 >> 172.21.15.7:0/699129568 conn(0x7f232c0fd600 msgr2=0x7f232c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.703+0000 7f23342d8640 1 -- 172.21.15.7:0/699129568 shutdown_connections 2024-09-21T04:52:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.703+0000 7f23342d8640 1 -- 172.21.15.7:0/699129568 wait complete. 2024-09-21T04:52:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f23342d8640 1 Processor -- start 2024-09-21T04:52:41.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f23342d8640 1 -- start start 2024-09-21T04:52:41.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f23342d8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f232c101900 0x7f232c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f23342d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c107900 0x7f232c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f23342d8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10ee30 con 0x7f232c107900 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f23342d8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10ecb0 con 0x7f232c101900 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f2331873640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c107900 0x7f232c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f2331873640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c107900 0x7f232c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49968/0 (socket says 172.21.15.7:49968) 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f2331873640 1 -- 172.21.15.7:0/4141768335 learned_addr learned my addr 172.21.15.7:0/4141768335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f2332074640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f232c101900 0x7f232c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f2331873640 1 -- 172.21.15.7:0/4141768335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f232c101900 msgr2=0x7f232c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.704+0000 7f2331873640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f232c101900 0x7f232c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.705+0000 7f2331873640 1 -- 172.21.15.7:0/4141768335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c197840 con 0x7f232c107900 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.705+0000 7f2331873640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c107900 0x7f232c19dc00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2320031460 tx=0x7f23200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.705+0000 7f232b7fe640 1 -- 172.21.15.7:0/4141768335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f232003f070 con 0x7f232c107900 2024-09-21T04:52:41.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.705+0000 7f232b7fe640 1 -- 172.21.15.7:0/4141768335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2320004cf0 con 0x7f232c107900 2024-09-21T04:52:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.705+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c197ad0 con 0x7f232c107900 2024-09-21T04:52:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.705+0000 7f232b7fe640 1 -- 172.21.15.7:0/4141768335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2320035030 con 0x7f232c107900 2024-09-21T04:52:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.705+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c198000 con 0x7f232c107900 2024-09-21T04:52:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.706+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c103040 con 0x7f232c107900 2024-09-21T04:52:41.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.707+0000 7f232b7fe640 1 -- 172.21.15.7:0/4141768335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2320042070 con 0x7f232c107900 2024-09-21T04:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.707+0000 7f232b7fe640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f231407af30 0x7f231407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.707+0000 7f232b7fe640 1 -- 172.21.15.7:0/4141768335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f23200beb60 con 0x7f232c107900 2024-09-21T04:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.707+0000 7f2332074640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f231407af30 0x7f231407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:41.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.708+0000 7f2332074640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f231407af30 0x7f231407d3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f231c007dc0 tx=0x7f231c0072f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:41.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.712+0000 7f232b7fe640 1 -- 172.21.15.7:0/4141768335 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f232008a100 con 0x7f232c107900 2024-09-21T04:52:41.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.914+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f232c1982b0 con 0x7f231407af30 2024-09-21T04:52:41.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.915+0000 7f232b7fe640 1 -- 172.21.15.7:0/4141768335 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f232c1982b0 con 0x7f231407af30 2024-09-21T04:52:41.919 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:52:41.919 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:52:41.919 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:52:41.919 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:52:41.920 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:52:41.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.917+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f231407af30 msgr2=0x7f231407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.917+0000 7f23342d8640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f231407af30 0x7f231407d3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f231c007dc0 tx=0x7f231c0072f0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.917+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c107900 msgr2=0x7f232c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.917+0000 7f23342d8640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c107900 0x7f232c19dc00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2320031460 tx=0x7f23200048f0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.918+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 shutdown_connections 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.918+0000 7f23342d8640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f231407af30 0x7f231407d3f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.918+0000 7f23342d8640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f232c107900 0x7f232c19dc00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.918+0000 7f23342d8640 1 --2- 172.21.15.7:0/4141768335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f232c101900 0x7f232c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.918+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 >> 172.21.15.7:0/4141768335 conn(0x7f232c0fd600 msgr2=0x7f232c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.918+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 shutdown_connections 2024-09-21T04:52:41.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:41.918+0000 7f23342d8640 1 -- 172.21.15.7:0/4141768335 wait complete. 2024-09-21T04:52:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.003+0000 7f47a441f640 1 -- 172.21.15.7:0/1082848880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c107910 msgr2=0x7f479c107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.003+0000 7f47a441f640 1 --2- 172.21.15.7:0/1082848880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c107910 0x7f479c107cf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4798009880 tx=0x7f479802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 -- 172.21.15.7:0/1082848880 shutdown_connections 2024-09-21T04:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 --2- 172.21.15.7:0/1082848880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c101910 0x7f479c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 --2- 172.21.15.7:0/1082848880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c107910 0x7f479c107cf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 -- 172.21.15.7:0/1082848880 >> 172.21.15.7:0/1082848880 conn(0x7f479c0fd630 msgr2=0x7f479c0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 -- 172.21.15.7:0/1082848880 shutdown_connections 2024-09-21T04:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 -- 172.21.15.7:0/1082848880 wait complete. 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 Processor -- start 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 -- start start 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c101910 0x7f479c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c107910 0x7f479c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10ede0 con 0x7f479c101910 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.004+0000 7f47a441f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10ec60 con 0x7f479c107910 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a21bb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c101910 0x7f479c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a21bb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c101910 0x7f479c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49986/0 (socket says 172.21.15.7:49986) 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a21bb640 1 -- 172.21.15.7:0/702869131 learned_addr learned my addr 172.21.15.7:0/702869131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a19ba640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c107910 0x7f479c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a21bb640 1 -- 172.21.15.7:0/702869131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c107910 msgr2=0x7f479c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a21bb640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c107910 0x7f479c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a21bb640 1 -- 172.21.15.7:0/702869131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c1977c0 con 0x7f479c101910 2024-09-21T04:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a19ba640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c107910 0x7f479c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a21bb640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c101910 0x7f479c19d640 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4798031460 tx=0x7f47980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47937fe640 1 -- 172.21.15.7:0/702869131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f479803f070 con 0x7f479c101910 2024-09-21T04:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c197a50 con 0x7f479c101910 2024-09-21T04:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c197f80 con 0x7f479c101910 2024-09-21T04:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47937fe640 1 -- 172.21.15.7:0/702869131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47980044d0 con 0x7f479c101910 2024-09-21T04:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.005+0000 7f47937fe640 1 -- 172.21.15.7:0/702869131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4798035030 con 0x7f479c101910 2024-09-21T04:52:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.006+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c103050 con 0x7f479c101910 2024-09-21T04:52:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.007+0000 7f47937fe640 1 -- 172.21.15.7:0/702869131 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4798042070 con 0x7f479c101910 2024-09-21T04:52:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.007+0000 7f47937fe640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f478807af40 0x7f478807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:52:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.008+0000 7f47937fe640 1 -- 172.21.15.7:0/702869131 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f47980beb60 con 0x7f479c101910 2024-09-21T04:52:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.008+0000 7f47a19ba640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f478807af40 0x7f478807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:52:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.008+0000 7f47a19ba640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f478807af40 0x7f478807d400 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f478c005fd0 tx=0x7f478c005ea0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:52:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.012+0000 7f47937fe640 1 -- 172.21.15.7:0/702869131 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4798004cf0 con 0x7f479c101910 2024-09-21T04:52:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:42 smithi007 ceph-mon[108238]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:42 smithi007 ceph-mon[108238]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:42 smithi007 ceph-mon[108238]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:42 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1229027626' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:52:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:42 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3797998549' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:52:42.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.278+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f479c198cf0 con 0x7f479c101910 2024-09-21T04:52:42.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.278+0000 7f47937fe640 1 -- 172.21.15.7:0/702869131 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f479808a100 con 0x7f479c101910 2024-09-21T04:52:42.282 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:52:42.282 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:52:42.282 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:52:42.282 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:52:42.282 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:52:42.283 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:52:42.283 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:52:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f478807af40 msgr2=0x7f478807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f478807af40 0x7f478807d400 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f478c005fd0 tx=0x7f478c005ea0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c101910 msgr2=0x7f479c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:52:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c101910 0x7f479c19d640 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4798031460 tx=0x7f47980048f0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 shutdown_connections 2024-09-21T04:52:42.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f478807af40 0x7f478807d400 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f479c107910 0x7f479c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 --2- 172.21.15.7:0/702869131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f479c101910 0x7f479c19d640 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:52:42.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 >> 172.21.15.7:0/702869131 conn(0x7f479c0fd630 msgr2=0x7f479c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:52:42.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 shutdown_connections 2024-09-21T04:52:42.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:52:42.280+0000 7f47a441f640 1 -- 172.21.15.7:0/702869131 wait complete. 2024-09-21T04:52:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:42 smithi053 ceph-mon[80176]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:42 smithi053 ceph-mon[80176]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:42 smithi053 ceph-mon[80176]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:42 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1229027626' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:52:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:42 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3797998549' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:52:43.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:43 smithi053 ceph-mon[80176]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:43.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:43 smithi053 ceph-mon[80176]: pgmap v538: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:43.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:43 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/702869131' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:52:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:43 smithi007 ceph-mon[108238]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:52:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:43 smithi007 ceph-mon[108238]: pgmap v538: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:43 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/702869131' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:52:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:52:44.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:52:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:45 smithi053 ceph-mon[80176]: pgmap v539: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:45 smithi007 ceph-mon[108238]: pgmap v539: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:47 smithi007 ceph-mon[108238]: pgmap v540: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:47 smithi053 ceph-mon[80176]: pgmap v540: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:49 smithi053 ceph-mon[80176]: pgmap v541: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:49 smithi007 ceph-mon[108238]: pgmap v541: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:51.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:51 smithi053 ceph-mon[80176]: pgmap v542: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:51 smithi007 ceph-mon[108238]: pgmap v542: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:53 smithi007 ceph-mon[108238]: pgmap v543: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:53 smithi053 ceph-mon[80176]: pgmap v543: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:55 smithi053 ceph-mon[80176]: pgmap v544: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:55 smithi007 ceph-mon[108238]: pgmap v544: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:57 smithi053 ceph-mon[80176]: pgmap v545: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:57 smithi007 ceph-mon[108238]: pgmap v545: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:59 smithi007 ceph-mon[108238]: pgmap v546: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:52:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:52:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:59 smithi053 ceph-mon[80176]: pgmap v546: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:52:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:52:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:53:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:01 smithi053 ceph-mon[80176]: pgmap v547: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:01 smithi007 ceph-mon[108238]: pgmap v547: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:03 smithi053 ceph-mon[80176]: pgmap v548: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:03 smithi007 ceph-mon[108238]: pgmap v548: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:05 smithi007 ceph-mon[108238]: pgmap v549: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:05 smithi053 ceph-mon[80176]: pgmap v549: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:07 smithi053 ceph-mon[80176]: pgmap v550: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:07 smithi007 ceph-mon[108238]: pgmap v550: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:09 smithi053 ceph-mon[80176]: pgmap v551: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:09 smithi007 ceph-mon[108238]: pgmap v551: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:11 smithi007 ceph-mon[108238]: pgmap v552: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:11 smithi053 ceph-mon[80176]: pgmap v552: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.380+0000 7fb5d3592640 1 -- 172.21.15.7:0/4253309222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc102b10 msgr2=0x7fb5cc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.380+0000 7fb5d3592640 1 --2- 172.21.15.7:0/4253309222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc102b10 0x7fb5cc102ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00098b0 tx=0x7fb5c002a560 comp rx=0 tx=0).stop 2024-09-21T04:53:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.380+0000 7fb5d3592640 1 -- 172.21.15.7:0/4253309222 shutdown_connections 2024-09-21T04:53:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.380+0000 7fb5d3592640 1 --2- 172.21.15.7:0/4253309222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5cc103430 0x7fb5cc10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.380+0000 7fb5d3592640 1 --2- 172.21.15.7:0/4253309222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc102b10 0x7fb5cc102ef0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.380+0000 7fb5d3592640 1 -- 172.21.15.7:0/4253309222 >> 172.21.15.7:0/4253309222 conn(0x7fb5cc0fc550 msgr2=0x7fb5cc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.380+0000 7fb5d3592640 1 -- 172.21.15.7:0/4253309222 shutdown_connections 2024-09-21T04:53:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.380+0000 7fb5d3592640 1 -- 172.21.15.7:0/4253309222 wait complete. 2024-09-21T04:53:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.381+0000 7fb5d3592640 1 Processor -- start 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.381+0000 7fb5d3592640 1 -- start start 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.381+0000 7fb5d3592640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5cc102b10 0x7fb5cc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.381+0000 7fb5d3592640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc103430 0x7fb5cc19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.381+0000 7fb5d3592640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc10edc0 con 0x7fb5cc103430 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.381+0000 7fb5d3592640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc10ec40 con 0x7fb5cc102b10 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d0b2d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc103430 0x7fb5cc19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d0b2d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc103430 0x7fb5cc19db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44718/0 (socket says 172.21.15.7:44718) 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d0b2d640 1 -- 172.21.15.7:0/36834164 learned_addr learned my addr 172.21.15.7:0/36834164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d132e640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5cc102b10 0x7fb5cc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d0b2d640 1 -- 172.21.15.7:0/36834164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5cc102b10 msgr2=0x7fb5cc19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d0b2d640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5cc102b10 0x7fb5cc19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d0b2d640 1 -- 172.21.15.7:0/36834164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc1977d0 con 0x7fb5cc103430 2024-09-21T04:53:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d0b2d640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc103430 0x7fb5cc19db90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800c810 tx=0x7fb5c800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5be7fc640 1 -- 172.21.15.7:0/36834164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb5c8004020 con 0x7fb5cc103430 2024-09-21T04:53:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5be7fc640 1 -- 172.21.15.7:0/36834164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb5c80041c0 con 0x7fb5cc103430 2024-09-21T04:53:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc197ac0 con 0x7fb5cc103430 2024-09-21T04:53:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.382+0000 7fb5be7fc640 1 -- 172.21.15.7:0/36834164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb5c80047a0 con 0x7fb5cc103430 2024-09-21T04:53:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.383+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc198050 con 0x7fb5cc103430 2024-09-21T04:53:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.384+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5cc06a520 con 0x7fb5cc103430 2024-09-21T04:53:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.384+0000 7fb5be7fc640 1 -- 172.21.15.7:0/36834164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb5c80079b0 con 0x7fb5cc103430 2024-09-21T04:53:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.385+0000 7fb5be7fc640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5b807af40 0x7fb5b807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.385+0000 7fb5be7fc640 1 -- 172.21.15.7:0/36834164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb5c809def0 con 0x7fb5cc103430 2024-09-21T04:53:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.385+0000 7fb5d132e640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5b807af40 0x7fb5b807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.386+0000 7fb5d132e640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5b807af40 0x7fb5b807d400 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0000c00 tx=0x7fb5c0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:12.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.391+0000 7fb5be7fc640 1 -- 172.21.15.7:0/36834164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5c8014030 con 0x7fb5cc103430 2024-09-21T04:53:12.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.605+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb5cc1983c0 con 0x7fb5b807af40 2024-09-21T04:53:12.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.606+0000 7fb5be7fc640 1 -- 172.21.15.7:0/36834164 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fb5cc1983c0 con 0x7fb5b807af40 2024-09-21T04:53:12.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5b807af40 msgr2=0x7fb5b807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:12.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5b807af40 0x7fb5b807d400 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0000c00 tx=0x7fb5c0031400 comp rx=0 tx=0).stop 2024-09-21T04:53:12.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc103430 msgr2=0x7fb5cc19db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:12.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc103430 0x7fb5cc19db90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800c810 tx=0x7fb5c800ccd0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 shutdown_connections 2024-09-21T04:53:12.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb5b807af40 0x7fb5b807d400 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5cc103430 0x7fb5cc19db90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 --2- 172.21.15.7:0/36834164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5cc102b10 0x7fb5cc19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 >> 172.21.15.7:0/36834164 conn(0x7fb5cc0fc550 msgr2=0x7fb5cc107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:12.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 shutdown_connections 2024-09-21T04:53:12.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.608+0000 7fb5d3592640 1 -- 172.21.15.7:0/36834164 wait complete. 2024-09-21T04:53:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:53:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.695+0000 7f419e92c640 1 -- 172.21.15.7:0/1504488794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198101900 msgr2=0x7f4198101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.695+0000 7f419e92c640 1 --2- 172.21.15.7:0/1504488794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198101900 0x7f4198101d60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f418c009880 tx=0x7f418c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.695+0000 7f419e92c640 1 -- 172.21.15.7:0/1504488794 shutdown_connections 2024-09-21T04:53:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.695+0000 7f419e92c640 1 --2- 172.21.15.7:0/1504488794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198101900 0x7f4198101d60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.695+0000 7f419e92c640 1 --2- 172.21.15.7:0/1504488794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4198107900 0x7f4198107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.695+0000 7f419e92c640 1 -- 172.21.15.7:0/1504488794 >> 172.21.15.7:0/1504488794 conn(0x7f41980fd600 msgr2=0x7f41980ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.695+0000 7f419e92c640 1 -- 172.21.15.7:0/1504488794 shutdown_connections 2024-09-21T04:53:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.695+0000 7f419e92c640 1 -- 172.21.15.7:0/1504488794 wait complete. 2024-09-21T04:53:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.696+0000 7f419e92c640 1 Processor -- start 2024-09-21T04:53:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.696+0000 7f419e92c640 1 -- start start 2024-09-21T04:53:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.696+0000 7f419e92c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4198101900 0x7f41980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f419e92c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198107900 0x7f4198074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f419e92c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f419810edb0 con 0x7f4198107900 2024-09-21T04:53:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f419e92c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f419810ec30 con 0x7f4198101900 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198107900 0x7f4198074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198107900 0x7f4198074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44726/0 (socket says 172.21.15.7:44726) 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41977fe640 1 -- 172.21.15.7:0/3473615861 learned_addr learned my addr 172.21.15.7:0/3473615861 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f4197fff640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4198101900 0x7f41980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41977fe640 1 -- 172.21.15.7:0/3473615861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4198101900 msgr2=0x7f41980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41977fe640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4198101900 0x7f41980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41977fe640 1 -- 172.21.15.7:0/3473615861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4198074f70 con 0x7f4198107900 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41977fe640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198107900 0x7f4198074a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f418c031460 tx=0x7f418c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41957fa640 1 -- 172.21.15.7:0/3473615861 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f418c03f070 con 0x7f4198107900 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41957fa640 1 -- 172.21.15.7:0/3473615861 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f418c004cf0 con 0x7f4198107900 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4198075200 con 0x7f4198107900 2024-09-21T04:53:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.697+0000 7f41957fa640 1 -- 172.21.15.7:0/3473615861 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f418c035030 con 0x7f4198107900 2024-09-21T04:53:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.698+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41981a3d10 con 0x7f4198107900 2024-09-21T04:53:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.699+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4198103040 con 0x7f4198107900 2024-09-21T04:53:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.699+0000 7f41957fa640 1 -- 172.21.15.7:0/3473615861 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f418c042070 con 0x7f4198107900 2024-09-21T04:53:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.700+0000 7f41957fa640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418007af30 0x7f418007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.700+0000 7f4197fff640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418007af30 0x7f418007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.700+0000 7f41957fa640 1 -- 172.21.15.7:0/3473615861 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f418c0beae0 con 0x7f4198107900 2024-09-21T04:53:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.700+0000 7f4197fff640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418007af30 0x7f418007d3f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f418800abe0 tx=0x7f418800a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.704+0000 7f41957fa640 1 -- 172.21.15.7:0/3473615861 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f418c089fe0 con 0x7f4198107900 2024-09-21T04:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.909+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4198062630 con 0x7f418007af30 2024-09-21T04:53:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.910+0000 7f41957fa640 1 -- 172.21.15.7:0/3473615861 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f4198062630 con 0x7f418007af30 2024-09-21T04:53:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418007af30 msgr2=0x7f418007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418007af30 0x7f418007d3f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f418800abe0 tx=0x7f418800a2d0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198107900 msgr2=0x7f4198074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198107900 0x7f4198074a30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f418c031460 tx=0x7f418c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 shutdown_connections 2024-09-21T04:53:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418007af30 0x7f418007d3f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4198107900 0x7f4198074a30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 --2- 172.21.15.7:0/3473615861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4198101900 0x7f41980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 >> 172.21.15.7:0/3473615861 conn(0x7f41980fd600 msgr2=0x7f41980fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 shutdown_connections 2024-09-21T04:53:12.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.912+0000 7f419e92c640 1 -- 172.21.15.7:0/3473615861 wait complete. 2024-09-21T04:53:13.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.997+0000 7ff97c4e5640 1 -- 172.21.15.7:0/636410994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 msgr2=0x7ff974100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.997+0000 7ff97c4e5640 1 --2- 172.21.15.7:0/636410994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 0x7ff974100c50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff964009880 tx=0x7ff96402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.997+0000 7ff97c4e5640 1 -- 172.21.15.7:0/636410994 shutdown_connections 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.997+0000 7ff97c4e5640 1 --2- 172.21.15.7:0/636410994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 0x7ff974100c50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.997+0000 7ff97c4e5640 1 --2- 172.21.15.7:0/636410994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9741067f0 0x7ff974106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.997+0000 7ff97c4e5640 1 -- 172.21.15.7:0/636410994 >> 172.21.15.7:0/636410994 conn(0x7ff9740fc550 msgr2=0x7ff9740fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.997+0000 7ff97c4e5640 1 -- 172.21.15.7:0/636410994 shutdown_connections 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.997+0000 7ff97c4e5640 1 -- 172.21.15.7:0/636410994 wait complete. 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97c4e5640 1 Processor -- start 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97c4e5640 1 -- start start 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97c4e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 0x7ff9740fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97c4e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9741067f0 0x7ff9740fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97c4e5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff97410dcc0 con 0x7ff9741007f0 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97c4e5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff97410db40 con 0x7ff9741067f0 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97a281640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 0x7ff9740fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97a281640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 0x7ff9740fccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44750/0 (socket says 172.21.15.7:44750) 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97a281640 1 -- 172.21.15.7:0/1366227867 learned_addr learned my addr 172.21.15.7:0/1366227867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff979a80640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9741067f0 0x7ff9740fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97a281640 1 -- 172.21.15.7:0/1366227867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9741067f0 msgr2=0x7ff9740fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97a281640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9741067f0 0x7ff9740fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff97a281640 1 -- 172.21.15.7:0/1366227867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9740fd750 con 0x7ff9741007f0 2024-09-21T04:53:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.998+0000 7ff979a80640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9741067f0 0x7ff9740fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:53:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.999+0000 7ff97a281640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 0x7ff9740fccd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff97000d6f0 tx=0x7ff97000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.999+0000 7ff96b7fe640 1 -- 172.21.15.7:0/1366227867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff970017070 con 0x7ff9741007f0 2024-09-21T04:53:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.999+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9740fda40 con 0x7ff9741007f0 2024-09-21T04:53:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.999+0000 7ff96b7fe640 1 -- 172.21.15.7:0/1366227867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff97000de00 con 0x7ff9741007f0 2024-09-21T04:53:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.999+0000 7ff96b7fe640 1 -- 172.21.15.7:0/1366227867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff970004bb0 con 0x7ff9741007f0 2024-09-21T04:53:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:12.999+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9741a3e60 con 0x7ff9741007f0 2024-09-21T04:53:13.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.000+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff974101f30 con 0x7ff9741007f0 2024-09-21T04:53:13.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.001+0000 7ff96b7fe640 1 -- 172.21.15.7:0/1366227867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff970004e60 con 0x7ff9741007f0 2024-09-21T04:53:13.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.001+0000 7ff96b7fe640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.001+0000 7ff96b7fe640 1 -- 172.21.15.7:0/1366227867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff97009dcb0 con 0x7ff9741007f0 2024-09-21T04:53:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.001+0000 7ff979a80640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.002+0000 7ff979a80640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff9740fe760 tx=0x7ff964031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.006+0000 7ff96b7fe640 1 -- 172.21.15.7:0/1366227867 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff970069250 con 0x7ff9741007f0 2024-09-21T04:53:13.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.206+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff974062630 con 0x7ff96007af30 2024-09-21T04:53:13.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.216+0000 7ff96b7fe640 1 -- 172.21.15.7:0/1366227867 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7ff974062630 con 0x7ff96007af30 2024-09-21T04:53:13.219 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:53:13.219 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (15m) 9m ago 26m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:53:13.219 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (18m) 9m ago 26m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:53:13.219 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (17m) 9m ago 25m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:53:13.219 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (13m) 9m ago 26m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:53:13.219 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (13m) 9m ago 25m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:53:13.219 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (14m) 9m ago 25m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (9m) 9m ago 20m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (9m) 9m ago 20m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (20m) 9m ago 20m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (9m) 9m ago 20m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (17m) 9m ago 27m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (16m) 9m ago 25m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (13m) 9m ago 27m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (13m) 9m ago 25m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (16m) 9m ago 26m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (16m) 9m ago 25m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (13m) 9m ago 24m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (12m) 9m ago 23m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (11m) 9m ago 23m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:53:13.220 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (11m) 9m ago 22m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:53:13.221 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (10m) 9m ago 22m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:53:13.221 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (9m) 9m ago 21m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:53:13.221 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (15m) 9m ago 25m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:53:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 msgr2=0x7ff96007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff9740fe760 tx=0x7ff964031400 comp rx=0 tx=0).stop 2024-09-21T04:53:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 msgr2=0x7ff9740fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 0x7ff9740fccd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff97000d6f0 tx=0x7ff97000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 shutdown_connections 2024-09-21T04:53:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9741067f0 0x7ff9740fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 --2- 172.21.15.7:0/1366227867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9741007f0 0x7ff9740fccd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.218+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 >> 172.21.15.7:0/1366227867 conn(0x7ff9740fc550 msgr2=0x7ff974069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.219+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 shutdown_connections 2024-09-21T04:53:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.219+0000 7ff97c4e5640 1 -- 172.21.15.7:0/1366227867 wait complete. 2024-09-21T04:53:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.301+0000 7f213c8d7640 1 -- 172.21.15.7:0/448613271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 msgr2=0x7f213810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.301+0000 7f213c8d7640 1 --2- 172.21.15.7:0/448613271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 0x7f213810b8d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f212c009880 tx=0x7f212c02a540 comp rx=0 tx=0).stop 2024-09-21T04:53:13.305 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:13 smithi007 ceph-mon[108238]: pgmap v553: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.301+0000 7f213c8d7640 1 -- 172.21.15.7:0/448613271 shutdown_connections 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.301+0000 7f213c8d7640 1 --2- 172.21.15.7:0/448613271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 0x7f213810b8d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.301+0000 7f213c8d7640 1 --2- 172.21.15.7:0/448613271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2138102ac0 0x7f2138102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.301+0000 7f213c8d7640 1 -- 172.21.15.7:0/448613271 >> 172.21.15.7:0/448613271 conn(0x7f21380fc520 msgr2=0x7f21380fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.301+0000 7f213c8d7640 1 -- 172.21.15.7:0/448613271 shutdown_connections 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.301+0000 7f213c8d7640 1 -- 172.21.15.7:0/448613271 wait complete. 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.302+0000 7f213c8d7640 1 Processor -- start 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.302+0000 7f213c8d7640 1 -- start start 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f213c8d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2138102ac0 0x7f213819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f213c8d7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 0x7f213819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f213c8d7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f213810edd0 con 0x7f21381033e0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f213c8d7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f213810ec50 con 0x7f2138102ac0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f2135d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 0x7f213819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f2135d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 0x7f213819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44764/0 (socket says 172.21.15.7:44764) 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f2135d9b640 1 -- 172.21.15.7:0/563073814 learned_addr learned my addr 172.21.15.7:0/563073814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f213659c640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2138102ac0 0x7f213819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f2135d9b640 1 -- 172.21.15.7:0/563073814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2138102ac0 msgr2=0x7f213819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f2135d9b640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2138102ac0 0x7f213819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.303+0000 7f2135d9b640 1 -- 172.21.15.7:0/563073814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21381977e0 con 0x7f21381033e0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.304+0000 7f2135d9b640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 0x7f213819dba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f212c031460 tx=0x7f212c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.304+0000 7f21277fe640 1 -- 172.21.15.7:0/563073814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f212c03f070 con 0x7f21381033e0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.304+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2138197a70 con 0x7f21381033e0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.304+0000 7f21277fe640 1 -- 172.21.15.7:0/563073814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f212c004ac0 con 0x7f21381033e0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.304+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2138197fa0 con 0x7f21381033e0 2024-09-21T04:53:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.304+0000 7f21277fe640 1 -- 172.21.15.7:0/563073814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f212c035030 con 0x7f21381033e0 2024-09-21T04:53:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.305+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213806aa90 con 0x7f21381033e0 2024-09-21T04:53:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.306+0000 7f21277fe640 1 -- 172.21.15.7:0/563073814 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f212c042070 con 0x7f21381033e0 2024-09-21T04:53:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.306+0000 7f21277fe640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f212007af40 0x7f212007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.306+0000 7f21277fe640 1 -- 172.21.15.7:0/563073814 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f212c0bea60 con 0x7f21381033e0 2024-09-21T04:53:13.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.306+0000 7f213659c640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f212007af40 0x7f212007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.307+0000 7f213659c640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f212007af40 0x7f212007d400 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f212800abe0 tx=0x7f212800a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:13.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.311+0000 7f21277fe640 1 -- 172.21.15.7:0/563073814 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f212c08a000 con 0x7f21381033e0 2024-09-21T04:53:13.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:13 smithi053 ceph-mon[80176]: pgmap v553: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.571+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f2138198d10 con 0x7f21381033e0 2024-09-21T04:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.571+0000 7f21277fe640 1 -- 172.21.15.7:0/563073814 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f212c08eeb0 con 0x7f21381033e0 2024-09-21T04:53:13.574 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:53:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:53:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:53:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:53:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:53:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:53:13.576 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.573+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f212007af40 msgr2=0x7f212007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.573+0000 7f213c8d7640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f212007af40 0x7f212007d400 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f212800abe0 tx=0x7f212800a2d0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.573+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 msgr2=0x7f213819dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.573+0000 7f213c8d7640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 0x7f213819dba0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f212c031460 tx=0x7f212c0046c0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.574+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 shutdown_connections 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.574+0000 7f213c8d7640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f212007af40 0x7f212007d400 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.574+0000 7f213c8d7640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21381033e0 0x7f213819dba0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.574+0000 7f213c8d7640 1 --2- 172.21.15.7:0/563073814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2138102ac0 0x7f213819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.574+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 >> 172.21.15.7:0/563073814 conn(0x7f21380fc520 msgr2=0x7f2138108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.574+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 shutdown_connections 2024-09-21T04:53:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.574+0000 7f213c8d7640 1 -- 172.21.15.7:0/563073814 wait complete. 2024-09-21T04:53:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 -- 172.21.15.7:0/3941306622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 msgr2=0x7fd278100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 --2- 172.21.15.7:0/3941306622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 0x7fd278100c60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009880 tx=0x7fd26c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 -- 172.21.15.7:0/3941306622 shutdown_connections 2024-09-21T04:53:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 --2- 172.21.15.7:0/3941306622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 0x7fd278100c60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 --2- 172.21.15.7:0/3941306622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd278106800 0x7fd278106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 -- 172.21.15.7:0/3941306622 >> 172.21.15.7:0/3941306622 conn(0x7fd2780fc520 msgr2=0x7fd2780fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 -- 172.21.15.7:0/3941306622 shutdown_connections 2024-09-21T04:53:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 -- 172.21.15.7:0/3941306622 wait complete. 2024-09-21T04:53:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.658+0000 7fd27fa9e640 1 Processor -- start 2024-09-21T04:53:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27fa9e640 1 -- start start 2024-09-21T04:53:13.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27fa9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 0x7fd2780fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27fa9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd278106800 0x7fd2780fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27fa9e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810dcb0 con 0x7fd278100800 2024-09-21T04:53:13.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27fa9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810db30 con 0x7fd278106800 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27d83a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 0x7fd2780fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27d83a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 0x7fd2780fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55020/0 (socket says 172.21.15.7:55020) 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27d83a640 1 -- 172.21.15.7:0/2020536146 learned_addr learned my addr 172.21.15.7:0/2020536146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27d039640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd278106800 0x7fd2780fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27d83a640 1 -- 172.21.15.7:0/2020536146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd278106800 msgr2=0x7fd2780fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27d83a640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd278106800 0x7fd2780fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27d83a640 1 -- 172.21.15.7:0/2020536146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2780fd650 con 0x7fd278100800 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.659+0000 7fd27d83a640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 0x7fd2780fcbd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd27000cc60 tx=0x7fd27000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.660+0000 7fd276ffd640 1 -- 172.21.15.7:0/2020536146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd27000ce10 con 0x7fd278100800 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.660+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2780fd940 con 0x7fd278100800 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.660+0000 7fd276ffd640 1 -- 172.21.15.7:0/2020536146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd27000f910 con 0x7fd278100800 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.660+0000 7fd276ffd640 1 -- 172.21.15.7:0/2020536146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd270018ae0 con 0x7fd278100800 2024-09-21T04:53:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.660+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2781a3e10 con 0x7fd278100800 2024-09-21T04:53:13.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.661+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd278101f40 con 0x7fd278100800 2024-09-21T04:53:13.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.661+0000 7fd276ffd640 1 -- 172.21.15.7:0/2020536146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd270018ca0 con 0x7fd278100800 2024-09-21T04:53:13.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.662+0000 7fd276ffd640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd26007af30 0x7fd26007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.662+0000 7fd276ffd640 1 -- 172.21.15.7:0/2020536146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd27009e6c0 con 0x7fd278100800 2024-09-21T04:53:13.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.662+0000 7fd27d039640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd26007af30 0x7fd26007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.662+0000 7fd27d039640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd26007af30 0x7fd26007d3f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2780fe660 tx=0x7fd26c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:13.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.666+0000 7fd276ffd640 1 -- 172.21.15.7:0/2020536146 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd270014030 con 0x7fd278100800 2024-09-21T04:53:13.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.892+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd27802d7c0 con 0x7fd278100800 2024-09-21T04:53:13.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.892+0000 7fd276ffd640 1 -- 172.21.15.7:0/2020536146 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7fd270069c60 con 0x7fd278100800 2024-09-21T04:53:13.895 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:53:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:53:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:53:13.898 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:53:13.899 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:53:13.899 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:53:13.899 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:53:13.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.894+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd26007af30 msgr2=0x7fd26007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.894+0000 7fd27fa9e640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd26007af30 0x7fd26007d3f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2780fe660 tx=0x7fd26c031400 comp rx=0 tx=0).stop 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.894+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 msgr2=0x7fd2780fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.894+0000 7fd27fa9e640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 0x7fd2780fcbd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd27000cc60 tx=0x7fd27000a620 comp rx=0 tx=0).stop 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.895+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 shutdown_connections 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.895+0000 7fd27fa9e640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd26007af30 0x7fd26007d3f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.895+0000 7fd27fa9e640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd278106800 0x7fd2780fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.895+0000 7fd27fa9e640 1 --2- 172.21.15.7:0/2020536146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd278100800 0x7fd2780fcbd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.895+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 >> 172.21.15.7:0/2020536146 conn(0x7fd2780fc520 msgr2=0x7fd27806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.895+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 shutdown_connections 2024-09-21T04:53:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.895+0000 7fd27fa9e640 1 -- 172.21.15.7:0/2020536146 wait complete. 2024-09-21T04:53:13.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.980+0000 7f0d61d90640 1 -- 172.21.15.7:0/3335312155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1067f0 msgr2=0x7f0d5c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.980+0000 7f0d61d90640 1 --2- 172.21.15.7:0/3335312155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1067f0 0x7f0d5c106bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009c20 tx=0x7f0d4c02a970 comp rx=0 tx=0).stop 2024-09-21T04:53:13.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.980+0000 7f0d61d90640 1 -- 172.21.15.7:0/3335312155 shutdown_connections 2024-09-21T04:53:13.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.980+0000 7f0d61d90640 1 --2- 172.21.15.7:0/3335312155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c1007f0 0x7f0d5c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.980+0000 7f0d61d90640 1 --2- 172.21.15.7:0/3335312155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1067f0 0x7f0d5c106bd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.980+0000 7f0d61d90640 1 -- 172.21.15.7:0/3335312155 >> 172.21.15.7:0/3335312155 conn(0x7f0d5c0fc550 msgr2=0x7f0d5c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:13.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.981+0000 7f0d61d90640 1 -- 172.21.15.7:0/3335312155 shutdown_connections 2024-09-21T04:53:13.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.981+0000 7f0d61d90640 1 -- 172.21.15.7:0/3335312155 wait complete. 2024-09-21T04:53:13.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.981+0000 7f0d61d90640 1 Processor -- start 2024-09-21T04:53:13.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.981+0000 7f0d61d90640 1 -- start start 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d61d90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1007f0 0x7f0d5c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d61d90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c1067f0 0x7f0d5c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d61d90640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c10dca0 con 0x7f0d5c1007f0 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d61d90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c10db20 con 0x7f0d5c1067f0 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d5b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1007f0 0x7f0d5c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d5b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1007f0 0x7f0d5c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55046/0 (socket says 172.21.15.7:55046) 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d5b7fe640 1 -- 172.21.15.7:0/1860019642 learned_addr learned my addr 172.21.15.7:0/1860019642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d5affd640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c1067f0 0x7f0d5c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d5b7fe640 1 -- 172.21.15.7:0/1860019642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c1067f0 msgr2=0x7f0d5c198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d5b7fe640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c1067f0 0x7f0d5c198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d5b7fe640 1 -- 172.21.15.7:0/1860019642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c199410 con 0x7f0d5c1007f0 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.982+0000 7f0d5b7fe640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1007f0 0x7f0d5c1988d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c006010 tx=0x7f0d4c02fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:13.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.983+0000 7f0d58ff9640 1 -- 172.21.15.7:0/1860019642 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d4c047070 con 0x7f0d5c1007f0 2024-09-21T04:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.983+0000 7f0d58ff9640 1 -- 172.21.15.7:0/1860019642 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d4c003e30 con 0x7f0d5c1007f0 2024-09-21T04:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.983+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c1996a0 con 0x7f0d5c1007f0 2024-09-21T04:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.983+0000 7f0d58ff9640 1 -- 172.21.15.7:0/1860019642 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d4c034070 con 0x7f0d5c1007f0 2024-09-21T04:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.983+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c1a1c50 con 0x7f0d5c1007f0 2024-09-21T04:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.984+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c101f30 con 0x7f0d5c1007f0 2024-09-21T04:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.984+0000 7f0d58ff9640 1 -- 172.21.15.7:0/1860019642 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0d4c04a070 con 0x7f0d5c1007f0 2024-09-21T04:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.985+0000 7f0d58ff9640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d4407af90 0x7f0d4407d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.985+0000 7f0d58ff9640 1 -- 172.21.15.7:0/1860019642 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0d4c0bec70 con 0x7f0d5c1007f0 2024-09-21T04:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.985+0000 7f0d5affd640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d4407af90 0x7f0d4407d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.986+0000 7f0d5affd640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d4407af90 0x7f0d4407d450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c06a7f0 tx=0x7f0d5000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:13.990+0000 7f0d58ff9640 1 -- 172.21.15.7:0/1860019642 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0d4c03f070 con 0x7f0d5c1007f0 2024-09-21T04:53:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:14 smithi007 ceph-mon[108238]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:14 smithi007 ceph-mon[108238]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:53:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:14 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/563073814' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:53:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:14 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2020536146' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:53:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.194+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0d5c062630 con 0x7f0d4407af90 2024-09-21T04:53:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.196+0000 7f0d58ff9640 1 -- 172.21.15.7:0/1860019642 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0d5c062630 con 0x7f0d4407af90 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:53:14.199 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d4407af90 msgr2=0x7f0d4407d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d4407af90 0x7f0d4407d450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c06a7f0 tx=0x7f0d5000b390 comp rx=0 tx=0).stop 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1007f0 msgr2=0x7f0d5c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1007f0 0x7f0d5c1988d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c006010 tx=0x7f0d4c02fdd0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 shutdown_connections 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d4407af90 0x7f0d4407d450 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d5c1067f0 0x7f0d5c198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 --2- 172.21.15.7:0/1860019642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d5c1007f0 0x7f0d5c1988d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 >> 172.21.15.7:0/1860019642 conn(0x7f0d5c0fc550 msgr2=0x7f0d5c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 shutdown_connections 2024-09-21T04:53:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.198+0000 7f0d61d90640 1 -- 172.21.15.7:0/1860019642 wait complete. 2024-09-21T04:53:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.282+0000 7fe402fe8640 1 -- 172.21.15.7:0/4108712213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc100650 msgr2=0x7fe3fc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.282+0000 7fe402fe8640 1 --2- 172.21.15.7:0/4108712213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc100650 0x7fe3fc100a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009880 tx=0x7fe3f002a490 comp rx=0 tx=0).stop 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.282+0000 7fe402fe8640 1 -- 172.21.15.7:0/4108712213 shutdown_connections 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.282+0000 7fe402fe8640 1 --2- 172.21.15.7:0/4108712213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3fc101000 0x7fe3fc10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.282+0000 7fe402fe8640 1 --2- 172.21.15.7:0/4108712213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc100650 0x7fe3fc100a30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.282+0000 7fe402fe8640 1 -- 172.21.15.7:0/4108712213 >> 172.21.15.7:0/4108712213 conn(0x7fe3fc0fc520 msgr2=0x7fe3fc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.282+0000 7fe402fe8640 1 -- 172.21.15.7:0/4108712213 shutdown_connections 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.282+0000 7fe402fe8640 1 -- 172.21.15.7:0/4108712213 wait complete. 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe402fe8640 1 Processor -- start 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe402fe8640 1 -- start start 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe402fe8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3fc100650 0x7fe3fc19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe402fe8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc101000 0x7fe3fc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe402fe8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc110fb0 con 0x7fe3fc101000 2024-09-21T04:53:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe402fe8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc110e30 con 0x7fe3fc100650 2024-09-21T04:53:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe3fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc101000 0x7fe3fc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe3fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc101000 0x7fe3fc19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55062/0 (socket says 172.21.15.7:55062) 2024-09-21T04:53:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.283+0000 7fe3fbfff640 1 -- 172.21.15.7:0/523378325 learned_addr learned my addr 172.21.15.7:0/523378325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe400d84640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3fc100650 0x7fe3fc19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe3fbfff640 1 -- 172.21.15.7:0/523378325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3fc100650 msgr2=0x7fe3fc19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe3fbfff640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3fc100650 0x7fe3fc19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe3fbfff640 1 -- 172.21.15.7:0/523378325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc199a40 con 0x7fe3fc101000 2024-09-21T04:53:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe3fbfff640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc101000 0x7fe3fc19fdc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d6f0 tx=0x7fe3e800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe3f9ffb640 1 -- 172.21.15.7:0/523378325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3e8017070 con 0x7fe3fc101000 2024-09-21T04:53:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe3f9ffb640 1 -- 172.21.15.7:0/523378325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3e800de00 con 0x7fe3fc101000 2024-09-21T04:53:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3fc199d30 con 0x7fe3fc101000 2024-09-21T04:53:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe3f9ffb640 1 -- 172.21.15.7:0/523378325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3e8004bb0 con 0x7fe3fc101000 2024-09-21T04:53:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.284+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3fc19a2c0 con 0x7fe3fc101000 2024-09-21T04:53:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.285+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3fc06aa90 con 0x7fe3fc101000 2024-09-21T04:53:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.286+0000 7fe3f9ffb640 1 -- 172.21.15.7:0/523378325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe3e8004e60 con 0x7fe3fc101000 2024-09-21T04:53:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.286+0000 7fe3f9ffb640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3ec07af40 0x7fe3ec07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.287+0000 7fe3f9ffb640 1 -- 172.21.15.7:0/523378325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe3e809dc00 con 0x7fe3fc101000 2024-09-21T04:53:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.287+0000 7fe400d84640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3ec07af40 0x7fe3ec07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.287+0000 7fe400d84640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3ec07af40 0x7fe3ec07d400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000c00 tx=0x7fe3f0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.291+0000 7fe3f9ffb640 1 -- 172.21.15.7:0/523378325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe3e80a3020 con 0x7fe3fc101000 2024-09-21T04:53:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:14 smithi053 ceph-mon[80176]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:14 smithi053 ceph-mon[80176]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:53:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:14 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/563073814' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:53:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:14 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2020536146' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:53:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.550+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe3fc10b3d0 con 0x7fe3fc101000 2024-09-21T04:53:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.550+0000 7fe3f9ffb640 1 -- 172.21.15.7:0/523378325 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fe3e80691a0 con 0x7fe3fc101000 2024-09-21T04:53:14.553 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:53:14.553 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:53:14.553 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:53:14.554 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:53:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:53:14.554 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:53:14.554 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:53:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3ec07af40 msgr2=0x7fe3ec07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3ec07af40 0x7fe3ec07d400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000c00 tx=0x7fe3f0031400 comp rx=0 tx=0).stop 2024-09-21T04:53:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc101000 msgr2=0x7fe3fc19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc101000 0x7fe3fc19fdc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d6f0 tx=0x7fe3e800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 shutdown_connections 2024-09-21T04:53:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3ec07af40 0x7fe3ec07d400 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3fc101000 0x7fe3fc19fdc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 --2- 172.21.15.7:0/523378325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3fc100650 0x7fe3fc19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.552+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 >> 172.21.15.7:0/523378325 conn(0x7fe3fc0fc520 msgr2=0x7fe3fc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.553+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 shutdown_connections 2024-09-21T04:53:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:14.553+0000 7fe402fe8640 1 -- 172.21.15.7:0/523378325 wait complete. 2024-09-21T04:53:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:15 smithi053 ceph-mon[80176]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:15 smithi053 ceph-mon[80176]: pgmap v554: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:15 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/523378325' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:53:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:15 smithi007 ceph-mon[108238]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:15 smithi007 ceph-mon[108238]: pgmap v554: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:15 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/523378325' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:53:16.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:16 smithi053 ceph-mon[80176]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:16.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:16 smithi007 ceph-mon[108238]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:17 smithi007 ceph-mon[108238]: pgmap v555: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:17 smithi053 ceph-mon[80176]: pgmap v555: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:19.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:19 smithi053 ceph-mon[80176]: pgmap v556: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:19 smithi007 ceph-mon[108238]: pgmap v556: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:21 smithi053 ceph-mon[80176]: pgmap v557: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:21 smithi007 ceph-mon[108238]: pgmap v557: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:23 smithi007 ceph-mon[108238]: pgmap v558: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:23 smithi053 ceph-mon[80176]: pgmap v558: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:25 smithi053 ceph-mon[80176]: pgmap v559: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:25 smithi007 ceph-mon[108238]: pgmap v559: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:27 smithi053 ceph-mon[80176]: pgmap v560: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:27 smithi007 ceph-mon[108238]: pgmap v560: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:29 smithi007 ceph-mon[108238]: pgmap v561: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:53:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:29 smithi053 ceph-mon[80176]: pgmap v561: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:53:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:31 smithi053 ceph-mon[80176]: pgmap v562: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:31 smithi007 ceph-mon[108238]: pgmap v562: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:33 smithi053 ceph-mon[80176]: pgmap v563: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:33 smithi007 ceph-mon[108238]: pgmap v563: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:35.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:34 smithi053 ceph-mon[80176]: pgmap v564: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:34 smithi007 ceph-mon[108238]: pgmap v564: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:37 smithi053 ceph-mon[80176]: pgmap v565: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:37 smithi007 ceph-mon[108238]: pgmap v565: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:39.415 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:39 smithi053 ceph-mon[80176]: pgmap v566: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:39 smithi007 ceph-mon[108238]: pgmap v566: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:41 smithi007 ceph-mon[108238]: pgmap v567: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:41 smithi053 ceph-mon[80176]: pgmap v567: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:43.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:43 smithi007 ceph-mon[108238]: pgmap v568: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:43 smithi053 ceph-mon[80176]: pgmap v568: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:53:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:53:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.650+0000 7f61f9c11640 1 -- 172.21.15.7:0/3282530742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 msgr2=0x7f61f4102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.650+0000 7f61f9c11640 1 --2- 172.21.15.7:0/3282530742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 0x7f61f4102f50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009880 tx=0x7f61e402a530 comp rx=0 tx=0).stop 2024-09-21T04:53:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.650+0000 7f61f9c11640 1 -- 172.21.15.7:0/3282530742 shutdown_connections 2024-09-21T04:53:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.650+0000 7f61f9c11640 1 --2- 172.21.15.7:0/3282530742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f4103490 0x7f61f410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.650+0000 7f61f9c11640 1 --2- 172.21.15.7:0/3282530742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 0x7f61f4102f50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.650+0000 7f61f9c11640 1 -- 172.21.15.7:0/3282530742 >> 172.21.15.7:0/3282530742 conn(0x7f61f40fc550 msgr2=0x7f61f40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.650+0000 7f61f9c11640 1 -- 172.21.15.7:0/3282530742 shutdown_connections 2024-09-21T04:53:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.650+0000 7f61f9c11640 1 -- 172.21.15.7:0/3282530742 wait complete. 2024-09-21T04:53:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.651+0000 7f61f9c11640 1 Processor -- start 2024-09-21T04:53:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.651+0000 7f61f9c11640 1 -- start start 2024-09-21T04:53:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.651+0000 7f61f9c11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 0x7f61f419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.651+0000 7f61f9c11640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f4103490 0x7f61f419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.651+0000 7f61f9c11640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61f410ede0 con 0x7f61f4102b70 2024-09-21T04:53:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.651+0000 7f61f9c11640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f61f410ec60 con 0x7f61f4103490 2024-09-21T04:53:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 0x7f61f419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 0x7f61f419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33792/0 (socket says 172.21.15.7:33792) 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f37fe640 1 -- 172.21.15.7:0/1363167298 learned_addr learned my addr 172.21.15.7:0/1363167298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f2ffd640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f4103490 0x7f61f419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f37fe640 1 -- 172.21.15.7:0/1363167298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f4103490 msgr2=0x7f61f419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f37fe640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f4103490 0x7f61f419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f37fe640 1 -- 172.21.15.7:0/1363167298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f41999e0 con 0x7f61f4102b70 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f37fe640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 0x7f61f419f820 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f61e4004f90 tx=0x7f61e40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f0ff9640 1 -- 172.21.15.7:0/1363167298 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f61e403f070 con 0x7f61f4102b70 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f4199c70 con 0x7f61f4102b70 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f0ff9640 1 -- 172.21.15.7:0/1363167298 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61e4004cf0 con 0x7f61f4102b70 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.652+0000 7f61f0ff9640 1 -- 172.21.15.7:0/1363167298 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f61e4035030 con 0x7f61f4102b70 2024-09-21T04:53:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.653+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f419a1a0 con 0x7f61f4102b70 2024-09-21T04:53:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.654+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f406a4c0 con 0x7f61f4102b70 2024-09-21T04:53:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.654+0000 7f61f0ff9640 1 -- 172.21.15.7:0/1363167298 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f61e4042070 con 0x7f61f4102b70 2024-09-21T04:53:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.655+0000 7f61f0ff9640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61dc07af40 0x7f61dc07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.655+0000 7f61f2ffd640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61dc07af40 0x7f61dc07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:44.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.655+0000 7f61f0ff9640 1 -- 172.21.15.7:0/1363167298 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f61e40bebd0 con 0x7f61f4102b70 2024-09-21T04:53:44.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.656+0000 7f61f2ffd640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61dc07af40 0x7f61dc07d400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005fd0 tx=0x7f61e8005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.660+0000 7f61f0ff9640 1 -- 172.21.15.7:0/1363167298 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61e408a170 con 0x7f61f4102b70 2024-09-21T04:53:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.865+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f61f419af10 con 0x7f61dc07af40 2024-09-21T04:53:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.866+0000 7f61f0ff9640 1 -- 172.21.15.7:0/1363167298 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f61f419af10 con 0x7f61dc07af40 2024-09-21T04:53:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61dc07af40 msgr2=0x7f61dc07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61dc07af40 0x7f61dc07d400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005fd0 tx=0x7f61e8005e20 comp rx=0 tx=0).stop 2024-09-21T04:53:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 msgr2=0x7f61f419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 0x7f61f419f820 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f61e4004f90 tx=0x7f61e40048f0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 shutdown_connections 2024-09-21T04:53:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f61dc07af40 0x7f61dc07d400 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f4103490 0x7f61f419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 --2- 172.21.15.7:0/1363167298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f4102b70 0x7f61f419f820 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 >> 172.21.15.7:0/1363167298 conn(0x7f61f40fc550 msgr2=0x7f61f4100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 shutdown_connections 2024-09-21T04:53:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.868+0000 7f61f9c11640 1 -- 172.21.15.7:0/1363167298 wait complete. 2024-09-21T04:53:44.881 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:53:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.954+0000 7f01b3349640 1 -- 172.21.15.7:0/1254303294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac101900 msgr2=0x7f01ac101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.954+0000 7f01b3349640 1 --2- 172.21.15.7:0/1254303294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac101900 0x7f01ac101d60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f019c009880 tx=0x7f019c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.954+0000 7f01b3349640 1 -- 172.21.15.7:0/1254303294 shutdown_connections 2024-09-21T04:53:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.954+0000 7f01b3349640 1 --2- 172.21.15.7:0/1254303294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac101900 0x7f01ac101d60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.954+0000 7f01b3349640 1 --2- 172.21.15.7:0/1254303294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac107900 0x7f01ac107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.954+0000 7f01b3349640 1 -- 172.21.15.7:0/1254303294 >> 172.21.15.7:0/1254303294 conn(0x7f01ac0fd600 msgr2=0x7f01ac0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.954+0000 7f01b3349640 1 -- 172.21.15.7:0/1254303294 shutdown_connections 2024-09-21T04:53:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.954+0000 7f01b3349640 1 -- 172.21.15.7:0/1254303294 wait complete. 2024-09-21T04:53:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b3349640 1 Processor -- start 2024-09-21T04:53:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b3349640 1 -- start start 2024-09-21T04:53:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b3349640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac101900 0x7f01ac19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b3349640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac107900 0x7f01ac19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b3349640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10edd0 con 0x7f01ac107900 2024-09-21T04:53:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b3349640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10ec50 con 0x7f01ac101900 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b08e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac107900 0x7f01ac19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b08e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac107900 0x7f01ac19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33804/0 (socket says 172.21.15.7:33804) 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b08e4640 1 -- 172.21.15.7:0/2323471753 learned_addr learned my addr 172.21.15.7:0/2323471753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b10e5640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac101900 0x7f01ac19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b08e4640 1 -- 172.21.15.7:0/2323471753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac101900 msgr2=0x7f01ac19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b08e4640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac101900 0x7f01ac19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.955+0000 7f01b08e4640 1 -- 172.21.15.7:0/2323471753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac1977a0 con 0x7f01ac107900 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.956+0000 7f01b08e4640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac107900 0x7f01ac19db60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f019c031460 tx=0x7f019c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.956+0000 7f01a27fc640 1 -- 172.21.15.7:0/2323471753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f019c03f070 con 0x7f01ac107900 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.956+0000 7f01a27fc640 1 -- 172.21.15.7:0/2323471753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f019c004cf0 con 0x7f01ac107900 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.956+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac197a30 con 0x7f01ac107900 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.956+0000 7f01a27fc640 1 -- 172.21.15.7:0/2323471753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f019c035030 con 0x7f01ac107900 2024-09-21T04:53:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.956+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac197f60 con 0x7f01ac107900 2024-09-21T04:53:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.957+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac103040 con 0x7f01ac107900 2024-09-21T04:53:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.957+0000 7f01a27fc640 1 -- 172.21.15.7:0/2323471753 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f019c042070 con 0x7f01ac107900 2024-09-21T04:53:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.958+0000 7f01a27fc640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f019807af30 0x7f019807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.958+0000 7f01a27fc640 1 -- 172.21.15.7:0/2323471753 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f019c0beb20 con 0x7f01ac107900 2024-09-21T04:53:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.958+0000 7f01b10e5640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f019807af30 0x7f019807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.958+0000 7f01b10e5640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f019807af30 0x7f019807d3f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005fd0 tx=0x7f01a8005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:44.962+0000 7f01a27fc640 1 -- 172.21.15.7:0/2323471753 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f019c08a0c0 con 0x7f01ac107900 2024-09-21T04:53:45.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.168+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f01ac1982d0 con 0x7f019807af30 2024-09-21T04:53:45.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.169+0000 7f01a27fc640 1 -- 172.21.15.7:0/2323471753 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f01ac1982d0 con 0x7f019807af30 2024-09-21T04:53:45.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.171+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f019807af30 msgr2=0x7f019807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.171+0000 7f01b3349640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f019807af30 0x7f019807d3f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005fd0 tx=0x7f01a8005e20 comp rx=0 tx=0).stop 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.171+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac107900 msgr2=0x7f01ac19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.171+0000 7f01b3349640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac107900 0x7f01ac19db60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f019c031460 tx=0x7f019c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.172+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 shutdown_connections 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.172+0000 7f01b3349640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f019807af30 0x7f019807d3f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.172+0000 7f01b3349640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01ac107900 0x7f01ac19db60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.172+0000 7f01b3349640 1 --2- 172.21.15.7:0/2323471753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01ac101900 0x7f01ac19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.172+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 >> 172.21.15.7:0/2323471753 conn(0x7f01ac0fd600 msgr2=0x7f01ac0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.172+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 shutdown_connections 2024-09-21T04:53:45.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.172+0000 7f01b3349640 1 -- 172.21.15.7:0/2323471753 wait complete. 2024-09-21T04:53:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.257+0000 7f6ea27d2640 1 -- 172.21.15.7:0/3615252772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c101900 msgr2=0x7f6e9c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.257+0000 7f6ea27d2640 1 --2- 172.21.15.7:0/3615252772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c101900 0x7f6e9c101d60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009880 tx=0x7f6e9002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.257+0000 7f6ea27d2640 1 -- 172.21.15.7:0/3615252772 shutdown_connections 2024-09-21T04:53:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.257+0000 7f6ea27d2640 1 --2- 172.21.15.7:0/3615252772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c101900 0x7f6e9c101d60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.257+0000 7f6ea27d2640 1 --2- 172.21.15.7:0/3615252772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e9c107900 0x7f6e9c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.257+0000 7f6ea27d2640 1 -- 172.21.15.7:0/3615252772 >> 172.21.15.7:0/3615252772 conn(0x7f6e9c0fd600 msgr2=0x7f6e9c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.257+0000 7f6ea27d2640 1 -- 172.21.15.7:0/3615252772 shutdown_connections 2024-09-21T04:53:45.262 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:45 smithi007 ceph-mon[108238]: pgmap v569: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.257+0000 7f6ea27d2640 1 -- 172.21.15.7:0/3615252772 wait complete. 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.258+0000 7f6ea27d2640 1 Processor -- start 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.258+0000 7f6ea27d2640 1 -- start start 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.258+0000 7f6ea27d2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e9c101900 0x7f6e9c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.258+0000 7f6ea27d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c107900 0x7f6e9c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.258+0000 7f6ea27d2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c10edd0 con 0x7f6e9c107900 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.258+0000 7f6ea27d2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c10ec50 con 0x7f6e9c101900 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c107900 0x7f6e9c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c107900 0x7f6e9c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33818/0 (socket says 172.21.15.7:33818) 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e9b7fe640 1 -- 172.21.15.7:0/1151659268 learned_addr learned my addr 172.21.15.7:0/1151659268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e9bfff640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e9c101900 0x7f6e9c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e9b7fe640 1 -- 172.21.15.7:0/1151659268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e9c101900 msgr2=0x7f6e9c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e9b7fe640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e9c101900 0x7f6e9c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e9b7fe640 1 -- 172.21.15.7:0/1151659268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c197800 con 0x7f6e9c107900 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e9b7fe640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c107900 0x7f6e9c19dbc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e90031460 tx=0x7f6e900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e997fa640 1 -- 172.21.15.7:0/1151659268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6e9003f070 con 0x7f6e9c107900 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e997fa640 1 -- 172.21.15.7:0/1151659268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e90004cf0 con 0x7f6e9c107900 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9c197a90 con 0x7f6e9c107900 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.259+0000 7f6e997fa640 1 -- 172.21.15.7:0/1151659268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6e90035030 con 0x7f6e9c107900 2024-09-21T04:53:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.260+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9c197fc0 con 0x7f6e9c107900 2024-09-21T04:53:45.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.261+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9c103040 con 0x7f6e9c107900 2024-09-21T04:53:45.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.261+0000 7f6e997fa640 1 -- 172.21.15.7:0/1151659268 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6e90042070 con 0x7f6e9c107900 2024-09-21T04:53:45.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.262+0000 7f6e997fa640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e8407af80 0x7f6e8407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.262+0000 7f6e997fa640 1 -- 172.21.15.7:0/1151659268 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6e900be990 con 0x7f6e9c107900 2024-09-21T04:53:45.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.262+0000 7f6e9bfff640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e8407af80 0x7f6e8407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.263+0000 7f6e9bfff640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e8407af80 0x7f6e8407d440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0058c0 tx=0x7f6e8c00a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.267+0000 7f6e997fa640 1 -- 172.21.15.7:0/1151659268 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e90089fb0 con 0x7f6e9c107900 2024-09-21T04:53:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:45 smithi053 ceph-mon[80176]: pgmap v569: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:45.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.462+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6e9c198270 con 0x7f6e8407af80 2024-09-21T04:53:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.472+0000 7f6e997fa640 1 -- 172.21.15.7:0/1151659268 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f6e9c198270 con 0x7f6e8407af80 2024-09-21T04:53:45.476 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:53:45.476 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (15m) 9m ago 26m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:53:45.476 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (18m) 9m ago 26m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:53:45.476 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (18m) 9m ago 26m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:53:45.476 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (13m) 9m ago 26m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:53:45.476 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (13m) 9m ago 26m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:53:45.476 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (15m) 9m ago 26m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (9m) 9m ago 21m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (10m) 9m ago 21m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (21m) 9m ago 21m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (9m) 9m ago 21m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (17m) 9m ago 28m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (17m) 9m ago 25m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (14m) 9m ago 28m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (14m) 9m ago 25m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (16m) 9m ago 26m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (16m) 9m ago 25m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (13m) 9m ago 24m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (12m) 9m ago 24m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (12m) 9m ago 23m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (11m) 9m ago 23m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:53:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (11m) 9m ago 22m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:53:45.478 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (10m) 9m ago 22m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:53:45.478 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (16m) 9m ago 26m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:53:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e8407af80 msgr2=0x7f6e8407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e8407af80 0x7f6e8407d440 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0058c0 tx=0x7f6e8c00a250 comp rx=0 tx=0).stop 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c107900 msgr2=0x7f6e9c19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c107900 0x7f6e9c19dbc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e90031460 tx=0x7f6e900048f0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 shutdown_connections 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e8407af80 0x7f6e8407d440 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c107900 0x7f6e9c19dbc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 --2- 172.21.15.7:0/1151659268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e9c101900 0x7f6e9c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 >> 172.21.15.7:0/1151659268 conn(0x7f6e9c0fd600 msgr2=0x7f6e9c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 shutdown_connections 2024-09-21T04:53:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.475+0000 7f6ea27d2640 1 -- 172.21.15.7:0/1151659268 wait complete. 2024-09-21T04:53:45.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.558+0000 7f83c59b4640 1 -- 172.21.15.7:0/3183480749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0102ab0 msgr2=0x7f83c0102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.558+0000 7f83c59b4640 1 --2- 172.21.15.7:0/3183480749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0102ab0 0x7f83c0102e90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009880 tx=0x7f83b002a530 comp rx=0 tx=0).stop 2024-09-21T04:53:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.558+0000 7f83c59b4640 1 -- 172.21.15.7:0/3183480749 shutdown_connections 2024-09-21T04:53:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.558+0000 7f83c59b4640 1 --2- 172.21.15.7:0/3183480749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c01033d0 0x7f83c010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.558+0000 7f83c59b4640 1 --2- 172.21.15.7:0/3183480749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0102ab0 0x7f83c0102e90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.558+0000 7f83c59b4640 1 -- 172.21.15.7:0/3183480749 >> 172.21.15.7:0/3183480749 conn(0x7f83c00fc550 msgr2=0x7f83c00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.559+0000 7f83c59b4640 1 -- 172.21.15.7:0/3183480749 shutdown_connections 2024-09-21T04:53:45.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.559+0000 7f83c59b4640 1 -- 172.21.15.7:0/3183480749 wait complete. 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.559+0000 7f83c59b4640 1 Processor -- start 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83c59b4640 1 -- start start 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83c59b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c0102ab0 0x7f83c019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83c59b4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c01033d0 0x7f83c019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83c59b4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83c010ede0 con 0x7f83c01033d0 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83c59b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83c010ec60 con 0x7f83c0102ab0 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83be7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c01033d0 0x7f83c019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83be7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c01033d0 0x7f83c019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33832/0 (socket says 172.21.15.7:33832) 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83be7fc640 1 -- 172.21.15.7:0/2206344925 learned_addr learned my addr 172.21.15.7:0/2206344925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:45.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83beffd640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c0102ab0 0x7f83c019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83be7fc640 1 -- 172.21.15.7:0/2206344925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c0102ab0 msgr2=0x7f83c019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83be7fc640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c0102ab0 0x7f83c019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.560+0000 7f83be7fc640 1 -- 172.21.15.7:0/2206344925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c01977b0 con 0x7f83c01033d0 2024-09-21T04:53:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.561+0000 7f83be7fc640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c01033d0 0x7f83c019db70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f83b400d6d0 tx=0x7f83b400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.561+0000 7f83c49b2640 1 -- 172.21.15.7:0/2206344925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f83b4017070 con 0x7f83c01033d0 2024-09-21T04:53:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.561+0000 7f83c49b2640 1 -- 172.21.15.7:0/2206344925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f83b40054e0 con 0x7f83c01033d0 2024-09-21T04:53:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.561+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c0197aa0 con 0x7f83c01033d0 2024-09-21T04:53:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.561+0000 7f83c49b2640 1 -- 172.21.15.7:0/2206344925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f83b4004ae0 con 0x7f83c01033d0 2024-09-21T04:53:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.561+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c0198030 con 0x7f83c01033d0 2024-09-21T04:53:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.562+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c006a580 con 0x7f83c01033d0 2024-09-21T04:53:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.563+0000 7f83c49b2640 1 -- 172.21.15.7:0/2206344925 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f83b4004de0 con 0x7f83c01033d0 2024-09-21T04:53:45.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.563+0000 7f83c49b2640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f83a807af30 0x7f83a807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.563+0000 7f83c49b2640 1 -- 172.21.15.7:0/2206344925 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f83b409dc50 con 0x7f83c01033d0 2024-09-21T04:53:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.563+0000 7f83beffd640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f83a807af30 0x7f83a807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.564+0000 7f83beffd640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f83a807af30 0x7f83a807d3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f83b00096c0 tx=0x7f83b0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:45.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.568+0000 7f83c49b2640 1 -- 172.21.15.7:0/2206344925 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83b40691f0 con 0x7f83c01033d0 2024-09-21T04:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.829+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f83c0198ce0 con 0x7f83c01033d0 2024-09-21T04:53:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.829+0000 7f83c49b2640 1 -- 172.21.15.7:0/2206344925 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f83b406e0a0 con 0x7f83c01033d0 2024-09-21T04:53:45.833 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:53:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:53:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:53:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:53:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:53:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:53:45.835 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:53:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f83a807af30 msgr2=0x7f83a807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f83a807af30 0x7f83a807d3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f83b00096c0 tx=0x7f83b0031400 comp rx=0 tx=0).stop 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c01033d0 msgr2=0x7f83c019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c01033d0 0x7f83c019db70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f83b400d6d0 tx=0x7f83b400db90 comp rx=0 tx=0).stop 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 shutdown_connections 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f83a807af30 0x7f83a807d3f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c01033d0 0x7f83c019db70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 --2- 172.21.15.7:0/2206344925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83c0102ab0 0x7f83c019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 >> 172.21.15.7:0/2206344925 conn(0x7f83c00fc550 msgr2=0x7f83c010aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 shutdown_connections 2024-09-21T04:53:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.831+0000 7f83c59b4640 1 -- 172.21.15.7:0/2206344925 wait complete. 2024-09-21T04:53:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.915+0000 7f412a5f8640 1 -- 172.21.15.7:0/3976796696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 msgr2=0x7f41241065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.915+0000 7f412a5f8640 1 --2- 172.21.15.7:0/3976796696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 0x7f41241065d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4118009880 tx=0x7f411802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.915+0000 7f412a5f8640 1 -- 172.21.15.7:0/3976796696 shutdown_connections 2024-09-21T04:53:45.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.915+0000 7f412a5f8640 1 --2- 172.21.15.7:0/3976796696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 0x7f41241065d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.915+0000 7f412a5f8640 1 --2- 172.21.15.7:0/3976796696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41241000e0 0x7f41241004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.915+0000 7f412a5f8640 1 -- 172.21.15.7:0/3976796696 >> 172.21.15.7:0/3976796696 conn(0x7f41240fbc90 msgr2=0x7f41240fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:45.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.915+0000 7f412a5f8640 1 -- 172.21.15.7:0/3976796696 shutdown_connections 2024-09-21T04:53:45.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.915+0000 7f412a5f8640 1 -- 172.21.15.7:0/3976796696 wait complete. 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.916+0000 7f412a5f8640 1 Processor -- start 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.916+0000 7f412a5f8640 1 -- start start 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.916+0000 7f412a5f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41241000e0 0x7f4124101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.916+0000 7f412a5f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 0x7f41241015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.916+0000 7f412a5f8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f412410ee50 con 0x7f4124100a90 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.916+0000 7f412a5f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f412410ecd0 con 0x7f41241000e0 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41237fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 0x7f41241015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41237fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 0x7f41241015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33854/0 (socket says 172.21.15.7:33854) 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41237fe640 1 -- 172.21.15.7:0/1638105515 learned_addr learned my addr 172.21.15.7:0/1638105515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f4123fff640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41241000e0 0x7f4124101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41237fe640 1 -- 172.21.15.7:0/1638105515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41241000e0 msgr2=0x7f4124101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41237fe640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41241000e0 0x7f4124101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41237fe640 1 -- 172.21.15.7:0/1638105515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4124101ae0 con 0x7f4124100a90 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41237fe640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 0x7f41241015a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4118031460 tx=0x7f4118004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41217fa640 1 -- 172.21.15.7:0/1638105515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f411803f070 con 0x7f4124100a90 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4124101d70 con 0x7f4124100a90 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41217fa640 1 -- 172.21.15.7:0/1638105515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f41180035f0 con 0x7f4124100a90 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f41217fa640 1 -- 172.21.15.7:0/1638105515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4118035030 con 0x7f4124100a90 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.917+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a3ec0 con 0x7f4124100a90 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.919+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41241031a0 con 0x7f4124100a90 2024-09-21T04:53:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.919+0000 7f41217fa640 1 -- 172.21.15.7:0/1638105515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4118042070 con 0x7f4124100a90 2024-09-21T04:53:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.920+0000 7f41217fa640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f410c07af30 0x7f410c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.920+0000 7f41217fa640 1 -- 172.21.15.7:0/1638105515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f41180bea10 con 0x7f4124100a90 2024-09-21T04:53:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.920+0000 7f4123fff640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f410c07af30 0x7f410c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.920+0000 7f4123fff640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f410c07af30 0x7f410c07d3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4114009b60 tx=0x7f4114005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:45.924+0000 7f41217fa640 1 -- 172.21.15.7:0/1638105515 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4118089fb0 con 0x7f4124100a90 2024-09-21T04:53:46.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.152+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4124102590 con 0x7f4124100a90 2024-09-21T04:53:46.156 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:46 smithi007 ceph-mon[108238]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:46.157 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:46 smithi007 ceph-mon[108238]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:46.157 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:46 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2206344925' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:53:46.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.153+0000 7f41217fa640 1 -- 172.21.15.7:0/1638105515 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f411808ee60 con 0x7f4124100a90 2024-09-21T04:53:46.157 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:53:46.157 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:53:46.157 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:53:46.157 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:53:46.157 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:53:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:53:46.158 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:53:46.159 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:53:46.160 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:53:46.161 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:53:46.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f410c07af30 msgr2=0x7f410c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f410c07af30 0x7f410c07d3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4114009b60 tx=0x7f4114005f90 comp rx=0 tx=0).stop 2024-09-21T04:53:46.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 msgr2=0x7f41241015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 0x7f41241015a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4118031460 tx=0x7f4118004530 comp rx=0 tx=0).stop 2024-09-21T04:53:46.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 shutdown_connections 2024-09-21T04:53:46.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f410c07af30 0x7f410c07d3f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4124100a90 0x7f41241015a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 --2- 172.21.15.7:0/1638105515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41241000e0 0x7f4124101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 >> 172.21.15.7:0/1638105515 conn(0x7f41240fbc90 msgr2=0x7f41240fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:46.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 shutdown_connections 2024-09-21T04:53:46.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.155+0000 7f412a5f8640 1 -- 172.21.15.7:0/1638105515 wait complete. 2024-09-21T04:53:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2456142049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 msgr2=0x7f52cc102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 --2- 172.21.15.7:0/2456142049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 0x7f52cc102f30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009c20 tx=0x7f52bc02aa00 comp rx=0 tx=0).stop 2024-09-21T04:53:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2456142049 shutdown_connections 2024-09-21T04:53:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 --2- 172.21.15.7:0/2456142049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52cc103470 0x7f52cc10b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 --2- 172.21.15.7:0/2456142049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 0x7f52cc102f30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2456142049 >> 172.21.15.7:0/2456142049 conn(0x7f52cc0fc550 msgr2=0x7f52cc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2456142049 shutdown_connections 2024-09-21T04:53:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2456142049 wait complete. 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 Processor -- start 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.241+0000 7f52d1f0a640 1 -- start start 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52d1f0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 0x7f52cc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52d1f0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52cc103470 0x7f52cc19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52d1f0a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc10ede0 con 0x7f52cc102b50 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52d1f0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc10ec60 con 0x7f52cc103470 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52cb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 0x7f52cc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52cb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 0x7f52cc19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33866/0 (socket says 172.21.15.7:33866) 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52cb7fe640 1 -- 172.21.15.7:0/2200307713 learned_addr learned my addr 172.21.15.7:0/2200307713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52caffd640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52cc103470 0x7f52cc19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52cb7fe640 1 -- 172.21.15.7:0/2200307713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52cc103470 msgr2=0x7f52cc19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52cb7fe640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52cc103470 0x7f52cc19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52cb7fe640 1 -- 172.21.15.7:0/2200307713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc1977d0 con 0x7f52cc102b50 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.242+0000 7f52cb7fe640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 0x7f52cc19d650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f52bc005cc0 tx=0x7f52bc02ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.243+0000 7f52c8ff9640 1 -- 172.21.15.7:0/2200307713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52bc047070 con 0x7f52cc102b50 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.243+0000 7f52c8ff9640 1 -- 172.21.15.7:0/2200307713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52bc003e30 con 0x7f52cc102b50 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.243+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc197a60 con 0x7f52cc102b50 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.243+0000 7f52c8ff9640 1 -- 172.21.15.7:0/2200307713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52bc033050 con 0x7f52cc102b50 2024-09-21T04:53:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.243+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc197f90 con 0x7f52cc102b50 2024-09-21T04:53:46.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.244+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc06a4e0 con 0x7f52cc102b50 2024-09-21T04:53:46.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.244+0000 7f52c8ff9640 1 -- 172.21.15.7:0/2200307713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f52bc04a070 con 0x7f52cc102b50 2024-09-21T04:53:46.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.245+0000 7f52c8ff9640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52b407af80 0x7f52b407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:46.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.245+0000 7f52c8ff9640 1 -- 172.21.15.7:0/2200307713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f52bc03e030 con 0x7f52cc102b50 2024-09-21T04:53:46.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.245+0000 7f52caffd640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52b407af80 0x7f52b407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:46.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.246+0000 7f52caffd640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52b407af80 0x7f52b407d440 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f52cc108920 tx=0x7f52c000a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:46.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.249+0000 7f52c8ff9640 1 -- 172.21.15.7:0/2200307713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52bc08a750 con 0x7f52cc102b50 2024-09-21T04:53:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:46 smithi053 ceph-mon[80176]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:46 smithi053 ceph-mon[80176]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:46 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2206344925' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:53:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.455+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f52cc198300 con 0x7f52b407af80 2024-09-21T04:53:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.456+0000 7f52c8ff9640 1 -- 172.21.15.7:0/2200307713 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f52cc198300 con 0x7f52b407af80 2024-09-21T04:53:46.460 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:53:46.461 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52b407af80 msgr2=0x7f52b407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52b407af80 0x7f52b407d440 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f52cc108920 tx=0x7f52c000a250 comp rx=0 tx=0).stop 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 msgr2=0x7f52cc19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 0x7f52cc19d650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f52bc005cc0 tx=0x7f52bc02ee50 comp rx=0 tx=0).stop 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 shutdown_connections 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f52b407af80 0x7f52b407d440 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52cc103470 0x7f52cc19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 --2- 172.21.15.7:0/2200307713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc102b50 0x7f52cc19d650 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 >> 172.21.15.7:0/2200307713 conn(0x7f52cc0fc550 msgr2=0x7f52cc108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 shutdown_connections 2024-09-21T04:53:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.459+0000 7f52d1f0a640 1 -- 172.21.15.7:0/2200307713 wait complete. 2024-09-21T04:53:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.543+0000 7ff1c4663640 1 -- 172.21.15.7:0/1420643143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074510 msgr2=0x7ff1bc0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.543+0000 7ff1c4663640 1 --2- 172.21.15.7:0/1420643143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074510 0x7ff1bc0748f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009880 tx=0x7ff1b402a530 comp rx=0 tx=0).stop 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.543+0000 7ff1c4663640 1 -- 172.21.15.7:0/1420643143 shutdown_connections 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.543+0000 7ff1c4663640 1 --2- 172.21.15.7:0/1420643143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1bc074e30 0x7ff1bc10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.543+0000 7ff1c4663640 1 --2- 172.21.15.7:0/1420643143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074510 0x7ff1bc0748f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.543+0000 7ff1c4663640 1 -- 172.21.15.7:0/1420643143 >> 172.21.15.7:0/1420643143 conn(0x7ff1bc0fd600 msgr2=0x7ff1bc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.543+0000 7ff1c4663640 1 -- 172.21.15.7:0/1420643143 shutdown_connections 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.543+0000 7ff1c4663640 1 -- 172.21.15.7:0/1420643143 wait complete. 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c4663640 1 Processor -- start 2024-09-21T04:53:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c4663640 1 -- start start 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c4663640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1bc074510 0x7ff1bc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c4663640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074e30 0x7ff1bc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c4663640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc110fe0 con 0x7ff1bc074e30 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c4663640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc110e60 con 0x7ff1bc074510 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c1bfe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074e30 0x7ff1bc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c1bfe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074e30 0x7ff1bc19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33878/0 (socket says 172.21.15.7:33878) 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.544+0000 7ff1c1bfe640 1 -- 172.21.15.7:0/4225840817 learned_addr learned my addr 172.21.15.7:0/4225840817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1c23ff640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1bc074510 0x7ff1bc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1c1bfe640 1 -- 172.21.15.7:0/4225840817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1bc074510 msgr2=0x7ff1bc19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1c1bfe640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1bc074510 0x7ff1bc19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1c1bfe640 1 -- 172.21.15.7:0/4225840817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc1999f0 con 0x7ff1bc074e30 2024-09-21T04:53:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1c1bfe640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074e30 0x7ff1bc19fd70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000d6f0 tx=0x7ff1b000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1bb7fe640 1 -- 172.21.15.7:0/4225840817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff1b0017070 con 0x7ff1bc074e30 2024-09-21T04:53:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1bb7fe640 1 -- 172.21.15.7:0/4225840817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1b000de00 con 0x7ff1bc074e30 2024-09-21T04:53:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1bc199ce0 con 0x7ff1bc074e30 2024-09-21T04:53:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1bb7fe640 1 -- 172.21.15.7:0/4225840817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff1b0004bb0 con 0x7ff1bc074e30 2024-09-21T04:53:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.545+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1bc19a270 con 0x7ff1bc074e30 2024-09-21T04:53:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.546+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1bc075c50 con 0x7ff1bc074e30 2024-09-21T04:53:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.547+0000 7ff1bb7fe640 1 -- 172.21.15.7:0/4225840817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff1b0004e60 con 0x7ff1bc074e30 2024-09-21T04:53:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.547+0000 7ff1bb7fe640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff1a407af30 0x7ff1a407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:53:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.548+0000 7ff1bb7fe640 1 -- 172.21.15.7:0/4225840817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff1b009dcb0 con 0x7ff1bc074e30 2024-09-21T04:53:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.548+0000 7ff1c23ff640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff1a407af30 0x7ff1a407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:53:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.548+0000 7ff1c23ff640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff1a407af30 0x7ff1a407d3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4000c00 tx=0x7ff1b4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:53:46.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.552+0000 7ff1bb7fe640 1 -- 172.21.15.7:0/4225840817 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff1b0069250 con 0x7ff1bc074e30 2024-09-21T04:53:46.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.816+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7ff1bc108370 con 0x7ff1bc074e30 2024-09-21T04:53:46.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.816+0000 7ff1bb7fe640 1 -- 172.21.15.7:0/4225840817 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7ff1b006e100 con 0x7ff1bc074e30 2024-09-21T04:53:46.820 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:53:46.821 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:53:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:53:46.821 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:53:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:53:46.821 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:53:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:53:46.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff1a407af30 msgr2=0x7ff1a407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff1a407af30 0x7ff1a407d3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4000c00 tx=0x7ff1b4031400 comp rx=0 tx=0).stop 2024-09-21T04:53:46.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074e30 msgr2=0x7ff1bc19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:53:46.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074e30 0x7ff1bc19fd70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000d6f0 tx=0x7ff1b000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 shutdown_connections 2024-09-21T04:53:46.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff1a407af30 0x7ff1a407d3f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1bc074e30 0x7ff1bc19fd70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 --2- 172.21.15.7:0/4225840817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1bc074510 0x7ff1bc19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:53:46.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 >> 172.21.15.7:0/4225840817 conn(0x7ff1bc0fd600 msgr2=0x7ff1bc10b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:53:46.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 shutdown_connections 2024-09-21T04:53:46.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:53:46.819+0000 7ff1c4663640 1 -- 172.21.15.7:0/4225840817 wait complete. 2024-09-21T04:53:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:47 smithi053 ceph-mon[80176]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:47 smithi053 ceph-mon[80176]: pgmap v570: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:47 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1638105515' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:53:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:47 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4225840817' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:53:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:47 smithi007 ceph-mon[108238]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:47 smithi007 ceph-mon[108238]: pgmap v570: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:47 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1638105515' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:53:47.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:47 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4225840817' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:53:48.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:48 smithi007 ceph-mon[108238]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:48.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:48 smithi053 ceph-mon[80176]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:53:49.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:49 smithi053 ceph-mon[80176]: pgmap v571: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:49 smithi007 ceph-mon[108238]: pgmap v571: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:51.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:51 smithi053 ceph-mon[80176]: pgmap v572: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:51 smithi007 ceph-mon[108238]: pgmap v572: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:53 smithi007 ceph-mon[108238]: pgmap v573: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:53 smithi053 ceph-mon[80176]: pgmap v573: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:55 smithi053 ceph-mon[80176]: pgmap v574: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:55 smithi007 ceph-mon[108238]: pgmap v574: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:57 smithi053 ceph-mon[80176]: pgmap v575: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:57 smithi007 ceph-mon[108238]: pgmap v575: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:59 smithi007 ceph-mon[108238]: pgmap v576: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:53:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:53:59.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:59 smithi053 ceph-mon[80176]: pgmap v576: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:53:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:53:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:54:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:01 smithi053 ceph-mon[80176]: pgmap v577: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:01 smithi007 ceph-mon[108238]: pgmap v577: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:03 smithi053 ceph-mon[80176]: pgmap v578: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:03 smithi007 ceph-mon[108238]: pgmap v578: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:05 smithi007 ceph-mon[108238]: pgmap v579: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:05 smithi053 ceph-mon[80176]: pgmap v579: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:07 smithi053 ceph-mon[80176]: pgmap v580: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:07 smithi007 ceph-mon[108238]: pgmap v580: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:09 smithi053 ceph-mon[80176]: pgmap v581: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:09 smithi007 ceph-mon[108238]: pgmap v581: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:11 smithi007 ceph-mon[108238]: pgmap v582: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:11 smithi053 ceph-mon[80176]: pgmap v582: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:13 smithi053 ceph-mon[80176]: pgmap v583: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:13 smithi007 ceph-mon[108238]: pgmap v583: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:54:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:54:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:15 smithi053 ceph-mon[80176]: pgmap v584: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:15 smithi007 ceph-mon[108238]: pgmap v584: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:16.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.917+0000 7f1499864640 1 -- 172.21.15.7:0/1007313505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1494102ac0 msgr2=0x7f1494102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:16.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.917+0000 7f1499864640 1 --2- 172.21.15.7:0/1007313505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1494102ac0 0x7f1494102ea0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1484009880 tx=0x7f148402a530 comp rx=0 tx=0).stop 2024-09-21T04:54:16.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.917+0000 7f1499864640 1 -- 172.21.15.7:0/1007313505 shutdown_connections 2024-09-21T04:54:16.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.917+0000 7f1499864640 1 --2- 172.21.15.7:0/1007313505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14941033e0 0x7f149410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:16.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.917+0000 7f1499864640 1 --2- 172.21.15.7:0/1007313505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1494102ac0 0x7f1494102ea0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:16.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.917+0000 7f1499864640 1 -- 172.21.15.7:0/1007313505 >> 172.21.15.7:0/1007313505 conn(0x7f14940fc520 msgr2=0x7f14940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:16.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.917+0000 7f1499864640 1 -- 172.21.15.7:0/1007313505 shutdown_connections 2024-09-21T04:54:16.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.917+0000 7f1499864640 1 -- 172.21.15.7:0/1007313505 wait complete. 2024-09-21T04:54:16.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.918+0000 7f1499864640 1 Processor -- start 2024-09-21T04:54:16.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.918+0000 7f1499864640 1 -- start start 2024-09-21T04:54:16.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.918+0000 7f1499864640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494102ac0 0x7f149419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:16.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.918+0000 7f1499864640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14941033e0 0x7f149419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:16.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.918+0000 7f1499864640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f149410edf0 con 0x7f14941033e0 2024-09-21T04:54:16.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.918+0000 7f1499864640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f149410ec70 con 0x7f1494102ac0 2024-09-21T04:54:16.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f14927fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14941033e0 0x7f149419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:16.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f1492ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494102ac0 0x7f149419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f14927fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14941033e0 0x7f149419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44808/0 (socket says 172.21.15.7:44808) 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f14927fc640 1 -- 172.21.15.7:0/3739473182 learned_addr learned my addr 172.21.15.7:0/3739473182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f14927fc640 1 -- 172.21.15.7:0/3739473182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494102ac0 msgr2=0x7f149419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f14927fc640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494102ac0 0x7f149419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f14927fc640 1 -- 172.21.15.7:0/3739473182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14941977c0 con 0x7f14941033e0 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f1492ffd640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494102ac0 0x7f149419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f14927fc640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14941033e0 0x7f149419db80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f148800e970 tx=0x7f148800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f1498862640 1 -- 172.21.15.7:0/3739473182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f148800cac0 con 0x7f14941033e0 2024-09-21T04:54:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f1498862640 1 -- 172.21.15.7:0/3739473182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14880054e0 con 0x7f14941033e0 2024-09-21T04:54:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1494197ab0 con 0x7f14941033e0 2024-09-21T04:54:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.919+0000 7f1498862640 1 -- 172.21.15.7:0/3739473182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1488004dd0 con 0x7f14941033e0 2024-09-21T04:54:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.920+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1494198040 con 0x7f14941033e0 2024-09-21T04:54:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.921+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149406aa90 con 0x7f14941033e0 2024-09-21T04:54:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.921+0000 7f1498862640 1 -- 172.21.15.7:0/3739473182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1488005020 con 0x7f14941033e0 2024-09-21T04:54:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.922+0000 7f1498862640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147c07aed0 0x7f147c07d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.922+0000 7f1498862640 1 -- 172.21.15.7:0/3739473182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f148809e670 con 0x7f14941033e0 2024-09-21T04:54:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.922+0000 7f1492ffd640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147c07aed0 0x7f147c07d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.922+0000 7f1492ffd640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147c07aed0 0x7f147c07d390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f14840096c0 tx=0x7f1484031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:16.926+0000 7f1498862640 1 -- 172.21.15.7:0/3739473182 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1488069c10 con 0x7f14941033e0 2024-09-21T04:54:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.135+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f14941983b0 con 0x7f147c07aed0 2024-09-21T04:54:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.136+0000 7f1498862640 1 -- 172.21.15.7:0/3739473182 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f14941983b0 con 0x7f147c07aed0 2024-09-21T04:54:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147c07aed0 msgr2=0x7f147c07d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147c07aed0 0x7f147c07d390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f14840096c0 tx=0x7f1484031400 comp rx=0 tx=0).stop 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14941033e0 msgr2=0x7f149419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14941033e0 0x7f149419db80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f148800e970 tx=0x7f148800ee30 comp rx=0 tx=0).stop 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 shutdown_connections 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f147c07aed0 0x7f147c07d390 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14941033e0 0x7f149419db80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 --2- 172.21.15.7:0/3739473182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494102ac0 0x7f149419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 >> 172.21.15.7:0/3739473182 conn(0x7f14940fc520 msgr2=0x7f149410af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 shutdown_connections 2024-09-21T04:54:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.139+0000 7f1499864640 1 -- 172.21.15.7:0/3739473182 wait complete. 2024-09-21T04:54:17.150 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:54:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:17 smithi007 ceph-mon[108238]: pgmap v585: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.224+0000 7eff85347640 1 -- 172.21.15.7:0/2057000699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 msgr2=0x7eff8010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.224+0000 7eff85347640 1 --2- 172.21.15.7:0/2057000699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 0x7eff8010b8c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7eff74009880 tx=0x7eff7402a530 comp rx=0 tx=0).stop 2024-09-21T04:54:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.224+0000 7eff85347640 1 -- 172.21.15.7:0/2057000699 shutdown_connections 2024-09-21T04:54:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.224+0000 7eff85347640 1 --2- 172.21.15.7:0/2057000699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 0x7eff8010b8c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.224+0000 7eff85347640 1 --2- 172.21.15.7:0/2057000699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80102ab0 0x7eff80102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.224+0000 7eff85347640 1 -- 172.21.15.7:0/2057000699 >> 172.21.15.7:0/2057000699 conn(0x7eff800fc4d0 msgr2=0x7eff800fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.225+0000 7eff85347640 1 -- 172.21.15.7:0/2057000699 shutdown_connections 2024-09-21T04:54:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.225+0000 7eff85347640 1 -- 172.21.15.7:0/2057000699 wait complete. 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.225+0000 7eff85347640 1 Processor -- start 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.225+0000 7eff85347640 1 -- start start 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff85347640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80102ab0 0x7eff8019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff85347640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 0x7eff8019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff85347640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010ee40 con 0x7eff801033d0 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff85347640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff8010ecc0 con 0x7eff80102ab0 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 0x7eff8019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff7e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 0x7eff8019dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44832/0 (socket says 172.21.15.7:44832) 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff7e7fc640 1 -- 172.21.15.7:0/4223968337 learned_addr learned my addr 172.21.15.7:0/4223968337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff7effd640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80102ab0 0x7eff8019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff7e7fc640 1 -- 172.21.15.7:0/4223968337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80102ab0 msgr2=0x7eff8019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff7e7fc640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80102ab0 0x7eff8019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff7e7fc640 1 -- 172.21.15.7:0/4223968337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff80197870 con 0x7eff801033d0 2024-09-21T04:54:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.226+0000 7eff7e7fc640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 0x7eff8019dbf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7eff74031460 tx=0x7eff740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.227+0000 7eff67fff640 1 -- 172.21.15.7:0/4223968337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7eff7403f070 con 0x7eff801033d0 2024-09-21T04:54:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.227+0000 7eff67fff640 1 -- 172.21.15.7:0/4223968337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7eff74004ac0 con 0x7eff801033d0 2024-09-21T04:54:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.227+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff80197b00 con 0x7eff801033d0 2024-09-21T04:54:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.227+0000 7eff67fff640 1 -- 172.21.15.7:0/4223968337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7eff74035030 con 0x7eff801033d0 2024-09-21T04:54:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.227+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff80198030 con 0x7eff801033d0 2024-09-21T04:54:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.228+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8006ada0 con 0x7eff801033d0 2024-09-21T04:54:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.228+0000 7eff67fff640 1 -- 172.21.15.7:0/4223968337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7eff74042070 con 0x7eff801033d0 2024-09-21T04:54:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.229+0000 7eff67fff640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff6807af80 0x7eff6807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.229+0000 7eff7effd640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff6807af80 0x7eff6807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.229+0000 7eff67fff640 1 -- 172.21.15.7:0/4223968337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7eff740bea50 con 0x7eff801033d0 2024-09-21T04:54:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.230+0000 7eff7effd640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff6807af80 0x7eff6807d440 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7eff801989b0 tx=0x7eff70009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.234+0000 7eff67fff640 1 -- 172.21.15.7:0/4223968337 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff74089fd0 con 0x7eff801033d0 2024-09-21T04:54:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.438+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7eff80198da0 con 0x7eff6807af80 2024-09-21T04:54:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.439+0000 7eff67fff640 1 -- 172.21.15.7:0/4223968337 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7eff80198da0 con 0x7eff6807af80 2024-09-21T04:54:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:17 smithi053 ceph-mon[80176]: pgmap v585: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff6807af80 msgr2=0x7eff6807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff6807af80 0x7eff6807d440 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7eff801989b0 tx=0x7eff70009110 comp rx=0 tx=0).stop 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 msgr2=0x7eff8019dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 0x7eff8019dbf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7eff74031460 tx=0x7eff740046c0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 shutdown_connections 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff6807af80 0x7eff6807d440 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff801033d0 0x7eff8019dbf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 --2- 172.21.15.7:0/4223968337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80102ab0 0x7eff8019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 >> 172.21.15.7:0/4223968337 conn(0x7eff800fc4d0 msgr2=0x7eff80108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 shutdown_connections 2024-09-21T04:54:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.441+0000 7eff85347640 1 -- 172.21.15.7:0/4223968337 wait complete. 2024-09-21T04:54:17.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.525+0000 7fa1acc31640 1 -- 172.21.15.7:0/3772196544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8106800 msgr2=0x7fa1a8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.525+0000 7fa1acc31640 1 --2- 172.21.15.7:0/3772196544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8106800 0x7fa1a8106be0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa198009880 tx=0x7fa19802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.526+0000 7fa1acc31640 1 -- 172.21.15.7:0/3772196544 shutdown_connections 2024-09-21T04:54:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.526+0000 7fa1acc31640 1 --2- 172.21.15.7:0/3772196544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1a8100800 0x7fa1a8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.526+0000 7fa1acc31640 1 --2- 172.21.15.7:0/3772196544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8106800 0x7fa1a8106be0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.526+0000 7fa1acc31640 1 -- 172.21.15.7:0/3772196544 >> 172.21.15.7:0/3772196544 conn(0x7fa1a80fc520 msgr2=0x7fa1a80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.526+0000 7fa1acc31640 1 -- 172.21.15.7:0/3772196544 shutdown_connections 2024-09-21T04:54:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.526+0000 7fa1acc31640 1 -- 172.21.15.7:0/3772196544 wait complete. 2024-09-21T04:54:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.526+0000 7fa1acc31640 1 Processor -- start 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.526+0000 7fa1acc31640 1 -- start start 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1acc31640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8100800 0x7fa1a819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1acc31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1a8106800 0x7fa1a819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1acc31640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810dcb0 con 0x7fa1a8100800 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1acc31640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a810db30 con 0x7fa1a8106800 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1a659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8100800 0x7fa1a819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1a659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8100800 0x7fa1a819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44850/0 (socket says 172.21.15.7:44850) 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1a659c640 1 -- 172.21.15.7:0/1570033192 learned_addr learned my addr 172.21.15.7:0/1570033192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1a5d9b640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1a8106800 0x7fa1a819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1a659c640 1 -- 172.21.15.7:0/1570033192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1a8106800 msgr2=0x7fa1a819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1a659c640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1a8106800 0x7fa1a819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1a659c640 1 -- 172.21.15.7:0/1570033192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a81978f0 con 0x7fa1a8100800 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.527+0000 7fa1a659c640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8100800 0x7fa1a819d730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa198031460 tx=0x7fa1980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:17.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.528+0000 7fa1977fe640 1 -- 172.21.15.7:0/1570033192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa19803f070 con 0x7fa1a8100800 2024-09-21T04:54:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.528+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a8197b80 con 0x7fa1a8100800 2024-09-21T04:54:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.528+0000 7fa1977fe640 1 -- 172.21.15.7:0/1570033192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa198004cf0 con 0x7fa1a8100800 2024-09-21T04:54:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.528+0000 7fa1977fe640 1 -- 172.21.15.7:0/1570033192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa198035030 con 0x7fa1a8100800 2024-09-21T04:54:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.528+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81980b0 con 0x7fa1a8100800 2024-09-21T04:54:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.529+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a8101f40 con 0x7fa1a8100800 2024-09-21T04:54:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.530+0000 7fa1977fe640 1 -- 172.21.15.7:0/1570033192 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa198042070 con 0x7fa1a8100800 2024-09-21T04:54:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.531+0000 7fa1977fe640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa19007af80 0x7fa19007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.531+0000 7fa1977fe640 1 -- 172.21.15.7:0/1570033192 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa1980beae0 con 0x7fa1a8100800 2024-09-21T04:54:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.531+0000 7fa1a5d9b640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa19007af80 0x7fa19007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.531+0000 7fa1a5d9b640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa19007af80 0x7fa19007d440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80ffbf0 tx=0x7fa19c00a4c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.536+0000 7fa1977fe640 1 -- 172.21.15.7:0/1570033192 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa19808a060 con 0x7fa1a8100800 2024-09-21T04:54:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.733+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa1a8198360 con 0x7fa19007af80 2024-09-21T04:54:17.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.743+0000 7fa1977fe640 1 -- 172.21.15.7:0/1570033192 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fa1a8198360 con 0x7fa19007af80 2024-09-21T04:54:17.746 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:54:17.746 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (16m) 10m ago 27m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:54:17.746 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (19m) 10m ago 27m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:54:17.747 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (19m) 10m ago 26m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:54:17.747 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (14m) 10m ago 27m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:54:17.747 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (14m) 10m ago 26m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:54:17.747 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (15m) 10m ago 27m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:54:17.747 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (10m) 10m ago 21m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:54:17.747 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (10m) 10m ago 21m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:54:17.748 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (21m) 10m ago 21m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:54:17.748 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (10m) 10m ago 21m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:54:17.748 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (18m) 10m ago 28m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:54:17.748 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (17m) 10m ago 26m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:54:17.748 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (14m) 10m ago 28m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:54:17.748 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (14m) 10m ago 26m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:54:17.748 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (17m) 10m ago 27m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:54:17.749 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (17m) 10m ago 26m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:54:17.749 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (14m) 10m ago 25m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:54:17.749 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (13m) 10m ago 24m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:54:17.749 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (12m) 10m ago 24m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:54:17.749 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (12m) 10m ago 23m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:54:17.749 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (11m) 10m ago 23m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:54:17.749 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (11m) 10m ago 22m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:54:17.750 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (16m) 10m ago 26m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:54:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa19007af80 msgr2=0x7fa19007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa19007af80 0x7fa19007d440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80ffbf0 tx=0x7fa19c00a4c0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8100800 msgr2=0x7fa1a819d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8100800 0x7fa1a819d730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa198031460 tx=0x7fa1980048f0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 shutdown_connections 2024-09-21T04:54:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa19007af80 0x7fa19007d440 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1a8106800 0x7fa1a819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 --2- 172.21.15.7:0/1570033192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a8100800 0x7fa1a819d730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 >> 172.21.15.7:0/1570033192 conn(0x7fa1a80fc520 msgr2=0x7fa1a806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 shutdown_connections 2024-09-21T04:54:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.745+0000 7fa1acc31640 1 -- 172.21.15.7:0/1570033192 wait complete. 2024-09-21T04:54:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.828+0000 7ff90d7dd640 1 -- 172.21.15.7:0/3485921285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908100800 msgr2=0x7ff908100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.828+0000 7ff90d7dd640 1 --2- 172.21.15.7:0/3485921285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908100800 0x7ff908100c60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009880 tx=0x7ff8fc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.828+0000 7ff90d7dd640 1 -- 172.21.15.7:0/3485921285 shutdown_connections 2024-09-21T04:54:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.828+0000 7ff90d7dd640 1 --2- 172.21.15.7:0/3485921285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908100800 0x7ff908100c60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.828+0000 7ff90d7dd640 1 --2- 172.21.15.7:0/3485921285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff908106800 0x7ff908106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.828+0000 7ff90d7dd640 1 -- 172.21.15.7:0/3485921285 >> 172.21.15.7:0/3485921285 conn(0x7ff9080fc520 msgr2=0x7ff9080fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.828+0000 7ff90d7dd640 1 -- 172.21.15.7:0/3485921285 shutdown_connections 2024-09-21T04:54:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.829+0000 7ff90d7dd640 1 -- 172.21.15.7:0/3485921285 wait complete. 2024-09-21T04:54:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.829+0000 7ff90d7dd640 1 Processor -- start 2024-09-21T04:54:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.829+0000 7ff90d7dd640 1 -- start start 2024-09-21T04:54:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff90d7dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff908100800 0x7ff9080fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff90d7dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908106800 0x7ff9080fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff90d7dd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff90810dcb0 con 0x7ff908106800 2024-09-21T04:54:17.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff90d7dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff90810db30 con 0x7ff908100800 2024-09-21T04:54:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff9067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908106800 0x7ff9080fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff9067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908106800 0x7ff9080fd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44868/0 (socket says 172.21.15.7:44868) 2024-09-21T04:54:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff906ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff908100800 0x7ff9080fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff9067fc640 1 -- 172.21.15.7:0/1356534041 learned_addr learned my addr 172.21.15.7:0/1356534041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff9067fc640 1 -- 172.21.15.7:0/1356534041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff908100800 msgr2=0x7ff9080fcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff9067fc640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff908100800 0x7ff9080fcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff9067fc640 1 -- 172.21.15.7:0/1356534041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9080fd6f0 con 0x7ff908106800 2024-09-21T04:54:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.830+0000 7ff9067fc640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908106800 0x7ff9080fd1b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc031460 tx=0x7ff8fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:17.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.831+0000 7ff8effff640 1 -- 172.21.15.7:0/1356534041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff8fc03f070 con 0x7ff908106800 2024-09-21T04:54:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.831+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9080fd980 con 0x7ff908106800 2024-09-21T04:54:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.831+0000 7ff8effff640 1 -- 172.21.15.7:0/1356534041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8fc0035f0 con 0x7ff908106800 2024-09-21T04:54:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.831+0000 7ff8effff640 1 -- 172.21.15.7:0/1356534041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff8fc035030 con 0x7ff908106800 2024-09-21T04:54:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.831+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9081a3db0 con 0x7ff908106800 2024-09-21T04:54:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.832+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff908101f40 con 0x7ff908106800 2024-09-21T04:54:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.832+0000 7ff8effff640 1 -- 172.21.15.7:0/1356534041 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff8fc042070 con 0x7ff908106800 2024-09-21T04:54:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.833+0000 7ff8effff640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8f007af30 0x7ff8f007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.833+0000 7ff8effff640 1 -- 172.21.15.7:0/1356534041 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff8fc0be9b0 con 0x7ff908106800 2024-09-21T04:54:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.833+0000 7ff906ffd640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8f007af30 0x7ff8f007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.834+0000 7ff906ffd640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8f007af30 0x7ff8f007d3f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800bbe0 tx=0x7ff8f800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:17.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:17.838+0000 7ff8effff640 1 -- 172.21.15.7:0/1356534041 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8fc089f50 con 0x7ff908106800 2024-09-21T04:54:18.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.099+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ff908109da0 con 0x7ff908106800 2024-09-21T04:54:18.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.100+0000 7ff8effff640 1 -- 172.21.15.7:0/1356534041 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7ff8fc08c470 con 0x7ff908106800 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:54:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:54:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:54:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:54:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:54:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:54:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:54:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:54:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:54:18.104 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8f007af30 msgr2=0x7ff8f007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8f007af30 0x7ff8f007d3f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800bbe0 tx=0x7ff8f800b390 comp rx=0 tx=0).stop 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908106800 msgr2=0x7ff9080fd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908106800 0x7ff9080fd1b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc031460 tx=0x7ff8fc0045e0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 shutdown_connections 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8f007af30 0x7ff8f007d3f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff908106800 0x7ff9080fd1b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 --2- 172.21.15.7:0/1356534041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff908100800 0x7ff9080fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 >> 172.21.15.7:0/1356534041 conn(0x7ff9080fc520 msgr2=0x7ff90806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 shutdown_connections 2024-09-21T04:54:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.102+0000 7ff90d7dd640 1 -- 172.21.15.7:0/1356534041 wait complete. 2024-09-21T04:54:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.187+0000 7f5b18d09640 1 -- 172.21.15.7:0/139992616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 msgr2=0x7f5b14107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.187+0000 7f5b18d09640 1 --2- 172.21.15.7:0/139992616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 0x7f5b14107ce0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009880 tx=0x7f5b0402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.187+0000 7f5b18d09640 1 -- 172.21.15.7:0/139992616 shutdown_connections 2024-09-21T04:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.187+0000 7f5b18d09640 1 --2- 172.21.15.7:0/139992616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b14101900 0x7f5b14101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.187+0000 7f5b18d09640 1 --2- 172.21.15.7:0/139992616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 0x7f5b14107ce0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.187+0000 7f5b18d09640 1 -- 172.21.15.7:0/139992616 >> 172.21.15.7:0/139992616 conn(0x7f5b140fd600 msgr2=0x7f5b140ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.187+0000 7f5b18d09640 1 -- 172.21.15.7:0/139992616 shutdown_connections 2024-09-21T04:54:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.187+0000 7f5b18d09640 1 -- 172.21.15.7:0/139992616 wait complete. 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.188+0000 7f5b18d09640 1 Processor -- start 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.188+0000 7f5b18d09640 1 -- start start 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.188+0000 7f5b18d09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b14101900 0x7f5b1419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.188+0000 7f5b18d09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 0x7f5b1419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.188+0000 7f5b18d09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410edb0 con 0x7f5b14107900 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.188+0000 7f5b18d09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1410ec30 con 0x7f5b14101900 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.188+0000 7f5b11d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 0x7f5b1419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.188+0000 7f5b11d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 0x7f5b1419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44880/0 (socket says 172.21.15.7:44880) 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b11d9b640 1 -- 172.21.15.7:0/968063301 learned_addr learned my addr 172.21.15.7:0/968063301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b1259c640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b14101900 0x7f5b1419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b11d9b640 1 -- 172.21.15.7:0/968063301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b14101900 msgr2=0x7f5b1419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b11d9b640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b14101900 0x7f5b1419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b11d9b640 1 -- 172.21.15.7:0/968063301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b141977c0 con 0x7f5b14107900 2024-09-21T04:54:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b1259c640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b14101900 0x7f5b1419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b11d9b640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 0x7f5b1419db80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c810 tx=0x7f5b0800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b037fe640 1 -- 172.21.15.7:0/968063301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b08004020 con 0x7f5b14107900 2024-09-21T04:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b037fe640 1 -- 172.21.15.7:0/968063301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b080041c0 con 0x7f5b14107900 2024-09-21T04:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b037fe640 1 -- 172.21.15.7:0/968063301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b080047a0 con 0x7f5b14107900 2024-09-21T04:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b14197ab0 con 0x7f5b14107900 2024-09-21T04:54:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.189+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b14198040 con 0x7f5b14107900 2024-09-21T04:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.190+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b14103040 con 0x7f5b14107900 2024-09-21T04:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.191+0000 7f5b037fe640 1 -- 172.21.15.7:0/968063301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5b080079b0 con 0x7f5b14107900 2024-09-21T04:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.191+0000 7f5b037fe640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5afc07af30 0x7f5afc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.192+0000 7f5b037fe640 1 -- 172.21.15.7:0/968063301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5b0809df60 con 0x7f5b14107900 2024-09-21T04:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.192+0000 7f5b1259c640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5afc07af30 0x7f5afc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.192+0000 7f5b1259c640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5afc07af30 0x7f5afc07d3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5b04031460 tx=0x7f5b04009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.196+0000 7f5b037fe640 1 -- 172.21.15.7:0/968063301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b08014030 con 0x7f5b14107900 2024-09-21T04:54:18.365 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:18 smithi007 ceph-mon[108238]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:18.365 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:18 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1356534041' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:54:18.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.427+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5b14104b10 con 0x7f5b14107900 2024-09-21T04:54:18.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.428+0000 7f5b037fe640 1 -- 172.21.15.7:0/968063301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f5b08069500 con 0x7f5b14107900 2024-09-21T04:54:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:54:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:54:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:54:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:54:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:54:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:54:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:54:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:54:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:54:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5afc07af30 msgr2=0x7f5afc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5afc07af30 0x7f5afc07d3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5b04031460 tx=0x7f5b04009560 comp rx=0 tx=0).stop 2024-09-21T04:54:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 msgr2=0x7f5b1419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 0x7f5b1419db80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800c810 tx=0x7f5b0800ccd0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 shutdown_connections 2024-09-21T04:54:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5afc07af30 0x7f5afc07d3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b14107900 0x7f5b1419db80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 --2- 172.21.15.7:0/968063301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b14101900 0x7f5b1419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 >> 172.21.15.7:0/968063301 conn(0x7f5b140fd600 msgr2=0x7f5b140ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 shutdown_connections 2024-09-21T04:54:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.430+0000 7f5b18d09640 1 -- 172.21.15.7:0/968063301 wait complete. 2024-09-21T04:54:18.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:18 smithi053 ceph-mon[80176]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:18.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:18 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1356534041' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:54:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.515+0000 7f4335480640 1 -- 172.21.15.7:0/1251710693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 msgr2=0x7f4330106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.515+0000 7f4335480640 1 --2- 172.21.15.7:0/1251710693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 0x7f4330106bd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4320009c20 tx=0x7f432002a980 comp rx=0 tx=0).stop 2024-09-21T04:54:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.516+0000 7f4335480640 1 -- 172.21.15.7:0/1251710693 shutdown_connections 2024-09-21T04:54:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.516+0000 7f4335480640 1 --2- 172.21.15.7:0/1251710693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43301007f0 0x7f4330100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.516+0000 7f4335480640 1 --2- 172.21.15.7:0/1251710693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 0x7f4330106bd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.516+0000 7f4335480640 1 -- 172.21.15.7:0/1251710693 >> 172.21.15.7:0/1251710693 conn(0x7f43300fc550 msgr2=0x7f43300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.516+0000 7f4335480640 1 -- 172.21.15.7:0/1251710693 shutdown_connections 2024-09-21T04:54:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.516+0000 7f4335480640 1 -- 172.21.15.7:0/1251710693 wait complete. 2024-09-21T04:54:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.516+0000 7f4335480640 1 Processor -- start 2024-09-21T04:54:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.516+0000 7f4335480640 1 -- start start 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f4335480640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43301007f0 0x7f433010b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f4335480640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 0x7f433010bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f4335480640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f433010edc0 con 0x7f43301067f0 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f4335480640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f433010ec40 con 0x7f43301007f0 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f432e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 0x7f433010bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f432e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 0x7f433010bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44900/0 (socket says 172.21.15.7:44900) 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f432e7fc640 1 -- 172.21.15.7:0/4044457731 learned_addr learned my addr 172.21.15.7:0/4044457731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f432effd640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43301007f0 0x7f433010b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f432e7fc640 1 -- 172.21.15.7:0/4044457731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43301007f0 msgr2=0x7f433010b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f432e7fc640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43301007f0 0x7f433010b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f432e7fc640 1 -- 172.21.15.7:0/4044457731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433010c200 con 0x7f43301067f0 2024-09-21T04:54:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.517+0000 7f432e7fc640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 0x7f433010bcc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f432400d6f0 tx=0x7f432400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.518+0000 7f4317fff640 1 -- 172.21.15.7:0/4044457731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4324017070 con 0x7f43301067f0 2024-09-21T04:54:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.518+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433010c4f0 con 0x7f43301067f0 2024-09-21T04:54:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.518+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43301a3e50 con 0x7f43301067f0 2024-09-21T04:54:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.518+0000 7f4317fff640 1 -- 172.21.15.7:0/4044457731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f432400de00 con 0x7f43301067f0 2024-09-21T04:54:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.518+0000 7f4317fff640 1 -- 172.21.15.7:0/4044457731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4324004bb0 con 0x7f43301067f0 2024-09-21T04:54:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.519+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c005290 con 0x7f43301067f0 2024-09-21T04:54:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.520+0000 7f4317fff640 1 -- 172.21.15.7:0/4044457731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4324009a30 con 0x7f43301067f0 2024-09-21T04:54:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.520+0000 7f4317fff640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f431807af80 0x7f431807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.520+0000 7f4317fff640 1 -- 172.21.15.7:0/4044457731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f432409da60 con 0x7f43301067f0 2024-09-21T04:54:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.520+0000 7f432effd640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f431807af80 0x7f431807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.521+0000 7f432effd640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f431807af80 0x7f431807d440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4320009430 tx=0x7f4320009360 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.525+0000 7f4317fff640 1 -- 172.21.15.7:0/4044457731 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4324069080 con 0x7f43301067f0 2024-09-21T04:54:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.730+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f431c000bc0 con 0x7f431807af80 2024-09-21T04:54:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.731+0000 7f4317fff640 1 -- 172.21.15.7:0/4044457731 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f431c000bc0 con 0x7f431807af80 2024-09-21T04:54:18.734 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:54:18.734 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:54:18.734 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:54:18.734 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:54:18.735 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:54:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f431807af80 msgr2=0x7f431807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f431807af80 0x7f431807d440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4320009430 tx=0x7f4320009360 comp rx=0 tx=0).stop 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 msgr2=0x7f433010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 0x7f433010bcc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f432400d6f0 tx=0x7f432400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 shutdown_connections 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f431807af80 0x7f431807d440 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43301067f0 0x7f433010bcc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 --2- 172.21.15.7:0/4044457731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43301007f0 0x7f433010b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 >> 172.21.15.7:0/4044457731 conn(0x7f43300fc550 msgr2=0x7f43300fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 shutdown_connections 2024-09-21T04:54:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.734+0000 7f4335480640 1 -- 172.21.15.7:0/4044457731 wait complete. 2024-09-21T04:54:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.818+0000 7f03b64a9640 1 -- 172.21.15.7:0/3666150622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0107900 msgr2=0x7f03b0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.818+0000 7f03b64a9640 1 --2- 172.21.15.7:0/3666150622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0107900 0x7f03b0107ce0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009880 tx=0x7f03a002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.818+0000 7f03b64a9640 1 -- 172.21.15.7:0/3666150622 shutdown_connections 2024-09-21T04:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.818+0000 7f03b64a9640 1 --2- 172.21.15.7:0/3666150622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b0101900 0x7f03b0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.818+0000 7f03b64a9640 1 --2- 172.21.15.7:0/3666150622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0107900 0x7f03b0107ce0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.818+0000 7f03b64a9640 1 -- 172.21.15.7:0/3666150622 >> 172.21.15.7:0/3666150622 conn(0x7f03b00fd600 msgr2=0x7f03b00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.818+0000 7f03b64a9640 1 -- 172.21.15.7:0/3666150622 shutdown_connections 2024-09-21T04:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.818+0000 7f03b64a9640 1 -- 172.21.15.7:0/3666150622 wait complete. 2024-09-21T04:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03b64a9640 1 Processor -- start 2024-09-21T04:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03b64a9640 1 -- start start 2024-09-21T04:54:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03b64a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0101900 0x7f03b019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03b64a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b0107900 0x7f03b019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03b64a9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03b010edb0 con 0x7f03b0101900 2024-09-21T04:54:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03b64a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03b010ec30 con 0x7f03b0107900 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0101900 0x7f03b019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0101900 0x7f03b019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44924/0 (socket says 172.21.15.7:44924) 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.819+0000 7f03affff640 1 -- 172.21.15.7:0/194294012 learned_addr learned my addr 172.21.15.7:0/194294012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03af7fe640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b0107900 0x7f03b019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03affff640 1 -- 172.21.15.7:0/194294012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b0107900 msgr2=0x7f03b019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03affff640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b0107900 0x7f03b019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03affff640 1 -- 172.21.15.7:0/194294012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b01a03e0 con 0x7f03b0101900 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03affff640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0101900 0x7f03b019f7f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03a0031460 tx=0x7f03a00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03ad7fa640 1 -- 172.21.15.7:0/194294012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f03a003f070 con 0x7f03b0101900 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03ad7fa640 1 -- 172.21.15.7:0/194294012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03a00044d0 con 0x7f03b0101900 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b0199ac0 con 0x7f03b0101900 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03ad7fa640 1 -- 172.21.15.7:0/194294012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f03a0035030 con 0x7f03b0101900 2024-09-21T04:54:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.820+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b0199fc0 con 0x7f03b0101900 2024-09-21T04:54:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.821+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b0103040 con 0x7f03b0101900 2024-09-21T04:54:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.822+0000 7f03ad7fa640 1 -- 172.21.15.7:0/194294012 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f03a0042070 con 0x7f03b0101900 2024-09-21T04:54:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.822+0000 7f03ad7fa640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f039807af40 0x7f039807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.823+0000 7f03af7fe640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f039807af40 0x7f039807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.823+0000 7f03ad7fa640 1 -- 172.21.15.7:0/194294012 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f03a00beb30 con 0x7f03b0101900 2024-09-21T04:54:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.823+0000 7f03af7fe640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f039807af40 0x7f039807d400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f03b0076510 tx=0x7f03a400a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:18.827+0000 7f03ad7fa640 1 -- 172.21.15.7:0/194294012 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03a0004cf0 con 0x7f03b0101900 2024-09-21T04:54:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.094+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f03b019a310 con 0x7f03b0101900 2024-09-21T04:54:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.094+0000 7f03ad7fa640 1 -- 172.21.15.7:0/194294012 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f03a008a0d0 con 0x7f03b0101900 2024-09-21T04:54:19.097 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:54:19.097 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:54:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:54:19.097 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:54:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:54:19.098 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:54:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:54:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f039807af40 msgr2=0x7f039807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f039807af40 0x7f039807d400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f03b0076510 tx=0x7f03a400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:54:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0101900 msgr2=0x7f03b019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0101900 0x7f03b019f7f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f03a0031460 tx=0x7f03a00048f0 comp rx=0 tx=0).stop 2024-09-21T04:54:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 shutdown_connections 2024-09-21T04:54:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f039807af40 0x7f039807d400 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b0107900 0x7f03b019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 --2- 172.21.15.7:0/194294012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03b0101900 0x7f03b019f7f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.096+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 >> 172.21.15.7:0/194294012 conn(0x7f03b00fd600 msgr2=0x7f03b00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.097+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 shutdown_connections 2024-09-21T04:54:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:19.097+0000 7f03b64a9640 1 -- 172.21.15.7:0/194294012 wait complete. 2024-09-21T04:54:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:19 smithi053 ceph-mon[80176]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:19 smithi053 ceph-mon[80176]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:19 smithi053 ceph-mon[80176]: pgmap v586: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:19 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/968063301' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:54:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:19 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/194294012' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:54:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:19 smithi007 ceph-mon[108238]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:19 smithi007 ceph-mon[108238]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:19 smithi007 ceph-mon[108238]: pgmap v586: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:19 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/968063301' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:54:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:19 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/194294012' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:54:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:20 smithi007 ceph-mon[108238]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:20.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:20 smithi053 ceph-mon[80176]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:21 smithi053 ceph-mon[80176]: pgmap v587: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:21 smithi007 ceph-mon[108238]: pgmap v587: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:23 smithi007 ceph-mon[108238]: pgmap v588: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:23 smithi053 ceph-mon[80176]: pgmap v588: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:25 smithi053 ceph-mon[80176]: pgmap v589: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:25 smithi007 ceph-mon[108238]: pgmap v589: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:27 smithi053 ceph-mon[80176]: pgmap v590: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:27 smithi007 ceph-mon[108238]: pgmap v590: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:29 smithi007 ceph-mon[108238]: pgmap v591: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:54:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:29 smithi053 ceph-mon[80176]: pgmap v591: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:54:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:31 smithi053 ceph-mon[80176]: pgmap v592: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:31 smithi007 ceph-mon[108238]: pgmap v592: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:33 smithi053 ceph-mon[80176]: pgmap v593: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:33 smithi007 ceph-mon[108238]: pgmap v593: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:35 smithi007 ceph-mon[108238]: pgmap v594: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:35 smithi053 ceph-mon[80176]: pgmap v594: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:37 smithi053 ceph-mon[80176]: pgmap v595: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:37 smithi007 ceph-mon[108238]: pgmap v595: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:39 smithi053 ceph-mon[80176]: pgmap v596: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:39 smithi007 ceph-mon[108238]: pgmap v596: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:41 smithi053 ceph-mon[80176]: pgmap v597: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:41 smithi007 ceph-mon[108238]: pgmap v597: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:43 smithi053 ceph-mon[80176]: pgmap v598: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:43 smithi007 ceph-mon[108238]: pgmap v598: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:54:44.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:54:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:45 smithi053 ceph-mon[80176]: pgmap v599: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:45 smithi007 ceph-mon[108238]: pgmap v599: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:47 smithi007 ceph-mon[108238]: pgmap v600: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:47 smithi053 ceph-mon[80176]: pgmap v600: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.194+0000 7f49d0e56640 1 -- 172.21.15.7:0/1459930769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 msgr2=0x7f49cc102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.194+0000 7f49d0e56640 1 --2- 172.21.15.7:0/1459930769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 0x7f49cc102f30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009880 tx=0x7f49bc02a530 comp rx=0 tx=0).stop 2024-09-21T04:54:49.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.194+0000 7f49d0e56640 1 -- 172.21.15.7:0/1459930769 shutdown_connections 2024-09-21T04:54:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.194+0000 7f49d0e56640 1 --2- 172.21.15.7:0/1459930769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49cc103470 0x7f49cc10b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.194+0000 7f49d0e56640 1 --2- 172.21.15.7:0/1459930769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 0x7f49cc102f30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.194+0000 7f49d0e56640 1 -- 172.21.15.7:0/1459930769 >> 172.21.15.7:0/1459930769 conn(0x7f49cc0fc550 msgr2=0x7f49cc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.194+0000 7f49d0e56640 1 -- 172.21.15.7:0/1459930769 shutdown_connections 2024-09-21T04:54:49.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.194+0000 7f49d0e56640 1 -- 172.21.15.7:0/1459930769 wait complete. 2024-09-21T04:54:49.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49d0e56640 1 Processor -- start 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49d0e56640 1 -- start start 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49d0e56640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 0x7f49cc19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49d0e56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49cc103470 0x7f49cc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49d0e56640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10edc0 con 0x7f49cc102b50 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49d0e56640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc10ec40 con 0x7f49cc103470 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 0x7f49cc19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 0x7f49cc19f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34746/0 (socket says 172.21.15.7:34746) 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.195+0000 7f49ca59c640 1 -- 172.21.15.7:0/627580050 learned_addr learned my addr 172.21.15.7:0/627580050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49c9d9b640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49cc103470 0x7f49cc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49ca59c640 1 -- 172.21.15.7:0/627580050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49cc103470 msgr2=0x7f49cc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49ca59c640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49cc103470 0x7f49cc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49ca59c640 1 -- 172.21.15.7:0/627580050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc1999c0 con 0x7f49cc102b50 2024-09-21T04:54:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49ca59c640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 0x7f49cc19f7b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49bc004f90 tx=0x7f49bc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49bb7fe640 1 -- 172.21.15.7:0/627580050 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49bc03f070 con 0x7f49cc102b50 2024-09-21T04:54:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49bb7fe640 1 -- 172.21.15.7:0/627580050 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49bc004c70 con 0x7f49cc102b50 2024-09-21T04:54:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc199c50 con 0x7f49cc102b50 2024-09-21T04:54:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49bb7fe640 1 -- 172.21.15.7:0/627580050 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49bc035030 con 0x7f49cc102b50 2024-09-21T04:54:49.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.196+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc19a180 con 0x7f49cc102b50 2024-09-21T04:54:49.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.197+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc06a4e0 con 0x7f49cc102b50 2024-09-21T04:54:49.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.198+0000 7f49bb7fe640 1 -- 172.21.15.7:0/627580050 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f49bc042070 con 0x7f49cc102b50 2024-09-21T04:54:49.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.199+0000 7f49bb7fe640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49b407af90 0x7f49b407d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.199+0000 7f49bb7fe640 1 -- 172.21.15.7:0/627580050 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f49bc0bea10 con 0x7f49cc102b50 2024-09-21T04:54:49.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.199+0000 7f49c9d9b640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49b407af90 0x7f49b407d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.199+0000 7f49c9d9b640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49b407af90 0x7f49b407d450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49cc102740 tx=0x7f49c000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:49.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.203+0000 7f49bb7fe640 1 -- 172.21.15.7:0/627580050 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49bc08a030 con 0x7f49cc102b50 2024-09-21T04:54:49.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.406+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f49cc19aea0 con 0x7f49b407af90 2024-09-21T04:54:49.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.407+0000 7f49bb7fe640 1 -- 172.21.15.7:0/627580050 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f49cc19aea0 con 0x7f49b407af90 2024-09-21T04:54:49.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49b407af90 msgr2=0x7f49b407d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49b407af90 0x7f49b407d450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49cc102740 tx=0x7f49c000b390 comp rx=0 tx=0).stop 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 msgr2=0x7f49cc19f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 0x7f49cc19f7b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49bc004f90 tx=0x7f49bc0048f0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 shutdown_connections 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49b407af90 0x7f49b407d450 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49cc103470 0x7f49cc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 --2- 172.21.15.7:0/627580050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc102b50 0x7f49cc19f7b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 >> 172.21.15.7:0/627580050 conn(0x7f49cc0fc550 msgr2=0x7f49cc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 shutdown_connections 2024-09-21T04:54:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.410+0000 7f49d0e56640 1 -- 172.21.15.7:0/627580050 wait complete. 2024-09-21T04:54:49.423 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:54:49.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:49 smithi053 ceph-mon[80176]: pgmap v601: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:49 smithi007 ceph-mon[108238]: pgmap v601: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.495+0000 7f4496a4f640 1 -- 172.21.15.7:0/829955118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 msgr2=0x7f4490107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.495+0000 7f4496a4f640 1 --2- 172.21.15.7:0/829955118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 0x7f4490107ce0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4480009880 tx=0x7f448002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.495+0000 7f4496a4f640 1 -- 172.21.15.7:0/829955118 shutdown_connections 2024-09-21T04:54:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.495+0000 7f4496a4f640 1 --2- 172.21.15.7:0/829955118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101900 0x7f4490101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.495+0000 7f4496a4f640 1 --2- 172.21.15.7:0/829955118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 0x7f4490107ce0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.495+0000 7f4496a4f640 1 -- 172.21.15.7:0/829955118 >> 172.21.15.7:0/829955118 conn(0x7f44900fd600 msgr2=0x7f44900ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.495+0000 7f4496a4f640 1 -- 172.21.15.7:0/829955118 shutdown_connections 2024-09-21T04:54:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.495+0000 7f4496a4f640 1 -- 172.21.15.7:0/829955118 wait complete. 2024-09-21T04:54:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f4496a4f640 1 Processor -- start 2024-09-21T04:54:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f4496a4f640 1 -- start start 2024-09-21T04:54:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f4496a4f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101900 0x7f449019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f4496a4f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 0x7f449019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f4496a4f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f449010ee30 con 0x7f4490107900 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f4496a4f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f449010ecb0 con 0x7f4490101900 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f448f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 0x7f449019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f448f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 0x7f449019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34766/0 (socket says 172.21.15.7:34766) 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.496+0000 7f448f7fe640 1 -- 172.21.15.7:0/1739837326 learned_addr learned my addr 172.21.15.7:0/1739837326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448ffff640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101900 0x7f449019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448f7fe640 1 -- 172.21.15.7:0/1739837326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101900 msgr2=0x7f449019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448f7fe640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101900 0x7f449019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448f7fe640 1 -- 172.21.15.7:0/1739837326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4490197810 con 0x7f4490107900 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448ffff640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101900 0x7f449019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448f7fe640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 0x7f449019dbd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f448400b420 tx=0x7f448400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448d7fa640 1 -- 172.21.15.7:0/1739837326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4484011630 con 0x7f4490107900 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448d7fa640 1 -- 172.21.15.7:0/1739837326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4484016070 con 0x7f4490107900 2024-09-21T04:54:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4490197b00 con 0x7f4490107900 2024-09-21T04:54:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f448d7fa640 1 -- 172.21.15.7:0/1739837326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4484010d40 con 0x7f4490107900 2024-09-21T04:54:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.497+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4490198090 con 0x7f4490107900 2024-09-21T04:54:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.498+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4490103040 con 0x7f4490107900 2024-09-21T04:54:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.499+0000 7f448d7fa640 1 -- 172.21.15.7:0/1739837326 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f448401b460 con 0x7f4490107900 2024-09-21T04:54:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.499+0000 7f448d7fa640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f447807af80 0x7f447807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.499+0000 7f448d7fa640 1 -- 172.21.15.7:0/1739837326 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f448409d750 con 0x7f4490107900 2024-09-21T04:54:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.499+0000 7f448ffff640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f447807af80 0x7f447807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.500+0000 7f448ffff640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f447807af80 0x7f447807d440 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4480006010 tx=0x7f44800095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.504+0000 7f448d7fa640 1 -- 172.21.15.7:0/1739837326 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4484021250 con 0x7f4490107900 2024-09-21T04:54:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.708+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4490198400 con 0x7f447807af80 2024-09-21T04:54:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.709+0000 7f448d7fa640 1 -- 172.21.15.7:0/1739837326 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f4490198400 con 0x7f447807af80 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f447807af80 msgr2=0x7f447807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f447807af80 0x7f447807d440 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4480006010 tx=0x7f44800095f0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 msgr2=0x7f449019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 0x7f449019dbd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f448400b420 tx=0x7f448400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 shutdown_connections 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f447807af80 0x7f447807d440 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4490107900 0x7f449019dbd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 --2- 172.21.15.7:0/1739837326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101900 0x7f449019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 >> 172.21.15.7:0/1739837326 conn(0x7f44900fd600 msgr2=0x7f44900fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 shutdown_connections 2024-09-21T04:54:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.711+0000 7f4496a4f640 1 -- 172.21.15.7:0/1739837326 wait complete. 2024-09-21T04:54:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.795+0000 7fdde9257640 1 -- 172.21.15.7:0/1474303330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4106800 msgr2=0x7fdde4106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.795+0000 7fdde9257640 1 --2- 172.21.15.7:0/1474303330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4106800 0x7fdde4106be0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009880 tx=0x7fddd402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.795+0000 7fdde9257640 1 -- 172.21.15.7:0/1474303330 shutdown_connections 2024-09-21T04:54:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.795+0000 7fdde9257640 1 --2- 172.21.15.7:0/1474303330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde4100800 0x7fdde4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.795+0000 7fdde9257640 1 --2- 172.21.15.7:0/1474303330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4106800 0x7fdde4106be0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.795+0000 7fdde9257640 1 -- 172.21.15.7:0/1474303330 >> 172.21.15.7:0/1474303330 conn(0x7fdde40fc520 msgr2=0x7fdde40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.795+0000 7fdde9257640 1 -- 172.21.15.7:0/1474303330 shutdown_connections 2024-09-21T04:54:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.795+0000 7fdde9257640 1 -- 172.21.15.7:0/1474303330 wait complete. 2024-09-21T04:54:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.796+0000 7fdde9257640 1 Processor -- start 2024-09-21T04:54:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.796+0000 7fdde9257640 1 -- start start 2024-09-21T04:54:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.796+0000 7fdde9257640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4100800 0x7fdde41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.796+0000 7fdde9257640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde4106800 0x7fdde4198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.796+0000 7fdde9257640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdde410dcd0 con 0x7fdde4100800 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.796+0000 7fdde9257640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdde410db50 con 0x7fdde4106800 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4100800 0x7fdde41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4100800 0x7fdde41987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34796/0 (socket says 172.21.15.7:34796) 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde2d9d640 1 -- 172.21.15.7:0/1360130150 learned_addr learned my addr 172.21.15.7:0/1360130150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdddbfff640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde4106800 0x7fdde4198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde2d9d640 1 -- 172.21.15.7:0/1360130150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde4106800 msgr2=0x7fdde4198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde2d9d640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde4106800 0x7fdde4198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde2d9d640 1 -- 172.21.15.7:0/1360130150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde4199360 con 0x7fdde4100800 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde2d9d640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4100800 0x7fdde41987d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fddd4031460 tx=0x7fddd40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde0d99640 1 -- 172.21.15.7:0/1360130150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fddd403f070 con 0x7fdde4100800 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde0d99640 1 -- 172.21.15.7:0/1360130150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fddd40035f0 con 0x7fdde4100800 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.797+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde41995f0 con 0x7fdde4100800 2024-09-21T04:54:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.798+0000 7fdde0d99640 1 -- 172.21.15.7:0/1360130150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fddd4035030 con 0x7fdde4100800 2024-09-21T04:54:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.798+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a1ba0 con 0x7fdde4100800 2024-09-21T04:54:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.799+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde4101f40 con 0x7fdde4100800 2024-09-21T04:54:49.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.799+0000 7fdde0d99640 1 -- 172.21.15.7:0/1360130150 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fddd4042070 con 0x7fdde4100800 2024-09-21T04:54:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.800+0000 7fdde0d99640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd007af80 0x7fddd007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.800+0000 7fdddbfff640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd007af80 0x7fddd007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.800+0000 7fdde0d99640 1 -- 172.21.15.7:0/1360130150 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fddd40be8c0 con 0x7fdde4100800 2024-09-21T04:54:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.801+0000 7fdddbfff640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd007af80 0x7fddd007d440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdde406ad40 tx=0x7fddcc00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:49.804+0000 7fdde0d99640 1 -- 172.21.15.7:0/1360130150 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fddd4089ee0 con 0x7fdde4100800 2024-09-21T04:54:50.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.003+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdde4062630 con 0x7fddd007af80 2024-09-21T04:54:50.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.013+0000 7fdde0d99640 1 -- 172.21.15.7:0/1360130150 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fdde4062630 con 0x7fddd007af80 2024-09-21T04:54:50.017 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:54:50.017 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (16m) 10m ago 27m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:54:50.017 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (19m) 10m ago 28m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:54:50.017 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (19m) 10m ago 27m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:54:50.017 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (15m) 10m ago 27m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (14m) 10m ago 27m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (16m) 10m ago 27m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (10m) 10m ago 22m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (11m) 10m ago 22m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (22m) 10m ago 22m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (10m) 10m ago 22m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (19m) 10m ago 29m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (18m) 10m ago 26m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (15m) 10m ago 29m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (15m) 10m ago 26m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (17m) 10m ago 27m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (17m) 10m ago 27m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (14m) 10m ago 25m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:54:50.018 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (13m) 10m ago 25m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:54:50.019 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (13m) 10m ago 24m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:54:50.019 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (12m) 10m ago 24m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:54:50.019 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (12m) 10m ago 23m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:54:50.019 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (11m) 10m ago 23m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:54:50.019 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (17m) 10m ago 27m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd007af80 msgr2=0x7fddd007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd007af80 0x7fddd007d440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdde406ad40 tx=0x7fddcc00b390 comp rx=0 tx=0).stop 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4100800 msgr2=0x7fdde41987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4100800 0x7fdde41987d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fddd4031460 tx=0x7fddd40045e0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 shutdown_connections 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd007af80 0x7fddd007d440 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdde4106800 0x7fdde4198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 --2- 172.21.15.7:0/1360130150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdde4100800 0x7fdde41987d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 >> 172.21.15.7:0/1360130150 conn(0x7fdde40fc520 msgr2=0x7fdde40fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 shutdown_connections 2024-09-21T04:54:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.015+0000 7fdde9257640 1 -- 172.21.15.7:0/1360130150 wait complete. 2024-09-21T04:54:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.099+0000 7fac40ee6640 1 -- 172.21.15.7:0/3980617498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c1033e0 msgr2=0x7fac3c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.099+0000 7fac40ee6640 1 --2- 172.21.15.7:0/3980617498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c1033e0 0x7fac3c10b8d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fac24009880 tx=0x7fac2402a530 comp rx=0 tx=0).stop 2024-09-21T04:54:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.099+0000 7fac40ee6640 1 -- 172.21.15.7:0/3980617498 shutdown_connections 2024-09-21T04:54:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.099+0000 7fac40ee6640 1 --2- 172.21.15.7:0/3980617498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c1033e0 0x7fac3c10b8d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.099+0000 7fac40ee6640 1 --2- 172.21.15.7:0/3980617498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac3c102ac0 0x7fac3c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.099+0000 7fac40ee6640 1 -- 172.21.15.7:0/3980617498 >> 172.21.15.7:0/3980617498 conn(0x7fac3c0fc520 msgr2=0x7fac3c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.099+0000 7fac40ee6640 1 -- 172.21.15.7:0/3980617498 shutdown_connections 2024-09-21T04:54:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.099+0000 7fac40ee6640 1 -- 172.21.15.7:0/3980617498 wait complete. 2024-09-21T04:54:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac40ee6640 1 Processor -- start 2024-09-21T04:54:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac40ee6640 1 -- start start 2024-09-21T04:54:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac40ee6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c102ac0 0x7fac3c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac40ee6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac3c1033e0 0x7fac3c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac40ee6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c10edf0 con 0x7fac3c102ac0 2024-09-21T04:54:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac40ee6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c10ec70 con 0x7fac3c1033e0 2024-09-21T04:54:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac3a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c102ac0 0x7fac3c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac3a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c102ac0 0x7fac3c19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34810/0 (socket says 172.21.15.7:34810) 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac3a59c640 1 -- 172.21.15.7:0/2290580052 learned_addr learned my addr 172.21.15.7:0/2290580052 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.100+0000 7fac39d9b640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac3c1033e0 0x7fac3c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac3a59c640 1 -- 172.21.15.7:0/2290580052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac3c1033e0 msgr2=0x7fac3c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac3a59c640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac3c1033e0 0x7fac3c19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac3a59c640 1 -- 172.21.15.7:0/2290580052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c199a10 con 0x7fac3c102ac0 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac3a59c640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c102ac0 0x7fac3c19f850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fac3400d6f0 tx=0x7fac3400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac337fe640 1 -- 172.21.15.7:0/2290580052 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac34017070 con 0x7fac3c102ac0 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac337fe640 1 -- 172.21.15.7:0/2290580052 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac3400de00 con 0x7fac3c102ac0 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac337fe640 1 -- 172.21.15.7:0/2290580052 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac34004bb0 con 0x7fac3c102ac0 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c199d00 con 0x7fac3c102ac0 2024-09-21T04:54:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.101+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c19a260 con 0x7fac3c102ac0 2024-09-21T04:54:50.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.102+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c06aa90 con 0x7fac3c102ac0 2024-09-21T04:54:50.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.103+0000 7fac337fe640 1 -- 172.21.15.7:0/2290580052 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fac34009a30 con 0x7fac3c102ac0 2024-09-21T04:54:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.104+0000 7fac337fe640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac2807af80 0x7fac2807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.104+0000 7fac337fe640 1 -- 172.21.15.7:0/2290580052 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fac3409db20 con 0x7fac3c102ac0 2024-09-21T04:54:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.104+0000 7fac39d9b640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac2807af80 0x7fac2807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.104+0000 7fac39d9b640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac2807af80 0x7fac2807d440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac3c19b200 tx=0x7fac24031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.108+0000 7fac337fe640 1 -- 172.21.15.7:0/2290580052 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fac34069140 con 0x7fac3c102ac0 2024-09-21T04:54:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.369+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fac3c19af40 con 0x7fac3c102ac0 2024-09-21T04:54:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.369+0000 7fac337fe640 1 -- 172.21.15.7:0/2290580052 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fac3406dff0 con 0x7fac3c102ac0 2024-09-21T04:54:50.373 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:54:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:54:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:54:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:54:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:54:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:54:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:54:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:54:50.375 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:54:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac2807af80 msgr2=0x7fac2807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac2807af80 0x7fac2807d440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac3c19b200 tx=0x7fac24031400 comp rx=0 tx=0).stop 2024-09-21T04:54:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c102ac0 msgr2=0x7fac3c19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c102ac0 0x7fac3c19f850 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fac3400d6f0 tx=0x7fac3400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 shutdown_connections 2024-09-21T04:54:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac2807af80 0x7fac2807d440 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac3c1033e0 0x7fac3c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 --2- 172.21.15.7:0/2290580052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac3c102ac0 0x7fac3c19f850 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 >> 172.21.15.7:0/2290580052 conn(0x7fac3c0fc520 msgr2=0x7fac3c100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.371+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 shutdown_connections 2024-09-21T04:54:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.372+0000 7fac40ee6640 1 -- 172.21.15.7:0/2290580052 wait complete. 2024-09-21T04:54:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.456+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3811899059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 msgr2=0x7fa2b810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.456+0000 7fa2bdc51640 1 --2- 172.21.15.7:0/3811899059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 0x7fa2b810b8c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009880 tx=0x7fa2ac02a530 comp rx=0 tx=0).stop 2024-09-21T04:54:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.456+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3811899059 shutdown_connections 2024-09-21T04:54:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.456+0000 7fa2bdc51640 1 --2- 172.21.15.7:0/3811899059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 0x7fa2b810b8c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.456+0000 7fa2bdc51640 1 --2- 172.21.15.7:0/3811899059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8102ab0 0x7fa2b8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.456+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3811899059 >> 172.21.15.7:0/3811899059 conn(0x7fa2b80fc550 msgr2=0x7fa2b80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.456+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3811899059 shutdown_connections 2024-09-21T04:54:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.456+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3811899059 wait complete. 2024-09-21T04:54:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2bdc51640 1 Processor -- start 2024-09-21T04:54:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2bdc51640 1 -- start start 2024-09-21T04:54:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2bdc51640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8102ab0 0x7fa2b819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2bdc51640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 0x7fa2b819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2bdc51640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b810edc0 con 0x7fa2b81033d0 2024-09-21T04:54:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2bdc51640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b810ec40 con 0x7fa2b8102ab0 2024-09-21T04:54:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2b6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 0x7fa2b819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2b6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 0x7fa2b819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34824/0 (socket says 172.21.15.7:34824) 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2b6ffd640 1 -- 172.21.15.7:0/3496570867 learned_addr learned my addr 172.21.15.7:0/3496570867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2b77fe640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8102ab0 0x7fa2b819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2b6ffd640 1 -- 172.21.15.7:0/3496570867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8102ab0 msgr2=0x7fa2b819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2b6ffd640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8102ab0 0x7fa2b819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.457+0000 7fa2b6ffd640 1 -- 172.21.15.7:0/3496570867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b8197790 con 0x7fa2b81033d0 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.458+0000 7fa2b6ffd640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 0x7fa2b819db50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac031460 tx=0x7fa2ac004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.458+0000 7fa2b4ff9640 1 -- 172.21.15.7:0/3496570867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa2ac03f070 con 0x7fa2b81033d0 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.458+0000 7fa2b4ff9640 1 -- 172.21.15.7:0/3496570867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa2ac0035f0 con 0x7fa2b81033d0 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.458+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b8197a20 con 0x7fa2b81033d0 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.458+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b8197f50 con 0x7fa2b81033d0 2024-09-21T04:54:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.458+0000 7fa2b4ff9640 1 -- 172.21.15.7:0/3496570867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa2ac035030 con 0x7fa2b81033d0 2024-09-21T04:54:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.459+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b806a580 con 0x7fa2b81033d0 2024-09-21T04:54:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.460+0000 7fa2b4ff9640 1 -- 172.21.15.7:0/3496570867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa2ac042070 con 0x7fa2b81033d0 2024-09-21T04:54:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.460+0000 7fa2b4ff9640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2a007af80 0x7fa2a007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.460+0000 7fa2b4ff9640 1 -- 172.21.15.7:0/3496570867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa2ac0be890 con 0x7fa2b81033d0 2024-09-21T04:54:50.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.460+0000 7fa2b77fe640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2a007af80 0x7fa2a007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.461+0000 7fa2b77fe640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2a007af80 0x7fa2a007d440 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800bbe0 tx=0x7fa2a800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.465+0000 7fa2b4ff9640 1 -- 172.21.15.7:0/3496570867 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa2ac089eb0 con 0x7fa2b81033d0 2024-09-21T04:54:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.694+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa2b8198200 con 0x7fa2b81033d0 2024-09-21T04:54:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.695+0000 7fa2b4ff9640 1 -- 172.21.15.7:0/3496570867 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7fa2ac08ed60 con 0x7fa2b81033d0 2024-09-21T04:54:50.699 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:54:50.699 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:54:50.699 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:54:50.699 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:54:50.699 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:54:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:54:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:54:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:54:50.702 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2a007af80 msgr2=0x7fa2a007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2a007af80 0x7fa2a007d440 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800bbe0 tx=0x7fa2a800b390 comp rx=0 tx=0).stop 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 msgr2=0x7fa2b819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 0x7fa2b819db50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac031460 tx=0x7fa2ac004530 comp rx=0 tx=0).stop 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 shutdown_connections 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa2a007af80 0x7fa2a007d440 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b81033d0 0x7fa2b819db50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 --2- 172.21.15.7:0/3496570867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2b8102ab0 0x7fa2b819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 >> 172.21.15.7:0/3496570867 conn(0x7fa2b80fc550 msgr2=0x7fa2b810aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 shutdown_connections 2024-09-21T04:54:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.698+0000 7fa2bdc51640 1 -- 172.21.15.7:0/3496570867 wait complete. 2024-09-21T04:54:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.783+0000 7f0cc0baa640 1 -- 172.21.15.7:0/904088668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 msgr2=0x7f0cbc106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.783+0000 7f0cc0baa640 1 --2- 172.21.15.7:0/904088668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 0x7f0cbc106be0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009880 tx=0x7f0cac02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.783+0000 7f0cc0baa640 1 -- 172.21.15.7:0/904088668 shutdown_connections 2024-09-21T04:54:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.783+0000 7f0cc0baa640 1 --2- 172.21.15.7:0/904088668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cbc100800 0x7f0cbc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.783+0000 7f0cc0baa640 1 --2- 172.21.15.7:0/904088668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 0x7f0cbc106be0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.783+0000 7f0cc0baa640 1 -- 172.21.15.7:0/904088668 >> 172.21.15.7:0/904088668 conn(0x7f0cbc0fc520 msgr2=0x7f0cbc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.783+0000 7f0cc0baa640 1 -- 172.21.15.7:0/904088668 shutdown_connections 2024-09-21T04:54:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.783+0000 7f0cc0baa640 1 -- 172.21.15.7:0/904088668 wait complete. 2024-09-21T04:54:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cc0baa640 1 Processor -- start 2024-09-21T04:54:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cc0baa640 1 -- start start 2024-09-21T04:54:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cc0baa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cbc100800 0x7f0cbc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cc0baa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 0x7f0cbc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cc0baa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc10dd30 con 0x7f0cbc106800 2024-09-21T04:54:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cc0baa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc10dbb0 con 0x7f0cbc100800 2024-09-21T04:54:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cb1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 0x7f0cbc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cb1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 0x7f0cbc19dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34842/0 (socket says 172.21.15.7:34842) 2024-09-21T04:54:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.784+0000 7f0cb1d9b640 1 -- 172.21.15.7:0/3122761716 learned_addr learned my addr 172.21.15.7:0/3122761716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cba59c640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cbc100800 0x7f0cbc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cb1d9b640 1 -- 172.21.15.7:0/3122761716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cbc100800 msgr2=0x7f0cbc19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cb1d9b640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cbc100800 0x7f0cbc19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cb1d9b640 1 -- 172.21.15.7:0/3122761716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc197820 con 0x7f0cbc106800 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cb1d9b640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 0x7f0cbc19dbe0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400b420 tx=0x7f0ca400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cb3fff640 1 -- 172.21.15.7:0/3122761716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ca4011630 con 0x7f0cbc106800 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cbc197b10 con 0x7f0cbc106800 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cb3fff640 1 -- 172.21.15.7:0/3122761716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ca4016070 con 0x7f0cbc106800 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cb3fff640 1 -- 172.21.15.7:0/3122761716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ca4010d40 con 0x7f0cbc106800 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.785+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cbc1980a0 con 0x7f0cbc106800 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.786+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cbc101f40 con 0x7f0cbc106800 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.787+0000 7f0cb3fff640 1 -- 172.21.15.7:0/3122761716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0ca401b480 con 0x7f0cbc106800 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.787+0000 7f0cb3fff640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ca807af30 0x7f0ca807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.788+0000 7f0cb3fff640 1 -- 172.21.15.7:0/3122761716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0ca409d840 con 0x7f0cbc106800 2024-09-21T04:54:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.788+0000 7f0cba59c640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ca807af30 0x7f0ca807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.788+0000 7f0cba59c640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ca807af30 0x7f0ca807d3f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0cac031460 tx=0x7f0cac009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:50.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.792+0000 7f0cb3fff640 1 -- 172.21.15.7:0/3122761716 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ca4068d40 con 0x7f0cbc106800 2024-09-21T04:54:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.997+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0cbc1983b0 con 0x7f0ca807af30 2024-09-21T04:54:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:50.998+0000 7f0cb3fff640 1 -- 172.21.15.7:0/3122761716 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0cbc1983b0 con 0x7f0ca807af30 2024-09-21T04:54:51.003 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:54:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:54:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:54:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:54:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:54:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:54:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:54:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:54:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:54:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:54:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:54:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:54:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:54:51.005 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.000+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ca807af30 msgr2=0x7f0ca807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.000+0000 7f0cc0baa640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ca807af30 0x7f0ca807d3f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0cac031460 tx=0x7f0cac009560 comp rx=0 tx=0).stop 2024-09-21T04:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.000+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 msgr2=0x7f0cbc19dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.000+0000 7f0cc0baa640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 0x7f0cbc19dbe0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400b420 tx=0x7f0ca400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.001+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 shutdown_connections 2024-09-21T04:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.001+0000 7f0cc0baa640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ca807af30 0x7f0ca807d3f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.001+0000 7f0cc0baa640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cbc106800 0x7f0cbc19dbe0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.001+0000 7f0cc0baa640 1 --2- 172.21.15.7:0/3122761716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cbc100800 0x7f0cbc19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.001+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 >> 172.21.15.7:0/3122761716 conn(0x7f0cbc0fc520 msgr2=0x7f0cbc1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.001+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 shutdown_connections 2024-09-21T04:54:51.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.001+0000 7f0cc0baa640 1 -- 172.21.15.7:0/3122761716 wait complete. 2024-09-21T04:54:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.085+0000 7f159b900640 1 -- 172.21.15.7:0/3859111013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 msgr2=0x7f1594106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.085+0000 7f159b900640 1 --2- 172.21.15.7:0/3859111013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 0x7f1594106610 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1584009880 tx=0x7f158402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.085+0000 7f159b900640 1 -- 172.21.15.7:0/3859111013 shutdown_connections 2024-09-21T04:54:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.085+0000 7f159b900640 1 --2- 172.21.15.7:0/3859111013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 0x7f1594106610 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.085+0000 7f159b900640 1 --2- 172.21.15.7:0/3859111013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594100120 0x7f1594100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.085+0000 7f159b900640 1 -- 172.21.15.7:0/3859111013 >> 172.21.15.7:0/3859111013 conn(0x7f15940fbc90 msgr2=0x7f15940fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.085+0000 7f159b900640 1 -- 172.21.15.7:0/3859111013 shutdown_connections 2024-09-21T04:54:51.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.085+0000 7f159b900640 1 -- 172.21.15.7:0/3859111013 wait complete. 2024-09-21T04:54:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f159b900640 1 Processor -- start 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f159b900640 1 -- start start 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f159b900640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594100120 0x7f15941010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f159b900640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 0x7f15941015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f159b900640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f159410ee60 con 0x7f1594100ad0 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f159b900640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f159410ece0 con 0x7f1594100120 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f1598e9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 0x7f15941015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f1598e9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 0x7f15941015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34858/0 (socket says 172.21.15.7:34858) 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f1598e9b640 1 -- 172.21.15.7:0/1735623869 learned_addr learned my addr 172.21.15.7:0/1735623869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f159969c640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594100120 0x7f15941010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f1598e9b640 1 -- 172.21.15.7:0/1735623869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594100120 msgr2=0x7f15941010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f1598e9b640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594100120 0x7f15941010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.086+0000 7f1598e9b640 1 -- 172.21.15.7:0/1735623869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1594101b20 con 0x7f1594100ad0 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.087+0000 7f1598e9b640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 0x7f15941015e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1584031460 tx=0x7f15840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.087+0000 7f158a7fc640 1 -- 172.21.15.7:0/1735623869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f158403f070 con 0x7f1594100ad0 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.087+0000 7f158a7fc640 1 -- 172.21.15.7:0/1735623869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1584004cf0 con 0x7f1594100ad0 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.087+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1594101db0 con 0x7f1594100ad0 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.087+0000 7f158a7fc640 1 -- 172.21.15.7:0/1735623869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1584035030 con 0x7f1594100ad0 2024-09-21T04:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.087+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15941a3e80 con 0x7f1594100ad0 2024-09-21T04:54:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.088+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15941031e0 con 0x7f1594100ad0 2024-09-21T04:54:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.089+0000 7f158a7fc640 1 -- 172.21.15.7:0/1735623869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1584042070 con 0x7f1594100ad0 2024-09-21T04:54:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.089+0000 7f158a7fc640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f158007af80 0x7f158007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.089+0000 7f158a7fc640 1 -- 172.21.15.7:0/1735623869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f15840be9e0 con 0x7f1594100ad0 2024-09-21T04:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.089+0000 7f159969c640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f158007af80 0x7f158007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.090+0000 7f159969c640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f158007af80 0x7f158007d440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1590005fd0 tx=0x7f1590005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:54:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.094+0000 7f158a7fc640 1 -- 172.21.15.7:0/1735623869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f158408a000 con 0x7f1594100ad0 2024-09-21T04:54:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.354+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f1594109ef0 con 0x7f1594100ad0 2024-09-21T04:54:51.358 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:51 smithi007 ceph-mon[108238]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:51.358 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:51 smithi007 ceph-mon[108238]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:51.359 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:51 smithi007 ceph-mon[108238]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:51.359 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:51 smithi007 ceph-mon[108238]: pgmap v602: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:51.359 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:51 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2290580052' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:54:51.359 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:51 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3496570867' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:54:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.355+0000 7f158a7fc640 1 -- 172.21.15.7:0/1735623869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f158408eeb0 con 0x7f1594100ad0 2024-09-21T04:54:51.359 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:54:51.359 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:54:51.359 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:54:51.359 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:54:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:54:51.360 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:54:51.360 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f158007af80 msgr2=0x7f158007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f158007af80 0x7f158007d440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1590005fd0 tx=0x7f1590005e20 comp rx=0 tx=0).stop 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 msgr2=0x7f15941015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 0x7f15941015e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1584031460 tx=0x7f15840048f0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 shutdown_connections 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f158007af80 0x7f158007d440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1594100ad0 0x7f15941015e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 --2- 172.21.15.7:0/1735623869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594100120 0x7f15941010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 >> 172.21.15.7:0/1735623869 conn(0x7f15940fbc90 msgr2=0x7f15940fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 shutdown_connections 2024-09-21T04:54:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:54:51.357+0000 7f159b900640 1 -- 172.21.15.7:0/1735623869 wait complete. 2024-09-21T04:54:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:51 smithi053 ceph-mon[80176]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:51 smithi053 ceph-mon[80176]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:51 smithi053 ceph-mon[80176]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:51 smithi053 ceph-mon[80176]: pgmap v602: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:51 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2290580052' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:54:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:51 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3496570867' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:54:52.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:52 smithi053 ceph-mon[80176]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:52.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:52 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1735623869' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:54:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:52 smithi007 ceph-mon[108238]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:54:52.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:52 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1735623869' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:54:53.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:53 smithi007 ceph-mon[108238]: pgmap v603: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:53.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:53 smithi053 ceph-mon[80176]: pgmap v603: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:55.554 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:55 smithi053 ceph-mon[80176]: pgmap v604: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:55.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:55 smithi007 ceph-mon[108238]: pgmap v604: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:57.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:57 smithi053 ceph-mon[80176]: pgmap v605: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:57.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:57 smithi007 ceph-mon[108238]: pgmap v605: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:59.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:59 smithi053 ceph-mon[80176]: pgmap v606: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:54:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:54:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:59 smithi007 ceph-mon[108238]: pgmap v606: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:54:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:54:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:55:01.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:01 smithi053 ceph-mon[80176]: pgmap v607: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:01 smithi007 ceph-mon[108238]: pgmap v607: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:03.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:03 smithi053 ceph-mon[80176]: pgmap v608: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:03.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:03 smithi007 ceph-mon[108238]: pgmap v608: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:05.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:05 smithi007 ceph-mon[108238]: pgmap v609: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:05.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:05 smithi053 ceph-mon[80176]: pgmap v609: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:07.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:07 smithi053 ceph-mon[80176]: pgmap v610: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:07.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:07 smithi007 ceph-mon[108238]: pgmap v610: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:09 smithi007 ceph-mon[108238]: pgmap v611: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:09.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:09 smithi053 ceph-mon[80176]: pgmap v611: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:11.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:11 smithi053 ceph-mon[80176]: pgmap v612: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:11.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:11 smithi007 ceph-mon[108238]: pgmap v612: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:12.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:12 smithi053 ceph-mon[80176]: pgmap v613: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:12.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:12 smithi007 ceph-mon[108238]: pgmap v613: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:13.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:13 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:55:13.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:13 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:55:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:14 smithi053 ceph-mon[80176]: pgmap v614: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:14 smithi007 ceph-mon[108238]: pgmap v614: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:17 smithi007 ceph-mon[108238]: pgmap v615: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:17 smithi053 ceph-mon[80176]: pgmap v615: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:19 smithi053 ceph-mon[80176]: pgmap v616: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:19 smithi007 ceph-mon[108238]: pgmap v616: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:21.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:21 smithi053 ceph-mon[80176]: pgmap v617: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:21 smithi007 ceph-mon[108238]: pgmap v617: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.457+0000 7f5732856640 1 -- 172.21.15.7:0/3720276613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c102ac0 msgr2=0x7f572c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.457+0000 7f5732856640 1 --2- 172.21.15.7:0/3720276613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c102ac0 0x7f572c102ea0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f571c009880 tx=0x7f571c02a530 comp rx=0 tx=0).stop 2024-09-21T04:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.457+0000 7f5732856640 1 -- 172.21.15.7:0/3720276613 shutdown_connections 2024-09-21T04:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.457+0000 7f5732856640 1 --2- 172.21.15.7:0/3720276613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f572c1033e0 0x7f572c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.457+0000 7f5732856640 1 --2- 172.21.15.7:0/3720276613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c102ac0 0x7f572c102ea0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.457+0000 7f5732856640 1 -- 172.21.15.7:0/3720276613 >> 172.21.15.7:0/3720276613 conn(0x7f572c0fc520 msgr2=0x7f572c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.457+0000 7f5732856640 1 -- 172.21.15.7:0/3720276613 shutdown_connections 2024-09-21T04:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.457+0000 7f5732856640 1 -- 172.21.15.7:0/3720276613 wait complete. 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f5732856640 1 Processor -- start 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f5732856640 1 -- start start 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f5732856640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f572c102ac0 0x7f572c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f5732856640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c1033e0 0x7f572c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f5732856640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f572c10edd0 con 0x7f572c1033e0 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f5732856640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f572c10ec50 con 0x7f572c102ac0 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f572b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c1033e0 0x7f572c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f572b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c1033e0 0x7f572c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35570/0 (socket says 172.21.15.7:35570) 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f572b7fe640 1 -- 172.21.15.7:0/3288387346 learned_addr learned my addr 172.21.15.7:0/3288387346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.458+0000 7f572bfff640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f572c102ac0 0x7f572c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f572b7fe640 1 -- 172.21.15.7:0/3288387346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f572c102ac0 msgr2=0x7f572c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f572b7fe640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f572c102ac0 0x7f572c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f572b7fe640 1 -- 172.21.15.7:0/3288387346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572c1a03f0 con 0x7f572c1033e0 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f572bfff640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f572c102ac0 0x7f572c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f572b7fe640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c1033e0 0x7f572c19fd40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f572000c8f0 tx=0x7f572000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f57297fa640 1 -- 172.21.15.7:0/3288387346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f57200079b0 con 0x7f572c1033e0 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572c199ae0 con 0x7f572c1033e0 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f57297fa640 1 -- 172.21.15.7:0/3288387346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5720007b50 con 0x7f572c1033e0 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f57297fa640 1 -- 172.21.15.7:0/3288387346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5720018610 con 0x7f572c1033e0 2024-09-21T04:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.459+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572c19a040 con 0x7f572c1033e0 2024-09-21T04:55:21.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.460+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572c06aa90 con 0x7f572c1033e0 2024-09-21T04:55:21.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.461+0000 7f57297fa640 1 -- 172.21.15.7:0/3288387346 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5720007d00 con 0x7f572c1033e0 2024-09-21T04:55:21.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.461+0000 7f57297fa640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f571407af30 0x7f571407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:21.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.461+0000 7f57297fa640 1 -- 172.21.15.7:0/3288387346 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f572009e2e0 con 0x7f572c1033e0 2024-09-21T04:55:21.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.461+0000 7f572bfff640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f571407af30 0x7f571407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.462+0000 7f572bfff640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f571407af30 0x7f571407d3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f571c000c00 tx=0x7f571c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:21.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.466+0000 7f57297fa640 1 -- 172.21.15.7:0/3288387346 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5720014030 con 0x7f572c1033e0 2024-09-21T04:55:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.670+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f572c19af60 con 0x7f571407af30 2024-09-21T04:55:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.672+0000 7f57297fa640 1 -- 172.21.15.7:0/3288387346 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f572c19af60 con 0x7f571407af30 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f571407af30 msgr2=0x7f571407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f571407af30 0x7f571407d3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f571c000c00 tx=0x7f571c031400 comp rx=0 tx=0).stop 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c1033e0 msgr2=0x7f572c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c1033e0 0x7f572c19fd40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f572000c8f0 tx=0x7f572000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 shutdown_connections 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f571407af30 0x7f571407d3f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f572c1033e0 0x7f572c19fd40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 --2- 172.21.15.7:0/3288387346 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f572c102ac0 0x7f572c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 >> 172.21.15.7:0/3288387346 conn(0x7f572c0fc520 msgr2=0x7f572c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 shutdown_connections 2024-09-21T04:55:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.674+0000 7f5732856640 1 -- 172.21.15.7:0/3288387346 wait complete. 2024-09-21T04:55:21.686 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:55:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.760+0000 7fb02c665640 1 -- 172.21.15.7:0/2355698560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024107900 msgr2=0x7fb024107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.760+0000 7fb02c665640 1 --2- 172.21.15.7:0/2355698560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024107900 0x7fb024107ce0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb020009880 tx=0x7fb02002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.760+0000 7fb02c665640 1 -- 172.21.15.7:0/2355698560 shutdown_connections 2024-09-21T04:55:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.760+0000 7fb02c665640 1 --2- 172.21.15.7:0/2355698560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb024101900 0x7fb024101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.760+0000 7fb02c665640 1 --2- 172.21.15.7:0/2355698560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024107900 0x7fb024107ce0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.760+0000 7fb02c665640 1 -- 172.21.15.7:0/2355698560 >> 172.21.15.7:0/2355698560 conn(0x7fb0240fd600 msgr2=0x7fb0240ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.761+0000 7fb02c665640 1 -- 172.21.15.7:0/2355698560 shutdown_connections 2024-09-21T04:55:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.761+0000 7fb02c665640 1 -- 172.21.15.7:0/2355698560 wait complete. 2024-09-21T04:55:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.761+0000 7fb02c665640 1 Processor -- start 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.761+0000 7fb02c665640 1 -- start start 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.761+0000 7fb02c665640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024101900 0x7fb02419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.761+0000 7fb02c665640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb024107900 0x7fb02419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.761+0000 7fb02c665640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb02410edb0 con 0x7fb024101900 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.761+0000 7fb02c665640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb02410ec30 con 0x7fb024107900 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02a401640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024101900 0x7fb02419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02a401640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024101900 0x7fb02419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35588/0 (socket says 172.21.15.7:35588) 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02a401640 1 -- 172.21.15.7:0/3961465883 learned_addr learned my addr 172.21.15.7:0/3961465883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb029c00640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb024107900 0x7fb02419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02a401640 1 -- 172.21.15.7:0/3961465883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb024107900 msgr2=0x7fb02419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02a401640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb024107900 0x7fb02419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02a401640 1 -- 172.21.15.7:0/3961465883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0241977e0 con 0x7fb024101900 2024-09-21T04:55:21.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02a401640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024101900 0x7fb02419d660 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb020031460 tx=0x7fb0200045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb01b7fe640 1 -- 172.21.15.7:0/3961465883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb02003f070 con 0x7fb024101900 2024-09-21T04:55:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb01b7fe640 1 -- 172.21.15.7:0/3961465883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb020004b10 con 0x7fb024101900 2024-09-21T04:55:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb024197a70 con 0x7fb024101900 2024-09-21T04:55:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb01b7fe640 1 -- 172.21.15.7:0/3961465883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb020035030 con 0x7fb024101900 2024-09-21T04:55:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.762+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb024197fa0 con 0x7fb024101900 2024-09-21T04:55:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.764+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb024103040 con 0x7fb024101900 2024-09-21T04:55:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.764+0000 7fb01b7fe640 1 -- 172.21.15.7:0/3961465883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb020042070 con 0x7fb024101900 2024-09-21T04:55:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.765+0000 7fb01b7fe640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb01007af90 0x7fb01007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.765+0000 7fb01b7fe640 1 -- 172.21.15.7:0/3961465883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb0200bea60 con 0x7fb024101900 2024-09-21T04:55:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.765+0000 7fb029c00640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb01007af90 0x7fb01007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.765+0000 7fb029c00640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb01007af90 0x7fb01007d450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb024100d10 tx=0x7fb01400b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.769+0000 7fb01b7fe640 1 -- 172.21.15.7:0/3961465883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb02008a020 con 0x7fb024101900 2024-09-21T04:55:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.976+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb024198310 con 0x7fb01007af90 2024-09-21T04:55:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.977+0000 7fb01b7fe640 1 -- 172.21.15.7:0/3961465883 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fb024198310 con 0x7fb01007af90 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb01007af90 msgr2=0x7fb01007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb01007af90 0x7fb01007d450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb024100d10 tx=0x7fb01400b410 comp rx=0 tx=0).stop 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024101900 msgr2=0x7fb02419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024101900 0x7fb02419d660 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb020031460 tx=0x7fb0200045e0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 shutdown_connections 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb01007af90 0x7fb01007d450 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb024107900 0x7fb02419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 --2- 172.21.15.7:0/3961465883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024101900 0x7fb02419d660 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 >> 172.21.15.7:0/3961465883 conn(0x7fb0240fd600 msgr2=0x7fb0240fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 shutdown_connections 2024-09-21T04:55:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:21.979+0000 7fb02c665640 1 -- 172.21.15.7:0/3961465883 wait complete. 2024-09-21T04:55:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.063+0000 7f4109227640 1 -- 172.21.15.7:0/4030431219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 msgr2=0x7f4104101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.063+0000 7f4109227640 1 --2- 172.21.15.7:0/4030431219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 0x7f4104101410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009880 tx=0x7f40ec02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.063+0000 7f4109227640 1 -- 172.21.15.7:0/4030431219 shutdown_connections 2024-09-21T04:55:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.063+0000 7f4109227640 1 --2- 172.21.15.7:0/4030431219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 0x7f4104101410 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.063+0000 7f4109227640 1 --2- 172.21.15.7:0/4030431219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4104106a90 0x7f4104106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.063+0000 7f4109227640 1 -- 172.21.15.7:0/4030431219 >> 172.21.15.7:0/4030431219 conn(0x7f41040fbc90 msgr2=0x7f41040fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.064+0000 7f4109227640 1 -- 172.21.15.7:0/4030431219 shutdown_connections 2024-09-21T04:55:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.064+0000 7f4109227640 1 -- 172.21.15.7:0/4030431219 wait complete. 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.064+0000 7f4109227640 1 Processor -- start 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.064+0000 7f4109227640 1 -- start start 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4109227640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 0x7f4104199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4109227640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4104106a90 0x7f410419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4109227640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f410410df60 con 0x7f4104100fb0 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4109227640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f410410dde0 con 0x7f4104106a90 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4102d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 0x7f4104199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4102d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 0x7f4104199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35618/0 (socket says 172.21.15.7:35618) 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4102d9d640 1 -- 172.21.15.7:0/3710272731 learned_addr learned my addr 172.21.15.7:0/3710272731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f40fbfff640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4104106a90 0x7f410419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4102d9d640 1 -- 172.21.15.7:0/3710272731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4104106a90 msgr2=0x7f410419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4102d9d640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4104106a90 0x7f410419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4102d9d640 1 -- 172.21.15.7:0/3710272731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410419e750 con 0x7f4104100fb0 2024-09-21T04:55:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.065+0000 7f4102d9d640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 0x7f4104199e80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c8f0 tx=0x7f40f400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.066+0000 7f4100d99640 1 -- 172.21.15.7:0/3710272731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f40f40079b0 con 0x7f4104100fb0 2024-09-21T04:55:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.066+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410419ea40 con 0x7f4104100fb0 2024-09-21T04:55:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.066+0000 7f4100d99640 1 -- 172.21.15.7:0/3710272731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f40f40054e0 con 0x7f4104100fb0 2024-09-21T04:55:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.066+0000 7f4100d99640 1 -- 172.21.15.7:0/3710272731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f40f4004dd0 con 0x7f4104100fb0 2024-09-21T04:55:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.066+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f410419efd0 con 0x7f4104100fb0 2024-09-21T04:55:22.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.067+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41041021d0 con 0x7f4104100fb0 2024-09-21T04:55:22.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.067+0000 7f4100d99640 1 -- 172.21.15.7:0/3710272731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f40f4005020 con 0x7f4104100fb0 2024-09-21T04:55:22.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.068+0000 7f4100d99640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007aed0 0x7f40f007d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.068+0000 7f4100d99640 1 -- 172.21.15.7:0/3710272731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f40f409e030 con 0x7f4104100fb0 2024-09-21T04:55:22.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.068+0000 7f40fbfff640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007aed0 0x7f40f007d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.069+0000 7f40fbfff640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007aed0 0x7f40f007d390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f41040fdb60 tx=0x7f40ec031420 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:22.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.072+0000 7f4100d99640 1 -- 172.21.15.7:0/3710272731 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40f4014030 con 0x7f4104100fb0 2024-09-21T04:55:22.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.271+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f410419b1e0 con 0x7f40f007aed0 2024-09-21T04:55:22.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.280+0000 7f4100d99640 1 -- 172.21.15.7:0/3710272731 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f410419b1e0 con 0x7f40f007aed0 2024-09-21T04:55:22.283 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:55:22.283 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (17m) 11m ago 28m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:55:22.283 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (20m) 11m ago 28m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (20m) 11m ago 27m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (15m) 11m ago 28m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (15m) 11m ago 27m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (17m) 11m ago 28m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (11m) 11m ago 22m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (11m) 11m ago 23m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (22m) 11m ago 22m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (11m) 11m ago 22m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (19m) 11m ago 29m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (18m) 11m ago 27m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (16m) 11m ago 29m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (15m) 11m ago 27m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:55:22.284 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (18m) 11m ago 28m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:55:22.285 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (18m) 11m ago 27m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:55:22.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (15m) 11m ago 26m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:55:22.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (14m) 11m ago 25m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:55:22.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (13m) 11m ago 25m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:55:22.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (13m) 11m ago 24m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:55:22.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (12m) 11m ago 24m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:55:22.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (12m) 11m ago 24m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:55:22.285 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (17m) 11m ago 27m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.282+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007aed0 msgr2=0x7f40f007d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007aed0 0x7f40f007d390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f41040fdb60 tx=0x7f40ec031420 comp rx=0 tx=0).stop 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 msgr2=0x7f4104199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 0x7f4104199e80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c8f0 tx=0x7f40f400cdb0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 shutdown_connections 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007aed0 0x7f40f007d390 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4104106a90 0x7f410419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 --2- 172.21.15.7:0/3710272731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4104100fb0 0x7f4104199e80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 >> 172.21.15.7:0/3710272731 conn(0x7f41040fbc90 msgr2=0x7f41040fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:22.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 shutdown_connections 2024-09-21T04:55:22.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.283+0000 7f4109227640 1 -- 172.21.15.7:0/3710272731 wait complete. 2024-09-21T04:55:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.366+0000 7f95c3d73640 1 -- 172.21.15.7:0/537121984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 msgr2=0x7f95bc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.366+0000 7f95c3d73640 1 --2- 172.21.15.7:0/537121984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 0x7f95bc107ce0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009880 tx=0x7f95b802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.366+0000 7f95c3d73640 1 -- 172.21.15.7:0/537121984 shutdown_connections 2024-09-21T04:55:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.366+0000 7f95c3d73640 1 --2- 172.21.15.7:0/537121984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc101900 0x7f95bc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.366+0000 7f95c3d73640 1 --2- 172.21.15.7:0/537121984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 0x7f95bc107ce0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.366+0000 7f95c3d73640 1 -- 172.21.15.7:0/537121984 >> 172.21.15.7:0/537121984 conn(0x7f95bc0fd600 msgr2=0x7f95bc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.366+0000 7f95c3d73640 1 -- 172.21.15.7:0/537121984 shutdown_connections 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.366+0000 7f95c3d73640 1 -- 172.21.15.7:0/537121984 wait complete. 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c3d73640 1 Processor -- start 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c3d73640 1 -- start start 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c3d73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc101900 0x7f95bc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c3d73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 0x7f95bc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c3d73640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc10edd0 con 0x7f95bc107900 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c3d73640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc10ec50 con 0x7f95bc101900 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c130e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 0x7f95bc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c130e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 0x7f95bc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35634/0 (socket says 172.21.15.7:35634) 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.367+0000 7f95c130e640 1 -- 172.21.15.7:0/2721172833 learned_addr learned my addr 172.21.15.7:0/2721172833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95c1b0f640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc101900 0x7f95bc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95c130e640 1 -- 172.21.15.7:0/2721172833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc101900 msgr2=0x7f95bc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95c130e640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc101900 0x7f95bc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95c130e640 1 -- 172.21.15.7:0/2721172833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc074f70 con 0x7f95bc107900 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95c130e640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 0x7f95bc074a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00d6f0 tx=0x7f95ac00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95b2ffd640 1 -- 172.21.15.7:0/2721172833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f95ac017070 con 0x7f95bc107900 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95b2ffd640 1 -- 172.21.15.7:0/2721172833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95ac00de00 con 0x7f95bc107900 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc075260 con 0x7f95bc107900 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95b2ffd640 1 -- 172.21.15.7:0/2721172833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f95ac004bb0 con 0x7f95bc107900 2024-09-21T04:55:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.368+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc1a3e20 con 0x7f95bc107900 2024-09-21T04:55:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.369+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95bc103040 con 0x7f95bc107900 2024-09-21T04:55:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.370+0000 7f95b2ffd640 1 -- 172.21.15.7:0/2721172833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f95ac009a30 con 0x7f95bc107900 2024-09-21T04:55:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.371+0000 7f95b2ffd640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f95a807af80 0x7f95a807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.371+0000 7f95b2ffd640 1 -- 172.21.15.7:0/2721172833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f95ac09dbe0 con 0x7f95bc107900 2024-09-21T04:55:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.371+0000 7f95c1b0f640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f95a807af80 0x7f95a807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.372+0000 7f95c1b0f640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f95a807af80 0x7f95a807d440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f95b80096c0 tx=0x7f95b8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:22.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.375+0000 7f95b2ffd640 1 -- 172.21.15.7:0/2721172833 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95ac069200 con 0x7f95bc107900 2024-09-21T04:55:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.638+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f95bc109e70 con 0x7f95bc107900 2024-09-21T04:55:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.638+0000 7f95b2ffd640 1 -- 172.21.15.7:0/2721172833 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f95ac05ee20 con 0x7f95bc107900 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:55:22.642 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:55:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:55:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:55:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:55:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:55:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:55:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:55:22.643 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f95a807af80 msgr2=0x7f95a807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f95a807af80 0x7f95a807d440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f95b80096c0 tx=0x7f95b8031400 comp rx=0 tx=0).stop 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 msgr2=0x7f95bc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 0x7f95bc074a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00d6f0 tx=0x7f95ac00dbb0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 shutdown_connections 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f95a807af80 0x7f95a807d440 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95bc107900 0x7f95bc074a30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 --2- 172.21.15.7:0/2721172833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc101900 0x7f95bc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.640+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 >> 172.21.15.7:0/2721172833 conn(0x7f95bc0fd600 msgr2=0x7f95bc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.641+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 shutdown_connections 2024-09-21T04:55:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.641+0000 7f95c3d73640 1 -- 172.21.15.7:0/2721172833 wait complete. 2024-09-21T04:55:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.725+0000 7f07353c0640 1 -- 172.21.15.7:0/686713983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 msgr2=0x7f0730106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.725+0000 7f07353c0640 1 --2- 172.21.15.7:0/686713983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 0x7f0730106be0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0720009880 tx=0x7f072002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.725+0000 7f07353c0640 1 -- 172.21.15.7:0/686713983 shutdown_connections 2024-09-21T04:55:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.725+0000 7f07353c0640 1 --2- 172.21.15.7:0/686713983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0730100800 0x7f0730100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.725+0000 7f07353c0640 1 --2- 172.21.15.7:0/686713983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 0x7f0730106be0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.725+0000 7f07353c0640 1 -- 172.21.15.7:0/686713983 >> 172.21.15.7:0/686713983 conn(0x7f07300fc520 msgr2=0x7f07300fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.725+0000 7f07353c0640 1 -- 172.21.15.7:0/686713983 shutdown_connections 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.725+0000 7f07353c0640 1 -- 172.21.15.7:0/686713983 wait complete. 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f07353c0640 1 Processor -- start 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f07353c0640 1 -- start start 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f07353c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0730100800 0x7f0730198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f07353c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 0x7f0730198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f07353c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f073010dcd0 con 0x7f0730106800 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f07353c0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f073010db50 con 0x7f0730100800 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f0727fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 0x7f0730198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f0727fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 0x7f0730198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35654/0 (socket says 172.21.15.7:35654) 2024-09-21T04:55:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f0727fff640 1 -- 172.21.15.7:0/3966225434 learned_addr learned my addr 172.21.15.7:0/3966225434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f072effd640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0730100800 0x7f0730198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f0727fff640 1 -- 172.21.15.7:0/3966225434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0730100800 msgr2=0x7f0730198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f0727fff640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0730100800 0x7f0730198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.726+0000 7f0727fff640 1 -- 172.21.15.7:0/3966225434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0730199350 con 0x7f0730106800 2024-09-21T04:55:22.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.727+0000 7f0727fff640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 0x7f0730198d50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f071800d6f0 tx=0x7f071800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:22.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.727+0000 7f072cff9640 1 -- 172.21.15.7:0/3966225434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0718017070 con 0x7f0730106800 2024-09-21T04:55:22.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.727+0000 7f072cff9640 1 -- 172.21.15.7:0/3966225434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f071800de00 con 0x7f0730106800 2024-09-21T04:55:22.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.727+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0730199640 con 0x7f0730106800 2024-09-21T04:55:22.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.727+0000 7f072cff9640 1 -- 172.21.15.7:0/3966225434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0718004bb0 con 0x7f0730106800 2024-09-21T04:55:22.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.727+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07301a1c40 con 0x7f0730106800 2024-09-21T04:55:22.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.728+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0730101f40 con 0x7f0730106800 2024-09-21T04:55:22.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.728+0000 7f072cff9640 1 -- 172.21.15.7:0/3966225434 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0718004e60 con 0x7f0730106800 2024-09-21T04:55:22.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.729+0000 7f072cff9640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f071c07af30 0x7f071c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:22.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.729+0000 7f072cff9640 1 -- 172.21.15.7:0/3966225434 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f071809dc00 con 0x7f0730106800 2024-09-21T04:55:22.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.729+0000 7f072effd640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f071c07af30 0x7f071c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:22.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.730+0000 7f072effd640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f071c07af30 0x7f071c07d3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f07200096c0 tx=0x7f0720031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.733+0000 7f072cff9640 1 -- 172.21.15.7:0/3966225434 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07180a3020 con 0x7f0730106800 2024-09-21T04:55:22.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.962+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f073006a7a0 con 0x7f0730106800 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.963+0000 7f072cff9640 1 -- 172.21.15.7:0/3966225434 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f0730199640 con 0x7f0730106800 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:55:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:55:22.967 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:55:22.968 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:55:22.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:55:22.969 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:55:22.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:55:22.969 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:55:22.969 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:55:22.969 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:55:22.969 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f071c07af30 msgr2=0x7f071c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f071c07af30 0x7f071c07d3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f07200096c0 tx=0x7f0720031400 comp rx=0 tx=0).stop 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 msgr2=0x7f0730198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 0x7f0730198d50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f071800d6f0 tx=0x7f071800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 shutdown_connections 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f071c07af30 0x7f071c07d3f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0730106800 0x7f0730198d50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 --2- 172.21.15.7:0/3966225434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0730100800 0x7f0730198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 >> 172.21.15.7:0/3966225434 conn(0x7f07300fc520 msgr2=0x7f07300fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:22.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 shutdown_connections 2024-09-21T04:55:22.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:22.965+0000 7f07353c0640 1 -- 172.21.15.7:0/3966225434 wait complete. 2024-09-21T04:55:23.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 -- 172.21.15.7:0/3357273237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 msgr2=0x7f1170106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:23.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 --2- 172.21.15.7:0/3357273237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 0x7f1170106bd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1160009880 tx=0x7f116002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 -- 172.21.15.7:0/3357273237 shutdown_connections 2024-09-21T04:55:23.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 --2- 172.21.15.7:0/3357273237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11701007f0 0x7f1170100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 --2- 172.21.15.7:0/3357273237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 0x7f1170106bd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 -- 172.21.15.7:0/3357273237 >> 172.21.15.7:0/3357273237 conn(0x7f11700fc550 msgr2=0x7f11700fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:23.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 -- 172.21.15.7:0/3357273237 shutdown_connections 2024-09-21T04:55:23.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 -- 172.21.15.7:0/3357273237 wait complete. 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.051+0000 7f1176109640 1 Processor -- start 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f1176109640 1 -- start start 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f1176109640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11701007f0 0x7f11700fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f1176109640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 0x7f11700fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f1176109640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f117010dcc0 con 0x7f11701067f0 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f1176109640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f117010db40 con 0x7f11701007f0 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f116effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 0x7f11700fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f116effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 0x7f11700fd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35682/0 (socket says 172.21.15.7:35682) 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f116effd640 1 -- 172.21.15.7:0/1301519654 learned_addr learned my addr 172.21.15.7:0/1301519654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:23.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f116f7fe640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11701007f0 0x7f11700fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f116effd640 1 -- 172.21.15.7:0/1301519654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11701007f0 msgr2=0x7f11700fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f116effd640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11701007f0 0x7f11700fccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f116effd640 1 -- 172.21.15.7:0/1301519654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11700fd770 con 0x7f11701067f0 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.052+0000 7f116effd640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 0x7f11700fd230 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f116400c8f0 tx=0x7f116400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.053+0000 7f116cff9640 1 -- 172.21.15.7:0/1301519654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f11640079b0 con 0x7f11701067f0 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.053+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11700fda60 con 0x7f11701067f0 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.053+0000 7f116cff9640 1 -- 172.21.15.7:0/1301519654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11640054e0 con 0x7f11701067f0 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.053+0000 7f116cff9640 1 -- 172.21.15.7:0/1301519654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1164004dd0 con 0x7f11701067f0 2024-09-21T04:55:23.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.053+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a3e30 con 0x7f11701067f0 2024-09-21T04:55:23.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.054+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1170101f30 con 0x7f11701067f0 2024-09-21T04:55:23.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.054+0000 7f116cff9640 1 -- 172.21.15.7:0/1301519654 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1164005020 con 0x7f11701067f0 2024-09-21T04:55:23.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.055+0000 7f116cff9640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f115807aed0 0x7f115807d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:23.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.055+0000 7f116cff9640 1 -- 172.21.15.7:0/1301519654 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f116409e080 con 0x7f11701067f0 2024-09-21T04:55:23.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.055+0000 7f116f7fe640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f115807aed0 0x7f115807d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:23.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.056+0000 7f116f7fe640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f115807aed0 0x7f115807d390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1160031460 tx=0x7f1160005bd0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.059+0000 7f116cff9640 1 -- 172.21.15.7:0/1301519654 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1164014030 con 0x7f11701067f0 2024-09-21T04:55:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:23 smithi007 ceph-mon[108238]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:23 smithi007 ceph-mon[108238]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:23 smithi007 ceph-mon[108238]: pgmap v618: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2721172833' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:55:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3966225434' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:55:23.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.268+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1170062630 con 0x7f115807aed0 2024-09-21T04:55:23.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.270+0000 7f116cff9640 1 -- 172.21.15.7:0/1301519654 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f1170062630 con 0x7f115807aed0 2024-09-21T04:55:23.272 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:55:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:55:23.273 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f115807aed0 msgr2=0x7f115807d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f115807aed0 0x7f115807d390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1160031460 tx=0x7f1160005bd0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 msgr2=0x7f11700fd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 0x7f11700fd230 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f116400c8f0 tx=0x7f116400cdb0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 shutdown_connections 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f115807aed0 0x7f115807d390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11701067f0 0x7f11700fd230 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 --2- 172.21.15.7:0/1301519654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11701007f0 0x7f11700fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 >> 172.21.15.7:0/1301519654 conn(0x7f11700fc550 msgr2=0x7f1170069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 shutdown_connections 2024-09-21T04:55:23.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.272+0000 7f1176109640 1 -- 172.21.15.7:0/1301519654 wait complete. 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.356+0000 7f587e926640 1 -- 172.21.15.7:0/1535418589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 msgr2=0x7f5878104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.356+0000 7f587e926640 1 --2- 172.21.15.7:0/1535418589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 0x7f5878104c50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f586c00a990 tx=0x7f586c02b7d0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.356+0000 7f587e926640 1 -- 172.21.15.7:0/1535418589 shutdown_connections 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.356+0000 7f587e926640 1 --2- 172.21.15.7:0/1535418589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 0x7f5878104c50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.356+0000 7f587e926640 1 --2- 172.21.15.7:0/1535418589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5878100250 0x7f5878100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.356+0000 7f587e926640 1 -- 172.21.15.7:0/1535418589 >> 172.21.15.7:0/1535418589 conn(0x7f58780fbc90 msgr2=0x7f58780fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.356+0000 7f587e926640 1 -- 172.21.15.7:0/1535418589 shutdown_connections 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.356+0000 7f587e926640 1 -- 172.21.15.7:0/1535418589 wait complete. 2024-09-21T04:55:23.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.357+0000 7f587e926640 1 Processor -- start 2024-09-21T04:55:23.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.357+0000 7f587e926640 1 -- start start 2024-09-21T04:55:23.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f587e926640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5878100250 0x7f5878101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:23.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f587e926640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 0x7f5878102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:23.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f587e926640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f587810cbb0 con 0x7f5878100c00 2024-09-21T04:55:23.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f587e926640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f587810ca30 con 0x7f5878100250 2024-09-21T04:55:23.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58777fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 0x7f5878102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:23.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58777fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 0x7f5878102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35694/0 (socket says 172.21.15.7:35694) 2024-09-21T04:55:23.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58777fe640 1 -- 172.21.15.7:0/1862483367 learned_addr learned my addr 172.21.15.7:0/1862483367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f5877fff640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5878100250 0x7f5878101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58777fe640 1 -- 172.21.15.7:0/1862483367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5878100250 msgr2=0x7f5878101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58777fe640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5878100250 0x7f5878101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58777fe640 1 -- 172.21.15.7:0/1862483367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5878102770 con 0x7f5878100c00 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f5877fff640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5878100250 0x7f5878101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58777fe640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 0x7f5878102230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f586c006010 tx=0x7f586c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58757fa640 1 -- 172.21.15.7:0/1862483367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f586c008e90 con 0x7f5878100c00 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58757fa640 1 -- 172.21.15.7:0/1862483367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f586c045070 con 0x7f5878100c00 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5878102a00 con 0x7f5878100c00 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.358+0000 7f58757fa640 1 -- 172.21.15.7:0/1862483367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f586c0035f0 con 0x7f5878100c00 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.359+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58781a3e70 con 0x7f5878100c00 2024-09-21T04:55:23.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.359+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5878101730 con 0x7f5878100c00 2024-09-21T04:55:23.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.360+0000 7f58757fa640 1 -- 172.21.15.7:0/1862483367 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f586c032070 con 0x7f5878100c00 2024-09-21T04:55:23.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.361+0000 7f58757fa640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f586007af80 0x7f586007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.361+0000 7f58757fa640 1 -- 172.21.15.7:0/1862483367 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f586c008030 con 0x7f5878100c00 2024-09-21T04:55:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.361+0000 7f5877fff640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f586007af80 0x7f586007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.361+0000 7f5877fff640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f586007af80 0x7f586007d440 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5868005d00 tx=0x7f5868009550 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.365+0000 7f58757fa640 1 -- 172.21.15.7:0/1862483367 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f586c08a500 con 0x7f5878100c00 2024-09-21T04:55:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:23 smithi053 ceph-mon[80176]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:23 smithi053 ceph-mon[80176]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:23 smithi053 ceph-mon[80176]: pgmap v618: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2721172833' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:55:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3966225434' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:55:23.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.625+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f5878103180 con 0x7f5878100c00 2024-09-21T04:55:23.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.625+0000 7f58757fa640 1 -- 172.21.15.7:0/1862483367 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f586c08f3b0 con 0x7f5878100c00 2024-09-21T04:55:23.628 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:55:23.629 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:55:23.629 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:55:23.629 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:55:23.629 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:55:23.629 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:55:23.629 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.627+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f586007af80 msgr2=0x7f586007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.627+0000 7f587e926640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f586007af80 0x7f586007d440 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5868005d00 tx=0x7f5868009550 comp rx=0 tx=0).stop 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 msgr2=0x7f5878102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 0x7f5878102230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f586c006010 tx=0x7f586c004270 comp rx=0 tx=0).stop 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 shutdown_connections 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f586007af80 0x7f586007d440 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5878100c00 0x7f5878102230 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 --2- 172.21.15.7:0/1862483367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5878100250 0x7f5878101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 >> 172.21.15.7:0/1862483367 conn(0x7f58780fbc90 msgr2=0x7f58780fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 shutdown_connections 2024-09-21T04:55:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:23.628+0000 7f587e926640 1 -- 172.21.15.7:0/1862483367 wait complete. 2024-09-21T04:55:24.353 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:24 smithi007 ceph-mon[108238]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:24.353 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:24 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1862483367' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:55:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:24 smithi053 ceph-mon[80176]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:24 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1862483367' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:55:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:25 smithi053 ceph-mon[80176]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:25 smithi053 ceph-mon[80176]: pgmap v619: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:25 smithi007 ceph-mon[108238]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:25 smithi007 ceph-mon[108238]: pgmap v619: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:27.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:27 smithi053 ceph-mon[80176]: pgmap v620: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:27 smithi007 ceph-mon[108238]: pgmap v620: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:29 smithi007 ceph-mon[108238]: pgmap v621: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:55:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:29 smithi053 ceph-mon[80176]: pgmap v621: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:55:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:31 smithi053 ceph-mon[80176]: pgmap v622: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:31 smithi007 ceph-mon[108238]: pgmap v622: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:33 smithi053 ceph-mon[80176]: pgmap v623: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:33 smithi007 ceph-mon[108238]: pgmap v623: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:35 smithi007 ceph-mon[108238]: pgmap v624: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:35 smithi053 ceph-mon[80176]: pgmap v624: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:37 smithi053 ceph-mon[80176]: pgmap v625: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:37 smithi007 ceph-mon[108238]: pgmap v625: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:39 smithi053 ceph-mon[80176]: pgmap v626: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:39 smithi007 ceph-mon[108238]: pgmap v626: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:41 smithi007 ceph-mon[108238]: pgmap v627: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:41 smithi053 ceph-mon[80176]: pgmap v627: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:43 smithi053 ceph-mon[80176]: pgmap v628: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:43 smithi007 ceph-mon[108238]: pgmap v628: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:44.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:55:44.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:55:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:45 smithi053 ceph-mon[80176]: pgmap v629: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:45 smithi007 ceph-mon[108238]: pgmap v629: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:47 smithi007 ceph-mon[108238]: pgmap v630: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:47 smithi053 ceph-mon[80176]: pgmap v630: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:49 smithi053 ceph-mon[80176]: pgmap v631: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:49 smithi007 ceph-mon[108238]: pgmap v631: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:51.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:51 smithi053 ceph-mon[80176]: pgmap v632: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:51 smithi007 ceph-mon[108238]: pgmap v632: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:53 smithi007 ceph-mon[108238]: pgmap v633: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:53 smithi053 ceph-mon[80176]: pgmap v633: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:53.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.725+0000 7f4e6427f640 1 -- 172.21.15.7:0/3753724860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c101900 msgr2=0x7f4e5c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:53.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.725+0000 7f4e6427f640 1 --2- 172.21.15.7:0/3753724860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c101900 0x7f4e5c101d60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009880 tx=0x7f4e5002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:53.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.725+0000 7f4e6427f640 1 -- 172.21.15.7:0/3753724860 shutdown_connections 2024-09-21T04:55:53.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.725+0000 7f4e6427f640 1 --2- 172.21.15.7:0/3753724860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c101900 0x7f4e5c101d60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:53.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.725+0000 7f4e6427f640 1 --2- 172.21.15.7:0/3753724860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c107900 0x7f4e5c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:53.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.725+0000 7f4e6427f640 1 -- 172.21.15.7:0/3753724860 >> 172.21.15.7:0/3753724860 conn(0x7f4e5c0fd600 msgr2=0x7f4e5c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:53.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.725+0000 7f4e6427f640 1 -- 172.21.15.7:0/3753724860 shutdown_connections 2024-09-21T04:55:53.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.725+0000 7f4e6427f640 1 -- 172.21.15.7:0/3753724860 wait complete. 2024-09-21T04:55:53.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6427f640 1 Processor -- start 2024-09-21T04:55:53.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6427f640 1 -- start start 2024-09-21T04:55:53.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6427f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c101900 0x7f4e5c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:53.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6427f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c107900 0x7f4e5c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:53.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6427f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c10edd0 con 0x7f4e5c107900 2024-09-21T04:55:53.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6427f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c10ec50 con 0x7f4e5c101900 2024-09-21T04:55:53.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6181a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c107900 0x7f4e5c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6181a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c107900 0x7f4e5c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40470/0 (socket says 172.21.15.7:40470) 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6181a640 1 -- 172.21.15.7:0/2538585176 learned_addr learned my addr 172.21.15.7:0/2538585176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6201b640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c101900 0x7f4e5c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6181a640 1 -- 172.21.15.7:0/2538585176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c101900 msgr2=0x7f4e5c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6181a640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c101900 0x7f4e5c19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.726+0000 7f4e6181a640 1 -- 172.21.15.7:0/2538585176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5c1977a0 con 0x7f4e5c107900 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.727+0000 7f4e6181a640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c107900 0x7f4e5c19db60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4e50031460 tx=0x7f4e500045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.727+0000 7f4e5b7fe640 1 -- 172.21.15.7:0/2538585176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e5003f070 con 0x7f4e5c107900 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.727+0000 7f4e5b7fe640 1 -- 172.21.15.7:0/2538585176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e500035f0 con 0x7f4e5c107900 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.727+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5c197a30 con 0x7f4e5c107900 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.727+0000 7f4e5b7fe640 1 -- 172.21.15.7:0/2538585176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e50035030 con 0x7f4e5c107900 2024-09-21T04:55:53.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.727+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5c197f60 con 0x7f4e5c107900 2024-09-21T04:55:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.728+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5c103040 con 0x7f4e5c107900 2024-09-21T04:55:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.729+0000 7f4e5b7fe640 1 -- 172.21.15.7:0/2538585176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4e50042070 con 0x7f4e5c107900 2024-09-21T04:55:53.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.729+0000 7f4e5b7fe640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e4407af80 0x7f4e4407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:53.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.730+0000 7f4e5b7fe640 1 -- 172.21.15.7:0/2538585176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4e500be960 con 0x7f4e5c107900 2024-09-21T04:55:53.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.730+0000 7f4e6201b640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e4407af80 0x7f4e4407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:53.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.730+0000 7f4e6201b640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e4407af80 0x7f4e4407d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005fd0 tx=0x7f4e4c009680 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:53.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.734+0000 7f4e5b7fe640 1 -- 172.21.15.7:0/2538585176 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e50089ee0 con 0x7f4e5c107900 2024-09-21T04:55:53.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.940+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4e5c1982d0 con 0x7f4e4407af80 2024-09-21T04:55:53.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.941+0000 7f4e5b7fe640 1 -- 172.21.15.7:0/2538585176 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f4e5c1982d0 con 0x7f4e4407af80 2024-09-21T04:55:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.943+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e4407af80 msgr2=0x7f4e4407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.943+0000 7f4e6427f640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e4407af80 0x7f4e4407d440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005fd0 tx=0x7f4e4c009680 comp rx=0 tx=0).stop 2024-09-21T04:55:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.943+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c107900 msgr2=0x7f4e5c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.943+0000 7f4e6427f640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c107900 0x7f4e5c19db60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4e50031460 tx=0x7f4e500045e0 comp rx=0 tx=0).stop 2024-09-21T04:55:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.944+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 shutdown_connections 2024-09-21T04:55:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.944+0000 7f4e6427f640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e4407af80 0x7f4e4407d440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.944+0000 7f4e6427f640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5c107900 0x7f4e5c19db60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.944+0000 7f4e6427f640 1 --2- 172.21.15.7:0/2538585176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e5c101900 0x7f4e5c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.944+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 >> 172.21.15.7:0/2538585176 conn(0x7f4e5c0fd600 msgr2=0x7f4e5c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.944+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 shutdown_connections 2024-09-21T04:55:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:53.944+0000 7f4e6427f640 1 -- 172.21.15.7:0/2538585176 wait complete. 2024-09-21T04:55:53.956 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:55:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.029+0000 7f158709b640 1 -- 172.21.15.7:0/929962156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580101000 msgr2=0x7f158010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.029+0000 7f158709b640 1 --2- 172.21.15.7:0/929962156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580101000 0x7f158010dcc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f156c009880 tx=0x7f156c02a490 comp rx=0 tx=0).stop 2024-09-21T04:55:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.029+0000 7f158709b640 1 -- 172.21.15.7:0/929962156 shutdown_connections 2024-09-21T04:55:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.029+0000 7f158709b640 1 --2- 172.21.15.7:0/929962156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580101000 0x7f158010dcc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.029+0000 7f158709b640 1 --2- 172.21.15.7:0/929962156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1580100650 0x7f1580100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.029+0000 7f158709b640 1 -- 172.21.15.7:0/929962156 >> 172.21.15.7:0/929962156 conn(0x7f15800fc520 msgr2=0x7f15800fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:54.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.029+0000 7f158709b640 1 -- 172.21.15.7:0/929962156 shutdown_connections 2024-09-21T04:55:54.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.029+0000 7f158709b640 1 -- 172.21.15.7:0/929962156 wait complete. 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f158709b640 1 Processor -- start 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f158709b640 1 -- start start 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f158709b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580100650 0x7f158019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f158709b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1580101000 0x7f158019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f158709b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1580110f70 con 0x7f1580100650 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f158709b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1580110df0 con 0x7f1580101000 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f1584e37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580100650 0x7f158019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f1584e37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580100650 0x7f158019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40488/0 (socket says 172.21.15.7:40488) 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f1584e37640 1 -- 172.21.15.7:0/2152545512 learned_addr learned my addr 172.21.15.7:0/2152545512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.030+0000 7f157ffff640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1580101000 0x7f158019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f1584e37640 1 -- 172.21.15.7:0/2152545512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1580101000 msgr2=0x7f158019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f1584e37640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1580101000 0x7f158019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f1584e37640 1 -- 172.21.15.7:0/2152545512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1580199a00 con 0x7f1580100650 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f1584e37640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580100650 0x7f158019f840 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f157400d8d0 tx=0x7f157400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f157dffb640 1 -- 172.21.15.7:0/2152545512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1574017030 con 0x7f1580100650 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1580199cf0 con 0x7f1580100650 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f157dffb640 1 -- 172.21.15.7:0/2152545512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1574004020 con 0x7f1580100650 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f157dffb640 1 -- 172.21.15.7:0/2152545512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f15740047a0 con 0x7f1580100650 2024-09-21T04:55:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.031+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158019a250 con 0x7f1580100650 2024-09-21T04:55:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.032+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158006aa90 con 0x7f1580100650 2024-09-21T04:55:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.033+0000 7f157dffb640 1 -- 172.21.15.7:0/2152545512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f15740041d0 con 0x7f1580100650 2024-09-21T04:55:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.034+0000 7f157dffb640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f157007af80 0x7f157007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.034+0000 7f157dffb640 1 -- 172.21.15.7:0/2152545512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f1574018070 con 0x7f1580100650 2024-09-21T04:55:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.034+0000 7f157ffff640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f157007af80 0x7f157007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.034+0000 7f157ffff640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f157007af80 0x7f157007d440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f158019b1f0 tx=0x7f156c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:54.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.038+0000 7f157dffb640 1 -- 172.21.15.7:0/2152545512 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1574069370 con 0x7f1580100650 2024-09-21T04:55:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.244+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f158019af30 con 0x7f157007af80 2024-09-21T04:55:54.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.245+0000 7f157dffb640 1 -- 172.21.15.7:0/2152545512 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f158019af30 con 0x7f157007af80 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f157007af80 msgr2=0x7f157007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f157007af80 0x7f157007d440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f158019b1f0 tx=0x7f156c031400 comp rx=0 tx=0).stop 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580100650 msgr2=0x7f158019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580100650 0x7f158019f840 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f157400d8d0 tx=0x7f157400dd90 comp rx=0 tx=0).stop 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 shutdown_connections 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f157007af80 0x7f157007d440 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1580101000 0x7f158019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 --2- 172.21.15.7:0/2152545512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1580100650 0x7f158019f840 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 >> 172.21.15.7:0/2152545512 conn(0x7f15800fc520 msgr2=0x7f15800fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:54.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 shutdown_connections 2024-09-21T04:55:54.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.247+0000 7f158709b640 1 -- 172.21.15.7:0/2152545512 wait complete. 2024-09-21T04:55:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.331+0000 7f648781b640 1 -- 172.21.15.7:0/1757746733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100140 msgr2=0x7f6480100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.331+0000 7f648781b640 1 --2- 172.21.15.7:0/1757746733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100140 0x7f6480100520 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f647c009880 tx=0x7f647c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.331+0000 7f648781b640 1 -- 172.21.15.7:0/1757746733 shutdown_connections 2024-09-21T04:55:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.331+0000 7f648781b640 1 --2- 172.21.15.7:0/1757746733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6480100af0 0x7f6480106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.331+0000 7f648781b640 1 --2- 172.21.15.7:0/1757746733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100140 0x7f6480100520 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.331+0000 7f648781b640 1 -- 172.21.15.7:0/1757746733 >> 172.21.15.7:0/1757746733 conn(0x7f64800fbc90 msgr2=0x7f64800fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.331+0000 7f648781b640 1 -- 172.21.15.7:0/1757746733 shutdown_connections 2024-09-21T04:55:54.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.331+0000 7f648781b640 1 -- 172.21.15.7:0/1757746733 wait complete. 2024-09-21T04:55:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.332+0000 7f648781b640 1 Processor -- start 2024-09-21T04:55:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.332+0000 7f648781b640 1 -- start start 2024-09-21T04:55:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.332+0000 7f648781b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6480100140 0x7f648019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.332+0000 7f648781b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100af0 0x7f648019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.332+0000 7f648781b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f648010ee60 con 0x7f6480100af0 2024-09-21T04:55:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.332+0000 7f648781b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f648010ece0 con 0x7f6480100140 2024-09-21T04:55:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.332+0000 7f6484db6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100af0 0x7f648019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f6484db6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100af0 0x7f648019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40508/0 (socket says 172.21.15.7:40508) 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f6484db6640 1 -- 172.21.15.7:0/411570003 learned_addr learned my addr 172.21.15.7:0/411570003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f64855b7640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6480100140 0x7f648019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f6484db6640 1 -- 172.21.15.7:0/411570003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6480100140 msgr2=0x7f648019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f6484db6640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6480100140 0x7f648019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f6484db6640 1 -- 172.21.15.7:0/411570003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64801977f0 con 0x7f6480100af0 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f6484db6640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100af0 0x7f648019dbb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f647000d6c0 tx=0x7f647000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f64767fc640 1 -- 172.21.15.7:0/411570003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6470017070 con 0x7f6480100af0 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f64767fc640 1 -- 172.21.15.7:0/411570003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f64700054e0 con 0x7f6480100af0 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6480197ae0 con 0x7f6480100af0 2024-09-21T04:55:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f64767fc640 1 -- 172.21.15.7:0/411570003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6470004b70 con 0x7f6480100af0 2024-09-21T04:55:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.333+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6480198070 con 0x7f6480100af0 2024-09-21T04:55:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.335+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6480103200 con 0x7f6480100af0 2024-09-21T04:55:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.335+0000 7f64767fc640 1 -- 172.21.15.7:0/411570003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6470009820 con 0x7f6480100af0 2024-09-21T04:55:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.336+0000 7f64767fc640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f646c07af90 0x7f646c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.336+0000 7f64767fc640 1 -- 172.21.15.7:0/411570003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f647009dae0 con 0x7f6480100af0 2024-09-21T04:55:54.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.336+0000 7f64855b7640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f646c07af90 0x7f646c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.337+0000 7f64855b7640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f646c07af90 0x7f646c07d450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f647c0096c0 tx=0x7f647c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:54.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.340+0000 7f64767fc640 1 -- 172.21.15.7:0/411570003 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6470069100 con 0x7f6480100af0 2024-09-21T04:55:54.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.538+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f64801983e0 con 0x7f646c07af90 2024-09-21T04:55:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.547+0000 7f64767fc640 1 -- 172.21.15.7:0/411570003 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f64801983e0 con 0x7f646c07af90 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (17m) 11m ago 28m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (21m) 11m ago 29m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (20m) 11m ago 28m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (16m) 11m ago 29m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (16m) 11m ago 28m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (17m) 11m ago 28m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (12m) 11m ago 23m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (12m) 11m ago 23m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (23m) 11m ago 23m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (11m) 11m ago 23m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (20m) 11m ago 30m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:55:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (19m) 11m ago 28m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (16m) 11m ago 30m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (16m) 11m ago 27m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (18m) 11m ago 28m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (18m) 11m ago 28m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (15m) 11m ago 26m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (15m) 11m ago 26m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (14m) 11m ago 25m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (13m) 11m ago 25m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (13m) 11m ago 25m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (12m) 11m ago 24m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:55:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (18m) 11m ago 28m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:55:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f646c07af90 msgr2=0x7f646c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f646c07af90 0x7f646c07d450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f647c0096c0 tx=0x7f647c031400 comp rx=0 tx=0).stop 2024-09-21T04:55:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100af0 msgr2=0x7f648019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100af0 0x7f648019dbb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f647000d6c0 tx=0x7f647000db80 comp rx=0 tx=0).stop 2024-09-21T04:55:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 shutdown_connections 2024-09-21T04:55:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f646c07af90 0x7f646c07d450 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6480100af0 0x7f648019dbb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 --2- 172.21.15.7:0/411570003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6480100140 0x7f648019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 >> 172.21.15.7:0/411570003 conn(0x7f64800fbc90 msgr2=0x7f6480105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 shutdown_connections 2024-09-21T04:55:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.550+0000 7f648781b640 1 -- 172.21.15.7:0/411570003 wait complete. 2024-09-21T04:55:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.633+0000 7fdc52d92640 1 -- 172.21.15.7:0/3023899736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1067f0 msgr2=0x7fdc4c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.633+0000 7fdc52d92640 1 --2- 172.21.15.7:0/3023899736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1067f0 0x7fdc4c106bd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009880 tx=0x7fdc3c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.633+0000 7fdc52d92640 1 -- 172.21.15.7:0/3023899736 shutdown_connections 2024-09-21T04:55:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.633+0000 7fdc52d92640 1 --2- 172.21.15.7:0/3023899736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc4c1007f0 0x7fdc4c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.633+0000 7fdc52d92640 1 --2- 172.21.15.7:0/3023899736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1067f0 0x7fdc4c106bd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.633+0000 7fdc52d92640 1 -- 172.21.15.7:0/3023899736 >> 172.21.15.7:0/3023899736 conn(0x7fdc4c0fc550 msgr2=0x7fdc4c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:54.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.634+0000 7fdc52d92640 1 -- 172.21.15.7:0/3023899736 shutdown_connections 2024-09-21T04:55:54.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.634+0000 7fdc52d92640 1 -- 172.21.15.7:0/3023899736 wait complete. 2024-09-21T04:55:54.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.634+0000 7fdc52d92640 1 Processor -- start 2024-09-21T04:55:54.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.634+0000 7fdc52d92640 1 -- start start 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.634+0000 7fdc52d92640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1007f0 0x7fdc4c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.634+0000 7fdc52d92640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc4c1067f0 0x7fdc4c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.634+0000 7fdc52d92640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c10dcc0 con 0x7fdc4c1007f0 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.634+0000 7fdc52d92640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c10db40 con 0x7fdc4c1067f0 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc50b2e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1007f0 0x7fdc4c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc50b2e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1007f0 0x7fdc4c19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40514/0 (socket says 172.21.15.7:40514) 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc50b2e640 1 -- 172.21.15.7:0/3709308575 learned_addr learned my addr 172.21.15.7:0/3709308575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc4bfff640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc4c1067f0 0x7fdc4c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc50b2e640 1 -- 172.21.15.7:0/3709308575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc4c1067f0 msgr2=0x7fdc4c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc50b2e640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc4c1067f0 0x7fdc4c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc50b2e640 1 -- 172.21.15.7:0/3709308575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c1977e0 con 0x7fdc4c1007f0 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc4bfff640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc4c1067f0 0x7fdc4c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc50b2e640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1007f0 0x7fdc4c19d660 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c031460 tx=0x7fdc3c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc49ffb640 1 -- 172.21.15.7:0/3709308575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdc3c03f070 con 0x7fdc4c1007f0 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc49ffb640 1 -- 172.21.15.7:0/3709308575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc3c0044d0 con 0x7fdc4c1007f0 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4c197a70 con 0x7fdc4c1007f0 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.635+0000 7fdc49ffb640 1 -- 172.21.15.7:0/3709308575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdc3c035030 con 0x7fdc4c1007f0 2024-09-21T04:55:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.636+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4c197fa0 con 0x7fdc4c1007f0 2024-09-21T04:55:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.637+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4c101f30 con 0x7fdc4c1007f0 2024-09-21T04:55:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.638+0000 7fdc49ffb640 1 -- 172.21.15.7:0/3709308575 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdc3c042070 con 0x7fdc4c1007f0 2024-09-21T04:55:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.638+0000 7fdc49ffb640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc3407af40 0x7fdc3407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.638+0000 7fdc49ffb640 1 -- 172.21.15.7:0/3709308575 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fdc3c0bebc0 con 0x7fdc4c1007f0 2024-09-21T04:55:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.638+0000 7fdc4bfff640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc3407af40 0x7fdc3407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:54.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.639+0000 7fdc4bfff640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc3407af40 0x7fdc3407d400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005fd0 tx=0x7fdc40005ea0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.643+0000 7fdc49ffb640 1 -- 172.21.15.7:0/3709308575 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc3c004cf0 con 0x7fdc4c1007f0 2024-09-21T04:55:54.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.909+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fdc4c109d70 con 0x7fdc4c1007f0 2024-09-21T04:55:54.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.910+0000 7fdc49ffb640 1 -- 172.21.15.7:0/3709308575 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fdc3c08a160 con 0x7fdc4c1007f0 2024-09-21T04:55:54.914 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:55:54.914 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:55:54.914 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:55:54.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:55:54.914 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:55:54.914 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:55:54.915 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc3407af40 msgr2=0x7fdc3407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc3407af40 0x7fdc3407d400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005fd0 tx=0x7fdc40005ea0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1007f0 msgr2=0x7fdc4c19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1007f0 0x7fdc4c19d660 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c031460 tx=0x7fdc3c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 shutdown_connections 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdc3407af40 0x7fdc3407d400 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc4c1067f0 0x7fdc4c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 --2- 172.21.15.7:0/3709308575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c1007f0 0x7fdc4c19d660 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 >> 172.21.15.7:0/3709308575 conn(0x7fdc4c0fc550 msgr2=0x7fdc4c069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 shutdown_connections 2024-09-21T04:55:54.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.913+0000 7fdc52d92640 1 -- 172.21.15.7:0/3709308575 wait complete. 2024-09-21T04:55:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.997+0000 7f6a32918640 1 -- 172.21.15.7:0/4168430238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 msgr2=0x7f6a2c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.997+0000 7f6a32918640 1 --2- 172.21.15.7:0/4168430238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 0x7f6a2c101d60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009880 tx=0x7f6a2002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.997+0000 7f6a32918640 1 -- 172.21.15.7:0/4168430238 shutdown_connections 2024-09-21T04:55:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.997+0000 7f6a32918640 1 --2- 172.21.15.7:0/4168430238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 0x7f6a2c101d60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.997+0000 7f6a32918640 1 --2- 172.21.15.7:0/4168430238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2c107900 0x7f6a2c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.997+0000 7f6a32918640 1 -- 172.21.15.7:0/4168430238 >> 172.21.15.7:0/4168430238 conn(0x7f6a2c0fd600 msgr2=0x7f6a2c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.997+0000 7f6a32918640 1 -- 172.21.15.7:0/4168430238 shutdown_connections 2024-09-21T04:55:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.998+0000 7f6a32918640 1 -- 172.21.15.7:0/4168430238 wait complete. 2024-09-21T04:55:55.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.998+0000 7f6a32918640 1 Processor -- start 2024-09-21T04:55:55.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.998+0000 7f6a32918640 1 -- start start 2024-09-21T04:55:55.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.998+0000 7f6a32918640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 0x7f6a2c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.998+0000 7f6a32918640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2c107900 0x7f6a2c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a32918640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c10edb0 con 0x7f6a2c101900 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a32918640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c10ec30 con 0x7f6a2c107900 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a2bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 0x7f6a2c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a2bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 0x7f6a2c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40522/0 (socket says 172.21.15.7:40522) 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a2bfff640 1 -- 172.21.15.7:0/2391313458 learned_addr learned my addr 172.21.15.7:0/2391313458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a2b7fe640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2c107900 0x7f6a2c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a2bfff640 1 -- 172.21.15.7:0/2391313458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2c107900 msgr2=0x7f6a2c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a2bfff640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2c107900 0x7f6a2c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a2bfff640 1 -- 172.21.15.7:0/2391313458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c074f70 con 0x7f6a2c101900 2024-09-21T04:55:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a2bfff640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 0x7f6a2c0744f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00d8d0 tx=0x7f6a1c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a297fa640 1 -- 172.21.15.7:0/2391313458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6a1c017030 con 0x7f6a2c101900 2024-09-21T04:55:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c075260 con 0x7f6a2c101900 2024-09-21T04:55:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a297fa640 1 -- 172.21.15.7:0/2391313458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a1c004020 con 0x7f6a2c101900 2024-09-21T04:55:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a297fa640 1 -- 172.21.15.7:0/2391313458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6a1c0047a0 con 0x7f6a2c101900 2024-09-21T04:55:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:54.999+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c1a3e60 con 0x7f6a2c101900 2024-09-21T04:55:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.001+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c103040 con 0x7f6a2c101900 2024-09-21T04:55:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.001+0000 7f6a297fa640 1 -- 172.21.15.7:0/2391313458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6a1c0041d0 con 0x7f6a2c101900 2024-09-21T04:55:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.002+0000 7f6a297fa640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a1407af30 0x7f6a1407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.002+0000 7f6a297fa640 1 -- 172.21.15.7:0/2391313458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6a1c018070 con 0x7f6a2c101900 2024-09-21T04:55:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.002+0000 7f6a2b7fe640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a1407af30 0x7f6a1407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.002+0000 7f6a2b7fe640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a1407af30 0x7f6a1407d3f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c075f80 tx=0x7f6a20031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.006+0000 7f6a297fa640 1 -- 172.21.15.7:0/2391313458 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6a1c0693b0 con 0x7f6a2c101900 2024-09-21T04:55:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:55 smithi007 ceph-mon[108238]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:55 smithi007 ceph-mon[108238]: pgmap v634: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:55.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:55 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3709308575' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:55:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.232+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6a2c109e90 con 0x7f6a2c101900 2024-09-21T04:55:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.233+0000 7f6a297fa640 1 -- 172.21.15.7:0/2391313458 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f6a1c06e260 con 0x7f6a2c101900 2024-09-21T04:55:55.237 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:55:55.237 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:55:55.237 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:55:55.238 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:55:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:55:55.240 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:55:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:55:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a1407af30 msgr2=0x7f6a1407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a1407af30 0x7f6a1407d3f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c075f80 tx=0x7f6a20031400 comp rx=0 tx=0).stop 2024-09-21T04:55:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 msgr2=0x7f6a2c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 0x7f6a2c0744f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00d8d0 tx=0x7f6a1c00dd90 comp rx=0 tx=0).stop 2024-09-21T04:55:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 shutdown_connections 2024-09-21T04:55:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a1407af30 0x7f6a1407d3f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a2c107900 0x7f6a2c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 --2- 172.21.15.7:0/2391313458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2c101900 0x7f6a2c0744f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 >> 172.21.15.7:0/2391313458 conn(0x7f6a2c0fd600 msgr2=0x7f6a2c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 shutdown_connections 2024-09-21T04:55:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.235+0000 7f6a32918640 1 -- 172.21.15.7:0/2391313458 wait complete. 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.320+0000 7f756d650640 1 -- 172.21.15.7:0/233997181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 msgr2=0x7f7568107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.320+0000 7f756d650640 1 --2- 172.21.15.7:0/233997181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 0x7f7568107ce0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7558009880 tx=0x7f755802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.320+0000 7f756d650640 1 -- 172.21.15.7:0/233997181 shutdown_connections 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.320+0000 7f756d650640 1 --2- 172.21.15.7:0/233997181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7568101900 0x7f7568101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.320+0000 7f756d650640 1 --2- 172.21.15.7:0/233997181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 0x7f7568107ce0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.320+0000 7f756d650640 1 -- 172.21.15.7:0/233997181 >> 172.21.15.7:0/233997181 conn(0x7f75680fd600 msgr2=0x7f75680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.320+0000 7f756d650640 1 -- 172.21.15.7:0/233997181 shutdown_connections 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.320+0000 7f756d650640 1 -- 172.21.15.7:0/233997181 wait complete. 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.321+0000 7f756d650640 1 Processor -- start 2024-09-21T04:55:55.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.321+0000 7f756d650640 1 -- start start 2024-09-21T04:55:55.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.321+0000 7f756d650640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7568101900 0x7f756819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.321+0000 7f756d650640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 0x7f756819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.321+0000 7f756d650640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f756810edd0 con 0x7f7568107900 2024-09-21T04:55:55.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.321+0000 7f756d650640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f756810ec50 con 0x7f7568101900 2024-09-21T04:55:55.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.321+0000 7f75667fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 0x7f756819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f75667fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 0x7f756819fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40538/0 (socket says 172.21.15.7:40538) 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f75667fc640 1 -- 172.21.15.7:0/3774484374 learned_addr learned my addr 172.21.15.7:0/3774484374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f7566ffd640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7568101900 0x7f756819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f75667fc640 1 -- 172.21.15.7:0/3774484374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7568101900 msgr2=0x7f756819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f75667fc640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7568101900 0x7f756819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f75667fc640 1 -- 172.21.15.7:0/3774484374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7568199a10 con 0x7f7568107900 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f75667fc640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 0x7f756819fd90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f755c00c8f0 tx=0x7f755c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f754ffff640 1 -- 172.21.15.7:0/3774484374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f755c0079b0 con 0x7f7568107900 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f754ffff640 1 -- 172.21.15.7:0/3774484374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f755c0054e0 con 0x7f7568107900 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7568199d00 con 0x7f7568107900 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f754ffff640 1 -- 172.21.15.7:0/3774484374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f755c004dd0 con 0x7f7568107900 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.322+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756819a290 con 0x7f7568107900 2024-09-21T04:55:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.323+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7568103040 con 0x7f7568107900 2024-09-21T04:55:55.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.324+0000 7f754ffff640 1 -- 172.21.15.7:0/3774484374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f755c005020 con 0x7f7568107900 2024-09-21T04:55:55.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.324+0000 7f754ffff640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f755007af40 0x7f755007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.325+0000 7f754ffff640 1 -- 172.21.15.7:0/3774484374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f755c09dfa0 con 0x7f7568107900 2024-09-21T04:55:55.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.325+0000 7f7566ffd640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f755007af40 0x7f755007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.325+0000 7f7566ffd640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f755007af40 0x7f755007d400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7558000c00 tx=0x7f7558031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:55.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.329+0000 7f754ffff640 1 -- 172.21.15.7:0/3774484374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f755c014030 con 0x7f7568107900 2024-09-21T04:55:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:55 smithi053 ceph-mon[80176]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:55 smithi053 ceph-mon[80176]: pgmap v634: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:55 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3709308575' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:55:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.539+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f756819b040 con 0x7f755007af40 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.540+0000 7f754ffff640 1 -- 172.21.15.7:0/3774484374 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f75580042e0 con 0x7f755007af40 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:55:55.544 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:55:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:55:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:55:55.545 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:55:55.545 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:55:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f755007af40 msgr2=0x7f755007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f755007af40 0x7f755007d400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7558000c00 tx=0x7f7558031400 comp rx=0 tx=0).stop 2024-09-21T04:55:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 msgr2=0x7f756819fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 0x7f756819fd90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f755c00c8f0 tx=0x7f755c00cdb0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 shutdown_connections 2024-09-21T04:55:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f755007af40 0x7f755007d400 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7568107900 0x7f756819fd90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 --2- 172.21.15.7:0/3774484374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7568101900 0x7f756819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.542+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 >> 172.21.15.7:0/3774484374 conn(0x7f75680fd600 msgr2=0x7f7568074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.543+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 shutdown_connections 2024-09-21T04:55:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.543+0000 7f756d650640 1 -- 172.21.15.7:0/3774484374 wait complete. 2024-09-21T04:55:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.627+0000 7f3b8b631640 1 -- 172.21.15.7:0/2941329714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 msgr2=0x7f3b841123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.627+0000 7f3b8b631640 1 --2- 172.21.15.7:0/2941329714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 0x7f3b841123d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009880 tx=0x7f3b7402a530 comp rx=0 tx=0).stop 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.627+0000 7f3b8b631640 1 -- 172.21.15.7:0/2941329714 shutdown_connections 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.627+0000 7f3b8b631640 1 --2- 172.21.15.7:0/2941329714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 0x7f3b841123d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.627+0000 7f3b8b631640 1 --2- 172.21.15.7:0/2941329714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b84074590 0x7f3b84074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.627+0000 7f3b8b631640 1 -- 172.21.15.7:0/2941329714 >> 172.21.15.7:0/2941329714 conn(0x7f3b840fd600 msgr2=0x7f3b840ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.627+0000 7f3b8b631640 1 -- 172.21.15.7:0/2941329714 shutdown_connections 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.627+0000 7f3b8b631640 1 -- 172.21.15.7:0/2941329714 wait complete. 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.628+0000 7f3b8b631640 1 Processor -- start 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.628+0000 7f3b8b631640 1 -- start start 2024-09-21T04:55:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.628+0000 7f3b8b631640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b84074590 0x7f3b8419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.628+0000 7f3b8b631640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 0x7f3b8419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.628+0000 7f3b8b631640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b841153c0 con 0x7f3b84074eb0 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.628+0000 7f3b8b631640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b84115240 con 0x7f3b84074590 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.628+0000 7f3b88bcc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 0x7f3b8419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.628+0000 7f3b88bcc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 0x7f3b8419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40550/0 (socket says 172.21.15.7:40550) 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b88bcc640 1 -- 172.21.15.7:0/4068869711 learned_addr learned my addr 172.21.15.7:0/4068869711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b893cd640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b84074590 0x7f3b8419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b88bcc640 1 -- 172.21.15.7:0/4068869711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b84074590 msgr2=0x7f3b8419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b88bcc640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b84074590 0x7f3b8419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b88bcc640 1 -- 172.21.15.7:0/4068869711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b841a2890 con 0x7f3b84074eb0 2024-09-21T04:55:55.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b88bcc640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 0x7f3b8419e500 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3b74031460 tx=0x7f3b740045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b7a7fc640 1 -- 172.21.15.7:0/4068869711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3b7403f070 con 0x7f3b84074eb0 2024-09-21T04:55:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b7a7fc640 1 -- 172.21.15.7:0/4068869711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b740035f0 con 0x7f3b84074eb0 2024-09-21T04:55:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b841a2b20 con 0x7f3b84074eb0 2024-09-21T04:55:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b7a7fc640 1 -- 172.21.15.7:0/4068869711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3b74035030 con 0x7f3b84074eb0 2024-09-21T04:55:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.629+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b841a3050 con 0x7f3b84074eb0 2024-09-21T04:55:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.630+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b84075cd0 con 0x7f3b84074eb0 2024-09-21T04:55:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.631+0000 7f3b7a7fc640 1 -- 172.21.15.7:0/4068869711 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3b74042070 con 0x7f3b84074eb0 2024-09-21T04:55:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.632+0000 7f3b7a7fc640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b7007af80 0x7f3b7007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:55:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.632+0000 7f3b7a7fc640 1 -- 172.21.15.7:0/4068869711 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f3b740be4a0 con 0x7f3b84074eb0 2024-09-21T04:55:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.632+0000 7f3b893cd640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b7007af80 0x7f3b7007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:55:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.632+0000 7f3b893cd640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b7007af80 0x7f3b7007d440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b80005d00 tx=0x7f3b8000b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:55:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.636+0000 7f3b7a7fc640 1 -- 172.21.15.7:0/4068869711 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b74089ac0 con 0x7f3b84074eb0 2024-09-21T04:55:55.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.901+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f3b8410fac0 con 0x7f3b84074eb0 2024-09-21T04:55:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.902+0000 7f3b7a7fc640 1 -- 172.21.15.7:0/4068869711 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f3b7408e970 con 0x7f3b84074eb0 2024-09-21T04:55:55.906 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:55:55.906 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:55:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:55:55.906 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:55:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:55:55.906 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:55:55.906 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:55:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b7007af80 msgr2=0x7f3b7007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b7007af80 0x7f3b7007d440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b80005d00 tx=0x7f3b8000b410 comp rx=0 tx=0).stop 2024-09-21T04:55:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 msgr2=0x7f3b8419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:55:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 0x7f3b8419e500 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3b74031460 tx=0x7f3b740045e0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 shutdown_connections 2024-09-21T04:55:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b7007af80 0x7f3b7007d440 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b84074eb0 0x7f3b8419e500 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 --2- 172.21.15.7:0/4068869711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b84074590 0x7f3b8419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:55:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 >> 172.21.15.7:0/4068869711 conn(0x7f3b840fd600 msgr2=0x7f3b84104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:55:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 shutdown_connections 2024-09-21T04:55:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:55:55.904+0000 7f3b8b631640 1 -- 172.21.15.7:0/4068869711 wait complete. 2024-09-21T04:55:56.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:56 smithi007 ceph-mon[108238]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:56.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:56 smithi007 ceph-mon[108238]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:56.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:56 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2391313458' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:55:56.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:56 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4068869711' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:55:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:56 smithi053 ceph-mon[80176]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:56 smithi053 ceph-mon[80176]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:56 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2391313458' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:55:56.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:56 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4068869711' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:55:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:57 smithi053 ceph-mon[80176]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:57 smithi053 ceph-mon[80176]: pgmap v635: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:57 smithi007 ceph-mon[108238]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:55:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:57 smithi007 ceph-mon[108238]: pgmap v635: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:59 smithi007 ceph-mon[108238]: pgmap v636: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:55:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:55:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:59 smithi053 ceph-mon[80176]: pgmap v636: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:55:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:55:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:01 smithi053 ceph-mon[80176]: pgmap v637: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:01 smithi007 ceph-mon[108238]: pgmap v637: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:03 smithi053 ceph-mon[80176]: pgmap v638: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:03 smithi007 ceph-mon[108238]: pgmap v638: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:05 smithi007 ceph-mon[108238]: pgmap v639: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:05 smithi053 ceph-mon[80176]: pgmap v639: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:07 smithi053 ceph-mon[80176]: pgmap v640: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:07 smithi007 ceph-mon[108238]: pgmap v640: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:09 smithi053 ceph-mon[80176]: pgmap v641: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:09 smithi007 ceph-mon[108238]: pgmap v641: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:11 smithi007 ceph-mon[108238]: pgmap v642: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:11 smithi053 ceph-mon[80176]: pgmap v642: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:13.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:13 smithi053 ceph-mon[80176]: pgmap v643: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:13 smithi007 ceph-mon[108238]: pgmap v643: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:15 smithi053 ceph-mon[80176]: pgmap v644: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:15 smithi007 ceph-mon[108238]: pgmap v644: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:17 smithi007 ceph-mon[108238]: pgmap v645: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:17 smithi053 ceph-mon[80176]: pgmap v645: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:19 smithi053 ceph-mon[80176]: pgmap v646: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:19 smithi007 ceph-mon[108238]: pgmap v646: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:21.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:21 smithi053 ceph-mon[80176]: pgmap v647: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:21 smithi007 ceph-mon[108238]: pgmap v647: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:23 smithi053 ceph-mon[80176]: pgmap v648: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:23.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:23 smithi007 ceph-mon[108238]: pgmap v648: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:25 smithi007 ceph-mon[108238]: pgmap v649: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:25.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:25 smithi053 ceph-mon[80176]: pgmap v649: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.005+0000 7f071e352640 1 -- 172.21.15.7:0/369107294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718101900 msgr2=0x7f0718101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.005+0000 7f071e352640 1 --2- 172.21.15.7:0/369107294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718101900 0x7f0718101d60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f070c009880 tx=0x7f070c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.005+0000 7f071e352640 1 -- 172.21.15.7:0/369107294 shutdown_connections 2024-09-21T04:56:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.005+0000 7f071e352640 1 --2- 172.21.15.7:0/369107294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718101900 0x7f0718101d60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.005+0000 7f071e352640 1 --2- 172.21.15.7:0/369107294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0718107900 0x7f0718107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.005+0000 7f071e352640 1 -- 172.21.15.7:0/369107294 >> 172.21.15.7:0/369107294 conn(0x7f07180fd600 msgr2=0x7f07180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.005+0000 7f071e352640 1 -- 172.21.15.7:0/369107294 shutdown_connections 2024-09-21T04:56:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.005+0000 7f071e352640 1 -- 172.21.15.7:0/369107294 wait complete. 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.006+0000 7f071e352640 1 Processor -- start 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.006+0000 7f071e352640 1 -- start start 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f071e352640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0718101900 0x7f071819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f071e352640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718107900 0x7f071819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f071e352640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f071810edd0 con 0x7f0718107900 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f071e352640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f071810ec50 con 0x7f0718101900 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f07177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718107900 0x7f071819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f07177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718107900 0x7f071819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34000/0 (socket says 172.21.15.7:34000) 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f07177fe640 1 -- 172.21.15.7:0/3725367952 learned_addr learned my addr 172.21.15.7:0/3725367952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f0717fff640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0718101900 0x7f071819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f07177fe640 1 -- 172.21.15.7:0/3725367952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0718101900 msgr2=0x7f071819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f07177fe640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0718101900 0x7f071819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f07177fe640 1 -- 172.21.15.7:0/3725367952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07181999d0 con 0x7f0718107900 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f0717fff640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0718101900 0x7f071819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.007+0000 7f07177fe640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718107900 0x7f071819fd50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f070c031460 tx=0x7f070c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.008+0000 7f07157fa640 1 -- 172.21.15.7:0/3725367952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f070c03f070 con 0x7f0718107900 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.008+0000 7f07157fa640 1 -- 172.21.15.7:0/3725367952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f070c0044d0 con 0x7f0718107900 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.008+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0718199c60 con 0x7f0718107900 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.008+0000 7f07157fa640 1 -- 172.21.15.7:0/3725367952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f070c035030 con 0x7f0718107900 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.008+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071819a190 con 0x7f0718107900 2024-09-21T04:56:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.009+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0718103040 con 0x7f0718107900 2024-09-21T04:56:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.010+0000 7f07157fa640 1 -- 172.21.15.7:0/3725367952 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f070c042070 con 0x7f0718107900 2024-09-21T04:56:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.010+0000 7f07157fa640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f070007af90 0x7f070007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.010+0000 7f0717fff640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f070007af90 0x7f070007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.010+0000 7f07157fa640 1 -- 172.21.15.7:0/3725367952 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f070c0beb40 con 0x7f0718107900 2024-09-21T04:56:26.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.011+0000 7f0717fff640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f070007af90 0x7f070007d450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0708005fd0 tx=0x7f0708005ea0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.015+0000 7f07157fa640 1 -- 172.21.15.7:0/3725367952 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f070c004cf0 con 0x7f0718107900 2024-09-21T04:56:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.219+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f071819afd0 con 0x7f070007af90 2024-09-21T04:56:26.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.220+0000 7f07157fa640 1 -- 172.21.15.7:0/3725367952 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f071819afd0 con 0x7f070007af90 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f070007af90 msgr2=0x7f070007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f070007af90 0x7f070007d450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0708005fd0 tx=0x7f0708005ea0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718107900 msgr2=0x7f071819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718107900 0x7f071819fd50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f070c031460 tx=0x7f070c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 shutdown_connections 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f070007af90 0x7f070007d450 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0718107900 0x7f071819fd50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 --2- 172.21.15.7:0/3725367952 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0718101900 0x7f071819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 >> 172.21.15.7:0/3725367952 conn(0x7f07180fd600 msgr2=0x7f0718074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 shutdown_connections 2024-09-21T04:56:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.222+0000 7f071e352640 1 -- 172.21.15.7:0/3725367952 wait complete. 2024-09-21T04:56:26.234 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:56:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.308+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/2384986874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4102b50 msgr2=0x7fcfd4102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.308+0000 7fcfd8dfb640 1 --2- 172.21.15.7:0/2384986874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4102b50 0x7fcfd4102f30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009880 tx=0x7fcfc402a530 comp rx=0 tx=0).stop 2024-09-21T04:56:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.309+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/2384986874 shutdown_connections 2024-09-21T04:56:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.309+0000 7fcfd8dfb640 1 --2- 172.21.15.7:0/2384986874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4103470 0x7fcfd410b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.309+0000 7fcfd8dfb640 1 --2- 172.21.15.7:0/2384986874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4102b50 0x7fcfd4102f30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.309+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/2384986874 >> 172.21.15.7:0/2384986874 conn(0x7fcfd40fc550 msgr2=0x7fcfd40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.309+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/2384986874 shutdown_connections 2024-09-21T04:56:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.309+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/2384986874 wait complete. 2024-09-21T04:56:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd8dfb640 1 Processor -- start 2024-09-21T04:56:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd8dfb640 1 -- start start 2024-09-21T04:56:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd8dfb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4102b50 0x7fcfd419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd8dfb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4103470 0x7fcfd419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd8dfb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd410ede0 con 0x7fcfd4103470 2024-09-21T04:56:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd8dfb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd410ec60 con 0x7fcfd4102b50 2024-09-21T04:56:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4103470 0x7fcfd419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4103470 0x7fcfd419db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34006/0 (socket says 172.21.15.7:34006) 2024-09-21T04:56:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd1d9b640 1 -- 172.21.15.7:0/3873291477 learned_addr learned my addr 172.21.15.7:0/3873291477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd259c640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4102b50 0x7fcfd419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd1d9b640 1 -- 172.21.15.7:0/3873291477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4102b50 msgr2=0x7fcfd419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd1d9b640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4102b50 0x7fcfd419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.310+0000 7fcfd1d9b640 1 -- 172.21.15.7:0/3873291477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd41977d0 con 0x7fcfd4103470 2024-09-21T04:56:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.311+0000 7fcfd1d9b640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4103470 0x7fcfd419db90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800d6f0 tx=0x7fcfc800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.311+0000 7fcfc37fe640 1 -- 172.21.15.7:0/3873291477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcfc8017070 con 0x7fcfd4103470 2024-09-21T04:56:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.311+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd4197ac0 con 0x7fcfd4103470 2024-09-21T04:56:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.311+0000 7fcfc37fe640 1 -- 172.21.15.7:0/3873291477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcfc800de00 con 0x7fcfd4103470 2024-09-21T04:56:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.311+0000 7fcfc37fe640 1 -- 172.21.15.7:0/3873291477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcfc8004bb0 con 0x7fcfd4103470 2024-09-21T04:56:26.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.311+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd4198050 con 0x7fcfd4103470 2024-09-21T04:56:26.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.312+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd406a4e0 con 0x7fcfd4103470 2024-09-21T04:56:26.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.313+0000 7fcfc37fe640 1 -- 172.21.15.7:0/3873291477 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fcfc8009a30 con 0x7fcfd4103470 2024-09-21T04:56:26.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.313+0000 7fcfc37fe640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfbc07af90 0x7fcfbc07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.313+0000 7fcfc37fe640 1 -- 172.21.15.7:0/3873291477 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcfc809db20 con 0x7fcfd4103470 2024-09-21T04:56:26.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.313+0000 7fcfd259c640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfbc07af90 0x7fcfbc07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.314+0000 7fcfd259c640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfbc07af90 0x7fcfbc07d450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40096c0 tx=0x7fcfc4004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:26.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.318+0000 7fcfc37fe640 1 -- 172.21.15.7:0/3873291477 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcfc8069140 con 0x7fcfd4103470 2024-09-21T04:56:26.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.522+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fcfd4198300 con 0x7fcfbc07af90 2024-09-21T04:56:26.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.523+0000 7fcfc37fe640 1 -- 172.21.15.7:0/3873291477 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fcfd4198300 con 0x7fcfbc07af90 2024-09-21T04:56:26.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfbc07af90 msgr2=0x7fcfbc07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfbc07af90 0x7fcfbc07d450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40096c0 tx=0x7fcfc4004f30 comp rx=0 tx=0).stop 2024-09-21T04:56:26.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4103470 msgr2=0x7fcfd419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4103470 0x7fcfd419db90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800d6f0 tx=0x7fcfc800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 shutdown_connections 2024-09-21T04:56:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcfbc07af90 0x7fcfbc07d450 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd4103470 0x7fcfd419db90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 --2- 172.21.15.7:0/3873291477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfd4102b50 0x7fcfd419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 >> 172.21.15.7:0/3873291477 conn(0x7fcfd40fc550 msgr2=0x7fcfd4108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 shutdown_connections 2024-09-21T04:56:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.525+0000 7fcfd8dfb640 1 -- 172.21.15.7:0/3873291477 wait complete. 2024-09-21T04:56:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.609+0000 7ff4e5e23640 1 -- 172.21.15.7:0/4222969510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 msgr2=0x7ff4e0105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.609+0000 7ff4e5e23640 1 --2- 172.21.15.7:0/4222969510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 0x7ff4e0105690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009880 tx=0x7ff4d402a530 comp rx=0 tx=0).stop 2024-09-21T04:56:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.609+0000 7ff4e5e23640 1 -- 172.21.15.7:0/4222969510 shutdown_connections 2024-09-21T04:56:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.609+0000 7ff4e5e23640 1 --2- 172.21.15.7:0/4222969510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 0x7ff4e0105690 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.609+0000 7ff4e5e23640 1 --2- 172.21.15.7:0/4222969510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e01008a0 0x7ff4e0100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.609+0000 7ff4e5e23640 1 -- 172.21.15.7:0/4222969510 >> 172.21.15.7:0/4222969510 conn(0x7ff4e00fc550 msgr2=0x7ff4e00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.609+0000 7ff4e5e23640 1 -- 172.21.15.7:0/4222969510 shutdown_connections 2024-09-21T04:56:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.609+0000 7ff4e5e23640 1 -- 172.21.15.7:0/4222969510 wait complete. 2024-09-21T04:56:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.610+0000 7ff4e5e23640 1 Processor -- start 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.610+0000 7ff4e5e23640 1 -- start start 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4e5e23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e01008a0 0x7ff4e019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4e5e23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 0x7ff4e019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4e5e23640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01153c0 con 0x7ff4e01011c0 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4e5e23640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0115240 con 0x7ff4e01008a0 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4deffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 0x7ff4e019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4deffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 0x7ff4e019e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34036/0 (socket says 172.21.15.7:34036) 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4deffd640 1 -- 172.21.15.7:0/2005995273 learned_addr learned my addr 172.21.15.7:0/2005995273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4df7fe640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e01008a0 0x7ff4e019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4deffd640 1 -- 172.21.15.7:0/2005995273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e01008a0 msgr2=0x7ff4e019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4deffd640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e01008a0 0x7ff4e019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4deffd640 1 -- 172.21.15.7:0/2005995273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e019eb30 con 0x7ff4e01011c0 2024-09-21T04:56:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4deffd640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 0x7ff4e019e480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4031460 tx=0x7ff4d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.611+0000 7ff4dcff9640 1 -- 172.21.15.7:0/2005995273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff4d403f070 con 0x7ff4e01011c0 2024-09-21T04:56:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.612+0000 7ff4dcff9640 1 -- 172.21.15.7:0/2005995273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4d4004ac0 con 0x7ff4e01011c0 2024-09-21T04:56:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.612+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e01a2870 con 0x7ff4e01011c0 2024-09-21T04:56:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.612+0000 7ff4dcff9640 1 -- 172.21.15.7:0/2005995273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff4d4035030 con 0x7ff4e01011c0 2024-09-21T04:56:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.612+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e01a2da0 con 0x7ff4e01011c0 2024-09-21T04:56:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.613+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e006a580 con 0x7ff4e01011c0 2024-09-21T04:56:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.614+0000 7ff4dcff9640 1 -- 172.21.15.7:0/2005995273 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff4d4042070 con 0x7ff4e01011c0 2024-09-21T04:56:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.614+0000 7ff4dcff9640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff4c807af80 0x7ff4c807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.614+0000 7ff4dcff9640 1 -- 172.21.15.7:0/2005995273 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff4d40be930 con 0x7ff4e01011c0 2024-09-21T04:56:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.614+0000 7ff4df7fe640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff4c807af80 0x7ff4c807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.615+0000 7ff4df7fe640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff4c807af80 0x7ff4c807d440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000abe0 tx=0x7ff4d000a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:26.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.618+0000 7ff4dcff9640 1 -- 172.21.15.7:0/2005995273 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4d4089f50 con 0x7ff4e01011c0 2024-09-21T04:56:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.812+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff4e0102810 con 0x7ff4c807af80 2024-09-21T04:56:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.822+0000 7ff4dcff9640 1 -- 172.21.15.7:0/2005995273 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7ff4e0102810 con 0x7ff4c807af80 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (18m) 12m ago 29m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (21m) 12m ago 29m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (21m) 12m ago 28m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (16m) 12m ago 29m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (16m) 12m ago 28m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (18m) 12m ago 29m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (12m) 12m ago 24m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (12m) 12m ago 24m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (23m) 12m ago 23m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:56:26.826 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (12m) 12m ago 24m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:56:26.827 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (20m) 12m ago 30m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:56:26.827 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (19m) 12m ago 28m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:56:26.827 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (17m) 12m ago 30m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:56:26.827 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (16m) 12m ago 28m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:56:26.827 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (19m) 12m ago 29m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:56:26.827 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (19m) 12m ago 28m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:56:26.827 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (16m) 12m ago 27m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:56:26.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (15m) 12m ago 26m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:56:26.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (14m) 12m ago 26m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:56:26.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (14m) 12m ago 26m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:56:26.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (13m) 12m ago 25m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:56:26.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (13m) 12m ago 25m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:56:26.828 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (18m) 12m ago 29m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff4c807af80 msgr2=0x7ff4c807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff4c807af80 0x7ff4c807d440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000abe0 tx=0x7ff4d000a2d0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 msgr2=0x7ff4e019e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 0x7ff4e019e480 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4031460 tx=0x7ff4d40046c0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 shutdown_connections 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff4c807af80 0x7ff4c807d440 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01011c0 0x7ff4e019e480 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 --2- 172.21.15.7:0/2005995273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4e01008a0 0x7ff4e019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 >> 172.21.15.7:0/2005995273 conn(0x7ff4e00fc550 msgr2=0x7ff4e0112b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 shutdown_connections 2024-09-21T04:56:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.825+0000 7ff4e5e23640 1 -- 172.21.15.7:0/2005995273 wait complete. 2024-09-21T04:56:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.908+0000 7fbfac444640 1 -- 172.21.15.7:0/1742531224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4107900 msgr2=0x7fbfa4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.908+0000 7fbfac444640 1 --2- 172.21.15.7:0/1742531224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4107900 0x7fbfa4107ce0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009880 tx=0x7fbfa002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.908+0000 7fbfac444640 1 -- 172.21.15.7:0/1742531224 shutdown_connections 2024-09-21T04:56:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.908+0000 7fbfac444640 1 --2- 172.21.15.7:0/1742531224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfa4101900 0x7fbfa4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.908+0000 7fbfac444640 1 --2- 172.21.15.7:0/1742531224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4107900 0x7fbfa4107ce0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.908+0000 7fbfac444640 1 -- 172.21.15.7:0/1742531224 >> 172.21.15.7:0/1742531224 conn(0x7fbfa40fd600 msgr2=0x7fbfa40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.908+0000 7fbfac444640 1 -- 172.21.15.7:0/1742531224 shutdown_connections 2024-09-21T04:56:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.908+0000 7fbfac444640 1 -- 172.21.15.7:0/1742531224 wait complete. 2024-09-21T04:56:26.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfac444640 1 Processor -- start 2024-09-21T04:56:26.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfac444640 1 -- start start 2024-09-21T04:56:26.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfac444640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4101900 0x7fbfa419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfac444640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfa4107900 0x7fbfa419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfac444640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa410edb0 con 0x7fbfa4101900 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfac444640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa410ec30 con 0x7fbfa4107900 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfaa1e0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4101900 0x7fbfa419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfaa1e0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4101900 0x7fbfa419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34042/0 (socket says 172.21.15.7:34042) 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.909+0000 7fbfaa1e0640 1 -- 172.21.15.7:0/3079953212 learned_addr learned my addr 172.21.15.7:0/3079953212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbfa99df640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfa4107900 0x7fbfa419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbfaa1e0640 1 -- 172.21.15.7:0/3079953212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfa4107900 msgr2=0x7fbfa419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbfaa1e0640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfa4107900 0x7fbfa419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbfaa1e0640 1 -- 172.21.15.7:0/3079953212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa41977c0 con 0x7fbfa4101900 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbfa99df640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfa4107900 0x7fbfa419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbfaa1e0640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4101900 0x7fbfa419d640 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0031460 tx=0x7fbfa00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbf9b7fe640 1 -- 172.21.15.7:0/3079953212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbfa003f070 con 0x7fbfa4101900 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbf9b7fe640 1 -- 172.21.15.7:0/3079953212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbfa0004cf0 con 0x7fbfa4101900 2024-09-21T04:56:26.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa4197a50 con 0x7fbfa4101900 2024-09-21T04:56:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa4197f80 con 0x7fbfa4101900 2024-09-21T04:56:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.910+0000 7fbf9b7fe640 1 -- 172.21.15.7:0/3079953212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbfa0035030 con 0x7fbfa4101900 2024-09-21T04:56:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.911+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf88005290 con 0x7fbfa4101900 2024-09-21T04:56:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.912+0000 7fbf9b7fe640 1 -- 172.21.15.7:0/3079953212 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbfa0042070 con 0x7fbfa4101900 2024-09-21T04:56:26.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.913+0000 7fbf9b7fe640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf9007af80 0x7fbf9007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.913+0000 7fbf9b7fe640 1 -- 172.21.15.7:0/3079953212 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fbfa00bea60 con 0x7fbfa4101900 2024-09-21T04:56:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.913+0000 7fbfa99df640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf9007af80 0x7fbf9007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:26.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.913+0000 7fbfa99df640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf9007af80 0x7fbf9007d440 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4100e00 tx=0x7fbf94005d90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:26.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:26.917+0000 7fbf9b7fe640 1 -- 172.21.15.7:0/3079953212 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbfa008a020 con 0x7fbfa4101900 2024-09-21T04:56:27.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.177+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fbf88005d90 con 0x7fbfa4101900 2024-09-21T04:56:27.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.178+0000 7fbf9b7fe640 1 -- 172.21.15.7:0/3079953212 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fbfa008c070 con 0x7fbfa4101900 2024-09-21T04:56:27.181 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:56:27.181 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:56:27.181 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:56:27.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:56:27.181 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:56:27.181 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:56:27.182 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:56:27.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf9007af80 msgr2=0x7fbf9007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf9007af80 0x7fbf9007d440 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4100e00 tx=0x7fbf94005d90 comp rx=0 tx=0).stop 2024-09-21T04:56:27.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4101900 msgr2=0x7fbfa419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4101900 0x7fbfa419d640 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0031460 tx=0x7fbfa00048f0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 shutdown_connections 2024-09-21T04:56:27.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf9007af80 0x7fbf9007d440 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfa4107900 0x7fbfa419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 --2- 172.21.15.7:0/3079953212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfa4101900 0x7fbfa419d640 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 >> 172.21.15.7:0/3079953212 conn(0x7fbfa40fd600 msgr2=0x7fbfa40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 shutdown_connections 2024-09-21T04:56:27.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.180+0000 7fbfac444640 1 -- 172.21.15.7:0/3079953212 wait complete. 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.264+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1135708401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 msgr2=0x7f6ff4100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.264+0000 7f6ff9a1d640 1 --2- 172.21.15.7:0/1135708401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 0x7f6ff4100220 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009880 tx=0x7f6fe802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.264+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1135708401 shutdown_connections 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.264+0000 7f6ff9a1d640 1 --2- 172.21.15.7:0/1135708401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 0x7f6ff4100220 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.264+0000 7f6ff9a1d640 1 --2- 172.21.15.7:0/1135708401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff41069a0 0x7f6ff4106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.264+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1135708401 >> 172.21.15.7:0/1135708401 conn(0x7f6ff40fbc90 msgr2=0x7f6ff40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.264+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1135708401 shutdown_connections 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.264+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1135708401 wait complete. 2024-09-21T04:56:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.265+0000 7f6ff9a1d640 1 Processor -- start 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.265+0000 7f6ff9a1d640 1 -- start start 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.265+0000 7f6ff9a1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 0x7f6ff419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.265+0000 7f6ff9a1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff41069a0 0x7f6ff419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.265+0000 7f6ff9a1d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff410ef40 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff9a1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff410edc0 con 0x7f6ff41069a0 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 0x7f6ff419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 0x7f6ff419d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34070/0 (socket says 172.21.15.7:34070) 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff2ffd640 1 -- 172.21.15.7:0/1505724866 learned_addr learned my addr 172.21.15.7:0/1505724866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff27fc640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff41069a0 0x7f6ff419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff2ffd640 1 -- 172.21.15.7:0/1505724866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff41069a0 msgr2=0x7f6ff419dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff2ffd640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff41069a0 0x7f6ff419dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff2ffd640 1 -- 172.21.15.7:0/1505724866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff41978f0 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff2ffd640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 0x7f6ff419d730 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400d6f0 tx=0x7f6fe400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff8a1b640 1 -- 172.21.15.7:0/1505724866 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6fe4017070 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff4197be0 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff8a1b640 1 -- 172.21.15.7:0/1505724866 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fe400de00 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff8a1b640 1 -- 172.21.15.7:0/1505724866 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6fe4004bb0 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.266+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff4198140 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.267+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff41030e0 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.268+0000 7f6ff8a1b640 1 -- 172.21.15.7:0/1505724866 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6fe4004e60 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.269+0000 7f6ff8a1b640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fdc07af30 0x7f6fdc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.269+0000 7f6ff8a1b640 1 -- 172.21.15.7:0/1505724866 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6fe409dcb0 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.269+0000 7f6ff27fc640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fdc07af30 0x7f6fdc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.269+0000 7f6ff27fc640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fdc07af30 0x7f6fdc07d3f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ff41990e0 tx=0x7f6fe8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.273+0000 7f6ff8a1b640 1 -- 172.21.15.7:0/1505724866 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6fe4069250 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:27 smithi007 ceph-mon[108238]: pgmap v650: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:27 smithi007 ceph-mon[108238]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:56:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:27 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3079953212' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:56:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.497+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6ff4109f00 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.498+0000 7f6ff8a1b640 1 -- 172.21.15.7:0/1505724866 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f6fe406e100 con 0x7f6ff40ffdc0 2024-09-21T04:56:27.501 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:56:27.501 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:56:27.501 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:56:27.501 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:56:27.501 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:56:27.501 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:56:27.501 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:56:27.502 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:56:27.503 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:56:27.504 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:56:27.505 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:56:27.505 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:56:27.505 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fdc07af30 msgr2=0x7f6fdc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fdc07af30 0x7f6fdc07d3f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ff41990e0 tx=0x7f6fe8031400 comp rx=0 tx=0).stop 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 msgr2=0x7f6ff419d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 0x7f6ff419d730 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400d6f0 tx=0x7f6fe400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 shutdown_connections 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6fdc07af30 0x7f6fdc07d3f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ff41069a0 0x7f6ff419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 --2- 172.21.15.7:0/1505724866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff40ffdc0 0x7f6ff419d730 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 >> 172.21.15.7:0/1505724866 conn(0x7f6ff40fbc90 msgr2=0x7f6ff40fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 shutdown_connections 2024-09-21T04:56:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.500+0000 7f6ff9a1d640 1 -- 172.21.15.7:0/1505724866 wait complete. 2024-09-21T04:56:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 -- 172.21.15.7:0/3907016611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1067f0 msgr2=0x7f160c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 --2- 172.21.15.7:0/3907016611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1067f0 0x7f160c106bd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1608009880 tx=0x7f160802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 -- 172.21.15.7:0/3907016611 shutdown_connections 2024-09-21T04:56:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 --2- 172.21.15.7:0/3907016611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c1007f0 0x7f160c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 --2- 172.21.15.7:0/3907016611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1067f0 0x7f160c106bd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 -- 172.21.15.7:0/3907016611 >> 172.21.15.7:0/3907016611 conn(0x7f160c0fc550 msgr2=0x7f160c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 -- 172.21.15.7:0/3907016611 shutdown_connections 2024-09-21T04:56:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 -- 172.21.15.7:0/3907016611 wait complete. 2024-09-21T04:56:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.586+0000 7f1613a0f640 1 Processor -- start 2024-09-21T04:56:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f1613a0f640 1 -- start start 2024-09-21T04:56:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f1613a0f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1007f0 0x7f160c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f1613a0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c1067f0 0x7f160c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f1613a0f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f160c10dcc0 con 0x7f160c1007f0 2024-09-21T04:56:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f1613a0f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f160c10db40 con 0x7f160c1067f0 2024-09-21T04:56:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f16117ab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1007f0 0x7f160c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f16117ab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1007f0 0x7f160c198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34086/0 (socket says 172.21.15.7:34086) 2024-09-21T04:56:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f16117ab640 1 -- 172.21.15.7:0/2380996593 learned_addr learned my addr 172.21.15.7:0/2380996593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f1610faa640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c1067f0 0x7f160c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f16117ab640 1 -- 172.21.15.7:0/2380996593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c1067f0 msgr2=0x7f160c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f16117ab640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c1067f0 0x7f160c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f16117ab640 1 -- 172.21.15.7:0/2380996593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c199350 con 0x7f160c1007f0 2024-09-21T04:56:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.587+0000 7f16117ab640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1007f0 0x7f160c198810 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1608031460 tx=0x7f1608004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.588+0000 7f16027fc640 1 -- 172.21.15.7:0/2380996593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f160803f070 con 0x7f160c1007f0 2024-09-21T04:56:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.588+0000 7f16027fc640 1 -- 172.21.15.7:0/2380996593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f16080035f0 con 0x7f160c1007f0 2024-09-21T04:56:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.588+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c1995e0 con 0x7f160c1007f0 2024-09-21T04:56:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.588+0000 7f16027fc640 1 -- 172.21.15.7:0/2380996593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1608035030 con 0x7f160c1007f0 2024-09-21T04:56:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.588+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c1a1b90 con 0x7f160c1007f0 2024-09-21T04:56:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.589+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160c101f30 con 0x7f160c1007f0 2024-09-21T04:56:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.589+0000 7f16027fc640 1 -- 172.21.15.7:0/2380996593 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1608042070 con 0x7f160c1007f0 2024-09-21T04:56:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.590+0000 7f16027fc640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f15f807af30 0x7f15f807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.590+0000 7f16027fc640 1 -- 172.21.15.7:0/2380996593 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f16080be960 con 0x7f160c1007f0 2024-09-21T04:56:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.590+0000 7f1610faa640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f15f807af30 0x7f15f807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.590+0000 7f1610faa640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f15f807af30 0x7f15f807d3f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f160c06a7f0 tx=0x7f15fc005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.594+0000 7f16027fc640 1 -- 172.21.15.7:0/2380996593 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1608089f00 con 0x7f160c1007f0 2024-09-21T04:56:27.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:27 smithi053 ceph-mon[80176]: pgmap v650: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:27.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:27 smithi053 ceph-mon[80176]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:56:27.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:27 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3079953212' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:56:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.799+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f160c062630 con 0x7f15f807af30 2024-09-21T04:56:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.800+0000 7f16027fc640 1 -- 172.21.15.7:0/2380996593 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f160c062630 con 0x7f15f807af30 2024-09-21T04:56:27.803 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:56:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:56:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:56:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:56:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:56:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:56:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:56:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:56:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:56:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:56:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:56:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:56:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:56:27.805 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:56:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.802+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f15f807af30 msgr2=0x7f15f807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.802+0000 7f1613a0f640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f15f807af30 0x7f15f807d3f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f160c06a7f0 tx=0x7f15fc005f90 comp rx=0 tx=0).stop 2024-09-21T04:56:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.802+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1007f0 msgr2=0x7f160c198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.802+0000 7f1613a0f640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1007f0 0x7f160c198810 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1608031460 tx=0x7f1608004530 comp rx=0 tx=0).stop 2024-09-21T04:56:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.803+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 shutdown_connections 2024-09-21T04:56:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.803+0000 7f1613a0f640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f15f807af30 0x7f15f807d3f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.803+0000 7f1613a0f640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f160c1067f0 0x7f160c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.803+0000 7f1613a0f640 1 --2- 172.21.15.7:0/2380996593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f160c1007f0 0x7f160c198810 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.803+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 >> 172.21.15.7:0/2380996593 conn(0x7f160c0fc550 msgr2=0x7f160c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.803+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 shutdown_connections 2024-09-21T04:56:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.803+0000 7f1613a0f640 1 -- 172.21.15.7:0/2380996593 wait complete. 2024-09-21T04:56:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 -- 172.21.15.7:0/97615584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8900102ac0 msgr2=0x7f8900102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 --2- 172.21.15.7:0/97615584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8900102ac0 0x7f8900102ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009880 tx=0x7f88f002a530 comp rx=0 tx=0).stop 2024-09-21T04:56:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 -- 172.21.15.7:0/97615584 shutdown_connections 2024-09-21T04:56:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 --2- 172.21.15.7:0/97615584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89001033e0 0x7f890010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 --2- 172.21.15.7:0/97615584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8900102ac0 0x7f8900102ea0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 -- 172.21.15.7:0/97615584 >> 172.21.15.7:0/97615584 conn(0x7f89000fc520 msgr2=0x7f89000fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:27.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 -- 172.21.15.7:0/97615584 shutdown_connections 2024-09-21T04:56:27.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 -- 172.21.15.7:0/97615584 wait complete. 2024-09-21T04:56:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.887+0000 7f8906355640 1 Processor -- start 2024-09-21T04:56:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f8906355640 1 -- start start 2024-09-21T04:56:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f8906355640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8900102ac0 0x7f8900100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f8906355640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89001033e0 0x7f8900100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f8906355640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f890010edf0 con 0x7f89001033e0 2024-09-21T04:56:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f8906355640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f890010ec70 con 0x7f8900102ac0 2024-09-21T04:56:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89001033e0 0x7f8900100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89001033e0 0x7f8900100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34092/0 (socket says 172.21.15.7:34092) 2024-09-21T04:56:27.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f88ff7fe640 1 -- 172.21.15.7:0/2145611972 learned_addr learned my addr 172.21.15.7:0/2145611972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:27.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f88fffff640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8900102ac0 0x7f8900100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f88ff7fe640 1 -- 172.21.15.7:0/2145611972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8900102ac0 msgr2=0x7f8900100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:27.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f88ff7fe640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8900102ac0 0x7f8900100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:27.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.888+0000 7f88ff7fe640 1 -- 172.21.15.7:0/2145611972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89001011a0 con 0x7f89001033e0 2024-09-21T04:56:27.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.889+0000 7f88ff7fe640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89001033e0 0x7f8900100c60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f88f400d6d0 tx=0x7f88f400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.889+0000 7f88fd7fa640 1 -- 172.21.15.7:0/2145611972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f88f4017070 con 0x7f89001033e0 2024-09-21T04:56:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.889+0000 7f88fd7fa640 1 -- 172.21.15.7:0/2145611972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f88f40054e0 con 0x7f89001033e0 2024-09-21T04:56:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.889+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8900101490 con 0x7f89001033e0 2024-09-21T04:56:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.889+0000 7f88fd7fa640 1 -- 172.21.15.7:0/2145611972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f88f4004ae0 con 0x7f89001033e0 2024-09-21T04:56:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.889+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001a3e60 con 0x7f89001033e0 2024-09-21T04:56:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.890+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890006aa90 con 0x7f89001033e0 2024-09-21T04:56:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.891+0000 7f88fd7fa640 1 -- 172.21.15.7:0/2145611972 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f88f4004d90 con 0x7f89001033e0 2024-09-21T04:56:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.891+0000 7f88fd7fa640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f88e807af80 0x7f88e807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.891+0000 7f88fffff640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f88e807af80 0x7f88e807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.891+0000 7f88fd7fa640 1 -- 172.21.15.7:0/2145611972 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f88f409daf0 con 0x7f89001033e0 2024-09-21T04:56:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.892+0000 7f88fffff640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f88e807af80 0x7f88e807d440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f88f00096c0 tx=0x7f88f0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:27.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:27.896+0000 7f88fd7fa640 1 -- 172.21.15.7:0/2145611972 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88f4069110 con 0x7f89001033e0 2024-09-21T04:56:28.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.159+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f8900102ea0 con 0x7f89001033e0 2024-09-21T04:56:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.160+0000 7f88fd7fa640 1 -- 172.21.15.7:0/2145611972 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f88f406dfc0 con 0x7f89001033e0 2024-09-21T04:56:28.163 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:56:28.163 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:56:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:56:28.163 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:56:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:56:28.163 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:56:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:56:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f88e807af80 msgr2=0x7f88e807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f88e807af80 0x7f88e807d440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f88f00096c0 tx=0x7f88f0031400 comp rx=0 tx=0).stop 2024-09-21T04:56:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89001033e0 msgr2=0x7f8900100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89001033e0 0x7f8900100c60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f88f400d6d0 tx=0x7f88f400db90 comp rx=0 tx=0).stop 2024-09-21T04:56:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 shutdown_connections 2024-09-21T04:56:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f88e807af80 0x7f88e807d440 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89001033e0 0x7f8900100c60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 --2- 172.21.15.7:0/2145611972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8900102ac0 0x7f8900100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 >> 172.21.15.7:0/2145611972 conn(0x7f89000fc520 msgr2=0x7f8900108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 shutdown_connections 2024-09-21T04:56:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:28.162+0000 7f8906355640 1 -- 172.21.15.7:0/2145611972 wait complete. 2024-09-21T04:56:28.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:28 smithi007 ceph-mon[108238]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:56:28.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:28 smithi007 ceph-mon[108238]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:56:28.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1505724866' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:56:28.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2145611972' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:56:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:28 smithi053 ceph-mon[80176]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:56:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:28 smithi053 ceph-mon[80176]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:56:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1505724866' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:56:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2145611972' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:56:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:29 smithi053 ceph-mon[80176]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:56:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:29 smithi053 ceph-mon[80176]: pgmap v651: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:29.703 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:29 smithi007 ceph-mon[108238]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:56:29.789 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:29 smithi007 ceph-mon[108238]: pgmap v651: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:29.789 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:31.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:31 smithi053 ceph-mon[80176]: pgmap v652: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:31.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:31 smithi007 ceph-mon[108238]: pgmap v652: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:33.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:33 smithi053 ceph-mon[80176]: pgmap v653: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:33 smithi007 ceph-mon[108238]: pgmap v653: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:35.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:35 smithi007 ceph-mon[108238]: pgmap v654: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:35.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:35 smithi053 ceph-mon[80176]: pgmap v654: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:37 smithi053 ceph-mon[80176]: pgmap v655: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:37 smithi007 ceph-mon[108238]: pgmap v655: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:39.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:39 smithi053 ceph-mon[80176]: pgmap v656: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:39 smithi007 ceph-mon[108238]: pgmap v656: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:41 smithi053 ceph-mon[80176]: pgmap v657: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:41 smithi007 ceph-mon[108238]: pgmap v657: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:43 smithi053 ceph-mon[80176]: pgmap v658: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:43 smithi007 ceph-mon[108238]: pgmap v658: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:44.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:45.561 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:45 smithi007 ceph-mon[108238]: pgmap v659: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:45.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:45 smithi053 ceph-mon[80176]: pgmap v659: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:47.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:47 smithi053 ceph-mon[80176]: pgmap v660: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:47 smithi007 ceph-mon[108238]: pgmap v660: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:49 smithi053 ceph-mon[80176]: pgmap v661: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:49.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:49 smithi007 ceph-mon[108238]: pgmap v661: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:50.637 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:50 smithi007 ceph-mon[108238]: pgmap v662: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:50 smithi053 ceph-mon[80176]: pgmap v662: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:53 smithi007 ceph-mon[108238]: pgmap v663: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:53 smithi053 ceph-mon[80176]: pgmap v663: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:55 smithi053 ceph-mon[80176]: pgmap v664: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:55 smithi007 ceph-mon[108238]: pgmap v664: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:57.366 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:57 smithi053 ceph-mon[80176]: pgmap v665: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:57 smithi007 ceph-mon[108238]: pgmap v665: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:58.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.259+0000 7f4485199640 1 -- 172.21.15.7:0/3097512113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480101900 msgr2=0x7f4480101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.259+0000 7f4485199640 1 --2- 172.21.15.7:0/3097512113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480101900 0x7f4480101d60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4468009880 tx=0x7f446802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.259+0000 7f4485199640 1 -- 172.21.15.7:0/3097512113 shutdown_connections 2024-09-21T04:56:58.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.259+0000 7f4485199640 1 --2- 172.21.15.7:0/3097512113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480101900 0x7f4480101d60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.259+0000 7f4485199640 1 --2- 172.21.15.7:0/3097512113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4480107900 0x7f4480107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.259+0000 7f4485199640 1 -- 172.21.15.7:0/3097512113 >> 172.21.15.7:0/3097512113 conn(0x7f44800fd600 msgr2=0x7f44800ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:58.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.260+0000 7f4485199640 1 -- 172.21.15.7:0/3097512113 shutdown_connections 2024-09-21T04:56:58.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.260+0000 7f4485199640 1 -- 172.21.15.7:0/3097512113 wait complete. 2024-09-21T04:56:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.260+0000 7f4485199640 1 Processor -- start 2024-09-21T04:56:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f4485199640 1 -- start start 2024-09-21T04:56:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f4485199640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4480101900 0x7f448019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f4485199640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480107900 0x7f448019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f4485199640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f448010edb0 con 0x7f4480107900 2024-09-21T04:56:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f4485199640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f448010ec30 con 0x7f4480101900 2024-09-21T04:56:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f447e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480107900 0x7f448019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f447e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480107900 0x7f448019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34648/0 (socket says 172.21.15.7:34648) 2024-09-21T04:56:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f447e59c640 1 -- 172.21.15.7:0/218056767 learned_addr learned my addr 172.21.15.7:0/218056767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.261+0000 7f447ed9d640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4480101900 0x7f448019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f447e59c640 1 -- 172.21.15.7:0/218056767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4480101900 msgr2=0x7f448019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f447e59c640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4480101900 0x7f448019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f447e59c640 1 -- 172.21.15.7:0/218056767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44801977d0 con 0x7f4480107900 2024-09-21T04:56:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f447e59c640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480107900 0x7f448019db90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4468031460 tx=0x7f44680045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f4477fff640 1 -- 172.21.15.7:0/218056767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f446803f070 con 0x7f4480107900 2024-09-21T04:56:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4480197a60 con 0x7f4480107900 2024-09-21T04:56:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f4477fff640 1 -- 172.21.15.7:0/218056767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44680035f0 con 0x7f4480107900 2024-09-21T04:56:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f4477fff640 1 -- 172.21.15.7:0/218056767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4468035030 con 0x7f4480107900 2024-09-21T04:56:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.262+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4480197f90 con 0x7f4480107900 2024-09-21T04:56:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.264+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4480103040 con 0x7f4480107900 2024-09-21T04:56:58.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.264+0000 7f4477fff640 1 -- 172.21.15.7:0/218056767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4468042070 con 0x7f4480107900 2024-09-21T04:56:58.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.265+0000 7f4477fff640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f446c07af30 0x7f446c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.265+0000 7f4477fff640 1 -- 172.21.15.7:0/218056767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f44680be9d0 con 0x7f4480107900 2024-09-21T04:56:58.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.265+0000 7f447ed9d640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f446c07af30 0x7f446c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.266+0000 7f447ed9d640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f446c07af30 0x7f446c07d3f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f447000bbe0 tx=0x7f447000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.269+0000 7f4477fff640 1 -- 172.21.15.7:0/218056767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4468089f70 con 0x7f4480107900 2024-09-21T04:56:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.478+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4480198300 con 0x7f446c07af30 2024-09-21T04:56:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.479+0000 7f4477fff640 1 -- 172.21.15.7:0/218056767 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f4480198300 con 0x7f446c07af30 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f446c07af30 msgr2=0x7f446c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f446c07af30 0x7f446c07d3f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f447000bbe0 tx=0x7f447000b390 comp rx=0 tx=0).stop 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480107900 msgr2=0x7f448019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480107900 0x7f448019db90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4468031460 tx=0x7f44680045e0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 shutdown_connections 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f446c07af30 0x7f446c07d3f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4480107900 0x7f448019db90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 --2- 172.21.15.7:0/218056767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4480101900 0x7f448019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 >> 172.21.15.7:0/218056767 conn(0x7f44800fd600 msgr2=0x7f44800fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 shutdown_connections 2024-09-21T04:56:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.482+0000 7f4485199640 1 -- 172.21.15.7:0/218056767 wait complete. 2024-09-21T04:56:58.495 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:56:58.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.568+0000 7f4e99183640 1 -- 172.21.15.7:0/2605161405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 msgr2=0x7f4e9410b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.568+0000 7f4e99183640 1 --2- 172.21.15.7:0/2605161405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 0x7f4e9410b930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4e880098b0 tx=0x7f4e8802a560 comp rx=0 tx=0).stop 2024-09-21T04:56:58.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.568+0000 7f4e99183640 1 -- 172.21.15.7:0/2605161405 shutdown_connections 2024-09-21T04:56:58.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.568+0000 7f4e99183640 1 --2- 172.21.15.7:0/2605161405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 0x7f4e9410b930 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.568+0000 7f4e99183640 1 --2- 172.21.15.7:0/2605161405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e94102b20 0x7f4e94102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.568+0000 7f4e99183640 1 -- 172.21.15.7:0/2605161405 >> 172.21.15.7:0/2605161405 conn(0x7f4e940fc520 msgr2=0x7f4e940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:58.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.568+0000 7f4e99183640 1 -- 172.21.15.7:0/2605161405 shutdown_connections 2024-09-21T04:56:58.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.568+0000 7f4e99183640 1 -- 172.21.15.7:0/2605161405 wait complete. 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.569+0000 7f4e99183640 1 Processor -- start 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.569+0000 7f4e99183640 1 -- start start 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.569+0000 7f4e99183640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e94102b20 0x7f4e9419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e99183640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 0x7f4e9419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e99183640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9410edd0 con 0x7f4e94103440 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e99183640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9410ec50 con 0x7f4e94102b20 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e9259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 0x7f4e9419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e9259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 0x7f4e9419dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34678/0 (socket says 172.21.15.7:34678) 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e9259c640 1 -- 172.21.15.7:0/717007453 learned_addr learned my addr 172.21.15.7:0/717007453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e92d9d640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e94102b20 0x7f4e9419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e9259c640 1 -- 172.21.15.7:0/717007453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e94102b20 msgr2=0x7f4e9419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e9259c640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e94102b20 0x7f4e9419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e9259c640 1 -- 172.21.15.7:0/717007453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e94197820 con 0x7f4e94103440 2024-09-21T04:56:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.570+0000 7f4e9259c640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 0x7f4e9419dbe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4e88031460 tx=0x7f4e880045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.571+0000 7f4e7bfff640 1 -- 172.21.15.7:0/717007453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e8803f070 con 0x7f4e94103440 2024-09-21T04:56:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.571+0000 7f4e7bfff640 1 -- 172.21.15.7:0/717007453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e880035f0 con 0x7f4e94103440 2024-09-21T04:56:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.571+0000 7f4e7bfff640 1 -- 172.21.15.7:0/717007453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e88035030 con 0x7f4e94103440 2024-09-21T04:56:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.571+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e94197ab0 con 0x7f4e94103440 2024-09-21T04:56:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.571+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e94197fe0 con 0x7f4e94103440 2024-09-21T04:56:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.572+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9406aa30 con 0x7f4e94103440 2024-09-21T04:56:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.572+0000 7f4e7bfff640 1 -- 172.21.15.7:0/717007453 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4e88042070 con 0x7f4e94103440 2024-09-21T04:56:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.573+0000 7f4e7bfff640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e7c07af80 0x7f4e7c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.573+0000 7f4e92d9d640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e7c07af80 0x7f4e7c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.573+0000 7f4e7bfff640 1 -- 172.21.15.7:0/717007453 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4e880be840 con 0x7f4e94103440 2024-09-21T04:56:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.574+0000 7f4e92d9d640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e7c07af80 0x7f4e7c07d440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005d00 tx=0x7f4e8400b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:58.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.578+0000 7f4e7bfff640 1 -- 172.21.15.7:0/717007453 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e88089e60 con 0x7f4e94103440 2024-09-21T04:56:58.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.781+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4e94198290 con 0x7f4e7c07af80 2024-09-21T04:56:58.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.783+0000 7f4e7bfff640 1 -- 172.21.15.7:0/717007453 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f4e94198290 con 0x7f4e7c07af80 2024-09-21T04:56:58.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e7c07af80 msgr2=0x7f4e7c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e7c07af80 0x7f4e7c07d440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005d00 tx=0x7f4e8400b410 comp rx=0 tx=0).stop 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 msgr2=0x7f4e9419dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 0x7f4e9419dbe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4e88031460 tx=0x7f4e880045c0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 shutdown_connections 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4e7c07af80 0x7f4e7c07d440 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e94103440 0x7f4e9419dbe0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 --2- 172.21.15.7:0/717007453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e94102b20 0x7f4e9419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 >> 172.21.15.7:0/717007453 conn(0x7f4e940fc520 msgr2=0x7f4e94103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 shutdown_connections 2024-09-21T04:56:58.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.785+0000 7f4e99183640 1 -- 172.21.15.7:0/717007453 wait complete. 2024-09-21T04:56:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.869+0000 7f0b84456640 1 -- 172.21.15.7:0/3029650351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c100650 msgr2=0x7f0b7c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.869+0000 7f0b84456640 1 --2- 172.21.15.7:0/3029650351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c100650 0x7f0b7c100a30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009880 tx=0x7f0b7002a490 comp rx=0 tx=0).stop 2024-09-21T04:56:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.869+0000 7f0b84456640 1 -- 172.21.15.7:0/3029650351 shutdown_connections 2024-09-21T04:56:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.869+0000 7f0b84456640 1 --2- 172.21.15.7:0/3029650351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c101000 0x7f0b7c10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.869+0000 7f0b84456640 1 --2- 172.21.15.7:0/3029650351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c100650 0x7f0b7c100a30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.869+0000 7f0b84456640 1 -- 172.21.15.7:0/3029650351 >> 172.21.15.7:0/3029650351 conn(0x7f0b7c0fc520 msgr2=0x7f0b7c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.869+0000 7f0b84456640 1 -- 172.21.15.7:0/3029650351 shutdown_connections 2024-09-21T04:56:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.869+0000 7f0b84456640 1 -- 172.21.15.7:0/3029650351 wait complete. 2024-09-21T04:56:58.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.870+0000 7f0b84456640 1 Processor -- start 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.870+0000 7f0b84456640 1 -- start start 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b84456640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c100650 0x7f0b7c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b84456640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c101000 0x7f0b7c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b84456640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c110f90 con 0x7f0b7c101000 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b84456640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c110e10 con 0x7f0b7c100650 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b819f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c101000 0x7f0b7c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b819f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c101000 0x7f0b7c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34702/0 (socket says 172.21.15.7:34702) 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b819f1640 1 -- 172.21.15.7:0/3255510817 learned_addr learned my addr 172.21.15.7:0/3255510817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b821f2640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c100650 0x7f0b7c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b819f1640 1 -- 172.21.15.7:0/3255510817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c100650 msgr2=0x7f0b7c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b819f1640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c100650 0x7f0b7c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b819f1640 1 -- 172.21.15.7:0/3255510817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c199980 con 0x7f0b7c101000 2024-09-21T04:56:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.871+0000 7f0b819f1640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c101000 0x7f0b7c19fd00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800e970 tx=0x7f0b7800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.872+0000 7f0b6f7fe640 1 -- 172.21.15.7:0/3255510817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0b7800cac0 con 0x7f0b7c101000 2024-09-21T04:56:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.872+0000 7f0b6f7fe640 1 -- 172.21.15.7:0/3255510817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b7800cc60 con 0x7f0b7c101000 2024-09-21T04:56:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.872+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c199c70 con 0x7f0b7c101000 2024-09-21T04:56:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.872+0000 7f0b6f7fe640 1 -- 172.21.15.7:0/3255510817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0b78018720 con 0x7f0b7c101000 2024-09-21T04:56:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.872+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c19a200 con 0x7f0b7c101000 2024-09-21T04:56:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.873+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c06aa90 con 0x7f0b7c101000 2024-09-21T04:56:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.873+0000 7f0b6f7fe640 1 -- 172.21.15.7:0/3255510817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0b78010430 con 0x7f0b7c101000 2024-09-21T04:56:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.874+0000 7f0b6f7fe640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b6807af40 0x7f0b6807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.874+0000 7f0b821f2640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b6807af40 0x7f0b6807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.874+0000 7f0b6f7fe640 1 -- 172.21.15.7:0/3255510817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0b7809e680 con 0x7f0b7c101000 2024-09-21T04:56:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.874+0000 7f0b821f2640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b6807af40 0x7f0b6807d400 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0b70000c00 tx=0x7f0b70031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:58.878+0000 7f0b6f7fe640 1 -- 172.21.15.7:0/3255510817 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b78069c20 con 0x7f0b7c101000 2024-09-21T04:56:59.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.080+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0b7c10b410 con 0x7f0b6807af40 2024-09-21T04:56:59.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.090+0000 7f0b6f7fe640 1 -- 172.21.15.7:0/3255510817 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f0b70004260 con 0x7f0b6807af40 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (19m) 13m ago 29m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (22m) 13m ago 30m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (21m) 12m ago 29m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (17m) 13m ago 30m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (17m) 12m ago 29m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (18m) 13m ago 29m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (13m) 13m ago 24m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (13m) 13m ago 24m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (24m) 12m ago 24m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (12m) 12m ago 24m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (21m) 13m ago 31m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (20m) 12m ago 29m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:56:59.094 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (17m) 13m ago 31m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (17m) 12m ago 29m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (19m) 13m ago 30m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (19m) 12m ago 29m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (16m) 13m ago 27m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (16m) 13m ago 27m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (15m) 13m ago 26m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (14m) 12m ago 26m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (14m) 12m ago 26m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (13m) 12m ago 25m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:56:59.095 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (19m) 13m ago 29m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:56:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b6807af40 msgr2=0x7f0b6807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b6807af40 0x7f0b6807d400 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0b70000c00 tx=0x7f0b70031400 comp rx=0 tx=0).stop 2024-09-21T04:56:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c101000 msgr2=0x7f0b7c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c101000 0x7f0b7c19fd00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800e970 tx=0x7f0b7800ee30 comp rx=0 tx=0).stop 2024-09-21T04:56:59.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 shutdown_connections 2024-09-21T04:56:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0b6807af40 0x7f0b6807d400 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b7c101000 0x7f0b7c19fd00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 --2- 172.21.15.7:0/3255510817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b7c100650 0x7f0b7c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 >> 172.21.15.7:0/3255510817 conn(0x7f0b7c0fc520 msgr2=0x7f0b7c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 shutdown_connections 2024-09-21T04:56:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.092+0000 7f0b84456640 1 -- 172.21.15.7:0/3255510817 wait complete. 2024-09-21T04:56:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.176+0000 7fd322645640 1 -- 172.21.15.7:0/2100673566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 msgr2=0x7fd31c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.176+0000 7fd322645640 1 --2- 172.21.15.7:0/2100673566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 0x7fd31c0748f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009880 tx=0x7fd30c02a530 comp rx=0 tx=0).stop 2024-09-21T04:56:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.176+0000 7fd322645640 1 -- 172.21.15.7:0/2100673566 shutdown_connections 2024-09-21T04:56:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.176+0000 7fd322645640 1 --2- 172.21.15.7:0/2100673566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd31c074e30 0x7fd31c10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.176+0000 7fd322645640 1 --2- 172.21.15.7:0/2100673566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 0x7fd31c0748f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.176+0000 7fd322645640 1 -- 172.21.15.7:0/2100673566 >> 172.21.15.7:0/2100673566 conn(0x7fd31c0fd600 msgr2=0x7fd31c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:59.181 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:59 smithi007 ceph-mon[108238]: pgmap v666: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:59.181 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:56:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.177+0000 7fd322645640 1 -- 172.21.15.7:0/2100673566 shutdown_connections 2024-09-21T04:56:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.177+0000 7fd322645640 1 -- 172.21.15.7:0/2100673566 wait complete. 2024-09-21T04:56:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.177+0000 7fd322645640 1 Processor -- start 2024-09-21T04:56:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.177+0000 7fd322645640 1 -- start start 2024-09-21T04:56:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.177+0000 7fd322645640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 0x7fd31c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd322645640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd31c074e30 0x7fd31c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd322645640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c110fe0 con 0x7fd31c074510 2024-09-21T04:56:59.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd322645640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c110e60 con 0x7fd31c074e30 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd31bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 0x7fd31c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd31bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 0x7fd31c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34714/0 (socket says 172.21.15.7:34714) 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd31bfff640 1 -- 172.21.15.7:0/2379339249 learned_addr learned my addr 172.21.15.7:0/2379339249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd31b7fe640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd31c074e30 0x7fd31c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd31bfff640 1 -- 172.21.15.7:0/2379339249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd31c074e30 msgr2=0x7fd31c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd31bfff640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd31c074e30 0x7fd31c19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd31bfff640 1 -- 172.21.15.7:0/2379339249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c1999f0 con 0x7fd31c074510 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd31bfff640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 0x7fd31c19f830 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd30c031460 tx=0x7fd30c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.178+0000 7fd3197fa640 1 -- 172.21.15.7:0/2379339249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd30c03f070 con 0x7fd31c074510 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.179+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c199c80 con 0x7fd31c074510 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.179+0000 7fd3197fa640 1 -- 172.21.15.7:0/2379339249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd30c0035f0 con 0x7fd31c074510 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.179+0000 7fd3197fa640 1 -- 172.21.15.7:0/2379339249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd30c035030 con 0x7fd31c074510 2024-09-21T04:56:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.179+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c19a1b0 con 0x7fd31c074510 2024-09-21T04:56:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.180+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c075c50 con 0x7fd31c074510 2024-09-21T04:56:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.181+0000 7fd3197fa640 1 -- 172.21.15.7:0/2379339249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd30c042070 con 0x7fd31c074510 2024-09-21T04:56:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.181+0000 7fd3197fa640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd30407af30 0x7fd30407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.181+0000 7fd3197fa640 1 -- 172.21.15.7:0/2379339249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd30c0be960 con 0x7fd31c074510 2024-09-21T04:56:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.181+0000 7fd31b7fe640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd30407af30 0x7fd30407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.182+0000 7fd31b7fe640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd30407af30 0x7fd30407d3f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd31c10acb0 tx=0x7fd310005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.186+0000 7fd3197fa640 1 -- 172.21.15.7:0/2379339249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd30c089f00 con 0x7fd31c074510 2024-09-21T04:56:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:59 smithi053 ceph-mon[80176]: pgmap v666: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:56:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:56:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:56:59.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.449+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fd31c19af20 con 0x7fd31c074510 2024-09-21T04:56:59.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.450+0000 7fd3197fa640 1 -- 172.21.15.7:0/2379339249 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fd30c08c420 con 0x7fd31c074510 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:56:59.454 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:56:59.455 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:56:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd30407af30 msgr2=0x7fd30407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd30407af30 0x7fd30407d3f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd31c10acb0 tx=0x7fd310005f90 comp rx=0 tx=0).stop 2024-09-21T04:56:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 msgr2=0x7fd31c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 0x7fd31c19f830 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd30c031460 tx=0x7fd30c004530 comp rx=0 tx=0).stop 2024-09-21T04:56:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 shutdown_connections 2024-09-21T04:56:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd30407af30 0x7fd30407d3f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd31c074e30 0x7fd31c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 --2- 172.21.15.7:0/2379339249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31c074510 0x7fd31c19f830 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 >> 172.21.15.7:0/2379339249 conn(0x7fd31c0fd600 msgr2=0x7fd31c10b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 shutdown_connections 2024-09-21T04:56:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.452+0000 7fd322645640 1 -- 172.21.15.7:0/2379339249 wait complete. 2024-09-21T04:56:59.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.536+0000 7f85b5435640 1 -- 172.21.15.7:0/3021800445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0101900 msgr2=0x7f85b0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.536+0000 7f85b5435640 1 --2- 172.21.15.7:0/3021800445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0101900 0x7f85b0101d60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009880 tx=0x7f85a402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.536+0000 7f85b5435640 1 -- 172.21.15.7:0/3021800445 shutdown_connections 2024-09-21T04:56:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.536+0000 7f85b5435640 1 --2- 172.21.15.7:0/3021800445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0101900 0x7f85b0101d60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.536+0000 7f85b5435640 1 --2- 172.21.15.7:0/3021800445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85b0107900 0x7f85b0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.536+0000 7f85b5435640 1 -- 172.21.15.7:0/3021800445 >> 172.21.15.7:0/3021800445 conn(0x7f85b00fd600 msgr2=0x7f85b00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.536+0000 7f85b5435640 1 -- 172.21.15.7:0/3021800445 shutdown_connections 2024-09-21T04:56:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.536+0000 7f85b5435640 1 -- 172.21.15.7:0/3021800445 wait complete. 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.537+0000 7f85b5435640 1 Processor -- start 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.537+0000 7f85b5435640 1 -- start start 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85b5435640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85b0101900 0x7f85b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85b5435640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0107900 0x7f85b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85b5435640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010ee30 con 0x7f85b0107900 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85b5435640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010ecb0 con 0x7f85b0101900 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85ae7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0107900 0x7f85b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85ae7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0107900 0x7f85b0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34718/0 (socket says 172.21.15.7:34718) 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85ae7fc640 1 -- 172.21.15.7:0/1259507369 learned_addr learned my addr 172.21.15.7:0/1259507369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85aeffd640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85b0101900 0x7f85b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85ae7fc640 1 -- 172.21.15.7:0/1259507369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85b0101900 msgr2=0x7f85b00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85ae7fc640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85b0101900 0x7f85b00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85ae7fc640 1 -- 172.21.15.7:0/1259507369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b0074f70 con 0x7f85b0107900 2024-09-21T04:56:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85ae7fc640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0107900 0x7f85b0074a30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f85a40099a0 tx=0x7f85a40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f8597fff640 1 -- 172.21.15.7:0/1259507369 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f85a403f070 con 0x7f85b0107900 2024-09-21T04:56:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f8597fff640 1 -- 172.21.15.7:0/1259507369 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85a4004c70 con 0x7f85b0107900 2024-09-21T04:56:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b0075200 con 0x7f85b0107900 2024-09-21T04:56:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f8597fff640 1 -- 172.21.15.7:0/1259507369 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f85a4035030 con 0x7f85b0107900 2024-09-21T04:56:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.538+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01a3e00 con 0x7f85b0107900 2024-09-21T04:56:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.539+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b0103040 con 0x7f85b0107900 2024-09-21T04:56:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.540+0000 7f8597fff640 1 -- 172.21.15.7:0/1259507369 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f85a4042070 con 0x7f85b0107900 2024-09-21T04:56:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.541+0000 7f8597fff640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f859807afe0 0x7f859807d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.541+0000 7f8597fff640 1 -- 172.21.15.7:0/1259507369 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f85a40bec40 con 0x7f85b0107900 2024-09-21T04:56:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.541+0000 7f85aeffd640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f859807afe0 0x7f859807d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.541+0000 7f85aeffd640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f859807afe0 0x7f859807d4a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009890 tx=0x7f85a0008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.545+0000 7f8597fff640 1 -- 172.21.15.7:0/1259507369 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f85a408a260 con 0x7f85b0107900 2024-09-21T04:56:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.773+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f85b0109e90 con 0x7f85b0107900 2024-09-21T04:56:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.773+0000 7f8597fff640 1 -- 172.21.15.7:0/1259507369 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f85a408f110 con 0x7f85b0107900 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:56:59.778 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:56:59.779 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:56:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:56:59.781 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:56:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:56:59.781 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:56:59.781 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:56:59.781 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:56:59.781 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f859807afe0 msgr2=0x7f859807d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f859807afe0 0x7f859807d4a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009890 tx=0x7f85a0008040 comp rx=0 tx=0).stop 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0107900 msgr2=0x7f85b0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0107900 0x7f85b0074a30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f85a40099a0 tx=0x7f85a40048f0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 shutdown_connections 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f859807afe0 0x7f859807d4a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b0107900 0x7f85b0074a30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 --2- 172.21.15.7:0/1259507369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85b0101900 0x7f85b00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 >> 172.21.15.7:0/1259507369 conn(0x7f85b00fd600 msgr2=0x7f85b00fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 shutdown_connections 2024-09-21T04:56:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.777+0000 7f85b5435640 1 -- 172.21.15.7:0/1259507369 wait complete. 2024-09-21T04:56:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 -- 172.21.15.7:0/2998635764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1007f0 msgr2=0x7f05cc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 --2- 172.21.15.7:0/2998635764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1007f0 0x7f05cc100c50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009880 tx=0x7f05c002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 -- 172.21.15.7:0/2998635764 shutdown_connections 2024-09-21T04:56:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 --2- 172.21.15.7:0/2998635764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1007f0 0x7f05cc100c50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 --2- 172.21.15.7:0/2998635764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05cc1067f0 0x7f05cc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 -- 172.21.15.7:0/2998635764 >> 172.21.15.7:0/2998635764 conn(0x7f05cc0fc550 msgr2=0x7f05cc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:56:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 -- 172.21.15.7:0/2998635764 shutdown_connections 2024-09-21T04:56:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 -- 172.21.15.7:0/2998635764 wait complete. 2024-09-21T04:56:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.864+0000 7f05d145a640 1 Processor -- start 2024-09-21T04:56:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05d145a640 1 -- start start 2024-09-21T04:56:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05d145a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05cc1007f0 0x7f05cc19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05d145a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1067f0 0x7f05cc19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05d145a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc10dcc0 con 0x7f05cc1067f0 2024-09-21T04:56:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05d145a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc10db40 con 0x7f05cc1007f0 2024-09-21T04:56:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05ca7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1067f0 0x7f05cc19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05caffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05cc1007f0 0x7f05cc19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05caffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05cc1007f0 0x7f05cc19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:58480/0 (socket says 172.21.15.7:58480) 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05caffd640 1 -- 172.21.15.7:0/721454975 learned_addr learned my addr 172.21.15.7:0/721454975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05ca7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1067f0 0x7f05cc19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34740/0 (socket says 172.21.15.7:34740) 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05ca7fc640 1 -- 172.21.15.7:0/721454975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05cc1007f0 msgr2=0x7f05cc19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05ca7fc640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05cc1007f0 0x7f05cc19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05ca7fc640 1 -- 172.21.15.7:0/721454975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05cc197780 con 0x7f05cc1067f0 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.865+0000 7f05caffd640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05cc1007f0 0x7f05cc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.866+0000 7f05ca7fc640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1067f0 0x7f05cc19db40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f05c0006010 tx=0x7f05c00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.866+0000 7f05b3fff640 1 -- 172.21.15.7:0/721454975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f05c003f070 con 0x7f05cc1067f0 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.866+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05cc197a10 con 0x7f05cc1067f0 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.866+0000 7f05b3fff640 1 -- 172.21.15.7:0/721454975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05c00035f0 con 0x7f05cc1067f0 2024-09-21T04:56:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.866+0000 7f05b3fff640 1 -- 172.21.15.7:0/721454975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f05c0035030 con 0x7f05cc1067f0 2024-09-21T04:56:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.866+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05cc197f40 con 0x7f05cc1067f0 2024-09-21T04:56:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.867+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05cc101f30 con 0x7f05cc1067f0 2024-09-21T04:56:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.868+0000 7f05b3fff640 1 -- 172.21.15.7:0/721454975 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f05c0042070 con 0x7f05cc1067f0 2024-09-21T04:56:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.868+0000 7f05b3fff640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f05b407af30 0x7f05b407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:56:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.868+0000 7f05b3fff640 1 -- 172.21.15.7:0/721454975 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f05c00be9b0 con 0x7f05cc1067f0 2024-09-21T04:56:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.868+0000 7f05caffd640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f05b407af30 0x7f05b407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:56:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.869+0000 7f05caffd640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f05b407af30 0x7f05b407d3f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0104e0 tx=0x7f05bc00b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:56:59.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:56:59.873+0000 7f05b3fff640 1 -- 172.21.15.7:0/721454975 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f05c0089f50 con 0x7f05cc1067f0 2024-09-21T04:57:00.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.077+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f05cc1981f0 con 0x7f05b407af30 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.078+0000 7f05b3fff640 1 -- 172.21.15.7:0/721454975 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f05cc1981f0 con 0x7f05b407af30 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:57:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:57:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:57:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:57:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:57:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:57:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:57:00.083 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f05b407af30 msgr2=0x7f05b407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f05b407af30 0x7f05b407d3f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0104e0 tx=0x7f05bc00b410 comp rx=0 tx=0).stop 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1067f0 msgr2=0x7f05cc19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1067f0 0x7f05cc19db40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f05c0006010 tx=0x7f05c00045e0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 shutdown_connections 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f05b407af30 0x7f05b407d3f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05cc1067f0 0x7f05cc19db40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 --2- 172.21.15.7:0/721454975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05cc1007f0 0x7f05cc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 >> 172.21.15.7:0/721454975 conn(0x7f05cc0fc550 msgr2=0x7f05cc104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 shutdown_connections 2024-09-21T04:57:00.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.081+0000 7f05d145a640 1 -- 172.21.15.7:0/721454975 wait complete. 2024-09-21T04:57:00.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.166+0000 7f951d7f6640 1 -- 172.21.15.7:0/366531743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518101900 msgr2=0x7f9518101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:00.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.166+0000 7f951d7f6640 1 --2- 172.21.15.7:0/366531743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518101900 0x7f9518101d60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9500009880 tx=0x7f950002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.166+0000 7f951d7f6640 1 -- 172.21.15.7:0/366531743 shutdown_connections 2024-09-21T04:57:00.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.166+0000 7f951d7f6640 1 --2- 172.21.15.7:0/366531743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518101900 0x7f9518101d60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.166+0000 7f951d7f6640 1 --2- 172.21.15.7:0/366531743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9518107900 0x7f9518107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.166+0000 7f951d7f6640 1 -- 172.21.15.7:0/366531743 >> 172.21.15.7:0/366531743 conn(0x7f95180fd600 msgr2=0x7f95180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.166+0000 7f951d7f6640 1 -- 172.21.15.7:0/366531743 shutdown_connections 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.166+0000 7f951d7f6640 1 -- 172.21.15.7:0/366531743 wait complete. 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f951d7f6640 1 Processor -- start 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f951d7f6640 1 -- start start 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f951d7f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9518101900 0x7f951819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f951d7f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518107900 0x7f951819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f951d7f6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f951810edb0 con 0x7f9518107900 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f951d7f6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f951810ec30 con 0x7f9518101900 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f95167fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518107900 0x7f951819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f95167fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518107900 0x7f951819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34756/0 (socket says 172.21.15.7:34756) 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f95167fc640 1 -- 172.21.15.7:0/2594522951 learned_addr learned my addr 172.21.15.7:0/2594522951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.167+0000 7f9516ffd640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9518101900 0x7f951819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:00.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.168+0000 7f95167fc640 1 -- 172.21.15.7:0/2594522951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9518101900 msgr2=0x7f951819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:00.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.168+0000 7f95167fc640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9518101900 0x7f951819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.168+0000 7f95167fc640 1 -- 172.21.15.7:0/2594522951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9518197800 con 0x7f9518107900 2024-09-21T04:57:00.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.168+0000 7f95167fc640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518107900 0x7f951819dbc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9500031460 tx=0x7f9500004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:00.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.169+0000 7f950ffff640 1 -- 172.21.15.7:0/2594522951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f950003f070 con 0x7f9518107900 2024-09-21T04:57:00.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.169+0000 7f950ffff640 1 -- 172.21.15.7:0/2594522951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95000035f0 con 0x7f9518107900 2024-09-21T04:57:00.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.169+0000 7f950ffff640 1 -- 172.21.15.7:0/2594522951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9500035030 con 0x7f9518107900 2024-09-21T04:57:00.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.169+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9518197a90 con 0x7f9518107900 2024-09-21T04:57:00.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.169+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9518197fc0 con 0x7f9518107900 2024-09-21T04:57:00.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.170+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9518103040 con 0x7f9518107900 2024-09-21T04:57:00.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.171+0000 7f950ffff640 1 -- 172.21.15.7:0/2594522951 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9500042070 con 0x7f9518107900 2024-09-21T04:57:00.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.171+0000 7f950ffff640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f950407af80 0x7f950407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:00.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.172+0000 7f950ffff640 1 -- 172.21.15.7:0/2594522951 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f95000be890 con 0x7f9518107900 2024-09-21T04:57:00.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.172+0000 7f9516ffd640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f950407af80 0x7f950407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:00.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.172+0000 7f9516ffd640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f950407af80 0x7f950407d440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9508009bd0 tx=0x7f9508005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:00.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.176+0000 7f950ffff640 1 -- 172.21.15.7:0/2594522951 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9500089eb0 con 0x7f9518107900 2024-09-21T04:57:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:00 smithi053 ceph-mon[80176]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:00 smithi053 ceph-mon[80176]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:00 smithi053 ceph-mon[80176]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:00 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2379339249' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:57:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:00 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1259507369' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:57:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:00 smithi007 ceph-mon[108238]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:00 smithi007 ceph-mon[108238]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:00 smithi007 ceph-mon[108238]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:00 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2379339249' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:57:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:00 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1259507369' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:57:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.444+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9518198d30 con 0x7f9518107900 2024-09-21T04:57:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.445+0000 7f950ffff640 1 -- 172.21.15.7:0/2594522951 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f950008ed60 con 0x7f9518107900 2024-09-21T04:57:00.449 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:57:00.449 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:57:00.449 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:57:00.449 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:57:00.449 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:57:00.449 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:57:00.449 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:57:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f950407af80 msgr2=0x7f950407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f950407af80 0x7f950407d440 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9508009bd0 tx=0x7f9508005f90 comp rx=0 tx=0).stop 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518107900 msgr2=0x7f951819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518107900 0x7f951819dbc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9500031460 tx=0x7f9500004530 comp rx=0 tx=0).stop 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 shutdown_connections 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f950407af80 0x7f950407d440 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9518107900 0x7f951819dbc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 --2- 172.21.15.7:0/2594522951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9518101900 0x7f951819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 >> 172.21.15.7:0/2594522951 conn(0x7f95180fd600 msgr2=0x7f95180fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 shutdown_connections 2024-09-21T04:57:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:00.447+0000 7f951d7f6640 1 -- 172.21.15.7:0/2594522951 wait complete. 2024-09-21T04:57:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:01 smithi053 ceph-mon[80176]: pgmap v667: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:01 smithi053 ceph-mon[80176]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:01 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2594522951' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:57:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:01 smithi007 ceph-mon[108238]: pgmap v667: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:01 smithi007 ceph-mon[108238]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:01 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2594522951' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:57:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:03 smithi053 ceph-mon[80176]: pgmap v668: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:03 smithi007 ceph-mon[108238]: pgmap v668: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:05 smithi007 ceph-mon[108238]: pgmap v669: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:05 smithi053 ceph-mon[80176]: pgmap v669: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:07 smithi053 ceph-mon[80176]: pgmap v670: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:07 smithi007 ceph-mon[108238]: pgmap v670: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:09 smithi053 ceph-mon[80176]: pgmap v671: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:09 smithi007 ceph-mon[108238]: pgmap v671: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:11 smithi007 ceph-mon[108238]: pgmap v672: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:11 smithi053 ceph-mon[80176]: pgmap v672: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:13 smithi053 ceph-mon[80176]: pgmap v673: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:13 smithi007 ceph-mon[108238]: pgmap v673: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:57:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:57:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:15 smithi053 ceph-mon[80176]: pgmap v674: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:15 smithi007 ceph-mon[108238]: pgmap v674: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:17 smithi007 ceph-mon[108238]: pgmap v675: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:17 smithi053 ceph-mon[80176]: pgmap v675: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:19 smithi053 ceph-mon[80176]: pgmap v676: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:19 smithi007 ceph-mon[108238]: pgmap v676: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:21 smithi053 ceph-mon[80176]: pgmap v677: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:21 smithi007 ceph-mon[108238]: pgmap v677: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:23 smithi007 ceph-mon[108238]: pgmap v678: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:23.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:23 smithi053 ceph-mon[80176]: pgmap v678: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:25 smithi053 ceph-mon[80176]: pgmap v679: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:25 smithi007 ceph-mon[108238]: pgmap v679: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:27 smithi053 ceph-mon[80176]: pgmap v680: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:27 smithi007 ceph-mon[108238]: pgmap v680: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:29 smithi007 ceph-mon[108238]: pgmap v681: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:57:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:29 smithi053 ceph-mon[80176]: pgmap v681: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:57:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.546+0000 7f562812e640 1 -- 172.21.15.7:0/3067243480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 msgr2=0x7f5620107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.546+0000 7f562812e640 1 --2- 172.21.15.7:0/3067243480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 0x7f5620107ce0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f561c009880 tx=0x7f561c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.546+0000 7f562812e640 1 -- 172.21.15.7:0/3067243480 shutdown_connections 2024-09-21T04:57:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.546+0000 7f562812e640 1 --2- 172.21.15.7:0/3067243480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5620101900 0x7f5620101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.546+0000 7f562812e640 1 --2- 172.21.15.7:0/3067243480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 0x7f5620107ce0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.546+0000 7f562812e640 1 -- 172.21.15.7:0/3067243480 >> 172.21.15.7:0/3067243480 conn(0x7f56200fd600 msgr2=0x7f56200ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.546+0000 7f562812e640 1 -- 172.21.15.7:0/3067243480 shutdown_connections 2024-09-21T04:57:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.546+0000 7f562812e640 1 -- 172.21.15.7:0/3067243480 wait complete. 2024-09-21T04:57:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.547+0000 7f562812e640 1 Processor -- start 2024-09-21T04:57:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.547+0000 7f562812e640 1 -- start start 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.547+0000 7f562812e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5620101900 0x7f562019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.547+0000 7f562812e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 0x7f562019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.547+0000 7f562812e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f562010edb0 con 0x7f5620107900 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.547+0000 7f562812e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f562010ec30 con 0x7f5620101900 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f56256c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 0x7f562019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f56256c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 0x7f562019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59254/0 (socket says 172.21.15.7:59254) 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f56256c9640 1 -- 172.21.15.7:0/3006394933 learned_addr learned my addr 172.21.15.7:0/3006394933 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f5625eca640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5620101900 0x7f562019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f56256c9640 1 -- 172.21.15.7:0/3006394933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5620101900 msgr2=0x7f562019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f56256c9640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5620101900 0x7f562019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f56256c9640 1 -- 172.21.15.7:0/3006394933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56201977e0 con 0x7f5620107900 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f56256c9640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 0x7f562019dba0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f561000c8a0 tx=0x7f561000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f5616ffd640 1 -- 172.21.15.7:0/3006394933 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f56100079b0 con 0x7f5620107900 2024-09-21T04:57:30.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5620197ad0 con 0x7f5620107900 2024-09-21T04:57:30.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f5616ffd640 1 -- 172.21.15.7:0/3006394933 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5610007b50 con 0x7f5620107900 2024-09-21T04:57:30.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.548+0000 7f5616ffd640 1 -- 172.21.15.7:0/3006394933 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5610018700 con 0x7f5620107900 2024-09-21T04:57:30.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.549+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5620198060 con 0x7f5620107900 2024-09-21T04:57:30.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.550+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5620103040 con 0x7f5620107900 2024-09-21T04:57:30.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.550+0000 7f5616ffd640 1 -- 172.21.15.7:0/3006394933 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5610007d00 con 0x7f5620107900 2024-09-21T04:57:30.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.551+0000 7f5616ffd640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f560c07af30 0x7f560c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:30.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.551+0000 7f5616ffd640 1 -- 172.21.15.7:0/3006394933 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f561009e370 con 0x7f5620107900 2024-09-21T04:57:30.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.551+0000 7f5625eca640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f560c07af30 0x7f560c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:30.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.552+0000 7f5625eca640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f560c07af30 0x7f560c07d3f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f561c000c00 tx=0x7f561c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.556+0000 7f5616ffd640 1 -- 172.21.15.7:0/3006394933 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5610014030 con 0x7f5620107900 2024-09-21T04:57:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.764+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5620198370 con 0x7f560c07af30 2024-09-21T04:57:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.765+0000 7f5616ffd640 1 -- 172.21.15.7:0/3006394933 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f5620198370 con 0x7f560c07af30 2024-09-21T04:57:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f560c07af30 msgr2=0x7f560c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f560c07af30 0x7f560c07d3f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f561c000c00 tx=0x7f561c031400 comp rx=0 tx=0).stop 2024-09-21T04:57:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 msgr2=0x7f562019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 0x7f562019dba0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f561000c8a0 tx=0x7f561000cd60 comp rx=0 tx=0).stop 2024-09-21T04:57:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 shutdown_connections 2024-09-21T04:57:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f560c07af30 0x7f560c07d3f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5620107900 0x7f562019dba0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 --2- 172.21.15.7:0/3006394933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5620101900 0x7f562019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 >> 172.21.15.7:0/3006394933 conn(0x7f56200fd600 msgr2=0x7f56200fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 shutdown_connections 2024-09-21T04:57:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.767+0000 7f562812e640 1 -- 172.21.15.7:0/3006394933 wait complete. 2024-09-21T04:57:30.779 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:57:30.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.853+0000 7f0290ad4640 1 -- 172.21.15.7:0/3921865086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c101900 msgr2=0x7f028c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:30.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.853+0000 7f0290ad4640 1 --2- 172.21.15.7:0/3921865086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c101900 0x7f028c101d60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0280009880 tx=0x7f028002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.853+0000 7f0290ad4640 1 -- 172.21.15.7:0/3921865086 shutdown_connections 2024-09-21T04:57:30.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.853+0000 7f0290ad4640 1 --2- 172.21.15.7:0/3921865086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c101900 0x7f028c101d60 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.853+0000 7f0290ad4640 1 --2- 172.21.15.7:0/3921865086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c107900 0x7f028c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.853+0000 7f0290ad4640 1 -- 172.21.15.7:0/3921865086 >> 172.21.15.7:0/3921865086 conn(0x7f028c0fd600 msgr2=0x7f028c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:30.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.853+0000 7f0290ad4640 1 -- 172.21.15.7:0/3921865086 shutdown_connections 2024-09-21T04:57:30.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.853+0000 7f0290ad4640 1 -- 172.21.15.7:0/3921865086 wait complete. 2024-09-21T04:57:30.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0290ad4640 1 Processor -- start 2024-09-21T04:57:30.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0290ad4640 1 -- start start 2024-09-21T04:57:30.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0290ad4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c101900 0x7f028c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:30.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0290ad4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c107900 0x7f028c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0290ad4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10edd0 con 0x7f028c107900 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0290ad4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f028c10ec50 con 0x7f028c101900 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0289d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c107900 0x7f028c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0289d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c107900 0x7f028c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59274/0 (socket says 172.21.15.7:59274) 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f0289d9b640 1 -- 172.21.15.7:0/1290257909 learned_addr learned my addr 172.21.15.7:0/1290257909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.854+0000 7f028a59c640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c101900 0x7f028c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f0289d9b640 1 -- 172.21.15.7:0/1290257909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c101900 msgr2=0x7f028c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f0289d9b640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c101900 0x7f028c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f0289d9b640 1 -- 172.21.15.7:0/1290257909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c074f70 con 0x7f028c107900 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f0289d9b640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c107900 0x7f028c074a30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0280031460 tx=0x7f02800048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f027b7fe640 1 -- 172.21.15.7:0/1290257909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f028003f070 con 0x7f028c107900 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c075200 con 0x7f028c107900 2024-09-21T04:57:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f027b7fe640 1 -- 172.21.15.7:0/1290257909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0280004cf0 con 0x7f028c107900 2024-09-21T04:57:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f027b7fe640 1 -- 172.21.15.7:0/1290257909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0280035030 con 0x7f028c107900 2024-09-21T04:57:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.855+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c1a3d60 con 0x7f028c107900 2024-09-21T04:57:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.856+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028c103040 con 0x7f028c107900 2024-09-21T04:57:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.857+0000 7f027b7fe640 1 -- 172.21.15.7:0/1290257909 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0280042070 con 0x7f028c107900 2024-09-21T04:57:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.858+0000 7f027b7fe640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f027407af30 0x7f027407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.858+0000 7f027b7fe640 1 -- 172.21.15.7:0/1290257909 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f02800bea70 con 0x7f028c107900 2024-09-21T04:57:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.858+0000 7f028a59c640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f027407af30 0x7f027407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.858+0000 7f028a59c640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f027407af30 0x7f027407d3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f027c0058c0 tx=0x7f027c00a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:30.862+0000 7f027b7fe640 1 -- 172.21.15.7:0/1290257909 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f028008a010 con 0x7f028c107900 2024-09-21T04:57:31.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.068+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f028c062630 con 0x7f027407af30 2024-09-21T04:57:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.069+0000 7f027b7fe640 1 -- 172.21.15.7:0/1290257909 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f028c062630 con 0x7f027407af30 2024-09-21T04:57:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f027407af30 msgr2=0x7f027407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f027407af30 0x7f027407d3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f027c0058c0 tx=0x7f027c00a250 comp rx=0 tx=0).stop 2024-09-21T04:57:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c107900 msgr2=0x7f028c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c107900 0x7f028c074a30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0280031460 tx=0x7f02800048f0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 shutdown_connections 2024-09-21T04:57:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f027407af30 0x7f027407d3f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c107900 0x7f028c074a30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 --2- 172.21.15.7:0/1290257909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f028c101900 0x7f028c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 >> 172.21.15.7:0/1290257909 conn(0x7f028c0fd600 msgr2=0x7f028c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 shutdown_connections 2024-09-21T04:57:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.071+0000 7f0290ad4640 1 -- 172.21.15.7:0/1290257909 wait complete. 2024-09-21T04:57:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 -- 172.21.15.7:0/2837126631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 msgr2=0x7f4634101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 --2- 172.21.15.7:0/2837126631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 0x7f4634101d60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4628009880 tx=0x7f462802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 -- 172.21.15.7:0/2837126631 shutdown_connections 2024-09-21T04:57:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 --2- 172.21.15.7:0/2837126631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 0x7f4634101d60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 --2- 172.21.15.7:0/2837126631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4634107900 0x7f4634107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 -- 172.21.15.7:0/2837126631 >> 172.21.15.7:0/2837126631 conn(0x7f46340fd600 msgr2=0x7f46340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 -- 172.21.15.7:0/2837126631 shutdown_connections 2024-09-21T04:57:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 -- 172.21.15.7:0/2837126631 wait complete. 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.156+0000 7f463bb25640 1 Processor -- start 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f463bb25640 1 -- start start 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f463bb25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 0x7f46340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f463bb25640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4634107900 0x7f4634074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f463bb25640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f463410edd0 con 0x7f4634101900 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f463bb25640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f463410ec50 con 0x7f4634107900 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f46398c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 0x7f46340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f46398c1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 0x7f46340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59294/0 (socket says 172.21.15.7:59294) 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f46398c1640 1 -- 172.21.15.7:0/4139539950 learned_addr learned my addr 172.21.15.7:0/4139539950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f46390c0640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4634107900 0x7f4634074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f46398c1640 1 -- 172.21.15.7:0/4139539950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4634107900 msgr2=0x7f4634074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f46398c1640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4634107900 0x7f4634074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.157+0000 7f46398c1640 1 -- 172.21.15.7:0/4139539950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4634074f70 con 0x7f4634101900 2024-09-21T04:57:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.158+0000 7f46398c1640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 0x7f46340744f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f463000d6f0 tx=0x7f463000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.158+0000 7f4626ffd640 1 -- 172.21.15.7:0/4139539950 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4630017070 con 0x7f4634101900 2024-09-21T04:57:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.158+0000 7f4626ffd640 1 -- 172.21.15.7:0/4139539950 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f463000de00 con 0x7f4634101900 2024-09-21T04:57:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.158+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4634075260 con 0x7f4634101900 2024-09-21T04:57:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.158+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341a3e80 con 0x7f4634101900 2024-09-21T04:57:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.158+0000 7f4626ffd640 1 -- 172.21.15.7:0/4139539950 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4630004bb0 con 0x7f4634101900 2024-09-21T04:57:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.159+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4634103040 con 0x7f4634101900 2024-09-21T04:57:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.160+0000 7f4626ffd640 1 -- 172.21.15.7:0/4139539950 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4630004e60 con 0x7f4634101900 2024-09-21T04:57:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.160+0000 7f4626ffd640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f462007af30 0x7f462007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.161+0000 7f46390c0640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f462007af30 0x7f462007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.161+0000 7f4626ffd640 1 -- 172.21.15.7:0/4139539950 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f463009dca0 con 0x7f4634101900 2024-09-21T04:57:31.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.161+0000 7f46390c0640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f462007af30 0x7f462007d3f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4634075f80 tx=0x7f4628031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:31.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.165+0000 7f4626ffd640 1 -- 172.21.15.7:0/4139539950 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46300a2070 con 0x7f4634101900 2024-09-21T04:57:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.370+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4634062630 con 0x7f462007af30 2024-09-21T04:57:31.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.380+0000 7f4626ffd640 1 -- 172.21.15.7:0/4139539950 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f4634062630 con 0x7f462007af30 2024-09-21T04:57:31.383 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:57:31.383 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (19m) 13m ago 30m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:57:31.383 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (22m) 13m ago 30m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:57:31.383 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (22m) 13m ago 29m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:57:31.383 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (17m) 13m ago 30m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (17m) 13m ago 29m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (19m) 13m ago 30m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (13m) 13m ago 25m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (13m) 13m ago 25m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (25m) 13m ago 25m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (13m) 13m ago 25m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (21m) 13m ago 31m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (20m) 13m ago 29m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (18m) 13m ago 31m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (17m) 13m ago 29m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (20m) 13m ago 30m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (20m) 13m ago 29m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (17m) 13m ago 28m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:57:31.384 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (16m) 13m ago 28m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:57:31.385 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (16m) 13m ago 27m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:57:31.385 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (15m) 13m ago 27m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:57:31.385 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (14m) 13m ago 26m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:57:31.385 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (14m) 13m ago 26m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:57:31.385 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (19m) 13m ago 30m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f462007af30 msgr2=0x7f462007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f462007af30 0x7f462007d3f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4634075f80 tx=0x7f4628031400 comp rx=0 tx=0).stop 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 msgr2=0x7f46340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 0x7f46340744f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f463000d6f0 tx=0x7f463000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 shutdown_connections 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f462007af30 0x7f462007d3f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4634107900 0x7f4634074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 --2- 172.21.15.7:0/4139539950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4634101900 0x7f46340744f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 >> 172.21.15.7:0/4139539950 conn(0x7f46340fd600 msgr2=0x7f46340fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.382+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 shutdown_connections 2024-09-21T04:57:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.383+0000 7f463bb25640 1 -- 172.21.15.7:0/4139539950 wait complete. 2024-09-21T04:57:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:31 smithi053 ceph-mon[80176]: pgmap v682: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:31 smithi007 ceph-mon[108238]: pgmap v682: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:31.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.466+0000 7fc407f50640 1 -- 172.21.15.7:0/3446974564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc400100fd0 msgr2=0x7fc400101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.466+0000 7fc407f50640 1 --2- 172.21.15.7:0/3446974564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc400100fd0 0x7fc400101430 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009880 tx=0x7fc3f002a530 comp rx=0 tx=0).stop 2024-09-21T04:57:31.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.466+0000 7fc407f50640 1 -- 172.21.15.7:0/3446974564 shutdown_connections 2024-09-21T04:57:31.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.466+0000 7fc407f50640 1 --2- 172.21.15.7:0/3446974564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc400100fd0 0x7fc400101430 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.466+0000 7fc407f50640 1 --2- 172.21.15.7:0/3446974564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4001068b0 0x7fc400106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.466+0000 7fc407f50640 1 -- 172.21.15.7:0/3446974564 >> 172.21.15.7:0/3446974564 conn(0x7fc4000fbc90 msgr2=0x7fc4000fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:31.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.466+0000 7fc407f50640 1 -- 172.21.15.7:0/3446974564 shutdown_connections 2024-09-21T04:57:31.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.466+0000 7fc407f50640 1 -- 172.21.15.7:0/3446974564 wait complete. 2024-09-21T04:57:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc407f50640 1 Processor -- start 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc407f50640 1 -- start start 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc407f50640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc400100fd0 0x7fc400196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc407f50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4001068b0 0x7fc400196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc407f50640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc40010ddd0 con 0x7fc4001068b0 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc407f50640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc40010dc50 con 0x7fc400100fd0 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc4054eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4001068b0 0x7fc400196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc4054eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4001068b0 0x7fc400196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59320/0 (socket says 172.21.15.7:59320) 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc4054eb640 1 -- 172.21.15.7:0/1637430987 learned_addr learned my addr 172.21.15.7:0/1637430987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc405cec640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc400100fd0 0x7fc400196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc4054eb640 1 -- 172.21.15.7:0/1637430987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc400100fd0 msgr2=0x7fc400196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc4054eb640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc400100fd0 0x7fc400196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.467+0000 7fc4054eb640 1 -- 172.21.15.7:0/1637430987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4001971a0 con 0x7fc4001068b0 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.468+0000 7fc405cec640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc400100fd0 0x7fc400196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:57:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.468+0000 7fc4054eb640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4001068b0 0x7fc400196c60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0031460 tx=0x7fc3f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.468+0000 7fc3f6ffd640 1 -- 172.21.15.7:0/1637430987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3f003f070 con 0x7fc4001068b0 2024-09-21T04:57:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.468+0000 7fc3f6ffd640 1 -- 172.21.15.7:0/1637430987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3f0004cf0 con 0x7fc4001068b0 2024-09-21T04:57:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.468+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc400197430 con 0x7fc4001068b0 2024-09-21T04:57:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.468+0000 7fc3f6ffd640 1 -- 172.21.15.7:0/1637430987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3f0035030 con 0x7fc4001068b0 2024-09-21T04:57:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.468+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4001a3f60 con 0x7fc4001068b0 2024-09-21T04:57:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.469+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc400101430 con 0x7fc4001068b0 2024-09-21T04:57:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.470+0000 7fc3f6ffd640 1 -- 172.21.15.7:0/1637430987 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc3f0042070 con 0x7fc4001068b0 2024-09-21T04:57:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.470+0000 7fc3f6ffd640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3ec07af80 0x7fc3ec07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.470+0000 7fc3f6ffd640 1 -- 172.21.15.7:0/1637430987 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc3f00bea40 con 0x7fc4001068b0 2024-09-21T04:57:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.470+0000 7fc405cec640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3ec07af80 0x7fc3ec07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.471+0000 7fc405cec640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3ec07af80 0x7fc3ec07d440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc006fd0 tx=0x7fc3fc008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:31.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.474+0000 7fc3f6ffd640 1 -- 172.21.15.7:0/1637430987 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3f008a060 con 0x7fc4001068b0 2024-09-21T04:57:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.741+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc400101fa0 con 0x7fc4001068b0 2024-09-21T04:57:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.742+0000 7fc3f6ffd640 1 -- 172.21.15.7:0/1637430987 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fc3f008ef10 con 0x7fc4001068b0 2024-09-21T04:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:57:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:57:31.747 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3ec07af80 msgr2=0x7fc3ec07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3ec07af80 0x7fc3ec07d440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc006fd0 tx=0x7fc3fc008040 comp rx=0 tx=0).stop 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4001068b0 msgr2=0x7fc400196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4001068b0 0x7fc400196c60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0031460 tx=0x7fc3f00048f0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 shutdown_connections 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc3ec07af80 0x7fc3ec07d440 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4001068b0 0x7fc400196c60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 --2- 172.21.15.7:0/1637430987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc400100fd0 0x7fc400196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 >> 172.21.15.7:0/1637430987 conn(0x7fc4000fbc90 msgr2=0x7fc4000fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 shutdown_connections 2024-09-21T04:57:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.745+0000 7fc407f50640 1 -- 172.21.15.7:0/1637430987 wait complete. 2024-09-21T04:57:31.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.829+0000 7f2a83fb0640 1 -- 172.21.15.7:0/3352969766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c101900 msgr2=0x7f2a7c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.829+0000 7f2a83fb0640 1 --2- 172.21.15.7:0/3352969766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c101900 0x7f2a7c101d60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009880 tx=0x7f2a6c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.829+0000 7f2a83fb0640 1 -- 172.21.15.7:0/3352969766 shutdown_connections 2024-09-21T04:57:31.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.829+0000 7f2a83fb0640 1 --2- 172.21.15.7:0/3352969766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c101900 0x7f2a7c101d60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.829+0000 7f2a83fb0640 1 --2- 172.21.15.7:0/3352969766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c107900 0x7f2a7c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.829+0000 7f2a83fb0640 1 -- 172.21.15.7:0/3352969766 >> 172.21.15.7:0/3352969766 conn(0x7f2a7c0fd600 msgr2=0x7f2a7c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:31.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.829+0000 7f2a83fb0640 1 -- 172.21.15.7:0/3352969766 shutdown_connections 2024-09-21T04:57:31.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.829+0000 7f2a83fb0640 1 -- 172.21.15.7:0/3352969766 wait complete. 2024-09-21T04:57:31.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a83fb0640 1 Processor -- start 2024-09-21T04:57:31.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a83fb0640 1 -- start start 2024-09-21T04:57:31.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a83fb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c101900 0x7f2a7c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a83fb0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c107900 0x7f2a7c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a83fb0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c10ee30 con 0x7f2a7c107900 2024-09-21T04:57:31.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a83fb0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c10ecb0 con 0x7f2a7c101900 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a8154b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c107900 0x7f2a7c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a8154b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c107900 0x7f2a7c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59344/0 (socket says 172.21.15.7:59344) 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a8154b640 1 -- 172.21.15.7:0/556090916 learned_addr learned my addr 172.21.15.7:0/556090916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a81d4c640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c101900 0x7f2a7c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a8154b640 1 -- 172.21.15.7:0/556090916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c101900 msgr2=0x7f2a7c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a8154b640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c101900 0x7f2a7c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a8154b640 1 -- 172.21.15.7:0/556090916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c197840 con 0x7f2a7c107900 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.830+0000 7f2a8154b640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c107900 0x7f2a7c19dc00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c031460 tx=0x7f2a6c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.831+0000 7f2a72ffd640 1 -- 172.21.15.7:0/556090916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a6c03f070 con 0x7f2a7c107900 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.831+0000 7f2a72ffd640 1 -- 172.21.15.7:0/556090916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a6c004cf0 con 0x7f2a7c107900 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.831+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c197ad0 con 0x7f2a7c107900 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.831+0000 7f2a72ffd640 1 -- 172.21.15.7:0/556090916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a6c035030 con 0x7f2a7c107900 2024-09-21T04:57:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.831+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c198000 con 0x7f2a7c107900 2024-09-21T04:57:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.832+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c103040 con 0x7f2a7c107900 2024-09-21T04:57:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.833+0000 7f2a72ffd640 1 -- 172.21.15.7:0/556090916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2a6c042070 con 0x7f2a7c107900 2024-09-21T04:57:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.833+0000 7f2a72ffd640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a6807af80 0x7f2a6807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.833+0000 7f2a81d4c640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a6807af80 0x7f2a6807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.833+0000 7f2a72ffd640 1 -- 172.21.15.7:0/556090916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2a6c0bea40 con 0x7f2a7c107900 2024-09-21T04:57:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.834+0000 7f2a81d4c640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a6807af80 0x7f2a6807d440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009890 tx=0x7f2a78008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:31.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:31.838+0000 7f2a72ffd640 1 -- 172.21.15.7:0/556090916 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a6c08a060 con 0x7f2a7c107900 2024-09-21T04:57:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.067+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2a7c109e90 con 0x7f2a7c107900 2024-09-21T04:57:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.067+0000 7f2a72ffd640 1 -- 172.21.15.7:0/556090916 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f2a6c08ef10 con 0x7f2a7c107900 2024-09-21T04:57:32.070 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:57:32.070 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:57:32.070 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:57:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:57:32.072 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:57:32.073 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:57:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:57:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a6807af80 msgr2=0x7f2a6807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a6807af80 0x7f2a6807d440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009890 tx=0x7f2a78008040 comp rx=0 tx=0).stop 2024-09-21T04:57:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c107900 msgr2=0x7f2a7c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c107900 0x7f2a7c19dc00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c031460 tx=0x7f2a6c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 shutdown_connections 2024-09-21T04:57:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a6807af80 0x7f2a6807d440 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c107900 0x7f2a7c19dc00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 --2- 172.21.15.7:0/556090916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c101900 0x7f2a7c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 >> 172.21.15.7:0/556090916 conn(0x7f2a7c0fd600 msgr2=0x7f2a7c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 shutdown_connections 2024-09-21T04:57:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.069+0000 7f2a83fb0640 1 -- 172.21.15.7:0/556090916 wait complete. 2024-09-21T04:57:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.155+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/2909492956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 msgr2=0x7fb0e0106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.155+0000 7fb0e4c3f640 1 --2- 172.21.15.7:0/2909492956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 0x7fb0e0106bf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009880 tx=0x7fb0d002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.155+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/2909492956 shutdown_connections 2024-09-21T04:57:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.155+0000 7fb0e4c3f640 1 --2- 172.21.15.7:0/2909492956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0e0100810 0x7fb0e0100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.155+0000 7fb0e4c3f640 1 --2- 172.21.15.7:0/2909492956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 0x7fb0e0106bf0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.155+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/2909492956 >> 172.21.15.7:0/2909492956 conn(0x7fb0e00fc510 msgr2=0x7fb0e00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.155+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/2909492956 shutdown_connections 2024-09-21T04:57:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.155+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/2909492956 wait complete. 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0e4c3f640 1 Processor -- start 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0e4c3f640 1 -- start start 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0e4c3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0e0100810 0x7fb0e00fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0e4c3f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 0x7fb0e00fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0e4c3f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e010dce0 con 0x7fb0e0106810 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0e4c3f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e010db60 con 0x7fb0e0100810 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 0x7fb0e00fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 0x7fb0e00fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59358/0 (socket says 172.21.15.7:59358) 2024-09-21T04:57:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0d7fff640 1 -- 172.21.15.7:0/659743628 learned_addr learned my addr 172.21.15.7:0/659743628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0de59c640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0e0100810 0x7fb0e00fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0d7fff640 1 -- 172.21.15.7:0/659743628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0e0100810 msgr2=0x7fb0e00fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0d7fff640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0e0100810 0x7fb0e00fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.156+0000 7fb0d7fff640 1 -- 172.21.15.7:0/659743628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e00fd670 con 0x7fb0e0106810 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.157+0000 7fb0d7fff640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 0x7fb0e00fd130 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800d6f0 tx=0x7fb0c800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.157+0000 7fb0d77fe640 1 -- 172.21.15.7:0/659743628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb0c8017070 con 0x7fb0e0106810 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.157+0000 7fb0d77fe640 1 -- 172.21.15.7:0/659743628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0c800de00 con 0x7fb0e0106810 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.157+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e00fd960 con 0x7fb0e0106810 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.157+0000 7fb0d77fe640 1 -- 172.21.15.7:0/659743628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb0c8004bb0 con 0x7fb0e0106810 2024-09-21T04:57:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.157+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e01a3ed0 con 0x7fb0e0106810 2024-09-21T04:57:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.158+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e0101f50 con 0x7fb0e0106810 2024-09-21T04:57:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.159+0000 7fb0d77fe640 1 -- 172.21.15.7:0/659743628 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb0c8004e60 con 0x7fb0e0106810 2024-09-21T04:57:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.159+0000 7fb0d77fe640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb0cc07af40 0x7fb0cc07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.159+0000 7fb0d77fe640 1 -- 172.21.15.7:0/659743628 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb0c809dc00 con 0x7fb0e0106810 2024-09-21T04:57:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.159+0000 7fb0de59c640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb0cc07af40 0x7fb0cc07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:32.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.160+0000 7fb0de59c640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb0cc07af40 0x7fb0cc07d400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00096c0 tx=0x7fb0d0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:32.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.163+0000 7fb0d77fe640 1 -- 172.21.15.7:0/659743628 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0c80a3020 con 0x7fb0e0106810 2024-09-21T04:57:32.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.371+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb0e0062630 con 0x7fb0cc07af40 2024-09-21T04:57:32.374 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:32 smithi007 ceph-mon[108238]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:32.374 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:32 smithi007 ceph-mon[108238]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:32.374 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:32 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1637430987' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:57:32.374 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:32 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/556090916' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:57:32.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.372+0000 7fb0d77fe640 1 -- 172.21.15.7:0/659743628 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fb0e0062630 con 0x7fb0cc07af40 2024-09-21T04:57:32.375 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:57:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:57:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:57:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:57:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:57:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:57:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:57:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:57:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:57:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:57:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:57:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:57:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:57:32.376 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb0cc07af40 msgr2=0x7fb0cc07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb0cc07af40 0x7fb0cc07d400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00096c0 tx=0x7fb0d0031400 comp rx=0 tx=0).stop 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 msgr2=0x7fb0e00fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 0x7fb0e00fd130 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800d6f0 tx=0x7fb0c800dbb0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 shutdown_connections 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb0cc07af40 0x7fb0cc07d400 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e0106810 0x7fb0e00fd130 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 --2- 172.21.15.7:0/659743628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0e0100810 0x7fb0e00fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 >> 172.21.15.7:0/659743628 conn(0x7fb0e00fc510 msgr2=0x7fb0e006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 shutdown_connections 2024-09-21T04:57:32.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.375+0000 7fb0e4c3f640 1 -- 172.21.15.7:0/659743628 wait complete. 2024-09-21T04:57:32.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:32 smithi053 ceph-mon[80176]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:32.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:32 smithi053 ceph-mon[80176]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:32.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:32 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1637430987' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:57:32.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:32 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/556090916' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:57:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.459+0000 7f9ddcdde640 1 -- 172.21.15.7:0/2144812242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 msgr2=0x7f9dd8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.459+0000 7f9ddcdde640 1 --2- 172.21.15.7:0/2144812242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 0x7f9dd8106be0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009880 tx=0x7f9dc802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.459+0000 7f9ddcdde640 1 -- 172.21.15.7:0/2144812242 shutdown_connections 2024-09-21T04:57:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.459+0000 7f9ddcdde640 1 --2- 172.21.15.7:0/2144812242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd8100800 0x7f9dd8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.459+0000 7f9ddcdde640 1 --2- 172.21.15.7:0/2144812242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 0x7f9dd8106be0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.459+0000 7f9ddcdde640 1 -- 172.21.15.7:0/2144812242 >> 172.21.15.7:0/2144812242 conn(0x7f9dd80fc520 msgr2=0x7f9dd80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.459+0000 7f9ddcdde640 1 -- 172.21.15.7:0/2144812242 shutdown_connections 2024-09-21T04:57:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.459+0000 7f9ddcdde640 1 -- 172.21.15.7:0/2144812242 wait complete. 2024-09-21T04:57:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9ddcdde640 1 Processor -- start 2024-09-21T04:57:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9ddcdde640 1 -- start start 2024-09-21T04:57:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9ddcdde640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd8100800 0x7f9dd819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9ddcdde640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 0x7f9dd819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:32.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9ddcdde640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd810dcd0 con 0x7f9dd8106800 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9ddcdde640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd810db50 con 0x7f9dd8100800 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9dcffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 0x7f9dd819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9dcffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 0x7f9dd819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59370/0 (socket says 172.21.15.7:59370) 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9dcffff640 1 -- 172.21.15.7:0/1830951077 learned_addr learned my addr 172.21.15.7:0/1830951077 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.460+0000 7f9dd659c640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd8100800 0x7f9dd819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9dcffff640 1 -- 172.21.15.7:0/1830951077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd8100800 msgr2=0x7f9dd819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9dcffff640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd8100800 0x7f9dd819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9dcffff640 1 -- 172.21.15.7:0/1830951077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd81977c0 con 0x7f9dd8106800 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9dcffff640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 0x7f9dd819db80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c8f0 tx=0x7f9dc000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9dcf7fe640 1 -- 172.21.15.7:0/1830951077 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9dc00079b0 con 0x7f9dd8106800 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9dcf7fe640 1 -- 172.21.15.7:0/1830951077 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9dc00054e0 con 0x7f9dd8106800 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd8197ab0 con 0x7f9dd8106800 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9dcf7fe640 1 -- 172.21.15.7:0/1830951077 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9dc0004dd0 con 0x7f9dd8106800 2024-09-21T04:57:32.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.461+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd8198040 con 0x7f9dd8106800 2024-09-21T04:57:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.462+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd8101f40 con 0x7f9dd8106800 2024-09-21T04:57:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.463+0000 7f9dcf7fe640 1 -- 172.21.15.7:0/1830951077 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9dc0005020 con 0x7f9dd8106800 2024-09-21T04:57:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.464+0000 7f9dcf7fe640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9dc407aed0 0x7f9dc407d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:57:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.464+0000 7f9dd659c640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9dc407aed0 0x7f9dc407d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:57:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.464+0000 7f9dcf7fe640 1 -- 172.21.15.7:0/1830951077 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f9dc009df80 con 0x7f9dd8106800 2024-09-21T04:57:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.464+0000 7f9dd659c640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9dc407aed0 0x7f9dc407d390 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8000c00 tx=0x7f9dc8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:57:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.468+0000 7f9dcf7fe640 1 -- 172.21.15.7:0/1830951077 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9dc0014030 con 0x7f9dd8106800 2024-09-21T04:57:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.736+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9dd8109d80 con 0x7f9dd8106800 2024-09-21T04:57:32.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.736+0000 7f9dcf7fe640 1 -- 172.21.15.7:0/1830951077 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f9dc0069520 con 0x7f9dd8106800 2024-09-21T04:57:32.739 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:57:32.739 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:57:32.740 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:57:32.740 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:57:32.740 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:57:32.740 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:57:32.740 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:57:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9dc407aed0 msgr2=0x7f9dc407d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9dc407aed0 0x7f9dc407d390 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8000c00 tx=0x7f9dc8031400 comp rx=0 tx=0).stop 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 msgr2=0x7f9dd819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 0x7f9dd819db80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c8f0 tx=0x7f9dc000cdb0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 shutdown_connections 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9dc407aed0 0x7f9dc407d390 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd8106800 0x7f9dd819db80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 --2- 172.21.15.7:0/1830951077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9dd8100800 0x7f9dd819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.738+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 >> 172.21.15.7:0/1830951077 conn(0x7f9dd80fc520 msgr2=0x7f9dd806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.739+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 shutdown_connections 2024-09-21T04:57:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:57:32.739+0000 7f9ddcdde640 1 -- 172.21.15.7:0/1830951077 wait complete. 2024-09-21T04:57:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:33 smithi053 ceph-mon[80176]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:33 smithi053 ceph-mon[80176]: pgmap v683: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:33 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1830951077' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:57:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:33 smithi007 ceph-mon[108238]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:33 smithi007 ceph-mon[108238]: pgmap v683: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:33 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1830951077' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:57:34.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:34 smithi053 ceph-mon[80176]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:34.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:34 smithi007 ceph-mon[108238]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:57:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:35 smithi007 ceph-mon[108238]: pgmap v684: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:35.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:35 smithi053 ceph-mon[80176]: pgmap v684: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:37 smithi053 ceph-mon[80176]: pgmap v685: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:37 smithi007 ceph-mon[108238]: pgmap v685: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:39 smithi053 ceph-mon[80176]: pgmap v686: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:39 smithi007 ceph-mon[108238]: pgmap v686: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:41 smithi007 ceph-mon[108238]: pgmap v687: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:41 smithi053 ceph-mon[80176]: pgmap v687: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:43 smithi053 ceph-mon[80176]: pgmap v688: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:43 smithi007 ceph-mon[108238]: pgmap v688: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:57:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:57:45.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:45 smithi007 ceph-mon[108238]: pgmap v689: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:45.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:45 smithi053 ceph-mon[80176]: pgmap v689: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:47.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:47 smithi053 ceph-mon[80176]: pgmap v690: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:47 smithi007 ceph-mon[108238]: pgmap v690: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:49 smithi053 ceph-mon[80176]: pgmap v691: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:49.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:49 smithi007 ceph-mon[108238]: pgmap v691: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:51.546 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:51 smithi007 ceph-mon[108238]: pgmap v692: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:51.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:51 smithi053 ceph-mon[80176]: pgmap v692: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:53.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:53 smithi053 ceph-mon[80176]: pgmap v693: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:53 smithi007 ceph-mon[108238]: pgmap v693: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:55.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:54 smithi053 ceph-mon[80176]: pgmap v694: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:55.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:54 smithi007 ceph-mon[108238]: pgmap v694: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:57 smithi053 ceph-mon[80176]: pgmap v695: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:57 smithi007 ceph-mon[108238]: pgmap v695: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:59 smithi007 ceph-mon[108238]: pgmap v696: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:57:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:57:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:59 smithi053 ceph-mon[80176]: pgmap v696: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:57:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:57:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:58:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:01 smithi053 ceph-mon[80176]: pgmap v697: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:01 smithi007 ceph-mon[108238]: pgmap v697: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:02.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.835+0000 7f05866c5640 1 -- 172.21.15.7:0/1287534839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580106800 msgr2=0x7f0580106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.835+0000 7f05866c5640 1 --2- 172.21.15.7:0/1287534839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580106800 0x7f0580106be0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0570009880 tx=0x7f057002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.835+0000 7f05866c5640 1 -- 172.21.15.7:0/1287534839 shutdown_connections 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.835+0000 7f05866c5640 1 --2- 172.21.15.7:0/1287534839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580100800 0x7f0580100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.835+0000 7f05866c5640 1 --2- 172.21.15.7:0/1287534839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580106800 0x7f0580106be0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.835+0000 7f05866c5640 1 -- 172.21.15.7:0/1287534839 >> 172.21.15.7:0/1287534839 conn(0x7f05800fc520 msgr2=0x7f05800fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.835+0000 7f05866c5640 1 -- 172.21.15.7:0/1287534839 shutdown_connections 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.835+0000 7f05866c5640 1 -- 172.21.15.7:0/1287534839 wait complete. 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.836+0000 7f05866c5640 1 Processor -- start 2024-09-21T04:58:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.836+0000 7f05866c5640 1 -- start start 2024-09-21T04:58:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.836+0000 7f05866c5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580100800 0x7f05800fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.836+0000 7f05866c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580106800 0x7f05800fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.836+0000 7f05866c5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f058010dcd0 con 0x7f0580100800 2024-09-21T04:58:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.836+0000 7f05866c5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f058010db50 con 0x7f0580106800 2024-09-21T04:58:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580100800 0x7f05800fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580100800 0x7f05800fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54840/0 (socket says 172.21.15.7:54840) 2024-09-21T04:58:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580106800 0x7f05800fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057ffff640 1 -- 172.21.15.7:0/175875957 learned_addr learned my addr 172.21.15.7:0/175875957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057ffff640 1 -- 172.21.15.7:0/175875957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580106800 msgr2=0x7f05800fd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057ffff640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580106800 0x7f05800fd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057ffff640 1 -- 172.21.15.7:0/175875957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05800fd680 con 0x7f0580100800 2024-09-21T04:58:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057f7fe640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580106800 0x7f05800fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:58:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.837+0000 7f057ffff640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580100800 0x7f05800fcc00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0570031460 tx=0x7f05700045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.838+0000 7f057d7fa640 1 -- 172.21.15.7:0/175875957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f057003f070 con 0x7f0580100800 2024-09-21T04:58:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.838+0000 7f057d7fa640 1 -- 172.21.15.7:0/175875957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05700035f0 con 0x7f0580100800 2024-09-21T04:58:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.838+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05800fd910 con 0x7f0580100800 2024-09-21T04:58:02.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.838+0000 7f057d7fa640 1 -- 172.21.15.7:0/175875957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0570035030 con 0x7f0580100800 2024-09-21T04:58:02.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.838+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05801a3da0 con 0x7f0580100800 2024-09-21T04:58:02.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.839+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0580101f40 con 0x7f0580100800 2024-09-21T04:58:02.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.840+0000 7f057d7fa640 1 -- 172.21.15.7:0/175875957 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0570042070 con 0x7f0580100800 2024-09-21T04:58:02.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.840+0000 7f057d7fa640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f056807af80 0x7f056807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:02.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.840+0000 7f057d7fa640 1 -- 172.21.15.7:0/175875957 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f05700be8c0 con 0x7f0580100800 2024-09-21T04:58:02.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.840+0000 7f057f7fe640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f056807af80 0x7f056807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:02.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.841+0000 7f057f7fe640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f056807af80 0x7f056807d440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f05800fe690 tx=0x7f057400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:02.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:02.845+0000 7f057d7fa640 1 -- 172.21.15.7:0/175875957 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0570089ee0 con 0x7f0580100800 2024-09-21T04:58:03.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.050+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0580062630 con 0x7f056807af80 2024-09-21T04:58:03.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.051+0000 7f057d7fa640 1 -- 172.21.15.7:0/175875957 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0580062630 con 0x7f056807af80 2024-09-21T04:58:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.053+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f056807af80 msgr2=0x7f056807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.053+0000 7f05866c5640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f056807af80 0x7f056807d440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f05800fe690 tx=0x7f057400b390 comp rx=0 tx=0).stop 2024-09-21T04:58:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.053+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580100800 msgr2=0x7f05800fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.053+0000 7f05866c5640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580100800 0x7f05800fcc00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0570031460 tx=0x7f05700045e0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.054+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 shutdown_connections 2024-09-21T04:58:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.054+0000 7f05866c5640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f056807af80 0x7f056807d440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.054+0000 7f05866c5640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580106800 0x7f05800fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.054+0000 7f05866c5640 1 --2- 172.21.15.7:0/175875957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0580100800 0x7f05800fcc00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.054+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 >> 172.21.15.7:0/175875957 conn(0x7f05800fc520 msgr2=0x7f058006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.054+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 shutdown_connections 2024-09-21T04:58:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.054+0000 7f05866c5640 1 -- 172.21.15.7:0/175875957 wait complete. 2024-09-21T04:58:03.066 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:58:03.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.139+0000 7fd708197640 1 -- 172.21.15.7:0/1442401216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001007f0 msgr2=0x7fd700100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.139+0000 7fd708197640 1 --2- 172.21.15.7:0/1442401216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001007f0 0x7fd700100c50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009880 tx=0x7fd6f402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.139+0000 7fd708197640 1 -- 172.21.15.7:0/1442401216 shutdown_connections 2024-09-21T04:58:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.139+0000 7fd708197640 1 --2- 172.21.15.7:0/1442401216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001007f0 0x7fd700100c50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.139+0000 7fd708197640 1 --2- 172.21.15.7:0/1442401216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7001067f0 0x7fd700106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.139+0000 7fd708197640 1 -- 172.21.15.7:0/1442401216 >> 172.21.15.7:0/1442401216 conn(0x7fd7000fc550 msgr2=0x7fd7000fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.140+0000 7fd708197640 1 -- 172.21.15.7:0/1442401216 shutdown_connections 2024-09-21T04:58:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.140+0000 7fd708197640 1 -- 172.21.15.7:0/1442401216 wait complete. 2024-09-21T04:58:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.140+0000 7fd708197640 1 Processor -- start 2024-09-21T04:58:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.140+0000 7fd708197640 1 -- start start 2024-09-21T04:58:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.140+0000 7fd708197640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7001007f0 0x7fd7000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.140+0000 7fd708197640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001067f0 0x7fd7000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.140+0000 7fd708197640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd70010dca0 con 0x7fd7001067f0 2024-09-21T04:58:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.140+0000 7fd708197640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd70010db20 con 0x7fd7001007f0 2024-09-21T04:58:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd705732640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001067f0 0x7fd7000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd705732640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001067f0 0x7fd7000fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54854/0 (socket says 172.21.15.7:54854) 2024-09-21T04:58:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd705732640 1 -- 172.21.15.7:0/2659721153 learned_addr learned my addr 172.21.15.7:0/2659721153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd705f33640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7001007f0 0x7fd7000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd705732640 1 -- 172.21.15.7:0/2659721153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7001007f0 msgr2=0x7fd7000fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd705732640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7001007f0 0x7fd7000fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd705732640 1 -- 172.21.15.7:0/2659721153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7000fd700 con 0x7fd7001067f0 2024-09-21T04:58:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd705732640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001067f0 0x7fd7000fd1c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4031460 tx=0x7fd6f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd6feffd640 1 -- 172.21.15.7:0/2659721153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd6f403f070 con 0x7fd7001067f0 2024-09-21T04:58:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7000fd990 con 0x7fd7001067f0 2024-09-21T04:58:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd6feffd640 1 -- 172.21.15.7:0/2659721153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd6f4004cf0 con 0x7fd7001067f0 2024-09-21T04:58:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd6feffd640 1 -- 172.21.15.7:0/2659721153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd6f4035030 con 0x7fd7001067f0 2024-09-21T04:58:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.141+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7001a3cf0 con 0x7fd7001067f0 2024-09-21T04:58:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.142+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd700101f30 con 0x7fd7001067f0 2024-09-21T04:58:03.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.143+0000 7fd6feffd640 1 -- 172.21.15.7:0/2659721153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd6f4042070 con 0x7fd7001067f0 2024-09-21T04:58:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.144+0000 7fd6feffd640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd6e807af30 0x7fd6e807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.144+0000 7fd6feffd640 1 -- 172.21.15.7:0/2659721153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd6f40bea40 con 0x7fd7001067f0 2024-09-21T04:58:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.144+0000 7fd705f33640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd6e807af30 0x7fd6e807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.144+0000 7fd705f33640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd6e807af30 0x7fd6e807d3f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0007ec0 tx=0x7fd6f00075b0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.148+0000 7fd6feffd640 1 -- 172.21.15.7:0/2659721153 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6f4089fe0 con 0x7fd7001067f0 2024-09-21T04:58:03.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.354+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd700062630 con 0x7fd6e807af30 2024-09-21T04:58:03.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.355+0000 7fd6feffd640 1 -- 172.21.15.7:0/2659721153 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd700062630 con 0x7fd6e807af30 2024-09-21T04:58:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.357+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd6e807af30 msgr2=0x7fd6e807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.357+0000 7fd708197640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd6e807af30 0x7fd6e807d3f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0007ec0 tx=0x7fd6f00075b0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.357+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001067f0 msgr2=0x7fd7000fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.357+0000 7fd708197640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001067f0 0x7fd7000fd1c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4031460 tx=0x7fd6f40048f0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.357+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 shutdown_connections 2024-09-21T04:58:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.357+0000 7fd708197640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd6e807af30 0x7fd6e807d3f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.358+0000 7fd708197640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7001067f0 0x7fd7000fd1c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.358+0000 7fd708197640 1 --2- 172.21.15.7:0/2659721153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7001007f0 0x7fd7000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.358+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 >> 172.21.15.7:0/2659721153 conn(0x7fd7000fc550 msgr2=0x7fd700069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.358+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 shutdown_connections 2024-09-21T04:58:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.358+0000 7fd708197640 1 -- 172.21.15.7:0/2659721153 wait complete. 2024-09-21T04:58:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:03 smithi053 ceph-mon[80176]: pgmap v698: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:03 smithi007 ceph-mon[108238]: pgmap v698: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:03.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.442+0000 7f217fcac640 1 -- 172.21.15.7:0/3658305858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178106800 msgr2=0x7f2178106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.442+0000 7f217fcac640 1 --2- 172.21.15.7:0/3658305858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178106800 0x7f2178106be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2174009880 tx=0x7f217402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.442+0000 7f217fcac640 1 -- 172.21.15.7:0/3658305858 shutdown_connections 2024-09-21T04:58:03.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.442+0000 7f217fcac640 1 --2- 172.21.15.7:0/3658305858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2178100800 0x7f2178100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.442+0000 7f217fcac640 1 --2- 172.21.15.7:0/3658305858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178106800 0x7f2178106be0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.442+0000 7f217fcac640 1 -- 172.21.15.7:0/3658305858 >> 172.21.15.7:0/3658305858 conn(0x7f21780fc520 msgr2=0x7f21780fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:03.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.443+0000 7f217fcac640 1 -- 172.21.15.7:0/3658305858 shutdown_connections 2024-09-21T04:58:03.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.443+0000 7f217fcac640 1 -- 172.21.15.7:0/3658305858 wait complete. 2024-09-21T04:58:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.443+0000 7f217fcac640 1 Processor -- start 2024-09-21T04:58:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.443+0000 7f217fcac640 1 -- start start 2024-09-21T04:58:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217fcac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178100800 0x7f21780fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217fcac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2178106800 0x7f21780fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217fcac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f217810dcd0 con 0x7f2178100800 2024-09-21T04:58:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217fcac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f217810db50 con 0x7f2178106800 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217da48640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178100800 0x7f21780fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217da48640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178100800 0x7f21780fcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45600/0 (socket says 172.21.15.7:45600) 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217da48640 1 -- 172.21.15.7:0/2098335719 learned_addr learned my addr 172.21.15.7:0/2098335719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217d247640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2178106800 0x7f21780fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217da48640 1 -- 172.21.15.7:0/2098335719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2178106800 msgr2=0x7f21780fd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217da48640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2178106800 0x7f21780fd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217da48640 1 -- 172.21.15.7:0/2098335719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21780fd6f0 con 0x7f2178100800 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217da48640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178100800 0x7f21780fcc70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2174031460 tx=0x7f2174004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f216effd640 1 -- 172.21.15.7:0/2098335719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f217403f070 con 0x7f2178100800 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f216effd640 1 -- 172.21.15.7:0/2098335719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21740035f0 con 0x7f2178100800 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21780fd980 con 0x7f2178100800 2024-09-21T04:58:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.444+0000 7f216effd640 1 -- 172.21.15.7:0/2098335719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2174035030 con 0x7f2178100800 2024-09-21T04:58:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.445+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21781a3d60 con 0x7f2178100800 2024-09-21T04:58:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.446+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2178101f40 con 0x7f2178100800 2024-09-21T04:58:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.446+0000 7f216effd640 1 -- 172.21.15.7:0/2098335719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2174042070 con 0x7f2178100800 2024-09-21T04:58:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.447+0000 7f216effd640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f216407af30 0x7f216407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.447+0000 7f216effd640 1 -- 172.21.15.7:0/2098335719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f21740bea10 con 0x7f2178100800 2024-09-21T04:58:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.447+0000 7f217d247640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f216407af30 0x7f216407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.447+0000 7f217d247640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f216407af30 0x7f216407d3f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f21780fe700 tx=0x7f2168005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.451+0000 7f216effd640 1 -- 172.21.15.7:0/2098335719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2174089fb0 con 0x7f2178100800 2024-09-21T04:58:03.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.648+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2178062630 con 0x7f216407af30 2024-09-21T04:58:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.658+0000 7f216effd640 1 -- 172.21.15.7:0/2098335719 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f2178062630 con 0x7f216407af30 2024-09-21T04:58:03.662 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:58:03.662 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (20m) 14m ago 30m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:58:03.662 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (23m) 14m ago 31m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:58:03.662 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (22m) 13m ago 30m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:58:03.662 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (18m) 14m ago 31m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:58:03.662 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (18m) 13m ago 30m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:58:03.662 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (19m) 14m ago 30m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (14m) 14m ago 25m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (14m) 14m ago 25m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (25m) 13m ago 25m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (13m) 13m ago 25m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (22m) 14m ago 32m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (21m) 13m ago 30m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (18m) 14m ago 32m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (18m) 13m ago 30m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (20m) 14m ago 31m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (20m) 13m ago 30m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (17m) 14m ago 29m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (17m) 14m ago 28m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (16m) 14m ago 28m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (15m) 13m ago 27m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:58:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (15m) 13m ago 27m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:58:03.664 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (14m) 13m ago 26m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:58:03.664 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (20m) 14m ago 30m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:58:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f216407af30 msgr2=0x7f216407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f216407af30 0x7f216407d3f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f21780fe700 tx=0x7f2168005f90 comp rx=0 tx=0).stop 2024-09-21T04:58:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178100800 msgr2=0x7f21780fcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178100800 0x7f21780fcc70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2174031460 tx=0x7f2174004530 comp rx=0 tx=0).stop 2024-09-21T04:58:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 shutdown_connections 2024-09-21T04:58:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f216407af30 0x7f216407d3f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2178106800 0x7f21780fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 --2- 172.21.15.7:0/2098335719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2178100800 0x7f21780fcc70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 >> 172.21.15.7:0/2098335719 conn(0x7f21780fc520 msgr2=0x7f217806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 shutdown_connections 2024-09-21T04:58:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.660+0000 7f217fcac640 1 -- 172.21.15.7:0/2098335719 wait complete. 2024-09-21T04:58:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.743+0000 7f076bb1d640 1 -- 172.21.15.7:0/2410967774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764100800 msgr2=0x7f0764100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.743+0000 7f076bb1d640 1 --2- 172.21.15.7:0/2410967774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764100800 0x7f0764100c60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0754009880 tx=0x7f075402a4a0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.743+0000 7f076bb1d640 1 -- 172.21.15.7:0/2410967774 shutdown_connections 2024-09-21T04:58:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.743+0000 7f076bb1d640 1 --2- 172.21.15.7:0/2410967774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764100800 0x7f0764100c60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.743+0000 7f076bb1d640 1 --2- 172.21.15.7:0/2410967774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0764106800 0x7f0764106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.743+0000 7f076bb1d640 1 -- 172.21.15.7:0/2410967774 >> 172.21.15.7:0/2410967774 conn(0x7f07640fc520 msgr2=0x7f07640fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.744+0000 7f076bb1d640 1 -- 172.21.15.7:0/2410967774 shutdown_connections 2024-09-21T04:58:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.744+0000 7f076bb1d640 1 -- 172.21.15.7:0/2410967774 wait complete. 2024-09-21T04:58:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.744+0000 7f076bb1d640 1 Processor -- start 2024-09-21T04:58:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.744+0000 7f076bb1d640 1 -- start start 2024-09-21T04:58:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f076bb1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0764100800 0x7f07641987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f076bb1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764106800 0x7f0764198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f076bb1d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f076410dcd0 con 0x7f0764106800 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f076bb1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f076410db50 con 0x7f0764100800 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f07690b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764106800 0x7f0764198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f07690b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764106800 0x7f0764198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45624/0 (socket says 172.21.15.7:45624) 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f07690b8640 1 -- 172.21.15.7:0/2959689188 learned_addr learned my addr 172.21.15.7:0/2959689188 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f07698b9640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0764100800 0x7f07641987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f07690b8640 1 -- 172.21.15.7:0/2959689188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0764100800 msgr2=0x7f07641987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f07690b8640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0764100800 0x7f07641987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f07690b8640 1 -- 172.21.15.7:0/2959689188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0764199370 con 0x7f0764106800 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.745+0000 7f07690b8640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764106800 0x7f0764198d20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0754031460 tx=0x7f07540048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.746+0000 7f075affd640 1 -- 172.21.15.7:0/2959689188 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f075403f070 con 0x7f0764106800 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.746+0000 7f075affd640 1 -- 172.21.15.7:0/2959689188 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07540044d0 con 0x7f0764106800 2024-09-21T04:58:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.746+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0764199600 con 0x7f0764106800 2024-09-21T04:58:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.746+0000 7f075affd640 1 -- 172.21.15.7:0/2959689188 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0754035030 con 0x7f0764106800 2024-09-21T04:58:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.746+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07641a1bb0 con 0x7f0764106800 2024-09-21T04:58:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.747+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0764101f40 con 0x7f0764106800 2024-09-21T04:58:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.747+0000 7f075affd640 1 -- 172.21.15.7:0/2959689188 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0754042070 con 0x7f0764106800 2024-09-21T04:58:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.748+0000 7f075affd640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f075007af40 0x7f075007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.748+0000 7f075affd640 1 -- 172.21.15.7:0/2959689188 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f07540bebc0 con 0x7f0764106800 2024-09-21T04:58:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.748+0000 7f07698b9640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f075007af40 0x7f075007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.748+0000 7f07698b9640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f075007af40 0x7f075007d400 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0760005fd0 tx=0x7f0760005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:03.752+0000 7f075affd640 1 -- 172.21.15.7:0/2959689188 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0754004cf0 con 0x7f0764106800 2024-09-21T04:58:04.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.017+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f076406a7e0 con 0x7f0764106800 2024-09-21T04:58:04.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.017+0000 7f075affd640 1 -- 172.21.15.7:0/2959689188 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f075408c680 con 0x7f0764106800 2024-09-21T04:58:04.022 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:58:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:58:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:58:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:58:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:58:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:58:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:58:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:58:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:58:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:58:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:58:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:58:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:58:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:58:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:58:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:58:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:58:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:58:04.025 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f075007af40 msgr2=0x7f075007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f075007af40 0x7f075007d400 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0760005fd0 tx=0x7f0760005e20 comp rx=0 tx=0).stop 2024-09-21T04:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764106800 msgr2=0x7f0764198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764106800 0x7f0764198d20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0754031460 tx=0x7f07540048f0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 shutdown_connections 2024-09-21T04:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f075007af40 0x7f075007d400 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764106800 0x7f0764198d20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 --2- 172.21.15.7:0/2959689188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0764100800 0x7f07641987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 >> 172.21.15.7:0/2959689188 conn(0x7f07640fc520 msgr2=0x7f07640fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 shutdown_connections 2024-09-21T04:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.019+0000 7f076bb1d640 1 -- 172.21.15.7:0/2959689188 wait complete. 2024-09-21T04:58:04.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 -- 172.21.15.7:0/1204481132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681007f0 msgr2=0x7ff468100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 --2- 172.21.15.7:0/1204481132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681007f0 0x7ff468100c50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff450009880 tx=0x7ff45002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 -- 172.21.15.7:0/1204481132 shutdown_connections 2024-09-21T04:58:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 --2- 172.21.15.7:0/1204481132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681007f0 0x7ff468100c50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 --2- 172.21.15.7:0/1204481132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4681067f0 0x7ff468106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 -- 172.21.15.7:0/1204481132 >> 172.21.15.7:0/1204481132 conn(0x7ff4680fc550 msgr2=0x7ff4680fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 -- 172.21.15.7:0/1204481132 shutdown_connections 2024-09-21T04:58:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 -- 172.21.15.7:0/1204481132 wait complete. 2024-09-21T04:58:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 Processor -- start 2024-09-21T04:58:04.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.104+0000 7ff46dda9640 1 -- start start 2024-09-21T04:58:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff46dda9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4681007f0 0x7ff46819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff46dda9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681067f0 0x7ff46819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff46dda9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff46810dcc0 con 0x7ff4681067f0 2024-09-21T04:58:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff46dda9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff46810db40 con 0x7ff4681007f0 2024-09-21T04:58:04.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff45effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681067f0 0x7ff46819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff45effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681067f0 0x7ff46819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45644/0 (socket says 172.21.15.7:45644) 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff45effd640 1 -- 172.21.15.7:0/2720099702 learned_addr learned my addr 172.21.15.7:0/2720099702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff4677fe640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4681007f0 0x7ff46819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff45effd640 1 -- 172.21.15.7:0/2720099702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4681007f0 msgr2=0x7ff46819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff45effd640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4681007f0 0x7ff46819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.105+0000 7ff45effd640 1 -- 172.21.15.7:0/2720099702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4681977e0 con 0x7ff4681067f0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.106+0000 7ff45effd640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681067f0 0x7ff46819dba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff450031460 tx=0x7ff450004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.106+0000 7ff4657fa640 1 -- 172.21.15.7:0/2720099702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff45003f070 con 0x7ff4681067f0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.106+0000 7ff4657fa640 1 -- 172.21.15.7:0/2720099702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4500035f0 con 0x7ff4681067f0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.106+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff468197a70 con 0x7ff4681067f0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.106+0000 7ff4657fa640 1 -- 172.21.15.7:0/2720099702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff450035030 con 0x7ff4681067f0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.106+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff468197fa0 con 0x7ff4681067f0 2024-09-21T04:58:04.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.107+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff468101f30 con 0x7ff4681067f0 2024-09-21T04:58:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.107+0000 7ff4657fa640 1 -- 172.21.15.7:0/2720099702 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff450042070 con 0x7ff4681067f0 2024-09-21T04:58:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.108+0000 7ff4657fa640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff45407af80 0x7ff45407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.108+0000 7ff4677fe640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff45407af80 0x7ff45407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.108+0000 7ff4657fa640 1 -- 172.21.15.7:0/2720099702 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff4500be940 con 0x7ff4681067f0 2024-09-21T04:58:04.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.109+0000 7ff4677fe640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff45407af80 0x7ff45407d440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff458009b60 tx=0x7ff458005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:04.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.112+0000 7ff4657fa640 1 -- 172.21.15.7:0/2720099702 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff450089f60 con 0x7ff4681067f0 2024-09-21T04:58:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.344+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff468109d70 con 0x7ff4681067f0 2024-09-21T04:58:04.348 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:04 smithi007 ceph-mon[108238]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:04.349 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:04 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2959689188' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:58:04.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.345+0000 7ff4657fa640 1 -- 172.21.15.7:0/2720099702 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7ff45008ee10 con 0x7ff4681067f0 2024-09-21T04:58:04.349 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:58:04.349 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:58:04.349 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:58:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:58:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:58:04.352 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:58:04.353 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:58:04.353 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:58:04.353 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:58:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:58:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff45407af80 msgr2=0x7ff45407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff45407af80 0x7ff45407d440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff458009b60 tx=0x7ff458005f90 comp rx=0 tx=0).stop 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681067f0 msgr2=0x7ff46819dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681067f0 0x7ff46819dba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff450031460 tx=0x7ff450004530 comp rx=0 tx=0).stop 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 shutdown_connections 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff45407af80 0x7ff45407d440 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4681067f0 0x7ff46819dba0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 --2- 172.21.15.7:0/2720099702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4681007f0 0x7ff46819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 >> 172.21.15.7:0/2720099702 conn(0x7ff4680fc550 msgr2=0x7ff468069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 shutdown_connections 2024-09-21T04:58:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.348+0000 7ff46dda9640 1 -- 172.21.15.7:0/2720099702 wait complete. 2024-09-21T04:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.433+0000 7f3bd2d54640 1 -- 172.21.15.7:0/1434468886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc101900 msgr2=0x7f3bcc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.433+0000 7f3bd2d54640 1 --2- 172.21.15.7:0/1434468886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc101900 0x7f3bcc101d60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009880 tx=0x7f3bbc02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.433+0000 7f3bd2d54640 1 -- 172.21.15.7:0/1434468886 shutdown_connections 2024-09-21T04:58:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.433+0000 7f3bd2d54640 1 --2- 172.21.15.7:0/1434468886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc101900 0x7f3bcc101d60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.433+0000 7f3bd2d54640 1 --2- 172.21.15.7:0/1434468886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc107900 0x7f3bcc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.433+0000 7f3bd2d54640 1 -- 172.21.15.7:0/1434468886 >> 172.21.15.7:0/1434468886 conn(0x7f3bcc0fd600 msgr2=0x7f3bcc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.433+0000 7f3bd2d54640 1 -- 172.21.15.7:0/1434468886 shutdown_connections 2024-09-21T04:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.433+0000 7f3bd2d54640 1 -- 172.21.15.7:0/1434468886 wait complete. 2024-09-21T04:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bd2d54640 1 Processor -- start 2024-09-21T04:58:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bd2d54640 1 -- start start 2024-09-21T04:58:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bd2d54640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc101900 0x7f3bcc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bd2d54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc107900 0x7f3bcc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bd2d54640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10ee30 con 0x7f3bcc107900 2024-09-21T04:58:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bd2d54640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10ecb0 con 0x7f3bcc101900 2024-09-21T04:58:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bc3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc107900 0x7f3bcc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bc3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc107900 0x7f3bcc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45666/0 (socket says 172.21.15.7:45666) 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bc3fff640 1 -- 172.21.15.7:0/2590522156 learned_addr learned my addr 172.21.15.7:0/2590522156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bd0af0640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc101900 0x7f3bcc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bc3fff640 1 -- 172.21.15.7:0/2590522156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc101900 msgr2=0x7f3bcc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bc3fff640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc101900 0x7f3bcc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.434+0000 7f3bc3fff640 1 -- 172.21.15.7:0/2590522156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc074f70 con 0x7f3bcc107900 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.435+0000 7f3bc3fff640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc107900 0x7f3bcc074a30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc031460 tx=0x7f3bbc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.435+0000 7f3bc1ffb640 1 -- 172.21.15.7:0/2590522156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3bbc03f070 con 0x7f3bcc107900 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.435+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc075200 con 0x7f3bcc107900 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.435+0000 7f3bc1ffb640 1 -- 172.21.15.7:0/2590522156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3bbc004cf0 con 0x7f3bcc107900 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.435+0000 7f3bc1ffb640 1 -- 172.21.15.7:0/2590522156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3bbc035030 con 0x7f3bcc107900 2024-09-21T04:58:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.435+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a3e00 con 0x7f3bcc107900 2024-09-21T04:58:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.436+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc103040 con 0x7f3bcc107900 2024-09-21T04:58:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.437+0000 7f3bc1ffb640 1 -- 172.21.15.7:0/2590522156 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3bbc042070 con 0x7f3bcc107900 2024-09-21T04:58:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.437+0000 7f3bc1ffb640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bb407af30 0x7f3bb407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.438+0000 7f3bc1ffb640 1 -- 172.21.15.7:0/2590522156 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f3bbc0bec00 con 0x7f3bcc107900 2024-09-21T04:58:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.438+0000 7f3bd0af0640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bb407af30 0x7f3bb407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.438+0000 7f3bd0af0640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bb407af30 0x7f3bb407d3f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009890 tx=0x7f3bc8008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:04.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:04 smithi053 ceph-mon[80176]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:04.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:04 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2959689188' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:58:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.442+0000 7f3bc1ffb640 1 -- 172.21.15.7:0/2590522156 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3bbc08a100 con 0x7f3bcc107900 2024-09-21T04:58:04.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.647+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3bcc062630 con 0x7f3bb407af30 2024-09-21T04:58:04.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.649+0000 7f3bc1ffb640 1 -- 172.21.15.7:0/2590522156 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f3bcc062630 con 0x7f3bb407af30 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:58:04.653 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:58:04.654 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:58:04.654 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bb407af30 msgr2=0x7f3bb407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bb407af30 0x7f3bb407d3f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009890 tx=0x7f3bc8008040 comp rx=0 tx=0).stop 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc107900 msgr2=0x7f3bcc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc107900 0x7f3bcc074a30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc031460 tx=0x7f3bbc0048f0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 shutdown_connections 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3bb407af30 0x7f3bb407d3f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bcc107900 0x7f3bcc074a30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 --2- 172.21.15.7:0/2590522156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc101900 0x7f3bcc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 >> 172.21.15.7:0/2590522156 conn(0x7f3bcc0fd600 msgr2=0x7f3bcc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 shutdown_connections 2024-09-21T04:58:04.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.651+0000 7f3bd2d54640 1 -- 172.21.15.7:0/2590522156 wait complete. 2024-09-21T04:58:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.736+0000 7f4de2bf4640 1 -- 172.21.15.7:0/3395395982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc100650 msgr2=0x7f4ddc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.736+0000 7f4de2bf4640 1 --2- 172.21.15.7:0/3395395982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc100650 0x7f4ddc100a30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009880 tx=0x7f4dd002a490 comp rx=0 tx=0).stop 2024-09-21T04:58:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.736+0000 7f4de2bf4640 1 -- 172.21.15.7:0/3395395982 shutdown_connections 2024-09-21T04:58:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.736+0000 7f4de2bf4640 1 --2- 172.21.15.7:0/3395395982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ddc101000 0x7f4ddc10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.736+0000 7f4de2bf4640 1 --2- 172.21.15.7:0/3395395982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc100650 0x7f4ddc100a30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.736+0000 7f4de2bf4640 1 -- 172.21.15.7:0/3395395982 >> 172.21.15.7:0/3395395982 conn(0x7f4ddc0fc520 msgr2=0x7f4ddc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.736+0000 7f4de2bf4640 1 -- 172.21.15.7:0/3395395982 shutdown_connections 2024-09-21T04:58:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.736+0000 7f4de2bf4640 1 -- 172.21.15.7:0/3395395982 wait complete. 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4de2bf4640 1 Processor -- start 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4de2bf4640 1 -- start start 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4de2bf4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ddc100650 0x7f4ddc19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4de2bf4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc101000 0x7f4ddc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4de2bf4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc110fb0 con 0x7f4ddc101000 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4de2bf4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc110e30 con 0x7f4ddc100650 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc101000 0x7f4ddc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc101000 0x7f4ddc19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45678/0 (socket says 172.21.15.7:45678) 2024-09-21T04:58:04.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.737+0000 7f4ddbfff640 1 -- 172.21.15.7:0/1389792792 learned_addr learned my addr 172.21.15.7:0/1389792792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4de0990640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ddc100650 0x7f4ddc19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4ddbfff640 1 -- 172.21.15.7:0/1389792792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ddc100650 msgr2=0x7f4ddc19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4ddbfff640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ddc100650 0x7f4ddc19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4ddbfff640 1 -- 172.21.15.7:0/1389792792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc199a40 con 0x7f4ddc101000 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4ddbfff640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc101000 0x7f4ddc19fdc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800e970 tx=0x7f4dc800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4dd9ffb640 1 -- 172.21.15.7:0/1389792792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4dc800cac0 con 0x7f4ddc101000 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc199d30 con 0x7f4ddc101000 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4dd9ffb640 1 -- 172.21.15.7:0/1389792792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4dc80054e0 con 0x7f4ddc101000 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4dd9ffb640 1 -- 172.21.15.7:0/1389792792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4dc8010430 con 0x7f4ddc101000 2024-09-21T04:58:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.738+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc19a2c0 con 0x7f4ddc101000 2024-09-21T04:58:04.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.739+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ddc06aa90 con 0x7f4ddc101000 2024-09-21T04:58:04.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.740+0000 7f4dd9ffb640 1 -- 172.21.15.7:0/1389792792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4dc8004f00 con 0x7f4ddc101000 2024-09-21T04:58:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.741+0000 7f4dd9ffb640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4dcc07af30 0x7f4dcc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.741+0000 7f4de0990640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4dcc07af30 0x7f4dcc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.741+0000 7f4dd9ffb640 1 -- 172.21.15.7:0/1389792792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4dc809e7d0 con 0x7f4ddc101000 2024-09-21T04:58:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.741+0000 7f4de0990640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4dcc07af30 0x7f4dcc07d3f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0000c00 tx=0x7f4dd0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:04.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:04.745+0000 7f4dd9ffb640 1 -- 172.21.15.7:0/1389792792 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4dc8069d70 con 0x7f4ddc101000 2024-09-21T04:58:05.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.011+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f4ddc10b3d0 con 0x7f4ddc101000 2024-09-21T04:58:05.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.012+0000 7f4dd9ffb640 1 -- 172.21.15.7:0/1389792792 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f4dc806ec20 con 0x7f4ddc101000 2024-09-21T04:58:05.016 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:58:05.016 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:58:05.016 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:58:05.016 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:58:05.016 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:58:05.016 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:58:05.016 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4dcc07af30 msgr2=0x7f4dcc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4dcc07af30 0x7f4dcc07d3f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0000c00 tx=0x7f4dd0031400 comp rx=0 tx=0).stop 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc101000 msgr2=0x7f4ddc19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc101000 0x7f4ddc19fdc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800e970 tx=0x7f4dc800ed70 comp rx=0 tx=0).stop 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 shutdown_connections 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4dcc07af30 0x7f4dcc07d3f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ddc101000 0x7f4ddc19fdc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 --2- 172.21.15.7:0/1389792792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ddc100650 0x7f4ddc19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 >> 172.21.15.7:0/1389792792 conn(0x7f4ddc0fc520 msgr2=0x7f4ddc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 shutdown_connections 2024-09-21T04:58:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:05.014+0000 7f4de2bf4640 1 -- 172.21.15.7:0/1389792792 wait complete. 2024-09-21T04:58:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:05 smithi007 ceph-mon[108238]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:05 smithi007 ceph-mon[108238]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:05 smithi007 ceph-mon[108238]: pgmap v699: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:05 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2720099702' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:58:05.195 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:05 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1389792792' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:58:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:05 smithi053 ceph-mon[80176]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:05 smithi053 ceph-mon[80176]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:05 smithi053 ceph-mon[80176]: pgmap v699: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:05 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2720099702' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:58:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:05 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1389792792' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:58:06.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:06 smithi053 ceph-mon[80176]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:06.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:06 smithi007 ceph-mon[108238]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:07 smithi053 ceph-mon[80176]: pgmap v700: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:07 smithi007 ceph-mon[108238]: pgmap v700: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:09.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:09 smithi053 ceph-mon[80176]: pgmap v701: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:09 smithi007 ceph-mon[108238]: pgmap v701: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:11 smithi007 ceph-mon[108238]: pgmap v702: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:11 smithi053 ceph-mon[80176]: pgmap v702: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:13 smithi053 ceph-mon[80176]: pgmap v703: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:13 smithi007 ceph-mon[108238]: pgmap v703: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:58:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:58:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:15 smithi053 ceph-mon[80176]: pgmap v704: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:15 smithi007 ceph-mon[108238]: pgmap v704: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:17 smithi007 ceph-mon[108238]: pgmap v705: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:17 smithi053 ceph-mon[80176]: pgmap v705: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:19 smithi053 ceph-mon[80176]: pgmap v706: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:19 smithi007 ceph-mon[108238]: pgmap v706: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:21 smithi053 ceph-mon[80176]: pgmap v707: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:21 smithi007 ceph-mon[108238]: pgmap v707: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:23.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:23 smithi053 ceph-mon[80176]: pgmap v708: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:23.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:23 smithi007 ceph-mon[108238]: pgmap v708: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:25 smithi053 ceph-mon[80176]: pgmap v709: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:25 smithi007 ceph-mon[108238]: pgmap v709: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:27 smithi053 ceph-mon[80176]: pgmap v710: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:27 smithi007 ceph-mon[108238]: pgmap v710: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:29 smithi007 ceph-mon[108238]: pgmap v711: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:58:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:29 smithi053 ceph-mon[80176]: pgmap v711: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:58:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:31 smithi053 ceph-mon[80176]: pgmap v712: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:31 smithi007 ceph-mon[108238]: pgmap v712: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:33 smithi053 ceph-mon[80176]: pgmap v713: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:33 smithi007 ceph-mon[108238]: pgmap v713: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.113+0000 7f5a399ce640 1 -- 172.21.15.7:0/2289606083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074620 msgr2=0x7f5a34074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.113+0000 7f5a399ce640 1 --2- 172.21.15.7:0/2289606083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074620 0x7f5a34074a00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009880 tx=0x7f5a2402a530 comp rx=0 tx=0).stop 2024-09-21T04:58:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.113+0000 7f5a399ce640 1 -- 172.21.15.7:0/2289606083 shutdown_connections 2024-09-21T04:58:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.113+0000 7f5a399ce640 1 --2- 172.21.15.7:0/2289606083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a34074f40 0x7f5a3410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.113+0000 7f5a399ce640 1 --2- 172.21.15.7:0/2289606083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074620 0x7f5a34074a00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.113+0000 7f5a399ce640 1 -- 172.21.15.7:0/2289606083 >> 172.21.15.7:0/2289606083 conn(0x7f5a340fd600 msgr2=0x7f5a340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.113+0000 7f5a399ce640 1 -- 172.21.15.7:0/2289606083 shutdown_connections 2024-09-21T04:58:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.113+0000 7f5a399ce640 1 -- 172.21.15.7:0/2289606083 wait complete. 2024-09-21T04:58:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a399ce640 1 Processor -- start 2024-09-21T04:58:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a399ce640 1 -- start start 2024-09-21T04:58:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a399ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a34074620 0x7f5a3419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a399ce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074f40 0x7f5a3419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a399ce640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34110fd0 con 0x7f5a34074f40 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a399ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34110e50 con 0x7f5a34074620 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a327fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074f40 0x7f5a3419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a327fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074f40 0x7f5a3419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57196/0 (socket says 172.21.15.7:57196) 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a327fc640 1 -- 172.21.15.7:0/4165253036 learned_addr learned my addr 172.21.15.7:0/4165253036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.114+0000 7f5a32ffd640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a34074620 0x7f5a3419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a327fc640 1 -- 172.21.15.7:0/4165253036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a34074620 msgr2=0x7f5a3419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a327fc640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a34074620 0x7f5a3419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a327fc640 1 -- 172.21.15.7:0/4165253036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a34199a00 con 0x7f5a34074f40 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a32ffd640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a34074620 0x7f5a3419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a327fc640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074f40 0x7f5a3419fd80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800d8d0 tx=0x7f5a2800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a389cc640 1 -- 172.21.15.7:0/4165253036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5a28017030 con 0x7f5a34074f40 2024-09-21T04:58:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a389cc640 1 -- 172.21.15.7:0/4165253036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a28004020 con 0x7f5a34074f40 2024-09-21T04:58:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a34199cf0 con 0x7f5a34074f40 2024-09-21T04:58:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a389cc640 1 -- 172.21.15.7:0/4165253036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5a280047a0 con 0x7f5a34074f40 2024-09-21T04:58:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.115+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3419a280 con 0x7f5a34074f40 2024-09-21T04:58:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.116+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a34075cd0 con 0x7f5a34074f40 2024-09-21T04:58:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.117+0000 7f5a389cc640 1 -- 172.21.15.7:0/4165253036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5a280041d0 con 0x7f5a34074f40 2024-09-21T04:58:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.118+0000 7f5a389cc640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a1c07af90 0x7f5a1c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.118+0000 7f5a389cc640 1 -- 172.21.15.7:0/4165253036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5a28018070 con 0x7f5a34074f40 2024-09-21T04:58:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.118+0000 7f5a32ffd640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a1c07af90 0x7f5a1c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.118+0000 7f5a32ffd640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a1c07af90 0x7f5a1c07d450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5a24000c00 tx=0x7f5a24031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:35.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.122+0000 7f5a389cc640 1 -- 172.21.15.7:0/4165253036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a280a4080 con 0x7f5a34074f40 2024-09-21T04:58:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:34 smithi007 ceph-mon[108238]: pgmap v714: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.329+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5a3419af30 con 0x7f5a1c07af90 2024-09-21T04:58:35.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.330+0000 7f5a389cc640 1 -- 172.21.15.7:0/4165253036 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f5a3419af30 con 0x7f5a1c07af90 2024-09-21T04:58:35.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a1c07af90 msgr2=0x7f5a1c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a1c07af90 0x7f5a1c07d450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5a24000c00 tx=0x7f5a24031400 comp rx=0 tx=0).stop 2024-09-21T04:58:35.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074f40 msgr2=0x7f5a3419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074f40 0x7f5a3419fd80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800d8d0 tx=0x7f5a2800dd90 comp rx=0 tx=0).stop 2024-09-21T04:58:35.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 shutdown_connections 2024-09-21T04:58:35.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5a1c07af90 0x7f5a1c07d450 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a34074f40 0x7f5a3419fd80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 --2- 172.21.15.7:0/4165253036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a34074620 0x7f5a3419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 >> 172.21.15.7:0/4165253036 conn(0x7f5a340fd600 msgr2=0x7f5a3410b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:35.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 shutdown_connections 2024-09-21T04:58:35.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.332+0000 7f5a399ce640 1 -- 172.21.15.7:0/4165253036 wait complete. 2024-09-21T04:58:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:58:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.418+0000 7fab2ce89640 1 -- 172.21.15.7:0/3994407400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28101900 msgr2=0x7fab28101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.418+0000 7fab2ce89640 1 --2- 172.21.15.7:0/3994407400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28101900 0x7fab28101d60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009880 tx=0x7fab1c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.418+0000 7fab2ce89640 1 -- 172.21.15.7:0/3994407400 shutdown_connections 2024-09-21T04:58:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.418+0000 7fab2ce89640 1 --2- 172.21.15.7:0/3994407400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28101900 0x7fab28101d60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.418+0000 7fab2ce89640 1 --2- 172.21.15.7:0/3994407400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab28107900 0x7fab28107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.418+0000 7fab2ce89640 1 -- 172.21.15.7:0/3994407400 >> 172.21.15.7:0/3994407400 conn(0x7fab280fd600 msgr2=0x7fab280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.418+0000 7fab2ce89640 1 -- 172.21.15.7:0/3994407400 shutdown_connections 2024-09-21T04:58:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.418+0000 7fab2ce89640 1 -- 172.21.15.7:0/3994407400 wait complete. 2024-09-21T04:58:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.419+0000 7fab2ce89640 1 Processor -- start 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.419+0000 7fab2ce89640 1 -- start start 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.419+0000 7fab2ce89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab28101900 0x7fab2819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.419+0000 7fab2ce89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28107900 0x7fab2819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.419+0000 7fab2ce89640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab2810edd0 con 0x7fab28107900 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.419+0000 7fab2ce89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab2810ec50 con 0x7fab28101900 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.419+0000 7fab25d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28107900 0x7fab2819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab25d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28107900 0x7fab2819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57224/0 (socket says 172.21.15.7:57224) 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab25d9b640 1 -- 172.21.15.7:0/260157206 learned_addr learned my addr 172.21.15.7:0/260157206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab2659c640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab28101900 0x7fab2819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab25d9b640 1 -- 172.21.15.7:0/260157206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab28101900 msgr2=0x7fab2819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab25d9b640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab28101900 0x7fab2819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab25d9b640 1 -- 172.21.15.7:0/260157206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab281977a0 con 0x7fab28107900 2024-09-21T04:58:35.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab25d9b640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28107900 0x7fab2819db60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fab1c031460 tx=0x7fab1c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab177fe640 1 -- 172.21.15.7:0/260157206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fab1c03f070 con 0x7fab28107900 2024-09-21T04:58:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab28197a30 con 0x7fab28107900 2024-09-21T04:58:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab177fe640 1 -- 172.21.15.7:0/260157206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab1c0035f0 con 0x7fab28107900 2024-09-21T04:58:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab177fe640 1 -- 172.21.15.7:0/260157206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fab1c035030 con 0x7fab28107900 2024-09-21T04:58:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.420+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab28197f60 con 0x7fab28107900 2024-09-21T04:58:35.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.421+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab28103040 con 0x7fab28107900 2024-09-21T04:58:35.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.422+0000 7fab177fe640 1 -- 172.21.15.7:0/260157206 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fab1c042070 con 0x7fab28107900 2024-09-21T04:58:35.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.422+0000 7fab177fe640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab1007af30 0x7fab1007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.423+0000 7fab177fe640 1 -- 172.21.15.7:0/260157206 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fab1c0bea30 con 0x7fab28107900 2024-09-21T04:58:35.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.423+0000 7fab2659c640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab1007af30 0x7fab1007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.423+0000 7fab2659c640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab1007af30 0x7fab1007d3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab18005fd0 tx=0x7fab1800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.427+0000 7fab177fe640 1 -- 172.21.15.7:0/260157206 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab1c089f70 con 0x7fab28107900 2024-09-21T04:58:35.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:34 smithi053 ceph-mon[80176]: pgmap v714: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.633+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fab281982d0 con 0x7fab1007af30 2024-09-21T04:58:35.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.634+0000 7fab177fe640 1 -- 172.21.15.7:0/260157206 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fab281982d0 con 0x7fab1007af30 2024-09-21T04:58:35.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab1007af30 msgr2=0x7fab1007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab1007af30 0x7fab1007d3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab18005fd0 tx=0x7fab1800b390 comp rx=0 tx=0).stop 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28107900 msgr2=0x7fab2819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28107900 0x7fab2819db60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fab1c031460 tx=0x7fab1c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 shutdown_connections 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab1007af30 0x7fab1007d3f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab28107900 0x7fab2819db60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 --2- 172.21.15.7:0/260157206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab28101900 0x7fab2819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 >> 172.21.15.7:0/260157206 conn(0x7fab280fd600 msgr2=0x7fab280fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 shutdown_connections 2024-09-21T04:58:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.637+0000 7fab2ce89640 1 -- 172.21.15.7:0/260157206 wait complete. 2024-09-21T04:58:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 -- 172.21.15.7:0/714990521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 msgr2=0x7fad78101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 --2- 172.21.15.7:0/714990521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 0x7fad78101d60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009880 tx=0x7fad6c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 -- 172.21.15.7:0/714990521 shutdown_connections 2024-09-21T04:58:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 --2- 172.21.15.7:0/714990521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 0x7fad78101d60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 --2- 172.21.15.7:0/714990521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad78107900 0x7fad78107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 -- 172.21.15.7:0/714990521 >> 172.21.15.7:0/714990521 conn(0x7fad780fd600 msgr2=0x7fad780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 -- 172.21.15.7:0/714990521 shutdown_connections 2024-09-21T04:58:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 -- 172.21.15.7:0/714990521 wait complete. 2024-09-21T04:58:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.721+0000 7fad7d920640 1 Processor -- start 2024-09-21T04:58:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad7d920640 1 -- start start 2024-09-21T04:58:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad7d920640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 0x7fad780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad7d920640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad78107900 0x7fad78074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad7d920640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810ee30 con 0x7fad78101900 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad7d920640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810ecb0 con 0x7fad78107900 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad76ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 0x7fad780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad76ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 0x7fad780744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57236/0 (socket says 172.21.15.7:57236) 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad76ffd640 1 -- 172.21.15.7:0/1020890162 learned_addr learned my addr 172.21.15.7:0/1020890162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad767fc640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad78107900 0x7fad78074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad76ffd640 1 -- 172.21.15.7:0/1020890162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad78107900 msgr2=0x7fad78074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad76ffd640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad78107900 0x7fad78074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad76ffd640 1 -- 172.21.15.7:0/1020890162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad78074f70 con 0x7fad78101900 2024-09-21T04:58:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.722+0000 7fad767fc640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad78107900 0x7fad78074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:58:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.723+0000 7fad76ffd640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 0x7fad780744f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fad6800b420 tx=0x7fad6800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.723+0000 7fad7c91e640 1 -- 172.21.15.7:0/1020890162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fad68011630 con 0x7fad78101900 2024-09-21T04:58:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.723+0000 7fad7c91e640 1 -- 172.21.15.7:0/1020890162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad68016070 con 0x7fad78101900 2024-09-21T04:58:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.723+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad78075260 con 0x7fad78101900 2024-09-21T04:58:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.723+0000 7fad7c91e640 1 -- 172.21.15.7:0/1020890162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fad68010d40 con 0x7fad78101900 2024-09-21T04:58:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.723+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad781a3ec0 con 0x7fad78101900 2024-09-21T04:58:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.724+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad78103040 con 0x7fad78101900 2024-09-21T04:58:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.724+0000 7fad7c91e640 1 -- 172.21.15.7:0/1020890162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fad6801b460 con 0x7fad78101900 2024-09-21T04:58:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.725+0000 7fad7c91e640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fad6007af90 0x7fad6007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.725+0000 7fad7c91e640 1 -- 172.21.15.7:0/1020890162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fad6809d7a0 con 0x7fad78101900 2024-09-21T04:58:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.725+0000 7fad767fc640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fad6007af90 0x7fad6007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.726+0000 7fad767fc640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fad6007af90 0x7fad6007d450 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fad6c031460 tx=0x7fad6c004280 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:35.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.730+0000 7fad7c91e640 1 -- 172.21.15.7:0/1020890162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad68021250 con 0x7fad78101900 2024-09-21T04:58:35.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.925+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fad78062630 con 0x7fad6007af90 2024-09-21T04:58:35.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.935+0000 7fad7c91e640 1 -- 172.21.15.7:0/1020890162 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fad78062630 con 0x7fad6007af90 2024-09-21T04:58:35.938 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:58:35.938 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (20m) 14m ago 31m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:58:35.938 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (23m) 14m ago 31m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:58:35.938 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (23m) 14m ago 30m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:58:35.938 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (18m) 14m ago 31m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:58:35.938 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (18m) 14m ago 30m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (20m) 14m ago 31m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (14m) 14m ago 26m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (15m) 14m ago 26m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (26m) 14m ago 26m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (14m) 14m ago 26m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (22m) 14m ago 32m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (21m) 14m ago 30m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (19m) 14m ago 33m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (19m) 14m ago 30m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (21m) 14m ago 31m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (21m) 14m ago 30m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:58:35.939 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (18m) 14m ago 29m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:58:35.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (17m) 14m ago 29m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:58:35.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (17m) 14m ago 28m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:58:35.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (16m) 14m ago 28m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:58:35.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (15m) 14m ago 27m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:58:35.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (15m) 14m ago 27m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:58:35.940 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (20m) 14m ago 31m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fad6007af90 msgr2=0x7fad6007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fad6007af90 0x7fad6007d450 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fad6c031460 tx=0x7fad6c004280 comp rx=0 tx=0).stop 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 msgr2=0x7fad780744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 0x7fad780744f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fad6800b420 tx=0x7fad6800b8e0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 shutdown_connections 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fad6007af90 0x7fad6007d450 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad78107900 0x7fad78074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 --2- 172.21.15.7:0/1020890162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad78101900 0x7fad780744f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 >> 172.21.15.7:0/1020890162 conn(0x7fad780fd600 msgr2=0x7fad780fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:35.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 shutdown_connections 2024-09-21T04:58:35.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:35.937+0000 7fad7d920640 1 -- 172.21.15.7:0/1020890162 wait complete. 2024-09-21T04:58:36.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.020+0000 7fdf2cba6640 1 -- 172.21.15.7:0/3020023101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 msgr2=0x7fdf28074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.020+0000 7fdf2cba6640 1 --2- 172.21.15.7:0/3020023101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 0x7fdf28074970 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009880 tx=0x7fdf1802a530 comp rx=0 tx=0).stop 2024-09-21T04:58:36.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.020+0000 7fdf2cba6640 1 -- 172.21.15.7:0/3020023101 shutdown_connections 2024-09-21T04:58:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.020+0000 7fdf2cba6640 1 --2- 172.21.15.7:0/3020023101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf28074eb0 0x7fdf281123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.020+0000 7fdf2cba6640 1 --2- 172.21.15.7:0/3020023101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 0x7fdf28074970 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.020+0000 7fdf2cba6640 1 -- 172.21.15.7:0/3020023101 >> 172.21.15.7:0/3020023101 conn(0x7fdf280fd600 msgr2=0x7fdf280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.020+0000 7fdf2cba6640 1 -- 172.21.15.7:0/3020023101 shutdown_connections 2024-09-21T04:58:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.020+0000 7fdf2cba6640 1 -- 172.21.15.7:0/3020023101 wait complete. 2024-09-21T04:58:36.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2cba6640 1 Processor -- start 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2cba6640 1 -- start start 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2cba6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 0x7fdf2819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2cba6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf28074eb0 0x7fdf2819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2cba6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf281153d0 con 0x7fdf28074590 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2cba6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf28115250 con 0x7fdf28074eb0 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 0x7fdf2819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 0x7fdf2819dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57244/0 (socket says 172.21.15.7:57244) 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.021+0000 7fdf2659c640 1 -- 172.21.15.7:0/1624222908 learned_addr learned my addr 172.21.15.7:0/1624222908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf1ffff640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf28074eb0 0x7fdf2819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf2659c640 1 -- 172.21.15.7:0/1624222908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf28074eb0 msgr2=0x7fdf2819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf2659c640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf28074eb0 0x7fdf2819e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf2659c640 1 -- 172.21.15.7:0/1624222908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2819ebc0 con 0x7fdf28074590 2024-09-21T04:58:36.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf1ffff640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf28074eb0 0x7fdf2819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:58:36.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf2659c640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 0x7fdf2819dfd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdf18031460 tx=0x7fdf180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:36.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf1f7fe640 1 -- 172.21.15.7:0/1624222908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf1803f070 con 0x7fdf28074590 2024-09-21T04:58:36.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf281a2900 con 0x7fdf28074590 2024-09-21T04:58:36.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf1f7fe640 1 -- 172.21.15.7:0/1624222908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf18004cf0 con 0x7fdf28074590 2024-09-21T04:58:36.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf1f7fe640 1 -- 172.21.15.7:0/1624222908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf18035030 con 0x7fdf28074590 2024-09-21T04:58:36.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.022+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf281a2e30 con 0x7fdf28074590 2024-09-21T04:58:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.023+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf28075cd0 con 0x7fdf28074590 2024-09-21T04:58:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.024+0000 7fdf1f7fe640 1 -- 172.21.15.7:0/1624222908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdf18042070 con 0x7fdf28074590 2024-09-21T04:58:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.024+0000 7fdf1f7fe640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf1407af30 0x7fdf1407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.025+0000 7fdf1f7fe640 1 -- 172.21.15.7:0/1624222908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fdf180bebc0 con 0x7fdf28074590 2024-09-21T04:58:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.025+0000 7fdf1ffff640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf1407af30 0x7fdf1407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.025+0000 7fdf1ffff640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf1407af30 0x7fdf1407d3f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf28104880 tx=0x7fdf1000a5c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.029+0000 7fdf1f7fe640 1 -- 172.21.15.7:0/1624222908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdf1808a0c0 con 0x7fdf28074590 2024-09-21T04:58:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.297+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fdf2810fac0 con 0x7fdf28074590 2024-09-21T04:58:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.298+0000 7fdf1f7fe640 1 -- 172.21.15.7:0/1624222908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fdf1808c5e0 con 0x7fdf28074590 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:58:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:58:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:58:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:58:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:58:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:58:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:58:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:58:36.302 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf1407af30 msgr2=0x7fdf1407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf1407af30 0x7fdf1407d3f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf28104880 tx=0x7fdf1000a5c0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 msgr2=0x7fdf2819dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 0x7fdf2819dfd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdf18031460 tx=0x7fdf180048f0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 shutdown_connections 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf1407af30 0x7fdf1407d3f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf28074eb0 0x7fdf2819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 --2- 172.21.15.7:0/1624222908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf28074590 0x7fdf2819dfd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 >> 172.21.15.7:0/1624222908 conn(0x7fdf280fd600 msgr2=0x7fdf28104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:36.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 shutdown_connections 2024-09-21T04:58:36.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.300+0000 7fdf2cba6640 1 -- 172.21.15.7:0/1624222908 wait complete. 2024-09-21T04:58:36.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.384+0000 7efece990640 1 -- 172.21.15.7:0/1252362479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 msgr2=0x7efec810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.384+0000 7efece990640 1 --2- 172.21.15.7:0/1252362479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 0x7efec810b8d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efebc009880 tx=0x7efebc02a530 comp rx=0 tx=0).stop 2024-09-21T04:58:36.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.384+0000 7efece990640 1 -- 172.21.15.7:0/1252362479 shutdown_connections 2024-09-21T04:58:36.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.384+0000 7efece990640 1 --2- 172.21.15.7:0/1252362479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 0x7efec810b8d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.384+0000 7efece990640 1 --2- 172.21.15.7:0/1252362479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec8102ac0 0x7efec8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.384+0000 7efece990640 1 -- 172.21.15.7:0/1252362479 >> 172.21.15.7:0/1252362479 conn(0x7efec80fc520 msgr2=0x7efec80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:36.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.385+0000 7efece990640 1 -- 172.21.15.7:0/1252362479 shutdown_connections 2024-09-21T04:58:36.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.385+0000 7efece990640 1 -- 172.21.15.7:0/1252362479 wait complete. 2024-09-21T04:58:36.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.385+0000 7efece990640 1 Processor -- start 2024-09-21T04:58:36.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.385+0000 7efece990640 1 -- start start 2024-09-21T04:58:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efece990640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec8102ac0 0x7efec8100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efece990640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 0x7efec8100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efece990640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efec810edd0 con 0x7efec81033e0 2024-09-21T04:58:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efece990640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efec810ec50 con 0x7efec8102ac0 2024-09-21T04:58:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 0x7efec8100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 0x7efec8100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57264/0 (socket says 172.21.15.7:57264) 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec77fe640 1 -- 172.21.15.7:0/3541882651 learned_addr learned my addr 172.21.15.7:0/3541882651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec7fff640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec8102ac0 0x7efec8100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec77fe640 1 -- 172.21.15.7:0/3541882651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec8102ac0 msgr2=0x7efec8100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec77fe640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec8102ac0 0x7efec8100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec77fe640 1 -- 172.21.15.7:0/3541882651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec81011a0 con 0x7efec81033e0 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec77fe640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 0x7efec8100c60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efebc031460 tx=0x7efebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.386+0000 7efec57fa640 1 -- 172.21.15.7:0/3541882651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efebc03f070 con 0x7efec81033e0 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.387+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec8101430 con 0x7efec81033e0 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.387+0000 7efec57fa640 1 -- 172.21.15.7:0/3541882651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efebc004ac0 con 0x7efec81033e0 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.387+0000 7efec57fa640 1 -- 172.21.15.7:0/3541882651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efebc035030 con 0x7efec81033e0 2024-09-21T04:58:36.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.387+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec81a3da0 con 0x7efec81033e0 2024-09-21T04:58:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.388+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec806aa90 con 0x7efec81033e0 2024-09-21T04:58:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.388+0000 7efec57fa640 1 -- 172.21.15.7:0/3541882651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7efebc042070 con 0x7efec81033e0 2024-09-21T04:58:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.389+0000 7efec57fa640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeb007af30 0x7efeb007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.389+0000 7efec57fa640 1 -- 172.21.15.7:0/3541882651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7efebc0be9e0 con 0x7efec81033e0 2024-09-21T04:58:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.389+0000 7efec7fff640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeb007af30 0x7efeb007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.389+0000 7efec7fff640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeb007af30 0x7efeb007d3f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efeb80058c0 tx=0x7efeb800a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.393+0000 7efec57fa640 1 -- 172.21.15.7:0/3541882651 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efebc089f80 con 0x7efec81033e0 2024-09-21T04:58:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.621+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7efec8101c50 con 0x7efec81033e0 2024-09-21T04:58:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.622+0000 7efec57fa640 1 -- 172.21.15.7:0/3541882651 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7efebc08ee30 con 0x7efec81033e0 2024-09-21T04:58:36.625 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:58:36.625 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:58:36.626 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:58:36.627 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:58:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:58:36.629 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:58:36.629 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:58:36.629 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:58:36.629 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:58:36.629 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:58:36.629 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:58:36.629 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:58:36.629 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeb007af30 msgr2=0x7efeb007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeb007af30 0x7efeb007d3f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efeb80058c0 tx=0x7efeb800a2d0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 msgr2=0x7efec8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 0x7efec8100c60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efebc031460 tx=0x7efebc0046c0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 shutdown_connections 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7efeb007af30 0x7efeb007d3f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efec81033e0 0x7efec8100c60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 --2- 172.21.15.7:0/3541882651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efec8102ac0 0x7efec8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 >> 172.21.15.7:0/3541882651 conn(0x7efec80fc520 msgr2=0x7efec8108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 shutdown_connections 2024-09-21T04:58:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.624+0000 7efece990640 1 -- 172.21.15.7:0/3541882651 wait complete. 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.709+0000 7f689ebe3640 1 -- 172.21.15.7:0/1444315435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68981068b0 msgr2=0x7f6898106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.709+0000 7f689ebe3640 1 --2- 172.21.15.7:0/1444315435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68981068b0 0x7f6898106c90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6888009880 tx=0x7f688802a530 comp rx=0 tx=0).stop 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.709+0000 7f689ebe3640 1 -- 172.21.15.7:0/1444315435 shutdown_connections 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.709+0000 7f689ebe3640 1 --2- 172.21.15.7:0/1444315435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6898100f70 0x7f68981013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.709+0000 7f689ebe3640 1 --2- 172.21.15.7:0/1444315435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68981068b0 0x7f6898106c90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.709+0000 7f689ebe3640 1 -- 172.21.15.7:0/1444315435 >> 172.21.15.7:0/1444315435 conn(0x7f68980fbc90 msgr2=0x7f68980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.709+0000 7f689ebe3640 1 -- 172.21.15.7:0/1444315435 shutdown_connections 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.709+0000 7f689ebe3640 1 -- 172.21.15.7:0/1444315435 wait complete. 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689ebe3640 1 Processor -- start 2024-09-21T04:58:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689ebe3640 1 -- start start 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689ebe3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6898100f70 0x7f6898102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689ebe3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68981068b0 0x7f68981029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689ebe3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f689810dd50 con 0x7f6898100f70 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689ebe3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f689810dbd0 con 0x7f68981068b0 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689c97f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6898100f70 0x7f6898102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689c97f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6898100f70 0x7f6898102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57286/0 (socket says 172.21.15.7:57286) 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.710+0000 7f689c97f640 1 -- 172.21.15.7:0/650206571 learned_addr learned my addr 172.21.15.7:0/650206571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f6897fff640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68981068b0 0x7f68981029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f689c97f640 1 -- 172.21.15.7:0/650206571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68981068b0 msgr2=0x7f68981029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f689c97f640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68981068b0 0x7f68981029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f689c97f640 1 -- 172.21.15.7:0/650206571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6898102ef0 con 0x7f6898100f70 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f6897fff640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68981068b0 0x7f68981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f689c97f640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6898100f70 0x7f6898102470 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6888031460 tx=0x7f68880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f6895ffb640 1 -- 172.21.15.7:0/650206571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f688803f070 con 0x7f6898100f70 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f6895ffb640 1 -- 172.21.15.7:0/650206571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6888004cf0 con 0x7f6898100f70 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f6895ffb640 1 -- 172.21.15.7:0/650206571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6888035030 con 0x7f6898100f70 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6898103180 con 0x7f6898100f70 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.711+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68981a3dd0 con 0x7f6898100f70 2024-09-21T04:58:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.712+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6898102190 con 0x7f6898100f70 2024-09-21T04:58:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.713+0000 7f6895ffb640 1 -- 172.21.15.7:0/650206571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6888042070 con 0x7f6898100f70 2024-09-21T04:58:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.714+0000 7f6895ffb640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f688007af80 0x7f688007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.714+0000 7f6895ffb640 1 -- 172.21.15.7:0/650206571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f68880bea50 con 0x7f6898100f70 2024-09-21T04:58:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.714+0000 7f6897fff640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f688007af80 0x7f688007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.714+0000 7f6897fff640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f688007af80 0x7f688007d440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6898103f00 tx=0x7f688c005d90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.718+0000 7f6895ffb640 1 -- 172.21.15.7:0/650206571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f688808a070 con 0x7f6898100f70 2024-09-21T04:58:36.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.922+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6898100690 con 0x7f688007af80 2024-09-21T04:58:36.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.924+0000 7f6895ffb640 1 -- 172.21.15.7:0/650206571 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f6898100690 con 0x7f688007af80 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:58:36.927 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:58:36.928 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:58:36.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f688007af80 msgr2=0x7f688007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f688007af80 0x7f688007d440 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6898103f00 tx=0x7f688c005d90 comp rx=0 tx=0).stop 2024-09-21T04:58:36.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6898100f70 msgr2=0x7f6898102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:36.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6898100f70 0x7f6898102470 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6888031460 tx=0x7f68880048f0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 shutdown_connections 2024-09-21T04:58:36.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f688007af80 0x7f688007d440 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68981068b0 0x7f68981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 --2- 172.21.15.7:0/650206571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6898100f70 0x7f6898102470 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:36.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 >> 172.21.15.7:0/650206571 conn(0x7f68980fbc90 msgr2=0x7f68980fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:36.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 shutdown_connections 2024-09-21T04:58:36.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:36.926+0000 7f689ebe3640 1 -- 172.21.15.7:0/650206571 wait complete. 2024-09-21T04:58:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.010+0000 7f4172b4b640 1 -- 172.21.15.7:0/2568292116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 msgr2=0x7f416c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.010+0000 7f4172b4b640 1 --2- 172.21.15.7:0/2568292116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 0x7f416c102ea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f415c009880 tx=0x7f415c02a530 comp rx=0 tx=0).stop 2024-09-21T04:58:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.010+0000 7f4172b4b640 1 -- 172.21.15.7:0/2568292116 shutdown_connections 2024-09-21T04:58:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.010+0000 7f4172b4b640 1 --2- 172.21.15.7:0/2568292116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f416c1033e0 0x7f416c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.010+0000 7f4172b4b640 1 --2- 172.21.15.7:0/2568292116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 0x7f416c102ea0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.010+0000 7f4172b4b640 1 -- 172.21.15.7:0/2568292116 >> 172.21.15.7:0/2568292116 conn(0x7f416c0fc520 msgr2=0x7f416c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.010+0000 7f4172b4b640 1 -- 172.21.15.7:0/2568292116 shutdown_connections 2024-09-21T04:58:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.011+0000 7f4172b4b640 1 -- 172.21.15.7:0/2568292116 wait complete. 2024-09-21T04:58:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.011+0000 7f4172b4b640 1 Processor -- start 2024-09-21T04:58:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.011+0000 7f4172b4b640 1 -- start start 2024-09-21T04:58:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.011+0000 7f4172b4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 0x7f416c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.011+0000 7f4172b4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f416c1033e0 0x7f416c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.011+0000 7f4172b4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10edd0 con 0x7f416c102ac0 2024-09-21T04:58:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.011+0000 7f4172b4b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10ec50 con 0x7f416c1033e0 2024-09-21T04:58:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.011+0000 7f41708e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 0x7f416c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f41708e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 0x7f416c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57312/0 (socket says 172.21.15.7:57312) 2024-09-21T04:58:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f41708e7640 1 -- 172.21.15.7:0/622626398 learned_addr learned my addr 172.21.15.7:0/622626398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f416bfff640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f416c1033e0 0x7f416c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f41708e7640 1 -- 172.21.15.7:0/622626398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f416c1033e0 msgr2=0x7f416c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f41708e7640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f416c1033e0 0x7f416c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f41708e7640 1 -- 172.21.15.7:0/622626398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1a03f0 con 0x7f416c102ac0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f416bfff640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f416c1033e0 0x7f416c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f41708e7640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 0x7f416c19f800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f415c031460 tx=0x7f415c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f4169ffb640 1 -- 172.21.15.7:0/622626398 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f415c03f070 con 0x7f416c102ac0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f4169ffb640 1 -- 172.21.15.7:0/622626398 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f415c004cf0 con 0x7f416c102ac0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c199ad0 con 0x7f416c102ac0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f4169ffb640 1 -- 172.21.15.7:0/622626398 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f415c035030 con 0x7f416c102ac0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.012+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c199fd0 con 0x7f416c102ac0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.013+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c06aa90 con 0x7f416c102ac0 2024-09-21T04:58:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.014+0000 7f4169ffb640 1 -- 172.21.15.7:0/622626398 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f415c042070 con 0x7f416c102ac0 2024-09-21T04:58:37.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.015+0000 7f4169ffb640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f415407af30 0x7f415407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:58:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.015+0000 7f4169ffb640 1 -- 172.21.15.7:0/622626398 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f415c0bebc0 con 0x7f416c102ac0 2024-09-21T04:58:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.015+0000 7f416bfff640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f415407af30 0x7f415407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:58:37.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.015+0000 7f416bfff640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f415407af30 0x7f415407d3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4160005e10 tx=0x7f4160005da0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:58:37.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.019+0000 7f4169ffb640 1 -- 172.21.15.7:0/622626398 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f415c08a0c0 con 0x7f416c102ac0 2024-09-21T04:58:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:37 smithi007 ceph-mon[108238]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:37 smithi007 ceph-mon[108238]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:37 smithi007 ceph-mon[108238]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:37 smithi007 ceph-mon[108238]: pgmap v715: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:37 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1624222908' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:58:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:37 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3541882651' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.313+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f416c19af90 con 0x7f416c102ac0 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.314+0000 7f4169ffb640 1 -- 172.21.15.7:0/622626398 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f415c08ef70 con 0x7f416c102ac0 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:58:37.317 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f415407af30 msgr2=0x7f415407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f415407af30 0x7f415407d3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4160005e10 tx=0x7f4160005da0 comp rx=0 tx=0).stop 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 msgr2=0x7f416c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 0x7f416c19f800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f415c031460 tx=0x7f415c0048f0 comp rx=0 tx=0).stop 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 shutdown_connections 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f415407af30 0x7f415407d3f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f416c1033e0 0x7f416c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 --2- 172.21.15.7:0/622626398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c102ac0 0x7f416c19f800 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 >> 172.21.15.7:0/622626398 conn(0x7f416c0fc520 msgr2=0x7f416c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 shutdown_connections 2024-09-21T04:58:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:58:37.317+0000 7f4172b4b640 1 -- 172.21.15.7:0/622626398 wait complete. 2024-09-21T04:58:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:37 smithi053 ceph-mon[80176]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:37 smithi053 ceph-mon[80176]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:37 smithi053 ceph-mon[80176]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:37 smithi053 ceph-mon[80176]: pgmap v715: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:37 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1624222908' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:58:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:37 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3541882651' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:58:38.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:38 smithi007 ceph-mon[108238]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:38.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:38 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/622626398' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:58:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:38 smithi053 ceph-mon[80176]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:58:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:38 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/622626398' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:58:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:39 smithi053 ceph-mon[80176]: pgmap v716: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:39 smithi007 ceph-mon[108238]: pgmap v716: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:41 smithi053 ceph-mon[80176]: pgmap v717: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:41.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:41 smithi007 ceph-mon[108238]: pgmap v717: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:43 smithi053 ceph-mon[80176]: pgmap v718: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:43 smithi007 ceph-mon[108238]: pgmap v718: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:58:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:58:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:45 smithi053 ceph-mon[80176]: pgmap v719: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:45 smithi007 ceph-mon[108238]: pgmap v719: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:47 smithi007 ceph-mon[108238]: pgmap v720: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:47 smithi053 ceph-mon[80176]: pgmap v720: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:49 smithi053 ceph-mon[80176]: pgmap v721: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:49 smithi007 ceph-mon[108238]: pgmap v721: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:51 smithi053 ceph-mon[80176]: pgmap v722: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:51 smithi007 ceph-mon[108238]: pgmap v722: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:53 smithi007 ceph-mon[108238]: pgmap v723: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:53 smithi053 ceph-mon[80176]: pgmap v723: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:55 smithi053 ceph-mon[80176]: pgmap v724: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:55 smithi007 ceph-mon[108238]: pgmap v724: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:57 smithi053 ceph-mon[80176]: pgmap v725: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:57 smithi007 ceph-mon[108238]: pgmap v725: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:59 smithi007 ceph-mon[108238]: pgmap v726: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:58:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:58:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:59 smithi053 ceph-mon[80176]: pgmap v726: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:58:59.448 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:58:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:59:01.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:01 smithi053 ceph-mon[80176]: pgmap v727: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:01 smithi007 ceph-mon[108238]: pgmap v727: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:03 smithi053 ceph-mon[80176]: pgmap v728: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:03 smithi007 ceph-mon[108238]: pgmap v728: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:05 smithi007 ceph-mon[108238]: pgmap v729: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:05 smithi053 ceph-mon[80176]: pgmap v729: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.411+0000 7f9144dea640 1 -- 172.21.15.7:0/4224076298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91401033e0 msgr2=0x7f914010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.411+0000 7f9144dea640 1 --2- 172.21.15.7:0/4224076298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91401033e0 0x7f914010b8d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9128009880 tx=0x7f912802a530 comp rx=0 tx=0).stop 2024-09-21T04:59:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.411+0000 7f9144dea640 1 -- 172.21.15.7:0/4224076298 shutdown_connections 2024-09-21T04:59:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.411+0000 7f9144dea640 1 --2- 172.21.15.7:0/4224076298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91401033e0 0x7f914010b8d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.411+0000 7f9144dea640 1 --2- 172.21.15.7:0/4224076298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9140102ac0 0x7f9140102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.411+0000 7f9144dea640 1 -- 172.21.15.7:0/4224076298 >> 172.21.15.7:0/4224076298 conn(0x7f91400fc520 msgr2=0x7f91400fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.411+0000 7f9144dea640 1 -- 172.21.15.7:0/4224076298 shutdown_connections 2024-09-21T04:59:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.411+0000 7f9144dea640 1 -- 172.21.15.7:0/4224076298 wait complete. 2024-09-21T04:59:07.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.412+0000 7f9144dea640 1 Processor -- start 2024-09-21T04:59:07.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.412+0000 7f9144dea640 1 -- start start 2024-09-21T04:59:07.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f9144dea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9140102ac0 0x7f9140100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f9144dea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91401033e0 0x7f9140100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f9144dea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f914010edf0 con 0x7f9140102ac0 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f9144dea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f914010ec70 con 0x7f91401033e0 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f913e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9140102ac0 0x7f9140100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f913e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9140102ac0 0x7f9140100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46336/0 (socket says 172.21.15.7:46336) 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f913e59c640 1 -- 172.21.15.7:0/3897757221 learned_addr learned my addr 172.21.15.7:0/3897757221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f9135d9b640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91401033e0 0x7f9140100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f913e59c640 1 -- 172.21.15.7:0/3897757221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91401033e0 msgr2=0x7f9140100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f913e59c640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91401033e0 0x7f9140100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f913e59c640 1 -- 172.21.15.7:0/3897757221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91401011a0 con 0x7f9140102ac0 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.413+0000 7f913e59c640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9140102ac0 0x7f9140100720 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f913000d6f0 tx=0x7f913000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:07.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.414+0000 7f9137fff640 1 -- 172.21.15.7:0/3897757221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9130017070 con 0x7f9140102ac0 2024-09-21T04:59:07.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.414+0000 7f9137fff640 1 -- 172.21.15.7:0/3897757221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f913000de00 con 0x7f9140102ac0 2024-09-21T04:59:07.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.414+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9140101490 con 0x7f9140102ac0 2024-09-21T04:59:07.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.414+0000 7f9137fff640 1 -- 172.21.15.7:0/3897757221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9130004bb0 con 0x7f9140102ac0 2024-09-21T04:59:07.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.414+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401a3ea0 con 0x7f9140102ac0 2024-09-21T04:59:07.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.415+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914006aa90 con 0x7f9140102ac0 2024-09-21T04:59:07.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.415+0000 7f9137fff640 1 -- 172.21.15.7:0/3897757221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9130004e60 con 0x7f9140102ac0 2024-09-21T04:59:07.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.416+0000 7f9137fff640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f912c07af30 0x7f912c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:07.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.416+0000 7f9137fff640 1 -- 172.21.15.7:0/3897757221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f913009dbe0 con 0x7f9140102ac0 2024-09-21T04:59:07.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.416+0000 7f9135d9b640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f912c07af30 0x7f912c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:07.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.417+0000 7f9135d9b640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f912c07af30 0x7f912c07d3f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f91401021b0 tx=0x7f9128031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:07.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.421+0000 7f9137fff640 1 -- 172.21.15.7:0/3897757221 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9130069180 con 0x7f9140102ac0 2024-09-21T04:59:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:07 smithi053 ceph-mon[80176]: pgmap v730: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:07 smithi007 ceph-mon[108238]: pgmap v730: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:07.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.625+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9140062630 con 0x7f912c07af30 2024-09-21T04:59:07.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.627+0000 7f9137fff640 1 -- 172.21.15.7:0/3897757221 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f9140062630 con 0x7f912c07af30 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f912c07af30 msgr2=0x7f912c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f912c07af30 0x7f912c07d3f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f91401021b0 tx=0x7f9128031400 comp rx=0 tx=0).stop 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9140102ac0 msgr2=0x7f9140100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9140102ac0 0x7f9140100720 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f913000d6f0 tx=0x7f913000dbb0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 shutdown_connections 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f912c07af30 0x7f912c07d3f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91401033e0 0x7f9140100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 --2- 172.21.15.7:0/3897757221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9140102ac0 0x7f9140100720 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 >> 172.21.15.7:0/3897757221 conn(0x7f91400fc520 msgr2=0x7f9140108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 shutdown_connections 2024-09-21T04:59:07.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.629+0000 7f9144dea640 1 -- 172.21.15.7:0/3897757221 wait complete. 2024-09-21T04:59:07.642 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:59:07.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.715+0000 7f740f269640 1 -- 172.21.15.7:0/802301939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408103480 msgr2=0x7f740810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:07.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.715+0000 7f740f269640 1 --2- 172.21.15.7:0/802301939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408103480 0x7f740810b970 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009880 tx=0x7f73fc02a530 comp rx=0 tx=0).stop 2024-09-21T04:59:07.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.715+0000 7f740f269640 1 -- 172.21.15.7:0/802301939 shutdown_connections 2024-09-21T04:59:07.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.715+0000 7f740f269640 1 --2- 172.21.15.7:0/802301939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408103480 0x7f740810b970 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.715+0000 7f740f269640 1 --2- 172.21.15.7:0/802301939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7408102b60 0x7f7408102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.715+0000 7f740f269640 1 -- 172.21.15.7:0/802301939 >> 172.21.15.7:0/802301939 conn(0x7f74080fc520 msgr2=0x7f74080fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:07.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.715+0000 7f740f269640 1 -- 172.21.15.7:0/802301939 shutdown_connections 2024-09-21T04:59:07.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.715+0000 7f740f269640 1 -- 172.21.15.7:0/802301939 wait complete. 2024-09-21T04:59:07.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740f269640 1 Processor -- start 2024-09-21T04:59:07.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740f269640 1 -- start start 2024-09-21T04:59:07.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740f269640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408102b60 0x7f74081007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:07.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740f269640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7408103480 0x7f7408100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:07.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740f269640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f740810edd0 con 0x7f7408102b60 2024-09-21T04:59:07.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740f269640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f740810ec50 con 0x7f7408103480 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740d005640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408102b60 0x7f74081007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740d005640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408102b60 0x7f74081007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46348/0 (socket says 172.21.15.7:46348) 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740d005640 1 -- 172.21.15.7:0/725351875 learned_addr learned my addr 172.21.15.7:0/725351875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740c804640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7408103480 0x7f7408100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740d005640 1 -- 172.21.15.7:0/725351875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7408103480 msgr2=0x7f7408100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740d005640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7408103480 0x7f7408100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.716+0000 7f740d005640 1 -- 172.21.15.7:0/725351875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7408101240 con 0x7f7408102b60 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.717+0000 7f740d005640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408102b60 0x7f74081007c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f740400d8d0 tx=0x7f740400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.717+0000 7f73fa7fc640 1 -- 172.21.15.7:0/725351875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7404017030 con 0x7f7408102b60 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.717+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7408101530 con 0x7f7408102b60 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.717+0000 7f73fa7fc640 1 -- 172.21.15.7:0/725351875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7404004020 con 0x7f7408102b60 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.717+0000 7f73fa7fc640 1 -- 172.21.15.7:0/725351875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74040047a0 con 0x7f7408102b60 2024-09-21T04:59:07.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.717+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74081a3e80 con 0x7f7408102b60 2024-09-21T04:59:07.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.718+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f740806a9f0 con 0x7f7408102b60 2024-09-21T04:59:07.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.718+0000 7f73fa7fc640 1 -- 172.21.15.7:0/725351875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f74040041d0 con 0x7f7408102b60 2024-09-21T04:59:07.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.719+0000 7f73fa7fc640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73f407af30 0x7f73f407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.719+0000 7f740c804640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73f407af30 0x7f73f407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.719+0000 7f73fa7fc640 1 -- 172.21.15.7:0/725351875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7404018070 con 0x7f7408102b60 2024-09-21T04:59:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.720+0000 7f740c804640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73f407af30 0x7f73f407d3f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7408102250 tx=0x7f73fc004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:07.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.723+0000 7f73fa7fc640 1 -- 172.21.15.7:0/725351875 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7404069430 con 0x7f7408102b60 2024-09-21T04:59:07.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.932+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7408062630 con 0x7f73f407af30 2024-09-21T04:59:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.934+0000 7f73fa7fc640 1 -- 172.21.15.7:0/725351875 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f7408062630 con 0x7f73f407af30 2024-09-21T04:59:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73f407af30 msgr2=0x7f73f407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73f407af30 0x7f73f407d3f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7408102250 tx=0x7f73fc004f30 comp rx=0 tx=0).stop 2024-09-21T04:59:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408102b60 msgr2=0x7f74081007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408102b60 0x7f74081007c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f740400d8d0 tx=0x7f740400dd90 comp rx=0 tx=0).stop 2024-09-21T04:59:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 shutdown_connections 2024-09-21T04:59:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f73f407af30 0x7f73f407d3f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7408103480 0x7f7408100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 --2- 172.21.15.7:0/725351875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7408102b60 0x7f74081007c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 >> 172.21.15.7:0/725351875 conn(0x7f74080fc520 msgr2=0x7f7408103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.936+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 shutdown_connections 2024-09-21T04:59:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:07.937+0000 7f740f269640 1 -- 172.21.15.7:0/725351875 wait complete. 2024-09-21T04:59:08.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.021+0000 7f771eeca640 1 -- 172.21.15.7:0/235288511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718107900 msgr2=0x7f7718107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.021+0000 7f771eeca640 1 --2- 172.21.15.7:0/235288511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718107900 0x7f7718107ce0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7708009880 tx=0x7f770802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.021+0000 7f771eeca640 1 -- 172.21.15.7:0/235288511 shutdown_connections 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.021+0000 7f771eeca640 1 --2- 172.21.15.7:0/235288511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7718101900 0x7f7718101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.021+0000 7f771eeca640 1 --2- 172.21.15.7:0/235288511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718107900 0x7f7718107ce0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.021+0000 7f771eeca640 1 -- 172.21.15.7:0/235288511 >> 172.21.15.7:0/235288511 conn(0x7f77180fd600 msgr2=0x7f77180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.021+0000 7f771eeca640 1 -- 172.21.15.7:0/235288511 shutdown_connections 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.021+0000 7f771eeca640 1 -- 172.21.15.7:0/235288511 wait complete. 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771eeca640 1 Processor -- start 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771eeca640 1 -- start start 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771eeca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718101900 0x7f771819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771eeca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7718107900 0x7f771819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771eeca640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f771810edd0 con 0x7f7718101900 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771eeca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f771810ec50 con 0x7f7718107900 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771cc66640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718101900 0x7f771819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771cc66640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718101900 0x7f771819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46360/0 (socket says 172.21.15.7:46360) 2024-09-21T04:59:08.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f771cc66640 1 -- 172.21.15.7:0/3547071708 learned_addr learned my addr 172.21.15.7:0/3547071708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:08.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.022+0000 7f7717fff640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7718107900 0x7f771819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f771cc66640 1 -- 172.21.15.7:0/3547071708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7718107900 msgr2=0x7f771819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f771cc66640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7718107900 0x7f771819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f771cc66640 1 -- 172.21.15.7:0/3547071708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77181977a0 con 0x7f7718101900 2024-09-21T04:59:08.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f771cc66640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718101900 0x7f771819d620 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7708031460 tx=0x7f7708004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f7715ffb640 1 -- 172.21.15.7:0/3547071708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f770803f070 con 0x7f7718101900 2024-09-21T04:59:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f7715ffb640 1 -- 172.21.15.7:0/3547071708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77080035f0 con 0x7f7718101900 2024-09-21T04:59:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7718197a30 con 0x7f7718101900 2024-09-21T04:59:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f7715ffb640 1 -- 172.21.15.7:0/3547071708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7708035030 con 0x7f7718101900 2024-09-21T04:59:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.023+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7718197f60 con 0x7f7718101900 2024-09-21T04:59:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.024+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718103040 con 0x7f7718101900 2024-09-21T04:59:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.025+0000 7f7715ffb640 1 -- 172.21.15.7:0/3547071708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7708042070 con 0x7f7718101900 2024-09-21T04:59:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.025+0000 7f7715ffb640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f770407af80 0x7f770407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.025+0000 7f7717fff640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f770407af80 0x7f770407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.025+0000 7f7715ffb640 1 -- 172.21.15.7:0/3547071708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f77080be890 con 0x7f7718101900 2024-09-21T04:59:08.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.026+0000 7f7717fff640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f770407af80 0x7f770407d440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f77180ffc20 tx=0x7f7700005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:08.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.030+0000 7f7715ffb640 1 -- 172.21.15.7:0/3547071708 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7708089eb0 con 0x7f7718101900 2024-09-21T04:59:08.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.228+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7718198210 con 0x7f770407af80 2024-09-21T04:59:08.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.237+0000 7f7715ffb640 1 -- 172.21.15.7:0/3547071708 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f7718198210 con 0x7f770407af80 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (21m) 15m ago 32m 20.0M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (24m) 15m ago 32m 9252k - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (23m) 14m ago 31m 9105k - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (19m) 15m ago 32m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (19m) 14m ago 31m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (20m) 15m ago 31m 77.1M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (15m) 15m ago 26m 11.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (15m) 15m ago 26m 13.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (26m) 14m ago 26m 12.5M - 19.1.1-373-g4e45e6b4 164465d2334a c9847719d5d6 2024-09-21T04:59:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (15m) 14m ago 26m 14.2M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (23m) 15m ago 33m 532M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (22m) 14m ago 31m 457M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (19m) 15m ago 33m 50.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (19m) 14m ago 31m 41.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (22m) 15m ago 32m 11.2M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (21m) 14m ago 31m 11.0M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (18m) 15m ago 30m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (18m) 15m ago 29m 94.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (17m) 15m ago 29m 87.2M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (17m) 14m ago 28m 96.0M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (16m) 14m ago 28m 85.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (15m) 14m ago 27m 82.8M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:59:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (21m) 15m ago 31m 58.3M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:59:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f770407af80 msgr2=0x7f770407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f770407af80 0x7f770407d440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f77180ffc20 tx=0x7f7700005f90 comp rx=0 tx=0).stop 2024-09-21T04:59:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718101900 msgr2=0x7f771819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718101900 0x7f771819d620 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7708031460 tx=0x7f7708004530 comp rx=0 tx=0).stop 2024-09-21T04:59:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 shutdown_connections 2024-09-21T04:59:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f770407af80 0x7f770407d440 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7718107900 0x7f771819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 --2- 172.21.15.7:0/3547071708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7718101900 0x7f771819d620 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 >> 172.21.15.7:0/3547071708 conn(0x7f77180fd600 msgr2=0x7f77180fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 shutdown_connections 2024-09-21T04:59:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.240+0000 7f771eeca640 1 -- 172.21.15.7:0/3547071708 wait complete. 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.323+0000 7f0adc662640 1 -- 172.21.15.7:0/2357078028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410cf10 msgr2=0x7f0ad4110f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.323+0000 7f0adc662640 1 --2- 172.21.15.7:0/2357078028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410cf10 0x7f0ad4110f60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009880 tx=0x7f0ad002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.323+0000 7f0adc662640 1 -- 172.21.15.7:0/2357078028 shutdown_connections 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.323+0000 7f0adc662640 1 --2- 172.21.15.7:0/2357078028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410cf10 0x7f0ad4110f60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.323+0000 7f0adc662640 1 --2- 172.21.15.7:0/2357078028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad410c560 0x7f0ad410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.323+0000 7f0adc662640 1 -- 172.21.15.7:0/2357078028 >> 172.21.15.7:0/2357078028 conn(0x7f0ad406c5b0 msgr2=0x7f0ad406c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.323+0000 7f0adc662640 1 -- 172.21.15.7:0/2357078028 shutdown_connections 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.323+0000 7f0adc662640 1 -- 172.21.15.7:0/2357078028 wait complete. 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0adc662640 1 Processor -- start 2024-09-21T04:59:08.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0adc662640 1 -- start start 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0adc662640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410c560 0x7f0ad41a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0adc662640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad410cf10 0x7f0ad41a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0adc662640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad41197e0 con 0x7f0ad410c560 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0adc662640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad4119660 con 0x7f0ad410cf10 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0ada3fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410c560 0x7f0ad41a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0ada3fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410c560 0x7f0ad41a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46372/0 (socket says 172.21.15.7:46372) 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.324+0000 7f0ada3fe640 1 -- 172.21.15.7:0/2592654956 learned_addr learned my addr 172.21.15.7:0/2592654956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0ad9bfd640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad410cf10 0x7f0ad41a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0ada3fe640 1 -- 172.21.15.7:0/2592654956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad410cf10 msgr2=0x7f0ad41a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0ada3fe640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad410cf10 0x7f0ad41a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0ada3fe640 1 -- 172.21.15.7:0/2592654956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad41a6c80 con 0x7f0ad410c560 2024-09-21T04:59:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0ada3fe640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410c560 0x7f0ad41a23b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400e970 tx=0x7f0ac400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0acb7fe640 1 -- 172.21.15.7:0/2592654956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ac400cac0 con 0x7f0ad410c560 2024-09-21T04:59:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0acb7fe640 1 -- 172.21.15.7:0/2592654956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ac40054e0 con 0x7f0ad410c560 2024-09-21T04:59:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0acb7fe640 1 -- 172.21.15.7:0/2592654956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ac4010430 con 0x7f0ad410c560 2024-09-21T04:59:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad41a6f70 con 0x7f0ad410c560 2024-09-21T04:59:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.325+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad41a7500 con 0x7f0ad410c560 2024-09-21T04:59:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.326+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad410da40 con 0x7f0ad410c560 2024-09-21T04:59:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.327+0000 7f0acb7fe640 1 -- 172.21.15.7:0/2592654956 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0ac4004e80 con 0x7f0ad410c560 2024-09-21T04:59:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.327+0000 7f0acb7fe640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ac007af30 0x7f0ac007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.328+0000 7f0acb7fe640 1 -- 172.21.15.7:0/2592654956 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0ac409e750 con 0x7f0ad410c560 2024-09-21T04:59:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.328+0000 7f0ad9bfd640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ac007af30 0x7f0ac007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.328+0000 7f0ad9bfd640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ac007af30 0x7f0ac007d3f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0ad410be20 tx=0x7f0ad0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:08.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.332+0000 7f0acb7fe640 1 -- 172.21.15.7:0/2592654956 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ac4069c50 con 0x7f0ad410c560 2024-09-21T04:59:08.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:59:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:59:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:59:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:08 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:59:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:59:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:59:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:08 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.595+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f0ad4114870 con 0x7f0ad410c560 2024-09-21T04:59:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.595+0000 7f0acb7fe640 1 -- 172.21.15.7:0/2592654956 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f0ac406eb00 con 0x7f0ad410c560 2024-09-21T04:59:08.599 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:59:08.599 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:59:08.599 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 3 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.1.1-373-g4e45e6b4 (4e45e6b490e6cbe082cdf155fea780438ec53fc7) squid (stable)": 1, 2024-09-21T04:59:08.600 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 13 2024-09-21T04:59:08.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:59:08.601 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:59:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.597+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ac007af30 msgr2=0x7f0ac007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.597+0000 7f0adc662640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ac007af30 0x7f0ac007d3f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0ad410be20 tx=0x7f0ad0031400 comp rx=0 tx=0).stop 2024-09-21T04:59:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.597+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410c560 msgr2=0x7f0ad41a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.597+0000 7f0adc662640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410c560 0x7f0ad41a23b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400e970 tx=0x7f0ac400ed70 comp rx=0 tx=0).stop 2024-09-21T04:59:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.597+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 shutdown_connections 2024-09-21T04:59:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.597+0000 7f0adc662640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ac007af30 0x7f0ac007d3f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.597+0000 7f0adc662640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ad410cf10 0x7f0ad41a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.597+0000 7f0adc662640 1 --2- 172.21.15.7:0/2592654956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad410c560 0x7f0ad41a23b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.598+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 >> 172.21.15.7:0/2592654956 conn(0x7f0ad406c5b0 msgr2=0x7f0ad410a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.598+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 shutdown_connections 2024-09-21T04:59:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.598+0000 7f0adc662640 1 -- 172.21.15.7:0/2592654956 wait complete. 2024-09-21T04:59:08.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.681+0000 7f67739d2640 1 -- 172.21.15.7:0/3492719517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 msgr2=0x7f676c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.681+0000 7f67739d2640 1 --2- 172.21.15.7:0/3492719517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 0x7f676c101d60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f675c009880 tx=0x7f675c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.681+0000 7f67739d2640 1 -- 172.21.15.7:0/3492719517 shutdown_connections 2024-09-21T04:59:08.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.681+0000 7f67739d2640 1 --2- 172.21.15.7:0/3492719517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 0x7f676c101d60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.681+0000 7f67739d2640 1 --2- 172.21.15.7:0/3492719517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c107900 0x7f676c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.681+0000 7f67739d2640 1 -- 172.21.15.7:0/3492719517 >> 172.21.15.7:0/3492719517 conn(0x7f676c0fd600 msgr2=0x7f676c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:08.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.682+0000 7f67739d2640 1 -- 172.21.15.7:0/3492719517 shutdown_connections 2024-09-21T04:59:08.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.682+0000 7f67739d2640 1 -- 172.21.15.7:0/3492719517 wait complete. 2024-09-21T04:59:08.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.682+0000 7f67739d2640 1 Processor -- start 2024-09-21T04:59:08.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.682+0000 7f67739d2640 1 -- start start 2024-09-21T04:59:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.682+0000 7f67739d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 0x7f676c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f67739d2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c107900 0x7f676c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f67739d2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f676c10edd0 con 0x7f676c101900 2024-09-21T04:59:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f67739d2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f676c10ec50 con 0x7f676c107900 2024-09-21T04:59:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f677176e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 0x7f676c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f677176e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 0x7f676c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46402/0 (socket says 172.21.15.7:46402) 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f677176e640 1 -- 172.21.15.7:0/3678317094 learned_addr learned my addr 172.21.15.7:0/3678317094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f6770f6d640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c107900 0x7f676c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f677176e640 1 -- 172.21.15.7:0/3678317094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c107900 msgr2=0x7f676c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f677176e640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c107900 0x7f676c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f677176e640 1 -- 172.21.15.7:0/3678317094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c074f70 con 0x7f676c101900 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f677176e640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 0x7f676c0744f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f676800e970 tx=0x7f676800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f67627fc640 1 -- 172.21.15.7:0/3678317094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f676800cac0 con 0x7f676c101900 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c075260 con 0x7f676c101900 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f67627fc640 1 -- 172.21.15.7:0/3678317094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f67680054e0 con 0x7f676c101900 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f67627fc640 1 -- 172.21.15.7:0/3678317094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6768010430 con 0x7f676c101900 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.683+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c1a3e80 con 0x7f676c101900 2024-09-21T04:59:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.685+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c103040 con 0x7f676c101900 2024-09-21T04:59:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.685+0000 7f67627fc640 1 -- 172.21.15.7:0/3678317094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6768004f00 con 0x7f676c101900 2024-09-21T04:59:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.686+0000 7f67627fc640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f675807af30 0x7f675807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.686+0000 7f67627fc640 1 -- 172.21.15.7:0/3678317094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f676809e740 con 0x7f676c101900 2024-09-21T04:59:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.686+0000 7f6770f6d640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f675807af30 0x7f675807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.686+0000 7f6770f6d640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f675807af30 0x7f675807d3f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f676c075f80 tx=0x7f675c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.690+0000 7f67627fc640 1 -- 172.21.15.7:0/3678317094 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6768069ce0 con 0x7f676c101900 2024-09-21T04:59:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.918+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f676c109e70 con 0x7f676c101900 2024-09-21T04:59:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.918+0000 7f67627fc640 1 -- 172.21.15.7:0/3678317094 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+1992 (secure 0 0 0) 0x7f676806eb90 con 0x7f676c101900 2024-09-21T04:59:08.922 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:44:07:773367+0000 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:59:08.923 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:59:08.924 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:59:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:59:08.926 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/606146784,v1:172.21.15.53:6827/606146784] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:59:08.926 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:59:08.926 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T04:59:08.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.920+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f675807af30 msgr2=0x7f675807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.920+0000 7f67739d2640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f675807af30 0x7f675807d3f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f676c075f80 tx=0x7f675c031400 comp rx=0 tx=0).stop 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 msgr2=0x7f676c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 0x7f676c0744f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f676800e970 tx=0x7f676800ed70 comp rx=0 tx=0).stop 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 shutdown_connections 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f675807af30 0x7f675807d3f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f676c107900 0x7f676c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 --2- 172.21.15.7:0/3678317094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f676c101900 0x7f676c0744f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 >> 172.21.15.7:0/3678317094 conn(0x7f676c0fd600 msgr2=0x7f676c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 shutdown_connections 2024-09-21T04:59:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:08.921+0000 7f67739d2640 1 -- 172.21.15.7:0/3678317094 wait complete. 2024-09-21T04:59:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.006+0000 7ff831d23640 1 -- 172.21.15.7:0/2884796530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 msgr2=0x7ff82c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.006+0000 7ff831d23640 1 --2- 172.21.15.7:0/2884796530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 0x7ff82c107ce0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009880 tx=0x7ff81c02a4a0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.006+0000 7ff831d23640 1 -- 172.21.15.7:0/2884796530 shutdown_connections 2024-09-21T04:59:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.006+0000 7ff831d23640 1 --2- 172.21.15.7:0/2884796530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82c101900 0x7ff82c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.006+0000 7ff831d23640 1 --2- 172.21.15.7:0/2884796530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 0x7ff82c107ce0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.006+0000 7ff831d23640 1 -- 172.21.15.7:0/2884796530 >> 172.21.15.7:0/2884796530 conn(0x7ff82c0fd600 msgr2=0x7ff82c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.006+0000 7ff831d23640 1 -- 172.21.15.7:0/2884796530 shutdown_connections 2024-09-21T04:59:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.006+0000 7ff831d23640 1 -- 172.21.15.7:0/2884796530 wait complete. 2024-09-21T04:59:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff831d23640 1 Processor -- start 2024-09-21T04:59:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff831d23640 1 -- start start 2024-09-21T04:59:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff831d23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82c101900 0x7ff82c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff831d23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 0x7ff82c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff831d23640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c10edb0 con 0x7ff82c107900 2024-09-21T04:59:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff831d23640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c10ec30 con 0x7ff82c101900 2024-09-21T04:59:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff82affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 0x7ff82c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff82affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 0x7ff82c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46430/0 (socket says 172.21.15.7:46430) 2024-09-21T04:59:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.007+0000 7ff82affd640 1 -- 172.21.15.7:0/2686390101 learned_addr learned my addr 172.21.15.7:0/2686390101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff82b7fe640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82c101900 0x7ff82c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff82affd640 1 -- 172.21.15.7:0/2686390101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82c101900 msgr2=0x7ff82c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff82affd640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82c101900 0x7ff82c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff82affd640 1 -- 172.21.15.7:0/2686390101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c074f70 con 0x7ff82c107900 2024-09-21T04:59:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff82b7fe640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82c101900 0x7ff82c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:59:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff82affd640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 0x7ff82c074a30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff82000d8d0 tx=0x7ff82000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff828ff9640 1 -- 172.21.15.7:0/2686390101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff820017070 con 0x7ff82c107900 2024-09-21T04:59:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff828ff9640 1 -- 172.21.15.7:0/2686390101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff820010460 con 0x7ff82c107900 2024-09-21T04:59:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff828ff9640 1 -- 172.21.15.7:0/2686390101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff82000f5f0 con 0x7ff82c107900 2024-09-21T04:59:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c075260 con 0x7ff82c107900 2024-09-21T04:59:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.008+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c1a3e60 con 0x7ff82c107900 2024-09-21T04:59:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.009+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82c103040 con 0x7ff82c107900 2024-09-21T04:59:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.010+0000 7ff828ff9640 1 -- 172.21.15.7:0/2686390101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff820009a30 con 0x7ff82c107900 2024-09-21T04:59:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.010+0000 7ff828ff9640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff81407af30 0x7ff81407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.011+0000 7ff828ff9640 1 -- 172.21.15.7:0/2686390101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff82009dfc0 con 0x7ff82c107900 2024-09-21T04:59:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.011+0000 7ff82b7fe640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff81407af30 0x7ff81407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.011+0000 7ff82b7fe640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff81407af30 0x7ff81407d3f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff81c000c00 tx=0x7ff81c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:09.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.015+0000 7ff828ff9640 1 -- 172.21.15.7:0/2686390101 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8200694c0 con 0x7ff82c107900 2024-09-21T04:59:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.220+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff82c062630 con 0x7ff81407af30 2024-09-21T04:59:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.222+0000 7ff828ff9640 1 -- 172.21.15.7:0/2686390101 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7ff82c062630 con 0x7ff81407af30 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "mon", 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "crash" 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/23 daemons upgraded", 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:59:09.226 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff81407af30 msgr2=0x7ff81407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff81407af30 0x7ff81407d3f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff81c000c00 tx=0x7ff81c031400 comp rx=0 tx=0).stop 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 msgr2=0x7ff82c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 0x7ff82c074a30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff82000d8d0 tx=0x7ff82000dd90 comp rx=0 tx=0).stop 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 shutdown_connections 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff81407af30 0x7ff81407d3f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff82c107900 0x7ff82c074a30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 --2- 172.21.15.7:0/2686390101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82c101900 0x7ff82c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 >> 172.21.15.7:0/2686390101 conn(0x7ff82c0fd600 msgr2=0x7ff82c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 shutdown_connections 2024-09-21T04:59:09.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.224+0000 7ff831d23640 1 -- 172.21.15.7:0/2686390101 wait complete. 2024-09-21T04:59:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.308+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/1438293983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4100800 msgr2=0x7fe3d4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.308+0000 7fe3d8d9e640 1 --2- 172.21.15.7:0/1438293983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4100800 0x7fe3d4100c60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009880 tx=0x7fe3c802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.309+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/1438293983 shutdown_connections 2024-09-21T04:59:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.309+0000 7fe3d8d9e640 1 --2- 172.21.15.7:0/1438293983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4100800 0x7fe3d4100c60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.309+0000 7fe3d8d9e640 1 --2- 172.21.15.7:0/1438293983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d4106800 0x7fe3d4106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.309+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/1438293983 >> 172.21.15.7:0/1438293983 conn(0x7fe3d40fc520 msgr2=0x7fe3d40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.309+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/1438293983 shutdown_connections 2024-09-21T04:59:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.309+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/1438293983 wait complete. 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.309+0000 7fe3d8d9e640 1 Processor -- start 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.309+0000 7fe3d8d9e640 1 -- start start 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d8d9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d4100800 0x7fe3d419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d8d9e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4106800 0x7fe3d419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d8d9e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d410dcb0 con 0x7fe3d4106800 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d8d9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d410db30 con 0x7fe3d4100800 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4106800 0x7fe3d419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4106800 0x7fe3d419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46446/0 (socket says 172.21.15.7:46446) 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d1d9b640 1 -- 172.21.15.7:0/4036924419 learned_addr learned my addr 172.21.15.7:0/4036924419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d259c640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d4100800 0x7fe3d419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d1d9b640 1 -- 172.21.15.7:0/4036924419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d4100800 msgr2=0x7fe3d419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d1d9b640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d4100800 0x7fe3d419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d1d9b640 1 -- 172.21.15.7:0/4036924419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d41977a0 con 0x7fe3d4106800 2024-09-21T04:59:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.310+0000 7fe3d1d9b640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4106800 0x7fe3d419db60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8031460 tx=0x7fe3c80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.311+0000 7fe3c37fe640 1 -- 172.21.15.7:0/4036924419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3c803f070 con 0x7fe3d4106800 2024-09-21T04:59:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.311+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d4197a30 con 0x7fe3d4106800 2024-09-21T04:59:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.311+0000 7fe3c37fe640 1 -- 172.21.15.7:0/4036924419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3c80044d0 con 0x7fe3d4106800 2024-09-21T04:59:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.311+0000 7fe3c37fe640 1 -- 172.21.15.7:0/4036924419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3c8035030 con 0x7fe3d4106800 2024-09-21T04:59:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.311+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d4197f60 con 0x7fe3d4106800 2024-09-21T04:59:09.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.312+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d4101f40 con 0x7fe3d4106800 2024-09-21T04:59:09.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.312+0000 7fe3c37fe640 1 -- 172.21.15.7:0/4036924419 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe3c8042070 con 0x7fe3d4106800 2024-09-21T04:59:09.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.313+0000 7fe3c37fe640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3bc07af40 0x7fe3bc07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:09.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.313+0000 7fe3c37fe640 1 -- 172.21.15.7:0/4036924419 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe3c80beb00 con 0x7fe3d4106800 2024-09-21T04:59:09.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.313+0000 7fe3d259c640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3bc07af40 0x7fe3bc07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:09.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.313+0000 7fe3d259c640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3bc07af40 0x7fe3bc07d400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400abe0 tx=0x7fe3c400a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.317+0000 7fe3c37fe640 1 -- 172.21.15.7:0/4036924419 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe3c8004cf0 con 0x7fe3d4106800 2024-09-21T04:59:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: pgmap v731: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2592654956' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.jvuwme", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:59:09.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:09 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3678317094' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: pgmap v731: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2592654956' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi053.jvuwme", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:59:09.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:09 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3678317094' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:59:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.578+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe3d4198cd0 con 0x7fe3d4106800 2024-09-21T04:59:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.578+0000 7fe3c37fe640 1 -- 172.21.15.7:0/4036924419 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fe3c808a0a0 con 0x7fe3d4106800 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T04:59:09.583 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:59:09.584 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:59:09.585 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:59:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3bc07af40 msgr2=0x7fe3bc07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3bc07af40 0x7fe3bc07d400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400abe0 tx=0x7fe3c400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4106800 msgr2=0x7fe3d419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4106800 0x7fe3d419db60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8031460 tx=0x7fe3c80048f0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 shutdown_connections 2024-09-21T04:59:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe3bc07af40 0x7fe3bc07d400 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4106800 0x7fe3d419db60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 --2- 172.21.15.7:0/4036924419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3d4100800 0x7fe3d419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 >> 172.21.15.7:0/4036924419 conn(0x7fe3d40fc520 msgr2=0x7fe3d41045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 shutdown_connections 2024-09-21T04:59:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:09.581+0000 7fe3d8d9e640 1 -- 172.21.15.7:0/4036924419 wait complete. 2024-09-21T04:59:10.315 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:10 smithi053 ceph-mon[80176]: Upgrade: Updating mds.cephfs.smithi053.jvuwme 2024-09-21T04:59:10.315 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:10 smithi053 ceph-mon[80176]: Deploying daemon mds.cephfs.smithi053.jvuwme on smithi053 2024-09-21T04:59:10.315 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:10 smithi053 ceph-mon[80176]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:10.315 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:10 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4036924419' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:59:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:10 smithi007 ceph-mon[108238]: Upgrade: Updating mds.cephfs.smithi053.jvuwme 2024-09-21T04:59:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:10 smithi007 ceph-mon[108238]: Deploying daemon mds.cephfs.smithi053.jvuwme on smithi053 2024-09-21T04:59:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:10 smithi007 ceph-mon[108238]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:10 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4036924419' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:59:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:11 smithi053 ceph-mon[80176]: pgmap v732: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:11.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:11 smithi007 ceph-mon[108238]: pgmap v732: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:12.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:12 smithi007 ceph-mon[108238]: osdmap e80: 6 total, 6 up, 6 in 2024-09-21T04:59:12.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:12 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-21T04:59:12.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:12 smithi053 ceph-mon[80176]: osdmap e80: 6 total, 6 up, 6 in 2024-09-21T04:59:12.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:12 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-21T04:59:13.594 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:13 smithi053 ceph-mon[80176]: pgmap v734: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:13.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:13 smithi007 ceph-mon[108238]: pgmap v734: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:14.537 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:14 smithi053 ceph-mon[80176]: pgmap v735: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:14.537 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:14.538 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:59:14.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:14 smithi007 ceph-mon[108238]: pgmap v735: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:14.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:59:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: pgmap v736: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:59:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] up:boot 2024-09-21T04:59:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:59:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"} : dispatch 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: pgmap v736: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] up:boot 2024-09-21T04:59:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T04:59:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "mds metadata", "who": "cephfs.smithi053.jvuwme"} : dispatch 2024-09-21T04:59:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:18.744 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:18 smithi007 ceph-mon[108238]: pgmap v737: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:18.745 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:18.745 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:18.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:18 smithi053 ceph-mon[80176]: pgmap v737: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:18.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:18.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:19 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:20.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:19 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:20.424 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:19 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:20.425 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:19 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:20 smithi053 ceph-mon[80176]: pgmap v738: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:20 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:20 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:20 smithi007 ceph-mon[108238]: pgmap v738: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:20 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:20 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T04:59:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:22 smithi007 ceph-mon[108238]: pgmap v739: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:22 smithi053 ceph-mon[80176]: pgmap v739: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:24 smithi053 ceph-mon[80176]: pgmap v740: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:24 smithi007 ceph-mon[108238]: pgmap v740: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:27.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:26 smithi053 ceph-mon[80176]: pgmap v741: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:26 smithi007 ceph-mon[108238]: pgmap v741: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:29 smithi007 ceph-mon[108238]: pgmap v742: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:59:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:29 smithi053 ceph-mon[80176]: pgmap v742: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:59:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:31 smithi053 ceph-mon[80176]: pgmap v743: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:31 smithi007 ceph-mon[108238]: pgmap v743: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:33 smithi053 ceph-mon[80176]: pgmap v744: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:33 smithi007 ceph-mon[108238]: pgmap v744: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:35 smithi007 ceph-mon[108238]: pgmap v745: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:35 smithi053 ceph-mon[80176]: pgmap v745: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:37 smithi007 ceph-mon[108238]: pgmap v746: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:37 smithi053 ceph-mon[80176]: pgmap v746: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:39 smithi053 ceph-mon[80176]: pgmap v747: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:39 smithi007 ceph-mon[108238]: pgmap v747: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.680+0000 7fdf959e5640 1 -- 172.21.15.7:0/2555704834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 msgr2=0x7fdf9010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.680+0000 7fdf959e5640 1 --2- 172.21.15.7:0/2555704834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 0x7fdf9010b8d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009880 tx=0x7fdf8402a530 comp rx=0 tx=0).stop 2024-09-21T04:59:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.680+0000 7fdf959e5640 1 -- 172.21.15.7:0/2555704834 shutdown_connections 2024-09-21T04:59:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.680+0000 7fdf959e5640 1 --2- 172.21.15.7:0/2555704834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 0x7fdf9010b8d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.680+0000 7fdf959e5640 1 --2- 172.21.15.7:0/2555704834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90102ac0 0x7fdf90102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.680+0000 7fdf959e5640 1 -- 172.21.15.7:0/2555704834 >> 172.21.15.7:0/2555704834 conn(0x7fdf900fc520 msgr2=0x7fdf900fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.680+0000 7fdf959e5640 1 -- 172.21.15.7:0/2555704834 shutdown_connections 2024-09-21T04:59:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.680+0000 7fdf959e5640 1 -- 172.21.15.7:0/2555704834 wait complete. 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.681+0000 7fdf959e5640 1 Processor -- start 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.681+0000 7fdf959e5640 1 -- start start 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf959e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90102ac0 0x7fdf9019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf959e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 0x7fdf9019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf959e5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9010edd0 con 0x7fdf901033e0 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf959e5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9010ec50 con 0x7fdf90102ac0 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf8e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 0x7fdf9019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf8e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 0x7fdf9019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42110/0 (socket says 172.21.15.7:42110) 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf8e7fc640 1 -- 172.21.15.7:0/2187777467 learned_addr learned my addr 172.21.15.7:0/2187777467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf8effd640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90102ac0 0x7fdf9019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf8e7fc640 1 -- 172.21.15.7:0/2187777467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90102ac0 msgr2=0x7fdf9019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf8e7fc640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90102ac0 0x7fdf9019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf8e7fc640 1 -- 172.21.15.7:0/2187777467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf901977e0 con 0x7fdf901033e0 2024-09-21T04:59:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.682+0000 7fdf8effd640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90102ac0 0x7fdf9019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:59:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.683+0000 7fdf8e7fc640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 0x7fdf9019dba0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdf84031460 tx=0x7fdf84004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.683+0000 7fdf949e3640 1 -- 172.21.15.7:0/2187777467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf8403f070 con 0x7fdf901033e0 2024-09-21T04:59:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.683+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf90197a70 con 0x7fdf901033e0 2024-09-21T04:59:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.683+0000 7fdf949e3640 1 -- 172.21.15.7:0/2187777467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf84004a90 con 0x7fdf901033e0 2024-09-21T04:59:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.683+0000 7fdf949e3640 1 -- 172.21.15.7:0/2187777467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf84035030 con 0x7fdf901033e0 2024-09-21T04:59:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.683+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf90197fa0 con 0x7fdf901033e0 2024-09-21T04:59:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.684+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9006aa90 con 0x7fdf901033e0 2024-09-21T04:59:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.685+0000 7fdf949e3640 1 -- 172.21.15.7:0/2187777467 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdf84042070 con 0x7fdf901033e0 2024-09-21T04:59:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.685+0000 7fdf949e3640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf7807af40 0x7fdf7807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.685+0000 7fdf949e3640 1 -- 172.21.15.7:0/2187777467 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fdf840bebc0 con 0x7fdf901033e0 2024-09-21T04:59:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.685+0000 7fdf8effd640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf7807af40 0x7fdf7807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.686+0000 7fdf8effd640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf7807af40 0x7fdf7807d400 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdf800058c0 tx=0x7fdf80005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.690+0000 7fdf949e3640 1 -- 172.21.15.7:0/2187777467 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdf8408a100 con 0x7fdf901033e0 2024-09-21T04:59:39.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.896+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdf90198310 con 0x7fdf7807af40 2024-09-21T04:59:39.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.898+0000 7fdf949e3640 1 -- 172.21.15.7:0/2187777467 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fdf90198310 con 0x7fdf7807af40 2024-09-21T04:59:39.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.899+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf7807af40 msgr2=0x7fdf7807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.899+0000 7fdf959e5640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf7807af40 0x7fdf7807d400 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdf800058c0 tx=0x7fdf80005850 comp rx=0 tx=0).stop 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.899+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 msgr2=0x7fdf9019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.900+0000 7fdf959e5640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 0x7fdf9019dba0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdf84031460 tx=0x7fdf84004530 comp rx=0 tx=0).stop 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.900+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 shutdown_connections 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.900+0000 7fdf959e5640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdf7807af40 0x7fdf7807d400 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.900+0000 7fdf959e5640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf901033e0 0x7fdf9019dba0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.900+0000 7fdf959e5640 1 --2- 172.21.15.7:0/2187777467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf90102ac0 0x7fdf9019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.900+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 >> 172.21.15.7:0/2187777467 conn(0x7fdf900fc520 msgr2=0x7fdf90108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.900+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 shutdown_connections 2024-09-21T04:59:39.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.900+0000 7fdf959e5640 1 -- 172.21.15.7:0/2187777467 wait complete. 2024-09-21T04:59:39.912 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T04:59:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.985+0000 7ff0c4863640 1 -- 172.21.15.7:0/2343152089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0102b60 msgr2=0x7ff0c0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.985+0000 7ff0c4863640 1 --2- 172.21.15.7:0/2343152089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0102b60 0x7ff0c0102f40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009880 tx=0x7ff0b002a530 comp rx=0 tx=0).stop 2024-09-21T04:59:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.985+0000 7ff0c4863640 1 -- 172.21.15.7:0/2343152089 shutdown_connections 2024-09-21T04:59:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.985+0000 7ff0c4863640 1 --2- 172.21.15.7:0/2343152089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0c0103480 0x7ff0c010b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.985+0000 7ff0c4863640 1 --2- 172.21.15.7:0/2343152089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0102b60 0x7ff0c0102f40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.985+0000 7ff0c4863640 1 -- 172.21.15.7:0/2343152089 >> 172.21.15.7:0/2343152089 conn(0x7ff0c00fc520 msgr2=0x7ff0c00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.986+0000 7ff0c4863640 1 -- 172.21.15.7:0/2343152089 shutdown_connections 2024-09-21T04:59:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.986+0000 7ff0c4863640 1 -- 172.21.15.7:0/2343152089 wait complete. 2024-09-21T04:59:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.986+0000 7ff0c4863640 1 Processor -- start 2024-09-21T04:59:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.986+0000 7ff0c4863640 1 -- start start 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0c4863640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0c0102b60 0x7ff0c01007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0c4863640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0103480 0x7ff0c0100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0c4863640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c010edf0 con 0x7ff0c0103480 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0c4863640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c010ec70 con 0x7ff0c0102b60 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0103480 0x7ff0c0100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0bdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0103480 0x7ff0c0100d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42116/0 (socket says 172.21.15.7:42116) 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0bdd9b640 1 -- 172.21.15.7:0/3494338674 learned_addr learned my addr 172.21.15.7:0/3494338674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0be59c640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0c0102b60 0x7ff0c01007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0bdd9b640 1 -- 172.21.15.7:0/3494338674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0c0102b60 msgr2=0x7ff0c01007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0bdd9b640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0c0102b60 0x7ff0c01007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0bdd9b640 1 -- 172.21.15.7:0/3494338674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c0101240 con 0x7ff0c0103480 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0bdd9b640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0103480 0x7ff0c0100d00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400d6f0 tx=0x7ff0b400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0af7fe640 1 -- 172.21.15.7:0/3494338674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff0b4017070 con 0x7ff0c0103480 2024-09-21T04:59:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0af7fe640 1 -- 172.21.15.7:0/3494338674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff0b400de00 con 0x7ff0c0103480 2024-09-21T04:59:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c0101530 con 0x7ff0c0103480 2024-09-21T04:59:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.987+0000 7ff0af7fe640 1 -- 172.21.15.7:0/3494338674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff0b4004bb0 con 0x7ff0c0103480 2024-09-21T04:59:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.988+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c01a3f00 con 0x7ff0c0103480 2024-09-21T04:59:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.988+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c006a9f0 con 0x7ff0c0103480 2024-09-21T04:59:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.989+0000 7ff0af7fe640 1 -- 172.21.15.7:0/3494338674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff0b4004e60 con 0x7ff0c0103480 2024-09-21T04:59:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.990+0000 7ff0af7fe640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff0a807af30 0x7ff0a807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.990+0000 7ff0af7fe640 1 -- 172.21.15.7:0/3494338674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff0b409dc80 con 0x7ff0c0103480 2024-09-21T04:59:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.990+0000 7ff0be59c640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff0a807af30 0x7ff0a807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.990+0000 7ff0be59c640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff0a807af30 0x7ff0a807d3f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00096c0 tx=0x7ff0b0004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:39.994+0000 7ff0af7fe640 1 -- 172.21.15.7:0/3494338674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0b4069180 con 0x7ff0c0103480 2024-09-21T04:59:40.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.202+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff0c0062630 con 0x7ff0a807af30 2024-09-21T04:59:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.204+0000 7ff0af7fe640 1 -- 172.21.15.7:0/3494338674 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ff0c0062630 con 0x7ff0a807af30 2024-09-21T04:59:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff0a807af30 msgr2=0x7ff0a807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff0a807af30 0x7ff0a807d3f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00096c0 tx=0x7ff0b0004f30 comp rx=0 tx=0).stop 2024-09-21T04:59:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0103480 msgr2=0x7ff0c0100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0103480 0x7ff0c0100d00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400d6f0 tx=0x7ff0b400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 shutdown_connections 2024-09-21T04:59:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff0a807af30 0x7ff0a807d3f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0c0103480 0x7ff0c0100d00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 --2- 172.21.15.7:0/3494338674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0c0102b60 0x7ff0c01007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 >> 172.21.15.7:0/3494338674 conn(0x7ff0c00fc520 msgr2=0x7ff0c0107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 shutdown_connections 2024-09-21T04:59:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.206+0000 7ff0c4863640 1 -- 172.21.15.7:0/3494338674 wait complete. 2024-09-21T04:59:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.291+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3146458252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 msgr2=0x7f13b4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.291+0000 7f13bb1d3640 1 --2- 172.21.15.7:0/3146458252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 0x7f13b4106bd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009880 tx=0x7f13b002a4a0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.291+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3146458252 shutdown_connections 2024-09-21T04:59:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.291+0000 7f13bb1d3640 1 --2- 172.21.15.7:0/3146458252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b41007f0 0x7f13b4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.291+0000 7f13bb1d3640 1 --2- 172.21.15.7:0/3146458252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 0x7f13b4106bd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.291+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3146458252 >> 172.21.15.7:0/3146458252 conn(0x7f13b40fc550 msgr2=0x7f13b40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.291+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3146458252 shutdown_connections 2024-09-21T04:59:40.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.291+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3146458252 wait complete. 2024-09-21T04:59:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13bb1d3640 1 Processor -- start 2024-09-21T04:59:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13bb1d3640 1 -- start start 2024-09-21T04:59:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13bb1d3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b41007f0 0x7f13b40fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13bb1d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 0x7f13b40fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13bb1d3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13b410dcc0 con 0x7f13b41067f0 2024-09-21T04:59:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13bb1d3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13b410db40 con 0x7f13b41007f0 2024-09-21T04:59:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13abfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 0x7f13b40fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13abfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 0x7f13b40fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42142/0 (socket says 172.21.15.7:42142) 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13abfff640 1 -- 172.21.15.7:0/3048769698 learned_addr learned my addr 172.21.15.7:0/3048769698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13b8f6f640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b41007f0 0x7f13b40fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13abfff640 1 -- 172.21.15.7:0/3048769698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b41007f0 msgr2=0x7f13b40fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13abfff640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b41007f0 0x7f13b40fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.292+0000 7f13abfff640 1 -- 172.21.15.7:0/3048769698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b40fd7b0 con 0x7f13b41067f0 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.293+0000 7f13abfff640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 0x7f13b40fd270 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f13a400d6f0 tx=0x7f13a400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.293+0000 7f13a9ffb640 1 -- 172.21.15.7:0/3048769698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13a4017070 con 0x7f13b41067f0 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.293+0000 7f13a9ffb640 1 -- 172.21.15.7:0/3048769698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13a400de00 con 0x7f13b41067f0 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.293+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b40fdaa0 con 0x7f13b41067f0 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.293+0000 7f13a9ffb640 1 -- 172.21.15.7:0/3048769698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13a4004bb0 con 0x7f13b41067f0 2024-09-21T04:59:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.293+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41a3e30 con 0x7f13b41067f0 2024-09-21T04:59:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.294+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b4101f30 con 0x7f13b41067f0 2024-09-21T04:59:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.295+0000 7f13a9ffb640 1 -- 172.21.15.7:0/3048769698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f13a4004e60 con 0x7f13b41067f0 2024-09-21T04:59:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.296+0000 7f13a9ffb640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f139c07af30 0x7f139c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.296+0000 7f13a9ffb640 1 -- 172.21.15.7:0/3048769698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f13a409dc80 con 0x7f13b41067f0 2024-09-21T04:59:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.296+0000 7f13b8f6f640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f139c07af30 0x7f139c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.296+0000 7f13b8f6f640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f139c07af30 0x7f139c07d3f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f13b00096c0 tx=0x7f13b0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.300+0000 7f13a9ffb640 1 -- 172.21.15.7:0/3048769698 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13a4069180 con 0x7f13b41067f0 2024-09-21T04:59:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.497+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f13b4062630 con 0x7f139c07af30 2024-09-21T04:59:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.507+0000 7f13a9ffb640 1 -- 172.21.15.7:0/3048769698 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f13b4062630 con 0x7f139c07af30 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (21m) 19s ago 32m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (24m) 19s ago 32m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (24m) 22s ago 31m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (19m) 19s ago 32m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (19m) 22s ago 31m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (21m) 19s ago 32m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (15m) 19s ago 27m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T04:59:40.510 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (16m) 19s ago 27m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (25s) 22s ago 27m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (15m) 22s ago 27m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (23m) 19s ago 34m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (22m) 22s ago 31m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (20m) 19s ago 34m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (20m) 22s ago 31m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (22m) 19s ago 32m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (22m) 22s ago 31m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (19m) 19s ago 30m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (18m) 19s ago 30m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (18m) 19s ago 29m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (17m) 22s ago 29m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (16m) 22s ago 28m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T04:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (16m) 22s ago 28m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T04:59:40.512 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (22m) 19s ago 32m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T04:59:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f139c07af30 msgr2=0x7f139c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f139c07af30 0x7f139c07d3f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f13b00096c0 tx=0x7f13b0031400 comp rx=0 tx=0).stop 2024-09-21T04:59:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 msgr2=0x7f13b40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 0x7f13b40fd270 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f13a400d6f0 tx=0x7f13a400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 shutdown_connections 2024-09-21T04:59:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f139c07af30 0x7f139c07d3f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41067f0 0x7f13b40fd270 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 --2- 172.21.15.7:0/3048769698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13b41007f0 0x7f13b40fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.509+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 >> 172.21.15.7:0/3048769698 conn(0x7f13b40fc550 msgr2=0x7f13b4069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.510+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 shutdown_connections 2024-09-21T04:59:40.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.510+0000 7f13bb1d3640 1 -- 172.21.15.7:0/3048769698 wait complete. 2024-09-21T04:59:40.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.593+0000 7f98c9d82640 1 -- 172.21.15.7:0/3210828523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c4102ac0 msgr2=0x7f98c4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.593+0000 7f98c9d82640 1 --2- 172.21.15.7:0/3210828523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c4102ac0 0x7f98c4102ea0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009880 tx=0x7f98b402a530 comp rx=0 tx=0).stop 2024-09-21T04:59:40.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.593+0000 7f98c9d82640 1 -- 172.21.15.7:0/3210828523 shutdown_connections 2024-09-21T04:59:40.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.593+0000 7f98c9d82640 1 --2- 172.21.15.7:0/3210828523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c41033e0 0x7f98c410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.593+0000 7f98c9d82640 1 --2- 172.21.15.7:0/3210828523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c4102ac0 0x7f98c4102ea0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.593+0000 7f98c9d82640 1 -- 172.21.15.7:0/3210828523 >> 172.21.15.7:0/3210828523 conn(0x7f98c40fc520 msgr2=0x7f98c40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:40.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.593+0000 7f98c9d82640 1 -- 172.21.15.7:0/3210828523 shutdown_connections 2024-09-21T04:59:40.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.593+0000 7f98c9d82640 1 -- 172.21.15.7:0/3210828523 wait complete. 2024-09-21T04:59:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c9d82640 1 Processor -- start 2024-09-21T04:59:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c9d82640 1 -- start start 2024-09-21T04:59:40.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c9d82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c4102ac0 0x7f98c4100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c9d82640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c41033e0 0x7f98c4100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c9d82640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98c410edf0 con 0x7f98c41033e0 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c9d82640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98c410ec70 con 0x7f98c4102ac0 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c41033e0 0x7f98c4100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c41033e0 0x7f98c4100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42158/0 (socket says 172.21.15.7:42158) 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.594+0000 7f98c2ffd640 1 -- 172.21.15.7:0/2840603570 learned_addr learned my addr 172.21.15.7:0/2840603570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c37fe640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c4102ac0 0x7f98c4100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c2ffd640 1 -- 172.21.15.7:0/2840603570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c4102ac0 msgr2=0x7f98c4100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c2ffd640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c4102ac0 0x7f98c4100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c2ffd640 1 -- 172.21.15.7:0/2840603570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c41011a0 con 0x7f98c41033e0 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c2ffd640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c41033e0 0x7f98c4100c60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f98b800d6d0 tx=0x7f98b800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c0ff9640 1 -- 172.21.15.7:0/2840603570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98b8017070 con 0x7f98c41033e0 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c0ff9640 1 -- 172.21.15.7:0/2840603570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98b80054e0 con 0x7f98c41033e0 2024-09-21T04:59:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c4101490 con 0x7f98c41033e0 2024-09-21T04:59:40.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c0ff9640 1 -- 172.21.15.7:0/2840603570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98b8004ae0 con 0x7f98c41033e0 2024-09-21T04:59:40.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.595+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c41a3e60 con 0x7f98c41033e0 2024-09-21T04:59:40.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.597+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c406aa90 con 0x7f98c41033e0 2024-09-21T04:59:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.597+0000 7f98c0ff9640 1 -- 172.21.15.7:0/2840603570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f98b8004d90 con 0x7f98c41033e0 2024-09-21T04:59:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.598+0000 7f98c0ff9640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98ac07af80 0x7f98ac07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.598+0000 7f98c0ff9640 1 -- 172.21.15.7:0/2840603570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f98b809dba0 con 0x7f98c41033e0 2024-09-21T04:59:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.598+0000 7f98c37fe640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98ac07af80 0x7f98ac07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.598+0000 7f98c37fe640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98ac07af80 0x7f98ac07d440 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f98b40096c0 tx=0x7f98b4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.602+0000 7f98c0ff9640 1 -- 172.21.15.7:0/2840603570 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98b80691c0 con 0x7f98c41033e0 2024-09-21T04:59:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.866+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f98c4105f90 con 0x7f98c41033e0 2024-09-21T04:59:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.866+0000 7f98c0ff9640 1 -- 172.21.15.7:0/2840603570 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f98b806e070 con 0x7f98c41033e0 2024-09-21T04:59:40.869 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:59:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T04:59:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T04:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:59:40.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98ac07af80 msgr2=0x7f98ac07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98ac07af80 0x7f98ac07d440 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f98b40096c0 tx=0x7f98b4031400 comp rx=0 tx=0).stop 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c41033e0 msgr2=0x7f98c4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c41033e0 0x7f98c4100c60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f98b800d6d0 tx=0x7f98b800db90 comp rx=0 tx=0).stop 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 shutdown_connections 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98ac07af80 0x7f98ac07d440 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98c41033e0 0x7f98c4100c60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 --2- 172.21.15.7:0/2840603570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98c4102ac0 0x7f98c4100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 >> 172.21.15.7:0/2840603570 conn(0x7f98c40fc520 msgr2=0x7f98c4108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 shutdown_connections 2024-09-21T04:59:40.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.869+0000 7f98c9d82640 1 -- 172.21.15.7:0/2840603570 wait complete. 2024-09-21T04:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 -- 172.21.15.7:0/1778685972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248103410 msgr2=0x7f824810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 --2- 172.21.15.7:0/1778685972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248103410 0x7f824810b900 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f823c0098b0 tx=0x7f823c02a560 comp rx=0 tx=0).stop 2024-09-21T04:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 -- 172.21.15.7:0/1778685972 shutdown_connections 2024-09-21T04:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 --2- 172.21.15.7:0/1778685972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248103410 0x7f824810b900 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 --2- 172.21.15.7:0/1778685972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248102af0 0x7f8248102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 -- 172.21.15.7:0/1778685972 >> 172.21.15.7:0/1778685972 conn(0x7f82480fc550 msgr2=0x7f82480fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 -- 172.21.15.7:0/1778685972 shutdown_connections 2024-09-21T04:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 -- 172.21.15.7:0/1778685972 wait complete. 2024-09-21T04:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.956+0000 7f82503e6640 1 Processor -- start 2024-09-21T04:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f82503e6640 1 -- start start 2024-09-21T04:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f82503e6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248102af0 0x7f824819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f82503e6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248103410 0x7f824819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f82503e6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f824810ede0 con 0x7f8248102af0 2024-09-21T04:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f82503e6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f824810ec60 con 0x7f8248103410 2024-09-21T04:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f824e182640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248102af0 0x7f824819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f824e182640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248102af0 0x7f824819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42176/0 (socket says 172.21.15.7:42176) 2024-09-21T04:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f824e182640 1 -- 172.21.15.7:0/1481827899 learned_addr learned my addr 172.21.15.7:0/1481827899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f824d981640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248103410 0x7f824819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f824e182640 1 -- 172.21.15.7:0/1481827899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248103410 msgr2=0x7f824819dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f824e182640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248103410 0x7f824819dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f824e182640 1 -- 172.21.15.7:0/1481827899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8248197830 con 0x7f8248102af0 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.957+0000 7f824e182640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248102af0 0x7f824819d6b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f823800ea00 tx=0x7f823800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.958+0000 7f82477fe640 1 -- 172.21.15.7:0/1481827899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f823800cb80 con 0x7f8248102af0 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.958+0000 7f82477fe640 1 -- 172.21.15.7:0/1481827899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f823800cd20 con 0x7f8248102af0 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.958+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8248197b20 con 0x7f8248102af0 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.958+0000 7f82477fe640 1 -- 172.21.15.7:0/1481827899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8238011470 con 0x7f8248102af0 2024-09-21T04:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.958+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8248198080 con 0x7f8248102af0 2024-09-21T04:59:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.959+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824806a540 con 0x7f8248102af0 2024-09-21T04:59:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.959+0000 7f82477fe640 1 -- 172.21.15.7:0/1481827899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8238005020 con 0x7f8248102af0 2024-09-21T04:59:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.960+0000 7f82477fe640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823007af30 0x7f823007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.960+0000 7f82477fe640 1 -- 172.21.15.7:0/1481827899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f823809e470 con 0x7f8248102af0 2024-09-21T04:59:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.960+0000 7f824d981640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823007af30 0x7f823007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.960+0000 7f824d981640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823007af30 0x7f823007d3f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f823c000c00 tx=0x7f823c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:40.964+0000 7f82477fe640 1 -- 172.21.15.7:0/1481827899 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8238016030 con 0x7f8248102af0 2024-09-21T04:59:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.190+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8248105f80 con 0x7f8248102af0 2024-09-21T04:59:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:41 smithi007 ceph-mon[108238]: pgmap v748: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:41 smithi007 ceph-mon[108238]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:41 smithi007 ceph-mon[108238]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:41 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2840603570' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:41.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.191+0000 7f82477fe640 1 -- 172.21.15.7:0/1481827899 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f8238069a10 con 0x7f8248102af0 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T04:59:41.195 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T04:59:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T04:59:41.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T04:59:41.198 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:59:41.198 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:59:41.198 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:59:41.198 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:59:41.198 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823007af30 msgr2=0x7f823007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823007af30 0x7f823007d3f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f823c000c00 tx=0x7f823c031400 comp rx=0 tx=0).stop 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248102af0 msgr2=0x7f824819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248102af0 0x7f824819d6b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f823800ea00 tx=0x7f823800eec0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 shutdown_connections 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823007af30 0x7f823007d3f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248103410 0x7f824819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 --2- 172.21.15.7:0/1481827899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248102af0 0x7f824819d6b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 >> 172.21.15.7:0/1481827899 conn(0x7f82480fc550 msgr2=0x7f8248103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 shutdown_connections 2024-09-21T04:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.193+0000 7f82503e6640 1 -- 172.21.15.7:0/1481827899 wait complete. 2024-09-21T04:59:41.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.279+0000 7f420f692640 1 -- 172.21.15.7:0/3439656238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107900 msgr2=0x7f4208107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.279+0000 7f420f692640 1 --2- 172.21.15.7:0/3439656238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107900 0x7f4208107ce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009880 tx=0x7f41f802a4a0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.279+0000 7f420f692640 1 -- 172.21.15.7:0/3439656238 shutdown_connections 2024-09-21T04:59:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.279+0000 7f420f692640 1 --2- 172.21.15.7:0/3439656238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4208101900 0x7f4208101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.279+0000 7f420f692640 1 --2- 172.21.15.7:0/3439656238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208107900 0x7f4208107ce0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.279+0000 7f420f692640 1 -- 172.21.15.7:0/3439656238 >> 172.21.15.7:0/3439656238 conn(0x7f42080fd600 msgr2=0x7f42080ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.279+0000 7f420f692640 1 -- 172.21.15.7:0/3439656238 shutdown_connections 2024-09-21T04:59:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.279+0000 7f420f692640 1 -- 172.21.15.7:0/3439656238 wait complete. 2024-09-21T04:59:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.280+0000 7f420f692640 1 Processor -- start 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.280+0000 7f420f692640 1 -- start start 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.280+0000 7f420f692640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208101900 0x7f420819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.280+0000 7f420f692640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4208107900 0x7f420819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.280+0000 7f420f692640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f420810edb0 con 0x7f4208101900 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.280+0000 7f420f692640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f420810ec30 con 0x7f4208107900 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.280+0000 7f420d42e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208101900 0x7f420819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420d42e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208101900 0x7f420819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42204/0 (socket says 172.21.15.7:42204) 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420d42e640 1 -- 172.21.15.7:0/2660995239 learned_addr learned my addr 172.21.15.7:0/2660995239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420cc2d640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4208107900 0x7f420819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420d42e640 1 -- 172.21.15.7:0/2660995239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4208107900 msgr2=0x7f420819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420d42e640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4208107900 0x7f420819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420d42e640 1 -- 172.21.15.7:0/2660995239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42081977e0 con 0x7f4208101900 2024-09-21T04:59:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420d42e640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208101900 0x7f420819d660 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f41f8031460 tx=0x7f41f80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f41fe7fc640 1 -- 172.21.15.7:0/2660995239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f41f803f070 con 0x7f4208101900 2024-09-21T04:59:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f41fe7fc640 1 -- 172.21.15.7:0/2660995239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f41f8004cf0 con 0x7f4208101900 2024-09-21T04:59:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f41fe7fc640 1 -- 172.21.15.7:0/2660995239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f41f8035030 con 0x7f4208101900 2024-09-21T04:59:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4208197a70 con 0x7f4208101900 2024-09-21T04:59:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.281+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4208197fa0 con 0x7f4208101900 2024-09-21T04:59:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.282+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4208103040 con 0x7f4208101900 2024-09-21T04:59:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.283+0000 7f41fe7fc640 1 -- 172.21.15.7:0/2660995239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f41f8042070 con 0x7f4208101900 2024-09-21T04:59:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.284+0000 7f41fe7fc640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f407af80 0x7f41f407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.284+0000 7f41fe7fc640 1 -- 172.21.15.7:0/2660995239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f41f80be990 con 0x7f4208101900 2024-09-21T04:59:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.284+0000 7f420cc2d640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f407af80 0x7f41f407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.284+0000 7f420cc2d640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f407af80 0x7f41f407d440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4208100d10 tx=0x7f420400a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.288+0000 7f41fe7fc640 1 -- 172.21.15.7:0/2660995239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41f8089fb0 con 0x7f4208101900 2024-09-21T04:59:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:41 smithi053 ceph-mon[80176]: pgmap v748: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:41 smithi053 ceph-mon[80176]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:41 smithi053 ceph-mon[80176]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:41 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2840603570' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T04:59:41.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.496+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4208198310 con 0x7f41f407af80 2024-09-21T04:59:41.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.498+0000 7f41fe7fc640 1 -- 172.21.15.7:0/2660995239 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f4208198310 con 0x7f41f407af80 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T04:59:41.501 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T04:59:41.502 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T04:59:41.502 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T04:59:41.502 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T04:59:41.502 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T04:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f407af80 msgr2=0x7f41f407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f407af80 0x7f41f407d440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4208100d10 tx=0x7f420400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208101900 msgr2=0x7f420819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208101900 0x7f420819d660 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f41f8031460 tx=0x7f41f80048f0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 shutdown_connections 2024-09-21T04:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f407af80 0x7f41f407d440 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4208107900 0x7f420819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 --2- 172.21.15.7:0/2660995239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4208101900 0x7f420819d660 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 >> 172.21.15.7:0/2660995239 conn(0x7f42080fd600 msgr2=0x7f42080fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 shutdown_connections 2024-09-21T04:59:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.500+0000 7f420f692640 1 -- 172.21.15.7:0/2660995239 wait complete. 2024-09-21T04:59:41.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.584+0000 7fb945d9d640 1 -- 172.21.15.7:0/772155673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940103230 msgr2=0x7fb94010b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.584+0000 7fb945d9d640 1 --2- 172.21.15.7:0/772155673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940103230 0x7fb94010b720 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009880 tx=0x7fb92c02a530 comp rx=0 tx=0).stop 2024-09-21T04:59:41.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.584+0000 7fb945d9d640 1 -- 172.21.15.7:0/772155673 shutdown_connections 2024-09-21T04:59:41.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.584+0000 7fb945d9d640 1 --2- 172.21.15.7:0/772155673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940103230 0x7fb94010b720 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.584+0000 7fb945d9d640 1 --2- 172.21.15.7:0/772155673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb940102910 0x7fb940102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.584+0000 7fb945d9d640 1 -- 172.21.15.7:0/772155673 >> 172.21.15.7:0/772155673 conn(0x7fb9400fc350 msgr2=0x7fb9400fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:41.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.584+0000 7fb945d9d640 1 -- 172.21.15.7:0/772155673 shutdown_connections 2024-09-21T04:59:41.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.584+0000 7fb945d9d640 1 -- 172.21.15.7:0/772155673 wait complete. 2024-09-21T04:59:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb945d9d640 1 Processor -- start 2024-09-21T04:59:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb945d9d640 1 -- start start 2024-09-21T04:59:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb945d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940102910 0x7fb94019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb945d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb940103230 0x7fb94019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb945d9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb94010ec20 con 0x7fb940102910 2024-09-21T04:59:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb945d9d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb94010eaa0 con 0x7fb940103230 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb944d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940102910 0x7fb94019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb944d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940102910 0x7fb94019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42216/0 (socket says 172.21.15.7:42216) 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb944d9b640 1 -- 172.21.15.7:0/1177370344 learned_addr learned my addr 172.21.15.7:0/1177370344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb93ffff640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb940103230 0x7fb94019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb944d9b640 1 -- 172.21.15.7:0/1177370344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb940103230 msgr2=0x7fb94019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb944d9b640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb940103230 0x7fb94019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.585+0000 7fb944d9b640 1 -- 172.21.15.7:0/1177370344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb940197610 con 0x7fb940102910 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.586+0000 7fb944d9b640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940102910 0x7fb94019d450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c810 tx=0x7fb93400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.586+0000 7fb93dffb640 1 -- 172.21.15.7:0/1177370344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb934004020 con 0x7fb940102910 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.586+0000 7fb93dffb640 1 -- 172.21.15.7:0/1177370344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb9340041c0 con 0x7fb940102910 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.586+0000 7fb93dffb640 1 -- 172.21.15.7:0/1177370344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb9340047a0 con 0x7fb940102910 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.586+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb940197900 con 0x7fb940102910 2024-09-21T04:59:41.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.586+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb940197e60 con 0x7fb940102910 2024-09-21T04:59:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.587+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94006aa90 con 0x7fb940102910 2024-09-21T04:59:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.588+0000 7fb93dffb640 1 -- 172.21.15.7:0/1177370344 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb9340079b0 con 0x7fb940102910 2024-09-21T04:59:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.588+0000 7fb93dffb640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb93007af30 0x7fb93007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:59:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.589+0000 7fb93dffb640 1 -- 172.21.15.7:0/1177370344 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fb93409deb0 con 0x7fb940102910 2024-09-21T04:59:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.589+0000 7fb93ffff640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb93007af30 0x7fb93007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:59:41.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.589+0000 7fb93ffff640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb93007af30 0x7fb93007d3f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb940198750 tx=0x7fb92c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:59:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.593+0000 7fb93dffb640 1 -- 172.21.15.7:0/1177370344 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb934014030 con 0x7fb940102910 2024-09-21T04:59:41.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.853+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fb940105db0 con 0x7fb940102910 2024-09-21T04:59:41.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.853+0000 7fb93dffb640 1 -- 172.21.15.7:0/1177370344 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fb934069450 con 0x7fb940102910 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T04:59:41.857 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T04:59:41.858 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T04:59:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb93007af30 msgr2=0x7fb93007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb93007af30 0x7fb93007d3f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb940198750 tx=0x7fb92c031400 comp rx=0 tx=0).stop 2024-09-21T04:59:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940102910 msgr2=0x7fb94019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:59:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940102910 0x7fb94019d450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c810 tx=0x7fb93400ccd0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 shutdown_connections 2024-09-21T04:59:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb93007af30 0x7fb93007d3f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb940103230 0x7fb94019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 --2- 172.21.15.7:0/1177370344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb940102910 0x7fb94019d450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:59:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 >> 172.21.15.7:0/1177370344 conn(0x7fb9400fc350 msgr2=0x7fb940108d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:59:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 shutdown_connections 2024-09-21T04:59:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T04:59:41.856+0000 7fb945d9d640 1 -- 172.21.15.7:0/1177370344 wait complete. 2024-09-21T04:59:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:42 smithi007 ceph-mon[108238]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:42 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1481827899' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:59:42.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:42 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1177370344' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:59:42.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:42 smithi053 ceph-mon[80176]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:42 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1481827899' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:59:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:42 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1177370344' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T04:59:43.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:43 smithi053 ceph-mon[80176]: pgmap v749: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:43.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:43 smithi053 ceph-mon[80176]: from='client.35200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:43 smithi007 ceph-mon[108238]: pgmap v749: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:43 smithi007 ceph-mon[108238]: from='client.35200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:59:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:59:44.360 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:59:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:45 smithi053 ceph-mon[80176]: pgmap v750: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:45.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:45 smithi007 ceph-mon[108238]: pgmap v750: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:47 smithi007 ceph-mon[108238]: pgmap v751: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:47 smithi053 ceph-mon[80176]: pgmap v751: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:49 smithi053 ceph-mon[80176]: pgmap v752: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:49 smithi007 ceph-mon[108238]: pgmap v752: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:51 smithi053 ceph-mon[80176]: pgmap v753: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:51 smithi007 ceph-mon[108238]: pgmap v753: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:53 smithi007 ceph-mon[108238]: pgmap v754: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:53 smithi053 ceph-mon[80176]: pgmap v754: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:55 smithi053 ceph-mon[80176]: pgmap v755: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:55 smithi007 ceph-mon[108238]: pgmap v755: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:57 smithi053 ceph-mon[80176]: pgmap v756: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:57 smithi007 ceph-mon[108238]: pgmap v756: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:59 smithi007 ceph-mon[108238]: pgmap v757: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 04:59:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T04:59:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:59 smithi053 ceph-mon[80176]: pgmap v757: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T04:59:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 04:59:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:00:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:00:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:00:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:00:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: /usr/bin/podman: timeout after 895 seconds 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: Traceback (most recent call last): 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: return _run_code(code, main_globals, None, 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: exec(code, run_globals) 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:00:00.444 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:00:00.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:00.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:00:00.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: fs cephfs is degraded 2024-09-21T05:00:00.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:00:00.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:00:00.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:00:00.445 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:00 smithi053 ceph-mon[80176]: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:00:00.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:00:00.447 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T04:59:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: /usr/bin/podman: timeout after 895 seconds 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: Traceback (most recent call last): 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: return _run_code(code, main_globals, None, 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: exec(code, run_globals) 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:00:00.448 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: fs cephfs is degraded 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:00:00.449 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:00 smithi007 ceph-mon[108238]: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:00:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:01 smithi053 ceph-mon[80176]: pgmap v758: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:01 smithi007 ceph-mon[108238]: pgmap v758: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:03 smithi053 ceph-mon[80176]: pgmap v759: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:03 smithi007 ceph-mon[108238]: pgmap v759: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:05 smithi007 ceph-mon[108238]: pgmap v760: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:05 smithi053 ceph-mon[80176]: pgmap v760: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:07 smithi053 ceph-mon[80176]: pgmap v761: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:07 smithi007 ceph-mon[108238]: pgmap v761: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:09 smithi053 ceph-mon[80176]: pgmap v762: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:09 smithi007 ceph-mon[108238]: pgmap v762: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:11 smithi007 ceph-mon[108238]: pgmap v763: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:11 smithi053 ceph-mon[80176]: pgmap v763: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.953+0000 7f26c82a3640 1 -- 172.21.15.7:0/1541143834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 msgr2=0x7f26c0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.953+0000 7f26c82a3640 1 --2- 172.21.15.7:0/1541143834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0102ea0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009c20 tx=0x7f26b002aa00 comp rx=0 tx=0).stop 2024-09-21T05:00:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.953+0000 7f26c82a3640 1 -- 172.21.15.7:0/1541143834 shutdown_connections 2024-09-21T05:00:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.953+0000 7f26c82a3640 1 --2- 172.21.15.7:0/1541143834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.953+0000 7f26c82a3640 1 --2- 172.21.15.7:0/1541143834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c0102ea0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.953+0000 7f26c82a3640 1 -- 172.21.15.7:0/1541143834 >> 172.21.15.7:0/1541143834 conn(0x7f26c00fc520 msgr2=0x7f26c00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.953+0000 7f26c82a3640 1 -- 172.21.15.7:0/1541143834 shutdown_connections 2024-09-21T05:00:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.954+0000 7f26c82a3640 1 -- 172.21.15.7:0/1541143834 wait complete. 2024-09-21T05:00:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.954+0000 7f26c82a3640 1 Processor -- start 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.954+0000 7f26c82a3640 1 -- start start 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c82a3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c82a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c82a3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010edf0 con 0x7f26c0102ac0 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c82a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010ec70 con 0x7f26c01033e0 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c603f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c603f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34766/0 (socket says 172.21.15.7:34766) 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c603f640 1 -- 172.21.15.7:0/3647233162 learned_addr learned my addr 172.21.15.7:0/3647233162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c583e640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c603f640 1 -- 172.21.15.7:0/3647233162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 msgr2=0x7f26c019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c603f640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c603f640 1 -- 172.21.15.7:0/3647233162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c01977c0 con 0x7f26c0102ac0 2024-09-21T05:00:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26c603f640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c019d640 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f26b002af20 tx=0x7f26b002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3647233162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f26b0047070 con 0x7f26c0102ac0 2024-09-21T05:00:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.955+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3647233162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26b0003e30 con 0x7f26c0102ac0 2024-09-21T05:00:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.956+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c0197a50 con 0x7f26c0102ac0 2024-09-21T05:00:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.956+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3647233162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f26b0033050 con 0x7f26c0102ac0 2024-09-21T05:00:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.956+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c0197f80 con 0x7f26c0102ac0 2024-09-21T05:00:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.957+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c006aa90 con 0x7f26c0102ac0 2024-09-21T05:00:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.957+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3647233162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f26b004a070 con 0x7f26c0102ac0 2024-09-21T05:00:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.958+0000 7f26bf7fe640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26a807af90 0x7f26a807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.958+0000 7f26c583e640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26a807af90 0x7f26a807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.958+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3647233162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f26b003e030 con 0x7f26c0102ac0 2024-09-21T05:00:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.959+0000 7f26c583e640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26a807af90 0x7f26a807d450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f26c0108900 tx=0x7f26b400b2e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:11.963+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3647233162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26b008a670 con 0x7f26c0102ac0 2024-09-21T05:00:12.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.169+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f26c0198230 con 0x7f26a807af90 2024-09-21T05:00:12.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.171+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3647233162 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f26c0198230 con 0x7f26a807af90 2024-09-21T05:00:12.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26a807af90 msgr2=0x7f26a807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26a807af90 0x7f26a807d450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f26c0108900 tx=0x7f26b400b2e0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 msgr2=0x7f26c019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c019d640 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f26b002af20 tx=0x7f26b002ee60 comp rx=0 tx=0).stop 2024-09-21T05:00:12.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 shutdown_connections 2024-09-21T05:00:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f26a807af90 0x7f26a807d450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c01033e0 0x7f26c019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 --2- 172.21.15.7:0/3647233162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26c0102ac0 0x7f26c019d640 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 >> 172.21.15.7:0/3647233162 conn(0x7f26c00fc520 msgr2=0x7f26c010af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 shutdown_connections 2024-09-21T05:00:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.173+0000 7f26c82a3640 1 -- 172.21.15.7:0/3647233162 wait complete. 2024-09-21T05:00:12.187 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:00:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.260+0000 7f9990d60640 1 -- 172.21.15.7:0/3923838735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c102ac0 msgr2=0x7f998c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.260+0000 7f9990d60640 1 --2- 172.21.15.7:0/3923838735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c102ac0 0x7f998c102ea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f997c009880 tx=0x7f997c02a530 comp rx=0 tx=0).stop 2024-09-21T05:00:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.260+0000 7f9990d60640 1 -- 172.21.15.7:0/3923838735 shutdown_connections 2024-09-21T05:00:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.260+0000 7f9990d60640 1 --2- 172.21.15.7:0/3923838735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f998c1033e0 0x7f998c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.260+0000 7f9990d60640 1 --2- 172.21.15.7:0/3923838735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c102ac0 0x7f998c102ea0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.260+0000 7f9990d60640 1 -- 172.21.15.7:0/3923838735 >> 172.21.15.7:0/3923838735 conn(0x7f998c0fc520 msgr2=0x7f998c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.260+0000 7f9990d60640 1 -- 172.21.15.7:0/3923838735 shutdown_connections 2024-09-21T05:00:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.260+0000 7f9990d60640 1 -- 172.21.15.7:0/3923838735 wait complete. 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9990d60640 1 Processor -- start 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9990d60640 1 -- start start 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9990d60640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f998c102ac0 0x7f998c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9990d60640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c1033e0 0x7f998c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9990d60640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f998c10edd0 con 0x7f998c1033e0 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9990d60640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f998c10ec50 con 0x7f998c102ac0 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9989d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c1033e0 0x7f998c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9989d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c1033e0 0x7f998c19dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34786/0 (socket says 172.21.15.7:34786) 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9989d9b640 1 -- 172.21.15.7:0/621413176 learned_addr learned my addr 172.21.15.7:0/621413176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f998a59c640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f998c102ac0 0x7f998c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9989d9b640 1 -- 172.21.15.7:0/621413176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f998c102ac0 msgr2=0x7f998c19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9989d9b640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f998c102ac0 0x7f998c19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f9989d9b640 1 -- 172.21.15.7:0/621413176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c19e310 con 0x7f998c1033e0 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.261+0000 7f998a59c640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f998c102ac0 0x7f998c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.262+0000 7f9989d9b640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c1033e0 0x7f998c19dc60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f997400c8f0 tx=0x7f997400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.262+0000 7f99837fe640 1 -- 172.21.15.7:0/621413176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f99740079b0 con 0x7f998c1033e0 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.262+0000 7f99837fe640 1 -- 172.21.15.7:0/621413176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9974007b50 con 0x7f998c1033e0 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.262+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c197a00 con 0x7f998c1033e0 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.262+0000 7f99837fe640 1 -- 172.21.15.7:0/621413176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9974018610 con 0x7f998c1033e0 2024-09-21T05:00:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.262+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c197f60 con 0x7f998c1033e0 2024-09-21T05:00:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.263+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998c06aa90 con 0x7f998c1033e0 2024-09-21T05:00:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.264+0000 7f99837fe640 1 -- 172.21.15.7:0/621413176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9974007d00 con 0x7f998c1033e0 2024-09-21T05:00:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.264+0000 7f99837fe640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f997807af80 0x7f997807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.264+0000 7f99837fe640 1 -- 172.21.15.7:0/621413176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f997409e200 con 0x7f998c1033e0 2024-09-21T05:00:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.264+0000 7f998a59c640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f997807af80 0x7f997807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.265+0000 7f998a59c640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f997807af80 0x7f997807d440 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f997c006010 tx=0x7f997c004220 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.269+0000 7f99837fe640 1 -- 172.21.15.7:0/621413176 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9974014030 con 0x7f998c1033e0 2024-09-21T05:00:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.473+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f998c198e80 con 0x7f997807af80 2024-09-21T05:00:12.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.475+0000 7f99837fe640 1 -- 172.21.15.7:0/621413176 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f998c198e80 con 0x7f997807af80 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f997807af80 msgr2=0x7f997807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f997807af80 0x7f997807d440 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f997c006010 tx=0x7f997c004220 comp rx=0 tx=0).stop 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c1033e0 msgr2=0x7f998c19dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c1033e0 0x7f998c19dc60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f997400c8f0 tx=0x7f997400cdb0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 shutdown_connections 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f997807af80 0x7f997807d440 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c1033e0 0x7f998c19dc60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 --2- 172.21.15.7:0/621413176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f998c102ac0 0x7f998c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 >> 172.21.15.7:0/621413176 conn(0x7f998c0fc520 msgr2=0x7f998c108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 shutdown_connections 2024-09-21T05:00:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.477+0000 7f9990d60640 1 -- 172.21.15.7:0/621413176 wait complete. 2024-09-21T05:00:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 -- 172.21.15.7:0/2187415425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 msgr2=0x7f187810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 --2- 172.21.15.7:0/2187415425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 0x7f187810b970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f186c009880 tx=0x7f186c02a530 comp rx=0 tx=0).stop 2024-09-21T05:00:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 -- 172.21.15.7:0/2187415425 shutdown_connections 2024-09-21T05:00:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 --2- 172.21.15.7:0/2187415425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 0x7f187810b970 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 --2- 172.21.15.7:0/2187415425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1878102b60 0x7f1878102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 -- 172.21.15.7:0/2187415425 >> 172.21.15.7:0/2187415425 conn(0x7f18780fc520 msgr2=0x7f18780fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 -- 172.21.15.7:0/2187415425 shutdown_connections 2024-09-21T05:00:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 -- 172.21.15.7:0/2187415425 wait complete. 2024-09-21T05:00:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.561+0000 7f187e958640 1 Processor -- start 2024-09-21T05:00:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f187e958640 1 -- start start 2024-09-21T05:00:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f187e958640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1878102b60 0x7f18781007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f187e958640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 0x7f1878100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f187e958640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f187810ee50 con 0x7f1878103480 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f187e958640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f187810ecd0 con 0x7f1878102b60 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f18777fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 0x7f1878100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f18777fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 0x7f1878100d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34808/0 (socket says 172.21.15.7:34808) 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f18777fe640 1 -- 172.21.15.7:0/1486645563 learned_addr learned my addr 172.21.15.7:0/1486645563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f1877fff640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1878102b60 0x7f18781007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f18777fe640 1 -- 172.21.15.7:0/1486645563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1878102b60 msgr2=0x7f18781007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f18777fe640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1878102b60 0x7f18781007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.562+0000 7f18777fe640 1 -- 172.21.15.7:0/1486645563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1878101240 con 0x7f1878103480 2024-09-21T05:00:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.563+0000 7f18777fe640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 0x7f1878100d00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f186c004f90 tx=0x7f186c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.563+0000 7f18757fa640 1 -- 172.21.15.7:0/1486645563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f186c03f070 con 0x7f1878103480 2024-09-21T05:00:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.563+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18781014d0 con 0x7f1878103480 2024-09-21T05:00:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.563+0000 7f18757fa640 1 -- 172.21.15.7:0/1486645563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f186c004cf0 con 0x7f1878103480 2024-09-21T05:00:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.563+0000 7f18757fa640 1 -- 172.21.15.7:0/1486645563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f186c035030 con 0x7f1878103480 2024-09-21T05:00:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.563+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781a3de0 con 0x7f1878103480 2024-09-21T05:00:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.564+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187806a9f0 con 0x7f1878103480 2024-09-21T05:00:12.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.565+0000 7f18757fa640 1 -- 172.21.15.7:0/1486645563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f186c042070 con 0x7f1878103480 2024-09-21T05:00:12.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.565+0000 7f18757fa640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f186007af30 0x7f186007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.566+0000 7f18757fa640 1 -- 172.21.15.7:0/1486645563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f186c0beb10 con 0x7f1878103480 2024-09-21T05:00:12.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.566+0000 7f1877fff640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f186007af30 0x7f186007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.566+0000 7f1877fff640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f186007af30 0x7f186007d3f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1868009990 tx=0x7f1868009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:12.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.570+0000 7f18757fa640 1 -- 172.21.15.7:0/1486645563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f186c08a0b0 con 0x7f1878103480 2024-09-21T05:00:12.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.765+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1878106060 con 0x7f186007af30 2024-09-21T05:00:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.775+0000 7f18757fa640 1 -- 172.21.15.7:0/1486645563 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f186800e850 con 0x7f186007af30 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (22m) 51s ago 33m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (25m) 51s ago 33m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (24m) 54s ago 32m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (20m) 51s ago 33m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (20m) 54s ago 32m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (21m) 51s ago 32m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (16m) 51s ago 27m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (16m) 51s ago 27m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (57s) 54s ago 27m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (16m) 54s ago 27m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (24m) 51s ago 34m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:00:12.779 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (23m) 54s ago 32m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (20m) 51s ago 34m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (20m) 54s ago 32m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (23m) 51s ago 33m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (23m) 54s ago 32m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (20m) 51s ago 31m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (19m) 51s ago 30m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (18m) 51s ago 30m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (18m) 54s ago 29m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (17m) 54s ago 29m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (16m) 54s ago 28m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:00:12.780 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (22m) 51s ago 32m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f186007af30 msgr2=0x7f186007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f186007af30 0x7f186007d3f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1868009990 tx=0x7f1868009110 comp rx=0 tx=0).stop 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 msgr2=0x7f1878100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 0x7f1878100d00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f186c004f90 tx=0x7f186c0048f0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 shutdown_connections 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f186007af30 0x7f186007d3f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1878103480 0x7f1878100d00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 --2- 172.21.15.7:0/1486645563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1878102b60 0x7f18781007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 >> 172.21.15.7:0/1486645563 conn(0x7f18780fc520 msgr2=0x7f1878103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 shutdown_connections 2024-09-21T05:00:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.777+0000 7f187e958640 1 -- 172.21.15.7:0/1486645563 wait complete. 2024-09-21T05:00:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.860+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/3201901682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 msgr2=0x7fbe14107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.860+0000 7fbe1a6ee640 1 --2- 172.21.15.7:0/3201901682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 0x7fbe14107ce0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009880 tx=0x7fbe0402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.860+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/3201901682 shutdown_connections 2024-09-21T05:00:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.860+0000 7fbe1a6ee640 1 --2- 172.21.15.7:0/3201901682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe14101900 0x7fbe14101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.860+0000 7fbe1a6ee640 1 --2- 172.21.15.7:0/3201901682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 0x7fbe14107ce0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.860+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/3201901682 >> 172.21.15.7:0/3201901682 conn(0x7fbe140fd600 msgr2=0x7fbe140ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.860+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/3201901682 shutdown_connections 2024-09-21T05:00:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.860+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/3201901682 wait complete. 2024-09-21T05:00:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.861+0000 7fbe1a6ee640 1 Processor -- start 2024-09-21T05:00:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.861+0000 7fbe1a6ee640 1 -- start start 2024-09-21T05:00:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.861+0000 7fbe1a6ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe14101900 0x7fbe140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.861+0000 7fbe1a6ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 0x7fbe14074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.861+0000 7fbe1a6ee640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1410edb0 con 0x7fbe14107900 2024-09-21T05:00:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.861+0000 7fbe1a6ee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1410ec30 con 0x7fbe14101900 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 0x7fbe14074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 0x7fbe14074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34830/0 (socket says 172.21.15.7:34830) 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe137fe640 1 -- 172.21.15.7:0/42061153 learned_addr learned my addr 172.21.15.7:0/42061153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe13fff640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe14101900 0x7fbe140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe137fe640 1 -- 172.21.15.7:0/42061153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe14101900 msgr2=0x7fbe140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe137fe640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe14101900 0x7fbe140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe137fe640 1 -- 172.21.15.7:0/42061153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe14074f70 con 0x7fbe14107900 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe137fe640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 0x7fbe14074a30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800d8d0 tx=0x7fbe0800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe117fa640 1 -- 172.21.15.7:0/42061153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbe08017070 con 0x7fbe14107900 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe117fa640 1 -- 172.21.15.7:0/42061153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbe08010460 con 0x7fbe14107900 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe14075260 con 0x7fbe14107900 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.862+0000 7fbe117fa640 1 -- 172.21.15.7:0/42061153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbe0800f5f0 con 0x7fbe14107900 2024-09-21T05:00:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.863+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141a3e40 con 0x7fbe14107900 2024-09-21T05:00:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.864+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe14103040 con 0x7fbe14107900 2024-09-21T05:00:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.864+0000 7fbe117fa640 1 -- 172.21.15.7:0/42061153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbe08009a30 con 0x7fbe14107900 2024-09-21T05:00:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.865+0000 7fbe117fa640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbdfc07af80 0x7fbdfc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.865+0000 7fbe117fa640 1 -- 172.21.15.7:0/42061153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbe0809ddc0 con 0x7fbe14107900 2024-09-21T05:00:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.865+0000 7fbe13fff640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbdfc07af80 0x7fbdfc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.865+0000 7fbe13fff640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbdfc07af80 0x7fbdfc07d440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbe040096c0 tx=0x7fbe04031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:12.869+0000 7fbe117fa640 1 -- 172.21.15.7:0/42061153 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe080693e0 con 0x7fbe14107900 2024-09-21T05:00:13.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.132+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fbe14101d60 con 0x7fbe14107900 2024-09-21T05:00:13.136 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:13 smithi007 ceph-mon[108238]: pgmap v764: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:13.136 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:13 smithi007 ceph-mon[108238]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.133+0000 7fbe117fa640 1 -- 172.21.15.7:0/42061153 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fbe14101d60 con 0x7fbe14107900 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:00:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:00:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:00:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:00:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:00:13.138 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbdfc07af80 msgr2=0x7fbdfc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbdfc07af80 0x7fbdfc07d440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbe040096c0 tx=0x7fbe04031400 comp rx=0 tx=0).stop 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 msgr2=0x7fbe14074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 0x7fbe14074a30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800d8d0 tx=0x7fbe0800dd90 comp rx=0 tx=0).stop 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 shutdown_connections 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbdfc07af80 0x7fbdfc07d440 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe14107900 0x7fbe14074a30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 --2- 172.21.15.7:0/42061153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe14101900 0x7fbe140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 >> 172.21.15.7:0/42061153 conn(0x7fbe140fd600 msgr2=0x7fbe140ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 shutdown_connections 2024-09-21T05:00:13.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.135+0000 7fbe1a6ee640 1 -- 172.21.15.7:0/42061153 wait complete. 2024-09-21T05:00:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.219+0000 7f9b1815f640 1 -- 172.21.15.7:0/2392981951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10100800 msgr2=0x7f9b10100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.219+0000 7f9b1815f640 1 --2- 172.21.15.7:0/2392981951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10100800 0x7f9b10100c60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009880 tx=0x7f9b0002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.219+0000 7f9b1815f640 1 -- 172.21.15.7:0/2392981951 shutdown_connections 2024-09-21T05:00:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.219+0000 7f9b1815f640 1 --2- 172.21.15.7:0/2392981951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10100800 0x7f9b10100c60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.219+0000 7f9b1815f640 1 --2- 172.21.15.7:0/2392981951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b10106800 0x7f9b10106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.219+0000 7f9b1815f640 1 -- 172.21.15.7:0/2392981951 >> 172.21.15.7:0/2392981951 conn(0x7f9b100fc520 msgr2=0x7f9b100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.220+0000 7f9b1815f640 1 -- 172.21.15.7:0/2392981951 shutdown_connections 2024-09-21T05:00:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.220+0000 7f9b1815f640 1 -- 172.21.15.7:0/2392981951 wait complete. 2024-09-21T05:00:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.220+0000 7f9b1815f640 1 Processor -- start 2024-09-21T05:00:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.220+0000 7f9b1815f640 1 -- start start 2024-09-21T05:00:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.220+0000 7f9b1815f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b10100800 0x7f9b100fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.220+0000 7f9b1815f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10106800 0x7f9b100fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.220+0000 7f9b1815f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1010dcb0 con 0x7f9b10106800 2024-09-21T05:00:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.220+0000 7f9b1815f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1010db30 con 0x7f9b10100800 2024-09-21T05:00:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b156fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10106800 0x7f9b100fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b156fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10106800 0x7f9b100fd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34848/0 (socket says 172.21.15.7:34848) 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b156fa640 1 -- 172.21.15.7:0/3775650736 learned_addr learned my addr 172.21.15.7:0/3775650736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b15efb640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b10100800 0x7f9b100fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b156fa640 1 -- 172.21.15.7:0/3775650736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b10100800 msgr2=0x7f9b100fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b156fa640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b10100800 0x7f9b100fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b156fa640 1 -- 172.21.15.7:0/3775650736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b100fd690 con 0x7f9b10106800 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b156fa640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10106800 0x7f9b100fd150 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9b00031460 tx=0x7f9b000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b06ffd640 1 -- 172.21.15.7:0/3775650736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b0003f070 con 0x7f9b10106800 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b06ffd640 1 -- 172.21.15.7:0/3775650736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b000035f0 con 0x7f9b10106800 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b100fd920 con 0x7f9b10106800 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b06ffd640 1 -- 172.21.15.7:0/3775650736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b00035030 con 0x7f9b10106800 2024-09-21T05:00:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.221+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101a3db0 con 0x7f9b10106800 2024-09-21T05:00:13.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.222+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b10101f40 con 0x7f9b10106800 2024-09-21T05:00:13.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.223+0000 7f9b06ffd640 1 -- 172.21.15.7:0/3775650736 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9b00042070 con 0x7f9b10106800 2024-09-21T05:00:13.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.224+0000 7f9b06ffd640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9afc07af80 0x7f9afc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.224+0000 7f9b06ffd640 1 -- 172.21.15.7:0/3775650736 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f9b000be8c0 con 0x7f9b10106800 2024-09-21T05:00:13.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.224+0000 7f9b15efb640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9afc07af80 0x7f9afc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.224+0000 7f9b15efb640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9afc07af80 0x7f9afc07d440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005d00 tx=0x7f9b0c00b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:13.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.228+0000 7f9b06ffd640 1 -- 172.21.15.7:0/3775650736 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9b00089ee0 con 0x7f9b10106800 2024-09-21T05:00:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:13 smithi053 ceph-mon[80176]: pgmap v764: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:13 smithi053 ceph-mon[80176]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.454+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9b100fe140 con 0x7f9b10106800 2024-09-21T05:00:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.455+0000 7f9b06ffd640 1 -- 172.21.15.7:0/3775650736 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f9b0008ed90 con 0x7f9b10106800 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:00:13.459 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:00:13.460 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:00:13.461 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:00:13.462 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:00:13.462 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:00:13.462 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:00:13.462 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9afc07af80 msgr2=0x7f9afc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9afc07af80 0x7f9afc07d440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005d00 tx=0x7f9b0c00b410 comp rx=0 tx=0).stop 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10106800 msgr2=0x7f9b100fd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10106800 0x7f9b100fd150 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9b00031460 tx=0x7f9b000045e0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 shutdown_connections 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9afc07af80 0x7f9afc07d440 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b10106800 0x7f9b100fd150 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 --2- 172.21.15.7:0/3775650736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b10100800 0x7f9b100fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 >> 172.21.15.7:0/3775650736 conn(0x7f9b100fc520 msgr2=0x7f9b1006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 shutdown_connections 2024-09-21T05:00:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.458+0000 7f9b1815f640 1 -- 172.21.15.7:0/3775650736 wait complete. 2024-09-21T05:00:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.544+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/402051117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68101900 msgr2=0x7fcd68101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.544+0000 7fcd6c8c9640 1 --2- 172.21.15.7:0/402051117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68101900 0x7fcd68101d60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009880 tx=0x7fcd5c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.544+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/402051117 shutdown_connections 2024-09-21T05:00:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.544+0000 7fcd6c8c9640 1 --2- 172.21.15.7:0/402051117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68101900 0x7fcd68101d60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.544+0000 7fcd6c8c9640 1 --2- 172.21.15.7:0/402051117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd68107900 0x7fcd68107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.544+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/402051117 >> 172.21.15.7:0/402051117 conn(0x7fcd680fd600 msgr2=0x7fcd680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.544+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/402051117 shutdown_connections 2024-09-21T05:00:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.544+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/402051117 wait complete. 2024-09-21T05:00:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd6c8c9640 1 Processor -- start 2024-09-21T05:00:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd6c8c9640 1 -- start start 2024-09-21T05:00:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd6c8c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd68101900 0x7fcd680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd6c8c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68107900 0x7fcd68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd6c8c9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6810ee30 con 0x7fcd68107900 2024-09-21T05:00:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd6c8c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6810ecb0 con 0x7fcd68101900 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd65d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68107900 0x7fcd68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd65d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68107900 0x7fcd68074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54814/0 (socket says 172.21.15.7:54814) 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.545+0000 7fcd65d9b640 1 -- 172.21.15.7:0/953108008 learned_addr learned my addr 172.21.15.7:0/953108008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd6659c640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd68101900 0x7fcd680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd65d9b640 1 -- 172.21.15.7:0/953108008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd68101900 msgr2=0x7fcd680744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd65d9b640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd68101900 0x7fcd680744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd65d9b640 1 -- 172.21.15.7:0/953108008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd68074f70 con 0x7fcd68107900 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd65d9b640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68107900 0x7fcd68074a30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c031460 tx=0x7fcd5c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd577fe640 1 -- 172.21.15.7:0/953108008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcd5c03f070 con 0x7fcd68107900 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd68075200 con 0x7fcd68107900 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd577fe640 1 -- 172.21.15.7:0/953108008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcd5c004cf0 con 0x7fcd68107900 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd577fe640 1 -- 172.21.15.7:0/953108008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcd5c035030 con 0x7fcd68107900 2024-09-21T05:00:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.546+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd681a3e00 con 0x7fcd68107900 2024-09-21T05:00:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.547+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd68103040 con 0x7fcd68107900 2024-09-21T05:00:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.548+0000 7fcd577fe640 1 -- 172.21.15.7:0/953108008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fcd5c042070 con 0x7fcd68107900 2024-09-21T05:00:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.548+0000 7fcd577fe640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcd5007af30 0x7fcd5007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.549+0000 7fcd577fe640 1 -- 172.21.15.7:0/953108008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fcd5c0beb60 con 0x7fcd68107900 2024-09-21T05:00:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.549+0000 7fcd6659c640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcd5007af30 0x7fcd5007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.549+0000 7fcd6659c640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcd5007af30 0x7fcd5007d3f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009890 tx=0x7fcd58008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.553+0000 7fcd577fe640 1 -- 172.21.15.7:0/953108008 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd5c08a100 con 0x7fcd68107900 2024-09-21T05:00:13.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.757+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fcd68062630 con 0x7fcd5007af30 2024-09-21T05:00:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.759+0000 7fcd577fe640 1 -- 172.21.15.7:0/953108008 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fcd68062630 con 0x7fcd5007af30 2024-09-21T05:00:13.763 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:00:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:00:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:00:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:00:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:00:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:00:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:00:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:00:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:00:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:00:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:00:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:00:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:00:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:00:13.764 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:00:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcd5007af30 msgr2=0x7fcd5007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcd5007af30 0x7fcd5007d3f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009890 tx=0x7fcd58008040 comp rx=0 tx=0).stop 2024-09-21T05:00:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68107900 msgr2=0x7fcd68074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68107900 0x7fcd68074a30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c031460 tx=0x7fcd5c0048f0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 shutdown_connections 2024-09-21T05:00:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcd5007af30 0x7fcd5007d3f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd68107900 0x7fcd68074a30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 --2- 172.21.15.7:0/953108008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd68101900 0x7fcd680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 >> 172.21.15.7:0/953108008 conn(0x7fcd680fd600 msgr2=0x7fcd680fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 shutdown_connections 2024-09-21T05:00:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.761+0000 7fcd6c8c9640 1 -- 172.21.15.7:0/953108008 wait complete. 2024-09-21T05:00:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.847+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2992309691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074620 msgr2=0x7f53c4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.847+0000 7f53ca9ac640 1 --2- 172.21.15.7:0/2992309691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074620 0x7f53c4074a00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009880 tx=0x7f53b402a530 comp rx=0 tx=0).stop 2024-09-21T05:00:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.847+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2992309691 shutdown_connections 2024-09-21T05:00:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.847+0000 7f53ca9ac640 1 --2- 172.21.15.7:0/2992309691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4074f40 0x7f53c410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.847+0000 7f53ca9ac640 1 --2- 172.21.15.7:0/2992309691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074620 0x7f53c4074a00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.847+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2992309691 >> 172.21.15.7:0/2992309691 conn(0x7f53c40fd600 msgr2=0x7f53c40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.847+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2992309691 shutdown_connections 2024-09-21T05:00:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.847+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2992309691 wait complete. 2024-09-21T05:00:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53ca9ac640 1 Processor -- start 2024-09-21T05:00:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53ca9ac640 1 -- start start 2024-09-21T05:00:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53ca9ac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4074620 0x7f53c419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53ca9ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074f40 0x7f53c419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53ca9ac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4110fd0 con 0x7f53c4074f40 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53ca9ac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4110e50 con 0x7f53c4074620 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074f40 0x7f53c419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074f40 0x7f53c419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54842/0 (socket says 172.21.15.7:54842) 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.848+0000 7f53c37fe640 1 -- 172.21.15.7:0/2331501880 learned_addr learned my addr 172.21.15.7:0/2331501880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c3fff640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4074620 0x7f53c419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c37fe640 1 -- 172.21.15.7:0/2331501880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4074620 msgr2=0x7f53c419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c37fe640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4074620 0x7f53c419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c37fe640 1 -- 172.21.15.7:0/2331501880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c41a0410 con 0x7f53c4074f40 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c3fff640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4074620 0x7f53c419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c37fe640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074f40 0x7f53c419fd60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c8a0 tx=0x7f53b800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c17fa640 1 -- 172.21.15.7:0/2331501880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f53b80079b0 con 0x7f53c4074f40 2024-09-21T05:00:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c17fa640 1 -- 172.21.15.7:0/2331501880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53b8007b50 con 0x7f53c4074f40 2024-09-21T05:00:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c4199b00 con 0x7f53c4074f40 2024-09-21T05:00:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53c17fa640 1 -- 172.21.15.7:0/2331501880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f53b8018700 con 0x7f53c4074f40 2024-09-21T05:00:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.849+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c419a060 con 0x7f53c4074f40 2024-09-21T05:00:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.850+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c4075cd0 con 0x7f53c4074f40 2024-09-21T05:00:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.851+0000 7f53c17fa640 1 -- 172.21.15.7:0/2331501880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f53b8007d00 con 0x7f53c4074f40 2024-09-21T05:00:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.851+0000 7f53c17fa640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f53ac07af80 0x7f53ac07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.851+0000 7f53c17fa640 1 -- 172.21.15.7:0/2331501880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f53b809e2b0 con 0x7f53c4074f40 2024-09-21T05:00:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.851+0000 7f53c3fff640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f53ac07af80 0x7f53ac07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.852+0000 7f53c3fff640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f53ac07af80 0x7f53ac07d440 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f53b4006010 tx=0x7f53b4004220 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:13.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:13.856+0000 7f53c17fa640 1 -- 172.21.15.7:0/2331501880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53b8014030 con 0x7f53c4074f40 2024-09-21T05:00:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.117+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f53c419af40 con 0x7f53c4074f40 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.118+0000 7f53c17fa640 1 -- 172.21.15.7:0/2331501880 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f53b80698d0 con 0x7f53c4074f40 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:00:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:00:14.123 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:00:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:00:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f53ac07af80 msgr2=0x7f53ac07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f53ac07af80 0x7f53ac07d440 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f53b4006010 tx=0x7f53b4004220 comp rx=0 tx=0).stop 2024-09-21T05:00:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074f40 msgr2=0x7f53c419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074f40 0x7f53c419fd60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c8a0 tx=0x7f53b800cd60 comp rx=0 tx=0).stop 2024-09-21T05:00:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 shutdown_connections 2024-09-21T05:00:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f53ac07af80 0x7f53ac07d440 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53c4074f40 0x7f53c419fd60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 --2- 172.21.15.7:0/2331501880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53c4074620 0x7f53c419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 >> 172.21.15.7:0/2331501880 conn(0x7f53c40fd600 msgr2=0x7f53c410b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 shutdown_connections 2024-09-21T05:00:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:14.120+0000 7f53ca9ac640 1 -- 172.21.15.7:0/2331501880 wait complete. 2024-09-21T05:00:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:14 smithi053 ceph-mon[80176]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:14 smithi053 ceph-mon[80176]: from='client.35216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:14 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/42061153' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:00:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:14 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3775650736' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:00:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:00:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:14 smithi007 ceph-mon[108238]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:14 smithi007 ceph-mon[108238]: from='client.35216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:14 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/42061153' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:00:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:14 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3775650736' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:00:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:00:15.337 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:15 smithi053 ceph-mon[80176]: pgmap v765: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:15.337 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:15 smithi053 ceph-mon[80176]: from='client.35228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:15.337 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:15 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2331501880' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:00:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:15 smithi007 ceph-mon[108238]: pgmap v765: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:15 smithi007 ceph-mon[108238]: from='client.35228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:15 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2331501880' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:00:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:17 smithi007 ceph-mon[108238]: pgmap v766: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:17 smithi053 ceph-mon[80176]: pgmap v766: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:19 smithi053 ceph-mon[80176]: pgmap v767: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:19 smithi007 ceph-mon[108238]: pgmap v767: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:21 smithi053 ceph-mon[80176]: pgmap v768: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:21 smithi007 ceph-mon[108238]: pgmap v768: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:23 smithi007 ceph-mon[108238]: pgmap v769: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:23.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:23 smithi053 ceph-mon[80176]: pgmap v769: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:25 smithi053 ceph-mon[80176]: pgmap v770: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:25 smithi007 ceph-mon[108238]: pgmap v770: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:27 smithi053 ceph-mon[80176]: pgmap v771: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:27 smithi007 ceph-mon[108238]: pgmap v771: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:29 smithi007 ceph-mon[108238]: pgmap v772: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:00:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:29 smithi053 ceph-mon[80176]: pgmap v772: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:00:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:31 smithi053 ceph-mon[80176]: pgmap v773: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:31 smithi007 ceph-mon[108238]: pgmap v773: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:33 smithi053 ceph-mon[80176]: pgmap v774: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:33 smithi007 ceph-mon[108238]: pgmap v774: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:35.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:35 smithi053 ceph-mon[80176]: pgmap v775: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:35 smithi007 ceph-mon[108238]: pgmap v775: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:37 smithi053 ceph-mon[80176]: pgmap v776: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:37 smithi007 ceph-mon[108238]: pgmap v776: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:39 smithi053 ceph-mon[80176]: pgmap v777: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:39 smithi007 ceph-mon[108238]: pgmap v777: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:41 smithi007 ceph-mon[108238]: pgmap v778: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:41 smithi053 ceph-mon[80176]: pgmap v778: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:43 smithi053 ceph-mon[80176]: pgmap v779: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:43 smithi007 ceph-mon[108238]: pgmap v779: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:00:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.221+0000 7f15b414a640 1 -- 172.21.15.7:0/1131675999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 msgr2=0x7f15ac100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.221+0000 7f15b414a640 1 --2- 172.21.15.7:0/1131675999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 0x7f15ac100c50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f159c009880 tx=0x7f159c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.221+0000 7f15b414a640 1 -- 172.21.15.7:0/1131675999 shutdown_connections 2024-09-21T05:00:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.221+0000 7f15b414a640 1 --2- 172.21.15.7:0/1131675999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 0x7f15ac100c50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.221+0000 7f15b414a640 1 --2- 172.21.15.7:0/1131675999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ac1067f0 0x7f15ac106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.221+0000 7f15b414a640 1 -- 172.21.15.7:0/1131675999 >> 172.21.15.7:0/1131675999 conn(0x7f15ac0fc550 msgr2=0x7f15ac0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.221+0000 7f15b414a640 1 -- 172.21.15.7:0/1131675999 shutdown_connections 2024-09-21T05:00:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.221+0000 7f15b414a640 1 -- 172.21.15.7:0/1131675999 wait complete. 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b414a640 1 Processor -- start 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b414a640 1 -- start start 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b414a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 0x7f15ac0fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b414a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ac1067f0 0x7f15ac0fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b414a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10dcc0 con 0x7f15ac1007f0 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b414a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac10db40 con 0x7f15ac1067f0 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b1ee6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 0x7f15ac0fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b1ee6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 0x7f15ac0fccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45274/0 (socket says 172.21.15.7:45274) 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b1ee6640 1 -- 172.21.15.7:0/191503538 learned_addr learned my addr 172.21.15.7:0/191503538 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.222+0000 7f15b16e5640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ac1067f0 0x7f15ac0fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15b1ee6640 1 -- 172.21.15.7:0/191503538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ac1067f0 msgr2=0x7f15ac0fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15b1ee6640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ac1067f0 0x7f15ac0fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15b1ee6640 1 -- 172.21.15.7:0/191503538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac0fd750 con 0x7f15ac1007f0 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15b16e5640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ac1067f0 0x7f15ac0fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15b1ee6640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 0x7f15ac0fccd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f15a800d6f0 tx=0x7f15a800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15a2ffd640 1 -- 172.21.15.7:0/191503538 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f15a8017070 con 0x7f15ac1007f0 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac0fda40 con 0x7f15ac1007f0 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15a2ffd640 1 -- 172.21.15.7:0/191503538 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15a800de00 con 0x7f15ac1007f0 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15a2ffd640 1 -- 172.21.15.7:0/191503538 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f15a8004bb0 con 0x7f15ac1007f0 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.223+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1a3e60 con 0x7f15ac1007f0 2024-09-21T05:00:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.224+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac101f30 con 0x7f15ac1007f0 2024-09-21T05:00:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.225+0000 7f15a2ffd640 1 -- 172.21.15.7:0/191503538 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f15a8004e60 con 0x7f15ac1007f0 2024-09-21T05:00:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.226+0000 7f15a2ffd640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f159807af30 0x7f159807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.226+0000 7f15a2ffd640 1 -- 172.21.15.7:0/191503538 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f15a809e050 con 0x7f15ac1007f0 2024-09-21T05:00:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.226+0000 7f15b16e5640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f159807af30 0x7f159807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.227+0000 7f15b16e5640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f159807af30 0x7f159807d3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0fe760 tx=0x7f159c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.234+0000 7f15a2ffd640 1 -- 172.21.15.7:0/191503538 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15a8069710 con 0x7f15ac1007f0 2024-09-21T05:00:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.438+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f15ac062630 con 0x7f159807af30 2024-09-21T05:00:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.439+0000 7f15a2ffd640 1 -- 172.21.15.7:0/191503538 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f15ac062630 con 0x7f159807af30 2024-09-21T05:00:44.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:00:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f159807af30 msgr2=0x7f159807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f159807af30 0x7f159807d3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0fe760 tx=0x7f159c031400 comp rx=0 tx=0).stop 2024-09-21T05:00:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 msgr2=0x7f15ac0fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 0x7f15ac0fccd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f15a800d6f0 tx=0x7f15a800dbb0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 shutdown_connections 2024-09-21T05:00:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f159807af30 0x7f159807d3f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ac1067f0 0x7f15ac0fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 --2- 172.21.15.7:0/191503538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15ac1007f0 0x7f15ac0fccd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 >> 172.21.15.7:0/191503538 conn(0x7f15ac0fc550 msgr2=0x7f15ac069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 shutdown_connections 2024-09-21T05:00:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.441+0000 7f15b414a640 1 -- 172.21.15.7:0/191503538 wait complete. 2024-09-21T05:00:44.453 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:00:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.526+0000 7f24a8f31640 1 -- 172.21.15.7:0/1771332528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41067f0 msgr2=0x7f24a4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.526+0000 7f24a8f31640 1 --2- 172.21.15.7:0/1771332528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41067f0 0x7f24a4106bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2494009880 tx=0x7f249402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.527+0000 7f24a8f31640 1 -- 172.21.15.7:0/1771332528 shutdown_connections 2024-09-21T05:00:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.527+0000 7f24a8f31640 1 --2- 172.21.15.7:0/1771332528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a41007f0 0x7f24a4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.527+0000 7f24a8f31640 1 --2- 172.21.15.7:0/1771332528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41067f0 0x7f24a4106bd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.527+0000 7f24a8f31640 1 -- 172.21.15.7:0/1771332528 >> 172.21.15.7:0/1771332528 conn(0x7f24a40fc550 msgr2=0x7f24a40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.527+0000 7f24a8f31640 1 -- 172.21.15.7:0/1771332528 shutdown_connections 2024-09-21T05:00:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.527+0000 7f24a8f31640 1 -- 172.21.15.7:0/1771332528 wait complete. 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.527+0000 7f24a8f31640 1 Processor -- start 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a8f31640 1 -- start start 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a8f31640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41007f0 0x7f24a4107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a8f31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a41067f0 0x7f24a4107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a8f31640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410dca0 con 0x7f24a41007f0 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a8f31640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410db20 con 0x7f24a41067f0 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41007f0 0x7f24a4107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41007f0 0x7f24a4107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45288/0 (socket says 172.21.15.7:45288) 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a259c640 1 -- 172.21.15.7:0/2624921180 learned_addr learned my addr 172.21.15.7:0/2624921180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a1d9b640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a41067f0 0x7f24a4107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a259c640 1 -- 172.21.15.7:0/2624921180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a41067f0 msgr2=0x7f24a4107c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a259c640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a41067f0 0x7f24a4107c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a259c640 1 -- 172.21.15.7:0/2624921180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a41081a0 con 0x7f24a41007f0 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a1d9b640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a41067f0 0x7f24a4107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.528+0000 7f24a259c640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41007f0 0x7f24a4107720 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2494031460 tx=0x7f24940048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.529+0000 7f24937fe640 1 -- 172.21.15.7:0/2624921180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f249403f070 con 0x7f24a41007f0 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.529+0000 7f24937fe640 1 -- 172.21.15.7:0/2624921180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2494004cf0 con 0x7f24a41007f0 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.529+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a4108430 con 0x7f24a41007f0 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.529+0000 7f24937fe640 1 -- 172.21.15.7:0/2624921180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2494035030 con 0x7f24a41007f0 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.529+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41a3cc0 con 0x7f24a41007f0 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.530+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a4101f30 con 0x7f24a41007f0 2024-09-21T05:00:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.531+0000 7f24937fe640 1 -- 172.21.15.7:0/2624921180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2494042070 con 0x7f24a41007f0 2024-09-21T05:00:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.531+0000 7f24937fe640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f248c07af80 0x7f248c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.531+0000 7f24937fe640 1 -- 172.21.15.7:0/2624921180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f24940beab0 con 0x7f24a41007f0 2024-09-21T05:00:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.531+0000 7f24a1d9b640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f248c07af80 0x7f248c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.532+0000 7f24a1d9b640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f248c07af80 0x7f248c07d440 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24a41091b0 tx=0x7f2498005d90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:44.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.536+0000 7f24937fe640 1 -- 172.21.15.7:0/2624921180 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f249408a070 con 0x7f24a41007f0 2024-09-21T05:00:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.741+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f24a406a520 con 0x7f248c07af80 2024-09-21T05:00:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.742+0000 7f24937fe640 1 -- 172.21.15.7:0/2624921180 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f24a406a520 con 0x7f248c07af80 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f248c07af80 msgr2=0x7f248c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f248c07af80 0x7f248c07d440 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24a41091b0 tx=0x7f2498005d90 comp rx=0 tx=0).stop 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41007f0 msgr2=0x7f24a4107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41007f0 0x7f24a4107720 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2494031460 tx=0x7f24940048f0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 shutdown_connections 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f248c07af80 0x7f248c07d440 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a41067f0 0x7f24a4107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 --2- 172.21.15.7:0/2624921180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a41007f0 0x7f24a4107720 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 >> 172.21.15.7:0/2624921180 conn(0x7f24a40fc550 msgr2=0x7f24a4069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 shutdown_connections 2024-09-21T05:00:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.744+0000 7f24a8f31640 1 -- 172.21.15.7:0/2624921180 wait complete. 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.828+0000 7f98d2d7c640 1 -- 172.21.15.7:0/182591889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc106800 msgr2=0x7f98cc106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.828+0000 7f98d2d7c640 1 --2- 172.21.15.7:0/182591889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc106800 0x7f98cc106be0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009880 tx=0x7f98c002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.828+0000 7f98d2d7c640 1 -- 172.21.15.7:0/182591889 shutdown_connections 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.828+0000 7f98d2d7c640 1 --2- 172.21.15.7:0/182591889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc100800 0x7f98cc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.828+0000 7f98d2d7c640 1 --2- 172.21.15.7:0/182591889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc106800 0x7f98cc106be0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.828+0000 7f98d2d7c640 1 -- 172.21.15.7:0/182591889 >> 172.21.15.7:0/182591889 conn(0x7f98cc0fc520 msgr2=0x7f98cc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.828+0000 7f98d2d7c640 1 -- 172.21.15.7:0/182591889 shutdown_connections 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.828+0000 7f98d2d7c640 1 -- 172.21.15.7:0/182591889 wait complete. 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d2d7c640 1 Processor -- start 2024-09-21T05:00:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d2d7c640 1 -- start start 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d2d7c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc100800 0x7f98cc107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d2d7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc106800 0x7f98cc107c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d2d7c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc10dcb0 con 0x7f98cc100800 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d2d7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc10db30 con 0x7f98cc106800 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d0b18640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc100800 0x7f98cc107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d0b18640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc100800 0x7f98cc107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45306/0 (socket says 172.21.15.7:45306) 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.829+0000 7f98d0b18640 1 -- 172.21.15.7:0/1255016963 learned_addr learned my addr 172.21.15.7:0/1255016963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98cbfff640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc106800 0x7f98cc107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98d0b18640 1 -- 172.21.15.7:0/1255016963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc106800 msgr2=0x7f98cc107c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98d0b18640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc106800 0x7f98cc107c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98d0b18640 1 -- 172.21.15.7:0/1255016963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc1081b0 con 0x7f98cc100800 2024-09-21T05:00:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98d0b18640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc100800 0x7f98cc107730 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f98c0031460 tx=0x7f98c00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98c9ffb640 1 -- 172.21.15.7:0/1255016963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98c003f070 con 0x7f98cc100800 2024-09-21T05:00:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98c9ffb640 1 -- 172.21.15.7:0/1255016963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98c0004cf0 con 0x7f98cc100800 2024-09-21T05:00:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc108440 con 0x7f98cc100800 2024-09-21T05:00:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.830+0000 7f98c9ffb640 1 -- 172.21.15.7:0/1255016963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98c0035030 con 0x7f98cc100800 2024-09-21T05:00:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.831+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc1a3db0 con 0x7f98cc100800 2024-09-21T05:00:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.832+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98cc101f40 con 0x7f98cc100800 2024-09-21T05:00:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.832+0000 7f98c9ffb640 1 -- 172.21.15.7:0/1255016963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f98c0042070 con 0x7f98cc100800 2024-09-21T05:00:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.833+0000 7f98c9ffb640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98bc07af80 0x7f98bc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.833+0000 7f98c9ffb640 1 -- 172.21.15.7:0/1255016963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f98c00be990 con 0x7f98cc100800 2024-09-21T05:00:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.833+0000 7f98cbfff640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98bc07af80 0x7f98bc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.833+0000 7f98cbfff640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98bc07af80 0x7f98bc07d440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f98cc1091c0 tx=0x7f98b800a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:44.837+0000 7f98c9ffb640 1 -- 172.21.15.7:0/1255016963 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98c0089fb0 con 0x7f98cc100800 2024-09-21T05:00:45.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.031+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f98cc06a9f0 con 0x7f98bc07af80 2024-09-21T05:00:45.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.041+0000 7f98c9ffb640 1 -- 172.21.15.7:0/1255016963 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f98cc06a9f0 con 0x7f98bc07af80 2024-09-21T05:00:45.044 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:00:45.044 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (22m) 84s ago 33m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (25m) 84s ago 33m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (25m) 86s ago 33m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (20m) 84s ago 33m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (20m) 86s ago 33m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (22m) 84s ago 33m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (16m) 84s ago 28m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (17m) 84s ago 28m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (90s) 86s ago 28m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (16m) 86s ago 28m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (24m) 84s ago 35m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (24m) 86s ago 32m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (21m) 84s ago 35m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (21m) 86s ago 32m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:00:45.045 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (23m) 84s ago 33m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:00:45.046 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (23m) 86s ago 32m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:00:45.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (20m) 84s ago 31m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:00:45.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (19m) 84s ago 31m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:00:45.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (19m) 84s ago 30m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:00:45.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (18m) 86s ago 30m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:00:45.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (18m) 86s ago 29m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:00:45.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (17m) 86s ago 29m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:00:45.046 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (23m) 84s ago 33m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98bc07af80 msgr2=0x7f98bc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98bc07af80 0x7f98bc07d440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f98cc1091c0 tx=0x7f98b800a2d0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc100800 msgr2=0x7f98cc107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc100800 0x7f98cc107730 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f98c0031460 tx=0x7f98c00048f0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 shutdown_connections 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f98bc07af80 0x7f98bc07d440 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98cc106800 0x7f98cc107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 --2- 172.21.15.7:0/1255016963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98cc100800 0x7f98cc107730 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 >> 172.21.15.7:0/1255016963 conn(0x7f98cc0fc520 msgr2=0x7f98cc06a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 shutdown_connections 2024-09-21T05:00:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.044+0000 7f98d2d7c640 1 -- 172.21.15.7:0/1255016963 wait complete. 2024-09-21T05:00:45.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.127+0000 7fb834155640 1 -- 172.21.15.7:0/2168893007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 msgr2=0x7fb82c10dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.127+0000 7fb834155640 1 --2- 172.21.15.7:0/2168893007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 0x7fb82c10dcc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009880 tx=0x7fb81c02a490 comp rx=0 tx=0).stop 2024-09-21T05:00:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.127+0000 7fb834155640 1 -- 172.21.15.7:0/2168893007 shutdown_connections 2024-09-21T05:00:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.127+0000 7fb834155640 1 --2- 172.21.15.7:0/2168893007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 0x7fb82c10dcc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.127+0000 7fb834155640 1 --2- 172.21.15.7:0/2168893007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb82c100650 0x7fb82c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.127+0000 7fb834155640 1 -- 172.21.15.7:0/2168893007 >> 172.21.15.7:0/2168893007 conn(0x7fb82c0fc520 msgr2=0x7fb82c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.128+0000 7fb834155640 1 -- 172.21.15.7:0/2168893007 shutdown_connections 2024-09-21T05:00:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.128+0000 7fb834155640 1 -- 172.21.15.7:0/2168893007 wait complete. 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.128+0000 7fb834155640 1 Processor -- start 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.128+0000 7fb834155640 1 -- start start 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb834155640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb82c100650 0x7fb82c19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb834155640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 0x7fb82c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb834155640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c110f70 con 0x7fb82c101000 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb834155640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c110df0 con 0x7fb82c100650 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb8316f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 0x7fb82c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb8316f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 0x7fb82c19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45322/0 (socket says 172.21.15.7:45322) 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb8316f0640 1 -- 172.21.15.7:0/2204842141 learned_addr learned my addr 172.21.15.7:0/2204842141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb831ef1640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb82c100650 0x7fb82c19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb8316f0640 1 -- 172.21.15.7:0/2204842141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb82c100650 msgr2=0x7fb82c19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb8316f0640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb82c100650 0x7fb82c19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb8316f0640 1 -- 172.21.15.7:0/2204842141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c199a80 con 0x7fb82c101000 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb831ef1640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb82c100650 0x7fb82c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb8316f0640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 0x7fb82c19fe00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb81c031460 tx=0x7fb81c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb822ffd640 1 -- 172.21.15.7:0/2204842141 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb81c03f070 con 0x7fb82c101000 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb822ffd640 1 -- 172.21.15.7:0/2204842141 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb81c0035f0 con 0x7fb82c101000 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c199d10 con 0x7fb82c101000 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb822ffd640 1 -- 172.21.15.7:0/2204842141 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb81c035030 con 0x7fb82c101000 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.129+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c19a2a0 con 0x7fb82c101000 2024-09-21T05:00:45.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.131+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82c06aa90 con 0x7fb82c101000 2024-09-21T05:00:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.132+0000 7fb822ffd640 1 -- 172.21.15.7:0/2204842141 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb81c042070 con 0x7fb82c101000 2024-09-21T05:00:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.132+0000 7fb822ffd640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81807af30 0x7fb81807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.133+0000 7fb822ffd640 1 -- 172.21.15.7:0/2204842141 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fb81c0bea80 con 0x7fb82c101000 2024-09-21T05:00:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.133+0000 7fb831ef1640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81807af30 0x7fb81807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.133+0000 7fb831ef1640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81807af30 0x7fb81807d3f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb828005d00 tx=0x7fb828005ed0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.139+0000 7fb822ffd640 1 -- 172.21.15.7:0/2204842141 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb81c08a020 con 0x7fb82c101000 2024-09-21T05:00:45.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.402+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fb82c10b3a0 con 0x7fb82c101000 2024-09-21T05:00:45.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:45 smithi007 ceph-mon[108238]: pgmap v780: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:45.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.402+0000 7fb822ffd640 1 -- 172.21.15.7:0/2204842141 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fb81c08c070 con 0x7fb82c101000 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:00:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:00:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:00:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:00:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:00:45.407 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:00:45.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.404+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81807af30 msgr2=0x7fb81807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.404+0000 7fb834155640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81807af30 0x7fb81807d3f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb828005d00 tx=0x7fb828005ed0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.404+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 msgr2=0x7fb82c19fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.404+0000 7fb834155640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 0x7fb82c19fe00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb81c031460 tx=0x7fb81c0045e0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.405+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 shutdown_connections 2024-09-21T05:00:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.405+0000 7fb834155640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb81807af30 0x7fb81807d3f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.405+0000 7fb834155640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82c101000 0x7fb82c19fe00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.405+0000 7fb834155640 1 --2- 172.21.15.7:0/2204842141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb82c100650 0x7fb82c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.405+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 >> 172.21.15.7:0/2204842141 conn(0x7fb82c0fc520 msgr2=0x7fb82c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.405+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 shutdown_connections 2024-09-21T05:00:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.405+0000 7fb834155640 1 -- 172.21.15.7:0/2204842141 wait complete. 2024-09-21T05:00:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:45 smithi053 ceph-mon[80176]: pgmap v780: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 -- 172.21.15.7:0/882694728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 msgr2=0x7ff3b0106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 --2- 172.21.15.7:0/882694728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 0x7ff3b0106bf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009880 tx=0x7ff3a402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 -- 172.21.15.7:0/882694728 shutdown_connections 2024-09-21T05:00:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 --2- 172.21.15.7:0/882694728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3b0100810 0x7ff3b0100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 --2- 172.21.15.7:0/882694728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 0x7ff3b0106bf0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 -- 172.21.15.7:0/882694728 >> 172.21.15.7:0/882694728 conn(0x7ff3b00fc510 msgr2=0x7ff3b00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 -- 172.21.15.7:0/882694728 shutdown_connections 2024-09-21T05:00:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 -- 172.21.15.7:0/882694728 wait complete. 2024-09-21T05:00:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.489+0000 7ff3b6e42640 1 Processor -- start 2024-09-21T05:00:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3b6e42640 1 -- start start 2024-09-21T05:00:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3b6e42640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3b0100810 0x7ff3b00fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3b6e42640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 0x7ff3b00fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3b6e42640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010dce0 con 0x7ff3b0106810 2024-09-21T05:00:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3b6e42640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b010db60 con 0x7ff3b0100810 2024-09-21T05:00:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 0x7ff3b00fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 0x7ff3b00fd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45330/0 (socket says 172.21.15.7:45330) 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3affff640 1 -- 172.21.15.7:0/3280126720 learned_addr learned my addr 172.21.15.7:0/3280126720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3b4bde640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3b0100810 0x7ff3b00fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3affff640 1 -- 172.21.15.7:0/3280126720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3b0100810 msgr2=0x7ff3b00fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3affff640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3b0100810 0x7ff3b00fcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3affff640 1 -- 172.21.15.7:0/3280126720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b00fd690 con 0x7ff3b0106810 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.490+0000 7ff3affff640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 0x7ff3b00fd150 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00d6f0 tx=0x7ff39c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.491+0000 7ff3adffb640 1 -- 172.21.15.7:0/3280126720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff39c017070 con 0x7ff3b0106810 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.491+0000 7ff3adffb640 1 -- 172.21.15.7:0/3280126720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff39c00de00 con 0x7ff3b0106810 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.491+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b00fd980 con 0x7ff3b0106810 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.491+0000 7ff3adffb640 1 -- 172.21.15.7:0/3280126720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff39c004bb0 con 0x7ff3b0106810 2024-09-21T05:00:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.491+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b01a3ed0 con 0x7ff3b0106810 2024-09-21T05:00:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.492+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b0101f50 con 0x7ff3b0106810 2024-09-21T05:00:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.492+0000 7ff3adffb640 1 -- 172.21.15.7:0/3280126720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff39c009a30 con 0x7ff3b0106810 2024-09-21T05:00:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.493+0000 7ff3adffb640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff3a007af80 0x7ff3a007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.493+0000 7ff3adffb640 1 -- 172.21.15.7:0/3280126720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff39c09dbe0 con 0x7ff3b0106810 2024-09-21T05:00:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.493+0000 7ff3b4bde640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff3a007af80 0x7ff3a007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.493+0000 7ff3b4bde640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff3a007af80 0x7ff3a007d440 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4031460 tx=0x7ff3a4009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:45.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.497+0000 7ff3adffb640 1 -- 172.21.15.7:0/3280126720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff39c069200 con 0x7ff3b0106810 2024-09-21T05:00:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.725+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff3b0109d90 con 0x7ff3b0106810 2024-09-21T05:00:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.725+0000 7ff3adffb640 1 -- 172.21.15.7:0/3280126720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7ff39c05ee20 con 0x7ff3b0106810 2024-09-21T05:00:45.728 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:00:45.728 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:00:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:00:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:00:45.731 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:00:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:00:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff3a007af80 msgr2=0x7ff3a007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff3a007af80 0x7ff3a007d440 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4031460 tx=0x7ff3a4009560 comp rx=0 tx=0).stop 2024-09-21T05:00:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 msgr2=0x7ff3b00fd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 0x7ff3b00fd150 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00d6f0 tx=0x7ff39c00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 shutdown_connections 2024-09-21T05:00:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff3a007af80 0x7ff3a007d440 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3b0106810 0x7ff3b00fd150 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 --2- 172.21.15.7:0/3280126720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3b0100810 0x7ff3b00fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 >> 172.21.15.7:0/3280126720 conn(0x7ff3b00fc510 msgr2=0x7ff3b006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 shutdown_connections 2024-09-21T05:00:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.728+0000 7ff3b6e42640 1 -- 172.21.15.7:0/3280126720 wait complete. 2024-09-21T05:00:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.813+0000 7f4d9bf98640 1 -- 172.21.15.7:0/3385916236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941008b0 msgr2=0x7f4d94100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.813+0000 7f4d9bf98640 1 --2- 172.21.15.7:0/3385916236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941008b0 0x7f4d94100c90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009880 tx=0x7f4d9002a530 comp rx=0 tx=0).stop 2024-09-21T05:00:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.813+0000 7f4d9bf98640 1 -- 172.21.15.7:0/3385916236 shutdown_connections 2024-09-21T05:00:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.813+0000 7f4d9bf98640 1 --2- 172.21.15.7:0/3385916236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d941011d0 0x7f4d941056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.813+0000 7f4d9bf98640 1 --2- 172.21.15.7:0/3385916236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941008b0 0x7f4d94100c90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.813+0000 7f4d9bf98640 1 -- 172.21.15.7:0/3385916236 >> 172.21.15.7:0/3385916236 conn(0x7f4d940fc520 msgr2=0x7f4d940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.814+0000 7f4d9bf98640 1 -- 172.21.15.7:0/3385916236 shutdown_connections 2024-09-21T05:00:45.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.814+0000 7f4d9bf98640 1 -- 172.21.15.7:0/3385916236 wait complete. 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.814+0000 7f4d9bf98640 1 Processor -- start 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.814+0000 7f4d9bf98640 1 -- start start 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.814+0000 7f4d9bf98640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d941008b0 0x7f4d9419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.814+0000 7f4d9bf98640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941011d0 0x7f4d9419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.814+0000 7f4d9bf98640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d941153f0 con 0x7f4d941011d0 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.814+0000 7f4d9bf98640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94115270 con 0x7f4d941008b0 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d99533640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941011d0 0x7f4d9419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d99533640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941011d0 0x7f4d9419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45344/0 (socket says 172.21.15.7:45344) 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d99533640 1 -- 172.21.15.7:0/2970681960 learned_addr learned my addr 172.21.15.7:0/2970681960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:45.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d99d34640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d941008b0 0x7f4d9419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d99533640 1 -- 172.21.15.7:0/2970681960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d941008b0 msgr2=0x7f4d9419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d99533640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d941008b0 0x7f4d9419dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d99533640 1 -- 172.21.15.7:0/2970681960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d941a2880 con 0x7f4d941011d0 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d99533640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941011d0 0x7f4d9419e4f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400e970 tx=0x7f4d8400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d8affd640 1 -- 172.21.15.7:0/2970681960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4d8400cac0 con 0x7f4d941011d0 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d8affd640 1 -- 172.21.15.7:0/2970681960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d840054e0 con 0x7f4d941011d0 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d941a2b70 con 0x7f4d941011d0 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d8affd640 1 -- 172.21.15.7:0/2970681960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4d84004dd0 con 0x7f4d941011d0 2024-09-21T05:00:45.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.815+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d941a3100 con 0x7f4d941011d0 2024-09-21T05:00:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.816+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9406aa90 con 0x7f4d941011d0 2024-09-21T05:00:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.817+0000 7f4d8affd640 1 -- 172.21.15.7:0/2970681960 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4d84005020 con 0x7f4d941011d0 2024-09-21T05:00:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.817+0000 7f4d8affd640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8007aed0 0x7f4d8007d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.818+0000 7f4d8affd640 1 -- 172.21.15.7:0/2970681960 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f4d8409e690 con 0x7f4d941011d0 2024-09-21T05:00:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.818+0000 7f4d99d34640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8007aed0 0x7f4d8007d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:45.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.818+0000 7f4d99d34640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8007aed0 0x7f4d8007d390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4d900096c0 tx=0x7f4d90031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:45.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:45.822+0000 7f4d8affd640 1 -- 172.21.15.7:0/2970681960 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d84069c30 con 0x7f4d941011d0 2024-09-21T05:00:46.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.025+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4d941a3410 con 0x7f4d8007aed0 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.026+0000 7f4d8affd640 1 -- 172.21.15.7:0/2970681960 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f4d941a3410 con 0x7f4d8007aed0 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:00:46.029 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:00:46.030 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:00:46.030 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:00:46.030 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:00:46.030 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:00:46.030 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:00:46.030 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:00:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8007aed0 msgr2=0x7f4d8007d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8007aed0 0x7f4d8007d390 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4d900096c0 tx=0x7f4d90031400 comp rx=0 tx=0).stop 2024-09-21T05:00:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941011d0 msgr2=0x7f4d9419e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941011d0 0x7f4d9419e4f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400e970 tx=0x7f4d8400ee30 comp rx=0 tx=0).stop 2024-09-21T05:00:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 shutdown_connections 2024-09-21T05:00:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8007aed0 0x7f4d8007d390 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d941011d0 0x7f4d9419e4f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 --2- 172.21.15.7:0/2970681960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d941008b0 0x7f4d9419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 >> 172.21.15.7:0/2970681960 conn(0x7f4d940fc520 msgr2=0x7f4d94112ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:46.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 shutdown_connections 2024-09-21T05:00:46.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.028+0000 7f4d9bf98640 1 -- 172.21.15.7:0/2970681960 wait complete. 2024-09-21T05:00:46.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.113+0000 7f0d215e2640 1 -- 172.21.15.7:0/2721262468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c101900 msgr2=0x7f0d1c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:46.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.113+0000 7f0d215e2640 1 --2- 172.21.15.7:0/2721262468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c101900 0x7f0d1c101d60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009880 tx=0x7f0d1002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.113+0000 7f0d215e2640 1 -- 172.21.15.7:0/2721262468 shutdown_connections 2024-09-21T05:00:46.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.113+0000 7f0d215e2640 1 --2- 172.21.15.7:0/2721262468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c101900 0x7f0d1c101d60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.113+0000 7f0d215e2640 1 --2- 172.21.15.7:0/2721262468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d1c107900 0x7f0d1c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.113+0000 7f0d215e2640 1 -- 172.21.15.7:0/2721262468 >> 172.21.15.7:0/2721262468 conn(0x7f0d1c0fd600 msgr2=0x7f0d1c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:46.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.113+0000 7f0d215e2640 1 -- 172.21.15.7:0/2721262468 shutdown_connections 2024-09-21T05:00:46.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.113+0000 7f0d215e2640 1 -- 172.21.15.7:0/2721262468 wait complete. 2024-09-21T05:00:46.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d215e2640 1 Processor -- start 2024-09-21T05:00:46.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d215e2640 1 -- start start 2024-09-21T05:00:46.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d215e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d1c101900 0x7f0d1c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d215e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c107900 0x7f0d1c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d215e2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c10edb0 con 0x7f0d1c107900 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d215e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c10ec30 con 0x7f0d1c101900 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d1a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c107900 0x7f0d1c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d1a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c107900 0x7f0d1c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45366/0 (socket says 172.21.15.7:45366) 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d1a7fc640 1 -- 172.21.15.7:0/2908418873 learned_addr learned my addr 172.21.15.7:0/2908418873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d1affd640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d1c101900 0x7f0d1c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d1a7fc640 1 -- 172.21.15.7:0/2908418873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d1c101900 msgr2=0x7f0d1c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d1a7fc640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d1c101900 0x7f0d1c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d1a7fc640 1 -- 172.21.15.7:0/2908418873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c074f70 con 0x7f0d1c107900 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.114+0000 7f0d1affd640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d1c101900 0x7f0d1c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:00:46.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.115+0000 7f0d1a7fc640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c107900 0x7f0d1c074a30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0d10031460 tx=0x7f0d100045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:46.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.115+0000 7f0d03fff640 1 -- 172.21.15.7:0/2908418873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d1003f070 con 0x7f0d1c107900 2024-09-21T05:00:46.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.115+0000 7f0d03fff640 1 -- 172.21.15.7:0/2908418873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d100035f0 con 0x7f0d1c107900 2024-09-21T05:00:46.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.115+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c075200 con 0x7f0d1c107900 2024-09-21T05:00:46.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.115+0000 7f0d03fff640 1 -- 172.21.15.7:0/2908418873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d10035030 con 0x7f0d1c107900 2024-09-21T05:00:46.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.115+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c1a3dc0 con 0x7f0d1c107900 2024-09-21T05:00:46.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.116+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c103040 con 0x7f0d1c107900 2024-09-21T05:00:46.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.117+0000 7f0d03fff640 1 -- 172.21.15.7:0/2908418873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0d10042070 con 0x7f0d1c107900 2024-09-21T05:00:46.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.117+0000 7f0d03fff640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d0407af80 0x7f0d0407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:00:46.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.117+0000 7f0d03fff640 1 -- 172.21.15.7:0/2908418873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0d100be970 con 0x7f0d1c107900 2024-09-21T05:00:46.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.117+0000 7f0d1affd640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d0407af80 0x7f0d0407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:00:46.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.118+0000 7f0d1affd640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d0407af80 0x7f0d0407d440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0058c0 tx=0x7f0d0c005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:00:46.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.122+0000 7f0d03fff640 1 -- 172.21.15.7:0/2908418873 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0d100c0e20 con 0x7f0d1c107900 2024-09-21T05:00:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.387+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f0d1c109e90 con 0x7f0d1c107900 2024-09-21T05:00:46.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:46 smithi007 ceph-mon[108238]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:46.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:46 smithi007 ceph-mon[108238]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:46.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:46 smithi007 ceph-mon[108238]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:46.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:46 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2204842141' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:00:46.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:46 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3280126720' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:00:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.388+0000 7f0d03fff640 1 -- 172.21.15.7:0/2908418873 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f0d10089f90 con 0x7f0d1c107900 2024-09-21T05:00:46.391 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:00:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:00:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:00:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d0407af80 msgr2=0x7f0d0407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d0407af80 0x7f0d0407d440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0058c0 tx=0x7f0d0c005850 comp rx=0 tx=0).stop 2024-09-21T05:00:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c107900 msgr2=0x7f0d1c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:00:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c107900 0x7f0d1c074a30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0d10031460 tx=0x7f0d100045e0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 shutdown_connections 2024-09-21T05:00:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0d0407af80 0x7f0d0407d440 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c107900 0x7f0d1c074a30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 --2- 172.21.15.7:0/2908418873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d1c101900 0x7f0d1c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:00:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.390+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 >> 172.21.15.7:0/2908418873 conn(0x7f0d1c0fd600 msgr2=0x7f0d1c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:00:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.391+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 shutdown_connections 2024-09-21T05:00:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:00:46.391+0000 7f0d215e2640 1 -- 172.21.15.7:0/2908418873 wait complete. 2024-09-21T05:00:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:46 smithi053 ceph-mon[80176]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:46 smithi053 ceph-mon[80176]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:46 smithi053 ceph-mon[80176]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:46 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2204842141' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:00:46.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:46 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3280126720' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:00:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:47 smithi007 ceph-mon[108238]: pgmap v781: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:47 smithi007 ceph-mon[108238]: from='client.35256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:47 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2908418873' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:00:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:47 smithi053 ceph-mon[80176]: pgmap v781: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:47 smithi053 ceph-mon[80176]: from='client.35256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:00:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:47 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2908418873' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:00:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:49 smithi053 ceph-mon[80176]: pgmap v782: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:49 smithi007 ceph-mon[108238]: pgmap v782: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:51 smithi053 ceph-mon[80176]: pgmap v783: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:51 smithi007 ceph-mon[108238]: pgmap v783: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:53 smithi007 ceph-mon[108238]: pgmap v784: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:53 smithi053 ceph-mon[80176]: pgmap v784: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:55 smithi053 ceph-mon[80176]: pgmap v785: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:55 smithi007 ceph-mon[108238]: pgmap v785: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:57 smithi053 ceph-mon[80176]: pgmap v786: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:57 smithi007 ceph-mon[108238]: pgmap v786: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:59 smithi007 ceph-mon[108238]: pgmap v787: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:00:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:00:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:59 smithi053 ceph-mon[80176]: pgmap v787: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:00:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:00:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:01:01.283 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:01 smithi053 ceph-mon[80176]: pgmap v788: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:01 smithi007 ceph-mon[108238]: pgmap v788: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:03 smithi053 ceph-mon[80176]: pgmap v789: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:03 smithi007 ceph-mon[108238]: pgmap v789: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:05 smithi007 ceph-mon[108238]: pgmap v790: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:05 smithi053 ceph-mon[80176]: pgmap v790: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:07 smithi053 ceph-mon[80176]: pgmap v791: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:07 smithi007 ceph-mon[108238]: pgmap v791: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:09.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:09 smithi053 ceph-mon[80176]: pgmap v792: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:09 smithi007 ceph-mon[108238]: pgmap v792: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:11 smithi007 ceph-mon[108238]: pgmap v793: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:11 smithi053 ceph-mon[80176]: pgmap v793: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:13 smithi053 ceph-mon[80176]: pgmap v794: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:13 smithi007 ceph-mon[108238]: pgmap v794: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:01:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:01:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:15 smithi053 ceph-mon[80176]: pgmap v795: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:15 smithi007 ceph-mon[108238]: pgmap v795: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.488+0000 7fea83081640 1 -- 172.21.15.7:0/2841879623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 msgr2=0x7fea7c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.488+0000 7fea83081640 1 --2- 172.21.15.7:0/2841879623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 0x7fea7c101d60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fea70009880 tx=0x7fea7002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.488+0000 7fea83081640 1 -- 172.21.15.7:0/2841879623 shutdown_connections 2024-09-21T05:01:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.488+0000 7fea83081640 1 --2- 172.21.15.7:0/2841879623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 0x7fea7c101d60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.488+0000 7fea83081640 1 --2- 172.21.15.7:0/2841879623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea7c107900 0x7fea7c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.488+0000 7fea83081640 1 -- 172.21.15.7:0/2841879623 >> 172.21.15.7:0/2841879623 conn(0x7fea7c0fd620 msgr2=0x7fea7c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:16.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.488+0000 7fea83081640 1 -- 172.21.15.7:0/2841879623 shutdown_connections 2024-09-21T05:01:16.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.488+0000 7fea83081640 1 -- 172.21.15.7:0/2841879623 wait complete. 2024-09-21T05:01:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea83081640 1 Processor -- start 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea83081640 1 -- start start 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea83081640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 0x7fea7c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea83081640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea7c107900 0x7fea7c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea83081640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c10ee30 con 0x7fea7c101900 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea83081640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c10ecb0 con 0x7fea7c107900 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea80e1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 0x7fea7c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea80e1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 0x7fea7c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45512/0 (socket says 172.21.15.7:45512) 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.489+0000 7fea80e1d640 1 -- 172.21.15.7:0/3527848842 learned_addr learned my addr 172.21.15.7:0/3527848842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea7bfff640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea7c107900 0x7fea7c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea80e1d640 1 -- 172.21.15.7:0/3527848842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea7c107900 msgr2=0x7fea7c19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea80e1d640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea7c107900 0x7fea7c19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea80e1d640 1 -- 172.21.15.7:0/3527848842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c197840 con 0x7fea7c101900 2024-09-21T05:01:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea80e1d640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 0x7fea7c19d6c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00b420 tx=0x7fea6c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea79ffb640 1 -- 172.21.15.7:0/3527848842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fea6c011630 con 0x7fea7c101900 2024-09-21T05:01:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea79ffb640 1 -- 172.21.15.7:0/3527848842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea6c016070 con 0x7fea7c101900 2024-09-21T05:01:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea79ffb640 1 -- 172.21.15.7:0/3527848842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fea6c010d40 con 0x7fea7c101900 2024-09-21T05:01:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.490+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c197b30 con 0x7fea7c101900 2024-09-21T05:01:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.491+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c198090 con 0x7fea7c101900 2024-09-21T05:01:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.492+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c103040 con 0x7fea7c101900 2024-09-21T05:01:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.492+0000 7fea79ffb640 1 -- 172.21.15.7:0/3527848842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fea6c01b460 con 0x7fea7c101900 2024-09-21T05:01:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.493+0000 7fea79ffb640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea6407af30 0x7fea6407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.493+0000 7fea79ffb640 1 -- 172.21.15.7:0/3527848842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fea6c09d7c0 con 0x7fea7c101900 2024-09-21T05:01:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.493+0000 7fea7bfff640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea6407af30 0x7fea6407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:16.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.494+0000 7fea7bfff640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea6407af30 0x7fea6407d3f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fea7c198b90 tx=0x7fea70031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.498+0000 7fea79ffb640 1 -- 172.21.15.7:0/3527848842 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea6c068cc0 con 0x7fea7c101900 2024-09-21T05:01:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.705+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fea7c1983a0 con 0x7fea6407af30 2024-09-21T05:01:16.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.706+0000 7fea79ffb640 1 -- 172.21.15.7:0/3527848842 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fea7c1983a0 con 0x7fea6407af30 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.708+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea6407af30 msgr2=0x7fea6407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.708+0000 7fea83081640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea6407af30 0x7fea6407d3f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fea7c198b90 tx=0x7fea70031400 comp rx=0 tx=0).stop 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.708+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 msgr2=0x7fea7c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.708+0000 7fea83081640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 0x7fea7c19d6c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00b420 tx=0x7fea6c00b8e0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.709+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 shutdown_connections 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.709+0000 7fea83081640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea6407af30 0x7fea6407d3f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.709+0000 7fea83081640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea7c107900 0x7fea7c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.709+0000 7fea83081640 1 --2- 172.21.15.7:0/3527848842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea7c101900 0x7fea7c19d6c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.709+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 >> 172.21.15.7:0/3527848842 conn(0x7fea7c0fd620 msgr2=0x7fea7c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.709+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 shutdown_connections 2024-09-21T05:01:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.709+0000 7fea83081640 1 -- 172.21.15.7:0/3527848842 wait complete. 2024-09-21T05:01:16.722 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:01:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.794+0000 7f5d726f7640 1 -- 172.21.15.7:0/1233986715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102ac0 msgr2=0x7f5d6c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.794+0000 7f5d726f7640 1 --2- 172.21.15.7:0/1233986715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102ac0 0x7f5d6c102ea0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009880 tx=0x7f5d5c02a530 comp rx=0 tx=0).stop 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.794+0000 7f5d726f7640 1 -- 172.21.15.7:0/1233986715 shutdown_connections 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.794+0000 7f5d726f7640 1 --2- 172.21.15.7:0/1233986715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c1033e0 0x7f5d6c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.794+0000 7f5d726f7640 1 --2- 172.21.15.7:0/1233986715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c102ac0 0x7f5d6c102ea0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.794+0000 7f5d726f7640 1 -- 172.21.15.7:0/1233986715 >> 172.21.15.7:0/1233986715 conn(0x7f5d6c0fc520 msgr2=0x7f5d6c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.794+0000 7f5d726f7640 1 -- 172.21.15.7:0/1233986715 shutdown_connections 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.794+0000 7f5d726f7640 1 -- 172.21.15.7:0/1233986715 wait complete. 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d726f7640 1 Processor -- start 2024-09-21T05:01:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d726f7640 1 -- start start 2024-09-21T05:01:16.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d726f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c102ac0 0x7f5d6c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d726f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c1033e0 0x7f5d6c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d726f7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10edd0 con 0x7f5d6c1033e0 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d726f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10ec50 con 0x7f5d6c102ac0 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d6b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c1033e0 0x7f5d6c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d6b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c1033e0 0x7f5d6c19dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45526/0 (socket says 172.21.15.7:45526) 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d6b7fe640 1 -- 172.21.15.7:0/3374808653 learned_addr learned my addr 172.21.15.7:0/3374808653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.795+0000 7f5d6bfff640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c102ac0 0x7f5d6c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d6b7fe640 1 -- 172.21.15.7:0/3374808653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c102ac0 msgr2=0x7f5d6c19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d6b7fe640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c102ac0 0x7f5d6c19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d6b7fe640 1 -- 172.21.15.7:0/3374808653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c1978e0 con 0x7f5d6c1033e0 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d6bfff640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c102ac0 0x7f5d6c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d6b7fe640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c1033e0 0x7f5d6c19dc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c8c0 tx=0x7f5d6000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d697fa640 1 -- 172.21.15.7:0/3374808653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d60004020 con 0x7f5d6c1033e0 2024-09-21T05:01:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d697fa640 1 -- 172.21.15.7:0/3374808653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d600041c0 con 0x7f5d6c1033e0 2024-09-21T05:01:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c197bd0 con 0x7f5d6c1033e0 2024-09-21T05:01:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d697fa640 1 -- 172.21.15.7:0/3374808653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d600047a0 con 0x7f5d6c1033e0 2024-09-21T05:01:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.796+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c198160 con 0x7f5d6c1033e0 2024-09-21T05:01:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.797+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c06aa90 con 0x7f5d6c1033e0 2024-09-21T05:01:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.798+0000 7f5d697fa640 1 -- 172.21.15.7:0/3374808653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5d600079b0 con 0x7f5d6c1033e0 2024-09-21T05:01:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.799+0000 7f5d697fa640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5d5407af90 0x7f5d5407d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.799+0000 7f5d697fa640 1 -- 172.21.15.7:0/3374808653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f5d6009de60 con 0x7f5d6c1033e0 2024-09-21T05:01:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.799+0000 7f5d6bfff640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5d5407af90 0x7f5d5407d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.799+0000 7f5d6bfff640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5d5407af90 0x7f5d5407d450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c000c00 tx=0x7f5d5c0095a0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:16.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:16.803+0000 7f5d697fa640 1 -- 172.21.15.7:0/3374808653 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d60014030 con 0x7f5d6c1033e0 2024-09-21T05:01:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.013+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5d6c106000 con 0x7f5d5407af90 2024-09-21T05:01:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.014+0000 7f5d697fa640 1 -- 172.21.15.7:0/3374808653 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f5d5c0043a0 con 0x7f5d5407af90 2024-09-21T05:01:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5d5407af90 msgr2=0x7f5d5407d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5d5407af90 0x7f5d5407d450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c000c00 tx=0x7f5d5c0095a0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c1033e0 msgr2=0x7f5d6c19dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c1033e0 0x7f5d6c19dc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c8c0 tx=0x7f5d6000cd80 comp rx=0 tx=0).stop 2024-09-21T05:01:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 shutdown_connections 2024-09-21T05:01:17.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5d5407af90 0x7f5d5407d450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6c1033e0 0x7f5d6c19dc60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 --2- 172.21.15.7:0/3374808653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d6c102ac0 0x7f5d6c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 >> 172.21.15.7:0/3374808653 conn(0x7f5d6c0fc520 msgr2=0x7f5d6c108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 shutdown_connections 2024-09-21T05:01:17.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.017+0000 7f5d726f7640 1 -- 172.21.15.7:0/3374808653 wait complete. 2024-09-21T05:01:17.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.101+0000 7f5fe85fa640 1 -- 172.21.15.7:0/711480349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 msgr2=0x7f5fe0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.101+0000 7f5fe85fa640 1 --2- 172.21.15.7:0/711480349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 0x7f5fe0100c50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009880 tx=0x7f5fd002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.101+0000 7f5fe85fa640 1 -- 172.21.15.7:0/711480349 shutdown_connections 2024-09-21T05:01:17.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.101+0000 7f5fe85fa640 1 --2- 172.21.15.7:0/711480349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 0x7f5fe0100c50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.101+0000 7f5fe85fa640 1 --2- 172.21.15.7:0/711480349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe01067f0 0x7f5fe0106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.101+0000 7f5fe85fa640 1 -- 172.21.15.7:0/711480349 >> 172.21.15.7:0/711480349 conn(0x7f5fe00fc550 msgr2=0x7f5fe00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:17.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.101+0000 7f5fe85fa640 1 -- 172.21.15.7:0/711480349 shutdown_connections 2024-09-21T05:01:17.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.101+0000 7f5fe85fa640 1 -- 172.21.15.7:0/711480349 wait complete. 2024-09-21T05:01:17.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe85fa640 1 Processor -- start 2024-09-21T05:01:17.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe85fa640 1 -- start start 2024-09-21T05:01:17.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe85fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 0x7f5fe019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe85fa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe01067f0 0x7f5fe019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe85fa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe010dcc0 con 0x7f5fe01007f0 2024-09-21T05:01:17.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe85fa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe010db40 con 0x7f5fe01067f0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe6396640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 0x7f5fe019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe6396640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 0x7f5fe019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45544/0 (socket says 172.21.15.7:45544) 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe6396640 1 -- 172.21.15.7:0/1258370372 learned_addr learned my addr 172.21.15.7:0/1258370372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe5b95640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe01067f0 0x7f5fe019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe6396640 1 -- 172.21.15.7:0/1258370372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe01067f0 msgr2=0x7f5fe019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe6396640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe01067f0 0x7f5fe019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe6396640 1 -- 172.21.15.7:0/1258370372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe01977e0 con 0x7f5fe01007f0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.102+0000 7f5fe6396640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 0x7f5fe019d660 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00d6f0 tx=0x7f5fdc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.103+0000 7f5fd77fe640 1 -- 172.21.15.7:0/1258370372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5fdc017070 con 0x7f5fe01007f0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.103+0000 7f5fd77fe640 1 -- 172.21.15.7:0/1258370372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5fdc00de00 con 0x7f5fe01007f0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.103+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe0197ad0 con 0x7f5fe01007f0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.103+0000 7f5fd77fe640 1 -- 172.21.15.7:0/1258370372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5fdc004bb0 con 0x7f5fe01007f0 2024-09-21T05:01:17.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.103+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe0198060 con 0x7f5fe01007f0 2024-09-21T05:01:17.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.104+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe0101f30 con 0x7f5fe01007f0 2024-09-21T05:01:17.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.105+0000 7f5fd77fe640 1 -- 172.21.15.7:0/1258370372 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5fdc009a30 con 0x7f5fe01007f0 2024-09-21T05:01:17.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.105+0000 7f5fd77fe640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5fcc07af80 0x7f5fcc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.105+0000 7f5fd77fe640 1 -- 172.21.15.7:0/1258370372 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f5fdc09dbd0 con 0x7f5fe01007f0 2024-09-21T05:01:17.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.105+0000 7f5fe5b95640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5fcc07af80 0x7f5fcc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.106+0000 7f5fe5b95640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5fcc07af80 0x7f5fcc07d440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0031460 tx=0x7f5fd0009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:17.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.110+0000 7f5fd77fe640 1 -- 172.21.15.7:0/1258370372 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5fdc0691f0 con 0x7f5fe01007f0 2024-09-21T05:01:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.304+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5fe0198310 con 0x7f5fcc07af80 2024-09-21T05:01:17.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.314+0000 7f5fd77fe640 1 -- 172.21.15.7:0/1258370372 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f5fe0198310 con 0x7f5fcc07af80 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (23m) 116s ago 34m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (26m) 116s ago 34m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (26m) 118s ago 33m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (21m) 116s ago 34m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (21m) 118s ago 33m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (22m) 116s ago 34m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (17m) 116s ago 28m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (17m) 116s ago 28m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (2m) 118s ago 28m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:01:17.318 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (17m) 118s ago 28m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (25m) 116s ago 35m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (24m) 118s ago 33m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (21m) 116s ago 35m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (21m) 118s ago 33m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (24m) 116s ago 34m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (24m) 118s ago 33m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (21m) 116s ago 32m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (20m) 116s ago 31m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (19m) 116s ago 31m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (19m) 118s ago 30m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (18m) 118s ago 30m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (18m) 118s ago 29m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:01:17.319 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (23m) 116s ago 33m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:01:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5fcc07af80 msgr2=0x7f5fcc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5fcc07af80 0x7f5fcc07d440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0031460 tx=0x7f5fd0009560 comp rx=0 tx=0).stop 2024-09-21T05:01:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 msgr2=0x7f5fe019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 0x7f5fe019d660 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00d6f0 tx=0x7f5fdc00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 shutdown_connections 2024-09-21T05:01:17.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5fcc07af80 0x7f5fcc07d440 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe01067f0 0x7f5fe019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 --2- 172.21.15.7:0/1258370372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe01007f0 0x7f5fe019d660 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 >> 172.21.15.7:0/1258370372 conn(0x7f5fe00fc550 msgr2=0x7f5fe0069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:17.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 shutdown_connections 2024-09-21T05:01:17.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.316+0000 7f5fe85fa640 1 -- 172.21.15.7:0/1258370372 wait complete. 2024-09-21T05:01:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.398+0000 7f140d3b2640 1 -- 172.21.15.7:0/3099964175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408102b30 msgr2=0x7f1408102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.398+0000 7f140d3b2640 1 --2- 172.21.15.7:0/3099964175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408102b30 0x7f1408102f10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009880 tx=0x7f13f802a530 comp rx=0 tx=0).stop 2024-09-21T05:01:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.399+0000 7f140d3b2640 1 -- 172.21.15.7:0/3099964175 shutdown_connections 2024-09-21T05:01:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.399+0000 7f140d3b2640 1 --2- 172.21.15.7:0/3099964175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1408103450 0x7f140810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.399+0000 7f140d3b2640 1 --2- 172.21.15.7:0/3099964175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408102b30 0x7f1408102f10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.399+0000 7f140d3b2640 1 -- 172.21.15.7:0/3099964175 >> 172.21.15.7:0/3099964175 conn(0x7f14080fc550 msgr2=0x7f14080fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.399+0000 7f140d3b2640 1 -- 172.21.15.7:0/3099964175 shutdown_connections 2024-09-21T05:01:17.404 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:17 smithi007 ceph-mon[108238]: pgmap v796: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.399+0000 7f140d3b2640 1 -- 172.21.15.7:0/3099964175 wait complete. 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.399+0000 7f140d3b2640 1 Processor -- start 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f140d3b2640 1 -- start start 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f140d3b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1408102b30 0x7f1408100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f140d3b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408103450 0x7f1408100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f140d3b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f140810edc0 con 0x7f1408103450 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f140d3b2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f140810ec40 con 0x7f1408102b30 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f14067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408103450 0x7f1408100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f14067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408103450 0x7f1408100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45568/0 (socket says 172.21.15.7:45568) 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f14067fc640 1 -- 172.21.15.7:0/331249818 learned_addr learned my addr 172.21.15.7:0/331249818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f1406ffd640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1408102b30 0x7f1408100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f14067fc640 1 -- 172.21.15.7:0/331249818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1408102b30 msgr2=0x7f1408100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f14067fc640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1408102b30 0x7f1408100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.400+0000 7f14067fc640 1 -- 172.21.15.7:0/331249818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1408101210 con 0x7f1408103450 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.401+0000 7f14067fc640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408103450 0x7f1408100cd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00cc60 tx=0x7f13fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.401+0000 7f13effff640 1 -- 172.21.15.7:0/331249818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13fc00ad30 con 0x7f1408103450 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.401+0000 7f13effff640 1 -- 172.21.15.7:0/331249818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13fc005020 con 0x7f1408103450 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.401+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1408101500 con 0x7f1408103450 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.401+0000 7f13effff640 1 -- 172.21.15.7:0/331249818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13fc00f660 con 0x7f1408103450 2024-09-21T05:01:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.401+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14081a3e30 con 0x7f1408103450 2024-09-21T05:01:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.402+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f140806a500 con 0x7f1408103450 2024-09-21T05:01:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.403+0000 7f13effff640 1 -- 172.21.15.7:0/331249818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f13fc0051d0 con 0x7f1408103450 2024-09-21T05:01:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.404+0000 7f13effff640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13f007af80 0x7f13f007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.404+0000 7f1406ffd640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13f007af80 0x7f13f007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.404+0000 7f13effff640 1 -- 172.21.15.7:0/331249818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f13fc09e4c0 con 0x7f1408103450 2024-09-21T05:01:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.404+0000 7f1406ffd640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13f007af80 0x7f13f007d440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f13f8000c00 tx=0x7f13f8004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.408+0000 7f13effff640 1 -- 172.21.15.7:0/331249818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13fc014030 con 0x7f1408103450 2024-09-21T05:01:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:17 smithi053 ceph-mon[80176]: pgmap v796: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:17.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.675+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f1408105f90 con 0x7f1408103450 2024-09-21T05:01:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.675+0000 7f13effff640 1 -- 172.21.15.7:0/331249818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f13fc069ae0 con 0x7f1408103450 2024-09-21T05:01:17.679 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:01:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:01:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:01:17.680 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13f007af80 msgr2=0x7f13f007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13f007af80 0x7f13f007d440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f13f8000c00 tx=0x7f13f8004f30 comp rx=0 tx=0).stop 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408103450 msgr2=0x7f1408100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408103450 0x7f1408100cd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00cc60 tx=0x7f13fc00a620 comp rx=0 tx=0).stop 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 shutdown_connections 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13f007af80 0x7f13f007d440 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1408103450 0x7f1408100cd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 --2- 172.21.15.7:0/331249818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1408102b30 0x7f1408100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 >> 172.21.15.7:0/331249818 conn(0x7f14080fc550 msgr2=0x7f1408108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 shutdown_connections 2024-09-21T05:01:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.678+0000 7f140d3b2640 1 -- 172.21.15.7:0/331249818 wait complete. 2024-09-21T05:01:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.762+0000 7f96cd1aa640 1 -- 172.21.15.7:0/3394096169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81007f0 msgr2=0x7f96c8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.762+0000 7f96cd1aa640 1 --2- 172.21.15.7:0/3394096169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81007f0 0x7f96c8100c50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009880 tx=0x7f96b002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.762+0000 7f96cd1aa640 1 -- 172.21.15.7:0/3394096169 shutdown_connections 2024-09-21T05:01:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.762+0000 7f96cd1aa640 1 --2- 172.21.15.7:0/3394096169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81007f0 0x7f96c8100c50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.762+0000 7f96cd1aa640 1 --2- 172.21.15.7:0/3394096169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c81067f0 0x7f96c8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.762+0000 7f96cd1aa640 1 -- 172.21.15.7:0/3394096169 >> 172.21.15.7:0/3394096169 conn(0x7f96c80fc550 msgr2=0x7f96c80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.762+0000 7f96cd1aa640 1 -- 172.21.15.7:0/3394096169 shutdown_connections 2024-09-21T05:01:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.762+0000 7f96cd1aa640 1 -- 172.21.15.7:0/3394096169 wait complete. 2024-09-21T05:01:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96cd1aa640 1 Processor -- start 2024-09-21T05:01:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96cd1aa640 1 -- start start 2024-09-21T05:01:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96cd1aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c81007f0 0x7f96c81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96cd1aa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81067f0 0x7f96c8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96cd1aa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96c810dca0 con 0x7f96c81067f0 2024-09-21T05:01:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96cd1aa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f96c810db20 con 0x7f96c81007f0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96be59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81067f0 0x7f96c8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96be59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81067f0 0x7f96c8198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45586/0 (socket says 172.21.15.7:45586) 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96be59c640 1 -- 172.21.15.7:0/1775888028 learned_addr learned my addr 172.21.15.7:0/1775888028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.763+0000 7f96c6d9d640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c81007f0 0x7f96c81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96be59c640 1 -- 172.21.15.7:0/1775888028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c81007f0 msgr2=0x7f96c81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96be59c640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c81007f0 0x7f96c81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96be59c640 1 -- 172.21.15.7:0/1775888028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c8199330 con 0x7f96c81067f0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96be59c640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81067f0 0x7f96c8198d30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f96b0031460 tx=0x7f96b00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96c4d99640 1 -- 172.21.15.7:0/1775888028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f96b003f070 con 0x7f96c81067f0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96c4d99640 1 -- 172.21.15.7:0/1775888028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96b00035f0 con 0x7f96c81067f0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96c4d99640 1 -- 172.21.15.7:0/1775888028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f96b0035030 con 0x7f96c81067f0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c81995c0 con 0x7f96c81067f0 2024-09-21T05:01:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.764+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c81968c0 con 0x7f96c81067f0 2024-09-21T05:01:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.765+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c8101f30 con 0x7f96c81067f0 2024-09-21T05:01:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.766+0000 7f96c4d99640 1 -- 172.21.15.7:0/1775888028 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f96b0042070 con 0x7f96c81067f0 2024-09-21T05:01:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.766+0000 7f96c4d99640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f96b407af30 0x7f96b407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.767+0000 7f96c4d99640 1 -- 172.21.15.7:0/1775888028 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f96b00be940 con 0x7f96c81067f0 2024-09-21T05:01:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.767+0000 7f96c6d9d640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f96b407af30 0x7f96b407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.767+0000 7f96c6d9d640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f96b407af30 0x7f96b407d3f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f96b800bbe0 tx=0x7f96b800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:17.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.771+0000 7f96c4d99640 1 -- 172.21.15.7:0/1775888028 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f96b0089ee0 con 0x7f96c81067f0 2024-09-21T05:01:17.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.995+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f96c8108d60 con 0x7f96c81067f0 2024-09-21T05:01:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.996+0000 7f96c4d99640 1 -- 172.21.15.7:0/1775888028 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f96b008ed90 con 0x7f96c81067f0 2024-09-21T05:01:18.000 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:01:18.000 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:01:18.000 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:01:18.000 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:01:18.000 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:01:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:01:18.000 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:01:18.001 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:01:18.002 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:01:18.003 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f96b407af30 msgr2=0x7f96b407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f96b407af30 0x7f96b407d3f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f96b800bbe0 tx=0x7f96b800b390 comp rx=0 tx=0).stop 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81067f0 msgr2=0x7f96c8198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81067f0 0x7f96c8198d30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f96b0031460 tx=0x7f96b00045e0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 shutdown_connections 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f96b407af30 0x7f96b407d3f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c81067f0 0x7f96c8198d30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 --2- 172.21.15.7:0/1775888028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96c81007f0 0x7f96c81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 >> 172.21.15.7:0/1775888028 conn(0x7f96c80fc550 msgr2=0x7f96c80fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 shutdown_connections 2024-09-21T05:01:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:17.998+0000 7f96cd1aa640 1 -- 172.21.15.7:0/1775888028 wait complete. 2024-09-21T05:01:18.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.083+0000 7f46711a6640 1 -- 172.21.15.7:0/2418541912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c100800 msgr2=0x7f466c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.083+0000 7f46711a6640 1 --2- 172.21.15.7:0/2418541912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c100800 0x7f466c100c60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4660009880 tx=0x7f466002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.083+0000 7f46711a6640 1 -- 172.21.15.7:0/2418541912 shutdown_connections 2024-09-21T05:01:18.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.083+0000 7f46711a6640 1 --2- 172.21.15.7:0/2418541912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c100800 0x7f466c100c60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.083+0000 7f46711a6640 1 --2- 172.21.15.7:0/2418541912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c106800 0x7f466c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.083+0000 7f46711a6640 1 -- 172.21.15.7:0/2418541912 >> 172.21.15.7:0/2418541912 conn(0x7f466c0fc520 msgr2=0x7f466c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:18.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.083+0000 7f46711a6640 1 -- 172.21.15.7:0/2418541912 shutdown_connections 2024-09-21T05:01:18.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.083+0000 7f46711a6640 1 -- 172.21.15.7:0/2418541912 wait complete. 2024-09-21T05:01:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f46711a6640 1 Processor -- start 2024-09-21T05:01:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f46711a6640 1 -- start start 2024-09-21T05:01:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f46711a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c100800 0x7f466c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f46711a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c106800 0x7f466c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f46711a6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f466c10dcd0 con 0x7f466c106800 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f46711a6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f466c10db50 con 0x7f466c100800 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f466a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c106800 0x7f466c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f466a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c106800 0x7f466c19fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45610/0 (socket says 172.21.15.7:45610) 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.084+0000 7f466a59c640 1 -- 172.21.15.7:0/65019624 learned_addr learned my addr 172.21.15.7:0/65019624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f466ad9d640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c100800 0x7f466c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f466a59c640 1 -- 172.21.15.7:0/65019624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c100800 msgr2=0x7f466c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f466a59c640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c100800 0x7f466c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f466a59c640 1 -- 172.21.15.7:0/65019624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c199a00 con 0x7f466c106800 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f466a59c640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c106800 0x7f466c19fd80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4660031460 tx=0x7f46600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f4653fff640 1 -- 172.21.15.7:0/65019624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f466003f070 con 0x7f466c106800 2024-09-21T05:01:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c199c90 con 0x7f466c106800 2024-09-21T05:01:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f4653fff640 1 -- 172.21.15.7:0/65019624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4660004cf0 con 0x7f466c106800 2024-09-21T05:01:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f4653fff640 1 -- 172.21.15.7:0/65019624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4660035030 con 0x7f466c106800 2024-09-21T05:01:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.085+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c19a1c0 con 0x7f466c106800 2024-09-21T05:01:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.086+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c101f40 con 0x7f466c106800 2024-09-21T05:01:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.087+0000 7f4653fff640 1 -- 172.21.15.7:0/65019624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4660042070 con 0x7f466c106800 2024-09-21T05:01:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.087+0000 7f4653fff640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f465407af30 0x7f465407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.088+0000 7f466ad9d640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f465407af30 0x7f465407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.088+0000 7f4653fff640 1 -- 172.21.15.7:0/65019624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f46600bebc0 con 0x7f466c106800 2024-09-21T05:01:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.088+0000 7f466ad9d640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f465407af30 0x7f465407d3f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f465c005fd0 tx=0x7f465c005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:18.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.092+0000 7f4653fff640 1 -- 172.21.15.7:0/65019624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f466008a0c0 con 0x7f466c106800 2024-09-21T05:01:18.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.296+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f466c19afa0 con 0x7f465407af30 2024-09-21T05:01:18.300 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:18 smithi007 ceph-mon[108238]: from='client.35264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:18.300 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:18 smithi007 ceph-mon[108238]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:18.300 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:18 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/331249818' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:01:18.300 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:18 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1775888028' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:01:18.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.297+0000 7f4653fff640 1 -- 172.21.15.7:0/65019624 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f466c19afa0 con 0x7f465407af30 2024-09-21T05:01:18.301 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:01:18.301 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:01:18.301 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:01:18.301 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:01:18.301 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:01:18.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:01:18.301 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:01:18.302 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:01:18.302 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:01:18.302 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:01:18.302 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:01:18.302 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:01:18.302 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:01:18.302 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:01:18.302 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:01:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f465407af30 msgr2=0x7f465407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f465407af30 0x7f465407d3f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f465c005fd0 tx=0x7f465c005e20 comp rx=0 tx=0).stop 2024-09-21T05:01:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c106800 msgr2=0x7f466c19fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c106800 0x7f466c19fd80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4660031460 tx=0x7f46600048f0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 shutdown_connections 2024-09-21T05:01:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f465407af30 0x7f465407d3f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f466c106800 0x7f466c19fd80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 --2- 172.21.15.7:0/65019624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f466c100800 0x7f466c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.299+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 >> 172.21.15.7:0/65019624 conn(0x7f466c0fc520 msgr2=0x7f466c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.300+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 shutdown_connections 2024-09-21T05:01:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.300+0000 7f46711a6640 1 -- 172.21.15.7:0/65019624 wait complete. 2024-09-21T05:01:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.383+0000 7f4aa407b640 1 -- 172.21.15.7:0/825126770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c107900 msgr2=0x7f4a9c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.383+0000 7f4aa407b640 1 --2- 172.21.15.7:0/825126770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c107900 0x7f4a9c107ce0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009880 tx=0x7f4a9802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.383+0000 7f4aa407b640 1 -- 172.21.15.7:0/825126770 shutdown_connections 2024-09-21T05:01:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.383+0000 7f4aa407b640 1 --2- 172.21.15.7:0/825126770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a9c101900 0x7f4a9c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.383+0000 7f4aa407b640 1 --2- 172.21.15.7:0/825126770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c107900 0x7f4a9c107ce0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.383+0000 7f4aa407b640 1 -- 172.21.15.7:0/825126770 >> 172.21.15.7:0/825126770 conn(0x7f4a9c0fd600 msgr2=0x7f4a9c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.383+0000 7f4aa407b640 1 -- 172.21.15.7:0/825126770 shutdown_connections 2024-09-21T05:01:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.383+0000 7f4aa407b640 1 -- 172.21.15.7:0/825126770 wait complete. 2024-09-21T05:01:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa407b640 1 Processor -- start 2024-09-21T05:01:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa407b640 1 -- start start 2024-09-21T05:01:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa407b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c101900 0x7f4a9c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa407b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a9c107900 0x7f4a9c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa407b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c10edb0 con 0x7f4a9c101900 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa407b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c10ec30 con 0x7f4a9c107900 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa1e17640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c101900 0x7f4a9c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa1e17640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c101900 0x7f4a9c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45640/0 (socket says 172.21.15.7:45640) 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.384+0000 7f4aa1e17640 1 -- 172.21.15.7:0/2832794153 learned_addr learned my addr 172.21.15.7:0/2832794153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4aa1616640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a9c107900 0x7f4a9c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4aa1e17640 1 -- 172.21.15.7:0/2832794153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a9c107900 msgr2=0x7f4a9c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4aa1e17640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a9c107900 0x7f4a9c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4aa1e17640 1 -- 172.21.15.7:0/2832794153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c074f70 con 0x7f4a9c101900 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4aa1e17640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c101900 0x7f4a9c0744f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4a98031460 tx=0x7f4a980045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4a92ffd640 1 -- 172.21.15.7:0/2832794153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a9803f070 con 0x7f4a9c101900 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4a92ffd640 1 -- 172.21.15.7:0/2832794153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a980035f0 con 0x7f4a9c101900 2024-09-21T05:01:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c075200 con 0x7f4a9c101900 2024-09-21T05:01:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4a92ffd640 1 -- 172.21.15.7:0/2832794153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a98035030 con 0x7f4a9c101900 2024-09-21T05:01:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.385+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1a3d80 con 0x7f4a9c101900 2024-09-21T05:01:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.386+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c103040 con 0x7f4a9c101900 2024-09-21T05:01:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.387+0000 7f4a92ffd640 1 -- 172.21.15.7:0/2832794153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4a98042070 con 0x7f4a9c101900 2024-09-21T05:01:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.387+0000 7f4a92ffd640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a8807af30 0x7f4a8807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.388+0000 7f4a92ffd640 1 -- 172.21.15.7:0/2832794153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f4a980bea70 con 0x7f4a9c101900 2024-09-21T05:01:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.388+0000 7f4aa1616640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a8807af30 0x7f4a8807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.388+0000 7f4aa1616640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a8807af30 0x7f4a8807d3f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c075f80 tx=0x7f4a8c00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:18.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.392+0000 7f4a92ffd640 1 -- 172.21.15.7:0/2832794153 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4a98089f70 con 0x7f4a9c101900 2024-09-21T05:01:18.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.653+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f4a9c104b10 con 0x7f4a9c101900 2024-09-21T05:01:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.654+0000 7f4a92ffd640 1 -- 172.21.15.7:0/2832794153 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f4a9808ee20 con 0x7f4a9c101900 2024-09-21T05:01:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:01:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:01:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:01:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:01:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:01:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:01:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:01:18.659 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:01:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:01:18.660 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:01:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:01:18.660 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:01:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a8807af30 msgr2=0x7f4a8807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a8807af30 0x7f4a8807d3f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c075f80 tx=0x7f4a8c00b390 comp rx=0 tx=0).stop 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c101900 msgr2=0x7f4a9c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c101900 0x7f4a9c0744f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4a98031460 tx=0x7f4a980045e0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 shutdown_connections 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a8807af30 0x7f4a8807d3f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a9c107900 0x7f4a9c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 --2- 172.21.15.7:0/2832794153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a9c101900 0x7f4a9c0744f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 >> 172.21.15.7:0/2832794153 conn(0x7f4a9c0fd600 msgr2=0x7f4a9c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 shutdown_connections 2024-09-21T05:01:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:18.656+0000 7f4aa407b640 1 -- 172.21.15.7:0/2832794153 wait complete. 2024-09-21T05:01:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:18 smithi053 ceph-mon[80176]: from='client.35264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:18 smithi053 ceph-mon[80176]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:18 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/331249818' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:01:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:18 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1775888028' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:01:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:19 smithi053 ceph-mon[80176]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:19 smithi053 ceph-mon[80176]: pgmap v797: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:19 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2832794153' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:01:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:19 smithi007 ceph-mon[108238]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:19 smithi007 ceph-mon[108238]: pgmap v797: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:19 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2832794153' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:01:20.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:20 smithi053 ceph-mon[80176]: from='client.35284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:20 smithi007 ceph-mon[108238]: from='client.35284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:21.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:21 smithi053 ceph-mon[80176]: pgmap v798: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:21.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:21 smithi007 ceph-mon[108238]: pgmap v798: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:23.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:23 smithi053 ceph-mon[80176]: pgmap v799: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:23 smithi007 ceph-mon[108238]: pgmap v799: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:25.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:25 smithi007 ceph-mon[108238]: pgmap v800: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:25.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:25 smithi053 ceph-mon[80176]: pgmap v800: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:27.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:27 smithi053 ceph-mon[80176]: pgmap v801: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:27 smithi007 ceph-mon[108238]: pgmap v801: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:29 smithi053 ceph-mon[80176]: pgmap v802: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:29.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:01:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:29 smithi007 ceph-mon[108238]: pgmap v802: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:01:31.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:31 smithi053 ceph-mon[80176]: pgmap v803: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:31.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:31 smithi007 ceph-mon[108238]: pgmap v803: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:33.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:33 smithi053 ceph-mon[80176]: pgmap v804: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:33 smithi007 ceph-mon[108238]: pgmap v804: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:35.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:35 smithi007 ceph-mon[108238]: pgmap v805: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:35.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:35 smithi053 ceph-mon[80176]: pgmap v805: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:37 smithi053 ceph-mon[80176]: pgmap v806: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:37 smithi007 ceph-mon[108238]: pgmap v806: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:39.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:39 smithi053 ceph-mon[80176]: pgmap v807: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:39 smithi007 ceph-mon[108238]: pgmap v807: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:41.618 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:41 smithi053 ceph-mon[80176]: pgmap v808: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:41 smithi007 ceph-mon[108238]: pgmap v808: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:43.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:43 smithi053 ceph-mon[80176]: pgmap v809: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:43 smithi007 ceph-mon[108238]: pgmap v809: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:44.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:01:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:01:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:45 smithi007 ceph-mon[108238]: pgmap v810: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:45.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:45 smithi053 ceph-mon[80176]: pgmap v810: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:47.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:47 smithi053 ceph-mon[80176]: pgmap v811: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:47 smithi007 ceph-mon[108238]: pgmap v811: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.755+0000 7fa2013a1640 1 -- 172.21.15.7:0/1407878045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 msgr2=0x7fa1fc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.755+0000 7fa2013a1640 1 --2- 172.21.15.7:0/1407878045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 0x7fa1fc101d60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009880 tx=0x7fa1f002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.756+0000 7fa2013a1640 1 -- 172.21.15.7:0/1407878045 shutdown_connections 2024-09-21T05:01:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.756+0000 7fa2013a1640 1 --2- 172.21.15.7:0/1407878045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 0x7fa1fc101d60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.756+0000 7fa2013a1640 1 --2- 172.21.15.7:0/1407878045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1fc107900 0x7fa1fc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.756+0000 7fa2013a1640 1 -- 172.21.15.7:0/1407878045 >> 172.21.15.7:0/1407878045 conn(0x7fa1fc0fd600 msgr2=0x7fa1fc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.756+0000 7fa2013a1640 1 -- 172.21.15.7:0/1407878045 shutdown_connections 2024-09-21T05:01:48.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.756+0000 7fa2013a1640 1 -- 172.21.15.7:0/1407878045 wait complete. 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.756+0000 7fa2013a1640 1 Processor -- start 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa2013a1640 1 -- start start 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa2013a1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 0x7fa1fc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa2013a1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1fc107900 0x7fa1fc19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa2013a1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc10edb0 con 0x7fa1fc101900 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa2013a1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc10ec30 con 0x7fa1fc107900 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa1faffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 0x7fa1fc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa1faffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 0x7fa1fc19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58006/0 (socket says 172.21.15.7:58006) 2024-09-21T05:01:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa1faffd640 1 -- 172.21.15.7:0/1056304139 learned_addr learned my addr 172.21.15.7:0/1056304139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa1fa7fc640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1fc107900 0x7fa1fc19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa1faffd640 1 -- 172.21.15.7:0/1056304139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1fc107900 msgr2=0x7fa1fc19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa1faffd640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1fc107900 0x7fa1fc19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.757+0000 7fa1faffd640 1 -- 172.21.15.7:0/1056304139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc1977d0 con 0x7fa1fc101900 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.758+0000 7fa1faffd640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 0x7fa1fc19d650 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00d8d0 tx=0x7fa1ec00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.758+0000 7fa1e3fff640 1 -- 172.21.15.7:0/1056304139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa1ec017030 con 0x7fa1fc101900 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.758+0000 7fa1e3fff640 1 -- 172.21.15.7:0/1056304139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1ec004020 con 0x7fa1fc101900 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.758+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc197ac0 con 0x7fa1fc101900 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.758+0000 7fa1e3fff640 1 -- 172.21.15.7:0/1056304139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa1ec0047a0 con 0x7fa1fc101900 2024-09-21T05:01:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.758+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc198020 con 0x7fa1fc101900 2024-09-21T05:01:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.759+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1fc103040 con 0x7fa1fc101900 2024-09-21T05:01:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.759+0000 7fa1e3fff640 1 -- 172.21.15.7:0/1056304139 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa1ec0041d0 con 0x7fa1fc101900 2024-09-21T05:01:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.760+0000 7fa1e3fff640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa1e407af30 0x7fa1e407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.760+0000 7fa1e3fff640 1 -- 172.21.15.7:0/1056304139 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fa1ec018070 con 0x7fa1fc101900 2024-09-21T05:01:48.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.760+0000 7fa1fa7fc640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa1e407af30 0x7fa1e407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:48.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.761+0000 7fa1fa7fc640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa1e407af30 0x7fa1e407d3f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc198b20 tx=0x7fa1f0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:48.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.765+0000 7fa1e3fff640 1 -- 172.21.15.7:0/1056304139 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1ec069420 con 0x7fa1fc101900 2024-09-21T05:01:48.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.970+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa1fc198330 con 0x7fa1e407af30 2024-09-21T05:01:48.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.972+0000 7fa1e3fff640 1 -- 172.21.15.7:0/1056304139 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fa1fc198330 con 0x7fa1e407af30 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa1e407af30 msgr2=0x7fa1e407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa1e407af30 0x7fa1e407d3f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc198b20 tx=0x7fa1f0031400 comp rx=0 tx=0).stop 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 msgr2=0x7fa1fc19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 0x7fa1fc19d650 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00d8d0 tx=0x7fa1ec00dd90 comp rx=0 tx=0).stop 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 shutdown_connections 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa1e407af30 0x7fa1e407d3f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1fc107900 0x7fa1fc19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 --2- 172.21.15.7:0/1056304139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1fc101900 0x7fa1fc19d650 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:48.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 >> 172.21.15.7:0/1056304139 conn(0x7fa1fc0fd600 msgr2=0x7fa1fc0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:48.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 shutdown_connections 2024-09-21T05:01:48.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:48.974+0000 7fa2013a1640 1 -- 172.21.15.7:0/1056304139 wait complete. 2024-09-21T05:01:48.986 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:01:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.060+0000 7fda53760640 1 -- 172.21.15.7:0/3738750183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1067f0 msgr2=0x7fda4c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.060+0000 7fda53760640 1 --2- 172.21.15.7:0/3738750183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1067f0 0x7fda4c106bd0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fda44009880 tx=0x7fda4402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.060+0000 7fda53760640 1 -- 172.21.15.7:0/3738750183 shutdown_connections 2024-09-21T05:01:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.060+0000 7fda53760640 1 --2- 172.21.15.7:0/3738750183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c1007f0 0x7fda4c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.060+0000 7fda53760640 1 --2- 172.21.15.7:0/3738750183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1067f0 0x7fda4c106bd0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.060+0000 7fda53760640 1 -- 172.21.15.7:0/3738750183 >> 172.21.15.7:0/3738750183 conn(0x7fda4c0fc550 msgr2=0x7fda4c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.060+0000 7fda53760640 1 -- 172.21.15.7:0/3738750183 shutdown_connections 2024-09-21T05:01:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.060+0000 7fda53760640 1 -- 172.21.15.7:0/3738750183 wait complete. 2024-09-21T05:01:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda53760640 1 Processor -- start 2024-09-21T05:01:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda53760640 1 -- start start 2024-09-21T05:01:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda53760640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1007f0 0x7fda4c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda53760640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c1067f0 0x7fda4c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda53760640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c10dd20 con 0x7fda4c1007f0 2024-09-21T05:01:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda53760640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c10dba0 con 0x7fda4c1067f0 2024-09-21T05:01:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda514fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1007f0 0x7fda4c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda514fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1007f0 0x7fda4c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58034/0 (socket says 172.21.15.7:58034) 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda514fc640 1 -- 172.21.15.7:0/785202653 learned_addr learned my addr 172.21.15.7:0/785202653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda50cfb640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c1067f0 0x7fda4c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda514fc640 1 -- 172.21.15.7:0/785202653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c1067f0 msgr2=0x7fda4c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda514fc640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c1067f0 0x7fda4c198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda514fc640 1 -- 172.21.15.7:0/785202653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c1994b0 con 0x7fda4c1007f0 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.061+0000 7fda50cfb640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c1067f0 0x7fda4c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.062+0000 7fda514fc640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1007f0 0x7fda4c198970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fda44031460 tx=0x7fda440048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.062+0000 7fda4a7fc640 1 -- 172.21.15.7:0/785202653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda4403f070 con 0x7fda4c1007f0 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.062+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4c199740 con 0x7fda4c1007f0 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.062+0000 7fda4a7fc640 1 -- 172.21.15.7:0/785202653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fda440044d0 con 0x7fda4c1007f0 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.062+0000 7fda4a7fc640 1 -- 172.21.15.7:0/785202653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda44035030 con 0x7fda4c1007f0 2024-09-21T05:01:49.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.062+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4c1a1cf0 con 0x7fda4c1007f0 2024-09-21T05:01:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.063+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4c101f30 con 0x7fda4c1007f0 2024-09-21T05:01:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.064+0000 7fda4a7fc640 1 -- 172.21.15.7:0/785202653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fda44042070 con 0x7fda4c1007f0 2024-09-21T05:01:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.064+0000 7fda4a7fc640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda3407af40 0x7fda3407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.065+0000 7fda50cfb640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda3407af40 0x7fda3407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.065+0000 7fda4a7fc640 1 -- 172.21.15.7:0/785202653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fda440bec60 con 0x7fda4c1007f0 2024-09-21T05:01:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.065+0000 7fda50cfb640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda3407af40 0x7fda3407d400 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fda40006fd0 tx=0x7fda40008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.069+0000 7fda4a7fc640 1 -- 172.21.15.7:0/785202653 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda44004cf0 con 0x7fda4c1007f0 2024-09-21T05:01:49.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.273+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fda4c06a290 con 0x7fda3407af40 2024-09-21T05:01:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.275+0000 7fda4a7fc640 1 -- 172.21.15.7:0/785202653 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fda4c06a290 con 0x7fda3407af40 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda3407af40 msgr2=0x7fda3407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda3407af40 0x7fda3407d400 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fda40006fd0 tx=0x7fda40008040 comp rx=0 tx=0).stop 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1007f0 msgr2=0x7fda4c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1007f0 0x7fda4c198970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fda44031460 tx=0x7fda440048f0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 shutdown_connections 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fda3407af40 0x7fda3407d400 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda4c1067f0 0x7fda4c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 --2- 172.21.15.7:0/785202653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda4c1007f0 0x7fda4c198970 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 >> 172.21.15.7:0/785202653 conn(0x7fda4c0fc550 msgr2=0x7fda4c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 shutdown_connections 2024-09-21T05:01:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.277+0000 7fda53760640 1 -- 172.21.15.7:0/785202653 wait complete. 2024-09-21T05:01:49.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.361+0000 7f41a688f640 1 -- 172.21.15.7:0/2250782867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a01033e0 msgr2=0x7f41a010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.361+0000 7f41a688f640 1 --2- 172.21.15.7:0/2250782867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a01033e0 0x7f41a010b8d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4194009880 tx=0x7f419402a530 comp rx=0 tx=0).stop 2024-09-21T05:01:49.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.361+0000 7f41a688f640 1 -- 172.21.15.7:0/2250782867 shutdown_connections 2024-09-21T05:01:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.361+0000 7f41a688f640 1 --2- 172.21.15.7:0/2250782867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a01033e0 0x7f41a010b8d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.361+0000 7f41a688f640 1 --2- 172.21.15.7:0/2250782867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a0102ac0 0x7f41a0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.361+0000 7f41a688f640 1 -- 172.21.15.7:0/2250782867 >> 172.21.15.7:0/2250782867 conn(0x7f41a00fc520 msgr2=0x7f41a00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:49.365 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:49 smithi007 ceph-mon[108238]: pgmap v812: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.361+0000 7f41a688f640 1 -- 172.21.15.7:0/2250782867 shutdown_connections 2024-09-21T05:01:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.362+0000 7f41a688f640 1 -- 172.21.15.7:0/2250782867 wait complete. 2024-09-21T05:01:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.362+0000 7f41a688f640 1 Processor -- start 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.362+0000 7f41a688f640 1 -- start start 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.362+0000 7f41a688f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a0102ac0 0x7f41a019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.362+0000 7f41a688f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a01033e0 0x7f41a019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.362+0000 7f41a688f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41a010ee50 con 0x7f41a0102ac0 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f41a688f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41a010ecd0 con 0x7f41a01033e0 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a0102ac0 0x7f41a019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a0102ac0 0x7f41a019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58048/0 (socket says 172.21.15.7:58048) 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419ffff640 1 -- 172.21.15.7:0/1784222741 learned_addr learned my addr 172.21.15.7:0/1784222741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419f7fe640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a01033e0 0x7f41a019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419ffff640 1 -- 172.21.15.7:0/1784222741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a01033e0 msgr2=0x7f41a019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419ffff640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a01033e0 0x7f41a019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419ffff640 1 -- 172.21.15.7:0/1784222741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a0197860 con 0x7f41a0102ac0 2024-09-21T05:01:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419f7fe640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a01033e0 0x7f41a019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:01:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419ffff640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a0102ac0 0x7f41a019d6a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f419000b420 tx=0x7f419000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419d7fa640 1 -- 172.21.15.7:0/1784222741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4190011630 con 0x7f41a0102ac0 2024-09-21T05:01:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a0197b50 con 0x7f41a0102ac0 2024-09-21T05:01:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419d7fa640 1 -- 172.21.15.7:0/1784222741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4190016070 con 0x7f41a0102ac0 2024-09-21T05:01:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f419d7fa640 1 -- 172.21.15.7:0/1784222741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4190010d40 con 0x7f41a0102ac0 2024-09-21T05:01:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.363+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a01980e0 con 0x7f41a0102ac0 2024-09-21T05:01:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.364+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a006aa90 con 0x7f41a0102ac0 2024-09-21T05:01:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.365+0000 7f419d7fa640 1 -- 172.21.15.7:0/1784222741 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f419001b460 con 0x7f41a0102ac0 2024-09-21T05:01:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.366+0000 7f419d7fa640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418807f340 0x7f4188081800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.366+0000 7f419d7fa640 1 -- 172.21.15.7:0/1784222741 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f419009d7a0 con 0x7f41a0102ac0 2024-09-21T05:01:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.366+0000 7f419f7fe640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418807f340 0x7f4188081800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.366+0000 7f419f7fe640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418807f340 0x7f4188081800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4194031460 tx=0x7f41940095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:49.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.370+0000 7f419d7fa640 1 -- 172.21.15.7:0/1784222741 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4190068d40 con 0x7f41a0102ac0 2024-09-21T05:01:49.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.567+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f41a0198d90 con 0x7f418807f340 2024-09-21T05:01:49.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.577+0000 7f419d7fa640 1 -- 172.21.15.7:0/1784222741 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f41a0198d90 con 0x7f418807f340 2024-09-21T05:01:49.580 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:01:49.580 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (23m) 2m ago 34m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:01:49.580 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (26m) 2m ago 35m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (26m) 2m ago 34m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (22m) 2m ago 34m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (21m) 2m ago 34m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (23m) 2m ago 34m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (17m) 2m ago 29m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (18m) 2m ago 29m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (2m) 2m ago 29m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (17m) 2m ago 29m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (26m) 2m ago 36m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (25m) 2m ago 33m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (22m) 2m ago 36m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (22m) 2m ago 33m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (24m) 2m ago 34m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:01:49.581 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (24m) 2m ago 34m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:01:49.582 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (21m) 2m ago 32m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:01:49.582 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (20m) 2m ago 32m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:01:49.582 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (20m) 2m ago 31m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:01:49.582 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (19m) 2m ago 31m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:01:49.582 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (19m) 2m ago 30m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:01:49.582 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (18m) 2m ago 30m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:01:49.582 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (24m) 2m ago 34m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.579+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418807f340 msgr2=0x7f4188081800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.579+0000 7f41a688f640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418807f340 0x7f4188081800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4194031460 tx=0x7f41940095f0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.579+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a0102ac0 msgr2=0x7f41a019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.579+0000 7f41a688f640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a0102ac0 0x7f41a019d6a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f419000b420 tx=0x7f419000b8e0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.580+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 shutdown_connections 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.580+0000 7f41a688f640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f418807f340 0x7f4188081800 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.580+0000 7f41a688f640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41a01033e0 0x7f41a019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.580+0000 7f41a688f640 1 --2- 172.21.15.7:0/1784222741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41a0102ac0 0x7f41a019d6a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.580+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 >> 172.21.15.7:0/1784222741 conn(0x7f41a00fc520 msgr2=0x7f41a0108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.580+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 shutdown_connections 2024-09-21T05:01:49.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.580+0000 7f41a688f640 1 -- 172.21.15.7:0/1784222741 wait complete. 2024-09-21T05:01:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.663+0000 7fe4a639f640 1 -- 172.21.15.7:0/660562338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 msgr2=0x7fe4a0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.663+0000 7fe4a639f640 1 --2- 172.21.15.7:0/660562338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 0x7fe4a0107ce0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe490009880 tx=0x7fe49002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.663+0000 7fe4a639f640 1 -- 172.21.15.7:0/660562338 shutdown_connections 2024-09-21T05:01:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.663+0000 7fe4a639f640 1 --2- 172.21.15.7:0/660562338 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4a0101900 0x7fe4a0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.663+0000 7fe4a639f640 1 --2- 172.21.15.7:0/660562338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 0x7fe4a0107ce0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.663+0000 7fe4a639f640 1 -- 172.21.15.7:0/660562338 >> 172.21.15.7:0/660562338 conn(0x7fe4a00fd600 msgr2=0x7fe4a00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.663+0000 7fe4a639f640 1 -- 172.21.15.7:0/660562338 shutdown_connections 2024-09-21T05:01:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.663+0000 7fe4a639f640 1 -- 172.21.15.7:0/660562338 wait complete. 2024-09-21T05:01:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe4a639f640 1 Processor -- start 2024-09-21T05:01:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe4a639f640 1 -- start start 2024-09-21T05:01:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe4a639f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4a0101900 0x7fe4a019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe4a639f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 0x7fe4a019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe4a639f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a010edb0 con 0x7fe4a0107900 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe4a639f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a010ec30 con 0x7fe4a0101900 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe49f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 0x7fe4a019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe49f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 0x7fe4a019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58060/0 (socket says 172.21.15.7:58060) 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.664+0000 7fe49f7fe640 1 -- 172.21.15.7:0/1408577330 learned_addr learned my addr 172.21.15.7:0/1408577330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe49ffff640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4a0101900 0x7fe4a019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe49f7fe640 1 -- 172.21.15.7:0/1408577330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4a0101900 msgr2=0x7fe4a019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe49f7fe640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4a0101900 0x7fe4a019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe49f7fe640 1 -- 172.21.15.7:0/1408577330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a01977c0 con 0x7fe4a0107900 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe49f7fe640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 0x7fe4a019db80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d8d0 tx=0x7fe49400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe49d7fa640 1 -- 172.21.15.7:0/1408577330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe494017030 con 0x7fe4a0107900 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe49d7fa640 1 -- 172.21.15.7:0/1408577330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe494004020 con 0x7fe4a0107900 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a0197ab0 con 0x7fe4a0107900 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe49d7fa640 1 -- 172.21.15.7:0/1408577330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe4940047a0 con 0x7fe4a0107900 2024-09-21T05:01:49.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.665+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a0198040 con 0x7fe4a0107900 2024-09-21T05:01:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.666+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a0103040 con 0x7fe4a0107900 2024-09-21T05:01:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.667+0000 7fe49d7fa640 1 -- 172.21.15.7:0/1408577330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe4940041d0 con 0x7fe4a0107900 2024-09-21T05:01:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.668+0000 7fe49d7fa640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe48807af30 0x7fe48807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.668+0000 7fe49d7fa640 1 -- 172.21.15.7:0/1408577330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fe494018070 con 0x7fe4a0107900 2024-09-21T05:01:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.668+0000 7fe49ffff640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe48807af30 0x7fe48807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.668+0000 7fe49ffff640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe48807af30 0x7fe48807d3f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe490031460 tx=0x7fe490009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:49.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.673+0000 7fe49d7fa640 1 -- 172.21.15.7:0/1408577330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe494069420 con 0x7fe4a0107900 2024-09-21T05:01:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:49 smithi053 ceph-mon[80176]: pgmap v812: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:49.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.934+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fe4a0104b10 con 0x7fe4a0107900 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.935+0000 7fe49d7fa640 1 -- 172.21.15.7:0/1408577330 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fe49406e2d0 con 0x7fe4a0107900 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:01:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:01:49.939 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe48807af30 msgr2=0x7fe48807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe48807af30 0x7fe48807d3f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe490031460 tx=0x7fe490009560 comp rx=0 tx=0).stop 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 msgr2=0x7fe4a019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 0x7fe4a019db80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d8d0 tx=0x7fe49400dd90 comp rx=0 tx=0).stop 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 shutdown_connections 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe48807af30 0x7fe48807d3f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4a0107900 0x7fe4a019db80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 --2- 172.21.15.7:0/1408577330 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4a0101900 0x7fe4a019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 >> 172.21.15.7:0/1408577330 conn(0x7fe4a00fd600 msgr2=0x7fe4a00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:49.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 shutdown_connections 2024-09-21T05:01:49.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:49.937+0000 7fe4a639f640 1 -- 172.21.15.7:0/1408577330 wait complete. 2024-09-21T05:01:50.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.022+0000 7fdbdc323640 1 -- 172.21.15.7:0/3383409527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd41033e0 msgr2=0x7fdbd410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.022+0000 7fdbdc323640 1 --2- 172.21.15.7:0/3383409527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd41033e0 0x7fdbd410b8d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009880 tx=0x7fdbc402a530 comp rx=0 tx=0).stop 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.022+0000 7fdbdc323640 1 -- 172.21.15.7:0/3383409527 shutdown_connections 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.022+0000 7fdbdc323640 1 --2- 172.21.15.7:0/3383409527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd41033e0 0x7fdbd410b8d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.022+0000 7fdbdc323640 1 --2- 172.21.15.7:0/3383409527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd4102ac0 0x7fdbd4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.022+0000 7fdbdc323640 1 -- 172.21.15.7:0/3383409527 >> 172.21.15.7:0/3383409527 conn(0x7fdbd40fc520 msgr2=0x7fdbd40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.022+0000 7fdbdc323640 1 -- 172.21.15.7:0/3383409527 shutdown_connections 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.022+0000 7fdbdc323640 1 -- 172.21.15.7:0/3383409527 wait complete. 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbdc323640 1 Processor -- start 2024-09-21T05:01:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbdc323640 1 -- start start 2024-09-21T05:01:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbdc323640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4102ac0 0x7fdbd419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbdc323640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd41033e0 0x7fdbd419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbdc323640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd410edf0 con 0x7fdbd4102ac0 2024-09-21T05:01:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbdc323640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd410ec70 con 0x7fdbd41033e0 2024-09-21T05:01:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbda0bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4102ac0 0x7fdbd419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbda0bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4102ac0 0x7fdbd419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58086/0 (socket says 172.21.15.7:58086) 2024-09-21T05:01:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbda0bf640 1 -- 172.21.15.7:0/3124419991 learned_addr learned my addr 172.21.15.7:0/3124419991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.023+0000 7fdbda0bf640 1 -- 172.21.15.7:0/3124419991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd41033e0 msgr2=0x7fdbd419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbd98be640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd41033e0 0x7fdbd419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbda0bf640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd41033e0 0x7fdbd419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbda0bf640 1 -- 172.21.15.7:0/3124419991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd41999e0 con 0x7fdbd4102ac0 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbd98be640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd41033e0 0x7fdbd419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbda0bf640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4102ac0 0x7fdbd419f820 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000c8f0 tx=0x7fdbd000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbcb7fe640 1 -- 172.21.15.7:0/3124419991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdbd00079b0 con 0x7fdbd4102ac0 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd4199cd0 con 0x7fdbd4102ac0 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbcb7fe640 1 -- 172.21.15.7:0/3124419991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbd00054e0 con 0x7fdbd4102ac0 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbcb7fe640 1 -- 172.21.15.7:0/3124419991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdbd0004dd0 con 0x7fdbd4102ac0 2024-09-21T05:01:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.024+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd419a260 con 0x7fdbd4102ac0 2024-09-21T05:01:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.025+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd406aa90 con 0x7fdbd4102ac0 2024-09-21T05:01:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.026+0000 7fdbcb7fe640 1 -- 172.21.15.7:0/3124419991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdbd0005020 con 0x7fdbd4102ac0 2024-09-21T05:01:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.026+0000 7fdbcb7fe640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbc007aed0 0x7fdbc007d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.026+0000 7fdbcb7fe640 1 -- 172.21.15.7:0/3124419991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fdbd009e040 con 0x7fdbd4102ac0 2024-09-21T05:01:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.026+0000 7fdbd98be640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbc007aed0 0x7fdbc007d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.027+0000 7fdbd98be640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbc007aed0 0x7fdbc007d390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4031460 tx=0x7fdbc40095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.031+0000 7fdbcb7fe640 1 -- 172.21.15.7:0/3124419991 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdbd0014030 con 0x7fdbd4102ac0 2024-09-21T05:01:50.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.263+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fdbd419af10 con 0x7fdbd4102ac0 2024-09-21T05:01:50.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.263+0000 7fdbcb7fe640 1 -- 172.21.15.7:0/3124419991 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7fdbd00695e0 con 0x7fdbd4102ac0 2024-09-21T05:01:50.266 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:01:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:01:50.268 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:01:50.269 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:01:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:01:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbc007aed0 msgr2=0x7fdbc007d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbc007aed0 0x7fdbc007d390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4031460 tx=0x7fdbc40095f0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4102ac0 msgr2=0x7fdbd419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4102ac0 0x7fdbd419f820 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000c8f0 tx=0x7fdbd000cdb0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 shutdown_connections 2024-09-21T05:01:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbc007aed0 0x7fdbc007d390 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbd41033e0 0x7fdbd419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 --2- 172.21.15.7:0/3124419991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4102ac0 0x7fdbd419f820 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 >> 172.21.15.7:0/3124419991 conn(0x7fdbd40fc520 msgr2=0x7fdbd4100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 shutdown_connections 2024-09-21T05:01:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.265+0000 7fdbdc323640 1 -- 172.21.15.7:0/3124419991 wait complete. 2024-09-21T05:01:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.351+0000 7f06fe14f640 1 -- 172.21.15.7:0/1010328102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 msgr2=0x7f06f8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.351+0000 7f06fe14f640 1 --2- 172.21.15.7:0/1010328102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 0x7f06f8106be0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009880 tx=0x7f06e802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.352+0000 7f06fe14f640 1 -- 172.21.15.7:0/1010328102 shutdown_connections 2024-09-21T05:01:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.352+0000 7f06fe14f640 1 --2- 172.21.15.7:0/1010328102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06f8100800 0x7f06f8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.352+0000 7f06fe14f640 1 --2- 172.21.15.7:0/1010328102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 0x7f06f8106be0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.352+0000 7f06fe14f640 1 -- 172.21.15.7:0/1010328102 >> 172.21.15.7:0/1010328102 conn(0x7f06f80fc520 msgr2=0x7f06f80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.352+0000 7f06fe14f640 1 -- 172.21.15.7:0/1010328102 shutdown_connections 2024-09-21T05:01:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.352+0000 7f06fe14f640 1 -- 172.21.15.7:0/1010328102 wait complete. 2024-09-21T05:01:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.352+0000 7f06fe14f640 1 Processor -- start 2024-09-21T05:01:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.352+0000 7f06fe14f640 1 -- start start 2024-09-21T05:01:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06fe14f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06f8100800 0x7f06f819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06fe14f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 0x7f06f819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06fe14f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06f810dcd0 con 0x7f06f8106800 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06fe14f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f06f810db50 con 0x7f06f8100800 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 0x7f06f819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 0x7f06f819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58116/0 (socket says 172.21.15.7:58116) 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f6ffd640 1 -- 172.21.15.7:0/502422054 learned_addr learned my addr 172.21.15.7:0/502422054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f77fe640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06f8100800 0x7f06f819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f6ffd640 1 -- 172.21.15.7:0/502422054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06f8100800 msgr2=0x7f06f819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f6ffd640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06f8100800 0x7f06f819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f6ffd640 1 -- 172.21.15.7:0/502422054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f8199a20 con 0x7f06f8106800 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f6ffd640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 0x7f06f819fda0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00d6f0 tx=0x7f06ec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f4ff9640 1 -- 172.21.15.7:0/502422054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f06ec017070 con 0x7f06f8106800 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.353+0000 7f06f4ff9640 1 -- 172.21.15.7:0/502422054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f06ec00de00 con 0x7f06f8106800 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.354+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f8199d10 con 0x7f06f8106800 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.354+0000 7f06f4ff9640 1 -- 172.21.15.7:0/502422054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f06ec004bb0 con 0x7f06f8106800 2024-09-21T05:01:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.354+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f819a2a0 con 0x7f06f8106800 2024-09-21T05:01:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.355+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f8101f40 con 0x7f06f8106800 2024-09-21T05:01:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.355+0000 7f06f4ff9640 1 -- 172.21.15.7:0/502422054 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f06ec004e60 con 0x7f06f8106800 2024-09-21T05:01:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.356+0000 7f06f4ff9640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f06e007af30 0x7f06e007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.356+0000 7f06f77fe640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f06e007af30 0x7f06e007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.356+0000 7f06f4ff9640 1 -- 172.21.15.7:0/502422054 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f06ec09dbf0 con 0x7f06f8106800 2024-09-21T05:01:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.356+0000 7f06f77fe640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f06e007af30 0x7f06e007d3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000c00 tx=0x7f06e8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:50.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.360+0000 7f06f4ff9640 1 -- 172.21.15.7:0/502422054 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f06ec069190 con 0x7f06f8106800 2024-09-21T05:01:50.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.564+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f06f819af50 con 0x7f06e007af30 2024-09-21T05:01:50.568 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:50 smithi007 ceph-mon[108238]: from='client.35292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:50.568 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:50 smithi007 ceph-mon[108238]: from='client.35296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:50.568 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:50 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1408577330' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:01:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.566+0000 7f06f4ff9640 1 -- 172.21.15.7:0/502422054 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f06f819af50 con 0x7f06e007af30 2024-09-21T05:01:50.568 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:01:50.569 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:01:50.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f06e007af30 msgr2=0x7f06e007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f06e007af30 0x7f06e007d3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000c00 tx=0x7f06e8031400 comp rx=0 tx=0).stop 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 msgr2=0x7f06f819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 0x7f06f819fda0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00d6f0 tx=0x7f06ec00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 shutdown_connections 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f06e007af30 0x7f06e007d3f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06f8106800 0x7f06f819fda0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 --2- 172.21.15.7:0/502422054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06f8100800 0x7f06f819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 >> 172.21.15.7:0/502422054 conn(0x7f06f80fc520 msgr2=0x7f06f80fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.568+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 shutdown_connections 2024-09-21T05:01:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.569+0000 7f06fe14f640 1 -- 172.21.15.7:0/502422054 wait complete. 2024-09-21T05:01:50.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.652+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/1656605974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 msgr2=0x7f3b08100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.652+0000 7f3b0fcf1640 1 --2- 172.21.15.7:0/1656605974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 0x7f3b08100a30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009c20 tx=0x7f3af802a960 comp rx=0 tx=0).stop 2024-09-21T05:01:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.652+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/1656605974 shutdown_connections 2024-09-21T05:01:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.652+0000 7f3b0fcf1640 1 --2- 172.21.15.7:0/1656605974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b08101000 0x7f3b0810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.652+0000 7f3b0fcf1640 1 --2- 172.21.15.7:0/1656605974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 0x7f3b08100a30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.652+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/1656605974 >> 172.21.15.7:0/1656605974 conn(0x7f3b080fc520 msgr2=0x7f3b080fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.652+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/1656605974 shutdown_connections 2024-09-21T05:01:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.652+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/1656605974 wait complete. 2024-09-21T05:01:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0fcf1640 1 Processor -- start 2024-09-21T05:01:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0fcf1640 1 -- start start 2024-09-21T05:01:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0fcf1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 0x7f3b0819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0fcf1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b08101000 0x7f3b0819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0fcf1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08110fb0 con 0x7f3b08100650 2024-09-21T05:01:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0fcf1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08110e30 con 0x7f3b08101000 2024-09-21T05:01:50.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0da8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 0x7f3b0819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0da8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 0x7f3b0819f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58134/0 (socket says 172.21.15.7:58134) 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.653+0000 7f3b0da8d640 1 -- 172.21.15.7:0/2626965636 learned_addr learned my addr 172.21.15.7:0/2626965636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b0d28c640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b08101000 0x7f3b0819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b0da8d640 1 -- 172.21.15.7:0/2626965636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b08101000 msgr2=0x7f3b0819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b0da8d640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b08101000 0x7f3b0819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b0da8d640 1 -- 172.21.15.7:0/2626965636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b08199a40 con 0x7f3b08100650 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b0da8d640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 0x7f3b0819f880 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3af802ae80 tx=0x7f3af802fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b06ffd640 1 -- 172.21.15.7:0/2626965636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3af8034030 con 0x7f3b08100650 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b06ffd640 1 -- 172.21.15.7:0/2626965636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3af8003e20 con 0x7f3b08100650 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b08199cd0 con 0x7f3b08100650 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b06ffd640 1 -- 172.21.15.7:0/2626965636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3af80046c0 con 0x7f3b08100650 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.654+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0819a260 con 0x7f3b08100650 2024-09-21T05:01:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.655+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0806aa90 con 0x7f3b08100650 2024-09-21T05:01:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.656+0000 7f3b06ffd640 1 -- 172.21.15.7:0/2626965636 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3af8037020 con 0x7f3b08100650 2024-09-21T05:01:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.656+0000 7f3b06ffd640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3af007af30 0x7f3af007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:01:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.656+0000 7f3b06ffd640 1 -- 172.21.15.7:0/2626965636 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f3af80befc0 con 0x7f3b08100650 2024-09-21T05:01:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.657+0000 7f3b0d28c640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3af007af30 0x7f3af007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:01:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.657+0000 7f3b0d28c640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3af007af30 0x7f3af007d3f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b08102520 tx=0x7f3afc005c60 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:01:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.661+0000 7f3b06ffd640 1 -- 172.21.15.7:0/2626965636 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3af808a560 con 0x7f3b08100650 2024-09-21T05:01:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:50 smithi053 ceph-mon[80176]: from='client.35292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:50 smithi053 ceph-mon[80176]: from='client.35296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:50.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:50 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1408577330' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:01:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.923+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f3b0819af70 con 0x7f3b08100650 2024-09-21T05:01:50.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.923+0000 7f3b06ffd640 1 -- 172.21.15.7:0/2626965636 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f3af808f410 con 0x7f3b08100650 2024-09-21T05:01:50.926 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:01:50.926 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:01:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:01:50.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:01:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:01:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:01:50.928 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:01:50.928 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:01:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:01:50.928 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:01:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:01:50.928 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:01:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3af007af30 msgr2=0x7f3af007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3af007af30 0x7f3af007d3f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b08102520 tx=0x7f3afc005c60 comp rx=0 tx=0).stop 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 msgr2=0x7f3b0819f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 0x7f3b0819f880 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3af802ae80 tx=0x7f3af802fb90 comp rx=0 tx=0).stop 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 shutdown_connections 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3af007af30 0x7f3af007d3f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b08101000 0x7f3b0819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 --2- 172.21.15.7:0/2626965636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b08100650 0x7f3b0819f880 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:01:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 >> 172.21.15.7:0/2626965636 conn(0x7f3b080fc520 msgr2=0x7f3b080fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:01:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 shutdown_connections 2024-09-21T05:01:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:01:50.926+0000 7f3b0fcf1640 1 -- 172.21.15.7:0/2626965636 wait complete. 2024-09-21T05:01:51.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:51 smithi053 ceph-mon[80176]: pgmap v813: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:51.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:51 smithi053 ceph-mon[80176]: from='client.35300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:51.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:51 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3124419991' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:01:51.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:51 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2626965636' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:01:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:51 smithi007 ceph-mon[108238]: pgmap v813: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:51 smithi007 ceph-mon[108238]: from='client.35300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:51 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3124419991' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:01:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:51 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2626965636' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:01:52.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:52 smithi053 ceph-mon[80176]: from='client.35312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:52.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:52 smithi007 ceph-mon[108238]: from='client.35312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:01:53.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:53 smithi053 ceph-mon[80176]: pgmap v814: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:53 smithi007 ceph-mon[108238]: pgmap v814: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:55.496 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:55 smithi007 ceph-mon[108238]: pgmap v815: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:55.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:55 smithi053 ceph-mon[80176]: pgmap v815: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:57.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:57 smithi053 ceph-mon[80176]: pgmap v816: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:57.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:57 smithi007 ceph-mon[108238]: pgmap v816: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:59 smithi053 ceph-mon[80176]: pgmap v817: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:59.760 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:01:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:01:59.761 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:59 smithi007 ceph-mon[108238]: pgmap v817: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:01:59.761 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:01:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:02:01.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:01 smithi053 ceph-mon[80176]: pgmap v818: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:01 smithi007 ceph-mon[108238]: pgmap v818: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:03.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:03 smithi053 ceph-mon[80176]: pgmap v819: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:03.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:03 smithi007 ceph-mon[108238]: pgmap v819: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:05.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:05 smithi007 ceph-mon[108238]: pgmap v820: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:05.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:05 smithi053 ceph-mon[80176]: pgmap v820: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:07.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:07 smithi053 ceph-mon[80176]: pgmap v821: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:07.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:07 smithi007 ceph-mon[108238]: pgmap v821: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:09.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:09 smithi053 ceph-mon[80176]: pgmap v822: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:09.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:09 smithi007 ceph-mon[108238]: pgmap v822: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:10.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:10 smithi053 ceph-mon[80176]: pgmap v823: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:10.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:10 smithi007 ceph-mon[108238]: pgmap v823: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:12.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:12 smithi053 ceph-mon[80176]: pgmap v824: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:12.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:12 smithi007 ceph-mon[108238]: pgmap v824: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:14.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:14 smithi053 ceph-mon[80176]: pgmap v825: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:02:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:14 smithi007 ceph-mon[108238]: pgmap v825: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:02:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:16 smithi053 ceph-mon[80176]: pgmap v826: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:16 smithi007 ceph-mon[108238]: pgmap v826: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:18 smithi053 ceph-mon[80176]: pgmap v827: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:18.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:18 smithi007 ceph-mon[108238]: pgmap v827: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:20.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:20 smithi053 ceph-mon[80176]: pgmap v828: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:20 smithi007 ceph-mon[108238]: pgmap v828: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.020+0000 7f4300482640 1 -- 172.21.15.7:0/4217036788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 msgr2=0x7f42f8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.020+0000 7f4300482640 1 --2- 172.21.15.7:0/4217036788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 0x7f42f8102e90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009880 tx=0x7f42f402a530 comp rx=0 tx=0).stop 2024-09-21T05:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.020+0000 7f4300482640 1 -- 172.21.15.7:0/4217036788 shutdown_connections 2024-09-21T05:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.020+0000 7f4300482640 1 --2- 172.21.15.7:0/4217036788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f81033d0 0x7f42f810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.020+0000 7f4300482640 1 --2- 172.21.15.7:0/4217036788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 0x7f42f8102e90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.020+0000 7f4300482640 1 -- 172.21.15.7:0/4217036788 >> 172.21.15.7:0/4217036788 conn(0x7f42f80fc550 msgr2=0x7f42f80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.021+0000 7f4300482640 1 -- 172.21.15.7:0/4217036788 shutdown_connections 2024-09-21T05:02:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.021+0000 7f4300482640 1 -- 172.21.15.7:0/4217036788 wait complete. 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.021+0000 7f4300482640 1 Processor -- start 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.021+0000 7f4300482640 1 -- start start 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.021+0000 7f4300482640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 0x7f42f819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.021+0000 7f4300482640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f81033d0 0x7f42f819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.021+0000 7f4300482640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42f810edc0 con 0x7f42f8102ab0 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.021+0000 7f4300482640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f42f810ec40 con 0x7f42f81033d0 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42fe21e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 0x7f42f819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42fe21e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 0x7f42f819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56998/0 (socket says 172.21.15.7:56998) 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42fe21e640 1 -- 172.21.15.7:0/467323787 learned_addr learned my addr 172.21.15.7:0/467323787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42fda1d640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f81033d0 0x7f42f819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42fe21e640 1 -- 172.21.15.7:0/467323787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f81033d0 msgr2=0x7f42f819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42fe21e640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f81033d0 0x7f42f819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42fe21e640 1 -- 172.21.15.7:0/467323787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f81977b0 con 0x7f42f8102ab0 2024-09-21T05:02:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42fe21e640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 0x7f42f819d630 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f42f4031460 tx=0x7f42f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42ef7fe640 1 -- 172.21.15.7:0/467323787 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42f403f070 con 0x7f42f8102ab0 2024-09-21T05:02:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42ef7fe640 1 -- 172.21.15.7:0/467323787 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42f40035f0 con 0x7f42f8102ab0 2024-09-21T05:02:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f8197a40 con 0x7f42f8102ab0 2024-09-21T05:02:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.022+0000 7f42ef7fe640 1 -- 172.21.15.7:0/467323787 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42f4035030 con 0x7f42f8102ab0 2024-09-21T05:02:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.023+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f8197f70 con 0x7f42f8102ab0 2024-09-21T05:02:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.024+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f806a580 con 0x7f42f8102ab0 2024-09-21T05:02:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.024+0000 7f42ef7fe640 1 -- 172.21.15.7:0/467323787 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f42f4042070 con 0x7f42f8102ab0 2024-09-21T05:02:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.025+0000 7f42ef7fe640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f42e407af30 0x7f42e407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.025+0000 7f42ef7fe640 1 -- 172.21.15.7:0/467323787 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f42f40be9d0 con 0x7f42f8102ab0 2024-09-21T05:02:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.025+0000 7f42fda1d640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f42e407af30 0x7f42e407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.025+0000 7f42fda1d640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f42e407af30 0x7f42e407d3f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f42f80ff010 tx=0x7f42e800b2e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.029+0000 7f42ef7fe640 1 -- 172.21.15.7:0/467323787 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f42f4089f70 con 0x7f42f8102ab0 2024-09-21T05:02:21.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.234+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f42f8198220 con 0x7f42e407af30 2024-09-21T05:02:21.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.235+0000 7f42ef7fe640 1 -- 172.21.15.7:0/467323787 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f42f8198220 con 0x7f42e407af30 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f42e407af30 msgr2=0x7f42e407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f42e407af30 0x7f42e407d3f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f42f80ff010 tx=0x7f42e800b2e0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 msgr2=0x7f42f819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 0x7f42f819d630 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f42f4031460 tx=0x7f42f40045e0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 shutdown_connections 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f42e407af30 0x7f42e407d3f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f81033d0 0x7f42f819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 --2- 172.21.15.7:0/467323787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42f8102ab0 0x7f42f819d630 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 >> 172.21.15.7:0/467323787 conn(0x7f42f80fc550 msgr2=0x7f42f8108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 shutdown_connections 2024-09-21T05:02:21.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.238+0000 7f4300482640 1 -- 172.21.15.7:0/467323787 wait complete. 2024-09-21T05:02:21.251 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:02:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.323+0000 7f5dfb180640 1 -- 172.21.15.7:0/1043067771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4101900 msgr2=0x7f5df4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.323+0000 7f5dfb180640 1 --2- 172.21.15.7:0/1043067771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4101900 0x7f5df4101d60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009880 tx=0x7f5de402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.323+0000 7f5dfb180640 1 -- 172.21.15.7:0/1043067771 shutdown_connections 2024-09-21T05:02:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.323+0000 7f5dfb180640 1 --2- 172.21.15.7:0/1043067771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4101900 0x7f5df4101d60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.323+0000 7f5dfb180640 1 --2- 172.21.15.7:0/1043067771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df4107900 0x7f5df4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.323+0000 7f5dfb180640 1 -- 172.21.15.7:0/1043067771 >> 172.21.15.7:0/1043067771 conn(0x7f5df40fd600 msgr2=0x7f5df40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.323+0000 7f5dfb180640 1 -- 172.21.15.7:0/1043067771 shutdown_connections 2024-09-21T05:02:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.323+0000 7f5dfb180640 1 -- 172.21.15.7:0/1043067771 wait complete. 2024-09-21T05:02:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5dfb180640 1 Processor -- start 2024-09-21T05:02:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5dfb180640 1 -- start start 2024-09-21T05:02:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5dfb180640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df4101900 0x7f5df419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5dfb180640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4107900 0x7f5df419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5dfb180640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5df410edd0 con 0x7f5df4107900 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5dfb180640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5df410ec50 con 0x7f5df4101900 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5debfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4107900 0x7f5df419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5debfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4107900 0x7f5df419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57014/0 (socket says 172.21.15.7:57014) 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5debfff640 1 -- 172.21.15.7:0/858313844 learned_addr learned my addr 172.21.15.7:0/858313844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.324+0000 7f5df8f1c640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df4101900 0x7f5df419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5debfff640 1 -- 172.21.15.7:0/858313844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df4101900 msgr2=0x7f5df419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5debfff640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df4101900 0x7f5df419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5debfff640 1 -- 172.21.15.7:0/858313844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df41999d0 con 0x7f5df4107900 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5debfff640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4107900 0x7f5df419fd50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5de4031460 tx=0x7f5de40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5de9ffb640 1 -- 172.21.15.7:0/858313844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5de403f070 con 0x7f5df4107900 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5de9ffb640 1 -- 172.21.15.7:0/858313844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5de40035f0 con 0x7f5df4107900 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df4199c60 con 0x7f5df4107900 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5de9ffb640 1 -- 172.21.15.7:0/858313844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5de4035030 con 0x7f5df4107900 2024-09-21T05:02:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.325+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df419a190 con 0x7f5df4107900 2024-09-21T05:02:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.327+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df4103040 con 0x7f5df4107900 2024-09-21T05:02:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.327+0000 7f5de9ffb640 1 -- 172.21.15.7:0/858313844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5de4042070 con 0x7f5df4107900 2024-09-21T05:02:21.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.328+0000 7f5de9ffb640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ddc07af30 0x7f5ddc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.328+0000 7f5de9ffb640 1 -- 172.21.15.7:0/858313844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f5de40be9d0 con 0x7f5df4107900 2024-09-21T05:02:21.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.328+0000 7f5df8f1c640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ddc07af30 0x7f5ddc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.329+0000 7f5df8f1c640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ddc07af30 0x7f5ddc07d3f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5df419ab10 tx=0x7f5df000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.332+0000 7f5de9ffb640 1 -- 172.21.15.7:0/858313844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5de4089f70 con 0x7f5df4107900 2024-09-21T05:02:21.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.540+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5df419afa0 con 0x7f5ddc07af30 2024-09-21T05:02:21.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.541+0000 7f5de9ffb640 1 -- 172.21.15.7:0/858313844 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f5df419afa0 con 0x7f5ddc07af30 2024-09-21T05:02:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ddc07af30 msgr2=0x7f5ddc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ddc07af30 0x7f5ddc07d3f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5df419ab10 tx=0x7f5df000b390 comp rx=0 tx=0).stop 2024-09-21T05:02:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4107900 msgr2=0x7f5df419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4107900 0x7f5df419fd50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5de4031460 tx=0x7f5de40045e0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 shutdown_connections 2024-09-21T05:02:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5ddc07af30 0x7f5ddc07d3f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df4107900 0x7f5df419fd50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 --2- 172.21.15.7:0/858313844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5df4101900 0x7f5df419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 >> 172.21.15.7:0/858313844 conn(0x7f5df40fd600 msgr2=0x7f5df4074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 shutdown_connections 2024-09-21T05:02:21.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.543+0000 7f5dfb180640 1 -- 172.21.15.7:0/858313844 wait complete. 2024-09-21T05:02:21.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.628+0000 7f96938e7640 1 -- 172.21.15.7:0/2444542445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 msgr2=0x7f968c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.628+0000 7f96938e7640 1 --2- 172.21.15.7:0/2444542445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 0x7f968c100c60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f967c009880 tx=0x7f967c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.632 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:21 smithi007 ceph-mon[108238]: from='client.35320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.628+0000 7f96938e7640 1 -- 172.21.15.7:0/2444542445 shutdown_connections 2024-09-21T05:02:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.628+0000 7f96938e7640 1 --2- 172.21.15.7:0/2444542445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 0x7f968c100c60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.628+0000 7f96938e7640 1 --2- 172.21.15.7:0/2444542445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c106800 0x7f968c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.628+0000 7f96938e7640 1 -- 172.21.15.7:0/2444542445 >> 172.21.15.7:0/2444542445 conn(0x7f968c0fc520 msgr2=0x7f968c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.628+0000 7f96938e7640 1 -- 172.21.15.7:0/2444542445 shutdown_connections 2024-09-21T05:02:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.628+0000 7f96938e7640 1 -- 172.21.15.7:0/2444542445 wait complete. 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.629+0000 7f96938e7640 1 Processor -- start 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.629+0000 7f96938e7640 1 -- start start 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.629+0000 7f96938e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 0x7f968c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.629+0000 7f96938e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c106800 0x7f968c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.629+0000 7f96938e7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f968c10dcb0 con 0x7f968c100800 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.629+0000 7f96938e7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f968c10db30 con 0x7f968c106800 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f9691683640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 0x7f968c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f9691683640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 0x7f968c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57034/0 (socket says 172.21.15.7:57034) 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f9691683640 1 -- 172.21.15.7:0/563604190 learned_addr learned my addr 172.21.15.7:0/563604190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f9690e82640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c106800 0x7f968c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f9691683640 1 -- 172.21.15.7:0/563604190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c106800 msgr2=0x7f968c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f9691683640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c106800 0x7f968c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f9691683640 1 -- 172.21.15.7:0/563604190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c199340 con 0x7f968c100800 2024-09-21T05:02:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f9691683640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 0x7f968c198800 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f968800d8d0 tx=0x7f968800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f96827fc640 1 -- 172.21.15.7:0/563604190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9688017070 con 0x7f968c100800 2024-09-21T05:02:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f96827fc640 1 -- 172.21.15.7:0/563604190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9688010460 con 0x7f968c100800 2024-09-21T05:02:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c199630 con 0x7f968c100800 2024-09-21T05:02:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f96827fc640 1 -- 172.21.15.7:0/563604190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f968800f5f0 con 0x7f968c100800 2024-09-21T05:02:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.630+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c1a1c30 con 0x7f968c100800 2024-09-21T05:02:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.632+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968c101f40 con 0x7f968c100800 2024-09-21T05:02:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.632+0000 7f96827fc640 1 -- 172.21.15.7:0/563604190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9688009a30 con 0x7f968c100800 2024-09-21T05:02:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.633+0000 7f96827fc640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f967807af30 0x7f967807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.633+0000 7f96827fc640 1 -- 172.21.15.7:0/563604190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f968809de90 con 0x7f968c100800 2024-09-21T05:02:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.633+0000 7f9690e82640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f967807af30 0x7f967807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.634+0000 7f9690e82640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f967807af30 0x7f967807d3f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f968c06ad00 tx=0x7f967c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:21.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.637+0000 7f96827fc640 1 -- 172.21.15.7:0/563604190 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f968c199630 con 0x7f968c100800 2024-09-21T05:02:21.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:21 smithi053 ceph-mon[80176]: from='client.35320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.832+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f968c062630 con 0x7f967807af30 2024-09-21T05:02:21.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.842+0000 7f96827fc640 1 -- 172.21.15.7:0/563604190 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f968c062630 con 0x7f967807af30 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (24m) 3m ago 35m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (27m) 3m ago 35m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (27m) 3m ago 34m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (22m) 3m ago 35m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (22m) 3m ago 34m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (23m) 3m ago 35m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (18m) 3m ago 29m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (18m) 3m ago 30m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (3m) 3m ago 29m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (18m) 3m ago 29m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:02:21.846 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (26m) 3m ago 36m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (25m) 3m ago 34m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (23m) 3m ago 36m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (22m) 3m ago 34m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (25m) 3m ago 35m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (25m) 3m ago 34m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (22m) 3m ago 33m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (21m) 3m ago 32m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (20m) 3m ago 32m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (20m) 3m ago 31m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (19m) 3m ago 31m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (19m) 3m ago 31m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:02:21.847 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (24m) 3m ago 34m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:02:21.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f967807af30 msgr2=0x7f967807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f967807af30 0x7f967807d3f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f968c06ad00 tx=0x7f967c031400 comp rx=0 tx=0).stop 2024-09-21T05:02:21.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 msgr2=0x7f968c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 0x7f968c198800 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f968800d8d0 tx=0x7f968800dd90 comp rx=0 tx=0).stop 2024-09-21T05:02:21.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 shutdown_connections 2024-09-21T05:02:21.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f967807af30 0x7f967807d3f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f968c106800 0x7f968c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 --2- 172.21.15.7:0/563604190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c100800 0x7f968c198800 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 >> 172.21.15.7:0/563604190 conn(0x7f968c0fc520 msgr2=0x7f968c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:21.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 shutdown_connections 2024-09-21T05:02:21.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.844+0000 7f96938e7640 1 -- 172.21.15.7:0/563604190 wait complete. 2024-09-21T05:02:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.928+0000 7f6075e96640 1 -- 172.21.15.7:0/573575852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 msgr2=0x7f607010b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.928+0000 7f6075e96640 1 --2- 172.21.15.7:0/573575852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 0x7f607010b940 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6064009880 tx=0x7f606402a530 comp rx=0 tx=0).stop 2024-09-21T05:02:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.929+0000 7f6075e96640 1 -- 172.21.15.7:0/573575852 shutdown_connections 2024-09-21T05:02:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.929+0000 7f6075e96640 1 --2- 172.21.15.7:0/573575852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 0x7f607010b940 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.929+0000 7f6075e96640 1 --2- 172.21.15.7:0/573575852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6070102b30 0x7f6070102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.929+0000 7f6075e96640 1 -- 172.21.15.7:0/573575852 >> 172.21.15.7:0/573575852 conn(0x7f60700fc550 msgr2=0x7f60700fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:21.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.929+0000 7f6075e96640 1 -- 172.21.15.7:0/573575852 shutdown_connections 2024-09-21T05:02:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.929+0000 7f6075e96640 1 -- 172.21.15.7:0/573575852 wait complete. 2024-09-21T05:02:21.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.929+0000 7f6075e96640 1 Processor -- start 2024-09-21T05:02:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f6075e96640 1 -- start start 2024-09-21T05:02:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f6075e96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6070102b30 0x7f6070100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f6075e96640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 0x7f6070100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f6075e96640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f607010edc0 con 0x7f6070103450 2024-09-21T05:02:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f6075e96640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f607010ec40 con 0x7f6070102b30 2024-09-21T05:02:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f606effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 0x7f6070100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f606effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 0x7f6070100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57054/0 (socket says 172.21.15.7:57054) 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f606effd640 1 -- 172.21.15.7:0/2548922919 learned_addr learned my addr 172.21.15.7:0/2548922919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f606f7fe640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6070102b30 0x7f6070100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f606effd640 1 -- 172.21.15.7:0/2548922919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6070102b30 msgr2=0x7f6070100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f606effd640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6070102b30 0x7f6070100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f606effd640 1 -- 172.21.15.7:0/2548922919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6070101210 con 0x7f6070103450 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.930+0000 7f606f7fe640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6070102b30 0x7f6070100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.931+0000 7f606effd640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 0x7f6070100cd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6064004f90 tx=0x7f60640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.931+0000 7f606cff9640 1 -- 172.21.15.7:0/2548922919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f606403f070 con 0x7f6070103450 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.931+0000 7f606cff9640 1 -- 172.21.15.7:0/2548922919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6064004c70 con 0x7f6070103450 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.931+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60701014a0 con 0x7f6070103450 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.931+0000 7f606cff9640 1 -- 172.21.15.7:0/2548922919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6064035030 con 0x7f6070103450 2024-09-21T05:02:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.931+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60701a3e50 con 0x7f6070103450 2024-09-21T05:02:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.932+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607006a500 con 0x7f6070103450 2024-09-21T05:02:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.933+0000 7f606cff9640 1 -- 172.21.15.7:0/2548922919 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6064042070 con 0x7f6070103450 2024-09-21T05:02:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.933+0000 7f606cff9640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f605807af30 0x7f605807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.933+0000 7f606cff9640 1 -- 172.21.15.7:0/2548922919 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f60640bea90 con 0x7f6070103450 2024-09-21T05:02:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.933+0000 7f606f7fe640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f605807af30 0x7f605807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.934+0000 7f606f7fe640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f605807af30 0x7f605807d3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6060005fd0 tx=0x7f6060009520 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:21.937+0000 7f606cff9640 1 -- 172.21.15.7:0/2548922919 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f606408a030 con 0x7f6070103450 2024-09-21T05:02:22.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.202+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f6070101cc0 con 0x7f6070103450 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.203+0000 7f606cff9640 1 -- 172.21.15.7:0/2548922919 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f606408eee0 con 0x7f6070103450 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:02:22.207 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:02:22.208 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:02:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f605807af30 msgr2=0x7f605807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f605807af30 0x7f605807d3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6060005fd0 tx=0x7f6060009520 comp rx=0 tx=0).stop 2024-09-21T05:02:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 msgr2=0x7f6070100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 0x7f6070100cd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6064004f90 tx=0x7f60640048f0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 shutdown_connections 2024-09-21T05:02:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f605807af30 0x7f605807d3f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6070103450 0x7f6070100cd0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 --2- 172.21.15.7:0/2548922919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6070102b30 0x7f6070100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 >> 172.21.15.7:0/2548922919 conn(0x7f60700fc550 msgr2=0x7f6070107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 shutdown_connections 2024-09-21T05:02:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.205+0000 7f6075e96640 1 -- 172.21.15.7:0/2548922919 wait complete. 2024-09-21T05:02:22.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.289+0000 7ff2d1437640 1 -- 172.21.15.7:0/3719621059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc106800 msgr2=0x7ff2cc106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.289+0000 7ff2d1437640 1 --2- 172.21.15.7:0/3719621059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc106800 0x7ff2cc106be0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009880 tx=0x7ff2bc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.289+0000 7ff2d1437640 1 -- 172.21.15.7:0/3719621059 shutdown_connections 2024-09-21T05:02:22.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.289+0000 7ff2d1437640 1 --2- 172.21.15.7:0/3719621059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2cc100800 0x7ff2cc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.289+0000 7ff2d1437640 1 --2- 172.21.15.7:0/3719621059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc106800 0x7ff2cc106be0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.289+0000 7ff2d1437640 1 -- 172.21.15.7:0/3719621059 >> 172.21.15.7:0/3719621059 conn(0x7ff2cc0fc520 msgr2=0x7ff2cc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:22.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.289+0000 7ff2d1437640 1 -- 172.21.15.7:0/3719621059 shutdown_connections 2024-09-21T05:02:22.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.290+0000 7ff2d1437640 1 -- 172.21.15.7:0/3719621059 wait complete. 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.290+0000 7ff2d1437640 1 Processor -- start 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.290+0000 7ff2d1437640 1 -- start start 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2d1437640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc100800 0x7ff2cc19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2d1437640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2cc106800 0x7ff2cc19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2d1437640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc10dcb0 con 0x7ff2cc100800 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2d1437640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc10db30 con 0x7ff2cc106800 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2caffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc100800 0x7ff2cc19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2caffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc100800 0x7ff2cc19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57078/0 (socket says 172.21.15.7:57078) 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2caffd640 1 -- 172.21.15.7:0/3043987534 learned_addr learned my addr 172.21.15.7:0/3043987534 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2ca7fc640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2cc106800 0x7ff2cc19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2caffd640 1 -- 172.21.15.7:0/3043987534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2cc106800 msgr2=0x7ff2cc19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2caffd640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2cc106800 0x7ff2cc19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2caffd640 1 -- 172.21.15.7:0/3043987534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc197850 con 0x7ff2cc100800 2024-09-21T05:02:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2caffd640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc100800 0x7ff2cc19d6d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc031460 tx=0x7ff2bc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2b3fff640 1 -- 172.21.15.7:0/3043987534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff2bc03f070 con 0x7ff2cc100800 2024-09-21T05:02:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2cc197ae0 con 0x7ff2cc100800 2024-09-21T05:02:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2b3fff640 1 -- 172.21.15.7:0/3043987534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2bc004a90 con 0x7ff2cc100800 2024-09-21T05:02:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2b3fff640 1 -- 172.21.15.7:0/3043987534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff2bc035030 con 0x7ff2cc100800 2024-09-21T05:02:22.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.291+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2cc198010 con 0x7ff2cc100800 2024-09-21T05:02:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.293+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2cc101f40 con 0x7ff2cc100800 2024-09-21T05:02:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.293+0000 7ff2b3fff640 1 -- 172.21.15.7:0/3043987534 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff2bc042070 con 0x7ff2cc100800 2024-09-21T05:02:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.294+0000 7ff2b3fff640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff2b407af40 0x7ff2b407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.294+0000 7ff2b3fff640 1 -- 172.21.15.7:0/3043987534 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff2bc0beab0 con 0x7ff2cc100800 2024-09-21T05:02:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.294+0000 7ff2ca7fc640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff2b407af40 0x7ff2b407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.294+0000 7ff2ca7fc640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff2b407af40 0x7ff2b407d400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0ffbb0 tx=0x7ff2c0005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.298+0000 7ff2b3fff640 1 -- 172.21.15.7:0/3043987534 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2bc08a050 con 0x7ff2cc100800 2024-09-21T05:02:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.523+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff2cc198db0 con 0x7ff2cc100800 2024-09-21T05:02:22.527 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:22 smithi007 ceph-mon[108238]: pgmap v829: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:22.527 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:22 smithi007 ceph-mon[108238]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:22.527 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:22 smithi007 ceph-mon[108238]: from='client.35328 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:22.527 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:22 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2548922919' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:02:22.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.523+0000 7ff2b3fff640 1 -- 172.21.15.7:0/3043987534 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7ff2cc198db0 con 0x7ff2cc100800 2024-09-21T05:02:22.528 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:02:22.528 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:02:22.528 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:02:22.528 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:02:22.528 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:02:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:02:22.528 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:02:22.529 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:02:22.530 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:02:22.531 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff2b407af40 msgr2=0x7ff2b407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff2b407af40 0x7ff2b407d400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0ffbb0 tx=0x7ff2c0005f90 comp rx=0 tx=0).stop 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc100800 msgr2=0x7ff2cc19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc100800 0x7ff2cc19d6d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc031460 tx=0x7ff2bc004530 comp rx=0 tx=0).stop 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 shutdown_connections 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff2b407af40 0x7ff2b407d400 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2cc106800 0x7ff2cc19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 --2- 172.21.15.7:0/3043987534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2cc100800 0x7ff2cc19d6d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 >> 172.21.15.7:0/3043987534 conn(0x7ff2cc0fc520 msgr2=0x7ff2cc06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:22.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 shutdown_connections 2024-09-21T05:02:22.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.526+0000 7ff2d1437640 1 -- 172.21.15.7:0/3043987534 wait complete. 2024-09-21T05:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.611+0000 7f24de170640 1 -- 172.21.15.7:0/3196961209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 msgr2=0x7f24d810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.611+0000 7f24de170640 1 --2- 172.21.15.7:0/3196961209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 0x7f24d810b8d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009880 tx=0x7f24c002a530 comp rx=0 tx=0).stop 2024-09-21T05:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.611+0000 7f24de170640 1 -- 172.21.15.7:0/3196961209 shutdown_connections 2024-09-21T05:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.611+0000 7f24de170640 1 --2- 172.21.15.7:0/3196961209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 0x7f24d810b8d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.611+0000 7f24de170640 1 --2- 172.21.15.7:0/3196961209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d8102ac0 0x7f24d8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.611+0000 7f24de170640 1 -- 172.21.15.7:0/3196961209 >> 172.21.15.7:0/3196961209 conn(0x7f24d80fc520 msgr2=0x7f24d80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.611+0000 7f24de170640 1 -- 172.21.15.7:0/3196961209 shutdown_connections 2024-09-21T05:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.612+0000 7f24de170640 1 -- 172.21.15.7:0/3196961209 wait complete. 2024-09-21T05:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.612+0000 7f24de170640 1 Processor -- start 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.612+0000 7f24de170640 1 -- start start 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.612+0000 7f24de170640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d8102ac0 0x7f24d8100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.612+0000 7f24de170640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 0x7f24d8100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.612+0000 7f24de170640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24d810edf0 con 0x7f24d81033e0 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.612+0000 7f24de170640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24d810ec70 con 0x7f24d8102ac0 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24ceffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 0x7f24d8100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24ceffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 0x7f24d8100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57102/0 (socket says 172.21.15.7:57102) 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24ceffd640 1 -- 172.21.15.7:0/128439747 learned_addr learned my addr 172.21.15.7:0/128439747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24d77fe640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d8102ac0 0x7f24d8100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24ceffd640 1 -- 172.21.15.7:0/128439747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d8102ac0 msgr2=0x7f24d8100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24ceffd640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d8102ac0 0x7f24d8100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24ceffd640 1 -- 172.21.15.7:0/128439747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d81011a0 con 0x7f24d81033e0 2024-09-21T05:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24ceffd640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 0x7f24d8100c60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f24c0031460 tx=0x7f24c0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24d57fa640 1 -- 172.21.15.7:0/128439747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24c003f070 con 0x7f24d81033e0 2024-09-21T05:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d8101430 con 0x7f24d81033e0 2024-09-21T05:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24d57fa640 1 -- 172.21.15.7:0/128439747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f24c00035f0 con 0x7f24d81033e0 2024-09-21T05:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24d57fa640 1 -- 172.21.15.7:0/128439747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24c0035030 con 0x7f24d81033e0 2024-09-21T05:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.613+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d81a3de0 con 0x7f24d81033e0 2024-09-21T05:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.614+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d806aa90 con 0x7f24d81033e0 2024-09-21T05:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.615+0000 7f24d57fa640 1 -- 172.21.15.7:0/128439747 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f24c0042070 con 0x7f24d81033e0 2024-09-21T05:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.616+0000 7f24d57fa640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c407af30 0x7f24c407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.616+0000 7f24d57fa640 1 -- 172.21.15.7:0/128439747 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f24c00be960 con 0x7f24d81033e0 2024-09-21T05:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.616+0000 7f24d77fe640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c407af30 0x7f24c407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.616+0000 7f24d77fe640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c407af30 0x7f24c407d3f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005fd0 tx=0x7f24c800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.620+0000 7f24d57fa640 1 -- 172.21.15.7:0/128439747 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24c0089f00 con 0x7f24d81033e0 2024-09-21T05:02:22.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:22 smithi053 ceph-mon[80176]: pgmap v829: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:22.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:22 smithi053 ceph-mon[80176]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:22.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:22 smithi053 ceph-mon[80176]: from='client.35328 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:22.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:22 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2548922919' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:02:22.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.824+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f24d8062630 con 0x7f24c407af30 2024-09-21T05:02:22.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.825+0000 7f24d57fa640 1 -- 172.21.15.7:0/128439747 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f24d8062630 con 0x7f24c407af30 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:02:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:02:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:02:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:02:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:02:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:02:22.830 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:02:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c407af30 msgr2=0x7f24c407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c407af30 0x7f24c407d3f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005fd0 tx=0x7f24c800b390 comp rx=0 tx=0).stop 2024-09-21T05:02:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 msgr2=0x7f24d8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 0x7f24d8100c60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f24c0031460 tx=0x7f24c0004530 comp rx=0 tx=0).stop 2024-09-21T05:02:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 shutdown_connections 2024-09-21T05:02:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c407af30 0x7f24c407d3f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d81033e0 0x7f24d8100c60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 --2- 172.21.15.7:0/128439747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24d8102ac0 0x7f24d8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 >> 172.21.15.7:0/128439747 conn(0x7f24d80fc520 msgr2=0x7f24d8108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.827+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 shutdown_connections 2024-09-21T05:02:22.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.828+0000 7f24de170640 1 -- 172.21.15.7:0/128439747 wait complete. 2024-09-21T05:02:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.912+0000 7f1027a23640 1 -- 172.21.15.7:0/1161281036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201007f0 msgr2=0x7f1020100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.912+0000 7f1027a23640 1 --2- 172.21.15.7:0/1161281036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201007f0 0x7f1020100c50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1010009880 tx=0x7f101002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.912+0000 7f1027a23640 1 -- 172.21.15.7:0/1161281036 shutdown_connections 2024-09-21T05:02:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.912+0000 7f1027a23640 1 --2- 172.21.15.7:0/1161281036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201007f0 0x7f1020100c50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.912+0000 7f1027a23640 1 --2- 172.21.15.7:0/1161281036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10201067f0 0x7f1020106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.912+0000 7f1027a23640 1 -- 172.21.15.7:0/1161281036 >> 172.21.15.7:0/1161281036 conn(0x7f10200fc550 msgr2=0x7f10200fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.912+0000 7f1027a23640 1 -- 172.21.15.7:0/1161281036 shutdown_connections 2024-09-21T05:02:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.912+0000 7f1027a23640 1 -- 172.21.15.7:0/1161281036 wait complete. 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1027a23640 1 Processor -- start 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1027a23640 1 -- start start 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1027a23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10201007f0 0x7f102019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1027a23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201067f0 0x7f102019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1027a23640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f102010dcc0 con 0x7f10201067f0 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1027a23640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f102010db40 con 0x7f10201007f0 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1024fbe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201067f0 0x7f102019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1024fbe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201067f0 0x7f102019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57128/0 (socket says 172.21.15.7:57128) 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1024fbe640 1 -- 172.21.15.7:0/1310301451 learned_addr learned my addr 172.21.15.7:0/1310301451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f10257bf640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10201007f0 0x7f102019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1024fbe640 1 -- 172.21.15.7:0/1310301451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10201007f0 msgr2=0x7f102019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1024fbe640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10201007f0 0x7f102019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1024fbe640 1 -- 172.21.15.7:0/1310301451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10201977e0 con 0x7f10201067f0 2024-09-21T05:02:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.913+0000 7f1024fbe640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201067f0 0x7f102019dba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1010031460 tx=0x7f10100048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.914+0000 7f10167fc640 1 -- 172.21.15.7:0/1310301451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f101003f070 con 0x7f10201067f0 2024-09-21T05:02:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.914+0000 7f10167fc640 1 -- 172.21.15.7:0/1310301451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1010004cf0 con 0x7f10201067f0 2024-09-21T05:02:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.914+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1020197a70 con 0x7f10201067f0 2024-09-21T05:02:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.914+0000 7f10167fc640 1 -- 172.21.15.7:0/1310301451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1010035030 con 0x7f10201067f0 2024-09-21T05:02:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.914+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1020197fa0 con 0x7f10201067f0 2024-09-21T05:02:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.915+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1020101f30 con 0x7f10201067f0 2024-09-21T05:02:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.915+0000 7f10167fc640 1 -- 172.21.15.7:0/1310301451 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1010042070 con 0x7f10201067f0 2024-09-21T05:02:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.916+0000 7f10167fc640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f100c07af80 0x7f100c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.916+0000 7f10167fc640 1 -- 172.21.15.7:0/1310301451 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f10100bea40 con 0x7f10201067f0 2024-09-21T05:02:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.916+0000 7f10257bf640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f100c07af80 0x7f100c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.917+0000 7f10257bf640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f100c07af80 0x7f100c07d440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f101c0058c0 tx=0x7f101c005c60 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:22.920+0000 7f10167fc640 1 -- 172.21.15.7:0/1310301451 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f101008a060 con 0x7f10201067f0 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.198+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f1020109d70 con 0x7f10201067f0 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.199+0000 7f10167fc640 1 -- 172.21.15.7:0/1310301451 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f101008ef10 con 0x7f10201067f0 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:02:23.203 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:02:23.204 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:02:23.205 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:02:23.205 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:02:23.205 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:02:23.205 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:02:23.205 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f100c07af80 msgr2=0x7f100c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f100c07af80 0x7f100c07d440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f101c0058c0 tx=0x7f101c005c60 comp rx=0 tx=0).stop 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201067f0 msgr2=0x7f102019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201067f0 0x7f102019dba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1010031460 tx=0x7f10100048f0 comp rx=0 tx=0).stop 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 shutdown_connections 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f100c07af80 0x7f100c07d440 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10201067f0 0x7f102019dba0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 --2- 172.21.15.7:0/1310301451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10201007f0 0x7f102019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 >> 172.21.15.7:0/1310301451 conn(0x7f10200fc550 msgr2=0x7f1020069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 shutdown_connections 2024-09-21T05:02:23.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:23.201+0000 7f1027a23640 1 -- 172.21.15.7:0/1310301451 wait complete. 2024-09-21T05:02:23.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3043987534' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:02:23.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:23 smithi053 ceph-mon[80176]: from='client.35340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:23.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1310301451' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:02:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3043987534' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:02:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:23 smithi007 ceph-mon[108238]: from='client.35340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1310301451' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:02:24.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:24 smithi053 ceph-mon[80176]: pgmap v830: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:24.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:24 smithi007 ceph-mon[108238]: pgmap v830: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:26.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:26 smithi053 ceph-mon[80176]: pgmap v831: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:26.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:26 smithi007 ceph-mon[108238]: pgmap v831: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:28 smithi053 ceph-mon[80176]: pgmap v832: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:28.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:28 smithi007 ceph-mon[108238]: pgmap v832: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:29.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:02:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:02:30.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:30 smithi053 ceph-mon[80176]: pgmap v833: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:30.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:30 smithi007 ceph-mon[108238]: pgmap v833: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:32.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:32 smithi053 ceph-mon[80176]: pgmap v834: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:32.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:32 smithi007 ceph-mon[108238]: pgmap v834: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:34.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:34 smithi053 ceph-mon[80176]: pgmap v835: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:34.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:34 smithi007 ceph-mon[108238]: pgmap v835: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:36.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:36 smithi053 ceph-mon[80176]: pgmap v836: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:36.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:36 smithi007 ceph-mon[108238]: pgmap v836: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:38.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:38 smithi053 ceph-mon[80176]: pgmap v837: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:38.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:38 smithi007 ceph-mon[108238]: pgmap v837: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:40.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:40 smithi053 ceph-mon[80176]: pgmap v838: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:40.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:40 smithi007 ceph-mon[108238]: pgmap v838: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:42 smithi007 ceph-mon[108238]: pgmap v839: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:42.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:42 smithi053 ceph-mon[80176]: pgmap v839: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:44.745 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:44 smithi053 ceph-mon[80176]: pgmap v840: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:44.745 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:02:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:44 smithi007 ceph-mon[108238]: pgmap v840: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:02:46.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:46 smithi007 ceph-mon[108238]: pgmap v841: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:46.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:46 smithi053 ceph-mon[80176]: pgmap v841: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:48.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:48 smithi053 ceph-mon[80176]: pgmap v842: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:48.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:48 smithi007 ceph-mon[108238]: pgmap v842: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:50.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:50 smithi053 ceph-mon[80176]: pgmap v843: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:50.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:50 smithi007 ceph-mon[108238]: pgmap v843: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:52.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:52 smithi007 ceph-mon[108238]: pgmap v844: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:52.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:52 smithi053 ceph-mon[80176]: pgmap v844: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.299+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1524824477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 msgr2=0x7f6ac8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.299+0000 7f6acc9f3640 1 --2- 172.21.15.7:0/1524824477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 0x7f6ac8102ea0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009880 tx=0x7f6ab802a530 comp rx=0 tx=0).stop 2024-09-21T05:02:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.299+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1524824477 shutdown_connections 2024-09-21T05:02:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.299+0000 7f6acc9f3640 1 --2- 172.21.15.7:0/1524824477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ac81033e0 0x7f6ac810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.299+0000 7f6acc9f3640 1 --2- 172.21.15.7:0/1524824477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 0x7f6ac8102ea0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.299+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1524824477 >> 172.21.15.7:0/1524824477 conn(0x7f6ac80fc520 msgr2=0x7f6ac80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.299+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1524824477 shutdown_connections 2024-09-21T05:02:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.299+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1524824477 wait complete. 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.300+0000 7f6acc9f3640 1 Processor -- start 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.300+0000 7f6acc9f3640 1 -- start start 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6acc9f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 0x7f6ac819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6acc9f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ac81033e0 0x7f6ac819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6acc9f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac810edf0 con 0x7f6ac8102ac0 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6acc9f3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac810ec70 con 0x7f6ac81033e0 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6ac659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 0x7f6ac819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6ac659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 0x7f6ac819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54362/0 (socket says 172.21.15.7:54362) 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6ac659c640 1 -- 172.21.15.7:0/1456863242 learned_addr learned my addr 172.21.15.7:0/1456863242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6ac5d9b640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ac81033e0 0x7f6ac819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6ac659c640 1 -- 172.21.15.7:0/1456863242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ac81033e0 msgr2=0x7f6ac819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6ac659c640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ac81033e0 0x7f6ac819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6ac659c640 1 -- 172.21.15.7:0/1456863242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac8199a10 con 0x7f6ac8102ac0 2024-09-21T05:02:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.301+0000 7f6ac659c640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 0x7f6ac819f800 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8031460 tx=0x7f6ab80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.302+0000 7f6ab77fe640 1 -- 172.21.15.7:0/1456863242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ab803f070 con 0x7f6ac8102ac0 2024-09-21T05:02:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.302+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac8199ca0 con 0x7f6ac8102ac0 2024-09-21T05:02:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.302+0000 7f6ab77fe640 1 -- 172.21.15.7:0/1456863242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ab80035f0 con 0x7f6ac8102ac0 2024-09-21T05:02:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.302+0000 7f6ab77fe640 1 -- 172.21.15.7:0/1456863242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ab8035030 con 0x7f6ac8102ac0 2024-09-21T05:02:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.302+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac819a1d0 con 0x7f6ac8102ac0 2024-09-21T05:02:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.303+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac806aa90 con 0x7f6ac8102ac0 2024-09-21T05:02:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.303+0000 7f6ab77fe640 1 -- 172.21.15.7:0/1456863242 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6ab8042070 con 0x7f6ac8102ac0 2024-09-21T05:02:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.304+0000 7f6ab77fe640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ab007af30 0x7f6ab007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.304+0000 7f6ac5d9b640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ab007af30 0x7f6ab007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.304+0000 7f6ab77fe640 1 -- 172.21.15.7:0/1456863242 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f6ab80be9d0 con 0x7f6ac8102ac0 2024-09-21T05:02:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.305+0000 7f6ac5d9b640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ab007af30 0x7f6ab007d3f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80fee60 tx=0x7f6abc00b2e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:53.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.309+0000 7f6ab77fe640 1 -- 172.21.15.7:0/1456863242 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ab8089f70 con 0x7f6ac8102ac0 2024-09-21T05:02:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.515+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6ac819aef0 con 0x7f6ab007af30 2024-09-21T05:02:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.516+0000 7f6ab77fe640 1 -- 172.21.15.7:0/1456863242 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f6ac819aef0 con 0x7f6ab007af30 2024-09-21T05:02:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ab007af30 msgr2=0x7f6ab007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ab007af30 0x7f6ab007d3f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80fee60 tx=0x7f6abc00b2e0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 msgr2=0x7f6ac819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 0x7f6ac819f800 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8031460 tx=0x7f6ab80045e0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 shutdown_connections 2024-09-21T05:02:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ab007af30 0x7f6ab007d3f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ac81033e0 0x7f6ac819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 --2- 172.21.15.7:0/1456863242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ac8102ac0 0x7f6ac819f800 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 >> 172.21.15.7:0/1456863242 conn(0x7f6ac80fc520 msgr2=0x7f6ac8100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 shutdown_connections 2024-09-21T05:02:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.518+0000 7f6acc9f3640 1 -- 172.21.15.7:0/1456863242 wait complete. 2024-09-21T05:02:53.530 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:02:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.604+0000 7fddf0980640 1 -- 172.21.15.7:0/154555515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 msgr2=0x7fddec107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.604+0000 7fddf0980640 1 --2- 172.21.15.7:0/154555515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 0x7fddec107ce0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009880 tx=0x7fdddc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.604+0000 7fddf0980640 1 -- 172.21.15.7:0/154555515 shutdown_connections 2024-09-21T05:02:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.604+0000 7fddf0980640 1 --2- 172.21.15.7:0/154555515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec101900 0x7fddec101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.604+0000 7fddf0980640 1 --2- 172.21.15.7:0/154555515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 0x7fddec107ce0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.604+0000 7fddf0980640 1 -- 172.21.15.7:0/154555515 >> 172.21.15.7:0/154555515 conn(0x7fddec0fd600 msgr2=0x7fddec0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.604+0000 7fddf0980640 1 -- 172.21.15.7:0/154555515 shutdown_connections 2024-09-21T05:02:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.604+0000 7fddf0980640 1 -- 172.21.15.7:0/154555515 wait complete. 2024-09-21T05:02:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fddf0980640 1 Processor -- start 2024-09-21T05:02:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fddf0980640 1 -- start start 2024-09-21T05:02:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fddf0980640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec101900 0x7fddec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fddf0980640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 0x7fddec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fddf0980640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fddec10edd0 con 0x7fddec107900 2024-09-21T05:02:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fddf0980640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddec10ec50 con 0x7fddec101900 2024-09-21T05:02:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fdde9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 0x7fddec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fdde9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 0x7fddec074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48368/0 (socket says 172.21.15.7:48368) 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.605+0000 7fdde9d9b640 1 -- 172.21.15.7:0/2571361803 learned_addr learned my addr 172.21.15.7:0/2571361803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fddea59c640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec101900 0x7fddec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fdde9d9b640 1 -- 172.21.15.7:0/2571361803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec101900 msgr2=0x7fddec0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fdde9d9b640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec101900 0x7fddec0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fdde9d9b640 1 -- 172.21.15.7:0/2571361803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec074f70 con 0x7fddec107900 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fddea59c640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec101900 0x7fddec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fdde9d9b640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 0x7fddec074a30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c8f0 tx=0x7fddd400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fdde37fe640 1 -- 172.21.15.7:0/2571361803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fddd40079b0 con 0x7fddec107900 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fdde37fe640 1 -- 172.21.15.7:0/2571361803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fddd40054e0 con 0x7fddec107900 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fdde37fe640 1 -- 172.21.15.7:0/2571361803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fddd4004dd0 con 0x7fddec107900 2024-09-21T05:02:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddec075260 con 0x7fddec107900 2024-09-21T05:02:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.606+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddec1a3e40 con 0x7fddec107900 2024-09-21T05:02:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.607+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddec103040 con 0x7fddec107900 2024-09-21T05:02:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.608+0000 7fdde37fe640 1 -- 172.21.15.7:0/2571361803 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fddd4005020 con 0x7fddec107900 2024-09-21T05:02:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.609+0000 7fdde37fe640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd807aed0 0x7fddd807d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.609+0000 7fdde37fe640 1 -- 172.21.15.7:0/2571361803 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fddd409e020 con 0x7fddec107900 2024-09-21T05:02:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.609+0000 7fddea59c640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd807aed0 0x7fddd807d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.609+0000 7fddea59c640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd807aed0 0x7fddd807d390 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdddc031460 tx=0x7fdddc009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.613+0000 7fdde37fe640 1 -- 172.21.15.7:0/2571361803 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fddd4014030 con 0x7fddec107900 2024-09-21T05:02:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.819+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fddec062630 con 0x7fddd807aed0 2024-09-21T05:02:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.820+0000 7fdde37fe640 1 -- 172.21.15.7:0/2571361803 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fddec062630 con 0x7fddd807aed0 2024-09-21T05:02:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.822+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd807aed0 msgr2=0x7fddd807d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.822+0000 7fddf0980640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd807aed0 0x7fddd807d390 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdddc031460 tx=0x7fdddc009560 comp rx=0 tx=0).stop 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.822+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 msgr2=0x7fddec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.822+0000 7fddf0980640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 0x7fddec074a30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c8f0 tx=0x7fddd400cdb0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.823+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 shutdown_connections 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.823+0000 7fddf0980640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fddd807aed0 0x7fddd807d390 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.823+0000 7fddf0980640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddec107900 0x7fddec074a30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.823+0000 7fddf0980640 1 --2- 172.21.15.7:0/2571361803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec101900 0x7fddec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.823+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 >> 172.21.15.7:0/2571361803 conn(0x7fddec0fd600 msgr2=0x7fddec0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.823+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 shutdown_connections 2024-09-21T05:02:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.823+0000 7fddf0980640 1 -- 172.21.15.7:0/2571361803 wait complete. 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.907+0000 7f563f2c7640 1 -- 172.21.15.7:0/1630629984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638100800 msgr2=0x7f5638100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.907+0000 7f563f2c7640 1 --2- 172.21.15.7:0/1630629984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638100800 0x7f5638100c60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5628009880 tx=0x7f562802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.907+0000 7f563f2c7640 1 -- 172.21.15.7:0/1630629984 shutdown_connections 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.907+0000 7f563f2c7640 1 --2- 172.21.15.7:0/1630629984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638100800 0x7f5638100c60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.907+0000 7f563f2c7640 1 --2- 172.21.15.7:0/1630629984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5638106800 0x7f5638106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.907+0000 7f563f2c7640 1 -- 172.21.15.7:0/1630629984 >> 172.21.15.7:0/1630629984 conn(0x7f56380fc520 msgr2=0x7f56380fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.907+0000 7f563f2c7640 1 -- 172.21.15.7:0/1630629984 shutdown_connections 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.907+0000 7f563f2c7640 1 -- 172.21.15.7:0/1630629984 wait complete. 2024-09-21T05:02:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.908+0000 7f563f2c7640 1 Processor -- start 2024-09-21T05:02:53.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.908+0000 7f563f2c7640 1 -- start start 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.908+0000 7f563f2c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5638100800 0x7f563819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.908+0000 7f563f2c7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638106800 0x7f563819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.908+0000 7f563f2c7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f563810dcd0 con 0x7f5638106800 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.908+0000 7f563f2c7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f563810db50 con 0x7f5638100800 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.908+0000 7f563c862640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638106800 0x7f563819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563c862640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638106800 0x7f563819dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48390/0 (socket says 172.21.15.7:48390) 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563c862640 1 -- 172.21.15.7:0/4000176572 learned_addr learned my addr 172.21.15.7:0/4000176572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563d063640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5638100800 0x7f563819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563c862640 1 -- 172.21.15.7:0/4000176572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5638100800 msgr2=0x7f563819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563c862640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5638100800 0x7f563819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563c862640 1 -- 172.21.15.7:0/4000176572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56381977f0 con 0x7f5638106800 2024-09-21T05:02:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563c862640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638106800 0x7f563819dbb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5628031460 tx=0x7f56280045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f562e7fc640 1 -- 172.21.15.7:0/4000176572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f562803f070 con 0x7f5638106800 2024-09-21T05:02:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f562e7fc640 1 -- 172.21.15.7:0/4000176572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f56280035f0 con 0x7f5638106800 2024-09-21T05:02:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5638197a80 con 0x7f5638106800 2024-09-21T05:02:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f562e7fc640 1 -- 172.21.15.7:0/4000176572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5628035030 con 0x7f5638106800 2024-09-21T05:02:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.909+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5638197fb0 con 0x7f5638106800 2024-09-21T05:02:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.910+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5638101f40 con 0x7f5638106800 2024-09-21T05:02:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.911+0000 7f562e7fc640 1 -- 172.21.15.7:0/4000176572 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5628042070 con 0x7f5638106800 2024-09-21T05:02:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.912+0000 7f562e7fc640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f562407af80 0x7f562407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.912+0000 7f562e7fc640 1 -- 172.21.15.7:0/4000176572 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f56280be8c0 con 0x7f5638106800 2024-09-21T05:02:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.912+0000 7f563d063640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f562407af80 0x7f562407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.912+0000 7f563d063640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f562407af80 0x7f562407d440 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5634005fd0 tx=0x7f563400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:53.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:53.916+0000 7f562e7fc640 1 -- 172.21.15.7:0/4000176572 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5628089ee0 con 0x7f5638106800 2024-09-21T05:02:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.114+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5638198320 con 0x7f562407af80 2024-09-21T05:02:54.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.124+0000 7f562e7fc640 1 -- 172.21.15.7:0/4000176572 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f5638198320 con 0x7f562407af80 2024-09-21T05:02:54.127 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:02:54.127 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (24m) 3m ago 35m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:02:54.127 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (28m) 3m ago 36m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:02:54.127 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (27m) 3m ago 35m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:02:54.127 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (23m) 3m ago 36m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (23m) 3m ago 35m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (24m) 3m ago 35m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (19m) 3m ago 30m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (19m) 3m ago 30m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (3m) 3m ago 30m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (18m) 3m ago 30m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (27m) 3m ago 37m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (26m) 3m ago 35m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (23m) 3m ago 37m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (23m) 3m ago 34m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (25m) 3m ago 35m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (25m) 3m ago 35m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (22m) 3m ago 33m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (22m) 3m ago 33m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (21m) 3m ago 32m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:02:54.128 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (20m) 3m ago 32m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:02:54.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (20m) 3m ago 32m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:02:54.129 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (19m) 3m ago 31m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:02:54.129 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (25m) 3m ago 35m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:02:54.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f562407af80 msgr2=0x7f562407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f562407af80 0x7f562407d440 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5634005fd0 tx=0x7f563400b390 comp rx=0 tx=0).stop 2024-09-21T05:02:54.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638106800 msgr2=0x7f563819dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638106800 0x7f563819dbb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5628031460 tx=0x7f56280045e0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 shutdown_connections 2024-09-21T05:02:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f562407af80 0x7f562407d440 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5638106800 0x7f563819dbb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 --2- 172.21.15.7:0/4000176572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5638100800 0x7f563819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 >> 172.21.15.7:0/4000176572 conn(0x7f56380fc520 msgr2=0x7f563806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.126+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 shutdown_connections 2024-09-21T05:02:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.127+0000 7f563f2c7640 1 -- 172.21.15.7:0/4000176572 wait complete. 2024-09-21T05:02:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.210+0000 7f4233cfa640 1 -- 172.21.15.7:0/3377326766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 msgr2=0x7f422c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.210+0000 7f4233cfa640 1 --2- 172.21.15.7:0/3377326766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 0x7f422c107ce0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4228009880 tx=0x7f422802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.210+0000 7f4233cfa640 1 -- 172.21.15.7:0/3377326766 shutdown_connections 2024-09-21T05:02:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.210+0000 7f4233cfa640 1 --2- 172.21.15.7:0/3377326766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f422c101900 0x7f422c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.210+0000 7f4233cfa640 1 --2- 172.21.15.7:0/3377326766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 0x7f422c107ce0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.210+0000 7f4233cfa640 1 -- 172.21.15.7:0/3377326766 >> 172.21.15.7:0/3377326766 conn(0x7f422c0fd600 msgr2=0x7f422c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.210+0000 7f4233cfa640 1 -- 172.21.15.7:0/3377326766 shutdown_connections 2024-09-21T05:02:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.210+0000 7f4233cfa640 1 -- 172.21.15.7:0/3377326766 wait complete. 2024-09-21T05:02:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4233cfa640 1 Processor -- start 2024-09-21T05:02:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4233cfa640 1 -- start start 2024-09-21T05:02:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4233cfa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f422c101900 0x7f422c19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4233cfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 0x7f422c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4233cfa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10edd0 con 0x7f422c107900 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4233cfa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f422c10ec50 con 0x7f422c101900 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4231295640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 0x7f422c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4231295640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 0x7f422c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48412/0 (socket says 172.21.15.7:48412) 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4231295640 1 -- 172.21.15.7:0/1294642914 learned_addr learned my addr 172.21.15.7:0/1294642914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.211+0000 7f4231a96640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f422c101900 0x7f422c19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4231295640 1 -- 172.21.15.7:0/1294642914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f422c101900 msgr2=0x7f422c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4231295640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f422c101900 0x7f422c19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4231295640 1 -- 172.21.15.7:0/1294642914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c1977a0 con 0x7f422c107900 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4231295640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 0x7f422c19db60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f421c00d6f0 tx=0x7f421c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4222ffd640 1 -- 172.21.15.7:0/1294642914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f421c017070 con 0x7f422c107900 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4222ffd640 1 -- 172.21.15.7:0/1294642914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f421c00de00 con 0x7f422c107900 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4222ffd640 1 -- 172.21.15.7:0/1294642914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f421c004bb0 con 0x7f422c107900 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422c197a90 con 0x7f422c107900 2024-09-21T05:02:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.212+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c198020 con 0x7f422c107900 2024-09-21T05:02:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.213+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422c103040 con 0x7f422c107900 2024-09-21T05:02:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.214+0000 7f4222ffd640 1 -- 172.21.15.7:0/1294642914 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f421c009a30 con 0x7f422c107900 2024-09-21T05:02:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.215+0000 7f4222ffd640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f421807afd0 0x7f421807d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.215+0000 7f4222ffd640 1 -- 172.21.15.7:0/1294642914 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f421c09dbc0 con 0x7f422c107900 2024-09-21T05:02:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.215+0000 7f4231a96640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f421807afd0 0x7f421807d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.215+0000 7f4231a96640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f421807afd0 0x7f421807d490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f42280096c0 tx=0x7f4228031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.219+0000 7f4222ffd640 1 -- 172.21.15.7:0/1294642914 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f421c069140 con 0x7f422c107900 2024-09-21T05:02:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.481+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f422c109e70 con 0x7f422c107900 2024-09-21T05:02:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.481+0000 7f4222ffd640 1 -- 172.21.15.7:0/1294642914 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f421c06dff0 con 0x7f422c107900 2024-09-21T05:02:54.484 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:02:54.484 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:02:54.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:02:54.486 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.483+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f421807afd0 msgr2=0x7f421807d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.483+0000 7f4233cfa640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f421807afd0 0x7f421807d490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f42280096c0 tx=0x7f4228031400 comp rx=0 tx=0).stop 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.483+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 msgr2=0x7f422c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.483+0000 7f4233cfa640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 0x7f422c19db60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f421c00d6f0 tx=0x7f421c00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.484+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 shutdown_connections 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.484+0000 7f4233cfa640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f421807afd0 0x7f421807d490 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.484+0000 7f4233cfa640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f422c107900 0x7f422c19db60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.484+0000 7f4233cfa640 1 --2- 172.21.15.7:0/1294642914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f422c101900 0x7f422c19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.484+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 >> 172.21.15.7:0/1294642914 conn(0x7f422c0fd600 msgr2=0x7f422c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.484+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 shutdown_connections 2024-09-21T05:02:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.484+0000 7f4233cfa640 1 -- 172.21.15.7:0/1294642914 wait complete. 2024-09-21T05:02:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.567+0000 7f5567dc6640 1 -- 172.21.15.7:0/4067115834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560101900 msgr2=0x7f5560101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.567+0000 7f5567dc6640 1 --2- 172.21.15.7:0/4067115834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560101900 0x7f5560101d60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5550009880 tx=0x7f555002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.567+0000 7f5567dc6640 1 -- 172.21.15.7:0/4067115834 shutdown_connections 2024-09-21T05:02:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.567+0000 7f5567dc6640 1 --2- 172.21.15.7:0/4067115834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560101900 0x7f5560101d60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.567+0000 7f5567dc6640 1 --2- 172.21.15.7:0/4067115834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5560107900 0x7f5560107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.567+0000 7f5567dc6640 1 -- 172.21.15.7:0/4067115834 >> 172.21.15.7:0/4067115834 conn(0x7f55600fd600 msgr2=0x7f55600ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.567+0000 7f5567dc6640 1 -- 172.21.15.7:0/4067115834 shutdown_connections 2024-09-21T05:02:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.567+0000 7f5567dc6640 1 -- 172.21.15.7:0/4067115834 wait complete. 2024-09-21T05:02:54.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5567dc6640 1 Processor -- start 2024-09-21T05:02:54.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5567dc6640 1 -- start start 2024-09-21T05:02:54.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5567dc6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5560101900 0x7f556019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5567dc6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560107900 0x7f556019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5567dc6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f556010edb0 con 0x7f5560107900 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5567dc6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f556010ec30 con 0x7f5560101900 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5565361640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560107900 0x7f556019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5565361640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560107900 0x7f556019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48426/0 (socket says 172.21.15.7:48426) 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.568+0000 7f5565361640 1 -- 172.21.15.7:0/3908444816 learned_addr learned my addr 172.21.15.7:0/3908444816 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5565b62640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5560101900 0x7f556019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5565361640 1 -- 172.21.15.7:0/3908444816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5560101900 msgr2=0x7f556019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5565361640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5560101900 0x7f556019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5565361640 1 -- 172.21.15.7:0/3908444816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5560199a10 con 0x7f5560107900 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5565361640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560107900 0x7f556019fd90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5550031460 tx=0x7f55500045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5556ffd640 1 -- 172.21.15.7:0/3908444816 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f555003f070 con 0x7f5560107900 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5556ffd640 1 -- 172.21.15.7:0/3908444816 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f55500035f0 con 0x7f5560107900 2024-09-21T05:02:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5556ffd640 1 -- 172.21.15.7:0/3908444816 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5550035030 con 0x7f5560107900 2024-09-21T05:02:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5560199ca0 con 0x7f5560107900 2024-09-21T05:02:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.569+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556019a1d0 con 0x7f5560107900 2024-09-21T05:02:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.570+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5560103040 con 0x7f5560107900 2024-09-21T05:02:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.571+0000 7f5556ffd640 1 -- 172.21.15.7:0/3908444816 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5550042070 con 0x7f5560107900 2024-09-21T05:02:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.572+0000 7f5556ffd640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f554c07af80 0x7f554c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.572+0000 7f5565b62640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f554c07af80 0x7f554c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.572+0000 7f5556ffd640 1 -- 172.21.15.7:0/3908444816 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f55500be8c0 con 0x7f5560107900 2024-09-21T05:02:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.572+0000 7f5565b62640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f554c07af80 0x7f554c07d440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f556019ab50 tx=0x7f555c00b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.576+0000 7f5556ffd640 1 -- 172.21.15.7:0/3908444816 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5550089ee0 con 0x7f5560107900 2024-09-21T05:02:54.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.802+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f556019afa0 con 0x7f5560107900 2024-09-21T05:02:54.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:54 smithi007 ceph-mon[108238]: pgmap v845: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:54.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:54 smithi007 ceph-mon[108238]: from='client.35348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:54.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:54 smithi007 ceph-mon[108238]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:54.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:54 smithi007 ceph-mon[108238]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:54.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.802+0000 7f5556ffd640 1 -- 172.21.15.7:0/3908444816 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f555008ed90 con 0x7f5560107900 2024-09-21T05:02:54.806 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:02:54.806 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:02:54.806 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:02:54.806 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:02:54.806 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:02:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:02:54.806 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:02:54.807 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:02:54.808 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:02:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:02:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:02:54.809 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:02:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:02:54.809 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:02:54.809 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:02:54.809 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:02:54.809 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f554c07af80 msgr2=0x7f554c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f554c07af80 0x7f554c07d440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f556019ab50 tx=0x7f555c00b410 comp rx=0 tx=0).stop 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560107900 msgr2=0x7f556019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560107900 0x7f556019fd90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5550031460 tx=0x7f55500045e0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 shutdown_connections 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f554c07af80 0x7f554c07d440 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5560107900 0x7f556019fd90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 --2- 172.21.15.7:0/3908444816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5560101900 0x7f556019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 >> 172.21.15.7:0/3908444816 conn(0x7f55600fd600 msgr2=0x7f55600fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 shutdown_connections 2024-09-21T05:02:54.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.805+0000 7f5567dc6640 1 -- 172.21.15.7:0/3908444816 wait complete. 2024-09-21T05:02:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.890+0000 7f6edfd75640 1 -- 172.21.15.7:0/3379917127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 msgr2=0x7f6ed8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.890+0000 7f6edfd75640 1 --2- 172.21.15.7:0/3379917127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 0x7f6ed8101d60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009880 tx=0x7f6ecc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.890+0000 7f6edfd75640 1 -- 172.21.15.7:0/3379917127 shutdown_connections 2024-09-21T05:02:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.890+0000 7f6edfd75640 1 --2- 172.21.15.7:0/3379917127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 0x7f6ed8101d60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.890+0000 7f6edfd75640 1 --2- 172.21.15.7:0/3379917127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed8107900 0x7f6ed8107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.890+0000 7f6edfd75640 1 -- 172.21.15.7:0/3379917127 >> 172.21.15.7:0/3379917127 conn(0x7f6ed80fd600 msgr2=0x7f6ed80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.890+0000 7f6edfd75640 1 -- 172.21.15.7:0/3379917127 shutdown_connections 2024-09-21T05:02:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.890+0000 7f6edfd75640 1 -- 172.21.15.7:0/3379917127 wait complete. 2024-09-21T05:02:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6edfd75640 1 Processor -- start 2024-09-21T05:02:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6edfd75640 1 -- start start 2024-09-21T05:02:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6edfd75640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 0x7f6ed819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6edfd75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed8107900 0x7f6ed819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6edfd75640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed810edb0 con 0x7f6ed8101900 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6edfd75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed810ec30 con 0x7f6ed8107900 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6eddb11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 0x7f6ed819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6eddb11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 0x7f6ed819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48450/0 (socket says 172.21.15.7:48450) 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6eddb11640 1 -- 172.21.15.7:0/3114084914 learned_addr learned my addr 172.21.15.7:0/3114084914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6edd310640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed8107900 0x7f6ed819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6eddb11640 1 -- 172.21.15.7:0/3114084914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed8107900 msgr2=0x7f6ed819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6eddb11640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed8107900 0x7f6ed819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6eddb11640 1 -- 172.21.15.7:0/3114084914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed81977d0 con 0x7f6ed8101900 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.891+0000 7f6edd310640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed8107900 0x7f6ed819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.892+0000 7f6eddb11640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 0x7f6ed819d650 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c810 tx=0x7f6ed400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.892+0000 7f6ecaffd640 1 -- 172.21.15.7:0/3114084914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ed4004020 con 0x7f6ed8101900 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.892+0000 7f6ecaffd640 1 -- 172.21.15.7:0/3114084914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ed40041c0 con 0x7f6ed8101900 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.892+0000 7f6ecaffd640 1 -- 172.21.15.7:0/3114084914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ed40047a0 con 0x7f6ed8101900 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.892+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed8197ac0 con 0x7f6ed8101900 2024-09-21T05:02:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.892+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed8198050 con 0x7f6ed8101900 2024-09-21T05:02:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.893+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed8103040 con 0x7f6ed8101900 2024-09-21T05:02:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.894+0000 7f6ecaffd640 1 -- 172.21.15.7:0/3114084914 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6ed40079b0 con 0x7f6ed8101900 2024-09-21T05:02:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.894+0000 7f6ecaffd640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ec407af80 0x7f6ec407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.895+0000 7f6edd310640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ec407af80 0x7f6ec407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.895+0000 7f6ecaffd640 1 -- 172.21.15.7:0/3114084914 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f6ed409dec0 con 0x7f6ed8101900 2024-09-21T05:02:54.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.895+0000 7f6edd310640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ec407af80 0x7f6ec407d440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc031460 tx=0x7f6ecc0095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:54.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:54.899+0000 7f6ecaffd640 1 -- 172.21.15.7:0/3114084914 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ed4014030 con 0x7f6ed8101900 2024-09-21T05:02:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:54 smithi053 ceph-mon[80176]: pgmap v845: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:54 smithi053 ceph-mon[80176]: from='client.35348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:54 smithi053 ceph-mon[80176]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:54 smithi053 ceph-mon[80176]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:55.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.102+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6ed81983c0 con 0x7f6ec407af80 2024-09-21T05:02:55.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.104+0000 7f6ecaffd640 1 -- 172.21.15.7:0/3114084914 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f6ed81983c0 con 0x7f6ec407af80 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:02:55.107 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:02:55.108 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:02:55.108 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:02:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ec407af80 msgr2=0x7f6ec407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ec407af80 0x7f6ec407d440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc031460 tx=0x7f6ecc0095f0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 msgr2=0x7f6ed819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 0x7f6ed819d650 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c810 tx=0x7f6ed400ccd0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 shutdown_connections 2024-09-21T05:02:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6ec407af80 0x7f6ec407d440 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ed8107900 0x7f6ed819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 --2- 172.21.15.7:0/3114084914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed8101900 0x7f6ed819d650 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 >> 172.21.15.7:0/3114084914 conn(0x7f6ed80fd600 msgr2=0x7f6ed80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 shutdown_connections 2024-09-21T05:02:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.106+0000 7f6edfd75640 1 -- 172.21.15.7:0/3114084914 wait complete. 2024-09-21T05:02:55.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.190+0000 7fc719b40640 1 -- 172.21.15.7:0/3327596568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 msgr2=0x7fc714107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:55.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.190+0000 7fc719b40640 1 --2- 172.21.15.7:0/3327596568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 0x7fc714107ce0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc704009c20 tx=0x7fc70402a970 comp rx=0 tx=0).stop 2024-09-21T05:02:55.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.190+0000 7fc719b40640 1 -- 172.21.15.7:0/3327596568 shutdown_connections 2024-09-21T05:02:55.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.190+0000 7fc719b40640 1 --2- 172.21.15.7:0/3327596568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc714101900 0x7fc714101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.190+0000 7fc719b40640 1 --2- 172.21.15.7:0/3327596568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 0x7fc714107ce0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.190+0000 7fc719b40640 1 -- 172.21.15.7:0/3327596568 >> 172.21.15.7:0/3327596568 conn(0x7fc7140fd600 msgr2=0x7fc7140ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:55.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.190+0000 7fc719b40640 1 -- 172.21.15.7:0/3327596568 shutdown_connections 2024-09-21T05:02:55.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.190+0000 7fc719b40640 1 -- 172.21.15.7:0/3327596568 wait complete. 2024-09-21T05:02:55.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.191+0000 7fc719b40640 1 Processor -- start 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.191+0000 7fc719b40640 1 -- start start 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.191+0000 7fc719b40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc714101900 0x7fc71419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.191+0000 7fc719b40640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 0x7fc71419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.191+0000 7fc719b40640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc71410edd0 con 0x7fc714107900 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.191+0000 7fc719b40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc71410ec50 con 0x7fc714101900 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 0x7fc71419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 0x7fc71419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48480/0 (socket says 172.21.15.7:48480) 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc712ffd640 1 -- 172.21.15.7:0/3488601261 learned_addr learned my addr 172.21.15.7:0/3488601261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc7137fe640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc714101900 0x7fc71419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc712ffd640 1 -- 172.21.15.7:0/3488601261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc714101900 msgr2=0x7fc71419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc712ffd640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc714101900 0x7fc71419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc712ffd640 1 -- 172.21.15.7:0/3488601261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7141977a0 con 0x7fc714107900 2024-09-21T05:02:55.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc712ffd640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 0x7fc71419db60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc70800e970 tx=0x7fc70800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:55.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc710ff9640 1 -- 172.21.15.7:0/3488601261 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc70800cac0 con 0x7fc714107900 2024-09-21T05:02:55.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc710ff9640 1 -- 172.21.15.7:0/3488601261 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc7080054e0 con 0x7fc714107900 2024-09-21T05:02:55.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc714197a90 con 0x7fc714107900 2024-09-21T05:02:55.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc710ff9640 1 -- 172.21.15.7:0/3488601261 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc708010430 con 0x7fc714107900 2024-09-21T05:02:55.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.192+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc714198020 con 0x7fc714107900 2024-09-21T05:02:55.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.193+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc714103040 con 0x7fc714107900 2024-09-21T05:02:55.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.194+0000 7fc710ff9640 1 -- 172.21.15.7:0/3488601261 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc708004ea0 con 0x7fc714107900 2024-09-21T05:02:55.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.194+0000 7fc710ff9640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc6fc07af30 0x7fc6fc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:02:55.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.195+0000 7fc710ff9640 1 -- 172.21.15.7:0/3488601261 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc70809e6d0 con 0x7fc714107900 2024-09-21T05:02:55.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.195+0000 7fc7137fe640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc6fc07af30 0x7fc6fc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:02:55.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.195+0000 7fc7137fe640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc6fc07af30 0x7fc6fc07d3f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc704009430 tx=0x7fc70403c040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:02:55.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.199+0000 7fc710ff9640 1 -- 172.21.15.7:0/3488601261 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc708069c70 con 0x7fc714107900 2024-09-21T05:02:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.462+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fc714109e70 con 0x7fc714107900 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.463+0000 7fc710ff9640 1 -- 172.21.15.7:0/3488601261 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fc70806eb20 con 0x7fc714107900 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:02:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:02:55.467 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:02:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:02:55.468 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:02:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:02:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc6fc07af30 msgr2=0x7fc6fc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc6fc07af30 0x7fc6fc07d3f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc704009430 tx=0x7fc70403c040 comp rx=0 tx=0).stop 2024-09-21T05:02:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 msgr2=0x7fc71419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:02:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 0x7fc71419db60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc70800e970 tx=0x7fc70800ed70 comp rx=0 tx=0).stop 2024-09-21T05:02:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 shutdown_connections 2024-09-21T05:02:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc6fc07af30 0x7fc6fc07d3f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc714107900 0x7fc71419db60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 --2- 172.21.15.7:0/3488601261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc714101900 0x7fc71419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:02:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 >> 172.21.15.7:0/3488601261 conn(0x7fc7140fd600 msgr2=0x7fc7140fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:02:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 shutdown_connections 2024-09-21T05:02:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:02:55.465+0000 7fc719b40640 1 -- 172.21.15.7:0/3488601261 wait complete. 2024-09-21T05:02:55.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:55 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1294642914' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:02:55.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:55 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3908444816' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:02:55.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:55 smithi007 ceph-mon[108238]: from='client.35368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:55.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:55 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1294642914' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:02:55.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:55 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3908444816' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:02:55.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:55 smithi053 ceph-mon[80176]: from='client.35368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:02:56.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:56 smithi053 ceph-mon[80176]: pgmap v846: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:56.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:56 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3488601261' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:02:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:56 smithi007 ceph-mon[108238]: pgmap v846: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:56 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3488601261' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:02:58.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:58 smithi007 ceph-mon[108238]: pgmap v847: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:58 smithi053 ceph-mon[80176]: pgmap v847: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:02:59.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:02:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:02:59.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:02:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:00 smithi053 ceph-mon[80176]: pgmap v848: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:00 smithi007 ceph-mon[108238]: pgmap v848: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:02.827 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:02 smithi053 ceph-mon[80176]: pgmap v849: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:02 smithi007 ceph-mon[108238]: pgmap v849: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:04.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:04 smithi007 ceph-mon[108238]: pgmap v850: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:04.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:04 smithi053 ceph-mon[80176]: pgmap v850: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:06.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:06 smithi007 ceph-mon[108238]: pgmap v851: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:06.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:06 smithi053 ceph-mon[80176]: pgmap v851: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:08 smithi053 ceph-mon[80176]: pgmap v852: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:08 smithi007 ceph-mon[108238]: pgmap v852: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:10.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:10 smithi007 ceph-mon[108238]: pgmap v853: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:10.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:10 smithi053 ceph-mon[80176]: pgmap v853: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:12.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:12 smithi053 ceph-mon[80176]: pgmap v854: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:12 smithi007 ceph-mon[108238]: pgmap v854: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:14 smithi053 ceph-mon[80176]: pgmap v855: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:14 smithi007 ceph-mon[108238]: pgmap v855: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:16.937 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:16 smithi007 ceph-mon[108238]: pgmap v856: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:16.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:16 smithi053 ceph-mon[80176]: pgmap v856: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:18.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:18 smithi053 ceph-mon[80176]: pgmap v857: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:18.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:18 smithi007 ceph-mon[108238]: pgmap v857: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:20.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:20 smithi053 ceph-mon[80176]: pgmap v858: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:20.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:20 smithi007 ceph-mon[108238]: pgmap v858: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:22.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:22 smithi007 ceph-mon[108238]: pgmap v859: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:22.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:22 smithi053 ceph-mon[80176]: pgmap v859: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:24 smithi053 ceph-mon[80176]: pgmap v860: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:24 smithi007 ceph-mon[108238]: pgmap v860: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.564+0000 7fb47b770640 1 -- 172.21.15.7:0/1422456514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474107900 msgr2=0x7fb474107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.564+0000 7fb47b770640 1 --2- 172.21.15.7:0/1422456514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474107900 0x7fb474107ce0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb470009880 tx=0x7fb47002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.564+0000 7fb47b770640 1 -- 172.21.15.7:0/1422456514 shutdown_connections 2024-09-21T05:03:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.564+0000 7fb47b770640 1 --2- 172.21.15.7:0/1422456514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb474101900 0x7fb474101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.564+0000 7fb47b770640 1 --2- 172.21.15.7:0/1422456514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474107900 0x7fb474107ce0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.564+0000 7fb47b770640 1 -- 172.21.15.7:0/1422456514 >> 172.21.15.7:0/1422456514 conn(0x7fb4740fd600 msgr2=0x7fb4740ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.564+0000 7fb47b770640 1 -- 172.21.15.7:0/1422456514 shutdown_connections 2024-09-21T05:03:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.565+0000 7fb47b770640 1 -- 172.21.15.7:0/1422456514 wait complete. 2024-09-21T05:03:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.565+0000 7fb47b770640 1 Processor -- start 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.565+0000 7fb47b770640 1 -- start start 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47b770640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474101900 0x7fb47419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47b770640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb474107900 0x7fb47419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47b770640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb47410edb0 con 0x7fb474101900 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47b770640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb47410ec30 con 0x7fb474107900 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47950c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474101900 0x7fb47419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47950c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474101900 0x7fb47419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49352/0 (socket says 172.21.15.7:49352) 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47950c640 1 -- 172.21.15.7:0/4275466612 learned_addr learned my addr 172.21.15.7:0/4275466612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb478d0b640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb474107900 0x7fb47419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47950c640 1 -- 172.21.15.7:0/4275466612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb474107900 msgr2=0x7fb47419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47950c640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb474107900 0x7fb47419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47950c640 1 -- 172.21.15.7:0/4275466612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4741977e0 con 0x7fb474101900 2024-09-21T05:03:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.566+0000 7fb47950c640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474101900 0x7fb47419d660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb470031460 tx=0x7fb4700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.567+0000 7fb46a7fc640 1 -- 172.21.15.7:0/4275466612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb47003f070 con 0x7fb474101900 2024-09-21T05:03:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.567+0000 7fb46a7fc640 1 -- 172.21.15.7:0/4275466612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4700044d0 con 0x7fb474101900 2024-09-21T05:03:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.567+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb474197a70 con 0x7fb474101900 2024-09-21T05:03:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.567+0000 7fb46a7fc640 1 -- 172.21.15.7:0/4275466612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb470035030 con 0x7fb474101900 2024-09-21T05:03:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.567+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb474197fa0 con 0x7fb474101900 2024-09-21T05:03:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.568+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb474103040 con 0x7fb474101900 2024-09-21T05:03:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.569+0000 7fb46a7fc640 1 -- 172.21.15.7:0/4275466612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb470042070 con 0x7fb474101900 2024-09-21T05:03:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.569+0000 7fb46a7fc640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb46007af40 0x7fb46007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.569+0000 7fb46a7fc640 1 -- 172.21.15.7:0/4275466612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fb4700beb00 con 0x7fb474101900 2024-09-21T05:03:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.569+0000 7fb478d0b640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb46007af40 0x7fb46007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.570+0000 7fb478d0b640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb46007af40 0x7fb46007d400 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb474100d10 tx=0x7fb46400a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.574+0000 7fb46a7fc640 1 -- 172.21.15.7:0/4275466612 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb470004cf0 con 0x7fb474101900 2024-09-21T05:03:25.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.781+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb474198250 con 0x7fb46007af40 2024-09-21T05:03:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.782+0000 7fb46a7fc640 1 -- 172.21.15.7:0/4275466612 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fb474198250 con 0x7fb46007af40 2024-09-21T05:03:25.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb46007af40 msgr2=0x7fb46007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:25.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb46007af40 0x7fb46007d400 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb474100d10 tx=0x7fb46400a2d0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474101900 msgr2=0x7fb47419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:25.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474101900 0x7fb47419d660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb470031460 tx=0x7fb4700048f0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 shutdown_connections 2024-09-21T05:03:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb46007af40 0x7fb46007d400 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb474107900 0x7fb47419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 --2- 172.21.15.7:0/4275466612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb474101900 0x7fb47419d660 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.784+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 >> 172.21.15.7:0/4275466612 conn(0x7fb4740fd600 msgr2=0x7fb4740fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.785+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 shutdown_connections 2024-09-21T05:03:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.785+0000 7fb47b770640 1 -- 172.21.15.7:0/4275466612 wait complete. 2024-09-21T05:03:25.797 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:03:25.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.870+0000 7ff438fe8640 1 -- 172.21.15.7:0/786628255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 msgr2=0x7ff434107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:25.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.870+0000 7ff438fe8640 1 --2- 172.21.15.7:0/786628255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 0x7ff434107ce0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff424009880 tx=0x7ff42402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.870+0000 7ff438fe8640 1 -- 172.21.15.7:0/786628255 shutdown_connections 2024-09-21T05:03:25.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.870+0000 7ff438fe8640 1 --2- 172.21.15.7:0/786628255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff434101900 0x7ff434101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.870+0000 7ff438fe8640 1 --2- 172.21.15.7:0/786628255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 0x7ff434107ce0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.870+0000 7ff438fe8640 1 -- 172.21.15.7:0/786628255 >> 172.21.15.7:0/786628255 conn(0x7ff4340fd600 msgr2=0x7ff4340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:25.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.870+0000 7ff438fe8640 1 -- 172.21.15.7:0/786628255 shutdown_connections 2024-09-21T05:03:25.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.870+0000 7ff438fe8640 1 -- 172.21.15.7:0/786628255 wait complete. 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff438fe8640 1 Processor -- start 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff438fe8640 1 -- start start 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff438fe8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff434101900 0x7ff43419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff438fe8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 0x7ff43419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff438fe8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410edd0 con 0x7ff434107900 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff438fe8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410ec50 con 0x7ff434101900 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff431d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 0x7ff43419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff431d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 0x7ff43419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49380/0 (socket says 172.21.15.7:49380) 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.871+0000 7ff431d9b640 1 -- 172.21.15.7:0/3341014514 learned_addr learned my addr 172.21.15.7:0/3341014514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff43259c640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff434101900 0x7ff43419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff431d9b640 1 -- 172.21.15.7:0/3341014514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff434101900 msgr2=0x7ff43419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff431d9b640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff434101900 0x7ff43419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff431d9b640 1 -- 172.21.15.7:0/3341014514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4341977c0 con 0x7ff434107900 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff431d9b640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 0x7ff43419db80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff42800e970 tx=0x7ff42800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff4237fe640 1 -- 172.21.15.7:0/3341014514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff42800cac0 con 0x7ff434107900 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff4237fe640 1 -- 172.21.15.7:0/3341014514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4280054e0 con 0x7ff434107900 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff434197ab0 con 0x7ff434107900 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff4237fe640 1 -- 172.21.15.7:0/3341014514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff428010430 con 0x7ff434107900 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.872+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff434198040 con 0x7ff434107900 2024-09-21T05:03:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.873+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff434103040 con 0x7ff434107900 2024-09-21T05:03:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.874+0000 7ff4237fe640 1 -- 172.21.15.7:0/3341014514 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff428004e80 con 0x7ff434107900 2024-09-21T05:03:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.874+0000 7ff4237fe640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff41c07af40 0x7ff41c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.874+0000 7ff4237fe640 1 -- 172.21.15.7:0/3341014514 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff42809e750 con 0x7ff434107900 2024-09-21T05:03:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.874+0000 7ff43259c640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff41c07af40 0x7ff41c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.875+0000 7ff43259c640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff41c07af40 0x7ff41c07d400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff424031460 tx=0x7ff4240095c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:25.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:25.879+0000 7ff4237fe640 1 -- 172.21.15.7:0/3341014514 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff428069cf0 con 0x7ff434107900 2024-09-21T05:03:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.082+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff434198350 con 0x7ff41c07af40 2024-09-21T05:03:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.084+0000 7ff4237fe640 1 -- 172.21.15.7:0/3341014514 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ff434198350 con 0x7ff41c07af40 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff41c07af40 msgr2=0x7ff41c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff41c07af40 0x7ff41c07d400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff424031460 tx=0x7ff4240095c0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 msgr2=0x7ff43419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 0x7ff43419db80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff42800e970 tx=0x7ff42800ed70 comp rx=0 tx=0).stop 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 shutdown_connections 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff41c07af40 0x7ff41c07d400 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff434107900 0x7ff43419db80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 --2- 172.21.15.7:0/3341014514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff434101900 0x7ff43419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 >> 172.21.15.7:0/3341014514 conn(0x7ff4340fd600 msgr2=0x7ff4340fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:26.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 shutdown_connections 2024-09-21T05:03:26.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.086+0000 7ff438fe8640 1 -- 172.21.15.7:0/3341014514 wait complete. 2024-09-21T05:03:26.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.170+0000 7f009e96b640 1 -- 172.21.15.7:0/1624614966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 msgr2=0x7f0098101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.170+0000 7f009e96b640 1 --2- 172.21.15.7:0/1624614966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 0x7f0098101d60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f008c009880 tx=0x7f008c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.170+0000 7f009e96b640 1 -- 172.21.15.7:0/1624614966 shutdown_connections 2024-09-21T05:03:26.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.170+0000 7f009e96b640 1 --2- 172.21.15.7:0/1624614966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 0x7f0098101d60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.170+0000 7f009e96b640 1 --2- 172.21.15.7:0/1624614966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0098107900 0x7f0098107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.170+0000 7f009e96b640 1 -- 172.21.15.7:0/1624614966 >> 172.21.15.7:0/1624614966 conn(0x7f00980fd600 msgr2=0x7f00980ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:26.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.170+0000 7f009e96b640 1 -- 172.21.15.7:0/1624614966 shutdown_connections 2024-09-21T05:03:26.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.170+0000 7f009e96b640 1 -- 172.21.15.7:0/1624614966 wait complete. 2024-09-21T05:03:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.171+0000 7f009e96b640 1 Processor -- start 2024-09-21T05:03:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.171+0000 7f009e96b640 1 -- start start 2024-09-21T05:03:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.171+0000 7f009e96b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 0x7f009819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.171+0000 7f009e96b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0098107900 0x7f009819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.171+0000 7f009e96b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f009810edd0 con 0x7f0098101900 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.171+0000 7f009e96b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f009810ec50 con 0x7f0098107900 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f0097fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 0x7f009819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f0097fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 0x7f009819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49406/0 (socket says 172.21.15.7:49406) 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f0097fff640 1 -- 172.21.15.7:0/582504365 learned_addr learned my addr 172.21.15.7:0/582504365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f00977fe640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0098107900 0x7f009819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f0097fff640 1 -- 172.21.15.7:0/582504365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0098107900 msgr2=0x7f009819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f0097fff640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0098107900 0x7f009819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f0097fff640 1 -- 172.21.15.7:0/582504365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0098197800 con 0x7f0098101900 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f0097fff640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 0x7f009819d680 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f008800c8f0 tx=0x7f008800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f00957fa640 1 -- 172.21.15.7:0/582504365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00880079b0 con 0x7f0098101900 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f00957fa640 1 -- 172.21.15.7:0/582504365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00880054e0 con 0x7f0098101900 2024-09-21T05:03:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0098197af0 con 0x7f0098101900 2024-09-21T05:03:26.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.172+0000 7f00957fa640 1 -- 172.21.15.7:0/582504365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0088004dd0 con 0x7f0098101900 2024-09-21T05:03:26.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.173+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0098198050 con 0x7f0098101900 2024-09-21T05:03:26.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.174+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0098103040 con 0x7f0098101900 2024-09-21T05:03:26.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.174+0000 7f00957fa640 1 -- 172.21.15.7:0/582504365 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0088005020 con 0x7f0098101900 2024-09-21T05:03:26.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.175+0000 7f00957fa640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f008007aed0 0x7f008007d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.175+0000 7f00977fe640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f008007aed0 0x7f008007d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.175+0000 7f00957fa640 1 -- 172.21.15.7:0/582504365 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f008809e0c0 con 0x7f0098101900 2024-09-21T05:03:26.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.176+0000 7f00977fe640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f008007aed0 0x7f008007d390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0098198b50 tx=0x7f008c031420 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:26.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.180+0000 7f00957fa640 1 -- 172.21.15.7:0/582504365 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0088014030 con 0x7f0098101900 2024-09-21T05:03:26.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.374+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0098198370 con 0x7f008007aed0 2024-09-21T05:03:26.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.384+0000 7f00957fa640 1 -- 172.21.15.7:0/582504365 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f0098198370 con 0x7f008007aed0 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (25m) 4m ago 36m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (28m) 4m ago 36m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (28m) 4m ago 35m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (23m) 4m ago 36m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (23m) 4m ago 35m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (25m) 4m ago 36m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (19m) 4m ago 31m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (19m) 4m ago 31m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (4m) 4m ago 30m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:03:26.388 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (19m) 4m ago 31m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (27m) 4m ago 37m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (26m) 4m ago 35m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (24m) 4m ago 37m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (23m) 4m ago 35m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (26m) 4m ago 36m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (26m) 4m ago 35m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (23m) 4m ago 34m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (22m) 4m ago 33m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (21m) 4m ago 33m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (21m) 4m ago 32m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (20m) 4m ago 32m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (20m) 4m ago 32m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:03:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (25m) 4m ago 36m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.386+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f008007aed0 msgr2=0x7f008007d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.386+0000 7f009e96b640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f008007aed0 0x7f008007d390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0098198b50 tx=0x7f008c031420 comp rx=0 tx=0).stop 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 msgr2=0x7f009819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 0x7f009819d680 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f008800c8f0 tx=0x7f008800cdb0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 shutdown_connections 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f008007aed0 0x7f008007d390 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0098107900 0x7f009819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 --2- 172.21.15.7:0/582504365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0098101900 0x7f009819d680 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 >> 172.21.15.7:0/582504365 conn(0x7f00980fd600 msgr2=0x7f00980fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 shutdown_connections 2024-09-21T05:03:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.387+0000 7f009e96b640 1 -- 172.21.15.7:0/582504365 wait complete. 2024-09-21T05:03:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.470+0000 7f7ff3d06640 1 -- 172.21.15.7:0/3336164923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 msgr2=0x7f7fec101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.470+0000 7f7ff3d06640 1 --2- 172.21.15.7:0/3336164923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 0x7f7fec101d60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009880 tx=0x7f7fdc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.471+0000 7f7ff3d06640 1 -- 172.21.15.7:0/3336164923 shutdown_connections 2024-09-21T05:03:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.471+0000 7f7ff3d06640 1 --2- 172.21.15.7:0/3336164923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 0x7f7fec101d60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.471+0000 7f7ff3d06640 1 --2- 172.21.15.7:0/3336164923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec107900 0x7f7fec107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.471+0000 7f7ff3d06640 1 -- 172.21.15.7:0/3336164923 >> 172.21.15.7:0/3336164923 conn(0x7f7fec0fd600 msgr2=0x7f7fec0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.471+0000 7f7ff3d06640 1 -- 172.21.15.7:0/3336164923 shutdown_connections 2024-09-21T05:03:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.471+0000 7f7ff3d06640 1 -- 172.21.15.7:0/3336164923 wait complete. 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.471+0000 7f7ff3d06640 1 Processor -- start 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.471+0000 7f7ff3d06640 1 -- start start 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff3d06640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 0x7f7fec19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff3d06640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec107900 0x7f7fec19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff3d06640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10edd0 con 0x7f7fec101900 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff3d06640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec10ec50 con 0x7f7fec107900 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff1aa2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 0x7f7fec19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff1aa2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 0x7f7fec19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49418/0 (socket says 172.21.15.7:49418) 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff1aa2640 1 -- 172.21.15.7:0/440456165 learned_addr learned my addr 172.21.15.7:0/440456165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff12a1640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec107900 0x7f7fec19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff1aa2640 1 -- 172.21.15.7:0/440456165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec107900 msgr2=0x7f7fec19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff1aa2640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec107900 0x7f7fec19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff1aa2640 1 -- 172.21.15.7:0/440456165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec197800 con 0x7f7fec101900 2024-09-21T05:03:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff1aa2640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 0x7f7fec19d680 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800e970 tx=0x7f7fe800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7fe2ffd640 1 -- 172.21.15.7:0/440456165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7fe800cac0 con 0x7f7fec101900 2024-09-21T05:03:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7fe2ffd640 1 -- 172.21.15.7:0/440456165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fe80054e0 con 0x7f7fec101900 2024-09-21T05:03:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7fe2ffd640 1 -- 172.21.15.7:0/440456165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7fe8010430 con 0x7f7fec101900 2024-09-21T05:03:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.472+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec197af0 con 0x7f7fec101900 2024-09-21T05:03:26.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.473+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec198050 con 0x7f7fec101900 2024-09-21T05:03:26.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.474+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec103040 con 0x7f7fec101900 2024-09-21T05:03:26.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.474+0000 7f7fe2ffd640 1 -- 172.21.15.7:0/440456165 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7fe8004f00 con 0x7f7fec101900 2024-09-21T05:03:26.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.475+0000 7f7fe2ffd640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7fd807af80 0x7f7fd807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.475+0000 7f7fe2ffd640 1 -- 172.21.15.7:0/440456165 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f7fe809e600 con 0x7f7fec101900 2024-09-21T05:03:26.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.475+0000 7f7ff12a1640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7fd807af80 0x7f7fd807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.475+0000 7f7ff12a1640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7fd807af80 0x7f7fd807d440 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fec198b50 tx=0x7f7fdc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:26.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.479+0000 7f7fe2ffd640 1 -- 172.21.15.7:0/440456165 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7fe8069c20 con 0x7f7fec101900 2024-09-21T05:03:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.749+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7fec109e70 con 0x7f7fec101900 2024-09-21T05:03:26.754 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:26 smithi007 ceph-mon[108238]: pgmap v861: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:26.754 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:26 smithi007 ceph-mon[108238]: from='client.35376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:26.754 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:26 smithi007 ceph-mon[108238]: from='client.35380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.750+0000 7f7fe2ffd640 1 -- 172.21.15.7:0/440456165 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f7fe806ead0 con 0x7f7fec101900 2024-09-21T05:03:26.754 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:03:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:03:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:03:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:03:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:03:26.756 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.752+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7fd807af80 msgr2=0x7f7fd807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.752+0000 7f7ff3d06640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7fd807af80 0x7f7fd807d440 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fec198b50 tx=0x7f7fdc031400 comp rx=0 tx=0).stop 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.752+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 msgr2=0x7f7fec19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.752+0000 7f7ff3d06640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 0x7f7fec19d680 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800e970 tx=0x7f7fe800ed70 comp rx=0 tx=0).stop 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.753+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 shutdown_connections 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.753+0000 7f7ff3d06640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7fd807af80 0x7f7fd807d440 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.753+0000 7f7ff3d06640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fec107900 0x7f7fec19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.753+0000 7f7ff3d06640 1 --2- 172.21.15.7:0/440456165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec101900 0x7f7fec19d680 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.753+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 >> 172.21.15.7:0/440456165 conn(0x7f7fec0fd600 msgr2=0x7f7fec0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.753+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 shutdown_connections 2024-09-21T05:03:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.753+0000 7f7ff3d06640 1 -- 172.21.15.7:0/440456165 wait complete. 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.838+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2355526969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8102b60 msgr2=0x7f2fd8102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.838+0000 7f2fdef5c640 1 --2- 172.21.15.7:0/2355526969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8102b60 0x7f2fd8102f40 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009880 tx=0x7f2fcc02a530 comp rx=0 tx=0).stop 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.838+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2355526969 shutdown_connections 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.838+0000 7f2fdef5c640 1 --2- 172.21.15.7:0/2355526969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd8103480 0x7f2fd810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.838+0000 7f2fdef5c640 1 --2- 172.21.15.7:0/2355526969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8102b60 0x7f2fd8102f40 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.838+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2355526969 >> 172.21.15.7:0/2355526969 conn(0x7f2fd80fc520 msgr2=0x7f2fd80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.838+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2355526969 shutdown_connections 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.838+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2355526969 wait complete. 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fdef5c640 1 Processor -- start 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fdef5c640 1 -- start start 2024-09-21T05:03:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fdef5c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd8102b60 0x7f2fd819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fdef5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8103480 0x7f2fd819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fdef5c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd810edd0 con 0x7f2fd8103480 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fdef5c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd810ec50 con 0x7f2fd8102b60 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8103480 0x7f2fd819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8103480 0x7f2fd819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49436/0 (socket says 172.21.15.7:49436) 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fd7fff640 1 -- 172.21.15.7:0/2230540801 learned_addr learned my addr 172.21.15.7:0/2230540801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fdccf8640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd8102b60 0x7f2fd819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fd7fff640 1 -- 172.21.15.7:0/2230540801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd8102b60 msgr2=0x7f2fd819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fd7fff640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd8102b60 0x7f2fd819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fd7fff640 1 -- 172.21.15.7:0/2230540801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd81977e0 con 0x7f2fd8103480 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.839+0000 7f2fdccf8640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd8102b60 0x7f2fd819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.840+0000 7f2fd7fff640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8103480 0x7f2fd819dba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c810 tx=0x7f2fc400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.840+0000 7f2fd5ffb640 1 -- 172.21.15.7:0/2230540801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2fc4004020 con 0x7f2fd8103480 2024-09-21T05:03:26.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.840+0000 7f2fd5ffb640 1 -- 172.21.15.7:0/2230540801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2fc40041c0 con 0x7f2fd8103480 2024-09-21T05:03:26.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.840+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd8197ad0 con 0x7f2fd8103480 2024-09-21T05:03:26.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.840+0000 7f2fd5ffb640 1 -- 172.21.15.7:0/2230540801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2fc40047a0 con 0x7f2fd8103480 2024-09-21T05:03:26.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.840+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd8198060 con 0x7f2fd8103480 2024-09-21T05:03:26.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.841+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd806a9f0 con 0x7f2fd8103480 2024-09-21T05:03:26.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.842+0000 7f2fd5ffb640 1 -- 172.21.15.7:0/2230540801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2fc40079b0 con 0x7f2fd8103480 2024-09-21T05:03:26.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.842+0000 7f2fd5ffb640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2fc807af80 0x7f2fc807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:26.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.842+0000 7f2fd5ffb640 1 -- 172.21.15.7:0/2230540801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f2fc409de80 con 0x7f2fd8103480 2024-09-21T05:03:26.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.842+0000 7f2fdccf8640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2fc807af80 0x7f2fc807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:26.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.843+0000 7f2fdccf8640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2fc807af80 0x7f2fc807d440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc004f90 tx=0x7f2fcc009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:26.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:26.847+0000 7f2fd5ffb640 1 -- 172.21.15.7:0/2230540801 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2fc4014030 con 0x7f2fd8103480 2024-09-21T05:03:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:26 smithi053 ceph-mon[80176]: pgmap v861: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:26 smithi053 ceph-mon[80176]: from='client.35376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:26 smithi053 ceph-mon[80176]: from='client.35380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:27.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.072+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2fd8105fc0 con 0x7f2fd8103480 2024-09-21T05:03:27.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.073+0000 7f2fd5ffb640 1 -- 172.21.15.7:0/2230540801 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f2fc40694a0 con 0x7f2fd8103480 2024-09-21T05:03:27.080 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:03:27.080 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:03:27.080 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:03:27.080 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:03:27.080 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:03:27.081 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:03:27.081 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:03:27.081 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:03:27.081 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:03:27.081 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:03:27.081 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:03:27.081 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:03:27.081 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:03:27.082 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:03:27.082 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:03:27.082 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:03:27.082 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:03:27.082 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:03:27.082 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:03:27.083 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:03:27.083 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:03:27.083 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:03:27.083 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:03:27.083 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:03:27.083 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:03:27.083 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:03:27.083 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:03:27.084 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:03:27.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:03:27.085 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:03:27.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:03:27.085 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:03:27.085 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:03:27.085 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:03:27.085 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:03:27.086 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:03:27.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2fc807af80 msgr2=0x7f2fc807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2fc807af80 0x7f2fc807d440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc004f90 tx=0x7f2fcc009560 comp rx=0 tx=0).stop 2024-09-21T05:03:27.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8103480 msgr2=0x7f2fd819dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8103480 0x7f2fd819dba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c810 tx=0x7f2fc400ccd0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 shutdown_connections 2024-09-21T05:03:27.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2fc807af80 0x7f2fc807d440 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd8103480 0x7f2fd819dba0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 --2- 172.21.15.7:0/2230540801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2fd8102b60 0x7f2fd819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 >> 172.21.15.7:0/2230540801 conn(0x7f2fd80fc520 msgr2=0x7f2fd8108f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:27.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 shutdown_connections 2024-09-21T05:03:27.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.075+0000 7f2fdef5c640 1 -- 172.21.15.7:0/2230540801 wait complete. 2024-09-21T05:03:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 -- 172.21.15.7:0/488136727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834101900 msgr2=0x7f2834101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 --2- 172.21.15.7:0/488136727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834101900 0x7f2834101d60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2824009c20 tx=0x7f282402a970 comp rx=0 tx=0).stop 2024-09-21T05:03:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 -- 172.21.15.7:0/488136727 shutdown_connections 2024-09-21T05:03:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 --2- 172.21.15.7:0/488136727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834101900 0x7f2834101d60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 --2- 172.21.15.7:0/488136727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834107900 0x7f2834107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 -- 172.21.15.7:0/488136727 >> 172.21.15.7:0/488136727 conn(0x7f28340fd600 msgr2=0x7f28340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 -- 172.21.15.7:0/488136727 shutdown_connections 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 -- 172.21.15.7:0/488136727 wait complete. 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 Processor -- start 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.160+0000 7f283ab30640 1 -- start start 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f283ab30640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834101900 0x7f28340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f283ab30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834107900 0x7f2834074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f283ab30640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f283410edb0 con 0x7f2834107900 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f283ab30640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f283410ec30 con 0x7f2834101900 2024-09-21T05:03:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2833fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834107900 0x7f2834074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2833fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834107900 0x7f2834074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49440/0 (socket says 172.21.15.7:49440) 2024-09-21T05:03:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2833fff640 1 -- 172.21.15.7:0/1172492027 learned_addr learned my addr 172.21.15.7:0/1172492027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f28388cc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834101900 0x7f28340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2833fff640 1 -- 172.21.15.7:0/1172492027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834101900 msgr2=0x7f28340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2833fff640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834101900 0x7f28340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2833fff640 1 -- 172.21.15.7:0/1172492027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2834074f70 con 0x7f2834107900 2024-09-21T05:03:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2833fff640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834107900 0x7f2834074a30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f282402ae90 tx=0x7f282402f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2831ffb640 1 -- 172.21.15.7:0/1172492027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2824047070 con 0x7f2834107900 2024-09-21T05:03:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2831ffb640 1 -- 172.21.15.7:0/1172492027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f282402f7e0 con 0x7f2834107900 2024-09-21T05:03:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2834075200 con 0x7f2834107900 2024-09-21T05:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.161+0000 7f2831ffb640 1 -- 172.21.15.7:0/1172492027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2824034070 con 0x7f2834107900 2024-09-21T05:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.162+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28341a3d90 con 0x7f2834107900 2024-09-21T05:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.163+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2834103040 con 0x7f2834107900 2024-09-21T05:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.163+0000 7f2831ffb640 1 -- 172.21.15.7:0/1172492027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f282404a070 con 0x7f2834107900 2024-09-21T05:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.164+0000 7f2831ffb640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f282007af80 0x7f282007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.164+0000 7f2831ffb640 1 -- 172.21.15.7:0/1172492027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f28240becc0 con 0x7f2834107900 2024-09-21T05:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.164+0000 7f28388cc640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f282007af80 0x7f282007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.164+0000 7f28388cc640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f282007af80 0x7f282007d440 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f282c005d00 tx=0x7f282c00b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.168+0000 7f2831ffb640 1 -- 172.21.15.7:0/1172492027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f282403f070 con 0x7f2834107900 2024-09-21T05:03:27.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.371+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2834062630 con 0x7f282007af80 2024-09-21T05:03:27.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.372+0000 7f2831ffb640 1 -- 172.21.15.7:0/1172492027 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f2834062630 con 0x7f282007af80 2024-09-21T05:03:27.376 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:03:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:03:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:03:27.377 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:03:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.374+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f282007af80 msgr2=0x7f282007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.374+0000 7f283ab30640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f282007af80 0x7f282007d440 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f282c005d00 tx=0x7f282c00b410 comp rx=0 tx=0).stop 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.374+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834107900 msgr2=0x7f2834074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.374+0000 7f283ab30640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834107900 0x7f2834074a30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f282402ae90 tx=0x7f282402f5f0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.375+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 shutdown_connections 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.375+0000 7f283ab30640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f282007af80 0x7f282007d440 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.375+0000 7f283ab30640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2834107900 0x7f2834074a30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.375+0000 7f283ab30640 1 --2- 172.21.15.7:0/1172492027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2834101900 0x7f28340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.375+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 >> 172.21.15.7:0/1172492027 conn(0x7f28340fd600 msgr2=0x7f28340fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.375+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 shutdown_connections 2024-09-21T05:03:27.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.375+0000 7f283ab30640 1 -- 172.21.15.7:0/1172492027 wait complete. 2024-09-21T05:03:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.459+0000 7f9283d0a640 1 -- 172.21.15.7:0/284297271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c101900 msgr2=0x7f927c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.459+0000 7f9283d0a640 1 --2- 172.21.15.7:0/284297271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c101900 0x7f927c101d60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f926c009880 tx=0x7f926c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.459+0000 7f9283d0a640 1 -- 172.21.15.7:0/284297271 shutdown_connections 2024-09-21T05:03:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.459+0000 7f9283d0a640 1 --2- 172.21.15.7:0/284297271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c101900 0x7f927c101d60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.459+0000 7f9283d0a640 1 --2- 172.21.15.7:0/284297271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c107900 0x7f927c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.459+0000 7f9283d0a640 1 -- 172.21.15.7:0/284297271 >> 172.21.15.7:0/284297271 conn(0x7f927c0fd600 msgr2=0x7f927c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:27.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.459+0000 7f9283d0a640 1 -- 172.21.15.7:0/284297271 shutdown_connections 2024-09-21T05:03:27.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.459+0000 7f9283d0a640 1 -- 172.21.15.7:0/284297271 wait complete. 2024-09-21T05:03:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f9283d0a640 1 Processor -- start 2024-09-21T05:03:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f9283d0a640 1 -- start start 2024-09-21T05:03:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f9283d0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c101900 0x7f927c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f9283d0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c107900 0x7f927c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f9283d0a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f927c10ee30 con 0x7f927c107900 2024-09-21T05:03:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f9283d0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f927c10ecb0 con 0x7f927c101900 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f92812a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c107900 0x7f927c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f92812a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c107900 0x7f927c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49460/0 (socket says 172.21.15.7:49460) 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f92812a5640 1 -- 172.21.15.7:0/2738250099 learned_addr learned my addr 172.21.15.7:0/2738250099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f9281aa6640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c101900 0x7f927c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f92812a5640 1 -- 172.21.15.7:0/2738250099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c101900 msgr2=0x7f927c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f92812a5640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c101900 0x7f927c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f92812a5640 1 -- 172.21.15.7:0/2738250099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c074f70 con 0x7f927c107900 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.460+0000 7f9281aa6640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c101900 0x7f927c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.461+0000 7f92812a5640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c107900 0x7f927c074a30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f926c031460 tx=0x7f926c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.461+0000 7f9272ffd640 1 -- 172.21.15.7:0/2738250099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f926c03f070 con 0x7f927c107900 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.461+0000 7f9272ffd640 1 -- 172.21.15.7:0/2738250099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f926c0044d0 con 0x7f927c107900 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.461+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c075200 con 0x7f927c107900 2024-09-21T05:03:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.461+0000 7f9272ffd640 1 -- 172.21.15.7:0/2738250099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f926c035030 con 0x7f927c107900 2024-09-21T05:03:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.461+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1a3e00 con 0x7f927c107900 2024-09-21T05:03:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.462+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927c103040 con 0x7f927c107900 2024-09-21T05:03:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.463+0000 7f9272ffd640 1 -- 172.21.15.7:0/2738250099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f926c042070 con 0x7f927c107900 2024-09-21T05:03:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.463+0000 7f9272ffd640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f926807af40 0x7f926807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.463+0000 7f9272ffd640 1 -- 172.21.15.7:0/2738250099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f926c0bec60 con 0x7f927c107900 2024-09-21T05:03:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.463+0000 7f9281aa6640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f926807af40 0x7f926807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.464+0000 7f9281aa6640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f926807af40 0x7f926807d400 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9278009930 tx=0x7f9278008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:27.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.468+0000 7f9272ffd640 1 -- 172.21.15.7:0/2738250099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f926c004cf0 con 0x7f927c107900 2024-09-21T05:03:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:27 smithi007 ceph-mon[108238]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:27 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/440456165' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:03:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:27 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2230540801' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:03:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.727+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f927c075a20 con 0x7f927c107900 2024-09-21T05:03:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.728+0000 7f9272ffd640 1 -- 172.21.15.7:0/2738250099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f926c07fe20 con 0x7f927c107900 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:03:27.732 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:03:27.733 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f926807af40 msgr2=0x7f926807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f926807af40 0x7f926807d400 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9278009930 tx=0x7f9278008040 comp rx=0 tx=0).stop 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c107900 msgr2=0x7f927c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c107900 0x7f927c074a30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f926c031460 tx=0x7f926c0048f0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 shutdown_connections 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f926807af40 0x7f926807d400 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c107900 0x7f927c074a30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 --2- 172.21.15.7:0/2738250099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c101900 0x7f927c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:27.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 >> 172.21.15.7:0/2738250099 conn(0x7f927c0fd600 msgr2=0x7f927c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:27.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 shutdown_connections 2024-09-21T05:03:27.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:27.730+0000 7f9283d0a640 1 -- 172.21.15.7:0/2738250099 wait complete. 2024-09-21T05:03:27.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:27 smithi053 ceph-mon[80176]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:27.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:27 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/440456165' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:03:27.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:27 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2230540801' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:03:28.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:28 smithi007 ceph-mon[108238]: pgmap v862: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:28.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:28 smithi007 ceph-mon[108238]: from='client.35396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:28.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2738250099' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:03:28.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:28 smithi053 ceph-mon[80176]: pgmap v862: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:28 smithi053 ceph-mon[80176]: from='client.35396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2738250099' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:03:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:30.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:30 smithi053 ceph-mon[80176]: pgmap v863: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:30 smithi007 ceph-mon[108238]: pgmap v863: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:32.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:32 smithi053 ceph-mon[80176]: pgmap v864: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:32.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:32 smithi007 ceph-mon[108238]: pgmap v864: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:34.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:34 smithi007 ceph-mon[108238]: pgmap v865: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:34.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:34 smithi053 ceph-mon[80176]: pgmap v865: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:36 smithi053 ceph-mon[80176]: pgmap v866: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:36.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:36 smithi007 ceph-mon[108238]: pgmap v866: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:38.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:38 smithi053 ceph-mon[80176]: pgmap v867: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:38 smithi007 ceph-mon[108238]: pgmap v867: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:40.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:40 smithi007 ceph-mon[108238]: pgmap v868: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:40.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:40 smithi053 ceph-mon[80176]: pgmap v868: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:42.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:42 smithi053 ceph-mon[80176]: pgmap v869: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:42.945 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:42 smithi007 ceph-mon[108238]: pgmap v869: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:43.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:43.928 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:44.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:44 smithi053 ceph-mon[80176]: pgmap v870: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:44 smithi007 ceph-mon[108238]: pgmap v870: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:46.908 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:46 smithi007 ceph-mon[108238]: pgmap v871: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:46 smithi053 ceph-mon[80176]: pgmap v871: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:48.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:48 smithi053 ceph-mon[80176]: pgmap v872: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:48.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:48 smithi007 ceph-mon[108238]: pgmap v872: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:50.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:50 smithi053 ceph-mon[80176]: pgmap v873: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:50.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:50 smithi007 ceph-mon[108238]: pgmap v873: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:52.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:52 smithi007 ceph-mon[108238]: pgmap v874: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:52.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:52 smithi053 ceph-mon[80176]: pgmap v874: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:54 smithi053 ceph-mon[80176]: pgmap v875: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:54.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:54 smithi007 ceph-mon[108238]: pgmap v875: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:56.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:56 smithi053 ceph-mon[80176]: pgmap v876: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:56.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:56 smithi007 ceph-mon[108238]: pgmap v876: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:57.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.828+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1191630332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 msgr2=0x7f0cf0106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:57.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.828+0000 7f0cf79c2640 1 --2- 172.21.15.7:0/1191630332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 0x7f0cf0106be0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009880 tx=0x7f0cec02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:57.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.828+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1191630332 shutdown_connections 2024-09-21T05:03:57.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.828+0000 7f0cf79c2640 1 --2- 172.21.15.7:0/1191630332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf0100800 0x7f0cf0100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:57.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.828+0000 7f0cf79c2640 1 --2- 172.21.15.7:0/1191630332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 0x7f0cf0106be0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:57.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.828+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1191630332 >> 172.21.15.7:0/1191630332 conn(0x7f0cf00fc520 msgr2=0x7f0cf00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:57.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.828+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1191630332 shutdown_connections 2024-09-21T05:03:57.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.828+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1191630332 wait complete. 2024-09-21T05:03:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.829+0000 7f0cf79c2640 1 Processor -- start 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.829+0000 7f0cf79c2640 1 -- start start 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.829+0000 7f0cf79c2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf0100800 0x7f0cf00fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.829+0000 7f0cf79c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 0x7f0cf00fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.829+0000 7f0cf79c2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf010dd30 con 0x7f0cf0106800 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.829+0000 7f0cf79c2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf010dbb0 con 0x7f0cf0100800 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf4f5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 0x7f0cf00fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf4f5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 0x7f0cf00fd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56840/0 (socket says 172.21.15.7:56840) 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf4f5d640 1 -- 172.21.15.7:0/1367036272 learned_addr learned my addr 172.21.15.7:0/1367036272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf575e640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf0100800 0x7f0cf00fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf4f5d640 1 -- 172.21.15.7:0/1367036272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf0100800 msgr2=0x7f0cf00fcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf4f5d640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf0100800 0x7f0cf00fcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf4f5d640 1 -- 172.21.15.7:0/1367036272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf00fd6f0 con 0x7f0cf0106800 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf4f5d640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 0x7f0cf00fd1b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000b420 tx=0x7f0ce000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0ce67fc640 1 -- 172.21.15.7:0/1367036272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ce0011630 con 0x7f0cf0106800 2024-09-21T05:03:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0ce67fc640 1 -- 172.21.15.7:0/1367036272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ce0016070 con 0x7f0cf0106800 2024-09-21T05:03:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf00fd9e0 con 0x7f0cf0106800 2024-09-21T05:03:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.830+0000 7f0ce67fc640 1 -- 172.21.15.7:0/1367036272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ce0010d40 con 0x7f0cf0106800 2024-09-21T05:03:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.831+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf01a3ea0 con 0x7f0cf0106800 2024-09-21T05:03:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.832+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf0101f40 con 0x7f0cf0106800 2024-09-21T05:03:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.832+0000 7f0ce67fc640 1 -- 172.21.15.7:0/1367036272 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0ce001b460 con 0x7f0cf0106800 2024-09-21T05:03:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.833+0000 7f0ce67fc640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cdc07af40 0x7f0cdc07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.833+0000 7f0ce67fc640 1 -- 172.21.15.7:0/1367036272 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0ce009d750 con 0x7f0cf0106800 2024-09-21T05:03:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.833+0000 7f0cf575e640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cdc07af40 0x7f0cdc07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.834+0000 7f0cf575e640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cdc07af40 0x7f0cdc07d400 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0096c0 tx=0x7f0cec031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:57.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:57.838+0000 7f0ce67fc640 1 -- 172.21.15.7:0/1367036272 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ce0068cf0 con 0x7f0cf0106800 2024-09-21T05:03:58.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.046+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0cf0062630 con 0x7f0cdc07af40 2024-09-21T05:03:58.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.047+0000 7f0ce67fc640 1 -- 172.21.15.7:0/1367036272 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f0cf0062630 con 0x7f0cdc07af40 2024-09-21T05:03:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cdc07af40 msgr2=0x7f0cdc07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cdc07af40 0x7f0cdc07d400 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0096c0 tx=0x7f0cec031400 comp rx=0 tx=0).stop 2024-09-21T05:03:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 msgr2=0x7f0cf00fd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 0x7f0cf00fd1b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000b420 tx=0x7f0ce000b8e0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 shutdown_connections 2024-09-21T05:03:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cdc07af40 0x7f0cdc07d400 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf0106800 0x7f0cf00fd1b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 --2- 172.21.15.7:0/1367036272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf0100800 0x7f0cf00fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 >> 172.21.15.7:0/1367036272 conn(0x7f0cf00fc520 msgr2=0x7f0cf006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 shutdown_connections 2024-09-21T05:03:58.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.049+0000 7f0cf79c2640 1 -- 172.21.15.7:0/1367036272 wait complete. 2024-09-21T05:03:58.061 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:03:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.134+0000 7f0490727640 1 -- 172.21.15.7:0/588481473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 msgr2=0x7f0488100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.134+0000 7f0490727640 1 --2- 172.21.15.7:0/588481473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 0x7f0488100c60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0478009880 tx=0x7f047802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.134+0000 7f0490727640 1 -- 172.21.15.7:0/588481473 shutdown_connections 2024-09-21T05:03:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.134+0000 7f0490727640 1 --2- 172.21.15.7:0/588481473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 0x7f0488100c60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.134+0000 7f0490727640 1 --2- 172.21.15.7:0/588481473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0488106800 0x7f0488106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.134+0000 7f0490727640 1 -- 172.21.15.7:0/588481473 >> 172.21.15.7:0/588481473 conn(0x7f04880fc520 msgr2=0x7f04880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.134+0000 7f0490727640 1 -- 172.21.15.7:0/588481473 shutdown_connections 2024-09-21T05:03:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.134+0000 7f0490727640 1 -- 172.21.15.7:0/588481473 wait complete. 2024-09-21T05:03:58.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.135+0000 7f0490727640 1 Processor -- start 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.135+0000 7f0490727640 1 -- start start 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.135+0000 7f0490727640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 0x7f048819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.135+0000 7f0490727640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0488106800 0x7f048819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.135+0000 7f0490727640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f048810dd30 con 0x7f0488100800 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.135+0000 7f0490727640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f048810dbb0 con 0x7f0488106800 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.135+0000 7f048e4c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 0x7f048819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f048e4c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 0x7f048819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56852/0 (socket says 172.21.15.7:56852) 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f048e4c3640 1 -- 172.21.15.7:0/1389304875 learned_addr learned my addr 172.21.15.7:0/1389304875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f048dcc2640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0488106800 0x7f048819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f048e4c3640 1 -- 172.21.15.7:0/1389304875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0488106800 msgr2=0x7f048819dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f048e4c3640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0488106800 0x7f048819dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f048e4c3640 1 -- 172.21.15.7:0/1389304875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488197850 con 0x7f0488100800 2024-09-21T05:03:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f048e4c3640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 0x7f048819d6d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f048400b420 tx=0x7f048400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f047f7fe640 1 -- 172.21.15.7:0/1389304875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0484011630 con 0x7f0488100800 2024-09-21T05:03:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f047f7fe640 1 -- 172.21.15.7:0/1389304875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0484016070 con 0x7f0488100800 2024-09-21T05:03:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0488197b40 con 0x7f0488100800 2024-09-21T05:03:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f047f7fe640 1 -- 172.21.15.7:0/1389304875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0484010d40 con 0x7f0488100800 2024-09-21T05:03:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.136+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04881980a0 con 0x7f0488100800 2024-09-21T05:03:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.137+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0488101f40 con 0x7f0488100800 2024-09-21T05:03:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.138+0000 7f047f7fe640 1 -- 172.21.15.7:0/1389304875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f048401b460 con 0x7f0488100800 2024-09-21T05:03:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.139+0000 7f047f7fe640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f047407af80 0x7f047407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.139+0000 7f048dcc2640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f047407af80 0x7f047407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.139+0000 7f047f7fe640 1 -- 172.21.15.7:0/1389304875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f048409d650 con 0x7f0488100800 2024-09-21T05:03:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.140+0000 7f048dcc2640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f047407af80 0x7f047407d440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0488198ba0 tx=0x7f0478031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.143+0000 7f047f7fe640 1 -- 172.21.15.7:0/1389304875 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0484068c70 con 0x7f0488100800 2024-09-21T05:03:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.349+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0488198410 con 0x7f047407af80 2024-09-21T05:03:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.350+0000 7f047f7fe640 1 -- 172.21.15.7:0/1389304875 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f0488198410 con 0x7f047407af80 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f047407af80 msgr2=0x7f047407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f047407af80 0x7f047407d440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0488198ba0 tx=0x7f0478031400 comp rx=0 tx=0).stop 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 msgr2=0x7f048819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 0x7f048819d6d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f048400b420 tx=0x7f048400b8e0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 shutdown_connections 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f047407af80 0x7f047407d440 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0488106800 0x7f048819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 --2- 172.21.15.7:0/1389304875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0488100800 0x7f048819d6d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.352+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 >> 172.21.15.7:0/1389304875 conn(0x7f04880fc520 msgr2=0x7f048806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.353+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 shutdown_connections 2024-09-21T05:03:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.353+0000 7f0490727640 1 -- 172.21.15.7:0/1389304875 wait complete. 2024-09-21T05:03:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 -- 172.21.15.7:0/2144101087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 msgr2=0x7fb554106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 --2- 172.21.15.7:0/2144101087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 0x7fb554106bd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb544009880 tx=0x7fb54402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 -- 172.21.15.7:0/2144101087 shutdown_connections 2024-09-21T05:03:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 --2- 172.21.15.7:0/2144101087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5541007f0 0x7fb554100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 --2- 172.21.15.7:0/2144101087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 0x7fb554106bd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 -- 172.21.15.7:0/2144101087 >> 172.21.15.7:0/2144101087 conn(0x7fb5540fc550 msgr2=0x7fb5540fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 -- 172.21.15.7:0/2144101087 shutdown_connections 2024-09-21T05:03:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 -- 172.21.15.7:0/2144101087 wait complete. 2024-09-21T05:03:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 Processor -- start 2024-09-21T05:03:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.437+0000 7fb55a95c640 1 -- start start 2024-09-21T05:03:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb55a95c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5541007f0 0x7fb55419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb55a95c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 0x7fb55419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb55a95c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410dcc0 con 0x7fb5541067f0 2024-09-21T05:03:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb55a95c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410db40 con 0x7fb5541007f0 2024-09-21T05:03:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb5537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 0x7fb55419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb5537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 0x7fb55419dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56876/0 (socket says 172.21.15.7:56876) 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb5537fe640 1 -- 172.21.15.7:0/2946602504 learned_addr learned my addr 172.21.15.7:0/2946602504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb553fff640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5541007f0 0x7fb55419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb5537fe640 1 -- 172.21.15.7:0/2946602504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5541007f0 msgr2=0x7fb55419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb5537fe640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5541007f0 0x7fb55419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.438+0000 7fb5537fe640 1 -- 172.21.15.7:0/2946602504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5541977e0 con 0x7fb5541067f0 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.439+0000 7fb5537fe640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 0x7fb55419dba0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb54800d6f0 tx=0x7fb54800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.439+0000 7fb5517fa640 1 -- 172.21.15.7:0/2946602504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb548017070 con 0x7fb5541067f0 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.439+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb554197ad0 con 0x7fb5541067f0 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.439+0000 7fb5517fa640 1 -- 172.21.15.7:0/2946602504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb54800de00 con 0x7fb5541067f0 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.439+0000 7fb5517fa640 1 -- 172.21.15.7:0/2946602504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb548004bb0 con 0x7fb5541067f0 2024-09-21T05:03:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.439+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb554198060 con 0x7fb5541067f0 2024-09-21T05:03:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.440+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb554101f30 con 0x7fb5541067f0 2024-09-21T05:03:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.441+0000 7fb5517fa640 1 -- 172.21.15.7:0/2946602504 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb548009a30 con 0x7fb5541067f0 2024-09-21T05:03:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.441+0000 7fb5517fa640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb53c07af80 0x7fb53c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.441+0000 7fb5517fa640 1 -- 172.21.15.7:0/2946602504 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fb54809db00 con 0x7fb5541067f0 2024-09-21T05:03:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.441+0000 7fb553fff640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb53c07af80 0x7fb53c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.442+0000 7fb553fff640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb53c07af80 0x7fb53c07d440 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5440096c0 tx=0x7fb544031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:58.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.446+0000 7fb5517fa640 1 -- 172.21.15.7:0/2946602504 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb548069120 con 0x7fb5541067f0 2024-09-21T05:03:58.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.643+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb554198310 con 0x7fb53c07af80 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.658+0000 7fb5517fa640 1 -- 172.21.15.7:0/2946602504 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fb554198310 con 0x7fb53c07af80 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (25m) 4m ago 36m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (29m) 4m ago 37m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (28m) 4m ago 36m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (24m) 4m ago 37m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (24m) 4m ago 36m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (25m) 4m ago 36m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:03:58.661 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (20m) 4m ago 31m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (20m) 4m ago 31m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (4m) 4m ago 31m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (19m) 4m ago 31m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (28m) 4m ago 38m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (27m) 4m ago 36m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (24m) 4m ago 38m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (24m) 4m ago 36m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (26m) 4m ago 37m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (26m) 4m ago 36m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (23m) 4m ago 34m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (23m) 4m ago 34m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (22m) 4m ago 33m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (21m) 4m ago 33m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (21m) 4m ago 33m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:03:58.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (20m) 4m ago 32m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:03:58.663 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (26m) 4m ago 36m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb53c07af80 msgr2=0x7fb53c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb53c07af80 0x7fb53c07d440 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5440096c0 tx=0x7fb544031400 comp rx=0 tx=0).stop 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 msgr2=0x7fb55419dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 0x7fb55419dba0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb54800d6f0 tx=0x7fb54800dbb0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 shutdown_connections 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb53c07af80 0x7fb53c07d440 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5541067f0 0x7fb55419dba0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 --2- 172.21.15.7:0/2946602504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb5541007f0 0x7fb55419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 >> 172.21.15.7:0/2946602504 conn(0x7fb5540fc550 msgr2=0x7fb554069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:58.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 shutdown_connections 2024-09-21T05:03:58.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.661+0000 7fb55a95c640 1 -- 172.21.15.7:0/2946602504 wait complete. 2024-09-21T05:03:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:58 smithi007 ceph-mon[108238]: pgmap v877: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:58 smithi007 ceph-mon[108238]: from='client.35404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:58.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.747+0000 7fbebca50640 1 -- 172.21.15.7:0/2461132765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8106800 msgr2=0x7fbeb8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.747+0000 7fbebca50640 1 --2- 172.21.15.7:0/2461132765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8106800 0x7fbeb8106be0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009880 tx=0x7fbea802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.747+0000 7fbebca50640 1 -- 172.21.15.7:0/2461132765 shutdown_connections 2024-09-21T05:03:58.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.747+0000 7fbebca50640 1 --2- 172.21.15.7:0/2461132765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbeb8100800 0x7fbeb8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.747+0000 7fbebca50640 1 --2- 172.21.15.7:0/2461132765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8106800 0x7fbeb8106be0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.747+0000 7fbebca50640 1 -- 172.21.15.7:0/2461132765 >> 172.21.15.7:0/2461132765 conn(0x7fbeb80fc520 msgr2=0x7fbeb80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:58.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.747+0000 7fbebca50640 1 -- 172.21.15.7:0/2461132765 shutdown_connections 2024-09-21T05:03:58.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.747+0000 7fbebca50640 1 -- 172.21.15.7:0/2461132765 wait complete. 2024-09-21T05:03:58.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbebca50640 1 Processor -- start 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbebca50640 1 -- start start 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbebca50640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8100800 0x7fbeb810b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbebca50640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbeb8106800 0x7fbeb810bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbebca50640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb810edf0 con 0x7fbeb8100800 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbebca50640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb810ec70 con 0x7fbeb8106800 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbeb659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8100800 0x7fbeb810b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbeb659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8100800 0x7fbeb810b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56886/0 (socket says 172.21.15.7:56886) 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbeb659c640 1 -- 172.21.15.7:0/3426872864 learned_addr learned my addr 172.21.15.7:0/3426872864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.748+0000 7fbeaffff640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbeb8106800 0x7fbeb810bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbeb659c640 1 -- 172.21.15.7:0/3426872864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbeb8106800 msgr2=0x7fbeb810bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbeb659c640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbeb8106800 0x7fbeb810bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:58.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbeb659c640 1 -- 172.21.15.7:0/3426872864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb810c210 con 0x7fbeb8100800 2024-09-21T05:03:58.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbeb659c640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8100800 0x7fbeb810b790 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbea8031460 tx=0x7fbea80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:58.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbeaf7fe640 1 -- 172.21.15.7:0/3426872864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbea803f070 con 0x7fbeb8100800 2024-09-21T05:03:58.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb810c4a0 con 0x7fbeb8100800 2024-09-21T05:03:58.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbeaf7fe640 1 -- 172.21.15.7:0/3426872864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbea80035f0 con 0x7fbeb8100800 2024-09-21T05:03:58.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbeaf7fe640 1 -- 172.21.15.7:0/3426872864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbea8035030 con 0x7fbeb8100800 2024-09-21T05:03:58.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.749+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a3db0 con 0x7fbeb8100800 2024-09-21T05:03:58.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.750+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb8101f40 con 0x7fbeb8100800 2024-09-21T05:03:58.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.751+0000 7fbeaf7fe640 1 -- 172.21.15.7:0/3426872864 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbea8042070 con 0x7fbeb8100800 2024-09-21T05:03:58.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.752+0000 7fbeaf7fe640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbea407af30 0x7fbea407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:58.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.752+0000 7fbeaf7fe640 1 -- 172.21.15.7:0/3426872864 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbea80be9d0 con 0x7fbeb8100800 2024-09-21T05:03:58.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.752+0000 7fbeaffff640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbea407af30 0x7fbea407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:58.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.752+0000 7fbeaffff640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbea407af30 0x7fbea407d3f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbeb810d220 tx=0x7fbea000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:58.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:58.756+0000 7fbeaf7fe640 1 -- 172.21.15.7:0/3426872864 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbea8089f70 con 0x7fbeb8100800 2024-09-21T05:03:58.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:58 smithi053 ceph-mon[80176]: pgmap v877: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:03:58.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:58 smithi053 ceph-mon[80176]: from='client.35404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:58.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:03:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.024+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fbeb8108d70 con 0x7fbeb8100800 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.025+0000 7fbeaf7fe640 1 -- 172.21.15.7:0/3426872864 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fbea808ee20 con 0x7fbeb8100800 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:03:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:03:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:03:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:03:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:03:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:03:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:03:59.029 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:03:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbea407af30 msgr2=0x7fbea407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbea407af30 0x7fbea407d3f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbeb810d220 tx=0x7fbea000b390 comp rx=0 tx=0).stop 2024-09-21T05:03:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8100800 msgr2=0x7fbeb810b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8100800 0x7fbeb810b790 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbea8031460 tx=0x7fbea80045e0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 shutdown_connections 2024-09-21T05:03:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbea407af30 0x7fbea407d3f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbeb8106800 0x7fbeb810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 --2- 172.21.15.7:0/3426872864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbeb8100800 0x7fbeb810b790 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 >> 172.21.15.7:0/3426872864 conn(0x7fbeb80fc520 msgr2=0x7fbeb80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 shutdown_connections 2024-09-21T05:03:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.027+0000 7fbebca50640 1 -- 172.21.15.7:0/3426872864 wait complete. 2024-09-21T05:03:59.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.112+0000 7f1fa1360640 1 -- 172.21.15.7:0/1925038274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 msgr2=0x7f1f9c10dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.112+0000 7f1fa1360640 1 --2- 172.21.15.7:0/1925038274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 0x7f1f9c10dcc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009880 tx=0x7f1f8402a490 comp rx=0 tx=0).stop 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.112+0000 7f1fa1360640 1 -- 172.21.15.7:0/1925038274 shutdown_connections 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.112+0000 7f1fa1360640 1 --2- 172.21.15.7:0/1925038274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 0x7f1f9c10dcc0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.112+0000 7f1fa1360640 1 --2- 172.21.15.7:0/1925038274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9c100650 0x7f1f9c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.112+0000 7f1fa1360640 1 -- 172.21.15.7:0/1925038274 >> 172.21.15.7:0/1925038274 conn(0x7f1f9c0fc520 msgr2=0x7f1f9c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.112+0000 7f1fa1360640 1 -- 172.21.15.7:0/1925038274 shutdown_connections 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.112+0000 7f1fa1360640 1 -- 172.21.15.7:0/1925038274 wait complete. 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1fa1360640 1 Processor -- start 2024-09-21T05:03:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1fa1360640 1 -- start start 2024-09-21T05:03:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1fa1360640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9c100650 0x7f1f9c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1fa1360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 0x7f1f9c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1fa1360640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c110f90 con 0x7f1f9c101000 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1fa1360640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c110e10 con 0x7f1f9c100650 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1f93fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 0x7f1f9c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1f93fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 0x7f1f9c19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56896/0 (socket says 172.21.15.7:56896) 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1f93fff640 1 -- 172.21.15.7:0/3618652003 learned_addr learned my addr 172.21.15.7:0/3618652003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.113+0000 7f1f9affd640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9c100650 0x7f1f9c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1f93fff640 1 -- 172.21.15.7:0/3618652003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9c100650 msgr2=0x7f1f9c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1f93fff640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9c100650 0x7f1f9c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1f93fff640 1 -- 172.21.15.7:0/3618652003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c199a20 con 0x7f1f9c101000 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1f93fff640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 0x7f1f9c19fda0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1f84031460 tx=0x7f1f840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1f98ff9640 1 -- 172.21.15.7:0/3618652003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1f8403f070 con 0x7f1f9c101000 2024-09-21T05:03:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c199cb0 con 0x7f1f9c101000 2024-09-21T05:03:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1f98ff9640 1 -- 172.21.15.7:0/3618652003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f840035f0 con 0x7f1f9c101000 2024-09-21T05:03:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1f98ff9640 1 -- 172.21.15.7:0/3618652003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1f84035030 con 0x7f1f9c101000 2024-09-21T05:03:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.114+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c19a240 con 0x7f1f9c101000 2024-09-21T05:03:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.115+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c06aa90 con 0x7f1f9c101000 2024-09-21T05:03:59.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.116+0000 7f1f98ff9640 1 -- 172.21.15.7:0/3618652003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1f84042070 con 0x7f1f9c101000 2024-09-21T05:03:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.116+0000 7f1f98ff9640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f8807af30 0x7f1f8807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.117+0000 7f1f98ff9640 1 -- 172.21.15.7:0/3618652003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f1f840bea70 con 0x7f1f9c101000 2024-09-21T05:03:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.117+0000 7f1f9affd640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f8807af30 0x7f1f8807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.117+0000 7f1f9affd640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f8807af30 0x7f1f8807d3f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c19ab60 tx=0x7f1f8c00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.121+0000 7f1f98ff9640 1 -- 172.21.15.7:0/3618652003 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1f84089f70 con 0x7f1f9c101000 2024-09-21T05:03:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.348+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1f9c10b3a0 con 0x7f1f9c101000 2024-09-21T05:03:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.348+0000 7f1f98ff9640 1 -- 172.21.15.7:0/3618652003 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f1f8408ee20 con 0x7f1f9c101000 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:03:59.352 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:03:59.353 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:03:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:03:59.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:03:59.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:03:59.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:03:59.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:03:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f8807af30 msgr2=0x7f1f8807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f8807af30 0x7f1f8807d3f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c19ab60 tx=0x7f1f8c00b390 comp rx=0 tx=0).stop 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 msgr2=0x7f1f9c19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 0x7f1f9c19fda0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1f84031460 tx=0x7f1f840045e0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 shutdown_connections 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1f8807af30 0x7f1f8807d3f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101000 0x7f1f9c19fda0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 --2- 172.21.15.7:0/3618652003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9c100650 0x7f1f9c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 >> 172.21.15.7:0/3618652003 conn(0x7f1f9c0fc520 msgr2=0x7f1f9c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 shutdown_connections 2024-09-21T05:03:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.351+0000 7f1fa1360640 1 -- 172.21.15.7:0/3618652003 wait complete. 2024-09-21T05:03:59.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.436+0000 7faf42696640 1 -- 172.21.15.7:0/3767249596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c102b20 msgr2=0x7faf3c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.436+0000 7faf42696640 1 --2- 172.21.15.7:0/3767249596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c102b20 0x7faf3c102f00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0098b0 tx=0x7faf2c02a560 comp rx=0 tx=0).stop 2024-09-21T05:03:59.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.436+0000 7faf42696640 1 -- 172.21.15.7:0/3767249596 shutdown_connections 2024-09-21T05:03:59.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.436+0000 7faf42696640 1 --2- 172.21.15.7:0/3767249596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c103440 0x7faf3c10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.436+0000 7faf42696640 1 --2- 172.21.15.7:0/3767249596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c102b20 0x7faf3c102f00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.436+0000 7faf42696640 1 -- 172.21.15.7:0/3767249596 >> 172.21.15.7:0/3767249596 conn(0x7faf3c0fc520 msgr2=0x7faf3c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:59.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.436+0000 7faf42696640 1 -- 172.21.15.7:0/3767249596 shutdown_connections 2024-09-21T05:03:59.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.436+0000 7faf42696640 1 -- 172.21.15.7:0/3767249596 wait complete. 2024-09-21T05:03:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf42696640 1 Processor -- start 2024-09-21T05:03:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf42696640 1 -- start start 2024-09-21T05:03:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf42696640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c102b20 0x7faf3c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf42696640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c103440 0x7faf3c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf42696640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c10edd0 con 0x7faf3c103440 2024-09-21T05:03:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf42696640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c10ec50 con 0x7faf3c102b20 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf3b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c103440 0x7faf3c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf3b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c103440 0x7faf3c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56908/0 (socket says 172.21.15.7:56908) 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf3b7fe640 1 -- 172.21.15.7:0/421366898 learned_addr learned my addr 172.21.15.7:0/421366898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.437+0000 7faf3bfff640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c102b20 0x7faf3c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf3b7fe640 1 -- 172.21.15.7:0/421366898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c102b20 msgr2=0x7faf3c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf3b7fe640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c102b20 0x7faf3c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf3b7fe640 1 -- 172.21.15.7:0/421366898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c197800 con 0x7faf3c103440 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf3b7fe640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c103440 0x7faf3c19dbc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7faf3000d8d0 tx=0x7faf3000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf397fa640 1 -- 172.21.15.7:0/421366898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faf30017030 con 0x7faf3c103440 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf397fa640 1 -- 172.21.15.7:0/421366898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faf30004020 con 0x7faf3c103440 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3c197af0 con 0x7faf3c103440 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf397fa640 1 -- 172.21.15.7:0/421366898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faf300047a0 con 0x7faf3c103440 2024-09-21T05:03:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.438+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3c198080 con 0x7faf3c103440 2024-09-21T05:03:59.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.439+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3c06aa30 con 0x7faf3c103440 2024-09-21T05:03:59.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.440+0000 7faf397fa640 1 -- 172.21.15.7:0/421366898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7faf300041d0 con 0x7faf3c103440 2024-09-21T05:03:59.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.440+0000 7faf397fa640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf2407af30 0x7faf2407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.441+0000 7faf397fa640 1 -- 172.21.15.7:0/421366898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7faf30018070 con 0x7faf3c103440 2024-09-21T05:03:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.441+0000 7faf3bfff640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf2407af30 0x7faf2407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.441+0000 7faf3bfff640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf2407af30 0x7faf2407d3f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf2c000c00 tx=0x7faf2c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.445+0000 7faf397fa640 1 -- 172.21.15.7:0/421366898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf30069380 con 0x7faf3c103440 2024-09-21T05:03:59.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.648+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7faf3c198390 con 0x7faf2407af30 2024-09-21T05:03:59.651 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:59 smithi007 ceph-mon[108238]: from='client.35408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:59.651 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:59 smithi007 ceph-mon[108238]: from='client.35412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:59.651 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:59 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3426872864' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:03:59.651 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:03:59 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3618652003' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:03:59.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.649+0000 7faf397fa640 1 -- 172.21.15.7:0/421366898 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7faf3c198390 con 0x7faf2407af30 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:03:59.652 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:03:59.653 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:03:59.653 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:03:59.653 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:03:59.653 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:03:59.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf2407af30 msgr2=0x7faf2407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf2407af30 0x7faf2407d3f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf2c000c00 tx=0x7faf2c031400 comp rx=0 tx=0).stop 2024-09-21T05:03:59.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c103440 msgr2=0x7faf3c19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c103440 0x7faf3c19dbc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7faf3000d8d0 tx=0x7faf3000dd90 comp rx=0 tx=0).stop 2024-09-21T05:03:59.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 shutdown_connections 2024-09-21T05:03:59.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf2407af30 0x7faf2407d3f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf3c103440 0x7faf3c19dbc0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 --2- 172.21.15.7:0/421366898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf3c102b20 0x7faf3c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 >> 172.21.15.7:0/421366898 conn(0x7faf3c0fc520 msgr2=0x7faf3c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 shutdown_connections 2024-09-21T05:03:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.651+0000 7faf42696640 1 -- 172.21.15.7:0/421366898 wait complete. 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.735+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/88454825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100250 msgr2=0x7fb8e0100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.735+0000 7fb8e5ee9640 1 --2- 172.21.15.7:0/88454825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100250 0x7fb8e0100630 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009880 tx=0x7fb8d002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.735+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/88454825 shutdown_connections 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.735+0000 7fb8e5ee9640 1 --2- 172.21.15.7:0/88454825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8e0100c00 0x7fb8e0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.735+0000 7fb8e5ee9640 1 --2- 172.21.15.7:0/88454825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100250 0x7fb8e0100630 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.735+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/88454825 >> 172.21.15.7:0/88454825 conn(0x7fb8e00fbc90 msgr2=0x7fb8e00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.735+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/88454825 shutdown_connections 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.735+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/88454825 wait complete. 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.736+0000 7fb8e5ee9640 1 Processor -- start 2024-09-21T05:03:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.736+0000 7fb8e5ee9640 1 -- start start 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.736+0000 7fb8e5ee9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8e0100250 0x7fb8e019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.736+0000 7fb8e5ee9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100c00 0x7fb8e019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.736+0000 7fb8e5ee9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010dd50 con 0x7fb8e0100c00 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.736+0000 7fb8e5ee9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010dbd0 con 0x7fb8e0100250 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.736+0000 7fb8deffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100c00 0x7fb8e019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8deffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100c00 0x7fb8e019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56928/0 (socket says 172.21.15.7:56928) 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8deffd640 1 -- 172.21.15.7:0/2073085735 learned_addr learned my addr 172.21.15.7:0/2073085735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8df7fe640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8e0100250 0x7fb8e019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8deffd640 1 -- 172.21.15.7:0/2073085735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8e0100250 msgr2=0x7fb8e019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8deffd640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8e0100250 0x7fb8e019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:03:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8deffd640 1 -- 172.21.15.7:0/2073085735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e0197840 con 0x7fb8e0100c00 2024-09-21T05:03:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8deffd640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100c00 0x7fb8e019dc00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400d6f0 tx=0x7fb8d400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8dcff9640 1 -- 172.21.15.7:0/2073085735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb8d4017070 con 0x7fb8e0100c00 2024-09-21T05:03:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8dcff9640 1 -- 172.21.15.7:0/2073085735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb8d400de00 con 0x7fb8e0100c00 2024-09-21T05:03:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8dcff9640 1 -- 172.21.15.7:0/2073085735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb8d4004bb0 con 0x7fb8e0100c00 2024-09-21T05:03:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e0197b30 con 0x7fb8e0100c00 2024-09-21T05:03:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.737+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01980c0 con 0x7fb8e0100c00 2024-09-21T05:03:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.738+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e0101730 con 0x7fb8e0100c00 2024-09-21T05:03:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.739+0000 7fb8dcff9640 1 -- 172.21.15.7:0/2073085735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb8d4009a30 con 0x7fb8e0100c00 2024-09-21T05:03:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.740+0000 7fb8dcff9640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8c807af80 0x7fb8c807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:03:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.740+0000 7fb8df7fe640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8c807af80 0x7fb8c807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:03:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.740+0000 7fb8dcff9640 1 -- 172.21.15.7:0/2073085735 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fb8d409db20 con 0x7fb8e0100c00 2024-09-21T05:03:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.740+0000 7fb8df7fe640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8c807af80 0x7fb8c807d440 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00096c0 tx=0x7fb8d0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:03:59.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:03:59.744+0000 7fb8dcff9640 1 -- 172.21.15.7:0/2073085735 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8d4069140 con 0x7fb8e0100c00 2024-09-21T05:03:59.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:59 smithi053 ceph-mon[80176]: from='client.35408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:59.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:59 smithi053 ceph-mon[80176]: from='client.35412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:03:59.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:59 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3426872864' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:03:59.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:03:59 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3618652003' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:04:00.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.002+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fb8e010a570 con 0x7fb8e0100c00 2024-09-21T05:04:00.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.003+0000 7fb8dcff9640 1 -- 172.21.15.7:0/2073085735 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fb8d406dff0 con 0x7fb8e0100c00 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:04:00.006 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:04:00.007 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:04:00.008 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:04:00.008 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:04:00.008 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:04:00.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8c807af80 msgr2=0x7fb8c807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:00.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8c807af80 0x7fb8c807d440 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00096c0 tx=0x7fb8d0031400 comp rx=0 tx=0).stop 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100c00 msgr2=0x7fb8e019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100c00 0x7fb8e019dc00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400d6f0 tx=0x7fb8d400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 shutdown_connections 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8c807af80 0x7fb8c807d440 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e0100c00 0x7fb8e019dc00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 --2- 172.21.15.7:0/2073085735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8e0100250 0x7fb8e019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.005+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 >> 172.21.15.7:0/2073085735 conn(0x7fb8e00fbc90 msgr2=0x7fb8e00fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.006+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 shutdown_connections 2024-09-21T05:04:00.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:00.006+0000 7fb8e5ee9640 1 -- 172.21.15.7:0/2073085735 wait complete. 2024-09-21T05:04:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:00 smithi053 ceph-mon[80176]: pgmap v878: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:00 smithi053 ceph-mon[80176]: from='client.35424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:00 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2073085735' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:04:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:00 smithi007 ceph-mon[108238]: pgmap v878: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:00 smithi007 ceph-mon[108238]: from='client.35424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:00 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2073085735' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:04:02.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:02 smithi053 ceph-mon[80176]: pgmap v879: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:02 smithi007 ceph-mon[108238]: pgmap v879: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:04.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:04 smithi007 ceph-mon[108238]: pgmap v880: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:04.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:04 smithi053 ceph-mon[80176]: pgmap v880: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:06.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:06 smithi053 ceph-mon[80176]: pgmap v881: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:06.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:06 smithi007 ceph-mon[108238]: pgmap v881: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:08.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:08 smithi053 ceph-mon[80176]: pgmap v882: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:08 smithi007 ceph-mon[108238]: pgmap v882: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:10.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:10 smithi007 ceph-mon[108238]: pgmap v883: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:10.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:10 smithi053 ceph-mon[80176]: pgmap v883: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:12.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:12 smithi053 ceph-mon[80176]: pgmap v884: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:12 smithi007 ceph-mon[108238]: pgmap v884: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:13.886 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:13 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:04:13.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:13 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:04:14.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:14 smithi053 ceph-mon[80176]: pgmap v885: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:14 smithi007 ceph-mon[108238]: pgmap v885: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:16.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:16 smithi007 ceph-mon[108238]: pgmap v886: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:16.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:16 smithi053 ceph-mon[80176]: pgmap v886: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:18.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:18 smithi053 ceph-mon[80176]: pgmap v887: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:18.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:18 smithi007 ceph-mon[108238]: pgmap v887: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:20.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:20 smithi053 ceph-mon[80176]: pgmap v888: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:20.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:20 smithi007 ceph-mon[108238]: pgmap v888: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:22.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:22 smithi007 ceph-mon[108238]: pgmap v889: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:22.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:22 smithi053 ceph-mon[80176]: pgmap v889: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:24 smithi053 ceph-mon[80176]: pgmap v890: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:24 smithi007 ceph-mon[108238]: pgmap v890: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:26 smithi053 ceph-mon[80176]: pgmap v891: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:26.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:26 smithi007 ceph-mon[108238]: pgmap v891: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:28.932 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:28 smithi007 ceph-mon[108238]: pgmap v892: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:28.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:04:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:28 smithi053 ceph-mon[80176]: pgmap v892: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:04:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.102+0000 7fc17431a640 1 -- 172.21.15.7:0/230873676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 msgr2=0x7fc16c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.102+0000 7fc17431a640 1 --2- 172.21.15.7:0/230873676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 0x7fc16c101d60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc160009880 tx=0x7fc16002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.102+0000 7fc17431a640 1 -- 172.21.15.7:0/230873676 shutdown_connections 2024-09-21T05:04:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.102+0000 7fc17431a640 1 --2- 172.21.15.7:0/230873676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 0x7fc16c101d60 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.102+0000 7fc17431a640 1 --2- 172.21.15.7:0/230873676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc16c107900 0x7fc16c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.102+0000 7fc17431a640 1 -- 172.21.15.7:0/230873676 >> 172.21.15.7:0/230873676 conn(0x7fc16c0fd600 msgr2=0x7fc16c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.102+0000 7fc17431a640 1 -- 172.21.15.7:0/230873676 shutdown_connections 2024-09-21T05:04:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.102+0000 7fc17431a640 1 -- 172.21.15.7:0/230873676 wait complete. 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.103+0000 7fc17431a640 1 Processor -- start 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.103+0000 7fc17431a640 1 -- start start 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.103+0000 7fc17431a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 0x7fc16c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.103+0000 7fc17431a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc16c107900 0x7fc16c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc17431a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c10edb0 con 0x7fc16c101900 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc17431a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c10ec30 con 0x7fc16c107900 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1720b6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 0x7fc16c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1720b6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 0x7fc16c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45324/0 (socket says 172.21.15.7:45324) 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1720b6640 1 -- 172.21.15.7:0/3831708891 learned_addr learned my addr 172.21.15.7:0/3831708891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1718b5640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc16c107900 0x7fc16c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1720b6640 1 -- 172.21.15.7:0/3831708891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc16c107900 msgr2=0x7fc16c19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1720b6640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc16c107900 0x7fc16c19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1720b6640 1 -- 172.21.15.7:0/3831708891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c1977d0 con 0x7fc16c101900 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1718b5640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc16c107900 0x7fc16c19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc1720b6640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 0x7fc16c19d650 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc16800cc60 tx=0x7fc16800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc15f7fe640 1 -- 172.21.15.7:0/3831708891 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc16800ad30 con 0x7fc16c101900 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc15f7fe640 1 -- 172.21.15.7:0/3831708891 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc168005020 con 0x7fc16c101900 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c197ac0 con 0x7fc16c101900 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.104+0000 7fc15f7fe640 1 -- 172.21.15.7:0/3831708891 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc16800f660 con 0x7fc16c101900 2024-09-21T05:04:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.105+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c198050 con 0x7fc16c101900 2024-09-21T05:04:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.106+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16c103040 con 0x7fc16c101900 2024-09-21T05:04:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.106+0000 7fc15f7fe640 1 -- 172.21.15.7:0/3831708891 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc168004b20 con 0x7fc16c101900 2024-09-21T05:04:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.107+0000 7fc15f7fe640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc15807af30 0x7fc15807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.107+0000 7fc15f7fe640 1 -- 172.21.15.7:0/3831708891 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc16809e640 con 0x7fc16c101900 2024-09-21T05:04:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.107+0000 7fc1718b5640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc15807af30 0x7fc15807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.107+0000 7fc1718b5640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc15807af30 0x7fc15807d3f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc160031460 tx=0x7fc1600095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.111+0000 7fc15f7fe640 1 -- 172.21.15.7:0/3831708891 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc168014030 con 0x7fc16c101900 2024-09-21T05:04:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.317+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc16c198360 con 0x7fc15807af30 2024-09-21T05:04:30.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.318+0000 7fc15f7fe640 1 -- 172.21.15.7:0/3831708891 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc16c198360 con 0x7fc15807af30 2024-09-21T05:04:30.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc15807af30 msgr2=0x7fc15807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc15807af30 0x7fc15807d3f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc160031460 tx=0x7fc1600095f0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 msgr2=0x7fc16c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 0x7fc16c19d650 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc16800cc60 tx=0x7fc16800a620 comp rx=0 tx=0).stop 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 shutdown_connections 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc15807af30 0x7fc15807d3f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc16c107900 0x7fc16c19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 --2- 172.21.15.7:0/3831708891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc16c101900 0x7fc16c19d650 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 >> 172.21.15.7:0/3831708891 conn(0x7fc16c0fd600 msgr2=0x7fc16c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 shutdown_connections 2024-09-21T05:04:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.320+0000 7fc17431a640 1 -- 172.21.15.7:0/3831708891 wait complete. 2024-09-21T05:04:30.333 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:04:30.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.406+0000 7f10cc0d9640 1 -- 172.21.15.7:0/1790489514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 msgr2=0x7f10c410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.406+0000 7f10cc0d9640 1 --2- 172.21.15.7:0/1790489514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 0x7f10c410b8d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009880 tx=0x7f10b402a530 comp rx=0 tx=0).stop 2024-09-21T05:04:30.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.406+0000 7f10cc0d9640 1 -- 172.21.15.7:0/1790489514 shutdown_connections 2024-09-21T05:04:30.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.406+0000 7f10cc0d9640 1 --2- 172.21.15.7:0/1790489514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 0x7f10c410b8d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.406+0000 7f10cc0d9640 1 --2- 172.21.15.7:0/1790489514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c4102ac0 0x7f10c4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.406+0000 7f10cc0d9640 1 -- 172.21.15.7:0/1790489514 >> 172.21.15.7:0/1790489514 conn(0x7f10c40fc520 msgr2=0x7f10c40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.406+0000 7f10cc0d9640 1 -- 172.21.15.7:0/1790489514 shutdown_connections 2024-09-21T05:04:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.406+0000 7f10cc0d9640 1 -- 172.21.15.7:0/1790489514 wait complete. 2024-09-21T05:04:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10cc0d9640 1 Processor -- start 2024-09-21T05:04:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10cc0d9640 1 -- start start 2024-09-21T05:04:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10cc0d9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c4102ac0 0x7f10c419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10cc0d9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 0x7f10c419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10cc0d9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f10c410edd0 con 0x7f10c41033e0 2024-09-21T05:04:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10cc0d9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10c410ec50 con 0x7f10c4102ac0 2024-09-21T05:04:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10c9674640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 0x7f10c419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10c9674640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 0x7f10c419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45342/0 (socket says 172.21.15.7:45342) 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10c9674640 1 -- 172.21.15.7:0/980262400 learned_addr learned my addr 172.21.15.7:0/980262400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.407+0000 7f10c9e75640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c4102ac0 0x7f10c419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10c9674640 1 -- 172.21.15.7:0/980262400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c4102ac0 msgr2=0x7f10c419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10c9674640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c4102ac0 0x7f10c419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10c9674640 1 -- 172.21.15.7:0/980262400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c41999b0 con 0x7f10c41033e0 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10c9674640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 0x7f10c419fd30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f10b4031460 tx=0x7f10b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10baffd640 1 -- 172.21.15.7:0/980262400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f10b403f070 con 0x7f10c41033e0 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c4199c40 con 0x7f10c41033e0 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10baffd640 1 -- 172.21.15.7:0/980262400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f10b4004ac0 con 0x7f10c41033e0 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10baffd640 1 -- 172.21.15.7:0/980262400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f10b4035030 con 0x7f10c41033e0 2024-09-21T05:04:30.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.408+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c419a170 con 0x7f10c41033e0 2024-09-21T05:04:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.409+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c406aa90 con 0x7f10c41033e0 2024-09-21T05:04:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.410+0000 7f10baffd640 1 -- 172.21.15.7:0/980262400 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f10b4042070 con 0x7f10c41033e0 2024-09-21T05:04:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.410+0000 7f10baffd640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f10b007af30 0x7f10b007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.410+0000 7f10baffd640 1 -- 172.21.15.7:0/980262400 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f10b40beac0 con 0x7f10c41033e0 2024-09-21T05:04:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.410+0000 7f10c9e75640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f10b007af30 0x7f10b007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.411+0000 7f10c9e75640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f10b007af30 0x7f10b007d3f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10c0005fd0 tx=0x7f10c0005e40 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.415+0000 7f10baffd640 1 -- 172.21.15.7:0/980262400 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10b408a060 con 0x7f10c41033e0 2024-09-21T05:04:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.619+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f10c419aee0 con 0x7f10b007af30 2024-09-21T05:04:30.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.621+0000 7f10baffd640 1 -- 172.21.15.7:0/980262400 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f10c419aee0 con 0x7f10b007af30 2024-09-21T05:04:30.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f10b007af30 msgr2=0x7f10b007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f10b007af30 0x7f10b007d3f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f10c0005fd0 tx=0x7f10c0005e40 comp rx=0 tx=0).stop 2024-09-21T05:04:30.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 msgr2=0x7f10c419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 0x7f10c419fd30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f10b4031460 tx=0x7f10b40046c0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 shutdown_connections 2024-09-21T05:04:30.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f10b007af30 0x7f10b007d3f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10c41033e0 0x7f10c419fd30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 --2- 172.21.15.7:0/980262400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c4102ac0 0x7f10c419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 >> 172.21.15.7:0/980262400 conn(0x7f10c40fc520 msgr2=0x7f10c4100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:30.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 shutdown_connections 2024-09-21T05:04:30.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.623+0000 7f10cc0d9640 1 -- 172.21.15.7:0/980262400 wait complete. 2024-09-21T05:04:30.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:30 smithi007 ceph-mon[108238]: pgmap v893: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.708+0000 7fa279a6b640 1 -- 172.21.15.7:0/4188961888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 msgr2=0x7fa274100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.708+0000 7fa279a6b640 1 --2- 172.21.15.7:0/4188961888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 0x7fa274100c50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa268009880 tx=0x7fa26802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.709+0000 7fa279a6b640 1 -- 172.21.15.7:0/4188961888 shutdown_connections 2024-09-21T05:04:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.709+0000 7fa279a6b640 1 --2- 172.21.15.7:0/4188961888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 0x7fa274100c50 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.709+0000 7fa279a6b640 1 --2- 172.21.15.7:0/4188961888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2741067f0 0x7fa274106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.709+0000 7fa279a6b640 1 -- 172.21.15.7:0/4188961888 >> 172.21.15.7:0/4188961888 conn(0x7fa2740fc550 msgr2=0x7fa2740fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.709+0000 7fa279a6b640 1 -- 172.21.15.7:0/4188961888 shutdown_connections 2024-09-21T05:04:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.709+0000 7fa279a6b640 1 -- 172.21.15.7:0/4188961888 wait complete. 2024-09-21T05:04:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.709+0000 7fa279a6b640 1 Processor -- start 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.709+0000 7fa279a6b640 1 -- start start 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa279a6b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 0x7fa27419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa279a6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2741067f0 0x7fa27419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa279a6b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa27410dd20 con 0x7fa2741007f0 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa279a6b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa27410dba0 con 0x7fa2741067f0 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa2737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 0x7fa27419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa2737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 0x7fa27419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45350/0 (socket says 172.21.15.7:45350) 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa2737fe640 1 -- 172.21.15.7:0/2051425718 learned_addr learned my addr 172.21.15.7:0/2051425718 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa272ffd640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2741067f0 0x7fa27419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa2737fe640 1 -- 172.21.15.7:0/2051425718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2741067f0 msgr2=0x7fa27419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa2737fe640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2741067f0 0x7fa27419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa2737fe640 1 -- 172.21.15.7:0/2051425718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa274197830 con 0x7fa2741007f0 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa2737fe640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 0x7fa27419d6b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa26400ba50 tx=0x7fa26400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa270ff9640 1 -- 172.21.15.7:0/2051425718 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa264019070 con 0x7fa2741007f0 2024-09-21T05:04:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa274197b20 con 0x7fa2741007f0 2024-09-21T05:04:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.710+0000 7fa270ff9640 1 -- 172.21.15.7:0/2051425718 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa26400ce20 con 0x7fa2741007f0 2024-09-21T05:04:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.711+0000 7fa270ff9640 1 -- 172.21.15.7:0/2051425718 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa2640075a0 con 0x7fa2741007f0 2024-09-21T05:04:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.711+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa274198080 con 0x7fa2741007f0 2024-09-21T05:04:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.712+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa274101f30 con 0x7fa2741007f0 2024-09-21T05:04:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.712+0000 7fa270ff9640 1 -- 172.21.15.7:0/2051425718 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa26401c070 con 0x7fa2741007f0 2024-09-21T05:04:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.713+0000 7fa270ff9640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa25c07af30 0x7fa25c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.713+0000 7fa270ff9640 1 -- 172.21.15.7:0/2051425718 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fa26409d960 con 0x7fa2741007f0 2024-09-21T05:04:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.713+0000 7fa272ffd640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa25c07af30 0x7fa25c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.713+0000 7fa272ffd640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa25c07af30 0x7fa25c07d3f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa274198b80 tx=0x7fa268031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:30.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.717+0000 7fa270ff9640 1 -- 172.21.15.7:0/2051425718 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa264068f00 con 0x7fa2741007f0 2024-09-21T05:04:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.912+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa274198390 con 0x7fa25c07af30 2024-09-21T05:04:30.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.922+0000 7fa270ff9640 1 -- 172.21.15.7:0/2051425718 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fa274198390 con 0x7fa25c07af30 2024-09-21T05:04:30.926 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:04:30.926 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (26m) 5m ago 37m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:04:30.926 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (29m) 5m ago 37m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:04:30.926 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (29m) 5m ago 36m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:04:30.926 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (24m) 5m ago 37m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (24m) 5m ago 36m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (26m) 5m ago 37m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (20m) 5m ago 32m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (20m) 5m ago 32m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (5m) 5m ago 32m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (20m) 5m ago 32m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (28m) 5m ago 38m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (27m) 5m ago 36m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (25m) 5m ago 38m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (24m) 5m ago 36m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (27m) 5m ago 37m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (27m) 5m ago 36m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (24m) 5m ago 35m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (23m) 5m ago 35m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:04:30.927 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (23m) 5m ago 34m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:04:30.928 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (22m) 5m ago 34m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:04:30.928 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (21m) 5m ago 33m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:04:30.928 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (21m) 5m ago 33m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:04:30.928 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (26m) 5m ago 37m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:04:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa25c07af30 msgr2=0x7fa25c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa25c07af30 0x7fa25c07d3f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa274198b80 tx=0x7fa268031400 comp rx=0 tx=0).stop 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 msgr2=0x7fa27419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 0x7fa27419d6b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa26400ba50 tx=0x7fa26400bf10 comp rx=0 tx=0).stop 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 shutdown_connections 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa25c07af30 0x7fa25c07d3f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2741067f0 0x7fa27419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 --2- 172.21.15.7:0/2051425718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2741007f0 0x7fa27419d6b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 >> 172.21.15.7:0/2051425718 conn(0x7fa2740fc550 msgr2=0x7fa274069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 shutdown_connections 2024-09-21T05:04:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:30.924+0000 7fa279a6b640 1 -- 172.21.15.7:0/2051425718 wait complete. 2024-09-21T05:04:30.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:30 smithi053 ceph-mon[80176]: pgmap v893: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:31.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.008+0000 7f24165a0640 1 -- 172.21.15.7:0/3014151730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410103230 msgr2=0x7f241010b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.008+0000 7f24165a0640 1 --2- 172.21.15.7:0/3014151730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410103230 0x7f241010b720 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2400009880 tx=0x7f240002a530 comp rx=0 tx=0).stop 2024-09-21T05:04:31.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.008+0000 7f24165a0640 1 -- 172.21.15.7:0/3014151730 shutdown_connections 2024-09-21T05:04:31.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.008+0000 7f24165a0640 1 --2- 172.21.15.7:0/3014151730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410103230 0x7f241010b720 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.008+0000 7f24165a0640 1 --2- 172.21.15.7:0/3014151730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2410102910 0x7f2410102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.008+0000 7f24165a0640 1 -- 172.21.15.7:0/3014151730 >> 172.21.15.7:0/3014151730 conn(0x7f24100fc350 msgr2=0x7f24100fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:31.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.008+0000 7f24165a0640 1 -- 172.21.15.7:0/3014151730 shutdown_connections 2024-09-21T05:04:31.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.008+0000 7f24165a0640 1 -- 172.21.15.7:0/3014151730 wait complete. 2024-09-21T05:04:31.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f24165a0640 1 Processor -- start 2024-09-21T05:04:31.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f24165a0640 1 -- start start 2024-09-21T05:04:31.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f24165a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410102910 0x7f2410100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f24165a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2410103230 0x7f2410100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f24165a0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f241010ec20 con 0x7f2410102910 2024-09-21T05:04:31.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f24165a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f241010eaa0 con 0x7f2410103230 2024-09-21T05:04:31.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f241559e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410102910 0x7f2410100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f241559e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410102910 0x7f2410100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45372/0 (socket says 172.21.15.7:45372) 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.009+0000 7f241559e640 1 -- 172.21.15.7:0/1197726505 learned_addr learned my addr 172.21.15.7:0/1197726505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f2414d9d640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2410103230 0x7f2410100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f241559e640 1 -- 172.21.15.7:0/1197726505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2410103230 msgr2=0x7f2410100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f241559e640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2410103230 0x7f2410100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f241559e640 1 -- 172.21.15.7:0/1197726505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2410100ff0 con 0x7f2410102910 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f241559e640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410102910 0x7f2410100570 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c8a0 tx=0x7f240c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f24067fc640 1 -- 172.21.15.7:0/1197726505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f240c0079b0 con 0x7f2410102910 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f24067fc640 1 -- 172.21.15.7:0/1197726505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f240c007b50 con 0x7f2410102910 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101012e0 con 0x7f2410102910 2024-09-21T05:04:31.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a3c30 con 0x7f2410102910 2024-09-21T05:04:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.010+0000 7f24067fc640 1 -- 172.21.15.7:0/1197726505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f240c018700 con 0x7f2410102910 2024-09-21T05:04:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.011+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f4005290 con 0x7f2410102910 2024-09-21T05:04:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.012+0000 7f24067fc640 1 -- 172.21.15.7:0/1197726505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f240c00f450 con 0x7f2410102910 2024-09-21T05:04:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.013+0000 7f24067fc640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f23fc07af30 0x7f23fc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.013+0000 7f24067fc640 1 -- 172.21.15.7:0/1197726505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f240c09e1a0 con 0x7f2410102910 2024-09-21T05:04:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.013+0000 7f2414d9d640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f23fc07af30 0x7f23fc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.013+0000 7f2414d9d640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f23fc07af30 0x7f23fc07d3f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2410102000 tx=0x7f2400031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:31.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.017+0000 7f24067fc640 1 -- 172.21.15.7:0/1197726505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f240c014030 con 0x7f2410102910 2024-09-21T05:04:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.285+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f23f4005d90 con 0x7f2410102910 2024-09-21T05:04:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.285+0000 7f24067fc640 1 -- 172.21.15.7:0/1197726505 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f240c069740 con 0x7f2410102910 2024-09-21T05:04:31.289 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:04:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:04:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:04:31.291 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.287+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f23fc07af30 msgr2=0x7f23fc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f23fc07af30 0x7f23fc07d3f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2410102000 tx=0x7f2400031400 comp rx=0 tx=0).stop 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410102910 msgr2=0x7f2410100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410102910 0x7f2410100570 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c8a0 tx=0x7f240c00cd60 comp rx=0 tx=0).stop 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 shutdown_connections 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f23fc07af30 0x7f23fc07d3f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2410103230 0x7f2410100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 --2- 172.21.15.7:0/1197726505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2410102910 0x7f2410100570 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 >> 172.21.15.7:0/1197726505 conn(0x7f24100fc350 msgr2=0x7f2410108d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 shutdown_connections 2024-09-21T05:04:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.288+0000 7f24165a0640 1 -- 172.21.15.7:0/1197726505 wait complete. 2024-09-21T05:04:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.372+0000 7f6554cfb640 1 -- 172.21.15.7:0/867509706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 msgr2=0x7f6550100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.372+0000 7f6554cfb640 1 --2- 172.21.15.7:0/867509706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 0x7f6550100c50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6544009880 tx=0x7f654402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.372+0000 7f6554cfb640 1 -- 172.21.15.7:0/867509706 shutdown_connections 2024-09-21T05:04:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.372+0000 7f6554cfb640 1 --2- 172.21.15.7:0/867509706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 0x7f6550100c50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.372+0000 7f6554cfb640 1 --2- 172.21.15.7:0/867509706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65501067f0 0x7f6550106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.372+0000 7f6554cfb640 1 -- 172.21.15.7:0/867509706 >> 172.21.15.7:0/867509706 conn(0x7f65500fc550 msgr2=0x7f65500fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.372+0000 7f6554cfb640 1 -- 172.21.15.7:0/867509706 shutdown_connections 2024-09-21T05:04:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.372+0000 7f6554cfb640 1 -- 172.21.15.7:0/867509706 wait complete. 2024-09-21T05:04:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f6554cfb640 1 Processor -- start 2024-09-21T05:04:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f6554cfb640 1 -- start start 2024-09-21T05:04:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f6554cfb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 0x7f655019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f6554cfb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65501067f0 0x7f655019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f6554cfb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f655010dca0 con 0x7f65501007f0 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f6554cfb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f655010db20 con 0x7f65501067f0 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f654e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 0x7f655019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f654e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 0x7f655019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45398/0 (socket says 172.21.15.7:45398) 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f654e59c640 1 -- 172.21.15.7:0/855209399 learned_addr learned my addr 172.21.15.7:0/855209399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.373+0000 7f654dd9b640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65501067f0 0x7f655019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f654e59c640 1 -- 172.21.15.7:0/855209399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65501067f0 msgr2=0x7f655019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f654e59c640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65501067f0 0x7f655019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f654e59c640 1 -- 172.21.15.7:0/855209399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6550196670 con 0x7f65501007f0 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f654e59c640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 0x7f655019c5a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f654000d8d0 tx=0x7f654000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f653f7fe640 1 -- 172.21.15.7:0/855209399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6540017070 con 0x7f65501007f0 2024-09-21T05:04:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6550196960 con 0x7f65501007f0 2024-09-21T05:04:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f653f7fe640 1 -- 172.21.15.7:0/855209399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6540010460 con 0x7f65501007f0 2024-09-21T05:04:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f653f7fe640 1 -- 172.21.15.7:0/855209399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f654000f5f0 con 0x7f65501007f0 2024-09-21T05:04:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.374+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6550196ec0 con 0x7f65501007f0 2024-09-21T05:04:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.375+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6550101f30 con 0x7f65501007f0 2024-09-21T05:04:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.376+0000 7f653f7fe640 1 -- 172.21.15.7:0/855209399 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6540009a30 con 0x7f65501007f0 2024-09-21T05:04:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.376+0000 7f653f7fe640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f653807af40 0x7f653807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.376+0000 7f653f7fe640 1 -- 172.21.15.7:0/855209399 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f654009df40 con 0x7f65501007f0 2024-09-21T05:04:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.376+0000 7f654dd9b640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f653807af40 0x7f653807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.377+0000 7f654dd9b640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f653807af40 0x7f653807d400 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6550197ec0 tx=0x7f65440095c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.381+0000 7f653f7fe640 1 -- 172.21.15.7:0/855209399 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65400694e0 con 0x7f65501007f0 2024-09-21T05:04:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.604+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6550197bd0 con 0x7f65501007f0 2024-09-21T05:04:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.605+0000 7f653f7fe640 1 -- 172.21.15.7:0/855209399 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f6550197bd0 con 0x7f65501007f0 2024-09-21T05:04:31.609 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:04:31.609 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:04:31.609 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:04:31.609 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:04:31.609 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:04:31.610 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:04:31.611 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:04:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:04:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:04:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f653807af40 msgr2=0x7f653807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f653807af40 0x7f653807d400 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6550197ec0 tx=0x7f65440095c0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 msgr2=0x7f655019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 0x7f655019c5a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f654000d8d0 tx=0x7f654000dd90 comp rx=0 tx=0).stop 2024-09-21T05:04:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 shutdown_connections 2024-09-21T05:04:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f653807af40 0x7f653807d400 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65501067f0 0x7f655019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 --2- 172.21.15.7:0/855209399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65501007f0 0x7f655019c5a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 >> 172.21.15.7:0/855209399 conn(0x7f65500fc550 msgr2=0x7f65500fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 shutdown_connections 2024-09-21T05:04:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.608+0000 7f6554cfb640 1 -- 172.21.15.7:0/855209399 wait complete. 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.694+0000 7f1c28622640 1 -- 172.21.15.7:0/252255703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20107900 msgr2=0x7f1c20107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.694+0000 7f1c28622640 1 --2- 172.21.15.7:0/252255703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20107900 0x7f1c20107ce0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009880 tx=0x7f1c1802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.694+0000 7f1c28622640 1 -- 172.21.15.7:0/252255703 shutdown_connections 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.694+0000 7f1c28622640 1 --2- 172.21.15.7:0/252255703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c20101900 0x7f1c20101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.694+0000 7f1c28622640 1 --2- 172.21.15.7:0/252255703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20107900 0x7f1c20107ce0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.694+0000 7f1c28622640 1 -- 172.21.15.7:0/252255703 >> 172.21.15.7:0/252255703 conn(0x7f1c200fd600 msgr2=0x7f1c200ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.694+0000 7f1c28622640 1 -- 172.21.15.7:0/252255703 shutdown_connections 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.694+0000 7f1c28622640 1 -- 172.21.15.7:0/252255703 wait complete. 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c28622640 1 Processor -- start 2024-09-21T05:04:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c28622640 1 -- start start 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c28622640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20101900 0x7f1c200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c28622640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c20107900 0x7f1c20074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c28622640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010edd0 con 0x7f1c20101900 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c28622640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010ec50 con 0x7f1c20107900 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c263be640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20101900 0x7f1c200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c263be640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20101900 0x7f1c200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45418/0 (socket says 172.21.15.7:45418) 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c263be640 1 -- 172.21.15.7:0/2640629582 learned_addr learned my addr 172.21.15.7:0/2640629582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c25bbd640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c20107900 0x7f1c20074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c263be640 1 -- 172.21.15.7:0/2640629582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c20107900 msgr2=0x7f1c20074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c263be640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c20107900 0x7f1c20074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.695+0000 7f1c263be640 1 -- 172.21.15.7:0/2640629582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c20074f70 con 0x7f1c20101900 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.696+0000 7f1c263be640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20101900 0x7f1c200744f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1c18031460 tx=0x7f1c180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.696+0000 7f1c1f7fe640 1 -- 172.21.15.7:0/2640629582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1c1803f070 con 0x7f1c20101900 2024-09-21T05:04:31.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.696+0000 7f1c1f7fe640 1 -- 172.21.15.7:0/2640629582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c18004cf0 con 0x7f1c20101900 2024-09-21T05:04:31.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.696+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c20075200 con 0x7f1c20101900 2024-09-21T05:04:31.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.696+0000 7f1c1f7fe640 1 -- 172.21.15.7:0/2640629582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1c18035030 con 0x7f1c20101900 2024-09-21T05:04:31.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.696+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c201a3d80 con 0x7f1c20101900 2024-09-21T05:04:31.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.697+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c20103040 con 0x7f1c20101900 2024-09-21T05:04:31.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.697+0000 7f1c1f7fe640 1 -- 172.21.15.7:0/2640629582 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1c18042070 con 0x7f1c20101900 2024-09-21T05:04:31.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.698+0000 7f1c1f7fe640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c0807af80 0x7f1c0807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:31.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.698+0000 7f1c1f7fe640 1 -- 172.21.15.7:0/2640629582 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f1c180bea40 con 0x7f1c20101900 2024-09-21T05:04:31.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.698+0000 7f1c25bbd640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c0807af80 0x7f1c0807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:31.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.698+0000 7f1c25bbd640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c0807af80 0x7f1c0807d440 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005fd0 tx=0x7f1c14005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:31.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.702+0000 7f1c1f7fe640 1 -- 172.21.15.7:0/2640629582 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1c1808a060 con 0x7f1c20101900 2024-09-21T05:04:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.906+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1c20062630 con 0x7f1c0807af80 2024-09-21T05:04:31.911 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:31 smithi007 ceph-mon[108238]: from='client.35432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:31.911 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:31 smithi007 ceph-mon[108238]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:31.911 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:31 smithi007 ceph-mon[108238]: from='client.35440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:31.911 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:31 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1197726505' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:04:31.911 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:31 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/855209399' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:04:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.908+0000 7f1c1f7fe640 1 -- 172.21.15.7:0/2640629582 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f1c20062630 con 0x7f1c0807af80 2024-09-21T05:04:31.912 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:04:31.912 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:04:31.912 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:04:31.912 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:04:31.912 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:04:31.913 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c0807af80 msgr2=0x7f1c0807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c0807af80 0x7f1c0807d440 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005fd0 tx=0x7f1c14005e20 comp rx=0 tx=0).stop 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20101900 msgr2=0x7f1c200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20101900 0x7f1c200744f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1c18031460 tx=0x7f1c180048f0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 shutdown_connections 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1c0807af80 0x7f1c0807d440 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c20107900 0x7f1c20074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 --2- 172.21.15.7:0/2640629582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c20101900 0x7f1c200744f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 >> 172.21.15.7:0/2640629582 conn(0x7f1c200fd600 msgr2=0x7f1c200fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 shutdown_connections 2024-09-21T05:04:31.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:31.910+0000 7f1c28622640 1 -- 172.21.15.7:0/2640629582 wait complete. 2024-09-21T05:04:31.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:31 smithi053 ceph-mon[80176]: from='client.35432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:31.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:31 smithi053 ceph-mon[80176]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:31.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:31 smithi053 ceph-mon[80176]: from='client.35440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:31.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:31 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1197726505' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:04:31.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:31 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/855209399' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:04:32.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.000+0000 7fa51c7dd640 1 -- 172.21.15.7:0/4287294801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5141033e0 msgr2=0x7fa51410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:32.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.000+0000 7fa51c7dd640 1 --2- 172.21.15.7:0/4287294801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5141033e0 0x7fa51410b8d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa504009880 tx=0x7fa50402a530 comp rx=0 tx=0).stop 2024-09-21T05:04:32.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.000+0000 7fa51c7dd640 1 -- 172.21.15.7:0/4287294801 shutdown_connections 2024-09-21T05:04:32.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.000+0000 7fa51c7dd640 1 --2- 172.21.15.7:0/4287294801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5141033e0 0x7fa51410b8d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:32.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.000+0000 7fa51c7dd640 1 --2- 172.21.15.7:0/4287294801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa514102ac0 0x7fa514102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:32.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.000+0000 7fa51c7dd640 1 -- 172.21.15.7:0/4287294801 >> 172.21.15.7:0/4287294801 conn(0x7fa5140fc520 msgr2=0x7fa5140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:32.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.000+0000 7fa51c7dd640 1 -- 172.21.15.7:0/4287294801 shutdown_connections 2024-09-21T05:04:32.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.000+0000 7fa51c7dd640 1 -- 172.21.15.7:0/4287294801 wait complete. 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51c7dd640 1 Processor -- start 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51c7dd640 1 -- start start 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51c7dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa514102ac0 0x7fa51419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51c7dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5141033e0 0x7fa51419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51c7dd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa51410edf0 con 0x7fa514102ac0 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51c7dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa51410ec70 con 0x7fa5141033e0 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51a579640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa514102ac0 0x7fa51419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51a579640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa514102ac0 0x7fa51419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45440/0 (socket says 172.21.15.7:45440) 2024-09-21T05:04:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51a579640 1 -- 172.21.15.7:0/832003892 learned_addr learned my addr 172.21.15.7:0/832003892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa519d78640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5141033e0 0x7fa51419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51a579640 1 -- 172.21.15.7:0/832003892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5141033e0 msgr2=0x7fa51419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51a579640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5141033e0 0x7fa51419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa51a579640 1 -- 172.21.15.7:0/832003892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa514197800 con 0x7fa514102ac0 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.001+0000 7fa519d78640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5141033e0 0x7fa51419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.002+0000 7fa51a579640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa514102ac0 0x7fa51419d680 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c8f0 tx=0x7fa51000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.002+0000 7fa50b7fe640 1 -- 172.21.15.7:0/832003892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa5100079b0 con 0x7fa514102ac0 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.002+0000 7fa50b7fe640 1 -- 172.21.15.7:0/832003892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa5100054e0 con 0x7fa514102ac0 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.002+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa514197af0 con 0x7fa514102ac0 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.002+0000 7fa50b7fe640 1 -- 172.21.15.7:0/832003892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa510004dd0 con 0x7fa514102ac0 2024-09-21T05:04:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.002+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa514198080 con 0x7fa514102ac0 2024-09-21T05:04:32.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.003+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51406aa90 con 0x7fa514102ac0 2024-09-21T05:04:32.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.003+0000 7fa50b7fe640 1 -- 172.21.15.7:0/832003892 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa510005020 con 0x7fa514102ac0 2024-09-21T05:04:32.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.004+0000 7fa50b7fe640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa50007af30 0x7fa50007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:04:32.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.004+0000 7fa50b7fe640 1 -- 172.21.15.7:0/832003892 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fa51009dfa0 con 0x7fa514102ac0 2024-09-21T05:04:32.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.004+0000 7fa519d78640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa50007af30 0x7fa50007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:04:32.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.005+0000 7fa519d78640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa50007af30 0x7fa50007d3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa504031460 tx=0x7fa5040095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:04:32.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.008+0000 7fa50b7fe640 1 -- 172.21.15.7:0/832003892 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa510014030 con 0x7fa514102ac0 2024-09-21T05:04:32.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.275+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fa514105f60 con 0x7fa514102ac0 2024-09-21T05:04:32.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.275+0000 7fa50b7fe640 1 -- 172.21.15.7:0/832003892 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fa510069520 con 0x7fa514102ac0 2024-09-21T05:04:32.279 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:04:32.279 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:04:32.279 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:04:32.279 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:04:32.279 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:04:32.280 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:04:32.281 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:04:32.281 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:04:32.281 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:04:32.281 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:04:32.281 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:04:32.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa50007af30 msgr2=0x7fa50007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa50007af30 0x7fa50007d3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa504031460 tx=0x7fa5040095f0 comp rx=0 tx=0).stop 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa514102ac0 msgr2=0x7fa51419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa514102ac0 0x7fa51419d680 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c8f0 tx=0x7fa51000cdb0 comp rx=0 tx=0).stop 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 shutdown_connections 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa50007af30 0x7fa50007d3f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5141033e0 0x7fa51419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 --2- 172.21.15.7:0/832003892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa514102ac0 0x7fa51419d680 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 >> 172.21.15.7:0/832003892 conn(0x7fa5140fc520 msgr2=0x7fa514108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 shutdown_connections 2024-09-21T05:04:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:04:32.277+0000 7fa51c7dd640 1 -- 172.21.15.7:0/832003892 wait complete. 2024-09-21T05:04:32.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:32 smithi053 ceph-mon[80176]: pgmap v894: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:32.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:32 smithi053 ceph-mon[80176]: from='client.35452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:32.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:32 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/832003892' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:04:32.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:32 smithi007 ceph-mon[108238]: pgmap v894: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:32.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:32 smithi007 ceph-mon[108238]: from='client.35452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:04:32.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:32 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/832003892' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:04:34.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:34 smithi007 ceph-mon[108238]: pgmap v895: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:34.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:34 smithi053 ceph-mon[80176]: pgmap v895: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:36 smithi053 ceph-mon[80176]: pgmap v896: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:36.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:36 smithi007 ceph-mon[108238]: pgmap v896: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:38.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:38 smithi053 ceph-mon[80176]: pgmap v897: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:38 smithi007 ceph-mon[108238]: pgmap v897: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:40.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:40 smithi007 ceph-mon[108238]: pgmap v898: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:40.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:40 smithi053 ceph-mon[80176]: pgmap v898: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:42.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:42 smithi053 ceph-mon[80176]: pgmap v899: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:42.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:42 smithi007 ceph-mon[108238]: pgmap v899: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:43.932 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:04:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:04:44.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:44 smithi053 ceph-mon[80176]: pgmap v900: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:44 smithi007 ceph-mon[108238]: pgmap v900: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:46.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:46 smithi007 ceph-mon[108238]: pgmap v901: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:46.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:46 smithi053 ceph-mon[80176]: pgmap v901: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:48.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:48 smithi053 ceph-mon[80176]: pgmap v902: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:48.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:48 smithi007 ceph-mon[108238]: pgmap v902: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:50.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:50 smithi053 ceph-mon[80176]: pgmap v903: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:50.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:50 smithi007 ceph-mon[108238]: pgmap v903: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:52.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:52 smithi007 ceph-mon[108238]: pgmap v904: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:52.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:52 smithi053 ceph-mon[80176]: pgmap v904: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:54.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:54 smithi053 ceph-mon[80176]: pgmap v905: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:54.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:54 smithi007 ceph-mon[108238]: pgmap v905: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:56.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:56 smithi053 ceph-mon[80176]: pgmap v906: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:56 smithi007 ceph-mon[108238]: pgmap v906: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:58.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:58 smithi007 ceph-mon[108238]: pgmap v907: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:58.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:04:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:04:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:58 smithi053 ceph-mon[80176]: pgmap v907: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:04:58.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:04:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:05:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:00 smithi053 ceph-mon[80176]: pgmap v908: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:00 smithi007 ceph-mon[108238]: pgmap v908: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.376+0000 7f8b14d01640 1 -- 172.21.15.7:0/484014198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b101006c0 msgr2=0x7f8b10100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.376+0000 7f8b14d01640 1 --2- 172.21.15.7:0/484014198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b101006c0 0x7f8b10100aa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009880 tx=0x7f8b0002a490 comp rx=0 tx=0).stop 2024-09-21T05:05:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.376+0000 7f8b14d01640 1 -- 172.21.15.7:0/484014198 shutdown_connections 2024-09-21T05:05:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.376+0000 7f8b14d01640 1 --2- 172.21.15.7:0/484014198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b10101070 0x7f8b1010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.376+0000 7f8b14d01640 1 --2- 172.21.15.7:0/484014198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b101006c0 0x7f8b10100aa0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.376+0000 7f8b14d01640 1 -- 172.21.15.7:0/484014198 >> 172.21.15.7:0/484014198 conn(0x7f8b100fc550 msgr2=0x7f8b100fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.376+0000 7f8b14d01640 1 -- 172.21.15.7:0/484014198 shutdown_connections 2024-09-21T05:05:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.376+0000 7f8b14d01640 1 -- 172.21.15.7:0/484014198 wait complete. 2024-09-21T05:05:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b14d01640 1 Processor -- start 2024-09-21T05:05:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b14d01640 1 -- start start 2024-09-21T05:05:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b14d01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b101006c0 0x7f8b1019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b14d01640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b10101070 0x7f8b1019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b14d01640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10110f80 con 0x7f8b10101070 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b14d01640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10110e00 con 0x7f8b101006c0 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b05d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b10101070 0x7f8b1019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b05d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b10101070 0x7f8b1019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36236/0 (socket says 172.21.15.7:36236) 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.377+0000 7f8b05d9b640 1 -- 172.21.15.7:0/534152196 learned_addr learned my addr 172.21.15.7:0/534152196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b0e59c640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b101006c0 0x7f8b1019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b05d9b640 1 -- 172.21.15.7:0/534152196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b101006c0 msgr2=0x7f8b1019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b05d9b640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b101006c0 0x7f8b1019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b05d9b640 1 -- 172.21.15.7:0/534152196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b101a0400 con 0x7f8b10101070 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b05d9b640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b10101070 0x7f8b1019fd50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8af800c8a0 tx=0x7f8af800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b07fff640 1 -- 172.21.15.7:0/534152196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8af80079b0 con 0x7f8b10101070 2024-09-21T05:05:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b07fff640 1 -- 172.21.15.7:0/534152196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8af8007b50 con 0x7f8b10101070 2024-09-21T05:05:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b10199af0 con 0x7f8b10101070 2024-09-21T05:05:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b07fff640 1 -- 172.21.15.7:0/534152196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8af8018700 con 0x7f8b10101070 2024-09-21T05:05:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.378+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1019a050 con 0x7f8b10101070 2024-09-21T05:05:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.379+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1006a500 con 0x7f8b10101070 2024-09-21T05:05:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.380+0000 7f8b07fff640 1 -- 172.21.15.7:0/534152196 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8af8007d00 con 0x7f8b10101070 2024-09-21T05:05:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.381+0000 7f8b07fff640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8afc07af80 0x7f8afc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.381+0000 7f8b07fff640 1 -- 172.21.15.7:0/534152196 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f8af809e1e0 con 0x7f8b10101070 2024-09-21T05:05:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.381+0000 7f8b0e59c640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8afc07af80 0x7f8afc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.381+0000 7f8b0e59c640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8afc07af80 0x7f8afc07d440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8b00000c00 tx=0x7f8b00031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.385+0000 7f8b07fff640 1 -- 172.21.15.7:0/534152196 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8af8014030 con 0x7f8b10101070 2024-09-21T05:05:02.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.591+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8b1019af00 con 0x7f8afc07af80 2024-09-21T05:05:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.592+0000 7f8b07fff640 1 -- 172.21.15.7:0/534152196 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8b1019af00 con 0x7f8afc07af80 2024-09-21T05:05:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8afc07af80 msgr2=0x7f8afc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8afc07af80 0x7f8afc07d440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8b00000c00 tx=0x7f8b00031400 comp rx=0 tx=0).stop 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b10101070 msgr2=0x7f8b1019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b10101070 0x7f8b1019fd50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8af800c8a0 tx=0x7f8af800cd60 comp rx=0 tx=0).stop 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 shutdown_connections 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8afc07af80 0x7f8afc07d440 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8b10101070 0x7f8b1019fd50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 --2- 172.21.15.7:0/534152196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b101006c0 0x7f8b1019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.594+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 >> 172.21.15.7:0/534152196 conn(0x7f8b100fc550 msgr2=0x7f8b100fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.595+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 shutdown_connections 2024-09-21T05:05:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.595+0000 7f8b14d01640 1 -- 172.21.15.7:0/534152196 wait complete. 2024-09-21T05:05:02.606 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.683+0000 7fafa2412640 1 -- 172.21.15.7:0/3091342588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 msgr2=0x7faf9c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.683+0000 7fafa2412640 1 --2- 172.21.15.7:0/3091342588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 0x7faf9c102ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009880 tx=0x7faf8c02a530 comp rx=0 tx=0).stop 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.683+0000 7fafa2412640 1 -- 172.21.15.7:0/3091342588 shutdown_connections 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.683+0000 7fafa2412640 1 --2- 172.21.15.7:0/3091342588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf9c1033e0 0x7faf9c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.683+0000 7fafa2412640 1 --2- 172.21.15.7:0/3091342588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 0x7faf9c102ea0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.683+0000 7fafa2412640 1 -- 172.21.15.7:0/3091342588 >> 172.21.15.7:0/3091342588 conn(0x7faf9c0fc520 msgr2=0x7faf9c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.683+0000 7fafa2412640 1 -- 172.21.15.7:0/3091342588 shutdown_connections 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.683+0000 7fafa2412640 1 -- 172.21.15.7:0/3091342588 wait complete. 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.684+0000 7fafa2412640 1 Processor -- start 2024-09-21T05:05:02.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.684+0000 7fafa2412640 1 -- start start 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.684+0000 7fafa2412640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 0x7faf9c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.684+0000 7fafa2412640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf9c1033e0 0x7faf9c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.684+0000 7fafa2412640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10edd0 con 0x7faf9c102ac0 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.684+0000 7fafa2412640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10ec50 con 0x7faf9c1033e0 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 0x7faf9c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 0x7faf9c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36260/0 (socket says 172.21.15.7:36260) 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf9bfff640 1 -- 172.21.15.7:0/2524739079 learned_addr learned my addr 172.21.15.7:0/2524739079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf9b7fe640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf9c1033e0 0x7faf9c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf9bfff640 1 -- 172.21.15.7:0/2524739079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf9c1033e0 msgr2=0x7faf9c100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf9bfff640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf9c1033e0 0x7faf9c100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf9bfff640 1 -- 172.21.15.7:0/2524739079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c1011a0 con 0x7faf9c102ac0 2024-09-21T05:05:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf9bfff640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 0x7faf9c100720 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faf8c031460 tx=0x7faf8c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf997fa640 1 -- 172.21.15.7:0/2524739079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faf8c03f070 con 0x7faf9c102ac0 2024-09-21T05:05:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf997fa640 1 -- 172.21.15.7:0/2524739079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faf8c0035f0 con 0x7faf9c102ac0 2024-09-21T05:05:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9c101430 con 0x7faf9c102ac0 2024-09-21T05:05:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7faf997fa640 1 -- 172.21.15.7:0/2524739079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faf8c035030 con 0x7faf9c102ac0 2024-09-21T05:05:02.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.685+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9c1a3ea0 con 0x7faf9c102ac0 2024-09-21T05:05:02.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.686+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9c06aa90 con 0x7faf9c102ac0 2024-09-21T05:05:02.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.687+0000 7faf997fa640 1 -- 172.21.15.7:0/2524739079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7faf8c042070 con 0x7faf9c102ac0 2024-09-21T05:05:02.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.688+0000 7faf997fa640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf8407af30 0x7faf8407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.688+0000 7faf997fa640 1 -- 172.21.15.7:0/2524739079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7faf8c0bea30 con 0x7faf9c102ac0 2024-09-21T05:05:02.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.688+0000 7faf9b7fe640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf8407af30 0x7faf8407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.688+0000 7faf9b7fe640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf8407af30 0x7faf8407d3f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faf9c1021b0 tx=0x7faf9000b2e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:02.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.692+0000 7faf997fa640 1 -- 172.21.15.7:0/2524739079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf8c089f70 con 0x7faf9c102ac0 2024-09-21T05:05:02.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.896+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7faf9c062630 con 0x7faf8407af30 2024-09-21T05:05:02.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.898+0000 7faf997fa640 1 -- 172.21.15.7:0/2524739079 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7faf9c062630 con 0x7faf8407af30 2024-09-21T05:05:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf8407af30 msgr2=0x7faf8407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf8407af30 0x7faf8407d3f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7faf9c1021b0 tx=0x7faf9000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 msgr2=0x7faf9c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 0x7faf9c100720 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faf8c031460 tx=0x7faf8c0045e0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 shutdown_connections 2024-09-21T05:05:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7faf8407af30 0x7faf8407d3f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf9c1033e0 0x7faf9c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 --2- 172.21.15.7:0/2524739079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c102ac0 0x7faf9c100720 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 >> 172.21.15.7:0/2524739079 conn(0x7faf9c0fc520 msgr2=0x7faf9c108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 shutdown_connections 2024-09-21T05:05:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.900+0000 7fafa2412640 1 -- 172.21.15.7:0/2524739079 wait complete. 2024-09-21T05:05:02.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:02 smithi053 ceph-mon[80176]: pgmap v909: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:02 smithi007 ceph-mon[108238]: pgmap v909: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.984+0000 7fc0f435c640 1 -- 172.21.15.7:0/644868492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 msgr2=0x7fc0ec106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.984+0000 7fc0f435c640 1 --2- 172.21.15.7:0/644868492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 0x7fc0ec106bd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009880 tx=0x7fc0e802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.984+0000 7fc0f435c640 1 -- 172.21.15.7:0/644868492 shutdown_connections 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.984+0000 7fc0f435c640 1 --2- 172.21.15.7:0/644868492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0ec1007f0 0x7fc0ec100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.984+0000 7fc0f435c640 1 --2- 172.21.15.7:0/644868492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 0x7fc0ec106bd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.984+0000 7fc0f435c640 1 -- 172.21.15.7:0/644868492 >> 172.21.15.7:0/644868492 conn(0x7fc0ec0fc550 msgr2=0x7fc0ec0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.984+0000 7fc0f435c640 1 -- 172.21.15.7:0/644868492 shutdown_connections 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.984+0000 7fc0f435c640 1 -- 172.21.15.7:0/644868492 wait complete. 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f435c640 1 Processor -- start 2024-09-21T05:05:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f435c640 1 -- start start 2024-09-21T05:05:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f435c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0ec1007f0 0x7fc0ec0fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f435c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 0x7fc0ec0fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f435c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec10dd20 con 0x7fc0ec1067f0 2024-09-21T05:05:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f435c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec10dba0 con 0x7fc0ec1007f0 2024-09-21T05:05:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f18f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 0x7fc0ec0fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f18f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 0x7fc0ec0fd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36276/0 (socket says 172.21.15.7:36276) 2024-09-21T05:05:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.985+0000 7fc0f18f7640 1 -- 172.21.15.7:0/2394512957 learned_addr learned my addr 172.21.15.7:0/2394512957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0f20f8640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0ec1007f0 0x7fc0ec0fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0f18f7640 1 -- 172.21.15.7:0/2394512957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0ec1007f0 msgr2=0x7fc0ec0fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0f18f7640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0ec1007f0 0x7fc0ec0fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0f18f7640 1 -- 172.21.15.7:0/2394512957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0fd7a0 con 0x7fc0ec1067f0 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0f18f7640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 0x7fc0ec0fd260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00b420 tx=0x7fc0dc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0e37fe640 1 -- 172.21.15.7:0/2394512957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc0dc011630 con 0x7fc0ec1067f0 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0e37fe640 1 -- 172.21.15.7:0/2394512957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc0dc016070 con 0x7fc0ec1067f0 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec0fda90 con 0x7fc0ec1067f0 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.986+0000 7fc0e37fe640 1 -- 172.21.15.7:0/2394512957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc0dc010d40 con 0x7fc0ec1067f0 2024-09-21T05:05:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.987+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec1a3e80 con 0x7fc0ec1067f0 2024-09-21T05:05:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.988+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec101f30 con 0x7fc0ec1067f0 2024-09-21T05:05:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.988+0000 7fc0e37fe640 1 -- 172.21.15.7:0/2394512957 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc0dc01b460 con 0x7fc0ec1067f0 2024-09-21T05:05:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.989+0000 7fc0e37fe640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc0d807af30 0x7fc0d807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.989+0000 7fc0e37fe640 1 -- 172.21.15.7:0/2394512957 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc0dc09d3e0 con 0x7fc0ec1067f0 2024-09-21T05:05:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.989+0000 7fc0f20f8640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc0d807af30 0x7fc0d807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.989+0000 7fc0f20f8640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc0d807af30 0x7fc0d807d3f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80096c0 tx=0x7fc0e8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:02.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:02.993+0000 7fc0e37fe640 1 -- 172.21.15.7:0/2394512957 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0dc020060 con 0x7fc0ec1067f0 2024-09-21T05:05:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.194+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc0ec062630 con 0x7fc0d807af30 2024-09-21T05:05:03.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.203+0000 7fc0e37fe640 1 -- 172.21.15.7:0/2394512957 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fc0ec062630 con 0x7fc0d807af30 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (27m) 5m ago 37m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (30m) 5m ago 38m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (29m) 5m ago 37m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (25m) 5m ago 38m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (25m) 5m ago 37m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (26m) 5m ago 37m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (21m) 5m ago 32m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (21m) 5m ago 32m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (5m) 5m ago 32m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (20m) 5m ago 32m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (29m) 5m ago 39m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:05:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (28m) 5m ago 37m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (25m) 5m ago 39m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (25m) 5m ago 37m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (27m) 5m ago 38m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (27m) 5m ago 37m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (24m) 5m ago 36m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (24m) 5m ago 35m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (23m) 5m ago 35m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (22m) 5m ago 34m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (22m) 5m ago 34m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (21m) 5m ago 33m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:05:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (27m) 5m ago 37m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:05:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc0d807af30 msgr2=0x7fc0d807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc0d807af30 0x7fc0d807d3f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80096c0 tx=0x7fc0e8031400 comp rx=0 tx=0).stop 2024-09-21T05:05:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 msgr2=0x7fc0ec0fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 0x7fc0ec0fd260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00b420 tx=0x7fc0dc00b8e0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 shutdown_connections 2024-09-21T05:05:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc0d807af30 0x7fc0d807d3f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ec1067f0 0x7fc0ec0fd260 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 --2- 172.21.15.7:0/2394512957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0ec1007f0 0x7fc0ec0fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 >> 172.21.15.7:0/2394512957 conn(0x7fc0ec0fc550 msgr2=0x7fc0ec069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 shutdown_connections 2024-09-21T05:05:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.206+0000 7fc0f435c640 1 -- 172.21.15.7:0/2394512957 wait complete. 2024-09-21T05:05:03.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.288+0000 7ff38ede4640 1 -- 172.21.15.7:0/3491946102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388102ab0 msgr2=0x7ff388102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.288+0000 7ff38ede4640 1 --2- 172.21.15.7:0/3491946102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388102ab0 0x7ff388102e90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff378009880 tx=0x7ff37802a530 comp rx=0 tx=0).stop 2024-09-21T05:05:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.289+0000 7ff38ede4640 1 -- 172.21.15.7:0/3491946102 shutdown_connections 2024-09-21T05:05:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.289+0000 7ff38ede4640 1 --2- 172.21.15.7:0/3491946102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3881033d0 0x7ff38810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.289+0000 7ff38ede4640 1 --2- 172.21.15.7:0/3491946102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388102ab0 0x7ff388102e90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.289+0000 7ff38ede4640 1 -- 172.21.15.7:0/3491946102 >> 172.21.15.7:0/3491946102 conn(0x7ff3880fc550 msgr2=0x7ff3880fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.289+0000 7ff38ede4640 1 -- 172.21.15.7:0/3491946102 shutdown_connections 2024-09-21T05:05:03.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.289+0000 7ff38ede4640 1 -- 172.21.15.7:0/3491946102 wait complete. 2024-09-21T05:05:03.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.289+0000 7ff38ede4640 1 Processor -- start 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.289+0000 7ff38ede4640 1 -- start start 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff38ede4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff388102ab0 0x7ff38819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff38ede4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3881033d0 0x7ff38819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff38ede4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff38810ede0 con 0x7ff3881033d0 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff38ede4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff38810ec60 con 0x7ff388102ab0 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff387fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3881033d0 0x7ff38819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff387fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3881033d0 0x7ff38819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36292/0 (socket says 172.21.15.7:36292) 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff387fff640 1 -- 172.21.15.7:0/2295932706 learned_addr learned my addr 172.21.15.7:0/2295932706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff38cb80640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff388102ab0 0x7ff38819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff387fff640 1 -- 172.21.15.7:0/2295932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff388102ab0 msgr2=0x7ff38819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff387fff640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff388102ab0 0x7ff38819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff387fff640 1 -- 172.21.15.7:0/2295932706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3881977b0 con 0x7ff3881033d0 2024-09-21T05:05:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.290+0000 7ff387fff640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3881033d0 0x7ff38819db70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00c8f0 tx=0x7ff37c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.291+0000 7ff385ffb640 1 -- 172.21.15.7:0/2295932706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff37c00ae50 con 0x7ff3881033d0 2024-09-21T05:05:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.291+0000 7ff385ffb640 1 -- 172.21.15.7:0/2295932706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff37c005020 con 0x7ff3881033d0 2024-09-21T05:05:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.291+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff388197aa0 con 0x7ff3881033d0 2024-09-21T05:05:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.291+0000 7ff385ffb640 1 -- 172.21.15.7:0/2295932706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff37c004d70 con 0x7ff3881033d0 2024-09-21T05:05:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.291+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff388198030 con 0x7ff3881033d0 2024-09-21T05:05:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.292+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38806a580 con 0x7ff3881033d0 2024-09-21T05:05:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.292+0000 7ff385ffb640 1 -- 172.21.15.7:0/2295932706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff37c0051d0 con 0x7ff3881033d0 2024-09-21T05:05:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.293+0000 7ff385ffb640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff37007b130 0x7ff37007d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.293+0000 7ff385ffb640 1 -- 172.21.15.7:0/2295932706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff37c09de90 con 0x7ff3881033d0 2024-09-21T05:05:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.293+0000 7ff38cb80640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff37007b130 0x7ff37007d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.294+0000 7ff38cb80640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff37007b130 0x7ff37007d5f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff378031460 tx=0x7ff378005bd0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:03.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.298+0000 7ff385ffb640 1 -- 172.21.15.7:0/2295932706 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff37c014030 con 0x7ff3881033d0 2024-09-21T05:05:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.563+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ff388198340 con 0x7ff3881033d0 2024-09-21T05:05:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.563+0000 7ff385ffb640 1 -- 172.21.15.7:0/2295932706 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7ff37c069450 con 0x7ff3881033d0 2024-09-21T05:05:03.566 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:05:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:05:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:05:03.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:05:03.567 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:05:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.565+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff37007b130 msgr2=0x7ff37007d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.565+0000 7ff38ede4640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff37007b130 0x7ff37007d5f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff378031460 tx=0x7ff378005bd0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.565+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3881033d0 msgr2=0x7ff38819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.565+0000 7ff38ede4640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3881033d0 0x7ff38819db70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00c8f0 tx=0x7ff37c00cdb0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.565+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 shutdown_connections 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.566+0000 7ff38ede4640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff37007b130 0x7ff37007d5f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.566+0000 7ff38ede4640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3881033d0 0x7ff38819db70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.566+0000 7ff38ede4640 1 --2- 172.21.15.7:0/2295932706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff388102ab0 0x7ff38819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.566+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 >> 172.21.15.7:0/2295932706 conn(0x7ff3880fc550 msgr2=0x7ff38810aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.566+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 shutdown_connections 2024-09-21T05:05:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.566+0000 7ff38ede4640 1 -- 172.21.15.7:0/2295932706 wait complete. 2024-09-21T05:05:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.650+0000 7f50c9a12640 1 -- 172.21.15.7:0/686216758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 msgr2=0x7f50c4106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.650+0000 7f50c9a12640 1 --2- 172.21.15.7:0/686216758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 0x7f50c4106be0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009880 tx=0x7f50b402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.650+0000 7f50c9a12640 1 -- 172.21.15.7:0/686216758 shutdown_connections 2024-09-21T05:05:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.650+0000 7f50c9a12640 1 --2- 172.21.15.7:0/686216758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c4100800 0x7f50c4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.650+0000 7f50c9a12640 1 --2- 172.21.15.7:0/686216758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 0x7f50c4106be0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.650+0000 7f50c9a12640 1 -- 172.21.15.7:0/686216758 >> 172.21.15.7:0/686216758 conn(0x7f50c40fc520 msgr2=0x7f50c40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.650+0000 7f50c9a12640 1 -- 172.21.15.7:0/686216758 shutdown_connections 2024-09-21T05:05:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.650+0000 7f50c9a12640 1 -- 172.21.15.7:0/686216758 wait complete. 2024-09-21T05:05:03.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.651+0000 7f50c9a12640 1 Processor -- start 2024-09-21T05:05:03.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.651+0000 7f50c9a12640 1 -- start start 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.651+0000 7f50c9a12640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c4100800 0x7f50c419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.651+0000 7f50c9a12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 0x7f50c419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.651+0000 7f50c9a12640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50c410dcd0 con 0x7f50c4106800 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.651+0000 7f50c9a12640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f50c410db50 con 0x7f50c4100800 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.651+0000 7f50c27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 0x7f50c419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.651+0000 7f50c27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 0x7f50c419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48942/0 (socket says 172.21.15.7:48942) 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c27fc640 1 -- 172.21.15.7:0/4178459085 learned_addr learned my addr 172.21.15.7:0/4178459085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c2ffd640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c4100800 0x7f50c419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c27fc640 1 -- 172.21.15.7:0/4178459085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c4100800 msgr2=0x7f50c419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c27fc640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c4100800 0x7f50c419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c27fc640 1 -- 172.21.15.7:0/4178459085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c41977c0 con 0x7f50c4106800 2024-09-21T05:05:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c2ffd640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c4100800 0x7f50c419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:05:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c27fc640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 0x7f50c419db80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f50b800d6f0 tx=0x7f50b800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c8a10640 1 -- 172.21.15.7:0/4178459085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f50b8017070 con 0x7f50c4106800 2024-09-21T05:05:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c8a10640 1 -- 172.21.15.7:0/4178459085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f50b800de00 con 0x7f50c4106800 2024-09-21T05:05:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c4197ab0 con 0x7f50c4106800 2024-09-21T05:05:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c8a10640 1 -- 172.21.15.7:0/4178459085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f50b8004bb0 con 0x7f50c4106800 2024-09-21T05:05:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.652+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c4198040 con 0x7f50c4106800 2024-09-21T05:05:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.653+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c4101f40 con 0x7f50c4106800 2024-09-21T05:05:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.654+0000 7f50c8a10640 1 -- 172.21.15.7:0/4178459085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f50b8009a30 con 0x7f50c4106800 2024-09-21T05:05:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.654+0000 7f50c8a10640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f50ac07af80 0x7f50ac07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.654+0000 7f50c8a10640 1 -- 172.21.15.7:0/4178459085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f50b809dc50 con 0x7f50c4106800 2024-09-21T05:05:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.654+0000 7f50c2ffd640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f50ac07af80 0x7f50ac07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.655+0000 7f50c2ffd640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f50ac07af80 0x7f50ac07d440 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f50b4000c00 tx=0x7f50b4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.659+0000 7f50c8a10640 1 -- 172.21.15.7:0/4178459085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50b80691d0 con 0x7f50c4106800 2024-09-21T05:05:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.883+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f50c4109d80 con 0x7f50c4106800 2024-09-21T05:05:03.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:03 smithi007 ceph-mon[108238]: from='client.35460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:03.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:03 smithi007 ceph-mon[108238]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:03.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:03 smithi007 ceph-mon[108238]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:03.887 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:03 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2295932706' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:05:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.884+0000 7f50c8a10640 1 -- 172.21.15.7:0/4178459085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f50b806e080 con 0x7f50c4106800 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:05:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:05:03.889 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:05:03.890 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:05:03.891 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:05:03.891 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:05:03.891 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:05:03.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f50ac07af80 msgr2=0x7f50ac07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f50ac07af80 0x7f50ac07d440 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f50b4000c00 tx=0x7f50b4031400 comp rx=0 tx=0).stop 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 msgr2=0x7f50c419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 0x7f50c419db80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f50b800d6f0 tx=0x7f50b800dbb0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 shutdown_connections 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f50ac07af80 0x7f50ac07d440 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c4106800 0x7f50c419db80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 --2- 172.21.15.7:0/4178459085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50c4100800 0x7f50c419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 >> 172.21.15.7:0/4178459085 conn(0x7f50c40fc520 msgr2=0x7f50c406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 shutdown_connections 2024-09-21T05:05:03.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.886+0000 7f50c9a12640 1 -- 172.21.15.7:0/4178459085 wait complete. 2024-09-21T05:05:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:03 smithi053 ceph-mon[80176]: from='client.35460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:03 smithi053 ceph-mon[80176]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:03 smithi053 ceph-mon[80176]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:03.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:03 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2295932706' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:05:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 -- 172.21.15.7:0/1323095830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 msgr2=0x7fc1e0100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 --2- 172.21.15.7:0/1323095830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 0x7fc1e0100c70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009880 tx=0x7fc1cc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 -- 172.21.15.7:0/1323095830 shutdown_connections 2024-09-21T05:05:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 --2- 172.21.15.7:0/1323095830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 0x7fc1e0100c70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 --2- 172.21.15.7:0/1323095830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e0106810 0x7fc1e0106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 -- 172.21.15.7:0/1323095830 >> 172.21.15.7:0/1323095830 conn(0x7fc1e00fc510 msgr2=0x7fc1e00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 -- 172.21.15.7:0/1323095830 shutdown_connections 2024-09-21T05:05:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 -- 172.21.15.7:0/1323095830 wait complete. 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 Processor -- start 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.972+0000 7fc1e7128640 1 -- start start 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e7128640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 0x7fc1e019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e7128640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e0106810 0x7fc1e019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e7128640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e010dcc0 con 0x7fc1e0100810 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e7128640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e010db40 con 0x7fc1e0106810 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e4ec4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 0x7fc1e019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e4ec4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 0x7fc1e019d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48958/0 (socket says 172.21.15.7:48958) 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e4ec4640 1 -- 172.21.15.7:0/3398090897 learned_addr learned my addr 172.21.15.7:0/3398090897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1dffff640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e0106810 0x7fc1e019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e4ec4640 1 -- 172.21.15.7:0/3398090897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e0106810 msgr2=0x7fc1e019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e4ec4640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e0106810 0x7fc1e019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e4ec4640 1 -- 172.21.15.7:0/3398090897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e01977d0 con 0x7fc1e0100810 2024-09-21T05:05:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e4ec4640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 0x7fc1e019d650 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400d8d0 tx=0x7fc1d400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1ddffb640 1 -- 172.21.15.7:0/3398090897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc1d4017070 con 0x7fc1e0100810 2024-09-21T05:05:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1ddffb640 1 -- 172.21.15.7:0/3398090897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc1d4010460 con 0x7fc1e0100810 2024-09-21T05:05:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e0197ac0 con 0x7fc1e0100810 2024-09-21T05:05:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.973+0000 7fc1ddffb640 1 -- 172.21.15.7:0/3398090897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc1d400f5f0 con 0x7fc1e0100810 2024-09-21T05:05:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.974+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e0198050 con 0x7fc1e0100810 2024-09-21T05:05:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.975+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e0101f50 con 0x7fc1e0100810 2024-09-21T05:05:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.975+0000 7fc1ddffb640 1 -- 172.21.15.7:0/3398090897 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc1d4009a30 con 0x7fc1e0100810 2024-09-21T05:05:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.976+0000 7fc1ddffb640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc1d007af40 0x7fc1d007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.976+0000 7fc1dffff640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc1d007af40 0x7fc1d007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.976+0000 7fc1ddffb640 1 -- 172.21.15.7:0/3398090897 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc1d409df40 con 0x7fc1e0100810 2024-09-21T05:05:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.977+0000 7fc1dffff640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc1d007af40 0x7fc1d007d400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc031460 tx=0x7fc1cc000bc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:03.980+0000 7fc1ddffb640 1 -- 172.21.15.7:0/3398090897 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc1d40694e0 con 0x7fc1e0100810 2024-09-21T05:05:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.184+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc1e0198300 con 0x7fc1d007af40 2024-09-21T05:05:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.185+0000 7fc1ddffb640 1 -- 172.21.15.7:0/3398090897 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc1e0198300 con 0x7fc1d007af40 2024-09-21T05:05:04.188 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:05:04.189 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc1d007af40 msgr2=0x7fc1d007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc1d007af40 0x7fc1d007d400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc031460 tx=0x7fc1cc000bc0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 msgr2=0x7fc1e019d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 0x7fc1e019d650 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400d8d0 tx=0x7fc1d400dd90 comp rx=0 tx=0).stop 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 shutdown_connections 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc1d007af40 0x7fc1d007d400 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e0106810 0x7fc1e019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 --2- 172.21.15.7:0/3398090897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1e0100810 0x7fc1e019d650 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 >> 172.21.15.7:0/3398090897 conn(0x7fc1e00fc510 msgr2=0x7fc1e006a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 shutdown_connections 2024-09-21T05:05:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.188+0000 7fc1e7128640 1 -- 172.21.15.7:0/3398090897 wait complete. 2024-09-21T05:05:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.272+0000 7feab68af640 1 -- 172.21.15.7:0/505948668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0100800 msgr2=0x7feab0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.272+0000 7feab68af640 1 --2- 172.21.15.7:0/505948668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0100800 0x7feab0100c60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009880 tx=0x7feaa402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.272+0000 7feab68af640 1 -- 172.21.15.7:0/505948668 shutdown_connections 2024-09-21T05:05:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.272+0000 7feab68af640 1 --2- 172.21.15.7:0/505948668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0100800 0x7feab0100c60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.272+0000 7feab68af640 1 --2- 172.21.15.7:0/505948668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab0106800 0x7feab0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.272+0000 7feab68af640 1 -- 172.21.15.7:0/505948668 >> 172.21.15.7:0/505948668 conn(0x7feab00fc520 msgr2=0x7feab00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.272+0000 7feab68af640 1 -- 172.21.15.7:0/505948668 shutdown_connections 2024-09-21T05:05:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.272+0000 7feab68af640 1 -- 172.21.15.7:0/505948668 wait complete. 2024-09-21T05:05:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feab68af640 1 Processor -- start 2024-09-21T05:05:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feab68af640 1 -- start start 2024-09-21T05:05:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feab68af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab0100800 0x7feab00fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feab68af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0106800 0x7feab00fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feab68af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feab010dcd0 con 0x7feab0106800 2024-09-21T05:05:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feab68af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feab010db50 con 0x7feab0100800 2024-09-21T05:05:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feaaf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0106800 0x7feab00fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feaaf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0106800 0x7feab00fd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48976/0 (socket says 172.21.15.7:48976) 2024-09-21T05:05:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feaaf7fe640 1 -- 172.21.15.7:0/3789569841 learned_addr learned my addr 172.21.15.7:0/3789569841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.273+0000 7feaaffff640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab0100800 0x7feab00fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feaaf7fe640 1 -- 172.21.15.7:0/3789569841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab0100800 msgr2=0x7feab00fcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feaaf7fe640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab0100800 0x7feab00fcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feaaf7fe640 1 -- 172.21.15.7:0/3789569841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab00fd6f0 con 0x7feab0106800 2024-09-21T05:05:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feaaf7fe640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0106800 0x7feab00fd1b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feaa4031460 tx=0x7feaa40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feaad7fa640 1 -- 172.21.15.7:0/3789569841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feaa403f070 con 0x7feab0106800 2024-09-21T05:05:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feaad7fa640 1 -- 172.21.15.7:0/3789569841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feaa40035f0 con 0x7feab0106800 2024-09-21T05:05:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feaad7fa640 1 -- 172.21.15.7:0/3789569841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feaa4035030 con 0x7feab0106800 2024-09-21T05:05:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab00fd980 con 0x7feab0106800 2024-09-21T05:05:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.274+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab01a3db0 con 0x7feab0106800 2024-09-21T05:05:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.275+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab0101f40 con 0x7feab0106800 2024-09-21T05:05:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.276+0000 7feaad7fa640 1 -- 172.21.15.7:0/3789569841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7feaa4042070 con 0x7feab0106800 2024-09-21T05:05:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.276+0000 7feaad7fa640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea9807af80 0x7fea9807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:04.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.276+0000 7feaad7fa640 1 -- 172.21.15.7:0/3789569841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7feaa40be8c0 con 0x7feab0106800 2024-09-21T05:05:04.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.276+0000 7feaaffff640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea9807af80 0x7fea9807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:04.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.277+0000 7feaaffff640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea9807af80 0x7fea9807d440 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005fd0 tx=0x7feaa000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:04.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.281+0000 7feaad7fa640 1 -- 172.21.15.7:0/3789569841 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feaa4089ee0 con 0x7feab0106800 2024-09-21T05:05:04.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.543+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7feab0109d80 con 0x7feab0106800 2024-09-21T05:05:04.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.543+0000 7feaad7fa640 1 -- 172.21.15.7:0/3789569841 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7feaa408ed90 con 0x7feab0106800 2024-09-21T05:05:04.547 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:05:04.547 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:05:04.547 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:05:04.548 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:05:04.548 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:05:04.548 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:05:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:05:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:05:04.548 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:05:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:05:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:05:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:05:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:05:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:05:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:05:04.549 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:05:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:05:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:05:04.550 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:05:04.550 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:05:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:05:04.550 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:05:04.550 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:05:04.551 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:05:04.551 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:05:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.545+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea9807af80 msgr2=0x7fea9807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.545+0000 7feab68af640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea9807af80 0x7fea9807d440 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feaa0005fd0 tx=0x7feaa000b390 comp rx=0 tx=0).stop 2024-09-21T05:05:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.545+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0106800 msgr2=0x7feab00fd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.545+0000 7feab68af640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0106800 0x7feab00fd1b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feaa4031460 tx=0x7feaa40045e0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.546+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 shutdown_connections 2024-09-21T05:05:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.546+0000 7feab68af640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea9807af80 0x7fea9807d440 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.546+0000 7feab68af640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab0106800 0x7feab00fd1b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.546+0000 7feab68af640 1 --2- 172.21.15.7:0/3789569841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab0100800 0x7feab00fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.546+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 >> 172.21.15.7:0/3789569841 conn(0x7feab00fc520 msgr2=0x7feab006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.546+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 shutdown_connections 2024-09-21T05:05:04.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:04.546+0000 7feab68af640 1 -- 172.21.15.7:0/3789569841 wait complete. 2024-09-21T05:05:04.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:04 smithi007 ceph-mon[108238]: pgmap v910: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:04.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:04 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4178459085' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:05:04.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:04 smithi007 ceph-mon[108238]: from='client.35480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:04.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:04 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3789569841' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:05:04.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:04 smithi053 ceph-mon[80176]: pgmap v910: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:04.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:04 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4178459085' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:05:04.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:04 smithi053 ceph-mon[80176]: from='client.35480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:04.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:04 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3789569841' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:05:06.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:06 smithi053 ceph-mon[80176]: pgmap v911: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:06.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:06 smithi007 ceph-mon[108238]: pgmap v911: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:09.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:08 smithi053 ceph-mon[80176]: pgmap v912: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:09.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:08 smithi007 ceph-mon[108238]: pgmap v912: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:11 smithi007 ceph-mon[108238]: pgmap v913: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:11 smithi053 ceph-mon[80176]: pgmap v913: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:13 smithi053 ceph-mon[80176]: pgmap v914: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:13 smithi007 ceph-mon[108238]: pgmap v914: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:05:14.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:05:15.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:15 smithi053 ceph-mon[80176]: pgmap v915: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:15 smithi007 ceph-mon[108238]: pgmap v915: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:17 smithi007 ceph-mon[108238]: pgmap v916: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:17 smithi053 ceph-mon[80176]: pgmap v916: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:19 smithi053 ceph-mon[80176]: pgmap v917: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:19 smithi007 ceph-mon[108238]: pgmap v917: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:21 smithi053 ceph-mon[80176]: pgmap v918: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:21 smithi007 ceph-mon[108238]: pgmap v918: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:23 smithi007 ceph-mon[108238]: pgmap v919: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:23 smithi053 ceph-mon[80176]: pgmap v919: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:25.193 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:25 smithi053 ceph-mon[80176]: pgmap v920: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:25 smithi007 ceph-mon[108238]: pgmap v920: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:27 smithi053 ceph-mon[80176]: pgmap v921: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:27 smithi007 ceph-mon[108238]: pgmap v921: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:29 smithi007 ceph-mon[108238]: pgmap v922: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:05:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:29 smithi053 ceph-mon[80176]: pgmap v922: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:05:31.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:31 smithi053 ceph-mon[80176]: pgmap v923: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:31 smithi007 ceph-mon[108238]: pgmap v923: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:33 smithi053 ceph-mon[80176]: pgmap v924: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:33 smithi007 ceph-mon[108238]: pgmap v924: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:34.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.642+0000 7fc3596fe640 1 -- 172.21.15.7:0/3142703691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354107900 msgr2=0x7fc354107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.642+0000 7fc3596fe640 1 --2- 172.21.15.7:0/3142703691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354107900 0x7fc354107ce0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc344009880 tx=0x7fc34402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.642+0000 7fc3596fe640 1 -- 172.21.15.7:0/3142703691 shutdown_connections 2024-09-21T05:05:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.642+0000 7fc3596fe640 1 --2- 172.21.15.7:0/3142703691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc354101900 0x7fc354101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.642+0000 7fc3596fe640 1 --2- 172.21.15.7:0/3142703691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354107900 0x7fc354107ce0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.642+0000 7fc3596fe640 1 -- 172.21.15.7:0/3142703691 >> 172.21.15.7:0/3142703691 conn(0x7fc3540fd600 msgr2=0x7fc3540ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.643+0000 7fc3596fe640 1 -- 172.21.15.7:0/3142703691 shutdown_connections 2024-09-21T05:05:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.643+0000 7fc3596fe640 1 -- 172.21.15.7:0/3142703691 wait complete. 2024-09-21T05:05:34.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.643+0000 7fc3596fe640 1 Processor -- start 2024-09-21T05:05:34.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.643+0000 7fc3596fe640 1 -- start start 2024-09-21T05:05:34.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc3596fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354101900 0x7fc35419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:34.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc3596fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc354107900 0x7fc35419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:34.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc3596fe640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc35410edb0 con 0x7fc354101900 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc3596fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc35410ec30 con 0x7fc354107900 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc352ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354101900 0x7fc35419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc352ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354101900 0x7fc35419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54622/0 (socket says 172.21.15.7:54622) 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc352ffd640 1 -- 172.21.15.7:0/1529733979 learned_addr learned my addr 172.21.15.7:0/1529733979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc34a7fc640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc354107900 0x7fc35419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc352ffd640 1 -- 172.21.15.7:0/1529733979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc354107900 msgr2=0x7fc35419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc352ffd640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc354107900 0x7fc35419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc352ffd640 1 -- 172.21.15.7:0/1529733979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc354199990 con 0x7fc354101900 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.644+0000 7fc352ffd640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354101900 0x7fc35419f7d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc344031460 tx=0x7fc344004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.645+0000 7fc350ff9640 1 -- 172.21.15.7:0/1529733979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc34403f070 con 0x7fc354101900 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.645+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc354199c20 con 0x7fc354101900 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.645+0000 7fc350ff9640 1 -- 172.21.15.7:0/1529733979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3440035f0 con 0x7fc354101900 2024-09-21T05:05:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.645+0000 7fc350ff9640 1 -- 172.21.15.7:0/1529733979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc344035030 con 0x7fc354101900 2024-09-21T05:05:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.645+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35419a150 con 0x7fc354101900 2024-09-21T05:05:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.646+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc354103040 con 0x7fc354101900 2024-09-21T05:05:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.647+0000 7fc350ff9640 1 -- 172.21.15.7:0/1529733979 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc344042070 con 0x7fc354101900 2024-09-21T05:05:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.647+0000 7fc350ff9640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc34007af80 0x7fc34007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.647+0000 7fc350ff9640 1 -- 172.21.15.7:0/1529733979 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc3440be890 con 0x7fc354101900 2024-09-21T05:05:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.647+0000 7fc34a7fc640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc34007af80 0x7fc34007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.648+0000 7fc34a7fc640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc34007af80 0x7fc34007d440 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc354100060 tx=0x7fc33c005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.652+0000 7fc350ff9640 1 -- 172.21.15.7:0/1529733979 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc344089eb0 con 0x7fc354101900 2024-09-21T05:05:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.860+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc35419af90 con 0x7fc34007af80 2024-09-21T05:05:34.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.861+0000 7fc350ff9640 1 -- 172.21.15.7:0/1529733979 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc35419af90 con 0x7fc34007af80 2024-09-21T05:05:34.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc34007af80 msgr2=0x7fc34007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:34.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc34007af80 0x7fc34007d440 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc354100060 tx=0x7fc33c005f90 comp rx=0 tx=0).stop 2024-09-21T05:05:34.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354101900 msgr2=0x7fc35419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:34.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354101900 0x7fc35419f7d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc344031460 tx=0x7fc344004530 comp rx=0 tx=0).stop 2024-09-21T05:05:34.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 shutdown_connections 2024-09-21T05:05:34.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc34007af80 0x7fc34007d440 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc354107900 0x7fc35419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 --2- 172.21.15.7:0/1529733979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc354101900 0x7fc35419f7d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 >> 172.21.15.7:0/1529733979 conn(0x7fc3540fd600 msgr2=0x7fc3540feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 shutdown_connections 2024-09-21T05:05:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.863+0000 7fc3596fe640 1 -- 172.21.15.7:0/1529733979 wait complete. 2024-09-21T05:05:34.876 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:05:34.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.949+0000 7fbf212bb640 1 -- 172.21.15.7:0/3799576921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 msgr2=0x7fbf1c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.949+0000 7fbf212bb640 1 --2- 172.21.15.7:0/3799576921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 0x7fbf1c107ce0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009c20 tx=0x7fbf0c02a970 comp rx=0 tx=0).stop 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.949+0000 7fbf212bb640 1 -- 172.21.15.7:0/3799576921 shutdown_connections 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.949+0000 7fbf212bb640 1 --2- 172.21.15.7:0/3799576921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1c101900 0x7fbf1c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.949+0000 7fbf212bb640 1 --2- 172.21.15.7:0/3799576921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 0x7fbf1c107ce0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.949+0000 7fbf212bb640 1 -- 172.21.15.7:0/3799576921 >> 172.21.15.7:0/3799576921 conn(0x7fbf1c0fd600 msgr2=0x7fbf1c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.949+0000 7fbf212bb640 1 -- 172.21.15.7:0/3799576921 shutdown_connections 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.949+0000 7fbf212bb640 1 -- 172.21.15.7:0/3799576921 wait complete. 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf212bb640 1 Processor -- start 2024-09-21T05:05:34.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf212bb640 1 -- start start 2024-09-21T05:05:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf212bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1c101900 0x7fbf1c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf212bb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 0x7fbf1c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf212bb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10edd0 con 0x7fbf1c107900 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf212bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10ec50 con 0x7fbf1c101900 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf1a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 0x7fbf1c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf1a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 0x7fbf1c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54626/0 (socket says 172.21.15.7:54626) 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.950+0000 7fbf1a7fc640 1 -- 172.21.15.7:0/3333269859 learned_addr learned my addr 172.21.15.7:0/3333269859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf1affd640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1c101900 0x7fbf1c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf1a7fc640 1 -- 172.21.15.7:0/3333269859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1c101900 msgr2=0x7fbf1c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf1a7fc640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1c101900 0x7fbf1c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf1a7fc640 1 -- 172.21.15.7:0/3333269859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c197840 con 0x7fbf1c107900 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf1affd640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1c101900 0x7fbf1c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:05:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf1a7fc640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 0x7fbf1c19dc00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400d6f0 tx=0x7fbf0400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf13fff640 1 -- 172.21.15.7:0/3333269859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbf04017070 con 0x7fbf1c107900 2024-09-21T05:05:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf13fff640 1 -- 172.21.15.7:0/3333269859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbf0400de00 con 0x7fbf1c107900 2024-09-21T05:05:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf13fff640 1 -- 172.21.15.7:0/3333269859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbf04004bb0 con 0x7fbf1c107900 2024-09-21T05:05:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c197b30 con 0x7fbf1c107900 2024-09-21T05:05:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.951+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c1980c0 con 0x7fbf1c107900 2024-09-21T05:05:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.952+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1c103040 con 0x7fbf1c107900 2024-09-21T05:05:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.953+0000 7fbf13fff640 1 -- 172.21.15.7:0/3333269859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbf04009a30 con 0x7fbf1c107900 2024-09-21T05:05:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.953+0000 7fbf13fff640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf0807af30 0x7fbf0807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.953+0000 7fbf13fff640 1 -- 172.21.15.7:0/3333269859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbf0409dcc0 con 0x7fbf1c107900 2024-09-21T05:05:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.953+0000 7fbf1affd640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf0807af30 0x7fbf0807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.954+0000 7fbf1affd640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf0807af30 0x7fbf0807d3f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c006010 tx=0x7fbf0c02f9d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:34.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:34.958+0000 7fbf13fff640 1 -- 172.21.15.7:0/3333269859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf04069260 con 0x7fbf1c107900 2024-09-21T05:05:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.162+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbf1c198370 con 0x7fbf0807af30 2024-09-21T05:05:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.163+0000 7fbf13fff640 1 -- 172.21.15.7:0/3333269859 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fbf1c198370 con 0x7fbf0807af30 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf0807af30 msgr2=0x7fbf0807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf0807af30 0x7fbf0807d3f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c006010 tx=0x7fbf0c02f9d0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 msgr2=0x7fbf1c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 0x7fbf1c19dc00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400d6f0 tx=0x7fbf0400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 shutdown_connections 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbf0807af30 0x7fbf0807d3f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf1c107900 0x7fbf1c19dc00 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 --2- 172.21.15.7:0/3333269859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1c101900 0x7fbf1c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 >> 172.21.15.7:0/3333269859 conn(0x7fbf1c0fd600 msgr2=0x7fbf1c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 shutdown_connections 2024-09-21T05:05:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.165+0000 7fbf212bb640 1 -- 172.21.15.7:0/3333269859 wait complete. 2024-09-21T05:05:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:35 smithi007 ceph-mon[108238]: pgmap v925: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:35.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.250+0000 7f20a7ad2640 1 -- 172.21.15.7:0/704688553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0100810 msgr2=0x7f20a0100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.250+0000 7f20a7ad2640 1 --2- 172.21.15.7:0/704688553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0100810 0x7f20a0100c70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2090009880 tx=0x7f209002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.250+0000 7f20a7ad2640 1 -- 172.21.15.7:0/704688553 shutdown_connections 2024-09-21T05:05:35.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.250+0000 7f20a7ad2640 1 --2- 172.21.15.7:0/704688553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0100810 0x7f20a0100c70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.250+0000 7f20a7ad2640 1 --2- 172.21.15.7:0/704688553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a0106810 0x7f20a0106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.250+0000 7f20a7ad2640 1 -- 172.21.15.7:0/704688553 >> 172.21.15.7:0/704688553 conn(0x7f20a00fc510 msgr2=0x7f20a00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:35.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.250+0000 7f20a7ad2640 1 -- 172.21.15.7:0/704688553 shutdown_connections 2024-09-21T05:05:35.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.250+0000 7f20a7ad2640 1 -- 172.21.15.7:0/704688553 wait complete. 2024-09-21T05:05:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a7ad2640 1 Processor -- start 2024-09-21T05:05:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a7ad2640 1 -- start start 2024-09-21T05:05:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a7ad2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a0100810 0x7f20a019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a7ad2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0106810 0x7f20a019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a7ad2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20a010dce0 con 0x7f20a0106810 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a7ad2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20a010db60 con 0x7f20a0100810 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a506d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0106810 0x7f20a019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a506d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0106810 0x7f20a019dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54650/0 (socket says 172.21.15.7:54650) 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a506d640 1 -- 172.21.15.7:0/2489035921 learned_addr learned my addr 172.21.15.7:0/2489035921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a586e640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a0100810 0x7f20a019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a506d640 1 -- 172.21.15.7:0/2489035921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a0100810 msgr2=0x7f20a019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a506d640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a0100810 0x7f20a019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a506d640 1 -- 172.21.15.7:0/2489035921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a0197830 con 0x7f20a0106810 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.251+0000 7f20a586e640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a0100810 0x7f20a019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.252+0000 7f20a506d640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0106810 0x7f20a019dbf0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2090031460 tx=0x7f2090004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.252+0000 7f2096ffd640 1 -- 172.21.15.7:0/2489035921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f209003f070 con 0x7f20a0106810 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.252+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a0197ac0 con 0x7f20a0106810 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.252+0000 7f2096ffd640 1 -- 172.21.15.7:0/2489035921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20900035f0 con 0x7f20a0106810 2024-09-21T05:05:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.252+0000 7f2096ffd640 1 -- 172.21.15.7:0/2489035921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2090035030 con 0x7f20a0106810 2024-09-21T05:05:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.252+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a0197ff0 con 0x7f20a0106810 2024-09-21T05:05:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.253+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a0101f50 con 0x7f20a0106810 2024-09-21T05:05:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.254+0000 7f2096ffd640 1 -- 172.21.15.7:0/2489035921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2090042070 con 0x7f20a0106810 2024-09-21T05:05:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.254+0000 7f2096ffd640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f208c07af30 0x7f208c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.254+0000 7f2096ffd640 1 -- 172.21.15.7:0/2489035921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f20900beab0 con 0x7f20a0106810 2024-09-21T05:05:35.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.254+0000 7f20a586e640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f208c07af30 0x7f208c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.255+0000 7f20a586e640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f208c07af30 0x7f208c07d3f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f209c009b50 tx=0x7f209c005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:35.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.258+0000 7f2096ffd640 1 -- 172.21.15.7:0/2489035921 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2090089fb0 con 0x7f20a0106810 2024-09-21T05:05:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:35 smithi053 ceph-mon[80176]: pgmap v925: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:35.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.455+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f20a0198d60 con 0x7f208c07af30 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.465+0000 7f2096ffd640 1 -- 172.21.15.7:0/2489035921 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f20a0198d60 con 0x7f208c07af30 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (27m) 6m ago 38m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (30m) 6m ago 38m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (30m) 6m ago 37m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (25m) 6m ago 38m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (25m) 6m ago 37m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (27m) 6m ago 38m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (21m) 6m ago 33m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:05:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (22m) 6m ago 33m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (6m) 6m ago 33m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (21m) 6m ago 33m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (29m) 6m ago 39m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (28m) 6m ago 37m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (26m) 6m ago 40m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (26m) 6m ago 37m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (28m) 6m ago 38m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (28m) 6m ago 37m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (25m) 6m ago 36m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (24m) 6m ago 36m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (24m) 6m ago 35m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (23m) 6m ago 35m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:05:35.470 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (22m) 6m ago 34m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:05:35.471 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (22m) 6m ago 34m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:05:35.471 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (27m) 6m ago 38m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:05:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f208c07af30 msgr2=0x7f208c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f208c07af30 0x7f208c07d3f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f209c009b50 tx=0x7f209c005f90 comp rx=0 tx=0).stop 2024-09-21T05:05:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0106810 msgr2=0x7f20a019dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0106810 0x7f20a019dbf0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2090031460 tx=0x7f2090004530 comp rx=0 tx=0).stop 2024-09-21T05:05:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 shutdown_connections 2024-09-21T05:05:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f208c07af30 0x7f208c07d3f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a0106810 0x7f20a019dbf0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 --2- 172.21.15.7:0/2489035921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a0100810 0x7f20a019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 >> 172.21.15.7:0/2489035921 conn(0x7f20a00fc510 msgr2=0x7f20a0104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 shutdown_connections 2024-09-21T05:05:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.467+0000 7f20a7ad2640 1 -- 172.21.15.7:0/2489035921 wait complete. 2024-09-21T05:05:35.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.550+0000 7f9981ca0640 1 -- 172.21.15.7:0/1109609201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c102b70 msgr2=0x7f997c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.550+0000 7f9981ca0640 1 --2- 172.21.15.7:0/1109609201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c102b70 0x7f997c102f50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f996c009880 tx=0x7f996c02a530 comp rx=0 tx=0).stop 2024-09-21T05:05:35.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.551+0000 7f9981ca0640 1 -- 172.21.15.7:0/1109609201 shutdown_connections 2024-09-21T05:05:35.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.551+0000 7f9981ca0640 1 --2- 172.21.15.7:0/1109609201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f997c103490 0x7f997c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.551+0000 7f9981ca0640 1 --2- 172.21.15.7:0/1109609201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c102b70 0x7f997c102f50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.551+0000 7f9981ca0640 1 -- 172.21.15.7:0/1109609201 >> 172.21.15.7:0/1109609201 conn(0x7f997c0fc550 msgr2=0x7f997c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.551+0000 7f9981ca0640 1 -- 172.21.15.7:0/1109609201 shutdown_connections 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.551+0000 7f9981ca0640 1 -- 172.21.15.7:0/1109609201 wait complete. 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.551+0000 7f9981ca0640 1 Processor -- start 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f9981ca0640 1 -- start start 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f9981ca0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f997c102b70 0x7f997c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f9981ca0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c103490 0x7f997c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f9981ca0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f997c10edc0 con 0x7f997c103490 2024-09-21T05:05:35.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f9981ca0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f997c10ec40 con 0x7f997c102b70 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f997affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c103490 0x7f997c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f997affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c103490 0x7f997c19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54670/0 (socket says 172.21.15.7:54670) 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f997affd640 1 -- 172.21.15.7:0/2502147172 learned_addr learned my addr 172.21.15.7:0/2502147172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f997b7fe640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f997c102b70 0x7f997c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f997affd640 1 -- 172.21.15.7:0/2502147172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f997c102b70 msgr2=0x7f997c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f997affd640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f997c102b70 0x7f997c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f997affd640 1 -- 172.21.15.7:0/2502147172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c1977b0 con 0x7f997c103490 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.552+0000 7f997b7fe640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f997c102b70 0x7f997c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:05:35.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.553+0000 7f997affd640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c103490 0x7f997c19db70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f996400d8d0 tx=0x7f996400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:35.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.553+0000 7f9978ff9640 1 -- 172.21.15.7:0/2502147172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9964017070 con 0x7f997c103490 2024-09-21T05:05:35.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.553+0000 7f9978ff9640 1 -- 172.21.15.7:0/2502147172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9964010460 con 0x7f997c103490 2024-09-21T05:05:35.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.553+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c197aa0 con 0x7f997c103490 2024-09-21T05:05:35.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.553+0000 7f9978ff9640 1 -- 172.21.15.7:0/2502147172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f996400f5f0 con 0x7f997c103490 2024-09-21T05:05:35.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.553+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c198030 con 0x7f997c103490 2024-09-21T05:05:35.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.555+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997c06a4c0 con 0x7f997c103490 2024-09-21T05:05:35.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.555+0000 7f9978ff9640 1 -- 172.21.15.7:0/2502147172 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9964009a30 con 0x7f997c103490 2024-09-21T05:05:35.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.555+0000 7f9978ff9640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f996807af40 0x7f996807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.556+0000 7f9978ff9640 1 -- 172.21.15.7:0/2502147172 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f996409df00 con 0x7f997c103490 2024-09-21T05:05:35.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.556+0000 7f997b7fe640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f996807af40 0x7f996807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.556+0000 7f997b7fe640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f996807af40 0x7f996807d400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f996c004f90 tx=0x7f996c0041a0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:35.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.560+0000 7f9978ff9640 1 -- 172.21.15.7:0/2502147172 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99640694a0 con 0x7f997c103490 2024-09-21T05:05:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.822+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f997c105fd0 con 0x7f997c103490 2024-09-21T05:05:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.823+0000 7f9978ff9640 1 -- 172.21.15.7:0/2502147172 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f996406e350 con 0x7f997c103490 2024-09-21T05:05:35.827 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:05:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:05:35.827 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:05:35.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:05:35.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:05:35.829 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:05:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f996807af40 msgr2=0x7f996807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f996807af40 0x7f996807d400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f996c004f90 tx=0x7f996c0041a0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c103490 msgr2=0x7f997c19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c103490 0x7f997c19db70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f996400d8d0 tx=0x7f996400dd90 comp rx=0 tx=0).stop 2024-09-21T05:05:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 shutdown_connections 2024-09-21T05:05:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f996807af40 0x7f996807d400 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f997c103490 0x7f997c19db70 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 --2- 172.21.15.7:0/2502147172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f997c102b70 0x7f997c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 >> 172.21.15.7:0/2502147172 conn(0x7f997c0fc550 msgr2=0x7f997c108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 shutdown_connections 2024-09-21T05:05:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.825+0000 7f9981ca0640 1 -- 172.21.15.7:0/2502147172 wait complete. 2024-09-21T05:05:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.909+0000 7f14d5c30640 1 -- 172.21.15.7:0/795637690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 msgr2=0x7f14d0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.909+0000 7f14d5c30640 1 --2- 172.21.15.7:0/795637690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 0x7f14d0102ea0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009880 tx=0x7f14c002a530 comp rx=0 tx=0).stop 2024-09-21T05:05:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.909+0000 7f14d5c30640 1 -- 172.21.15.7:0/795637690 shutdown_connections 2024-09-21T05:05:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.909+0000 7f14d5c30640 1 --2- 172.21.15.7:0/795637690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d01033e0 0x7f14d010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.909+0000 7f14d5c30640 1 --2- 172.21.15.7:0/795637690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 0x7f14d0102ea0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.909+0000 7f14d5c30640 1 -- 172.21.15.7:0/795637690 >> 172.21.15.7:0/795637690 conn(0x7f14d00fc520 msgr2=0x7f14d00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:35.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.909+0000 7f14d5c30640 1 -- 172.21.15.7:0/795637690 shutdown_connections 2024-09-21T05:05:35.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.909+0000 7f14d5c30640 1 -- 172.21.15.7:0/795637690 wait complete. 2024-09-21T05:05:35.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.910+0000 7f14d5c30640 1 Processor -- start 2024-09-21T05:05:35.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.910+0000 7f14d5c30640 1 -- start start 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.910+0000 7f14d5c30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 0x7f14d019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.910+0000 7f14d5c30640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d01033e0 0x7f14d019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.910+0000 7f14d5c30640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010edf0 con 0x7f14d0102ac0 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.910+0000 7f14d5c30640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010ec70 con 0x7f14d01033e0 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14cf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 0x7f14d019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14cf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 0x7f14d019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54688/0 (socket says 172.21.15.7:54688) 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14cf7fe640 1 -- 172.21.15.7:0/3273811883 learned_addr learned my addr 172.21.15.7:0/3273811883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14ceffd640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d01033e0 0x7f14d019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14cf7fe640 1 -- 172.21.15.7:0/3273811883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d01033e0 msgr2=0x7f14d019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14cf7fe640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d01033e0 0x7f14d019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14cf7fe640 1 -- 172.21.15.7:0/3273811883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d01977e0 con 0x7f14d0102ac0 2024-09-21T05:05:35.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14cf7fe640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 0x7f14d019d660 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f14c0031460 tx=0x7f14c00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14ccff9640 1 -- 172.21.15.7:0/3273811883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14c003f070 con 0x7f14d0102ac0 2024-09-21T05:05:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d0197a70 con 0x7f14d0102ac0 2024-09-21T05:05:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14ccff9640 1 -- 172.21.15.7:0/3273811883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14c00035f0 con 0x7f14d0102ac0 2024-09-21T05:05:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.911+0000 7f14ccff9640 1 -- 172.21.15.7:0/3273811883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14c0035030 con 0x7f14d0102ac0 2024-09-21T05:05:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.912+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d0197fa0 con 0x7f14d0102ac0 2024-09-21T05:05:35.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.913+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d006aa90 con 0x7f14d0102ac0 2024-09-21T05:05:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.913+0000 7f14ccff9640 1 -- 172.21.15.7:0/3273811883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f14c0042070 con 0x7f14d0102ac0 2024-09-21T05:05:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.914+0000 7f14ccff9640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b807af30 0x7f14b807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.914+0000 7f14ceffd640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b807af30 0x7f14b807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.914+0000 7f14ccff9640 1 -- 172.21.15.7:0/3273811883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f14c00bea80 con 0x7f14d0102ac0 2024-09-21T05:05:35.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.915+0000 7f14ceffd640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b807af30 0x7f14b807d3f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f14c40058c0 tx=0x7f14c4005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:35.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:35.919+0000 7f14ccff9640 1 -- 172.21.15.7:0/3273811883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14c008a020 con 0x7f14d0102ac0 2024-09-21T05:05:36.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.147+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f14d0105f90 con 0x7f14d0102ac0 2024-09-21T05:05:36.151 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:36 smithi007 ceph-mon[108238]: from='client.35488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:36.151 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:36 smithi007 ceph-mon[108238]: from='client.35492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:36.151 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:36 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2502147172' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:05:36.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.147+0000 7f14ccff9640 1 -- 172.21.15.7:0/3273811883 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f14c008c020 con 0x7f14d0102ac0 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:05:36.152 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:05:36.153 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:05:36.154 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:05:36.155 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:05:36.155 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:05:36.155 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:05:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b807af30 msgr2=0x7f14b807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b807af30 0x7f14b807d3f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f14c40058c0 tx=0x7f14c4005850 comp rx=0 tx=0).stop 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 msgr2=0x7f14d019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 0x7f14d019d660 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f14c0031460 tx=0x7f14c00045e0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 shutdown_connections 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14b807af30 0x7f14b807d3f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d01033e0 0x7f14d019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 --2- 172.21.15.7:0/3273811883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d0102ac0 0x7f14d019d660 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 >> 172.21.15.7:0/3273811883 conn(0x7f14d00fc520 msgr2=0x7f14d0108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 shutdown_connections 2024-09-21T05:05:36.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.151+0000 7f14d5c30640 1 -- 172.21.15.7:0/3273811883 wait complete. 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.237+0000 7f28c9faf640 1 -- 172.21.15.7:0/1173678061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4107900 msgr2=0x7f28c4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.237+0000 7f28c9faf640 1 --2- 172.21.15.7:0/1173678061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4107900 0x7f28c4107ce0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009880 tx=0x7f28b402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.237+0000 7f28c9faf640 1 -- 172.21.15.7:0/1173678061 shutdown_connections 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.237+0000 7f28c9faf640 1 --2- 172.21.15.7:0/1173678061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28c4101900 0x7f28c4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.237+0000 7f28c9faf640 1 --2- 172.21.15.7:0/1173678061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4107900 0x7f28c4107ce0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.237+0000 7f28c9faf640 1 -- 172.21.15.7:0/1173678061 >> 172.21.15.7:0/1173678061 conn(0x7f28c40fd600 msgr2=0x7f28c40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.237+0000 7f28c9faf640 1 -- 172.21.15.7:0/1173678061 shutdown_connections 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.237+0000 7f28c9faf640 1 -- 172.21.15.7:0/1173678061 wait complete. 2024-09-21T05:05:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.238+0000 7f28c9faf640 1 Processor -- start 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.238+0000 7f28c9faf640 1 -- start start 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.238+0000 7f28c9faf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4101900 0x7f28c419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.238+0000 7f28c9faf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28c4107900 0x7f28c419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.238+0000 7f28c9faf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f28c410ee30 con 0x7f28c4101900 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.238+0000 7f28c9faf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f28c410ecb0 con 0x7f28c4107900 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4101900 0x7f28c419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4101900 0x7f28c419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54696/0 (socket says 172.21.15.7:54696) 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c37fe640 1 -- 172.21.15.7:0/3462339946 learned_addr learned my addr 172.21.15.7:0/3462339946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c2ffd640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28c4107900 0x7f28c419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c37fe640 1 -- 172.21.15.7:0/3462339946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28c4107900 msgr2=0x7f28c419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c37fe640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28c4107900 0x7f28c419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c37fe640 1 -- 172.21.15.7:0/3462339946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c4197840 con 0x7f28c4101900 2024-09-21T05:05:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c37fe640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4101900 0x7f28c419d6c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f28b4031460 tx=0x7f28b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c0ff9640 1 -- 172.21.15.7:0/3462339946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f28b403f070 con 0x7f28c4101900 2024-09-21T05:05:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c4197ad0 con 0x7f28c4101900 2024-09-21T05:05:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c0ff9640 1 -- 172.21.15.7:0/3462339946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f28b4004cf0 con 0x7f28c4101900 2024-09-21T05:05:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c0ff9640 1 -- 172.21.15.7:0/3462339946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f28b4035030 con 0x7f28c4101900 2024-09-21T05:05:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.239+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c4198000 con 0x7f28c4101900 2024-09-21T05:05:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.241+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c4103040 con 0x7f28c4101900 2024-09-21T05:05:36.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.241+0000 7f28c0ff9640 1 -- 172.21.15.7:0/3462339946 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f28b4042070 con 0x7f28c4101900 2024-09-21T05:05:36.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.242+0000 7f28c0ff9640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f28ac07af30 0x7f28ac07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:36.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.242+0000 7f28c0ff9640 1 -- 172.21.15.7:0/3462339946 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f28b40beaf0 con 0x7f28c4101900 2024-09-21T05:05:36.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.242+0000 7f28c2ffd640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f28ac07af30 0x7f28ac07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:36.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.242+0000 7f28c2ffd640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f28ac07af30 0x7f28ac07d3f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28c40ffc00 tx=0x7f28b8009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.246+0000 7f28c0ff9640 1 -- 172.21.15.7:0/3462339946 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28b408a090 con 0x7f28c4101900 2024-09-21T05:05:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:36 smithi053 ceph-mon[80176]: from='client.35488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:36 smithi053 ceph-mon[80176]: from='client.35492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:36 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2502147172' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:05:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.449+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f28c41982b0 con 0x7f28ac07af30 2024-09-21T05:05:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.451+0000 7f28c0ff9640 1 -- 172.21.15.7:0/3462339946 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f28c41982b0 con 0x7f28ac07af30 2024-09-21T05:05:36.454 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:05:36.455 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:05:36.456 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:05:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f28ac07af30 msgr2=0x7f28ac07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f28ac07af30 0x7f28ac07d3f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28c40ffc00 tx=0x7f28b8009110 comp rx=0 tx=0).stop 2024-09-21T05:05:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4101900 msgr2=0x7f28c419d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4101900 0x7f28c419d6c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f28b4031460 tx=0x7f28b40048f0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 shutdown_connections 2024-09-21T05:05:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f28ac07af30 0x7f28ac07d3f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28c4107900 0x7f28c419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 --2- 172.21.15.7:0/3462339946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28c4101900 0x7f28c419d6c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 >> 172.21.15.7:0/3462339946 conn(0x7f28c40fd600 msgr2=0x7f28c40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 shutdown_connections 2024-09-21T05:05:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.453+0000 7f28c9faf640 1 -- 172.21.15.7:0/3462339946 wait complete. 2024-09-21T05:05:36.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.537+0000 7fec60289640 1 -- 172.21.15.7:0/3969212231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58103480 msgr2=0x7fec5810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.537+0000 7fec60289640 1 --2- 172.21.15.7:0/3969212231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58103480 0x7fec5810b970 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fec54009880 tx=0x7fec5402a530 comp rx=0 tx=0).stop 2024-09-21T05:05:36.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.537+0000 7fec60289640 1 -- 172.21.15.7:0/3969212231 shutdown_connections 2024-09-21T05:05:36.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.537+0000 7fec60289640 1 --2- 172.21.15.7:0/3969212231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58103480 0x7fec5810b970 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.537+0000 7fec60289640 1 --2- 172.21.15.7:0/3969212231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec58102b60 0x7fec58102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.537+0000 7fec60289640 1 -- 172.21.15.7:0/3969212231 >> 172.21.15.7:0/3969212231 conn(0x7fec580fc520 msgr2=0x7fec580fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:36.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.537+0000 7fec60289640 1 -- 172.21.15.7:0/3969212231 shutdown_connections 2024-09-21T05:05:36.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.537+0000 7fec60289640 1 -- 172.21.15.7:0/3969212231 wait complete. 2024-09-21T05:05:36.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec60289640 1 Processor -- start 2024-09-21T05:05:36.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec60289640 1 -- start start 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec60289640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58102b60 0x7fec5819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec60289640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec58103480 0x7fec5819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec60289640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec5810edf0 con 0x7fec58102b60 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec60289640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fec5810ec70 con 0x7fec58103480 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec5e025640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58102b60 0x7fec5819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec5e025640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58102b60 0x7fec5819f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54718/0 (socket says 172.21.15.7:54718) 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec5e025640 1 -- 172.21.15.7:0/1332253922 learned_addr learned my addr 172.21.15.7:0/1332253922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.538+0000 7fec5d824640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec58103480 0x7fec5819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec5e025640 1 -- 172.21.15.7:0/1332253922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec58103480 msgr2=0x7fec5819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec5e025640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec58103480 0x7fec5819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec5e025640 1 -- 172.21.15.7:0/1332253922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec58199aa0 con 0x7fec58102b60 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec5e025640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58102b60 0x7fec5819f840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fec4800d6f0 tx=0x7fec4800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec4f7fe640 1 -- 172.21.15.7:0/1332253922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fec48017070 con 0x7fec58102b60 2024-09-21T05:05:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec4f7fe640 1 -- 172.21.15.7:0/1332253922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fec4800de00 con 0x7fec58102b60 2024-09-21T05:05:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec4f7fe640 1 -- 172.21.15.7:0/1332253922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fec48004bb0 con 0x7fec58102b60 2024-09-21T05:05:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec58199d90 con 0x7fec58102b60 2024-09-21T05:05:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.539+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5819a2f0 con 0x7fec58102b60 2024-09-21T05:05:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.540+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5806a9f0 con 0x7fec58102b60 2024-09-21T05:05:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.541+0000 7fec4f7fe640 1 -- 172.21.15.7:0/1332253922 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fec48009a30 con 0x7fec58102b60 2024-09-21T05:05:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.541+0000 7fec4f7fe640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fec4407af80 0x7fec4407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:05:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.541+0000 7fec4f7fe640 1 -- 172.21.15.7:0/1332253922 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fec4809db20 con 0x7fec58102b60 2024-09-21T05:05:36.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.542+0000 7fec5d824640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fec4407af80 0x7fec4407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:05:36.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.542+0000 7fec5d824640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fec4407af80 0x7fec4407d440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fec5819b1f0 tx=0x7fec54004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:05:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.546+0000 7fec4f7fe640 1 -- 172.21.15.7:0/1332253922 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec48069140 con 0x7fec58102b60 2024-09-21T05:05:36.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.808+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fec5819af30 con 0x7fec58102b60 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.808+0000 7fec4f7fe640 1 -- 172.21.15.7:0/1332253922 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fec4806dff0 con 0x7fec58102b60 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:05:36.813 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:05:36.814 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:05:36.815 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:05:36.815 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:05:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fec4407af80 msgr2=0x7fec4407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fec4407af80 0x7fec4407d440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fec5819b1f0 tx=0x7fec54004f30 comp rx=0 tx=0).stop 2024-09-21T05:05:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58102b60 msgr2=0x7fec5819f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:05:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58102b60 0x7fec5819f840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fec4800d6f0 tx=0x7fec4800dbb0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 shutdown_connections 2024-09-21T05:05:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fec4407af80 0x7fec4407d440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec58103480 0x7fec5819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 --2- 172.21.15.7:0/1332253922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec58102b60 0x7fec5819f840 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:05:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 >> 172.21.15.7:0/1332253922 conn(0x7fec580fc520 msgr2=0x7fec58100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:05:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 shutdown_connections 2024-09-21T05:05:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:05:36.811+0000 7fec60289640 1 -- 172.21.15.7:0/1332253922 wait complete. 2024-09-21T05:05:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:37 smithi053 ceph-mon[80176]: pgmap v926: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:37 smithi053 ceph-mon[80176]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:37 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3273811883' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:05:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:37 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1332253922' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:05:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:37 smithi007 ceph-mon[108238]: pgmap v926: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:37 smithi007 ceph-mon[108238]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:37 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3273811883' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:05:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:37 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1332253922' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:05:38.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:38 smithi007 ceph-mon[108238]: from='client.35508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:38.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:38 smithi053 ceph-mon[80176]: from='client.35508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:05:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:39 smithi053 ceph-mon[80176]: pgmap v927: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:39 smithi007 ceph-mon[108238]: pgmap v927: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:41 smithi007 ceph-mon[108238]: pgmap v928: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:41 smithi053 ceph-mon[80176]: pgmap v928: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:43 smithi053 ceph-mon[80176]: pgmap v929: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:43 smithi007 ceph-mon[108238]: pgmap v929: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:05:44.344 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:05:45.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:45 smithi053 ceph-mon[80176]: pgmap v930: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:45 smithi007 ceph-mon[108238]: pgmap v930: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:47 smithi007 ceph-mon[108238]: pgmap v931: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:47 smithi053 ceph-mon[80176]: pgmap v931: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:49.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:49 smithi053 ceph-mon[80176]: pgmap v932: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:49 smithi007 ceph-mon[108238]: pgmap v932: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:51.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:51 smithi053 ceph-mon[80176]: pgmap v933: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:51 smithi007 ceph-mon[108238]: pgmap v933: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:53 smithi007 ceph-mon[108238]: pgmap v934: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:53 smithi053 ceph-mon[80176]: pgmap v934: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:55.396 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:55 smithi053 ceph-mon[80176]: pgmap v935: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:55 smithi007 ceph-mon[108238]: pgmap v935: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:57.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:57 smithi053 ceph-mon[80176]: pgmap v936: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:57 smithi007 ceph-mon[108238]: pgmap v936: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:59 smithi007 ceph-mon[108238]: pgmap v937: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:05:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:05:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:59 smithi053 ceph-mon[80176]: pgmap v937: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:05:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:05:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:06:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:01 smithi053 ceph-mon[80176]: pgmap v938: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:01 smithi007 ceph-mon[108238]: pgmap v938: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:03.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:03 smithi053 ceph-mon[80176]: pgmap v939: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:03 smithi007 ceph-mon[108238]: pgmap v939: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:05 smithi007 ceph-mon[108238]: pgmap v940: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:05 smithi053 ceph-mon[80176]: pgmap v940: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.910+0000 7f36369c3640 1 -- 172.21.15.7:0/3650623719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 msgr2=0x7f3630100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.910+0000 7f36369c3640 1 --2- 172.21.15.7:0/3650623719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 0x7f3630100500 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3620009880 tx=0x7f362002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.910+0000 7f36369c3640 1 -- 172.21.15.7:0/3650623719 shutdown_connections 2024-09-21T05:06:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.910+0000 7f36369c3640 1 --2- 172.21.15.7:0/3650623719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3630100ad0 0x7f3630106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.910+0000 7f36369c3640 1 --2- 172.21.15.7:0/3650623719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 0x7f3630100500 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.910+0000 7f36369c3640 1 -- 172.21.15.7:0/3650623719 >> 172.21.15.7:0/3650623719 conn(0x7f36300fbc90 msgr2=0x7f36300fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.910+0000 7f36369c3640 1 -- 172.21.15.7:0/3650623719 shutdown_connections 2024-09-21T05:06:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.910+0000 7f36369c3640 1 -- 172.21.15.7:0/3650623719 wait complete. 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.911+0000 7f36369c3640 1 Processor -- start 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.911+0000 7f36369c3640 1 -- start start 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.911+0000 7f36369c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 0x7f36301010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.911+0000 7f36369c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3630100ad0 0x7f36301015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.911+0000 7f36369c3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f363010ee80 con 0x7f3630100120 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.911+0000 7f36369c3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f363010ed00 con 0x7f3630100ad0 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 0x7f36301010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 0x7f36301010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39998/0 (socket says 172.21.15.7:39998) 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362ffff640 1 -- 172.21.15.7:0/450461086 learned_addr learned my addr 172.21.15.7:0/450461086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f3627fff640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3630100ad0 0x7f36301015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362ffff640 1 -- 172.21.15.7:0/450461086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3630100ad0 msgr2=0x7f36301015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362ffff640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3630100ad0 0x7f36301015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362ffff640 1 -- 172.21.15.7:0/450461086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3630101b20 con 0x7f3630100120 2024-09-21T05:06:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f3627fff640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3630100ad0 0x7f36301015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:06:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362ffff640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 0x7f36301010a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3620031460 tx=0x7f36200045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362dffb640 1 -- 172.21.15.7:0/450461086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f362003f070 con 0x7f3630100120 2024-09-21T05:06:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362dffb640 1 -- 172.21.15.7:0/450461086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3620004b10 con 0x7f3630100120 2024-09-21T05:06:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3630101db0 con 0x7f3630100120 2024-09-21T05:06:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.912+0000 7f362dffb640 1 -- 172.21.15.7:0/450461086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3620035030 con 0x7f3630100120 2024-09-21T05:06:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.913+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36301a3e60 con 0x7f3630100120 2024-09-21T05:06:06.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.914+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36301031e0 con 0x7f3630100120 2024-09-21T05:06:06.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.914+0000 7f362dffb640 1 -- 172.21.15.7:0/450461086 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3620042070 con 0x7f3630100120 2024-09-21T05:06:06.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.915+0000 7f362dffb640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f361c07af40 0x7f361c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:06.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.915+0000 7f362dffb640 1 -- 172.21.15.7:0/450461086 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f36200bebb0 con 0x7f3630100120 2024-09-21T05:06:06.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.915+0000 7f3627fff640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f361c07af40 0x7f361c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:06.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.915+0000 7f3627fff640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f361c07af40 0x7f361c07d400 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f36180058c0 tx=0x7f3618005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:06.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:06.919+0000 7f362dffb640 1 -- 172.21.15.7:0/450461086 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f36200c5080 con 0x7f3630100120 2024-09-21T05:06:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.124+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3630062630 con 0x7f361c07af40 2024-09-21T05:06:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.126+0000 7f362dffb640 1 -- 172.21.15.7:0/450461086 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3630062630 con 0x7f361c07af40 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f361c07af40 msgr2=0x7f361c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f361c07af40 0x7f361c07d400 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f36180058c0 tx=0x7f3618005850 comp rx=0 tx=0).stop 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 msgr2=0x7f36301010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 0x7f36301010a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3620031460 tx=0x7f36200045e0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 shutdown_connections 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f361c07af40 0x7f361c07d400 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3630100ad0 0x7f36301015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 --2- 172.21.15.7:0/450461086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3630100120 0x7f36301010a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 >> 172.21.15.7:0/450461086 conn(0x7f36300fbc90 msgr2=0x7f36300fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 shutdown_connections 2024-09-21T05:06:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.128+0000 7f36369c3640 1 -- 172.21.15.7:0/450461086 wait complete. 2024-09-21T05:06:07.140 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:06:07.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:07 smithi007 ceph-mon[108238]: pgmap v941: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.215+0000 7f347860a640 1 -- 172.21.15.7:0/2355712104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 msgr2=0x7f3470107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.215+0000 7f347860a640 1 --2- 172.21.15.7:0/2355712104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 0x7f3470107ce0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3468009880 tx=0x7f346802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.215+0000 7f347860a640 1 -- 172.21.15.7:0/2355712104 shutdown_connections 2024-09-21T05:06:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.215+0000 7f347860a640 1 --2- 172.21.15.7:0/2355712104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3470101900 0x7f3470101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.215+0000 7f347860a640 1 --2- 172.21.15.7:0/2355712104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 0x7f3470107ce0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.215+0000 7f347860a640 1 -- 172.21.15.7:0/2355712104 >> 172.21.15.7:0/2355712104 conn(0x7f34700fd600 msgr2=0x7f34700ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.215+0000 7f347860a640 1 -- 172.21.15.7:0/2355712104 shutdown_connections 2024-09-21T05:06:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.215+0000 7f347860a640 1 -- 172.21.15.7:0/2355712104 wait complete. 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f347860a640 1 Processor -- start 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f347860a640 1 -- start start 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f347860a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3470101900 0x7f347019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f347860a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 0x7f347019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f347860a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f347010edd0 con 0x7f3470107900 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f347860a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f347010ec50 con 0x7f3470101900 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f3475ba5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 0x7f347019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f3475ba5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 0x7f347019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40010/0 (socket says 172.21.15.7:40010) 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.216+0000 7f3475ba5640 1 -- 172.21.15.7:0/1687076899 learned_addr learned my addr 172.21.15.7:0/1687076899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f34763a6640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3470101900 0x7f347019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f3475ba5640 1 -- 172.21.15.7:0/1687076899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3470101900 msgr2=0x7f347019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f3475ba5640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3470101900 0x7f347019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f3475ba5640 1 -- 172.21.15.7:0/1687076899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3470197840 con 0x7f3470107900 2024-09-21T05:06:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f34763a6640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3470101900 0x7f347019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:06:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f3475ba5640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 0x7f347019dc00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f346400d6f0 tx=0x7f346400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f346f7fe640 1 -- 172.21.15.7:0/1687076899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3464017070 con 0x7f3470107900 2024-09-21T05:06:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f346f7fe640 1 -- 172.21.15.7:0/1687076899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f346400de00 con 0x7f3470107900 2024-09-21T05:06:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3470197b30 con 0x7f3470107900 2024-09-21T05:06:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f346f7fe640 1 -- 172.21.15.7:0/1687076899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3464004bb0 con 0x7f3470107900 2024-09-21T05:06:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.217+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34701980c0 con 0x7f3470107900 2024-09-21T05:06:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.218+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3470103040 con 0x7f3470107900 2024-09-21T05:06:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.219+0000 7f346f7fe640 1 -- 172.21.15.7:0/1687076899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3464004e60 con 0x7f3470107900 2024-09-21T05:06:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.219+0000 7f346f7fe640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f345807af30 0x7f345807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.220+0000 7f346f7fe640 1 -- 172.21.15.7:0/1687076899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f346409dd50 con 0x7f3470107900 2024-09-21T05:06:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.220+0000 7f34763a6640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f345807af30 0x7f345807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.220+0000 7f34763a6640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f345807af30 0x7f345807d3f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3468000c00 tx=0x7f3468031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.224+0000 7f346f7fe640 1 -- 172.21.15.7:0/1687076899 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3464069250 con 0x7f3470107900 2024-09-21T05:06:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.428+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3470198430 con 0x7f345807af30 2024-09-21T05:06:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.429+0000 7f346f7fe640 1 -- 172.21.15.7:0/1687076899 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3470198430 con 0x7f345807af30 2024-09-21T05:06:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f345807af30 msgr2=0x7f345807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f345807af30 0x7f345807d3f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3468000c00 tx=0x7f3468031400 comp rx=0 tx=0).stop 2024-09-21T05:06:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 msgr2=0x7f347019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 0x7f347019dc00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f346400d6f0 tx=0x7f346400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 shutdown_connections 2024-09-21T05:06:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f345807af30 0x7f345807d3f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3470107900 0x7f347019dc00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 --2- 172.21.15.7:0/1687076899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3470101900 0x7f347019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 >> 172.21.15.7:0/1687076899 conn(0x7f34700fd600 msgr2=0x7f34700fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 shutdown_connections 2024-09-21T05:06:07.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.431+0000 7f347860a640 1 -- 172.21.15.7:0/1687076899 wait complete. 2024-09-21T05:06:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:07 smithi053 ceph-mon[80176]: pgmap v941: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.515+0000 7f1dc7299640 1 -- 172.21.15.7:0/207011678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc01033e0 msgr2=0x7f1dc010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.515+0000 7f1dc7299640 1 --2- 172.21.15.7:0/207011678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc01033e0 0x7f1dc010b8d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009880 tx=0x7f1db002a530 comp rx=0 tx=0).stop 2024-09-21T05:06:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.515+0000 7f1dc7299640 1 -- 172.21.15.7:0/207011678 shutdown_connections 2024-09-21T05:06:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.515+0000 7f1dc7299640 1 --2- 172.21.15.7:0/207011678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc01033e0 0x7f1dc010b8d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.515+0000 7f1dc7299640 1 --2- 172.21.15.7:0/207011678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dc0102ac0 0x7f1dc0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.515+0000 7f1dc7299640 1 -- 172.21.15.7:0/207011678 >> 172.21.15.7:0/207011678 conn(0x7f1dc00fc520 msgr2=0x7f1dc00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.516+0000 7f1dc7299640 1 -- 172.21.15.7:0/207011678 shutdown_connections 2024-09-21T05:06:07.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.516+0000 7f1dc7299640 1 -- 172.21.15.7:0/207011678 wait complete. 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.516+0000 7f1dc7299640 1 Processor -- start 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.516+0000 7f1dc7299640 1 -- start start 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc7299640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0102ac0 0x7f1dc019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc7299640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dc01033e0 0x7f1dc019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc7299640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc010edf0 con 0x7f1dc0102ac0 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc7299640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc010ec70 con 0x7f1dc01033e0 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc5035640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0102ac0 0x7f1dc019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc5035640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0102ac0 0x7f1dc019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40030/0 (socket says 172.21.15.7:40030) 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc5035640 1 -- 172.21.15.7:0/4024692409 learned_addr learned my addr 172.21.15.7:0/4024692409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc4834640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dc01033e0 0x7f1dc019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc5035640 1 -- 172.21.15.7:0/4024692409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dc01033e0 msgr2=0x7f1dc019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc5035640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dc01033e0 0x7f1dc019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.517+0000 7f1dc5035640 1 -- 172.21.15.7:0/4024692409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc0197800 con 0x7f1dc0102ac0 2024-09-21T05:06:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.518+0000 7f1dc5035640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0102ac0 0x7f1dc019d680 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00c8f0 tx=0x7f1dbc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.518+0000 7f1db67fc640 1 -- 172.21.15.7:0/4024692409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1dbc0079b0 con 0x7f1dc0102ac0 2024-09-21T05:06:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.518+0000 7f1db67fc640 1 -- 172.21.15.7:0/4024692409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1dbc0054e0 con 0x7f1dc0102ac0 2024-09-21T05:06:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.518+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc0197af0 con 0x7f1dc0102ac0 2024-09-21T05:06:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.518+0000 7f1db67fc640 1 -- 172.21.15.7:0/4024692409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1dbc004dd0 con 0x7f1dc0102ac0 2024-09-21T05:06:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.518+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc0198050 con 0x7f1dc0102ac0 2024-09-21T05:06:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.519+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc006aa90 con 0x7f1dc0102ac0 2024-09-21T05:06:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.520+0000 7f1db67fc640 1 -- 172.21.15.7:0/4024692409 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1dbc005020 con 0x7f1dc0102ac0 2024-09-21T05:06:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.520+0000 7f1db67fc640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1dac07af30 0x7f1dac07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.521+0000 7f1db67fc640 1 -- 172.21.15.7:0/4024692409 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f1dbc09dff0 con 0x7f1dc0102ac0 2024-09-21T05:06:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.521+0000 7f1dc4834640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1dac07af30 0x7f1dac07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.521+0000 7f1dc4834640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1dac07af30 0x7f1dac07d3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0198b50 tx=0x7f1db0031420 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.525+0000 7f1db67fc640 1 -- 172.21.15.7:0/4024692409 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1dbc014030 con 0x7f1dc0102ac0 2024-09-21T05:06:07.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.721+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1dc01983c0 con 0x7f1dac07af30 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.731+0000 7f1db67fc640 1 -- 172.21.15.7:0/4024692409 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f1dc01983c0 con 0x7f1dac07af30 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (28m) 6m ago 39m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (31m) 6m ago 39m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (30m) 6m ago 38m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (26m) 6m ago 39m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (26m) 6m ago 38m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (27m) 6m ago 38m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:06:07.734 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (22m) 6m ago 33m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (22m) 6m ago 33m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (6m) 6m ago 33m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (22m) 6m ago 33m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (30m) 6m ago 40m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (29m) 6m ago 38m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (26m) 6m ago 40m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (26m) 6m ago 38m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (29m) 6m ago 39m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (28m) 6m ago 38m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (25m) 6m ago 37m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (25m) 6m ago 36m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:06:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (24m) 6m ago 36m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:06:07.736 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (24m) 6m ago 35m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:06:07.736 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (23m) 6m ago 35m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:06:07.736 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (22m) 6m ago 34m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:06:07.736 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (28m) 6m ago 38m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:06:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1dac07af30 msgr2=0x7f1dac07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1dac07af30 0x7f1dac07d3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0198b50 tx=0x7f1db0031420 comp rx=0 tx=0).stop 2024-09-21T05:06:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0102ac0 msgr2=0x7f1dc019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0102ac0 0x7f1dc019d680 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00c8f0 tx=0x7f1dbc00cdb0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 shutdown_connections 2024-09-21T05:06:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1dac07af30 0x7f1dac07d3f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dc01033e0 0x7f1dc019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 --2- 172.21.15.7:0/4024692409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dc0102ac0 0x7f1dc019d680 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 >> 172.21.15.7:0/4024692409 conn(0x7f1dc00fc520 msgr2=0x7f1dc0108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 shutdown_connections 2024-09-21T05:06:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.733+0000 7f1dc7299640 1 -- 172.21.15.7:0/4024692409 wait complete. 2024-09-21T05:06:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.816+0000 7f350fd15640 1 -- 172.21.15.7:0/55299780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 msgr2=0x7f3508101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.816+0000 7f350fd15640 1 --2- 172.21.15.7:0/55299780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 0x7f3508101d60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009880 tx=0x7f34f802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.816+0000 7f350fd15640 1 -- 172.21.15.7:0/55299780 shutdown_connections 2024-09-21T05:06:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.816+0000 7f350fd15640 1 --2- 172.21.15.7:0/55299780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 0x7f3508101d60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.816+0000 7f350fd15640 1 --2- 172.21.15.7:0/55299780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3508107900 0x7f3508107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.816+0000 7f350fd15640 1 -- 172.21.15.7:0/55299780 >> 172.21.15.7:0/55299780 conn(0x7f35080fd600 msgr2=0x7f35080ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:07.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.816+0000 7f350fd15640 1 -- 172.21.15.7:0/55299780 shutdown_connections 2024-09-21T05:06:07.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.816+0000 7f350fd15640 1 -- 172.21.15.7:0/55299780 wait complete. 2024-09-21T05:06:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350fd15640 1 Processor -- start 2024-09-21T05:06:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350fd15640 1 -- start start 2024-09-21T05:06:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350fd15640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 0x7f35080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350fd15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3508107900 0x7f3508074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350fd15640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f350810edd0 con 0x7f3508101900 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350fd15640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f350810ec50 con 0x7f3508107900 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350dab1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 0x7f35080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350dab1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 0x7f35080744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40044/0 (socket says 172.21.15.7:40044) 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350dab1640 1 -- 172.21.15.7:0/705682748 learned_addr learned my addr 172.21.15.7:0/705682748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.817+0000 7f350d2b0640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3508107900 0x7f3508074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f350dab1640 1 -- 172.21.15.7:0/705682748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3508107900 msgr2=0x7f3508074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f350dab1640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3508107900 0x7f3508074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f350dab1640 1 -- 172.21.15.7:0/705682748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3508074f70 con 0x7f3508101900 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f350d2b0640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3508107900 0x7f3508074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f350dab1640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 0x7f35080744f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f350400c8f0 tx=0x7f350400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f34feffd640 1 -- 172.21.15.7:0/705682748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f35040079b0 con 0x7f3508101900 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3508075260 con 0x7f3508101900 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f34feffd640 1 -- 172.21.15.7:0/705682748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f35040054e0 con 0x7f3508101900 2024-09-21T05:06:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f34feffd640 1 -- 172.21.15.7:0/705682748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3504004dd0 con 0x7f3508101900 2024-09-21T05:06:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.818+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081a3e80 con 0x7f3508101900 2024-09-21T05:06:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.819+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3508103040 con 0x7f3508101900 2024-09-21T05:06:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.820+0000 7f34feffd640 1 -- 172.21.15.7:0/705682748 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3504005020 con 0x7f3508101900 2024-09-21T05:06:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.821+0000 7f34feffd640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f407aed0 0x7f34f407d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.821+0000 7f350d2b0640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f407aed0 0x7f34f407d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.821+0000 7f34feffd640 1 -- 172.21.15.7:0/705682748 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f350409e0c0 con 0x7f3508101900 2024-09-21T05:06:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.821+0000 7f350d2b0640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f407aed0 0x7f34f407d390 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f34f8006010 tx=0x7f34f80095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:07.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:07.825+0000 7f34feffd640 1 -- 172.21.15.7:0/705682748 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3504014030 con 0x7f3508101900 2024-09-21T05:06:08.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.087+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f3508109e70 con 0x7f3508101900 2024-09-21T05:06:08.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.088+0000 7f34feffd640 1 -- 172.21.15.7:0/705682748 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f3504007de0 con 0x7f3508101900 2024-09-21T05:06:08.091 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:06:08.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:06:08.091 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:06:08.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:06:08.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:06:08.091 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:06:08.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:06:08.092 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:06:08.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f407aed0 msgr2=0x7f34f407d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f407aed0 0x7f34f407d390 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f34f8006010 tx=0x7f34f80095f0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 msgr2=0x7f35080744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 0x7f35080744f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f350400c8f0 tx=0x7f350400cdb0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 shutdown_connections 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f407aed0 0x7f34f407d390 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3508107900 0x7f3508074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 --2- 172.21.15.7:0/705682748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3508101900 0x7f35080744f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 >> 172.21.15.7:0/705682748 conn(0x7f35080fd600 msgr2=0x7f35080fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 shutdown_connections 2024-09-21T05:06:08.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.090+0000 7f350fd15640 1 -- 172.21.15.7:0/705682748 wait complete. 2024-09-21T05:06:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 -- 172.21.15.7:0/1553230086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 msgr2=0x7f4108102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 --2- 172.21.15.7:0/1553230086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 0x7f4108102f30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009c20 tx=0x7f40f802aa00 comp rx=0 tx=0).stop 2024-09-21T05:06:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 -- 172.21.15.7:0/1553230086 shutdown_connections 2024-09-21T05:06:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 --2- 172.21.15.7:0/1553230086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4108103470 0x7f410810b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 --2- 172.21.15.7:0/1553230086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 0x7f4108102f30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 -- 172.21.15.7:0/1553230086 >> 172.21.15.7:0/1553230086 conn(0x7f41080fc550 msgr2=0x7f41080fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 -- 172.21.15.7:0/1553230086 shutdown_connections 2024-09-21T05:06:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 -- 172.21.15.7:0/1553230086 wait complete. 2024-09-21T05:06:08.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 Processor -- start 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.175+0000 7f410d63a640 1 -- start start 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f410d63a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 0x7f410819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f410d63a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4108103470 0x7f410819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f410d63a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f410810edc0 con 0x7f4108102b50 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f410d63a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f410810ec40 con 0x7f4108103470 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f4106ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 0x7f410819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f4106ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 0x7f410819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40064/0 (socket says 172.21.15.7:40064) 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f4106ffd640 1 -- 172.21.15.7:0/2188277146 learned_addr learned my addr 172.21.15.7:0/2188277146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f41067fc640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4108103470 0x7f410819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f4106ffd640 1 -- 172.21.15.7:0/2188277146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4108103470 msgr2=0x7f410819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f4106ffd640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4108103470 0x7f410819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f4106ffd640 1 -- 172.21.15.7:0/2188277146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41081977b0 con 0x7f4108102b50 2024-09-21T05:06:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.176+0000 7f4106ffd640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 0x7f410819d630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f40f802af20 tx=0x7f40f802ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.177+0000 7f40effff640 1 -- 172.21.15.7:0/2188277146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f40f8047070 con 0x7f4108102b50 2024-09-21T05:06:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.177+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4108197a40 con 0x7f4108102b50 2024-09-21T05:06:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.177+0000 7f40effff640 1 -- 172.21.15.7:0/2188277146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f40f8003e30 con 0x7f4108102b50 2024-09-21T05:06:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.177+0000 7f40effff640 1 -- 172.21.15.7:0/2188277146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f40f8033050 con 0x7f4108102b50 2024-09-21T05:06:08.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.177+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4108197f70 con 0x7f4108102b50 2024-09-21T05:06:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.178+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410806a4e0 con 0x7f4108102b50 2024-09-21T05:06:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.178+0000 7f40effff640 1 -- 172.21.15.7:0/2188277146 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f40f804a070 con 0x7f4108102b50 2024-09-21T05:06:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.179+0000 7f40effff640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007af30 0x7f40f007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.179+0000 7f41067fc640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007af30 0x7f40f007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.179+0000 7f40effff640 1 -- 172.21.15.7:0/2188277146 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f40f803e030 con 0x7f4108102b50 2024-09-21T05:06:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.180+0000 7f41067fc640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007af30 0x7f40f007d3f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4108108920 tx=0x7f40fc00b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:08.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.184+0000 7f40effff640 1 -- 172.21.15.7:0/2188277146 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40f808a6d0 con 0x7f4108102b50 2024-09-21T05:06:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.410+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4108105fb0 con 0x7f4108102b50 2024-09-21T05:06:08.413 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:08 smithi007 ceph-mon[108238]: from='client.35516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:08.413 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:08 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/705682748' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.410+0000 7f40effff640 1 -- 172.21.15.7:0/2188277146 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f40f808f580 con 0x7f4108102b50 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:06:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:06:08.415 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:06:08.416 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:06:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:06:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:06:08.417 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:06:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:06:08.417 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:06:08.417 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:06:08.417 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:06:08.417 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:06:08.418 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:06:08.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007af30 msgr2=0x7f40f007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007af30 0x7f40f007d3f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4108108920 tx=0x7f40fc00b410 comp rx=0 tx=0).stop 2024-09-21T05:06:08.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 msgr2=0x7f410819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 0x7f410819d630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f40f802af20 tx=0x7f40f802ee50 comp rx=0 tx=0).stop 2024-09-21T05:06:08.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 shutdown_connections 2024-09-21T05:06:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f40f007af30 0x7f40f007d3f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4108103470 0x7f410819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 --2- 172.21.15.7:0/2188277146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4108102b50 0x7f410819d630 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 >> 172.21.15.7:0/2188277146 conn(0x7f41080fc550 msgr2=0x7f4108108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 shutdown_connections 2024-09-21T05:06:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.413+0000 7f410d63a640 1 -- 172.21.15.7:0/2188277146 wait complete. 2024-09-21T05:06:08.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:08 smithi053 ceph-mon[80176]: from='client.35516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:08.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:08 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/705682748' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:06:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.498+0000 7f45e3341640 1 -- 172.21.15.7:0/2198048824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc102ac0 msgr2=0x7f45dc102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.498+0000 7f45e3341640 1 --2- 172.21.15.7:0/2198048824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc102ac0 0x7f45dc102ea0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009880 tx=0x7f45d802a530 comp rx=0 tx=0).stop 2024-09-21T05:06:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.498+0000 7f45e3341640 1 -- 172.21.15.7:0/2198048824 shutdown_connections 2024-09-21T05:06:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.498+0000 7f45e3341640 1 --2- 172.21.15.7:0/2198048824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45dc1033e0 0x7f45dc10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.498+0000 7f45e3341640 1 --2- 172.21.15.7:0/2198048824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc102ac0 0x7f45dc102ea0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.498+0000 7f45e3341640 1 -- 172.21.15.7:0/2198048824 >> 172.21.15.7:0/2198048824 conn(0x7f45dc0fc520 msgr2=0x7f45dc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.498+0000 7f45e3341640 1 -- 172.21.15.7:0/2198048824 shutdown_connections 2024-09-21T05:06:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.498+0000 7f45e3341640 1 -- 172.21.15.7:0/2198048824 wait complete. 2024-09-21T05:06:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e3341640 1 Processor -- start 2024-09-21T05:06:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e3341640 1 -- start start 2024-09-21T05:06:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e3341640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45dc102ac0 0x7f45dc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e3341640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc1033e0 0x7f45dc19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e3341640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc10edf0 con 0x7f45dc1033e0 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e3341640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc10ec70 con 0x7f45dc102ac0 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e08dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc1033e0 0x7f45dc19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e08dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc1033e0 0x7f45dc19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40084/0 (socket says 172.21.15.7:40084) 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e08dc640 1 -- 172.21.15.7:0/1223414208 learned_addr learned my addr 172.21.15.7:0/1223414208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.499+0000 7f45e10dd640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45dc102ac0 0x7f45dc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45e08dc640 1 -- 172.21.15.7:0/1223414208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45dc102ac0 msgr2=0x7f45dc19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45e08dc640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45dc102ac0 0x7f45dc19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45e08dc640 1 -- 172.21.15.7:0/1223414208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc1977c0 con 0x7f45dc1033e0 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45e10dd640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45dc102ac0 0x7f45dc19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45e08dc640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc1033e0 0x7f45dc19db80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00d6d0 tx=0x7f45cc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45d27fc640 1 -- 172.21.15.7:0/1223414208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f45cc017070 con 0x7f45dc1033e0 2024-09-21T05:06:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45d27fc640 1 -- 172.21.15.7:0/1223414208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f45cc0054e0 con 0x7f45dc1033e0 2024-09-21T05:06:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc197ab0 con 0x7f45dc1033e0 2024-09-21T05:06:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45d27fc640 1 -- 172.21.15.7:0/1223414208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f45cc004ae0 con 0x7f45dc1033e0 2024-09-21T05:06:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.500+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc198040 con 0x7f45dc1033e0 2024-09-21T05:06:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.501+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc06aa90 con 0x7f45dc1033e0 2024-09-21T05:06:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.502+0000 7f45d27fc640 1 -- 172.21.15.7:0/1223414208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f45cc004d90 con 0x7f45dc1033e0 2024-09-21T05:06:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.502+0000 7f45d27fc640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f45c807af80 0x7f45c807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.502+0000 7f45d27fc640 1 -- 172.21.15.7:0/1223414208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f45cc09dc50 con 0x7f45dc1033e0 2024-09-21T05:06:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.502+0000 7f45e10dd640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f45c807af80 0x7f45c807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.503+0000 7f45e10dd640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f45c807af80 0x7f45c807d440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000c00 tx=0x7f45d8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.507+0000 7f45d27fc640 1 -- 172.21.15.7:0/1223414208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45cc0691d0 con 0x7f45dc1033e0 2024-09-21T05:06:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.714+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f45dc1982f0 con 0x7f45c807af80 2024-09-21T05:06:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.716+0000 7f45d27fc640 1 -- 172.21.15.7:0/1223414208 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f45dc1982f0 con 0x7f45c807af80 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:06:08.719 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:06:08.720 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:06:08.720 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f45c807af80 msgr2=0x7f45c807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f45c807af80 0x7f45c807d440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000c00 tx=0x7f45d8031400 comp rx=0 tx=0).stop 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc1033e0 msgr2=0x7f45dc19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc1033e0 0x7f45dc19db80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00d6d0 tx=0x7f45cc00db90 comp rx=0 tx=0).stop 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 shutdown_connections 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f45c807af80 0x7f45c807d440 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45dc1033e0 0x7f45dc19db80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 --2- 172.21.15.7:0/1223414208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45dc102ac0 0x7f45dc19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 >> 172.21.15.7:0/1223414208 conn(0x7f45dc0fc520 msgr2=0x7f45dc10af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:08.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 shutdown_connections 2024-09-21T05:06:08.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.718+0000 7f45e3341640 1 -- 172.21.15.7:0/1223414208 wait complete. 2024-09-21T05:06:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.802+0000 7f5b1da57640 1 -- 172.21.15.7:0/3955867593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100250 msgr2=0x7f5b18100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.802+0000 7f5b1da57640 1 --2- 172.21.15.7:0/3955867593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100250 0x7f5b18100630 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009880 tx=0x7f5b0802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.802+0000 7f5b1da57640 1 -- 172.21.15.7:0/3955867593 shutdown_connections 2024-09-21T05:06:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.802+0000 7f5b1da57640 1 --2- 172.21.15.7:0/3955867593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b18100c00 0x7f5b18104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.802+0000 7f5b1da57640 1 --2- 172.21.15.7:0/3955867593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100250 0x7f5b18100630 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.802+0000 7f5b1da57640 1 -- 172.21.15.7:0/3955867593 >> 172.21.15.7:0/3955867593 conn(0x7f5b180fbc90 msgr2=0x7f5b180fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.802+0000 7f5b1da57640 1 -- 172.21.15.7:0/3955867593 shutdown_connections 2024-09-21T05:06:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.802+0000 7f5b1da57640 1 -- 172.21.15.7:0/3955867593 wait complete. 2024-09-21T05:06:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b1da57640 1 Processor -- start 2024-09-21T05:06:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b1da57640 1 -- start start 2024-09-21T05:06:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b1da57640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b18100250 0x7f5b1819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b1da57640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100c00 0x7f5b1819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b1da57640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1810cc40 con 0x7f5b18100c00 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b1da57640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1810cac0 con 0x7f5b18100250 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b167fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100c00 0x7f5b1819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b167fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100c00 0x7f5b1819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40100/0 (socket says 172.21.15.7:40100) 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b167fc640 1 -- 172.21.15.7:0/1842534583 learned_addr learned my addr 172.21.15.7:0/1842534583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b16ffd640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b18100250 0x7f5b1819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b167fc640 1 -- 172.21.15.7:0/1842534583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b18100250 msgr2=0x7f5b1819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b167fc640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b18100250 0x7f5b1819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.803+0000 7f5b167fc640 1 -- 172.21.15.7:0/1842534583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b18197840 con 0x7f5b18100c00 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.804+0000 7f5b167fc640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100c00 0x7f5b1819dc00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00db40 tx=0x7f5b0c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.804+0000 7f5b1ca55640 1 -- 172.21.15.7:0/1842534583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b0c00ac90 con 0x7f5b18100c00 2024-09-21T05:06:08.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.804+0000 7f5b1ca55640 1 -- 172.21.15.7:0/1842534583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b0c00ae30 con 0x7f5b18100c00 2024-09-21T05:06:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.804+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b18197b30 con 0x7f5b18100c00 2024-09-21T05:06:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.804+0000 7f5b1ca55640 1 -- 172.21.15.7:0/1842534583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b0c00e680 con 0x7f5b18100c00 2024-09-21T05:06:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.804+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b181980c0 con 0x7f5b18100c00 2024-09-21T05:06:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.805+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b18101730 con 0x7f5b18100c00 2024-09-21T05:06:08.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.806+0000 7f5b1ca55640 1 -- 172.21.15.7:0/1842534583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5b0c00e930 con 0x7f5b18100c00 2024-09-21T05:06:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.807+0000 7f5b1ca55640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5b0007af80 0x7f5b0007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.807+0000 7f5b16ffd640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5b0007af80 0x7f5b0007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.807+0000 7f5b1ca55640 1 -- 172.21.15.7:0/1842534583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f5b0c09e010 con 0x7f5b18100c00 2024-09-21T05:06:08.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.807+0000 7f5b16ffd640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5b0007af80 0x7f5b0007d440 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5b080096c0 tx=0x7f5b08031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:08.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:08.811+0000 7f5b1ca55640 1 -- 172.21.15.7:0/1842534583 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b0c069630 con 0x7f5b18100c00 2024-09-21T05:06:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.078+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f5b1810a570 con 0x7f5b18100c00 2024-09-21T05:06:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.078+0000 7f5b1ca55640 1 -- 172.21.15.7:0/1842534583 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f5b0c06e4e0 con 0x7f5b18100c00 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:06:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:06:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:06:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5b0007af80 msgr2=0x7f5b0007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5b0007af80 0x7f5b0007d440 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5b080096c0 tx=0x7f5b08031400 comp rx=0 tx=0).stop 2024-09-21T05:06:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100c00 msgr2=0x7f5b1819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100c00 0x7f5b1819dc00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00db40 tx=0x7f5b0c00a250 comp rx=0 tx=0).stop 2024-09-21T05:06:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 shutdown_connections 2024-09-21T05:06:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f5b0007af80 0x7f5b0007d440 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b18100c00 0x7f5b1819dc00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 --2- 172.21.15.7:0/1842534583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b18100250 0x7f5b1819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 >> 172.21.15.7:0/1842534583 conn(0x7f5b180fbc90 msgr2=0x7f5b180fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 shutdown_connections 2024-09-21T05:06:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:09.081+0000 7f5b1da57640 1 -- 172.21.15.7:0/1842534583 wait complete. 2024-09-21T05:06:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:09 smithi053 ceph-mon[80176]: pgmap v942: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:09 smithi053 ceph-mon[80176]: from='client.35520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:09 smithi053 ceph-mon[80176]: from='client.35524 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:09 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2188277146' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:06:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:09 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1842534583' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:06:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:09 smithi007 ceph-mon[108238]: pgmap v942: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:09 smithi007 ceph-mon[108238]: from='client.35520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:09 smithi007 ceph-mon[108238]: from='client.35524 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:09 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2188277146' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:06:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:09 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1842534583' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:06:10.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:10 smithi053 ceph-mon[80176]: from='client.35536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:10 smithi007 ceph-mon[108238]: from='client.35536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:11 smithi007 ceph-mon[108238]: pgmap v943: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:11 smithi053 ceph-mon[80176]: pgmap v943: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:13.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:13 smithi053 ceph-mon[80176]: pgmap v944: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:13 smithi007 ceph-mon[108238]: pgmap v944: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:06:14.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:06:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:15 smithi053 ceph-mon[80176]: pgmap v945: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:15 smithi007 ceph-mon[108238]: pgmap v945: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:17 smithi007 ceph-mon[108238]: pgmap v946: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:17 smithi053 ceph-mon[80176]: pgmap v946: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:19 smithi053 ceph-mon[80176]: pgmap v947: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:19 smithi007 ceph-mon[108238]: pgmap v947: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:21 smithi053 ceph-mon[80176]: pgmap v948: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:21.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:21 smithi007 ceph-mon[108238]: pgmap v948: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:23.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:23 smithi007 ceph-mon[108238]: pgmap v949: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:23.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:23 smithi053 ceph-mon[80176]: pgmap v949: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:25 smithi053 ceph-mon[80176]: pgmap v950: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:25 smithi007 ceph-mon[108238]: pgmap v950: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:27 smithi053 ceph-mon[80176]: pgmap v951: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:27 smithi007 ceph-mon[108238]: pgmap v951: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:29 smithi007 ceph-mon[108238]: pgmap v952: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:06:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:29 smithi053 ceph-mon[80176]: pgmap v952: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:06:31.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:31 smithi053 ceph-mon[80176]: pgmap v953: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:31 smithi007 ceph-mon[108238]: pgmap v953: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:33 smithi053 ceph-mon[80176]: pgmap v954: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:33 smithi007 ceph-mon[108238]: pgmap v954: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:35 smithi007 ceph-mon[108238]: pgmap v955: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:35 smithi053 ceph-mon[80176]: pgmap v955: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:37 smithi053 ceph-mon[80176]: pgmap v956: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:37 smithi007 ceph-mon[108238]: pgmap v956: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.182+0000 7f84284be640 1 -- 172.21.15.7:0/388031229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 msgr2=0x7f8420106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.182+0000 7f84284be640 1 --2- 172.21.15.7:0/388031229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 0x7f8420106bd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f841c009880 tx=0x7f841c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.182+0000 7f84284be640 1 -- 172.21.15.7:0/388031229 shutdown_connections 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.182+0000 7f84284be640 1 --2- 172.21.15.7:0/388031229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201007f0 0x7f8420100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.182+0000 7f84284be640 1 --2- 172.21.15.7:0/388031229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 0x7f8420106bd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.182+0000 7f84284be640 1 -- 172.21.15.7:0/388031229 >> 172.21.15.7:0/388031229 conn(0x7f84200fc550 msgr2=0x7f84200fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.182+0000 7f84284be640 1 -- 172.21.15.7:0/388031229 shutdown_connections 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.182+0000 7f84284be640 1 -- 172.21.15.7:0/388031229 wait complete. 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f84284be640 1 Processor -- start 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f84284be640 1 -- start start 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f84284be640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201007f0 0x7f842019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f84284be640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 0x7f842019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f84284be640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f842010dca0 con 0x7f84201067f0 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f84284be640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f842010db20 con 0x7f84201007f0 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f8425a59640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 0x7f842019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f8425a59640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 0x7f842019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42118/0 (socket says 172.21.15.7:42118) 2024-09-21T05:06:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f8425a59640 1 -- 172.21.15.7:0/3266413825 learned_addr learned my addr 172.21.15.7:0/3266413825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.183+0000 7f842625a640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201007f0 0x7f842019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f8425a59640 1 -- 172.21.15.7:0/3266413825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201007f0 msgr2=0x7f842019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f8425a59640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201007f0 0x7f842019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f8425a59640 1 -- 172.21.15.7:0/3266413825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8420199970 con 0x7f84201067f0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f8425a59640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 0x7f842019fcf0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f841000d8d0 tx=0x7f841000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f84177fe640 1 -- 172.21.15.7:0/3266413825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8410017030 con 0x7f84201067f0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f84177fe640 1 -- 172.21.15.7:0/3266413825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8410004020 con 0x7f84201067f0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8420199c60 con 0x7f84201067f0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f84177fe640 1 -- 172.21.15.7:0/3266413825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f84100047a0 con 0x7f84201067f0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.184+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842019a1f0 con 0x7f84201067f0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.185+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8420101f30 con 0x7f84201067f0 2024-09-21T05:06:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.186+0000 7f84177fe640 1 -- 172.21.15.7:0/3266413825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f84100041d0 con 0x7f84201067f0 2024-09-21T05:06:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.187+0000 7f84177fe640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f840c07af90 0x7f840c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.187+0000 7f84177fe640 1 -- 172.21.15.7:0/3266413825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f8410018070 con 0x7f84201067f0 2024-09-21T05:06:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.187+0000 7f842625a640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f840c07af90 0x7f840c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.187+0000 7f842625a640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f840c07af90 0x7f840c07d450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.191+0000 7f84177fe640 1 -- 172.21.15.7:0/3266413825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8410069370 con 0x7f84201067f0 2024-09-21T05:06:39.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.397+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f842019a500 con 0x7f840c07af90 2024-09-21T05:06:39.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.398+0000 7f84177fe640 1 -- 172.21.15.7:0/3266413825 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f842019a500 con 0x7f840c07af90 2024-09-21T05:06:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f840c07af90 msgr2=0x7f840c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f840c07af90 0x7f840c07d450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c031400 comp rx=0 tx=0).stop 2024-09-21T05:06:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 msgr2=0x7f842019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 0x7f842019fcf0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f841000d8d0 tx=0x7f841000dd90 comp rx=0 tx=0).stop 2024-09-21T05:06:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 shutdown_connections 2024-09-21T05:06:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f840c07af90 0x7f840c07d450 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201067f0 0x7f842019fcf0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 --2- 172.21.15.7:0/3266413825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201007f0 0x7f842019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 >> 172.21.15.7:0/3266413825 conn(0x7f84200fc550 msgr2=0x7f8420069dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:39.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 shutdown_connections 2024-09-21T05:06:39.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.400+0000 7f84284be640 1 -- 172.21.15.7:0/3266413825 wait complete. 2024-09-21T05:06:39.413 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:06:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:39 smithi053 ceph-mon[80176]: pgmap v957: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:39 smithi007 ceph-mon[108238]: pgmap v957: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:39.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.486+0000 7f342655d640 1 -- 172.21.15.7:0/2292302875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 msgr2=0x7f3420107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.486+0000 7f342655d640 1 --2- 172.21.15.7:0/2292302875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 0x7f3420107ce0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3410009880 tx=0x7f341002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.486+0000 7f342655d640 1 -- 172.21.15.7:0/2292302875 shutdown_connections 2024-09-21T05:06:39.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.486+0000 7f342655d640 1 --2- 172.21.15.7:0/2292302875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3420101900 0x7f3420101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.486+0000 7f342655d640 1 --2- 172.21.15.7:0/2292302875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 0x7f3420107ce0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.486+0000 7f342655d640 1 -- 172.21.15.7:0/2292302875 >> 172.21.15.7:0/2292302875 conn(0x7f34200fd600 msgr2=0x7f34200ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:39.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.486+0000 7f342655d640 1 -- 172.21.15.7:0/2292302875 shutdown_connections 2024-09-21T05:06:39.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.486+0000 7f342655d640 1 -- 172.21.15.7:0/2292302875 wait complete. 2024-09-21T05:06:39.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f342655d640 1 Processor -- start 2024-09-21T05:06:39.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f342655d640 1 -- start start 2024-09-21T05:06:39.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f342655d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3420101900 0x7f34200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f342655d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 0x7f3420074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f342655d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f342010edb0 con 0x7f3420107900 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f342655d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f342010ec30 con 0x7f3420101900 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f341f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 0x7f3420074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f341f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 0x7f3420074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42128/0 (socket says 172.21.15.7:42128) 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f341f7fe640 1 -- 172.21.15.7:0/4126959630 learned_addr learned my addr 172.21.15.7:0/4126959630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f341ffff640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3420101900 0x7f34200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f341f7fe640 1 -- 172.21.15.7:0/4126959630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3420101900 msgr2=0x7f34200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.487+0000 7f341f7fe640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3420101900 0x7f34200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.488+0000 7f341f7fe640 1 -- 172.21.15.7:0/4126959630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3420074f70 con 0x7f3420107900 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.488+0000 7f341ffff640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3420101900 0x7f34200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.488+0000 7f341f7fe640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 0x7f3420074a30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f341400c8a0 tx=0x7f341400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.488+0000 7f341d7fa640 1 -- 172.21.15.7:0/4126959630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f34140079b0 con 0x7f3420107900 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.488+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3420075260 con 0x7f3420107900 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.488+0000 7f341d7fa640 1 -- 172.21.15.7:0/4126959630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3414007b50 con 0x7f3420107900 2024-09-21T05:06:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.488+0000 7f341d7fa640 1 -- 172.21.15.7:0/4126959630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3414018700 con 0x7f3420107900 2024-09-21T05:06:39.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.488+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201a3e00 con 0x7f3420107900 2024-09-21T05:06:39.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.489+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3420103040 con 0x7f3420107900 2024-09-21T05:06:39.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.490+0000 7f341d7fa640 1 -- 172.21.15.7:0/4126959630 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3414007d00 con 0x7f3420107900 2024-09-21T05:06:39.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.490+0000 7f341d7fa640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f340807af80 0x7f340807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.491+0000 7f341d7fa640 1 -- 172.21.15.7:0/4126959630 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f341409e320 con 0x7f3420107900 2024-09-21T05:06:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.491+0000 7f341ffff640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f340807af80 0x7f340807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.491+0000 7f341ffff640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f340807af80 0x7f340807d440 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3410006010 tx=0x7f34100095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:39.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.495+0000 7f341d7fa640 1 -- 172.21.15.7:0/4126959630 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3414014030 con 0x7f3420107900 2024-09-21T05:06:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.703+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3420062630 con 0x7f340807af80 2024-09-21T05:06:39.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.705+0000 7f341d7fa640 1 -- 172.21.15.7:0/4126959630 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3420062630 con 0x7f340807af80 2024-09-21T05:06:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f340807af80 msgr2=0x7f340807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f340807af80 0x7f340807d440 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3410006010 tx=0x7f34100095f0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 msgr2=0x7f3420074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 0x7f3420074a30 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f341400c8a0 tx=0x7f341400cd60 comp rx=0 tx=0).stop 2024-09-21T05:06:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 shutdown_connections 2024-09-21T05:06:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f340807af80 0x7f340807d440 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3420107900 0x7f3420074a30 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 --2- 172.21.15.7:0/4126959630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3420101900 0x7f34200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 >> 172.21.15.7:0/4126959630 conn(0x7f34200fd600 msgr2=0x7f34200fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 shutdown_connections 2024-09-21T05:06:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.707+0000 7f342655d640 1 -- 172.21.15.7:0/4126959630 wait complete. 2024-09-21T05:06:39.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.791+0000 7fbd8111f640 1 -- 172.21.15.7:0/1003300737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c101900 msgr2=0x7fbd7c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.791+0000 7fbd8111f640 1 --2- 172.21.15.7:0/1003300737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c101900 0x7fbd7c101d60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009880 tx=0x7fbd6402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.791+0000 7fbd8111f640 1 -- 172.21.15.7:0/1003300737 shutdown_connections 2024-09-21T05:06:39.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.791+0000 7fbd8111f640 1 --2- 172.21.15.7:0/1003300737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c101900 0x7fbd7c101d60 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.791+0000 7fbd8111f640 1 --2- 172.21.15.7:0/1003300737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd7c107900 0x7fbd7c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.791+0000 7fbd8111f640 1 -- 172.21.15.7:0/1003300737 >> 172.21.15.7:0/1003300737 conn(0x7fbd7c0fd600 msgr2=0x7fbd7c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:39.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.791+0000 7fbd8111f640 1 -- 172.21.15.7:0/1003300737 shutdown_connections 2024-09-21T05:06:39.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.791+0000 7fbd8111f640 1 -- 172.21.15.7:0/1003300737 wait complete. 2024-09-21T05:06:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd8111f640 1 Processor -- start 2024-09-21T05:06:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd8111f640 1 -- start start 2024-09-21T05:06:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd8111f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd7c101900 0x7fbd7c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd8111f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c107900 0x7fbd7c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd8111f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c10edb0 con 0x7fbd7c107900 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd8111f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c10ec30 con 0x7fbd7c101900 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c107900 0x7fbd7c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c107900 0x7fbd7c19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42150/0 (socket says 172.21.15.7:42150) 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.792+0000 7fbd73fff640 1 -- 172.21.15.7:0/275283035 learned_addr learned my addr 172.21.15.7:0/275283035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd7ad9d640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd7c101900 0x7fbd7c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd73fff640 1 -- 172.21.15.7:0/275283035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd7c101900 msgr2=0x7fbd7c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd73fff640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd7c101900 0x7fbd7c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd73fff640 1 -- 172.21.15.7:0/275283035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c1a0440 con 0x7fbd7c107900 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd73fff640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c107900 0x7fbd7c19fd90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbd64031460 tx=0x7fbd640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd78d99640 1 -- 172.21.15.7:0/275283035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbd6403f070 con 0x7fbd7c107900 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7c199ad0 con 0x7fbd7c107900 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd78d99640 1 -- 172.21.15.7:0/275283035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd64004cf0 con 0x7fbd7c107900 2024-09-21T05:06:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd78d99640 1 -- 172.21.15.7:0/275283035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbd64035030 con 0x7fbd7c107900 2024-09-21T05:06:39.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.793+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7c199fd0 con 0x7fbd7c107900 2024-09-21T05:06:39.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.794+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd7c103040 con 0x7fbd7c107900 2024-09-21T05:06:39.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.795+0000 7fbd78d99640 1 -- 172.21.15.7:0/275283035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbd64042070 con 0x7fbd7c107900 2024-09-21T05:06:39.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.796+0000 7fbd78d99640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd6807af30 0x7fbd6807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:39.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.796+0000 7fbd78d99640 1 -- 172.21.15.7:0/275283035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbd640be6d0 con 0x7fbd7c107900 2024-09-21T05:06:39.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.796+0000 7fbd7ad9d640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd6807af30 0x7fbd6807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:39.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.796+0000 7fbd7ad9d640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd6807af30 0x7fbd6807d3f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c19ab50 tx=0x7fbd6c00a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:39.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.800+0000 7fbd78d99640 1 -- 172.21.15.7:0/275283035 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd64089c70 con 0x7fbd7c107900 2024-09-21T05:06:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:39.997+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbd7c19a340 con 0x7fbd6807af30 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.007+0000 7fbd78d99640 1 -- 172.21.15.7:0/275283035 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fbd7c19a340 con 0x7fbd6807af30 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (28m) 7m ago 39m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (31m) 7m ago 39m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (31m) 7m ago 38m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (26m) 7m ago 39m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (26m) 7m ago 38m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (28m) 7m ago 39m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (22m) 7m ago 34m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:06:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (23m) 7m ago 34m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (7m) 7m ago 34m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (22m) 7m ago 34m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (30m) 7m ago 41m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (29m) 7m ago 38m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (27m) 7m ago 41m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (27m) 7m ago 38m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (29m) 7m ago 39m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (29m) 7m ago 38m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (26m) 7m ago 37m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (25m) 7m ago 37m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (25m) 7m ago 36m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (24m) 7m ago 36m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (23m) 7m ago 35m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:06:40.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (23m) 7m ago 35m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:06:40.013 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (29m) 7m ago 39m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd6807af30 msgr2=0x7fbd6807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd6807af30 0x7fbd6807d3f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c19ab50 tx=0x7fbd6c00a2d0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c107900 msgr2=0x7fbd7c19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c107900 0x7fbd7c19fd90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbd64031460 tx=0x7fbd640048f0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 shutdown_connections 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd6807af30 0x7fbd6807d3f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd7c107900 0x7fbd7c19fd90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 --2- 172.21.15.7:0/275283035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd7c101900 0x7fbd7c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.009+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 >> 172.21.15.7:0/275283035 conn(0x7fbd7c0fd600 msgr2=0x7fbd7c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.010+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 shutdown_connections 2024-09-21T05:06:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.010+0000 7fbd8111f640 1 -- 172.21.15.7:0/275283035 wait complete. 2024-09-21T05:06:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.092+0000 7f4213f96640 1 -- 172.21.15.7:0/3548755813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c101900 msgr2=0x7f420c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.092+0000 7f4213f96640 1 --2- 172.21.15.7:0/3548755813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c101900 0x7f420c101d60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009880 tx=0x7f41fc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.092+0000 7f4213f96640 1 -- 172.21.15.7:0/3548755813 shutdown_connections 2024-09-21T05:06:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.092+0000 7f4213f96640 1 --2- 172.21.15.7:0/3548755813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c101900 0x7f420c101d60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.092+0000 7f4213f96640 1 --2- 172.21.15.7:0/3548755813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f420c107900 0x7f420c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.092+0000 7f4213f96640 1 -- 172.21.15.7:0/3548755813 >> 172.21.15.7:0/3548755813 conn(0x7f420c0fd600 msgr2=0x7f420c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:40.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.093+0000 7f4213f96640 1 -- 172.21.15.7:0/3548755813 shutdown_connections 2024-09-21T05:06:40.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.093+0000 7f4213f96640 1 -- 172.21.15.7:0/3548755813 wait complete. 2024-09-21T05:06:40.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.093+0000 7f4213f96640 1 Processor -- start 2024-09-21T05:06:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.093+0000 7f4213f96640 1 -- start start 2024-09-21T05:06:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4213f96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f420c101900 0x7f420c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4213f96640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c107900 0x7f420c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4213f96640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f420c10edb0 con 0x7f420c107900 2024-09-21T05:06:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4213f96640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f420c10ec30 con 0x7f420c101900 2024-09-21T05:06:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4211531640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c107900 0x7f420c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4211531640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c107900 0x7f420c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42166/0 (socket says 172.21.15.7:42166) 2024-09-21T05:06:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4211531640 1 -- 172.21.15.7:0/468389385 learned_addr learned my addr 172.21.15.7:0/468389385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4211d32640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f420c101900 0x7f420c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4211531640 1 -- 172.21.15.7:0/468389385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f420c101900 msgr2=0x7f420c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4211531640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f420c101900 0x7f420c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4211531640 1 -- 172.21.15.7:0/468389385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c197800 con 0x7f420c107900 2024-09-21T05:06:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4211531640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c107900 0x7f420c19dbc0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f41fc031460 tx=0x7f41fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.094+0000 7f4202ffd640 1 -- 172.21.15.7:0/468389385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f41fc03f070 con 0x7f420c107900 2024-09-21T05:06:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.095+0000 7f4202ffd640 1 -- 172.21.15.7:0/468389385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f41fc0035f0 con 0x7f420c107900 2024-09-21T05:06:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.095+0000 7f4202ffd640 1 -- 172.21.15.7:0/468389385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f41fc035030 con 0x7f420c107900 2024-09-21T05:06:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.095+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c197a90 con 0x7f420c107900 2024-09-21T05:06:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.095+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c197fc0 con 0x7f420c107900 2024-09-21T05:06:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.096+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420c103040 con 0x7f420c107900 2024-09-21T05:06:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.096+0000 7f4202ffd640 1 -- 172.21.15.7:0/468389385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f41fc042070 con 0x7f420c107900 2024-09-21T05:06:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.097+0000 7f4202ffd640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f807af30 0x7f41f807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.097+0000 7f4202ffd640 1 -- 172.21.15.7:0/468389385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f41fc0bea80 con 0x7f420c107900 2024-09-21T05:06:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.097+0000 7f4211d32640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f807af30 0x7f41f807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.098+0000 7f4211d32640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f807af30 0x7f41f807d3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4208005fd0 tx=0x7f4208005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.102+0000 7f4202ffd640 1 -- 172.21.15.7:0/468389385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41fc08a020 con 0x7f420c107900 2024-09-21T05:06:40.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.367+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f420c198d30 con 0x7f420c107900 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.368+0000 7f4202ffd640 1 -- 172.21.15.7:0/468389385 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f41fc08c070 con 0x7f420c107900 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:06:40.372 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:06:40.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:06:40.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:06:40.373 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:06:40.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:06:40.373 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:06:40.373 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:06:40.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:06:40.373 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f807af30 msgr2=0x7f41f807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f807af30 0x7f41f807d3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4208005fd0 tx=0x7f4208005850 comp rx=0 tx=0).stop 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c107900 msgr2=0x7f420c19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c107900 0x7f420c19dbc0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f41fc031460 tx=0x7f41fc0045e0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 shutdown_connections 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f41f807af30 0x7f41f807d3f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420c107900 0x7f420c19dbc0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 --2- 172.21.15.7:0/468389385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f420c101900 0x7f420c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 >> 172.21.15.7:0/468389385 conn(0x7f420c0fd600 msgr2=0x7f420c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 shutdown_connections 2024-09-21T05:06:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.370+0000 7f4213f96640 1 -- 172.21.15.7:0/468389385 wait complete. 2024-09-21T05:06:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.454+0000 7f1da50e4640 1 -- 172.21.15.7:0/186699268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0101900 msgr2=0x7f1da0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.454+0000 7f1da50e4640 1 --2- 172.21.15.7:0/186699268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0101900 0x7f1da0101d60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009880 tx=0x7f1d8802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.454+0000 7f1da50e4640 1 -- 172.21.15.7:0/186699268 shutdown_connections 2024-09-21T05:06:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.454+0000 7f1da50e4640 1 --2- 172.21.15.7:0/186699268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0101900 0x7f1da0101d60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.454+0000 7f1da50e4640 1 --2- 172.21.15.7:0/186699268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0107900 0x7f1da0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.454+0000 7f1da50e4640 1 -- 172.21.15.7:0/186699268 >> 172.21.15.7:0/186699268 conn(0x7f1da00fd600 msgr2=0x7f1da00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.454+0000 7f1da50e4640 1 -- 172.21.15.7:0/186699268 shutdown_connections 2024-09-21T05:06:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.454+0000 7f1da50e4640 1 -- 172.21.15.7:0/186699268 wait complete. 2024-09-21T05:06:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.455+0000 7f1da50e4640 1 Processor -- start 2024-09-21T05:06:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.455+0000 7f1da50e4640 1 -- start start 2024-09-21T05:06:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.455+0000 7f1da50e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0101900 0x7f1da019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.455+0000 7f1da50e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0107900 0x7f1da019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.455+0000 7f1da50e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010edd0 con 0x7f1da0107900 2024-09-21T05:06:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.455+0000 7f1da50e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010ec50 con 0x7f1da0101900 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0107900 0x7f1da019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d9e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0107900 0x7f1da019dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42186/0 (socket says 172.21.15.7:42186) 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d9e59c640 1 -- 172.21.15.7:0/36689607 learned_addr learned my addr 172.21.15.7:0/36689607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d9ed9d640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0101900 0x7f1da019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d9e59c640 1 -- 172.21.15.7:0/36689607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0101900 msgr2=0x7f1da019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d9e59c640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0101900 0x7f1da019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d9e59c640 1 -- 172.21.15.7:0/36689607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da0197800 con 0x7f1da0107900 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d9e59c640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0107900 0x7f1da019dbc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1d88031460 tx=0x7f1d880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d97fff640 1 -- 172.21.15.7:0/36689607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1d8803f070 con 0x7f1da0107900 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da0197a90 con 0x7f1da0107900 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d97fff640 1 -- 172.21.15.7:0/36689607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d88004cf0 con 0x7f1da0107900 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1d97fff640 1 -- 172.21.15.7:0/36689607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1d88035030 con 0x7f1da0107900 2024-09-21T05:06:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.456+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da0197fc0 con 0x7f1da0107900 2024-09-21T05:06:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.457+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da0103040 con 0x7f1da0107900 2024-09-21T05:06:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.458+0000 7f1d97fff640 1 -- 172.21.15.7:0/36689607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1d88042070 con 0x7f1da0107900 2024-09-21T05:06:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.459+0000 7f1d97fff640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d8c07af30 0x7f1d8c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.459+0000 7f1d9ed9d640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d8c07af30 0x7f1d8c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.459+0000 7f1d97fff640 1 -- 172.21.15.7:0/36689607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f1d880bea70 con 0x7f1da0107900 2024-09-21T05:06:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.459+0000 7f1d9ed9d640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d8c07af30 0x7f1d8c07d3f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1d900058c0 tx=0x7f1d9000a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.463+0000 7f1d97fff640 1 -- 172.21.15.7:0/36689607 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d8808a010 con 0x7f1da0107900 2024-09-21T05:06:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.687+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1da0109e70 con 0x7f1da0107900 2024-09-21T05:06:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.688+0000 7f1d97fff640 1 -- 172.21.15.7:0/36689607 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f1d8808eec0 con 0x7f1da0107900 2024-09-21T05:06:40.692 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:06:40.692 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:06:40.692 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:06:40.692 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:06:40.692 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:06:40.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:06:40.692 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:06:40.693 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:06:40.694 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:06:40.695 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d8c07af30 msgr2=0x7f1d8c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d8c07af30 0x7f1d8c07d3f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1d900058c0 tx=0x7f1d9000a250 comp rx=0 tx=0).stop 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0107900 msgr2=0x7f1da019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0107900 0x7f1da019dbc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1d88031460 tx=0x7f1d880048f0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 shutdown_connections 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1d8c07af30 0x7f1d8c07d3f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da0107900 0x7f1da019dbc0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 --2- 172.21.15.7:0/36689607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1da0101900 0x7f1da019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 >> 172.21.15.7:0/36689607 conn(0x7f1da00fd600 msgr2=0x7f1da00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 shutdown_connections 2024-09-21T05:06:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.690+0000 7f1da50e4640 1 -- 172.21.15.7:0/36689607 wait complete. 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.775+0000 7f317ee0e640 1 -- 172.21.15.7:0/2563444329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178100800 msgr2=0x7f3178100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.775+0000 7f317ee0e640 1 --2- 172.21.15.7:0/2563444329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178100800 0x7f3178100c60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3164009880 tx=0x7f316402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.775+0000 7f317ee0e640 1 -- 172.21.15.7:0/2563444329 shutdown_connections 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.775+0000 7f317ee0e640 1 --2- 172.21.15.7:0/2563444329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178100800 0x7f3178100c60 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.775+0000 7f317ee0e640 1 --2- 172.21.15.7:0/2563444329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3178106800 0x7f3178106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.775+0000 7f317ee0e640 1 -- 172.21.15.7:0/2563444329 >> 172.21.15.7:0/2563444329 conn(0x7f31780fc520 msgr2=0x7f31780fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.775+0000 7f317ee0e640 1 -- 172.21.15.7:0/2563444329 shutdown_connections 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.775+0000 7f317ee0e640 1 -- 172.21.15.7:0/2563444329 wait complete. 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f317ee0e640 1 Processor -- start 2024-09-21T05:06:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f317ee0e640 1 -- start start 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f317ee0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3178100800 0x7f31780fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f317ee0e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178106800 0x7f31780fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f317ee0e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f317810dcd0 con 0x7f3178106800 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f317ee0e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f317810db50 con 0x7f3178100800 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f3177fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178106800 0x7f31780fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f3177fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178106800 0x7f31780fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42208/0 (socket says 172.21.15.7:42208) 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f3177fff640 1 -- 172.21.15.7:0/3981087877 learned_addr learned my addr 172.21.15.7:0/3981087877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.776+0000 7f317cbaa640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3178100800 0x7f31780fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f3177fff640 1 -- 172.21.15.7:0/3981087877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3178100800 msgr2=0x7f31780fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f3177fff640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3178100800 0x7f31780fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f3177fff640 1 -- 172.21.15.7:0/3981087877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31780fd670 con 0x7f3178106800 2024-09-21T05:06:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f3177fff640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178106800 0x7f31780fd130 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3164031460 tx=0x7f3164004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f3175ffb640 1 -- 172.21.15.7:0/3981087877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f316403f070 con 0x7f3178106800 2024-09-21T05:06:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f3175ffb640 1 -- 172.21.15.7:0/3981087877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3164004a90 con 0x7f3178106800 2024-09-21T05:06:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f3175ffb640 1 -- 172.21.15.7:0/3981087877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3164035030 con 0x7f3178106800 2024-09-21T05:06:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31780fd900 con 0x7f3178106800 2024-09-21T05:06:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.777+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31781a3db0 con 0x7f3178106800 2024-09-21T05:06:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.778+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3178101f40 con 0x7f3178106800 2024-09-21T05:06:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.779+0000 7f3175ffb640 1 -- 172.21.15.7:0/3981087877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3164042070 con 0x7f3178106800 2024-09-21T05:06:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.780+0000 7f3175ffb640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f316807af40 0x7f316807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.780+0000 7f3175ffb640 1 -- 172.21.15.7:0/3981087877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f31640beab0 con 0x7f3178106800 2024-09-21T05:06:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.780+0000 7f317cbaa640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f316807af40 0x7f316807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.780+0000 7f317cbaa640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f316807af40 0x7f316807d400 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f316c009bd0 tx=0x7f316c005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:40.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.784+0000 7f3175ffb640 1 -- 172.21.15.7:0/3981087877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f316408a050 con 0x7f3178106800 2024-09-21T05:06:40.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.993+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3178062630 con 0x7f316807af40 2024-09-21T05:06:40.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.994+0000 7f3175ffb640 1 -- 172.21.15.7:0/3981087877 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3178062630 con 0x7f316807af40 2024-09-21T05:06:40.998 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:06:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:06:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:06:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:06:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:06:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f316807af40 msgr2=0x7f316807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f316807af40 0x7f316807d400 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f316c009bd0 tx=0x7f316c005f90 comp rx=0 tx=0).stop 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178106800 msgr2=0x7f31780fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178106800 0x7f31780fd130 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3164031460 tx=0x7f3164004530 comp rx=0 tx=0).stop 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 shutdown_connections 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f316807af40 0x7f316807d400 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3178106800 0x7f31780fd130 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 --2- 172.21.15.7:0/3981087877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3178100800 0x7f31780fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 >> 172.21.15.7:0/3981087877 conn(0x7f31780fc520 msgr2=0x7f317806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 shutdown_connections 2024-09-21T05:06:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:40.997+0000 7f317ee0e640 1 -- 172.21.15.7:0/3981087877 wait complete. 2024-09-21T05:06:41.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.080+0000 7f9fe7b03640 1 -- 172.21.15.7:0/3109904973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 msgr2=0x7f9fe0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:41.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.080+0000 7f9fe7b03640 1 --2- 172.21.15.7:0/3109904973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 0x7f9fe0107ce0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009880 tx=0x7f9fdc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.080+0000 7f9fe7b03640 1 -- 172.21.15.7:0/3109904973 shutdown_connections 2024-09-21T05:06:41.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.080+0000 7f9fe7b03640 1 --2- 172.21.15.7:0/3109904973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe0101900 0x7f9fe0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.080+0000 7f9fe7b03640 1 --2- 172.21.15.7:0/3109904973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 0x7f9fe0107ce0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.080+0000 7f9fe7b03640 1 -- 172.21.15.7:0/3109904973 >> 172.21.15.7:0/3109904973 conn(0x7f9fe00fd600 msgr2=0x7f9fe00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:41.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.081+0000 7f9fe7b03640 1 -- 172.21.15.7:0/3109904973 shutdown_connections 2024-09-21T05:06:41.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.081+0000 7f9fe7b03640 1 -- 172.21.15.7:0/3109904973 wait complete. 2024-09-21T05:06:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.081+0000 7f9fe7b03640 1 Processor -- start 2024-09-21T05:06:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.081+0000 7f9fe7b03640 1 -- start start 2024-09-21T05:06:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.081+0000 7f9fe7b03640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe0101900 0x7f9fe019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.081+0000 7f9fe7b03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 0x7f9fe019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.081+0000 7f9fe7b03640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe010edb0 con 0x7f9fe0107900 2024-09-21T05:06:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.081+0000 7f9fe7b03640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe010ec30 con 0x7f9fe0101900 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe509e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 0x7f9fe019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe509e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 0x7f9fe019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42218/0 (socket says 172.21.15.7:42218) 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe509e640 1 -- 172.21.15.7:0/1365883660 learned_addr learned my addr 172.21.15.7:0/1365883660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe589f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe0101900 0x7f9fe019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe509e640 1 -- 172.21.15.7:0/1365883660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe0101900 msgr2=0x7f9fe019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe509e640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe0101900 0x7f9fe019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe509e640 1 -- 172.21.15.7:0/1365883660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe01977c0 con 0x7f9fe0107900 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe509e640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 0x7f9fe019db80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000d8d0 tx=0x7f9fd000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fd6ffd640 1 -- 172.21.15.7:0/1365883660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9fd0017030 con 0x7f9fe0107900 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fd6ffd640 1 -- 172.21.15.7:0/1365883660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fd0004020 con 0x7f9fe0107900 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe0197ab0 con 0x7f9fe0107900 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fd6ffd640 1 -- 172.21.15.7:0/1365883660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9fd00047a0 con 0x7f9fe0107900 2024-09-21T05:06:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.082+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe0198040 con 0x7f9fe0107900 2024-09-21T05:06:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.083+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe0103040 con 0x7f9fe0107900 2024-09-21T05:06:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.084+0000 7f9fd6ffd640 1 -- 172.21.15.7:0/1365883660 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9fd00041d0 con 0x7f9fe0107900 2024-09-21T05:06:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.084+0000 7f9fd6ffd640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9fcc07af80 0x7f9fcc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:06:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.085+0000 7f9fd6ffd640 1 -- 172.21.15.7:0/1365883660 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f9fd0018070 con 0x7f9fe0107900 2024-09-21T05:06:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.085+0000 7f9fe589f640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9fcc07af80 0x7f9fcc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:06:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.085+0000 7f9fe589f640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9fcc07af80 0x7f9fcc07d440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0096c0 tx=0x7f9fdc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:06:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.089+0000 7f9fd6ffd640 1 -- 172.21.15.7:0/1365883660 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9fd00692f0 con 0x7f9fe0107900 2024-09-21T05:06:41.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.350+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9fe0104b10 con 0x7f9fe0107900 2024-09-21T05:06:41.354 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:41 smithi007 ceph-mon[108238]: from='client.35544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:41.354 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:41 smithi007 ceph-mon[108238]: pgmap v958: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:41.354 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:41 smithi007 ceph-mon[108238]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:41.354 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:41 smithi007 ceph-mon[108238]: from='client.35552 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:41.354 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:41 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/468389385' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:06:41.354 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:41 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/36689607' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:06:41.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.350+0000 7f9fd6ffd640 1 -- 172.21.15.7:0/1365883660 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f9fd006e1a0 con 0x7f9fe0107900 2024-09-21T05:06:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:06:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:06:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:06:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:06:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:06:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:06:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:06:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:06:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:06:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:06:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:06:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9fcc07af80 msgr2=0x7f9fcc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9fcc07af80 0x7f9fcc07d440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0096c0 tx=0x7f9fdc031400 comp rx=0 tx=0).stop 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 msgr2=0x7f9fe019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 0x7f9fe019db80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000d8d0 tx=0x7f9fd000dd90 comp rx=0 tx=0).stop 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 shutdown_connections 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9fcc07af80 0x7f9fcc07d440 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fe0107900 0x7f9fe019db80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 --2- 172.21.15.7:0/1365883660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe0101900 0x7f9fe019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 >> 172.21.15.7:0/1365883660 conn(0x7f9fe00fd600 msgr2=0x7f9fe00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 shutdown_connections 2024-09-21T05:06:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:06:41.352+0000 7f9fe7b03640 1 -- 172.21.15.7:0/1365883660 wait complete. 2024-09-21T05:06:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:41 smithi053 ceph-mon[80176]: from='client.35544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:41 smithi053 ceph-mon[80176]: pgmap v958: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:41 smithi053 ceph-mon[80176]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:41 smithi053 ceph-mon[80176]: from='client.35552 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:41 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/468389385' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:06:41.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:41 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/36689607' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:06:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:42 smithi053 ceph-mon[80176]: from='client.35564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:42.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:42 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1365883660' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:06:42.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:42 smithi007 ceph-mon[108238]: from='client.35564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:06:42.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:42 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1365883660' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:06:43.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:43 smithi053 ceph-mon[80176]: pgmap v959: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:43 smithi007 ceph-mon[108238]: pgmap v959: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:44.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:06:44.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:06:45.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:45 smithi007 ceph-mon[108238]: pgmap v960: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:45.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:45 smithi053 ceph-mon[80176]: pgmap v960: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:47.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:47 smithi053 ceph-mon[80176]: pgmap v961: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:47 smithi007 ceph-mon[108238]: pgmap v961: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:49.659 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:49 smithi007 ceph-mon[108238]: pgmap v962: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:49 smithi053 ceph-mon[80176]: pgmap v962: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:51.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:51 smithi053 ceph-mon[80176]: pgmap v963: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:51 smithi007 ceph-mon[108238]: pgmap v963: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:53.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:53 smithi053 ceph-mon[80176]: pgmap v964: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:53 smithi007 ceph-mon[108238]: pgmap v964: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:55.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:55 smithi007 ceph-mon[108238]: pgmap v965: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:55.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:55 smithi053 ceph-mon[80176]: pgmap v965: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:57.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:57 smithi053 ceph-mon[80176]: pgmap v966: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:57.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:57 smithi007 ceph-mon[108238]: pgmap v966: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:59 smithi053 ceph-mon[80176]: pgmap v967: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:06:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:06:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:59 smithi007 ceph-mon[108238]: pgmap v967: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:06:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:06:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:07:01.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:01 smithi053 ceph-mon[80176]: pgmap v968: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:01 smithi007 ceph-mon[108238]: pgmap v968: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:03.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:03 smithi053 ceph-mon[80176]: pgmap v969: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:03.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:03 smithi007 ceph-mon[108238]: pgmap v969: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:05.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:05 smithi053 ceph-mon[80176]: pgmap v970: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:05.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:05 smithi007 ceph-mon[108238]: pgmap v970: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:07.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:07 smithi053 ceph-mon[80176]: pgmap v971: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:07.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:07 smithi007 ceph-mon[108238]: pgmap v971: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:09.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:09 smithi053 ceph-mon[80176]: pgmap v972: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:09.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:09 smithi007 ceph-mon[108238]: pgmap v972: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:11.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.452+0000 7f0f9b909640 1 -- 172.21.15.7:0/957379706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94100800 msgr2=0x7f0f94100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:11.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.452+0000 7f0f9b909640 1 --2- 172.21.15.7:0/957379706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94100800 0x7f0f94100c60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009880 tx=0x7f0f8402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.452+0000 7f0f9b909640 1 -- 172.21.15.7:0/957379706 shutdown_connections 2024-09-21T05:07:11.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.452+0000 7f0f9b909640 1 --2- 172.21.15.7:0/957379706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94100800 0x7f0f94100c60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.452+0000 7f0f9b909640 1 --2- 172.21.15.7:0/957379706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f94106800 0x7f0f94106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.452+0000 7f0f9b909640 1 -- 172.21.15.7:0/957379706 >> 172.21.15.7:0/957379706 conn(0x7f0f940fc520 msgr2=0x7f0f940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:11.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.452+0000 7f0f9b909640 1 -- 172.21.15.7:0/957379706 shutdown_connections 2024-09-21T05:07:11.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.452+0000 7f0f9b909640 1 -- 172.21.15.7:0/957379706 wait complete. 2024-09-21T05:07:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.453+0000 7f0f9b909640 1 Processor -- start 2024-09-21T05:07:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.453+0000 7f0f9b909640 1 -- start start 2024-09-21T05:07:11.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.453+0000 7f0f9b909640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f94100800 0x7f0f940fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.453+0000 7f0f9b909640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94106800 0x7f0f940fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.453+0000 7f0f9b909640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9410dcb0 con 0x7f0f94106800 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.453+0000 7f0f9b909640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9410db30 con 0x7f0f94100800 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.453+0000 7f0f98ea4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94106800 0x7f0f940fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f996a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f94100800 0x7f0f940fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f98ea4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94106800 0x7f0f940fd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49434/0 (socket says 172.21.15.7:49434) 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f98ea4640 1 -- 172.21.15.7:0/1415326943 learned_addr learned my addr 172.21.15.7:0/1415326943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f98ea4640 1 -- 172.21.15.7:0/1415326943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f94100800 msgr2=0x7f0f940fcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f98ea4640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f94100800 0x7f0f940fcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f98ea4640 1 -- 172.21.15.7:0/1415326943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f940fd630 con 0x7f0f94106800 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f98ea4640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94106800 0x7f0f940fd0f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0f84031460 tx=0x7f0f840045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f8a7fc640 1 -- 172.21.15.7:0/1415326943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f8403f070 con 0x7f0f94106800 2024-09-21T05:07:11.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f940fd8c0 con 0x7f0f94106800 2024-09-21T05:07:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f8a7fc640 1 -- 172.21.15.7:0/1415326943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f840035f0 con 0x7f0f94106800 2024-09-21T05:07:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f8a7fc640 1 -- 172.21.15.7:0/1415326943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f84035030 con 0x7f0f94106800 2024-09-21T05:07:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.454+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f941a3d60 con 0x7f0f94106800 2024-09-21T05:07:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.456+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f94101f40 con 0x7f0f94106800 2024-09-21T05:07:11.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.456+0000 7f0f8a7fc640 1 -- 172.21.15.7:0/1415326943 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0f84042070 con 0x7f0f94106800 2024-09-21T05:07:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.457+0000 7f0f8a7fc640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f8007af30 0x7f0f8007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.457+0000 7f0f8a7fc640 1 -- 172.21.15.7:0/1415326943 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0f840be9d0 con 0x7f0f94106800 2024-09-21T05:07:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.457+0000 7f0f996a5640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f8007af30 0x7f0f8007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.457+0000 7f0f996a5640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f8007af30 0x7f0f8007d3f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000bbe0 tx=0x7f0f9000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:11.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.462+0000 7f0f8a7fc640 1 -- 172.21.15.7:0/1415326943 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f84089f70 con 0x7f0f94106800 2024-09-21T05:07:11.527 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:11 smithi053 ceph-mon[80176]: pgmap v973: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.666+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0f94062630 con 0x7f0f8007af30 2024-09-21T05:07:11.669 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:11 smithi007 ceph-mon[108238]: pgmap v973: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.668+0000 7f0f8a7fc640 1 -- 172.21.15.7:0/1415326943 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f0f94062630 con 0x7f0f8007af30 2024-09-21T05:07:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f8007af30 msgr2=0x7f0f8007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f8007af30 0x7f0f8007d3f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000bbe0 tx=0x7f0f9000b390 comp rx=0 tx=0).stop 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94106800 msgr2=0x7f0f940fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94106800 0x7f0f940fd0f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0f84031460 tx=0x7f0f840045e0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 shutdown_connections 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f8007af30 0x7f0f8007d3f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f94106800 0x7f0f940fd0f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 --2- 172.21.15.7:0/1415326943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f94100800 0x7f0f940fcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 >> 172.21.15.7:0/1415326943 conn(0x7f0f940fc520 msgr2=0x7f0f9406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 shutdown_connections 2024-09-21T05:07:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.670+0000 7f0f9b909640 1 -- 172.21.15.7:0/1415326943 wait complete. 2024-09-21T05:07:11.682 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:07:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.756+0000 7f66dcee2640 1 -- 172.21.15.7:0/1327519295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8107900 msgr2=0x7f66d8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:11.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.756+0000 7f66dcee2640 1 --2- 172.21.15.7:0/1327519295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8107900 0x7f66d8107ce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009880 tx=0x7f66c802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.756+0000 7f66dcee2640 1 -- 172.21.15.7:0/1327519295 shutdown_connections 2024-09-21T05:07:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.756+0000 7f66dcee2640 1 --2- 172.21.15.7:0/1327519295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8101900 0x7f66d8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.756+0000 7f66dcee2640 1 --2- 172.21.15.7:0/1327519295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8107900 0x7f66d8107ce0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.756+0000 7f66dcee2640 1 -- 172.21.15.7:0/1327519295 >> 172.21.15.7:0/1327519295 conn(0x7f66d80fd600 msgr2=0x7f66d80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.756+0000 7f66dcee2640 1 -- 172.21.15.7:0/1327519295 shutdown_connections 2024-09-21T05:07:11.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.756+0000 7f66dcee2640 1 -- 172.21.15.7:0/1327519295 wait complete. 2024-09-21T05:07:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66dcee2640 1 Processor -- start 2024-09-21T05:07:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66dcee2640 1 -- start start 2024-09-21T05:07:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66dcee2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8101900 0x7f66d819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:11.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66dcee2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8107900 0x7f66d819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66dcee2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810ee30 con 0x7f66d8101900 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66dcee2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810ecb0 con 0x7f66d8107900 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8101900 0x7f66d819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8101900 0x7f66d819f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49454/0 (socket says 172.21.15.7:49454) 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.757+0000 7f66d659c640 1 -- 172.21.15.7:0/2758482538 learned_addr learned my addr 172.21.15.7:0/2758482538 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66d5d9b640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8107900 0x7f66d819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66d659c640 1 -- 172.21.15.7:0/2758482538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8107900 msgr2=0x7f66d819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66d659c640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8107900 0x7f66d819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66d659c640 1 -- 172.21.15.7:0/2758482538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d8199a30 con 0x7f66d8101900 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66d659c640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8101900 0x7f66d819f870 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f66c8031460 tx=0x7f66c80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66c77fe640 1 -- 172.21.15.7:0/2758482538 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66c803f070 con 0x7f66d8101900 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66c77fe640 1 -- 172.21.15.7:0/2758482538 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66c8004cf0 con 0x7f66d8101900 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66c77fe640 1 -- 172.21.15.7:0/2758482538 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66c8035030 con 0x7f66d8101900 2024-09-21T05:07:11.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d8199cc0 con 0x7f66d8101900 2024-09-21T05:07:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.758+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d819a1f0 con 0x7f66d8101900 2024-09-21T05:07:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.759+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d8103040 con 0x7f66d8101900 2024-09-21T05:07:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.760+0000 7f66c77fe640 1 -- 172.21.15.7:0/2758482538 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f66c8042070 con 0x7f66d8101900 2024-09-21T05:07:11.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.760+0000 7f66c77fe640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66c007af80 0x7f66c007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.760+0000 7f66c77fe640 1 -- 172.21.15.7:0/2758482538 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f66c80bea40 con 0x7f66d8101900 2024-09-21T05:07:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.761+0000 7f66d5d9b640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66c007af80 0x7f66c007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:11.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.761+0000 7f66d5d9b640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66c007af80 0x7f66c007d440 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f66d8075f50 tx=0x7f66cc008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.765+0000 7f66c77fe640 1 -- 172.21.15.7:0/2758482538 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66c808a060 con 0x7f66d8101900 2024-09-21T05:07:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.971+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f66d819b070 con 0x7f66c007af80 2024-09-21T05:07:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.972+0000 7f66c77fe640 1 -- 172.21.15.7:0/2758482538 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f66d819b070 con 0x7f66c007af80 2024-09-21T05:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66c007af80 msgr2=0x7f66c007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66c007af80 0x7f66c007d440 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f66d8075f50 tx=0x7f66cc008040 comp rx=0 tx=0).stop 2024-09-21T05:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8101900 msgr2=0x7f66d819f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8101900 0x7f66d819f870 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f66c8031460 tx=0x7f66c80048f0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 shutdown_connections 2024-09-21T05:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66c007af80 0x7f66c007d440 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66d8107900 0x7f66d819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 --2- 172.21.15.7:0/2758482538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66d8101900 0x7f66d819f870 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 >> 172.21.15.7:0/2758482538 conn(0x7f66d80fd600 msgr2=0x7f66d8074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 shutdown_connections 2024-09-21T05:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:11.974+0000 7f66dcee2640 1 -- 172.21.15.7:0/2758482538 wait complete. 2024-09-21T05:07:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.058+0000 7f709fd03640 1 -- 172.21.15.7:0/1263789768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 msgr2=0x7f7098100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.058+0000 7f709fd03640 1 --2- 172.21.15.7:0/1263789768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 0x7f7098100c90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7094009880 tx=0x7f709402a530 comp rx=0 tx=0).stop 2024-09-21T05:07:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.059+0000 7f709fd03640 1 -- 172.21.15.7:0/1263789768 shutdown_connections 2024-09-21T05:07:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.059+0000 7f709fd03640 1 --2- 172.21.15.7:0/1263789768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70981011d0 0x7f70981056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.059+0000 7f709fd03640 1 --2- 172.21.15.7:0/1263789768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 0x7f7098100c90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.059+0000 7f709fd03640 1 -- 172.21.15.7:0/1263789768 >> 172.21.15.7:0/1263789768 conn(0x7f70980fc520 msgr2=0x7f70980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.059+0000 7f709fd03640 1 -- 172.21.15.7:0/1263789768 shutdown_connections 2024-09-21T05:07:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.059+0000 7f709fd03640 1 -- 172.21.15.7:0/1263789768 wait complete. 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.059+0000 7f709fd03640 1 Processor -- start 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.059+0000 7f709fd03640 1 -- start start 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709fd03640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 0x7f709819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709fd03640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70981011d0 0x7f709819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709fd03640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f70981153d0 con 0x7f70981008b0 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709fd03640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7098115250 con 0x7f70981011d0 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709da9f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 0x7f709819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709da9f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 0x7f709819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49466/0 (socket says 172.21.15.7:49466) 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709da9f640 1 -- 172.21.15.7:0/602096592 learned_addr learned my addr 172.21.15.7:0/602096592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709d29e640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70981011d0 0x7f709819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709da9f640 1 -- 172.21.15.7:0/602096592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70981011d0 msgr2=0x7f709819e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709da9f640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70981011d0 0x7f709819e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709da9f640 1 -- 172.21.15.7:0/602096592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70981a2860 con 0x7f70981008b0 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709da9f640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 0x7f709819df90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7094031460 tx=0x7f7094004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f708effd640 1 -- 172.21.15.7:0/602096592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f709403f070 con 0x7f70981008b0 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70981a2af0 con 0x7f70981008b0 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f708effd640 1 -- 172.21.15.7:0/602096592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f70940035f0 con 0x7f70981008b0 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f708effd640 1 -- 172.21.15.7:0/602096592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7094035030 con 0x7f70981008b0 2024-09-21T05:07:12.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.060+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70981a3020 con 0x7f70981008b0 2024-09-21T05:07:12.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.061+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f709806aa90 con 0x7f70981008b0 2024-09-21T05:07:12.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.062+0000 7f708effd640 1 -- 172.21.15.7:0/602096592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7094042070 con 0x7f70981008b0 2024-09-21T05:07:12.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.063+0000 7f708effd640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f708407af80 0x7f708407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.063+0000 7f708effd640 1 -- 172.21.15.7:0/602096592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f70940be890 con 0x7f70981008b0 2024-09-21T05:07:12.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.063+0000 7f709d29e640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f708407af80 0x7f708407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.063+0000 7f709d29e640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f708407af80 0x7f708407d440 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f70981124f0 tx=0x7f7088005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:12.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.067+0000 7f708effd640 1 -- 172.21.15.7:0/602096592 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7094089eb0 con 0x7f70981008b0 2024-09-21T05:07:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.265+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f70981a3390 con 0x7f708407af80 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.275+0000 7f708effd640 1 -- 172.21.15.7:0/602096592 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f70981a3390 con 0x7f708407af80 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (29m) 7m ago 40m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (32m) 7m ago 40m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (31m) 7m ago 39m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (27m) 7m ago 40m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (27m) 7m ago 39m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (28m) 7m ago 39m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (23m) 7m ago 34m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:07:12.278 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (23m) 7m ago 34m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (7m) 7m ago 34m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (23m) 7m ago 34m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (31m) 7m ago 41m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (30m) 7m ago 39m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (27m) 7m ago 41m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (27m) 7m ago 39m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (30m) 7m ago 40m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (30m) 7m ago 39m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (26m) 7m ago 38m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (26m) 7m ago 37m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (25m) 7m ago 37m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (25m) 7m ago 36m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (24m) 7m ago 36m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:07:12.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (23m) 7m ago 35m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:07:12.280 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (29m) 7m ago 39m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:07:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f708407af80 msgr2=0x7f708407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f708407af80 0x7f708407d440 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f70981124f0 tx=0x7f7088005f90 comp rx=0 tx=0).stop 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 msgr2=0x7f709819df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 0x7f709819df90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7094031460 tx=0x7f7094004530 comp rx=0 tx=0).stop 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 shutdown_connections 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f708407af80 0x7f708407d440 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70981011d0 0x7f709819e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 --2- 172.21.15.7:0/602096592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70981008b0 0x7f709819df90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.277+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 >> 172.21.15.7:0/602096592 conn(0x7f70980fc520 msgr2=0x7f7098112b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.278+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 shutdown_connections 2024-09-21T05:07:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.278+0000 7f709fd03640 1 -- 172.21.15.7:0/602096592 wait complete. 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.360+0000 7f4da9892640 1 -- 172.21.15.7:0/486485576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da41033e0 msgr2=0x7f4da410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.360+0000 7f4da9892640 1 --2- 172.21.15.7:0/486485576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da41033e0 0x7f4da410b8d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009880 tx=0x7f4d9802a530 comp rx=0 tx=0).stop 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.360+0000 7f4da9892640 1 -- 172.21.15.7:0/486485576 shutdown_connections 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.360+0000 7f4da9892640 1 --2- 172.21.15.7:0/486485576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da41033e0 0x7f4da410b8d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.360+0000 7f4da9892640 1 --2- 172.21.15.7:0/486485576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da4102ac0 0x7f4da4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.360+0000 7f4da9892640 1 -- 172.21.15.7:0/486485576 >> 172.21.15.7:0/486485576 conn(0x7f4da40fc520 msgr2=0x7f4da40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.360+0000 7f4da9892640 1 -- 172.21.15.7:0/486485576 shutdown_connections 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.360+0000 7f4da9892640 1 -- 172.21.15.7:0/486485576 wait complete. 2024-09-21T05:07:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.361+0000 7f4da9892640 1 Processor -- start 2024-09-21T05:07:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.361+0000 7f4da9892640 1 -- start start 2024-09-21T05:07:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.361+0000 7f4da9892640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da4102ac0 0x7f4da419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.361+0000 7f4da9892640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da41033e0 0x7f4da419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.361+0000 7f4da9892640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4da410edf0 con 0x7f4da4102ac0 2024-09-21T05:07:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.361+0000 7f4da9892640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4da410ec70 con 0x7f4da41033e0 2024-09-21T05:07:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da4102ac0 0x7f4da419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da4102ac0 0x7f4da419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49486/0 (socket says 172.21.15.7:49486) 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da2ffd640 1 -- 172.21.15.7:0/1227249883 learned_addr learned my addr 172.21.15.7:0/1227249883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da27fc640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da41033e0 0x7f4da419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da2ffd640 1 -- 172.21.15.7:0/1227249883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da41033e0 msgr2=0x7f4da419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da2ffd640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da41033e0 0x7f4da419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da2ffd640 1 -- 172.21.15.7:0/1227249883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da4197800 con 0x7f4da4102ac0 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da2ffd640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da4102ac0 0x7f4da419d680 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400d6f0 tx=0x7f4d9400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da8890640 1 -- 172.21.15.7:0/1227249883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4d94017070 con 0x7f4da4102ac0 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da8890640 1 -- 172.21.15.7:0/1227249883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d9400de00 con 0x7f4da4102ac0 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da4197af0 con 0x7f4da4102ac0 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da8890640 1 -- 172.21.15.7:0/1227249883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4d94004bb0 con 0x7f4da4102ac0 2024-09-21T05:07:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.362+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da4198050 con 0x7f4da4102ac0 2024-09-21T05:07:12.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.364+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da406aa90 con 0x7f4da4102ac0 2024-09-21T05:07:12.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.364+0000 7f4da8890640 1 -- 172.21.15.7:0/1227249883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4d94009a30 con 0x7f4da4102ac0 2024-09-21T05:07:12.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.364+0000 7f4da8890640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8c07af30 0x7f4d8c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.365+0000 7f4da8890640 1 -- 172.21.15.7:0/1227249883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f4d9409dcc0 con 0x7f4da4102ac0 2024-09-21T05:07:12.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.365+0000 7f4da27fc640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8c07af30 0x7f4d8c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.365+0000 7f4da27fc640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8c07af30 0x7f4d8c07d3f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4da4198b50 tx=0x7f4d98031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:12.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.369+0000 7f4da8890640 1 -- 172.21.15.7:0/1227249883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d94069260 con 0x7f4da4102ac0 2024-09-21T05:07:12.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.630+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f4da4105f60 con 0x7f4da4102ac0 2024-09-21T05:07:12.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.630+0000 7f4da8890640 1 -- 172.21.15.7:0/1227249883 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f4d9406e110 con 0x7f4da4102ac0 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:07:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:07:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:07:12.635 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8c07af30 msgr2=0x7f4d8c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8c07af30 0x7f4d8c07d3f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4da4198b50 tx=0x7f4d98031400 comp rx=0 tx=0).stop 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da4102ac0 msgr2=0x7f4da419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da4102ac0 0x7f4da419d680 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400d6f0 tx=0x7f4d9400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 shutdown_connections 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4d8c07af30 0x7f4d8c07d3f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4da41033e0 0x7f4da419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 --2- 172.21.15.7:0/1227249883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da4102ac0 0x7f4da419d680 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 >> 172.21.15.7:0/1227249883 conn(0x7f4da40fc520 msgr2=0x7f4da4108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 shutdown_connections 2024-09-21T05:07:12.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.633+0000 7f4da9892640 1 -- 172.21.15.7:0/1227249883 wait complete. 2024-09-21T05:07:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.718+0000 7f24e5e81640 1 -- 172.21.15.7:0/1322825267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01007f0 msgr2=0x7f24e0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.718+0000 7f24e5e81640 1 --2- 172.21.15.7:0/1322825267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01007f0 0x7f24e0100c50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009880 tx=0x7f24d402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.718+0000 7f24e5e81640 1 -- 172.21.15.7:0/1322825267 shutdown_connections 2024-09-21T05:07:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.718+0000 7f24e5e81640 1 --2- 172.21.15.7:0/1322825267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01007f0 0x7f24e0100c50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.718+0000 7f24e5e81640 1 --2- 172.21.15.7:0/1322825267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e01067f0 0x7f24e0106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.718+0000 7f24e5e81640 1 -- 172.21.15.7:0/1322825267 >> 172.21.15.7:0/1322825267 conn(0x7f24e00fc550 msgr2=0x7f24e00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.718+0000 7f24e5e81640 1 -- 172.21.15.7:0/1322825267 shutdown_connections 2024-09-21T05:07:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.718+0000 7f24e5e81640 1 -- 172.21.15.7:0/1322825267 wait complete. 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24e5e81640 1 Processor -- start 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24e5e81640 1 -- start start 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24e5e81640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e01007f0 0x7f24e01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24e5e81640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01067f0 0x7f24e0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24e5e81640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010dcc0 con 0x7f24e01067f0 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24e5e81640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010db40 con 0x7f24e01007f0 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24deffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01067f0 0x7f24e0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24deffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01067f0 0x7f24e0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49498/0 (socket says 172.21.15.7:49498) 2024-09-21T05:07:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24deffd640 1 -- 172.21.15.7:0/1744722859 learned_addr learned my addr 172.21.15.7:0/1744722859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.719+0000 7f24df7fe640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e01007f0 0x7f24e01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24deffd640 1 -- 172.21.15.7:0/1744722859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e01007f0 msgr2=0x7f24e01987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24deffd640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e01007f0 0x7f24e01987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24deffd640 1 -- 172.21.15.7:0/1744722859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e0199340 con 0x7f24e01067f0 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24deffd640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01067f0 0x7f24e0198cf0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f24d4031460 tx=0x7f24d4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24dcff9640 1 -- 172.21.15.7:0/1744722859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24d403f070 con 0x7f24e01067f0 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e01995d0 con 0x7f24e01067f0 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24dcff9640 1 -- 172.21.15.7:0/1744722859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f24d40035f0 con 0x7f24e01067f0 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24dcff9640 1 -- 172.21.15.7:0/1744722859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24d4035030 con 0x7f24e01067f0 2024-09-21T05:07:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.720+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e01a1b80 con 0x7f24e01067f0 2024-09-21T05:07:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.721+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e0101f30 con 0x7f24e01067f0 2024-09-21T05:07:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.722+0000 7f24dcff9640 1 -- 172.21.15.7:0/1744722859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f24d4042070 con 0x7f24e01067f0 2024-09-21T05:07:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.722+0000 7f24dcff9640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c807af80 0x7f24c807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.722+0000 7f24dcff9640 1 -- 172.21.15.7:0/1744722859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f24d40beab0 con 0x7f24e01067f0 2024-09-21T05:07:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.723+0000 7f24df7fe640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c807af80 0x7f24c807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.723+0000 7f24df7fe640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c807af80 0x7f24c807d440 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009b60 tx=0x7f24d0005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:12.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.727+0000 7f24dcff9640 1 -- 172.21.15.7:0/1744722859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24d4089fb0 con 0x7f24e01067f0 2024-09-21T05:07:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.951+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f24e0108d60 con 0x7f24e01067f0 2024-09-21T05:07:12.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.951+0000 7f24dcff9640 1 -- 172.21.15.7:0/1744722859 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f24d408ee60 con 0x7f24e01067f0 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:07:12.955 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:07:12.956 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:07:12.957 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:07:12.958 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:07:12.958 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:07:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:07:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c807af80 msgr2=0x7f24c807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c807af80 0x7f24c807d440 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009b60 tx=0x7f24d0005f90 comp rx=0 tx=0).stop 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01067f0 msgr2=0x7f24e0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01067f0 0x7f24e0198cf0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f24d4031460 tx=0x7f24d4004530 comp rx=0 tx=0).stop 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 shutdown_connections 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24c807af80 0x7f24c807d440 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e01067f0 0x7f24e0198cf0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 --2- 172.21.15.7:0/1744722859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24e01007f0 0x7f24e01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.954+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 >> 172.21.15.7:0/1744722859 conn(0x7f24e00fc550 msgr2=0x7f24e00fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.955+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 shutdown_connections 2024-09-21T05:07:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:12.955+0000 7f24e5e81640 1 -- 172.21.15.7:0/1744722859 wait complete. 2024-09-21T05:07:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.039+0000 7f7528f24640 1 -- 172.21.15.7:0/152510932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 msgr2=0x7f7524107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.039+0000 7f7528f24640 1 --2- 172.21.15.7:0/152510932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 0x7f7524107ce0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7514009880 tx=0x7f751402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.039+0000 7f7528f24640 1 -- 172.21.15.7:0/152510932 shutdown_connections 2024-09-21T05:07:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.039+0000 7f7528f24640 1 --2- 172.21.15.7:0/152510932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7524101900 0x7f7524101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.039+0000 7f7528f24640 1 --2- 172.21.15.7:0/152510932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 0x7f7524107ce0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.039+0000 7f7528f24640 1 -- 172.21.15.7:0/152510932 >> 172.21.15.7:0/152510932 conn(0x7f75240fd600 msgr2=0x7f75240ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.039+0000 7f7528f24640 1 -- 172.21.15.7:0/152510932 shutdown_connections 2024-09-21T05:07:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.039+0000 7f7528f24640 1 -- 172.21.15.7:0/152510932 wait complete. 2024-09-21T05:07:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.040+0000 7f7528f24640 1 Processor -- start 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.040+0000 7f7528f24640 1 -- start start 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7528f24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7524101900 0x7f75240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7528f24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 0x7f7524074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7528f24640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f752410edb0 con 0x7f7524107900 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7528f24640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f752410ec30 con 0x7f7524101900 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7521d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 0x7f7524074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7521d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 0x7f7524074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49512/0 (socket says 172.21.15.7:49512) 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7521d9b640 1 -- 172.21.15.7:0/2387307986 learned_addr learned my addr 172.21.15.7:0/2387307986 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f752259c640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7524101900 0x7f75240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7521d9b640 1 -- 172.21.15.7:0/2387307986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7524101900 msgr2=0x7f75240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7521d9b640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7524101900 0x7f75240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7521d9b640 1 -- 172.21.15.7:0/2387307986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7524074f70 con 0x7f7524107900 2024-09-21T05:07:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7521d9b640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 0x7f7524074a30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f751800d8d0 tx=0x7f751800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f75137fe640 1 -- 172.21.15.7:0/2387307986 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7518017070 con 0x7f7524107900 2024-09-21T05:07:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.041+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7524075260 con 0x7f7524107900 2024-09-21T05:07:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.042+0000 7f75137fe640 1 -- 172.21.15.7:0/2387307986 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7518010460 con 0x7f7524107900 2024-09-21T05:07:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.042+0000 7f75137fe640 1 -- 172.21.15.7:0/2387307986 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f751800f5f0 con 0x7f7524107900 2024-09-21T05:07:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.042+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75241a3e60 con 0x7f7524107900 2024-09-21T05:07:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.043+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7524103040 con 0x7f7524107900 2024-09-21T05:07:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.043+0000 7f75137fe640 1 -- 172.21.15.7:0/2387307986 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7518009a30 con 0x7f7524107900 2024-09-21T05:07:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.044+0000 7f75137fe640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f750c07af30 0x7f750c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.044+0000 7f752259c640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f750c07af30 0x7f750c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.044+0000 7f75137fe640 1 -- 172.21.15.7:0/2387307986 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f751809df30 con 0x7f7524107900 2024-09-21T05:07:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.044+0000 7f752259c640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f750c07af30 0x7f750c07d3f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f75140096c0 tx=0x7f7514031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.048+0000 7f75137fe640 1 -- 172.21.15.7:0/2387307986 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7518069430 con 0x7f7524107900 2024-09-21T05:07:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.252+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7524062630 con 0x7f750c07af30 2024-09-21T05:07:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.253+0000 7f75137fe640 1 -- 172.21.15.7:0/2387307986 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7524062630 con 0x7f750c07af30 2024-09-21T05:07:13.256 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:07:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:07:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:07:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:07:13.257 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:07:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f750c07af30 msgr2=0x7f750c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f750c07af30 0x7f750c07d3f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f75140096c0 tx=0x7f7514031400 comp rx=0 tx=0).stop 2024-09-21T05:07:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 msgr2=0x7f7524074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 0x7f7524074a30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f751800d8d0 tx=0x7f751800dd90 comp rx=0 tx=0).stop 2024-09-21T05:07:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 shutdown_connections 2024-09-21T05:07:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f750c07af30 0x7f750c07d3f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7524107900 0x7f7524074a30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 --2- 172.21.15.7:0/2387307986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7524101900 0x7f75240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 >> 172.21.15.7:0/2387307986 conn(0x7f75240fd600 msgr2=0x7f75240fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 shutdown_connections 2024-09-21T05:07:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.255+0000 7f7528f24640 1 -- 172.21.15.7:0/2387307986 wait complete. 2024-09-21T05:07:13.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.339+0000 7f76c8420640 1 -- 172.21.15.7:0/216437672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 msgr2=0x7f76c0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:13.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.339+0000 7f76c8420640 1 --2- 172.21.15.7:0/216437672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 0x7f76c0107ce0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009880 tx=0x7f76bc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.339+0000 7f76c8420640 1 -- 172.21.15.7:0/216437672 shutdown_connections 2024-09-21T05:07:13.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.339+0000 7f76c8420640 1 --2- 172.21.15.7:0/216437672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0101900 0x7f76c0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.339+0000 7f76c8420640 1 --2- 172.21.15.7:0/216437672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 0x7f76c0107ce0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.339+0000 7f76c8420640 1 -- 172.21.15.7:0/216437672 >> 172.21.15.7:0/216437672 conn(0x7f76c00fd600 msgr2=0x7f76c00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:13.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.339+0000 7f76c8420640 1 -- 172.21.15.7:0/216437672 shutdown_connections 2024-09-21T05:07:13.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.339+0000 7f76c8420640 1 -- 172.21.15.7:0/216437672 wait complete. 2024-09-21T05:07:13.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c8420640 1 Processor -- start 2024-09-21T05:07:13.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c8420640 1 -- start start 2024-09-21T05:07:13.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c8420640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0101900 0x7f76c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c8420640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 0x7f76c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c8420640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010edb0 con 0x7f76c0107900 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c8420640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010ec30 con 0x7f76c0101900 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c59bb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 0x7f76c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c59bb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 0x7f76c0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49518/0 (socket says 172.21.15.7:49518) 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.340+0000 7f76c59bb640 1 -- 172.21.15.7:0/758214183 learned_addr learned my addr 172.21.15.7:0/758214183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76c61bc640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0101900 0x7f76c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76c59bb640 1 -- 172.21.15.7:0/758214183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0101900 msgr2=0x7f76c00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76c59bb640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0101900 0x7f76c00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76c59bb640 1 -- 172.21.15.7:0/758214183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c0074f70 con 0x7f76c0107900 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76c59bb640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 0x7f76c0074a30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f76b000d8d0 tx=0x7f76b000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76b77fe640 1 -- 172.21.15.7:0/758214183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f76b0017070 con 0x7f76c0107900 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c0075260 con 0x7f76c0107900 2024-09-21T05:07:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76b77fe640 1 -- 172.21.15.7:0/758214183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f76b0010460 con 0x7f76c0107900 2024-09-21T05:07:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76b77fe640 1 -- 172.21.15.7:0/758214183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f76b000f5f0 con 0x7f76c0107900 2024-09-21T05:07:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.341+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c01a3e40 con 0x7f76c0107900 2024-09-21T05:07:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.342+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c0103040 con 0x7f76c0107900 2024-09-21T05:07:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.343+0000 7f76b77fe640 1 -- 172.21.15.7:0/758214183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f76b0009a30 con 0x7f76c0107900 2024-09-21T05:07:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.343+0000 7f76b77fe640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76ac07af40 0x7f76ac07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.344+0000 7f76b77fe640 1 -- 172.21.15.7:0/758214183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f76b009dfc0 con 0x7f76c0107900 2024-09-21T05:07:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.344+0000 7f76c61bc640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76ac07af40 0x7f76ac07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.344+0000 7f76c61bc640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76ac07af40 0x7f76ac07d400 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f76bc031460 tx=0x7f76bc0095c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:13.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.348+0000 7f76b77fe640 1 -- 172.21.15.7:0/758214183 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f76b00694c0 con 0x7f76c0107900 2024-09-21T05:07:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.622+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f76c0075a20 con 0x7f76c0107900 2024-09-21T05:07:13.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:13 smithi007 ceph-mon[108238]: pgmap v974: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:13.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:13 smithi007 ceph-mon[108238]: from='client.35572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:13.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:13 smithi007 ceph-mon[108238]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:13.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:13 smithi007 ceph-mon[108238]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:13.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:13 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1227249883' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:07:13.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:13 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1744722859' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:07:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.622+0000 7f76b77fe640 1 -- 172.21.15.7:0/758214183 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f76b006e370 con 0x7f76c0107900 2024-09-21T05:07:13.626 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:07:13.626 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:07:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:07:13.626 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:07:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:07:13.628 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:07:13.628 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:07:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:07:13.628 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:07:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:07:13.628 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:07:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76ac07af40 msgr2=0x7f76ac07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76ac07af40 0x7f76ac07d400 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f76bc031460 tx=0x7f76bc0095c0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 msgr2=0x7f76c0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 0x7f76c0074a30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f76b000d8d0 tx=0x7f76b000dd90 comp rx=0 tx=0).stop 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 shutdown_connections 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f76ac07af40 0x7f76ac07d400 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76c0107900 0x7f76c0074a30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 --2- 172.21.15.7:0/758214183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76c0101900 0x7f76c00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 >> 172.21.15.7:0/758214183 conn(0x7f76c00fd600 msgr2=0x7f76c00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 shutdown_connections 2024-09-21T05:07:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:13.625+0000 7f76c8420640 1 -- 172.21.15.7:0/758214183 wait complete. 2024-09-21T05:07:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:13 smithi053 ceph-mon[80176]: pgmap v974: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:13 smithi053 ceph-mon[80176]: from='client.35572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:13 smithi053 ceph-mon[80176]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:13 smithi053 ceph-mon[80176]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:13 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1227249883' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:07:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:13 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1744722859' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:07:14.594 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:14 smithi053 ceph-mon[80176]: from='client.35592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:14.595 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:07:14.595 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:14 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/758214183' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:07:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:14 smithi007 ceph-mon[108238]: from='client.35592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:07:14.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:14 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/758214183' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:07:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:15 smithi007 ceph-mon[108238]: pgmap v975: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:15.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:15 smithi053 ceph-mon[80176]: pgmap v975: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:17 smithi053 ceph-mon[80176]: pgmap v976: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:17 smithi007 ceph-mon[108238]: pgmap v976: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:19 smithi053 ceph-mon[80176]: pgmap v977: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:19 smithi007 ceph-mon[108238]: pgmap v977: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:21.678 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:21 smithi053 ceph-mon[80176]: pgmap v978: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:21.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:21 smithi007 ceph-mon[108238]: pgmap v978: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:23.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:23 smithi053 ceph-mon[80176]: pgmap v979: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:23 smithi007 ceph-mon[108238]: pgmap v979: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:25.497 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:25 smithi007 ceph-mon[108238]: pgmap v980: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:25.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:25 smithi053 ceph-mon[80176]: pgmap v980: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:27.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:27 smithi053 ceph-mon[80176]: pgmap v981: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:27 smithi007 ceph-mon[108238]: pgmap v981: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:29.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:29 smithi053 ceph-mon[80176]: pgmap v982: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:29.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:07:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:29 smithi007 ceph-mon[108238]: pgmap v982: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:07:31.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:31 smithi053 ceph-mon[80176]: pgmap v983: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:31.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:31 smithi007 ceph-mon[108238]: pgmap v983: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:33.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:33 smithi053 ceph-mon[80176]: pgmap v984: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:33 smithi007 ceph-mon[108238]: pgmap v984: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:35.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:35 smithi007 ceph-mon[108238]: pgmap v985: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:35.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:35 smithi053 ceph-mon[80176]: pgmap v985: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:37.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:37 smithi053 ceph-mon[80176]: pgmap v986: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:37 smithi007 ceph-mon[108238]: pgmap v986: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:39.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:39 smithi053 ceph-mon[80176]: pgmap v987: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:39 smithi007 ceph-mon[108238]: pgmap v987: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:41 smithi053 ceph-mon[80176]: pgmap v988: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:41 smithi007 ceph-mon[108238]: pgmap v988: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:43 smithi053 ceph-mon[80176]: pgmap v989: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:43 smithi007 ceph-mon[108238]: pgmap v989: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.723+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/1032597806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 msgr2=0x7f3b44102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.723+0000 7f3b4b6a4640 1 --2- 172.21.15.7:0/1032597806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 0x7f3b44102ef0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3b400098b0 tx=0x7f3b4002a560 comp rx=0 tx=0).stop 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.723+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/1032597806 shutdown_connections 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.723+0000 7f3b4b6a4640 1 --2- 172.21.15.7:0/1032597806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44103430 0x7f3b4410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.723+0000 7f3b4b6a4640 1 --2- 172.21.15.7:0/1032597806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 0x7f3b44102ef0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.723+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/1032597806 >> 172.21.15.7:0/1032597806 conn(0x7f3b440fc550 msgr2=0x7f3b440fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.723+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/1032597806 shutdown_connections 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.723+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/1032597806 wait complete. 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b4b6a4640 1 Processor -- start 2024-09-21T05:07:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b4b6a4640 1 -- start start 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b4b6a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 0x7f3b44100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b4b6a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44103430 0x7f3b44100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b4b6a4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410edc0 con 0x7f3b44102b10 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b4b6a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410ec40 con 0x7f3b44103430 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b49440640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 0x7f3b44100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b49440640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 0x7f3b44100770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54008/0 (socket says 172.21.15.7:54008) 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b49440640 1 -- 172.21.15.7:0/691575440 learned_addr learned my addr 172.21.15.7:0/691575440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.724+0000 7f3b48c3f640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44103430 0x7f3b44100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b49440640 1 -- 172.21.15.7:0/691575440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44103430 msgr2=0x7f3b44100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b49440640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44103430 0x7f3b44100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b49440640 1 -- 172.21.15.7:0/691575440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b441011f0 con 0x7f3b44102b10 2024-09-21T05:07:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b49440640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 0x7f3b44100770 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009230 tx=0x7f3b400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b3a7fc640 1 -- 172.21.15.7:0/691575440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3b4003f070 con 0x7f3b44102b10 2024-09-21T05:07:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b44101480 con 0x7f3b44102b10 2024-09-21T05:07:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b3a7fc640 1 -- 172.21.15.7:0/691575440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b40004ac0 con 0x7f3b44102b10 2024-09-21T05:07:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b3a7fc640 1 -- 172.21.15.7:0/691575440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3b40035030 con 0x7f3b44102b10 2024-09-21T05:07:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.725+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441a3d40 con 0x7f3b44102b10 2024-09-21T05:07:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.726+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4406a520 con 0x7f3b44102b10 2024-09-21T05:07:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.727+0000 7f3b3a7fc640 1 -- 172.21.15.7:0/691575440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3b40042070 con 0x7f3b44102b10 2024-09-21T05:07:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.727+0000 7f3b3a7fc640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b3007af90 0x7f3b3007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.728+0000 7f3b3a7fc640 1 -- 172.21.15.7:0/691575440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f3b400be9f0 con 0x7f3b44102b10 2024-09-21T05:07:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.728+0000 7f3b48c3f640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b3007af90 0x7f3b3007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.728+0000 7f3b48c3f640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b3007af90 0x7f3b3007d450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3b44102200 tx=0x7f3b3400a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.732+0000 7f3b3a7fc640 1 -- 172.21.15.7:0/691575440 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b4008a010 con 0x7f3b44102b10 2024-09-21T05:07:43.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.937+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3b44062630 con 0x7f3b3007af90 2024-09-21T05:07:43.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.938+0000 7f3b3a7fc640 1 -- 172.21.15.7:0/691575440 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3b44062630 con 0x7f3b3007af90 2024-09-21T05:07:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b3007af90 msgr2=0x7f3b3007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b3007af90 0x7f3b3007d450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3b44102200 tx=0x7f3b3400a2d0 comp rx=0 tx=0).stop 2024-09-21T05:07:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 msgr2=0x7f3b44100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 0x7f3b44100770 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009230 tx=0x7f3b400046c0 comp rx=0 tx=0).stop 2024-09-21T05:07:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 shutdown_connections 2024-09-21T05:07:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b3007af90 0x7f3b3007d450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b44103430 0x7f3b44100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 --2- 172.21.15.7:0/691575440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b44102b10 0x7f3b44100770 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:43.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 >> 172.21.15.7:0/691575440 conn(0x7f3b440fc550 msgr2=0x7f3b44107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 shutdown_connections 2024-09-21T05:07:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:43.940+0000 7f3b4b6a4640 1 -- 172.21.15.7:0/691575440 wait complete. 2024-09-21T05:07:43.953 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:07:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.025+0000 7f471cbcc640 1 -- 172.21.15.7:0/2162714496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 msgr2=0x7f4718102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.025+0000 7f471cbcc640 1 --2- 172.21.15.7:0/2162714496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 0x7f4718102e90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4708009880 tx=0x7f470802a530 comp rx=0 tx=0).stop 2024-09-21T05:07:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.026+0000 7f471cbcc640 1 -- 172.21.15.7:0/2162714496 shutdown_connections 2024-09-21T05:07:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.026+0000 7f471cbcc640 1 --2- 172.21.15.7:0/2162714496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47181033d0 0x7f471810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.026+0000 7f471cbcc640 1 --2- 172.21.15.7:0/2162714496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 0x7f4718102e90 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.026+0000 7f471cbcc640 1 -- 172.21.15.7:0/2162714496 >> 172.21.15.7:0/2162714496 conn(0x7f47180fc550 msgr2=0x7f47180fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.026+0000 7f471cbcc640 1 -- 172.21.15.7:0/2162714496 shutdown_connections 2024-09-21T05:07:44.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.026+0000 7f471cbcc640 1 -- 172.21.15.7:0/2162714496 wait complete. 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.026+0000 7f471cbcc640 1 Processor -- start 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471cbcc640 1 -- start start 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471cbcc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 0x7f471819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471cbcc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47181033d0 0x7f471819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471cbcc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f471810edc0 con 0x7f4718102ab0 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471cbcc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f471810ec40 con 0x7f47181033d0 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 0x7f471819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 0x7f471819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54040/0 (socket says 172.21.15.7:54040) 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471659c640 1 -- 172.21.15.7:0/497935034 learned_addr learned my addr 172.21.15.7:0/497935034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:44.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f4715d9b640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47181033d0 0x7f471819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471659c640 1 -- 172.21.15.7:0/497935034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47181033d0 msgr2=0x7f471819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471659c640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47181033d0 0x7f471819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.027+0000 7f471659c640 1 -- 172.21.15.7:0/497935034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47181a03d0 con 0x7f4718102ab0 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.028+0000 7f471659c640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 0x7f471819f7e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4708031460 tx=0x7f47080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.028+0000 7f47077fe640 1 -- 172.21.15.7:0/497935034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f470803f070 con 0x7f4718102ab0 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.028+0000 7f47077fe640 1 -- 172.21.15.7:0/497935034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4708004cf0 con 0x7f4718102ab0 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.028+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4718199a60 con 0x7f4718102ab0 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.028+0000 7f47077fe640 1 -- 172.21.15.7:0/497935034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4708035030 con 0x7f4718102ab0 2024-09-21T05:07:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.028+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4718199f60 con 0x7f4718102ab0 2024-09-21T05:07:44.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.029+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471806a580 con 0x7f4718102ab0 2024-09-21T05:07:44.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.030+0000 7f47077fe640 1 -- 172.21.15.7:0/497935034 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4708042070 con 0x7f4718102ab0 2024-09-21T05:07:44.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.030+0000 7f47077fe640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f470007af80 0x7f470007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.030+0000 7f47077fe640 1 -- 172.21.15.7:0/497935034 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f47080beae0 con 0x7f4718102ab0 2024-09-21T05:07:44.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.030+0000 7f4715d9b640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f470007af80 0x7f470007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.031+0000 7f4715d9b640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f470007af80 0x7f470007d440 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f470c005fd0 tx=0x7f470c005f60 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:44.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.035+0000 7f47077fe640 1 -- 172.21.15.7:0/497935034 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f470808a060 con 0x7f4718102ab0 2024-09-21T05:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.241+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f471819afe0 con 0x7f470007af80 2024-09-21T05:07:44.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.242+0000 7f47077fe640 1 -- 172.21.15.7:0/497935034 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f471819afe0 con 0x7f470007af80 2024-09-21T05:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f470007af80 msgr2=0x7f470007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f470007af80 0x7f470007d440 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f470c005fd0 tx=0x7f470c005f60 comp rx=0 tx=0).stop 2024-09-21T05:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 msgr2=0x7f471819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 0x7f471819f7e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4708031460 tx=0x7f47080048f0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 shutdown_connections 2024-09-21T05:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f470007af80 0x7f470007d440 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47181033d0 0x7f471819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 --2- 172.21.15.7:0/497935034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718102ab0 0x7f471819f7e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 >> 172.21.15.7:0/497935034 conn(0x7f47180fc550 msgr2=0x7f4718100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 shutdown_connections 2024-09-21T05:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.244+0000 7f471cbcc640 1 -- 172.21.15.7:0/497935034 wait complete. 2024-09-21T05:07:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.329+0000 7ff422d09640 1 -- 172.21.15.7:0/474775288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c102b20 msgr2=0x7ff41c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.329+0000 7ff422d09640 1 --2- 172.21.15.7:0/474775288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c102b20 0x7ff41c102f00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0098b0 tx=0x7ff40c02a560 comp rx=0 tx=0).stop 2024-09-21T05:07:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.329+0000 7ff422d09640 1 -- 172.21.15.7:0/474775288 shutdown_connections 2024-09-21T05:07:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.329+0000 7ff422d09640 1 --2- 172.21.15.7:0/474775288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c103440 0x7ff41c10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.329+0000 7ff422d09640 1 --2- 172.21.15.7:0/474775288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c102b20 0x7ff41c102f00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.329+0000 7ff422d09640 1 -- 172.21.15.7:0/474775288 >> 172.21.15.7:0/474775288 conn(0x7ff41c0fc520 msgr2=0x7ff41c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.329+0000 7ff422d09640 1 -- 172.21.15.7:0/474775288 shutdown_connections 2024-09-21T05:07:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.329+0000 7ff422d09640 1 -- 172.21.15.7:0/474775288 wait complete. 2024-09-21T05:07:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff422d09640 1 Processor -- start 2024-09-21T05:07:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff422d09640 1 -- start start 2024-09-21T05:07:44.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff422d09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c102b20 0x7ff41c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff422d09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c103440 0x7ff41c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff422d09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c10edf0 con 0x7ff41c103440 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff422d09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c10ec70 con 0x7ff41c102b20 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff41bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c103440 0x7ff41c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff41bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c103440 0x7ff41c19fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54058/0 (socket says 172.21.15.7:54058) 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.330+0000 7ff41bfff640 1 -- 172.21.15.7:0/4135726574 learned_addr learned my addr 172.21.15.7:0/4135726574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff420aa5640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c102b20 0x7ff41c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff41bfff640 1 -- 172.21.15.7:0/4135726574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c102b20 msgr2=0x7ff41c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff41bfff640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c102b20 0x7ff41c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff41bfff640 1 -- 172.21.15.7:0/4135726574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c199a50 con 0x7ff41c103440 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff41bfff640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c103440 0x7ff41c19fd80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff41000d6f0 tx=0x7ff41000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff419ffb640 1 -- 172.21.15.7:0/4135726574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff410017070 con 0x7ff41c103440 2024-09-21T05:07:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff419ffb640 1 -- 172.21.15.7:0/4135726574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff41000de00 con 0x7ff41c103440 2024-09-21T05:07:44.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c199d40 con 0x7ff41c103440 2024-09-21T05:07:44.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff419ffb640 1 -- 172.21.15.7:0/4135726574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff410004bb0 con 0x7ff41c103440 2024-09-21T05:07:44.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.331+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c19a2d0 con 0x7ff41c103440 2024-09-21T05:07:44.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.332+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c06aa30 con 0x7ff41c103440 2024-09-21T05:07:44.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.333+0000 7ff419ffb640 1 -- 172.21.15.7:0/4135726574 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff410009a30 con 0x7ff41c103440 2024-09-21T05:07:44.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.333+0000 7ff419ffb640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff40407af80 0x7ff40407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.334+0000 7ff419ffb640 1 -- 172.21.15.7:0/4135726574 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff41009db20 con 0x7ff41c103440 2024-09-21T05:07:44.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.334+0000 7ff420aa5640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff40407af80 0x7ff40407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.334+0000 7ff420aa5640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff40407af80 0x7ff40407d440 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009230 tx=0x7ff40c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.338+0000 7ff419ffb640 1 -- 172.21.15.7:0/4135726574 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff410069140 con 0x7ff41c103440 2024-09-21T05:07:44.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.534+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff41c19af30 con 0x7ff40407af80 2024-09-21T05:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.544+0000 7ff419ffb640 1 -- 172.21.15.7:0/4135726574 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7ff41c19af30 con 0x7ff40407af80 2024-09-21T05:07:44.548 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (29m) 8m ago 40m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (32m) 8m ago 40m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (32m) 8m ago 40m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (27m) 8m ago 40m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (27m) 8m ago 39m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (29m) 8m ago 40m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (23m) 8m ago 35m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (24m) 8m ago 35m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (8m) 8m ago 35m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:07:44.548 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (23m) 8m ago 35m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (31m) 8m ago 42m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (30m) 8m ago 39m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (28m) 8m ago 42m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (28m) 8m ago 39m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (30m) 8m ago 40m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (30m) 8m ago 39m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (27m) 8m ago 38m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (26m) 8m ago 38m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (26m) 8m ago 37m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (25m) 8m ago 37m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (25m) 8m ago 36m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (24m) 8m ago 36m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:07:44.549 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (30m) 8m ago 40m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff40407af80 msgr2=0x7ff40407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff40407af80 0x7ff40407d440 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009230 tx=0x7ff40c031400 comp rx=0 tx=0).stop 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c103440 msgr2=0x7ff41c19fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c103440 0x7ff41c19fd80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff41000d6f0 tx=0x7ff41000dbb0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 shutdown_connections 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff40407af80 0x7ff40407d440 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff41c103440 0x7ff41c19fd80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 --2- 172.21.15.7:0/4135726574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff41c102b20 0x7ff41c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 >> 172.21.15.7:0/4135726574 conn(0x7ff41c0fc520 msgr2=0x7ff41c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 shutdown_connections 2024-09-21T05:07:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.547+0000 7ff422d09640 1 -- 172.21.15.7:0/4135726574 wait complete. 2024-09-21T05:07:44.594 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:07:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.630+0000 7f0a1f419640 1 -- 172.21.15.7:0/2184253162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 msgr2=0x7f0a18107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.630+0000 7f0a1f419640 1 --2- 172.21.15.7:0/2184253162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 0x7f0a18107ce0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009880 tx=0x7f0a1402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.630+0000 7f0a1f419640 1 -- 172.21.15.7:0/2184253162 shutdown_connections 2024-09-21T05:07:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.630+0000 7f0a1f419640 1 --2- 172.21.15.7:0/2184253162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a18101900 0x7f0a18101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.630+0000 7f0a1f419640 1 --2- 172.21.15.7:0/2184253162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 0x7f0a18107ce0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.630+0000 7f0a1f419640 1 -- 172.21.15.7:0/2184253162 >> 172.21.15.7:0/2184253162 conn(0x7f0a180fd600 msgr2=0x7f0a180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.630+0000 7f0a1f419640 1 -- 172.21.15.7:0/2184253162 shutdown_connections 2024-09-21T05:07:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.630+0000 7f0a1f419640 1 -- 172.21.15.7:0/2184253162 wait complete. 2024-09-21T05:07:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1f419640 1 Processor -- start 2024-09-21T05:07:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1f419640 1 -- start start 2024-09-21T05:07:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1f419640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a18101900 0x7f0a1819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1f419640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 0x7f0a1819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1f419640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810edb0 con 0x7f0a18107900 2024-09-21T05:07:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1f419640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810ec30 con 0x7f0a18101900 2024-09-21T05:07:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1c9b4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 0x7f0a1819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1c9b4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 0x7f0a1819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54084/0 (socket says 172.21.15.7:54084) 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1c9b4640 1 -- 172.21.15.7:0/1395983766 learned_addr learned my addr 172.21.15.7:0/1395983766 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1d1b5640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a18101900 0x7f0a1819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1c9b4640 1 -- 172.21.15.7:0/1395983766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a18101900 msgr2=0x7f0a1819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1c9b4640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a18101900 0x7f0a1819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.631+0000 7f0a1c9b4640 1 -- 172.21.15.7:0/1395983766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a181977c0 con 0x7f0a18107900 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.632+0000 7f0a1c9b4640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 0x7f0a1819db80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00c8a0 tx=0x7f0a0c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.632+0000 7f0a0a7fc640 1 -- 172.21.15.7:0/1395983766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0a0c0079b0 con 0x7f0a18107900 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.632+0000 7f0a0a7fc640 1 -- 172.21.15.7:0/1395983766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0a0c007b50 con 0x7f0a18107900 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.632+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a18197ab0 con 0x7f0a18107900 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.632+0000 7f0a0a7fc640 1 -- 172.21.15.7:0/1395983766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0a0c018700 con 0x7f0a18107900 2024-09-21T05:07:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.632+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a18198040 con 0x7f0a18107900 2024-09-21T05:07:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.633+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a18103040 con 0x7f0a18107900 2024-09-21T05:07:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.634+0000 7f0a0a7fc640 1 -- 172.21.15.7:0/1395983766 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0a0c007d00 con 0x7f0a18107900 2024-09-21T05:07:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.634+0000 7f0a0a7fc640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0407af80 0x7f0a0407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.634+0000 7f0a0a7fc640 1 -- 172.21.15.7:0/1395983766 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0a0c09e2b0 con 0x7f0a18107900 2024-09-21T05:07:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.634+0000 7f0a1d1b5640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0407af80 0x7f0a0407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.635+0000 7f0a1d1b5640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0407af80 0x7f0a0407d440 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0a14031460 tx=0x7f0a14005bd0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:44.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.639+0000 7f0a0a7fc640 1 -- 172.21.15.7:0/1395983766 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a0c014030 con 0x7f0a18107900 2024-09-21T05:07:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.903+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f0a18198d20 con 0x7f0a18107900 2024-09-21T05:07:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.904+0000 7f0a0a7fc640 1 -- 172.21.15.7:0/1395983766 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f0a18198d20 con 0x7f0a18107900 2024-09-21T05:07:44.908 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:07:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:07:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:07:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:07:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:07:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:07:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:07:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:07:44.910 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:07:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0407af80 msgr2=0x7f0a0407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0407af80 0x7f0a0407d440 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0a14031460 tx=0x7f0a14005bd0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 msgr2=0x7f0a1819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 0x7f0a1819db80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00c8a0 tx=0x7f0a0c00cd60 comp rx=0 tx=0).stop 2024-09-21T05:07:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 shutdown_connections 2024-09-21T05:07:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0a0407af80 0x7f0a0407d440 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a18107900 0x7f0a1819db80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 --2- 172.21.15.7:0/1395983766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a18101900 0x7f0a1819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 >> 172.21.15.7:0/1395983766 conn(0x7f0a180fd600 msgr2=0x7f0a180ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.906+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 shutdown_connections 2024-09-21T05:07:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.907+0000 7f0a1f419640 1 -- 172.21.15.7:0/1395983766 wait complete. 2024-09-21T05:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.990+0000 7fa357b11640 1 -- 172.21.15.7:0/3337093071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350107900 msgr2=0x7fa350107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.990+0000 7fa357b11640 1 --2- 172.21.15.7:0/3337093071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350107900 0x7fa350107ce0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009880 tx=0x7fa34c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.990+0000 7fa357b11640 1 -- 172.21.15.7:0/3337093071 shutdown_connections 2024-09-21T05:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.990+0000 7fa357b11640 1 --2- 172.21.15.7:0/3337093071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa350101900 0x7fa350101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.990+0000 7fa357b11640 1 --2- 172.21.15.7:0/3337093071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350107900 0x7fa350107ce0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.990+0000 7fa357b11640 1 -- 172.21.15.7:0/3337093071 >> 172.21.15.7:0/3337093071 conn(0x7fa3500fd600 msgr2=0x7fa3500ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.990+0000 7fa357b11640 1 -- 172.21.15.7:0/3337093071 shutdown_connections 2024-09-21T05:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.990+0000 7fa357b11640 1 -- 172.21.15.7:0/3337093071 wait complete. 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.991+0000 7fa357b11640 1 Processor -- start 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.991+0000 7fa357b11640 1 -- start start 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.991+0000 7fa357b11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350101900 0x7fa35019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.991+0000 7fa357b11640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa350107900 0x7fa35019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.991+0000 7fa357b11640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa35010ee30 con 0x7fa350101900 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.991+0000 7fa357b11640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa35010ecb0 con 0x7fa350107900 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa3558ad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350101900 0x7fa35019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa3558ad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350101900 0x7fa35019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54092/0 (socket says 172.21.15.7:54092) 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa3558ad640 1 -- 172.21.15.7:0/1757781174 learned_addr learned my addr 172.21.15.7:0/1757781174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa3550ac640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa350107900 0x7fa35019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa3558ad640 1 -- 172.21.15.7:0/1757781174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa350107900 msgr2=0x7fa35019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:44.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa3558ad640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa350107900 0x7fa35019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:44.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa3558ad640 1 -- 172.21.15.7:0/1757781174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa350197860 con 0x7fa350101900 2024-09-21T05:07:44.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa3558ad640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350101900 0x7fa35019d6a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa34c031460 tx=0x7fa34c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:44.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa346ffd640 1 -- 172.21.15.7:0/1757781174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa34c03f070 con 0x7fa350101900 2024-09-21T05:07:44.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa350197af0 con 0x7fa350101900 2024-09-21T05:07:44.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa346ffd640 1 -- 172.21.15.7:0/1757781174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa34c004cf0 con 0x7fa350101900 2024-09-21T05:07:44.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa346ffd640 1 -- 172.21.15.7:0/1757781174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa34c035030 con 0x7fa350101900 2024-09-21T05:07:44.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.992+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa350198020 con 0x7fa350101900 2024-09-21T05:07:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.993+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa350103040 con 0x7fa350101900 2024-09-21T05:07:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.994+0000 7fa346ffd640 1 -- 172.21.15.7:0/1757781174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa34c042070 con 0x7fa350101900 2024-09-21T05:07:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.995+0000 7fa346ffd640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa33c07af30 0x7fa33c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.995+0000 7fa346ffd640 1 -- 172.21.15.7:0/1757781174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fa34c0beb90 con 0x7fa350101900 2024-09-21T05:07:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.995+0000 7fa3550ac640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa33c07af30 0x7fa33c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:44.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.995+0000 7fa3550ac640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa33c07af30 0x7fa33c07d3f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa3500ffc20 tx=0x7fa340009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:45.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:44.999+0000 7fa346ffd640 1 -- 172.21.15.7:0/1757781174 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa34c08a090 con 0x7fa350101900 2024-09-21T05:07:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.223+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa350198d90 con 0x7fa350101900 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.224+0000 7fa346ffd640 1 -- 172.21.15.7:0/1757781174 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7fa34c08ef40 con 0x7fa350101900 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:07:45.228 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:07:45.229 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:07:45.230 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:07:45.231 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:07:45.231 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:07:45.231 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:07:45.231 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:07:45.231 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:07:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa33c07af30 msgr2=0x7fa33c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa33c07af30 0x7fa33c07d3f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa3500ffc20 tx=0x7fa340009110 comp rx=0 tx=0).stop 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350101900 msgr2=0x7fa35019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350101900 0x7fa35019d6a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa34c031460 tx=0x7fa34c0048f0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 shutdown_connections 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa33c07af30 0x7fa33c07d3f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa350107900 0x7fa35019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 --2- 172.21.15.7:0/1757781174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa350101900 0x7fa35019d6a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 >> 172.21.15.7:0/1757781174 conn(0x7fa3500fd600 msgr2=0x7fa3500fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 shutdown_connections 2024-09-21T05:07:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.226+0000 7fa357b11640 1 -- 172.21.15.7:0/1757781174 wait complete. 2024-09-21T05:07:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.314+0000 7fa0f2f09640 1 -- 172.21.15.7:0/3205725554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 msgr2=0x7fa0ec100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.314+0000 7fa0f2f09640 1 --2- 172.21.15.7:0/3205725554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 0x7fa0ec100c60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009c20 tx=0x7fa0dc02a970 comp rx=0 tx=0).stop 2024-09-21T05:07:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.314+0000 7fa0f2f09640 1 -- 172.21.15.7:0/3205725554 shutdown_connections 2024-09-21T05:07:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.314+0000 7fa0f2f09640 1 --2- 172.21.15.7:0/3205725554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 0x7fa0ec100c60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.314+0000 7fa0f2f09640 1 --2- 172.21.15.7:0/3205725554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec106800 0x7fa0ec106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.314+0000 7fa0f2f09640 1 -- 172.21.15.7:0/3205725554 >> 172.21.15.7:0/3205725554 conn(0x7fa0ec0fc520 msgr2=0x7fa0ec0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.314+0000 7fa0f2f09640 1 -- 172.21.15.7:0/3205725554 shutdown_connections 2024-09-21T05:07:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.314+0000 7fa0f2f09640 1 -- 172.21.15.7:0/3205725554 wait complete. 2024-09-21T05:07:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f2f09640 1 Processor -- start 2024-09-21T05:07:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f2f09640 1 -- start start 2024-09-21T05:07:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f2f09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 0x7fa0ec1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f2f09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec106800 0x7fa0ec102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:45.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f2f09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec10dd30 con 0x7fa0ec100800 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f2f09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec10dbb0 con 0x7fa0ec106800 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f0ca5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 0x7fa0ec1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f0ca5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 0x7fa0ec1023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54116/0 (socket says 172.21.15.7:54116) 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f0ca5640 1 -- 172.21.15.7:0/2167533069 learned_addr learned my addr 172.21.15.7:0/2167533069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0ebfff640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec106800 0x7fa0ec102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f0ca5640 1 -- 172.21.15.7:0/2167533069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec106800 msgr2=0x7fa0ec102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.315+0000 7fa0f0ca5640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec106800 0x7fa0ec102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.316+0000 7fa0f0ca5640 1 -- 172.21.15.7:0/2167533069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec102e50 con 0x7fa0ec100800 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.316+0000 7fa0f0ca5640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 0x7fa0ec1023d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400b420 tx=0x7fa0e400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.316+0000 7fa0e9ffb640 1 -- 172.21.15.7:0/2167533069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0e4011630 con 0x7fa0ec100800 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.316+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec103140 con 0x7fa0ec100800 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.316+0000 7fa0e9ffb640 1 -- 172.21.15.7:0/2167533069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa0e4016070 con 0x7fa0ec100800 2024-09-21T05:07:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.316+0000 7fa0e9ffb640 1 -- 172.21.15.7:0/2167533069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0e4010d40 con 0x7fa0ec100800 2024-09-21T05:07:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.316+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec1a3f30 con 0x7fa0ec100800 2024-09-21T05:07:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.317+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec101f40 con 0x7fa0ec100800 2024-09-21T05:07:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.318+0000 7fa0e9ffb640 1 -- 172.21.15.7:0/2167533069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa0e401b460 con 0x7fa0ec100800 2024-09-21T05:07:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.318+0000 7fa0e9ffb640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa0d807af30 0x7fa0d807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.319+0000 7fa0ebfff640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa0d807af30 0x7fa0d807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.319+0000 7fa0e9ffb640 1 -- 172.21.15.7:0/2167533069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fa0e409d810 con 0x7fa0ec100800 2024-09-21T05:07:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.319+0000 7fa0ebfff640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa0d807af30 0x7fa0d807d3f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009430 tx=0x7fa0dc0093a0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:45.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.323+0000 7fa0e9ffb640 1 -- 172.21.15.7:0/2167533069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa0e4021250 con 0x7fa0ec100800 2024-09-21T05:07:45.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:45 smithi007 ceph-mon[108238]: pgmap v990: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:45.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:45 smithi007 ceph-mon[108238]: from='client.35600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:45.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:45 smithi007 ceph-mon[108238]: from='client.35604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:45.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:45 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1395983766' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:07:45.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:45 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1757781174' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:07:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.530+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa0ec0fcd00 con 0x7fa0d807af30 2024-09-21T05:07:45.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.531+0000 7fa0e9ffb640 1 -- 172.21.15.7:0/2167533069 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fa0ec0fcd00 con 0x7fa0d807af30 2024-09-21T05:07:45.535 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:07:45.535 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:07:45.536 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:07:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa0d807af30 msgr2=0x7fa0d807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa0d807af30 0x7fa0d807d3f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009430 tx=0x7fa0dc0093a0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 msgr2=0x7fa0ec1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 0x7fa0ec1023d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400b420 tx=0x7fa0e400b8e0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 shutdown_connections 2024-09-21T05:07:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa0d807af30 0x7fa0d807d3f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0ec106800 0x7fa0ec102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 --2- 172.21.15.7:0/2167533069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0ec100800 0x7fa0ec1023d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 >> 172.21.15.7:0/2167533069 conn(0x7fa0ec0fc520 msgr2=0x7fa0ec104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 shutdown_connections 2024-09-21T05:07:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.534+0000 7fa0f2f09640 1 -- 172.21.15.7:0/2167533069 wait complete. 2024-09-21T05:07:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 -- 172.21.15.7:0/494537687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 msgr2=0x7fb638106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 --2- 172.21.15.7:0/494537687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 0x7fb638106be0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb628009880 tx=0x7fb62802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 -- 172.21.15.7:0/494537687 shutdown_connections 2024-09-21T05:07:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 --2- 172.21.15.7:0/494537687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb638100800 0x7fb638100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 --2- 172.21.15.7:0/494537687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 0x7fb638106be0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 -- 172.21.15.7:0/494537687 >> 172.21.15.7:0/494537687 conn(0x7fb6380fc520 msgr2=0x7fb6380fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 -- 172.21.15.7:0/494537687 shutdown_connections 2024-09-21T05:07:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 -- 172.21.15.7:0/494537687 wait complete. 2024-09-21T05:07:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.623+0000 7fb63d808640 1 Processor -- start 2024-09-21T05:07:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb63d808640 1 -- start start 2024-09-21T05:07:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb63d808640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb638100800 0x7fb63819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb63d808640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 0x7fb63819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb63d808640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb63810dcb0 con 0x7fb638106800 2024-09-21T05:07:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb63d808640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb63810db30 con 0x7fb638100800 2024-09-21T05:07:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb6367fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 0x7fb63819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb6367fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 0x7fb63819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54134/0 (socket says 172.21.15.7:54134) 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb6367fc640 1 -- 172.21.15.7:0/3746815624 learned_addr learned my addr 172.21.15.7:0/3746815624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb636ffd640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb638100800 0x7fb63819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb6367fc640 1 -- 172.21.15.7:0/3746815624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb638100800 msgr2=0x7fb63819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb6367fc640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb638100800 0x7fb63819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb6367fc640 1 -- 172.21.15.7:0/3746815624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6381977c0 con 0x7fb638106800 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.624+0000 7fb6367fc640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 0x7fb63819db80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d8d0 tx=0x7fb62c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.625+0000 7fb63c806640 1 -- 172.21.15.7:0/3746815624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb62c017030 con 0x7fb638106800 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.625+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb638197ab0 con 0x7fb638106800 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.625+0000 7fb63c806640 1 -- 172.21.15.7:0/3746815624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb62c004020 con 0x7fb638106800 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.625+0000 7fb63c806640 1 -- 172.21.15.7:0/3746815624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb62c0047a0 con 0x7fb638106800 2024-09-21T05:07:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.625+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb638198040 con 0x7fb638106800 2024-09-21T05:07:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.626+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb638101f40 con 0x7fb638106800 2024-09-21T05:07:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.626+0000 7fb63c806640 1 -- 172.21.15.7:0/3746815624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb62c0041d0 con 0x7fb638106800 2024-09-21T05:07:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.627+0000 7fb63c806640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb62007af30 0x7fb62007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:07:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.627+0000 7fb63c806640 1 -- 172.21.15.7:0/3746815624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fb62c018070 con 0x7fb638106800 2024-09-21T05:07:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.627+0000 7fb636ffd640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb62007af30 0x7fb62007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:07:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.627+0000 7fb636ffd640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb62007af30 0x7fb62007d3f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb6280096c0 tx=0x7fb628031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:07:45.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.631+0000 7fb63c806640 1 -- 172.21.15.7:0/3746815624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb62c069420 con 0x7fb638106800 2024-09-21T05:07:45.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:45 smithi053 ceph-mon[80176]: pgmap v990: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:45.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:45 smithi053 ceph-mon[80176]: from='client.35600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:45.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:45 smithi053 ceph-mon[80176]: from='client.35604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:45.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:45 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1395983766' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:07:45.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:45 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1757781174' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:07:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.959+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fb638198350 con 0x7fb638106800 2024-09-21T05:07:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.959+0000 7fb63c806640 1 -- 172.21.15.7:0/3746815624 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fb62c06e2d0 con 0x7fb638106800 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:07:45.964 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:07:45.965 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:07:45.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb62007af30 msgr2=0x7fb62007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb62007af30 0x7fb62007d3f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb6280096c0 tx=0x7fb628031400 comp rx=0 tx=0).stop 2024-09-21T05:07:45.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 msgr2=0x7fb63819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:07:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 0x7fb63819db80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d8d0 tx=0x7fb62c00dd90 comp rx=0 tx=0).stop 2024-09-21T05:07:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 shutdown_connections 2024-09-21T05:07:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb62007af30 0x7fb62007d3f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb638106800 0x7fb63819db80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 --2- 172.21.15.7:0/3746815624 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb638100800 0x7fb63819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:07:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 >> 172.21.15.7:0/3746815624 conn(0x7fb6380fc520 msgr2=0x7fb63806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:07:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 shutdown_connections 2024-09-21T05:07:45.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:07:45.962+0000 7fb63d808640 1 -- 172.21.15.7:0/3746815624 wait complete. 2024-09-21T05:07:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:46 smithi053 ceph-mon[80176]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:46.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:46 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3746815624' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:07:46.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:46 smithi007 ceph-mon[108238]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:46.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:46 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3746815624' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:07:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:47 smithi053 ceph-mon[80176]: pgmap v991: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:47.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:47 smithi053 ceph-mon[80176]: from='client.35620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:47 smithi007 ceph-mon[108238]: pgmap v991: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:47 smithi007 ceph-mon[108238]: from='client.35620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:07:49.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:49 smithi053 ceph-mon[80176]: pgmap v992: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:49.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:49 smithi007 ceph-mon[108238]: pgmap v992: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:51.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:51 smithi053 ceph-mon[80176]: pgmap v993: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:51 smithi007 ceph-mon[108238]: pgmap v993: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:53.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:53 smithi053 ceph-mon[80176]: pgmap v994: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:53 smithi007 ceph-mon[108238]: pgmap v994: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:55.652 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:55 smithi007 ceph-mon[108238]: pgmap v995: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:55.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:55 smithi053 ceph-mon[80176]: pgmap v995: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:57.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:57 smithi053 ceph-mon[80176]: pgmap v996: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:57.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:57 smithi007 ceph-mon[108238]: pgmap v996: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:59 smithi053 ceph-mon[80176]: pgmap v997: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:07:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:07:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:59 smithi007 ceph-mon[108238]: pgmap v997: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:07:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:07:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:08:01.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:01 smithi053 ceph-mon[80176]: pgmap v998: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:01 smithi007 ceph-mon[108238]: pgmap v998: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:03.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:03 smithi053 ceph-mon[80176]: pgmap v999: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:03.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:03 smithi007 ceph-mon[108238]: pgmap v999: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:05.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:05 smithi007 ceph-mon[108238]: pgmap v1000: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:05.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:05 smithi053 ceph-mon[80176]: pgmap v1000: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:07.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:07 smithi053 ceph-mon[80176]: pgmap v1001: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:07.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:07 smithi007 ceph-mon[108238]: pgmap v1001: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:09.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:09 smithi053 ceph-mon[80176]: pgmap v1002: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:09.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:09 smithi007 ceph-mon[108238]: pgmap v1002: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:11.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:11 smithi053 ceph-mon[80176]: pgmap v1003: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:11.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:11 smithi007 ceph-mon[108238]: pgmap v1003: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:13 smithi053 ceph-mon[80176]: pgmap v1004: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:13.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:13 smithi007 ceph-mon[108238]: pgmap v1004: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:14.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:08:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:08:15.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:15 smithi053 ceph-mon[80176]: pgmap v1005: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:15.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:15 smithi007 ceph-mon[108238]: pgmap v1005: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:16.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.058+0000 7fabf1917640 1 -- 172.21.15.7:0/3418874911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 msgr2=0x7fabec10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.058+0000 7fabf1917640 1 --2- 172.21.15.7:0/3418874911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 0x7fabec10b8d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009880 tx=0x7fabe002a530 comp rx=0 tx=0).stop 2024-09-21T05:08:16.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.058+0000 7fabf1917640 1 -- 172.21.15.7:0/3418874911 shutdown_connections 2024-09-21T05:08:16.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.058+0000 7fabf1917640 1 --2- 172.21.15.7:0/3418874911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 0x7fabec10b8d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.058+0000 7fabf1917640 1 --2- 172.21.15.7:0/3418874911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabec102ac0 0x7fabec102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.058+0000 7fabf1917640 1 -- 172.21.15.7:0/3418874911 >> 172.21.15.7:0/3418874911 conn(0x7fabec0fc520 msgr2=0x7fabec0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.058+0000 7fabf1917640 1 -- 172.21.15.7:0/3418874911 shutdown_connections 2024-09-21T05:08:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.058+0000 7fabf1917640 1 -- 172.21.15.7:0/3418874911 wait complete. 2024-09-21T05:08:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabf1917640 1 Processor -- start 2024-09-21T05:08:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabf1917640 1 -- start start 2024-09-21T05:08:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabf1917640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabec102ac0 0x7fabec100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabf1917640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 0x7fabec100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabf1917640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabec10edd0 con 0x7fabec1033e0 2024-09-21T05:08:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabf1917640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fabec10ec50 con 0x7fabec102ac0 2024-09-21T05:08:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabea7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 0x7fabec100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabea7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 0x7fabec100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59774/0 (socket says 172.21.15.7:59774) 2024-09-21T05:08:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.059+0000 7fabea7fc640 1 -- 172.21.15.7:0/3568849940 learned_addr learned my addr 172.21.15.7:0/3568849940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabeaffd640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabec102ac0 0x7fabec100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabea7fc640 1 -- 172.21.15.7:0/3568849940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabec102ac0 msgr2=0x7fabec100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabea7fc640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabec102ac0 0x7fabec100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabea7fc640 1 -- 172.21.15.7:0/3568849940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabec1011a0 con 0x7fabec1033e0 2024-09-21T05:08:16.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabea7fc640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 0x7fabec100c60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fabe0031460 tx=0x7fabe0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:16.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabf0915640 1 -- 172.21.15.7:0/3568849940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fabe003f070 con 0x7fabec1033e0 2024-09-21T05:08:16.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabf0915640 1 -- 172.21.15.7:0/3568849940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fabe0004a90 con 0x7fabec1033e0 2024-09-21T05:08:16.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabec101430 con 0x7fabec1033e0 2024-09-21T05:08:16.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabf0915640 1 -- 172.21.15.7:0/3568849940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fabe0035030 con 0x7fabec1033e0 2024-09-21T05:08:16.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.060+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabec1a3da0 con 0x7fabec1033e0 2024-09-21T05:08:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.062+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabec06aa90 con 0x7fabec1033e0 2024-09-21T05:08:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.062+0000 7fabf0915640 1 -- 172.21.15.7:0/3568849940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fabe0042070 con 0x7fabec1033e0 2024-09-21T05:08:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.063+0000 7fabf0915640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabd407af40 0x7fabd407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.063+0000 7fabf0915640 1 -- 172.21.15.7:0/3568849940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fabe00beb40 con 0x7fabec1033e0 2024-09-21T05:08:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.063+0000 7fabeaffd640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabd407af40 0x7fabd407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.063+0000 7fabeaffd640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabd407af40 0x7fabd407d400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fabdc005fd0 tx=0x7fabdc005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.067+0000 7fabf0915640 1 -- 172.21.15.7:0/3568849940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fabe008a0e0 con 0x7fabec1033e0 2024-09-21T05:08:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.273+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fabec062630 con 0x7fabd407af40 2024-09-21T05:08:16.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.275+0000 7fabf0915640 1 -- 172.21.15.7:0/3568849940 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fabec062630 con 0x7fabd407af40 2024-09-21T05:08:16.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabd407af40 msgr2=0x7fabd407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabd407af40 0x7fabd407d400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fabdc005fd0 tx=0x7fabdc005850 comp rx=0 tx=0).stop 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 msgr2=0x7fabec100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 0x7fabec100c60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fabe0031460 tx=0x7fabe0004530 comp rx=0 tx=0).stop 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 shutdown_connections 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fabd407af40 0x7fabd407d400 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabec1033e0 0x7fabec100c60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 --2- 172.21.15.7:0/3568849940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabec102ac0 0x7fabec100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 >> 172.21.15.7:0/3568849940 conn(0x7fabec0fc520 msgr2=0x7fabec108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 shutdown_connections 2024-09-21T05:08:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.277+0000 7fabf1917640 1 -- 172.21.15.7:0/3568849940 wait complete. 2024-09-21T05:08:16.289 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:08:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.363+0000 7ffa72c94640 1 -- 172.21.15.7:0/3016530743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c107900 msgr2=0x7ffa6c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.363+0000 7ffa72c94640 1 --2- 172.21.15.7:0/3016530743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c107900 0x7ffa6c107ce0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009880 tx=0x7ffa6002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.363+0000 7ffa72c94640 1 -- 172.21.15.7:0/3016530743 shutdown_connections 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.363+0000 7ffa72c94640 1 --2- 172.21.15.7:0/3016530743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa6c101900 0x7ffa6c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.363+0000 7ffa72c94640 1 --2- 172.21.15.7:0/3016530743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c107900 0x7ffa6c107ce0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.363+0000 7ffa72c94640 1 -- 172.21.15.7:0/3016530743 >> 172.21.15.7:0/3016530743 conn(0x7ffa6c0fd600 msgr2=0x7ffa6c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.363+0000 7ffa72c94640 1 -- 172.21.15.7:0/3016530743 shutdown_connections 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.363+0000 7ffa72c94640 1 -- 172.21.15.7:0/3016530743 wait complete. 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa72c94640 1 Processor -- start 2024-09-21T05:08:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa72c94640 1 -- start start 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa72c94640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c101900 0x7ffa6c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa72c94640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa6c107900 0x7ffa6c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa72c94640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c10edb0 con 0x7ffa6c101900 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa72c94640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c10ec30 con 0x7ffa6c107900 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa70a30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c101900 0x7ffa6c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa70a30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c101900 0x7ffa6c19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59786/0 (socket says 172.21.15.7:59786) 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa70a30640 1 -- 172.21.15.7:0/3525434009 learned_addr learned my addr 172.21.15.7:0/3525434009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa6bfff640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa6c107900 0x7ffa6c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa70a30640 1 -- 172.21.15.7:0/3525434009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa6c107900 msgr2=0x7ffa6c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa70a30640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa6c107900 0x7ffa6c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.364+0000 7ffa70a30640 1 -- 172.21.15.7:0/3525434009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c1977e0 con 0x7ffa6c101900 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.365+0000 7ffa70a30640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c101900 0x7ffa6c19d660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffa60031460 tx=0x7ffa600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.365+0000 7ffa69ffb640 1 -- 172.21.15.7:0/3525434009 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffa6003f070 con 0x7ffa6c101900 2024-09-21T05:08:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.365+0000 7ffa69ffb640 1 -- 172.21.15.7:0/3525434009 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa60004cf0 con 0x7ffa6c101900 2024-09-21T05:08:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.365+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c197a70 con 0x7ffa6c101900 2024-09-21T05:08:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.365+0000 7ffa69ffb640 1 -- 172.21.15.7:0/3525434009 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffa60035030 con 0x7ffa6c101900 2024-09-21T05:08:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.365+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c197fa0 con 0x7ffa6c101900 2024-09-21T05:08:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.366+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6c103040 con 0x7ffa6c101900 2024-09-21T05:08:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.367+0000 7ffa69ffb640 1 -- 172.21.15.7:0/3525434009 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ffa60042070 con 0x7ffa6c101900 2024-09-21T05:08:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.367+0000 7ffa69ffb640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffa5c07af80 0x7ffa5c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.367+0000 7ffa69ffb640 1 -- 172.21.15.7:0/3525434009 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ffa600bea90 con 0x7ffa6c101900 2024-09-21T05:08:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.367+0000 7ffa6bfff640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffa5c07af80 0x7ffa5c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.368+0000 7ffa6bfff640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffa5c07af80 0x7ffa5c07d440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c100d10 tx=0x7ffa5800a4c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.372+0000 7ffa69ffb640 1 -- 172.21.15.7:0/3525434009 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffa6008a0b0 con 0x7ffa6c101900 2024-09-21T05:08:16.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.575+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ffa6c198250 con 0x7ffa5c07af80 2024-09-21T05:08:16.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.577+0000 7ffa69ffb640 1 -- 172.21.15.7:0/3525434009 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ffa6c198250 con 0x7ffa5c07af80 2024-09-21T05:08:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffa5c07af80 msgr2=0x7ffa5c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffa5c07af80 0x7ffa5c07d440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c100d10 tx=0x7ffa5800a4c0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c101900 msgr2=0x7ffa6c19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c101900 0x7ffa6c19d660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffa60031460 tx=0x7ffa600048f0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 shutdown_connections 2024-09-21T05:08:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffa5c07af80 0x7ffa5c07d440 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa6c107900 0x7ffa6c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 --2- 172.21.15.7:0/3525434009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa6c101900 0x7ffa6c19d660 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 >> 172.21.15.7:0/3525434009 conn(0x7ffa6c0fd600 msgr2=0x7ffa6c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 shutdown_connections 2024-09-21T05:08:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.579+0000 7ffa72c94640 1 -- 172.21.15.7:0/3525434009 wait complete. 2024-09-21T05:08:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.663+0000 7f928b450640 1 -- 172.21.15.7:0/4045930831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284100650 msgr2=0x7f9284100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.663+0000 7f928b450640 1 --2- 172.21.15.7:0/4045930831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284100650 0x7f9284100a30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9280009880 tx=0x7f928002a490 comp rx=0 tx=0).stop 2024-09-21T05:08:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.663+0000 7f928b450640 1 -- 172.21.15.7:0/4045930831 shutdown_connections 2024-09-21T05:08:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.663+0000 7f928b450640 1 --2- 172.21.15.7:0/4045930831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9284101000 0x7f928410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.663+0000 7f928b450640 1 --2- 172.21.15.7:0/4045930831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284100650 0x7f9284100a30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.663+0000 7f928b450640 1 -- 172.21.15.7:0/4045930831 >> 172.21.15.7:0/4045930831 conn(0x7f92840fc520 msgr2=0x7f92840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.664+0000 7f928b450640 1 -- 172.21.15.7:0/4045930831 shutdown_connections 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.664+0000 7f928b450640 1 -- 172.21.15.7:0/4045930831 wait complete. 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.664+0000 7f928b450640 1 Processor -- start 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.664+0000 7f928b450640 1 -- start start 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.664+0000 7f928b450640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9284100650 0x7f928419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.664+0000 7f928b450640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284101000 0x7f928419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.664+0000 7f928b450640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9284110fb0 con 0x7f9284101000 2024-09-21T05:08:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.664+0000 7f928b450640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9284110e30 con 0x7f9284100650 2024-09-21T05:08:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f92889eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284101000 0x7f928419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f92889eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284101000 0x7f928419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59804/0 (socket says 172.21.15.7:59804) 2024-09-21T05:08:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f92889eb640 1 -- 172.21.15.7:0/2248175670 learned_addr learned my addr 172.21.15.7:0/2248175670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f92891ec640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9284100650 0x7f928419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f92889eb640 1 -- 172.21.15.7:0/2248175670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9284100650 msgr2=0x7f928419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f92889eb640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9284100650 0x7f928419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f92889eb640 1 -- 172.21.15.7:0/2248175670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9284199a40 con 0x7f9284101000 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f92889eb640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284101000 0x7f928419fdc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f927400d6f0 tx=0x7f927400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f927a7fc640 1 -- 172.21.15.7:0/2248175670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9274017070 con 0x7f9284101000 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9284199d30 con 0x7f9284101000 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f927a7fc640 1 -- 172.21.15.7:0/2248175670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f927400de00 con 0x7f9284101000 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f927a7fc640 1 -- 172.21.15.7:0/2248175670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9274004bb0 con 0x7f9284101000 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.665+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928419a2c0 con 0x7f9284101000 2024-09-21T05:08:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.666+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928406aa90 con 0x7f9284101000 2024-09-21T05:08:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.667+0000 7f927a7fc640 1 -- 172.21.15.7:0/2248175670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9274004e60 con 0x7f9284101000 2024-09-21T05:08:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.668+0000 7f927a7fc640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f927007af30 0x7f927007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.668+0000 7f927a7fc640 1 -- 172.21.15.7:0/2248175670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f927409dc00 con 0x7f9284101000 2024-09-21T05:08:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.668+0000 7f92891ec640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f927007af30 0x7f927007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.668+0000 7f92891ec640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f927007af30 0x7f927007d3f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9280000c00 tx=0x7f9280031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.672+0000 7f927a7fc640 1 -- 172.21.15.7:0/2248175670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92740a3020 con 0x7f9284101000 2024-09-21T05:08:16.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.869+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f928419af70 con 0x7f927007af30 2024-09-21T05:08:16.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.880+0000 7f927a7fc640 1 -- 172.21.15.7:0/2248175670 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f928419af70 con 0x7f927007af30 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (30m) 8m ago 41m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (33m) 8m ago 41m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (33m) 8m ago 40m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (28m) 8m ago 41m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (28m) 8m ago 40m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (29m) 8m ago 41m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (24m) 8m ago 35m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (24m) 8m ago 35m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (9m) 8m ago 35m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:08:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (24m) 8m ago 35m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (32m) 8m ago 42m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (31m) 8m ago 40m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (28m) 8m ago 42m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (28m) 8m ago 40m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (31m) 8m ago 41m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (31m) 8m ago 40m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (28m) 8m ago 39m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (27m) 8m ago 38m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (26m) 8m ago 38m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (26m) 8m ago 37m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (25m) 8m ago 37m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (24m) 8m ago 36m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:08:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (30m) 8m ago 40m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:08:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f927007af30 msgr2=0x7f927007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f927007af30 0x7f927007d3f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9280000c00 tx=0x7f9280031400 comp rx=0 tx=0).stop 2024-09-21T05:08:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284101000 msgr2=0x7f928419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284101000 0x7f928419fdc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f927400d6f0 tx=0x7f927400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 shutdown_connections 2024-09-21T05:08:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f927007af30 0x7f927007d3f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9284101000 0x7f928419fdc0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 --2- 172.21.15.7:0/2248175670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9284100650 0x7f928419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 >> 172.21.15.7:0/2248175670 conn(0x7f92840fc520 msgr2=0x7f92840fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 shutdown_connections 2024-09-21T05:08:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.883+0000 7f928b450640 1 -- 172.21.15.7:0/2248175670 wait complete. 2024-09-21T05:08:16.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.967+0000 7fdbf86cd640 1 -- 172.21.15.7:0/2451587137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0103460 msgr2=0x7fdbf010b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.967+0000 7fdbf86cd640 1 --2- 172.21.15.7:0/2451587137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0103460 0x7fdbf010b950 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009880 tx=0x7fdbe002a540 comp rx=0 tx=0).stop 2024-09-21T05:08:16.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.967+0000 7fdbf86cd640 1 -- 172.21.15.7:0/2451587137 shutdown_connections 2024-09-21T05:08:16.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.967+0000 7fdbf86cd640 1 --2- 172.21.15.7:0/2451587137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0103460 0x7fdbf010b950 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.967+0000 7fdbf86cd640 1 --2- 172.21.15.7:0/2451587137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf0102b40 0x7fdbf0102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.967+0000 7fdbf86cd640 1 -- 172.21.15.7:0/2451587137 >> 172.21.15.7:0/2451587137 conn(0x7fdbf00fc520 msgr2=0x7fdbf00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:16.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.967+0000 7fdbf86cd640 1 -- 172.21.15.7:0/2451587137 shutdown_connections 2024-09-21T05:08:16.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.967+0000 7fdbf86cd640 1 -- 172.21.15.7:0/2451587137 wait complete. 2024-09-21T05:08:16.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf86cd640 1 Processor -- start 2024-09-21T05:08:16.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf86cd640 1 -- start start 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf86cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0102b40 0x7fdbf01007a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf86cd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf0103460 0x7fdbf0100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf86cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf010edd0 con 0x7fdbf0102b40 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf86cd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf010ec50 con 0x7fdbf0103460 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf6469640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0102b40 0x7fdbf01007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf6469640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0102b40 0x7fdbf01007a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59822/0 (socket says 172.21.15.7:59822) 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf6469640 1 -- 172.21.15.7:0/846666091 learned_addr learned my addr 172.21.15.7:0/846666091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf5c68640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf0103460 0x7fdbf0100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf6469640 1 -- 172.21.15.7:0/846666091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf0103460 msgr2=0x7fdbf0100ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf6469640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf0103460 0x7fdbf0100ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.968+0000 7fdbf6469640 1 -- 172.21.15.7:0/846666091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf0101220 con 0x7fdbf0102b40 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.969+0000 7fdbf6469640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0102b40 0x7fdbf01007a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c8a0 tx=0x7fdbec00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.969+0000 7fdbe77fe640 1 -- 172.21.15.7:0/846666091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdbec0079b0 con 0x7fdbf0102b40 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.969+0000 7fdbe77fe640 1 -- 172.21.15.7:0/846666091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbec007b50 con 0x7fdbf0102b40 2024-09-21T05:08:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.969+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf0101510 con 0x7fdbf0102b40 2024-09-21T05:08:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.969+0000 7fdbe77fe640 1 -- 172.21.15.7:0/846666091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdbec018700 con 0x7fdbf0102b40 2024-09-21T05:08:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.969+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf01a3de0 con 0x7fdbf0102b40 2024-09-21T05:08:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.970+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf006aa10 con 0x7fdbf0102b40 2024-09-21T05:08:16.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.971+0000 7fdbe77fe640 1 -- 172.21.15.7:0/846666091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdbec007d00 con 0x7fdbf0102b40 2024-09-21T05:08:16.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.971+0000 7fdbe77fe640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbdc07af30 0x7fdbdc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:16.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.971+0000 7fdbe77fe640 1 -- 172.21.15.7:0/846666091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fdbec09e410 con 0x7fdbf0102b40 2024-09-21T05:08:16.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.971+0000 7fdbf5c68640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbdc07af30 0x7fdbdc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:16.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.972+0000 7fdbf5c68640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbdc07af30 0x7fdbdc07d3f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0102230 tx=0x7fdbe0004f50 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:16.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:16.975+0000 7fdbe77fe640 1 -- 172.21.15.7:0/846666091 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdbec014030 con 0x7fdbf0102b40 2024-09-21T05:08:17.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.237+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fdbf002d7c0 con 0x7fdbf0102b40 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.238+0000 7fdbe77fe640 1 -- 172.21.15.7:0/846666091 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fdbec069910 con 0x7fdbf0102b40 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:08:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:08:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:08:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:08:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:08:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:08:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbdc07af30 msgr2=0x7fdbdc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbdc07af30 0x7fdbdc07d3f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0102230 tx=0x7fdbe0004f50 comp rx=0 tx=0).stop 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0102b40 msgr2=0x7fdbf01007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0102b40 0x7fdbf01007a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c8a0 tx=0x7fdbec00cd60 comp rx=0 tx=0).stop 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 shutdown_connections 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdbdc07af30 0x7fdbdc07d3f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf0103460 0x7fdbf0100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 --2- 172.21.15.7:0/846666091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf0102b40 0x7fdbf01007a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 >> 172.21.15.7:0/846666091 conn(0x7fdbf00fc520 msgr2=0x7fdbf0107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 shutdown_connections 2024-09-21T05:08:17.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.240+0000 7fdbf86cd640 1 -- 172.21.15.7:0/846666091 wait complete. 2024-09-21T05:08:17.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.323+0000 7fab57ca6640 1 -- 172.21.15.7:0/225329077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 msgr2=0x7fab50100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.323+0000 7fab57ca6640 1 --2- 172.21.15.7:0/225329077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 0x7fab50100c90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009880 tx=0x7fab4c02a530 comp rx=0 tx=0).stop 2024-09-21T05:08:17.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.323+0000 7fab57ca6640 1 -- 172.21.15.7:0/225329077 shutdown_connections 2024-09-21T05:08:17.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.323+0000 7fab57ca6640 1 --2- 172.21.15.7:0/225329077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab501011d0 0x7fab501056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.323+0000 7fab57ca6640 1 --2- 172.21.15.7:0/225329077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 0x7fab50100c90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.323+0000 7fab57ca6640 1 -- 172.21.15.7:0/225329077 >> 172.21.15.7:0/225329077 conn(0x7fab500fc520 msgr2=0x7fab500fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:17.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.324+0000 7fab57ca6640 1 -- 172.21.15.7:0/225329077 shutdown_connections 2024-09-21T05:08:17.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.324+0000 7fab57ca6640 1 -- 172.21.15.7:0/225329077 wait complete. 2024-09-21T05:08:17.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.324+0000 7fab57ca6640 1 Processor -- start 2024-09-21T05:08:17.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.324+0000 7fab57ca6640 1 -- start start 2024-09-21T05:08:17.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.324+0000 7fab57ca6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 0x7fab5019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.324+0000 7fab57ca6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab501011d0 0x7fab5019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.324+0000 7fab57ca6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab501153f0 con 0x7fab501008b0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.324+0000 7fab57ca6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab50115270 con 0x7fab501011d0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab55a42640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 0x7fab5019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab55a42640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 0x7fab5019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59832/0 (socket says 172.21.15.7:59832) 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab55a42640 1 -- 172.21.15.7:0/1533746390 learned_addr learned my addr 172.21.15.7:0/1533746390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab55241640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab501011d0 0x7fab5019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab55a42640 1 -- 172.21.15.7:0/1533746390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab501011d0 msgr2=0x7fab5019e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab55a42640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab501011d0 0x7fab5019e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab55a42640 1 -- 172.21.15.7:0/1533746390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab501a2880 con 0x7fab501008b0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab55a42640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 0x7fab5019dfb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fab4c031460 tx=0x7fab4c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab46ffd640 1 -- 172.21.15.7:0/1533746390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fab4c03f070 con 0x7fab501008b0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab501a2b10 con 0x7fab501008b0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab46ffd640 1 -- 172.21.15.7:0/1533746390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab4c004b10 con 0x7fab501008b0 2024-09-21T05:08:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab46ffd640 1 -- 172.21.15.7:0/1533746390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fab4c035030 con 0x7fab501008b0 2024-09-21T05:08:17.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.325+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab501a3040 con 0x7fab501008b0 2024-09-21T05:08:17.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.326+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5006aa90 con 0x7fab501008b0 2024-09-21T05:08:17.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.327+0000 7fab46ffd640 1 -- 172.21.15.7:0/1533746390 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fab4c042070 con 0x7fab501008b0 2024-09-21T05:08:17.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.328+0000 7fab46ffd640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab3c07af90 0x7fab3c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.328+0000 7fab46ffd640 1 -- 172.21.15.7:0/1533746390 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fab4c0bea20 con 0x7fab501008b0 2024-09-21T05:08:17.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.328+0000 7fab55241640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab3c07af90 0x7fab3c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.328+0000 7fab55241640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab3c07af90 0x7fab3c07d450 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab501124f0 tx=0x7fab4000b2e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:17.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.332+0000 7fab46ffd640 1 -- 172.21.15.7:0/1533746390 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab4c08a040 con 0x7fab501008b0 2024-09-21T05:08:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.556+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fab5019f310 con 0x7fab501008b0 2024-09-21T05:08:17.560 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:17 smithi007 ceph-mon[108238]: pgmap v1006: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:17.560 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:17 smithi007 ceph-mon[108238]: from='client.35628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:17.560 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:17 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/846666091' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:08:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.557+0000 7fab46ffd640 1 -- 172.21.15.7:0/1533746390 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7fab4c08eef0 con 0x7fab501008b0 2024-09-21T05:08:17.560 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:08:17.560 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:08:17.561 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:08:17.562 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:08:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:08:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:08:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:08:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab3c07af90 msgr2=0x7fab3c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab3c07af90 0x7fab3c07d450 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fab501124f0 tx=0x7fab4000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 msgr2=0x7fab5019dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 0x7fab5019dfb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fab4c031460 tx=0x7fab4c0045e0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 shutdown_connections 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fab3c07af90 0x7fab3c07d450 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab501011d0 0x7fab5019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 --2- 172.21.15.7:0/1533746390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab501008b0 0x7fab5019dfb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 >> 172.21.15.7:0/1533746390 conn(0x7fab500fc520 msgr2=0x7fab50112b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 shutdown_connections 2024-09-21T05:08:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.559+0000 7fab57ca6640 1 -- 172.21.15.7:0/1533746390 wait complete. 2024-09-21T05:08:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.644+0000 7f3330477640 1 -- 172.21.15.7:0/2032552011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328107900 msgr2=0x7f3328107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.644+0000 7f3330477640 1 --2- 172.21.15.7:0/2032552011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328107900 0x7f3328107ce0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3324009880 tx=0x7f332402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.644+0000 7f3330477640 1 -- 172.21.15.7:0/2032552011 shutdown_connections 2024-09-21T05:08:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.644+0000 7f3330477640 1 --2- 172.21.15.7:0/2032552011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3328101900 0x7f3328101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.644+0000 7f3330477640 1 --2- 172.21.15.7:0/2032552011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328107900 0x7f3328107ce0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.644+0000 7f3330477640 1 -- 172.21.15.7:0/2032552011 >> 172.21.15.7:0/2032552011 conn(0x7f33280fd600 msgr2=0x7f33280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.644+0000 7f3330477640 1 -- 172.21.15.7:0/2032552011 shutdown_connections 2024-09-21T05:08:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.644+0000 7f3330477640 1 -- 172.21.15.7:0/2032552011 wait complete. 2024-09-21T05:08:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f3330477640 1 Processor -- start 2024-09-21T05:08:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f3330477640 1 -- start start 2024-09-21T05:08:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f3330477640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328101900 0x7f33280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f3330477640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3328107900 0x7f3328074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f3330477640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f332810edb0 con 0x7f3328101900 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f3330477640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f332810ec30 con 0x7f3328107900 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f332e213640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328101900 0x7f33280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f332e213640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328101900 0x7f33280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59844/0 (socket says 172.21.15.7:59844) 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f332e213640 1 -- 172.21.15.7:0/3947416779 learned_addr learned my addr 172.21.15.7:0/3947416779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.645+0000 7f332da12640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3328107900 0x7f3328074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f332e213640 1 -- 172.21.15.7:0/3947416779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3328107900 msgr2=0x7f3328074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f332e213640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3328107900 0x7f3328074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f332e213640 1 -- 172.21.15.7:0/3947416779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3328074f70 con 0x7f3328101900 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f332e213640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328101900 0x7f33280744f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3324031460 tx=0x7f33240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f331f7fe640 1 -- 172.21.15.7:0/3947416779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f332403f070 con 0x7f3328101900 2024-09-21T05:08:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f331f7fe640 1 -- 172.21.15.7:0/3947416779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33240044d0 con 0x7f3328101900 2024-09-21T05:08:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f331f7fe640 1 -- 172.21.15.7:0/3947416779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3324035030 con 0x7f3328101900 2024-09-21T05:08:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3328075200 con 0x7f3328101900 2024-09-21T05:08:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.646+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281a3d50 con 0x7f3328101900 2024-09-21T05:08:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.647+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3328103040 con 0x7f3328101900 2024-09-21T05:08:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.648+0000 7f331f7fe640 1 -- 172.21.15.7:0/3947416779 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3324042070 con 0x7f3328101900 2024-09-21T05:08:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.648+0000 7f331f7fe640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f331407af90 0x7f331407d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.648+0000 7f331f7fe640 1 -- 172.21.15.7:0/3947416779 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f33240bea10 con 0x7f3328101900 2024-09-21T05:08:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.648+0000 7f332da12640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f331407af90 0x7f331407d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.649+0000 7f332da12640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f331407af90 0x7f331407d450 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3328075f80 tx=0x7f331800a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.652+0000 7f331f7fe640 1 -- 172.21.15.7:0/3947416779 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3324004cf0 con 0x7f3328101900 2024-09-21T05:08:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:17 smithi053 ceph-mon[80176]: pgmap v1006: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:17 smithi053 ceph-mon[80176]: from='client.35628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:17 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/846666091' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:08:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.856+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3328062630 con 0x7f331407af90 2024-09-21T05:08:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.857+0000 7f331f7fe640 1 -- 172.21.15.7:0/3947416779 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3328062630 con 0x7f331407af90 2024-09-21T05:08:17.860 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:08:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:08:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:08:17.861 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:08:17.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f331407af90 msgr2=0x7f331407d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f331407af90 0x7f331407d450 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3328075f80 tx=0x7f331800a2d0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328101900 msgr2=0x7f33280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328101900 0x7f33280744f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3324031460 tx=0x7f33240048f0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 shutdown_connections 2024-09-21T05:08:17.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f331407af90 0x7f331407d450 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3328107900 0x7f3328074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 --2- 172.21.15.7:0/3947416779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3328101900 0x7f33280744f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.859+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 >> 172.21.15.7:0/3947416779 conn(0x7f33280fd600 msgr2=0x7f33280fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:17.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.860+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 shutdown_connections 2024-09-21T05:08:17.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.860+0000 7f3330477640 1 -- 172.21.15.7:0/3947416779 wait complete. 2024-09-21T05:08:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 -- 172.21.15.7:0/2900892355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09581069a0 msgr2=0x7f0958106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 --2- 172.21.15.7:0/2900892355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09581069a0 0x7f0958106d80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0948009880 tx=0x7f094802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 -- 172.21.15.7:0/2900892355 shutdown_connections 2024-09-21T05:08:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 --2- 172.21.15.7:0/2900892355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09580ffdc0 0x7f0958100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 --2- 172.21.15.7:0/2900892355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09581069a0 0x7f0958106d80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 -- 172.21.15.7:0/2900892355 >> 172.21.15.7:0/2900892355 conn(0x7f09580fbc90 msgr2=0x7f09580fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 -- 172.21.15.7:0/2900892355 shutdown_connections 2024-09-21T05:08:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 -- 172.21.15.7:0/2900892355 wait complete. 2024-09-21T05:08:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 Processor -- start 2024-09-21T05:08:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.944+0000 7f095e449640 1 -- start start 2024-09-21T05:08:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f095e449640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09580ffdc0 0x7f0958100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f095e449640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09581069a0 0x7f09581014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f095e449640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f095810ef20 con 0x7f09580ffdc0 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f095e449640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f095810eda0 con 0x7f09581069a0 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f0957fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09580ffdc0 0x7f0958100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f0957fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09580ffdc0 0x7f0958100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59874/0 (socket says 172.21.15.7:59874) 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f0957fff640 1 -- 172.21.15.7:0/2665510755 learned_addr learned my addr 172.21.15.7:0/2665510755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f09577fe640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09581069a0 0x7f09581014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f0957fff640 1 -- 172.21.15.7:0/2665510755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09581069a0 msgr2=0x7f09581014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f0957fff640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09581069a0 0x7f09581014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f0957fff640 1 -- 172.21.15.7:0/2665510755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0958101a20 con 0x7f09580ffdc0 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.945+0000 7f09577fe640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09581069a0 0x7f09581014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.946+0000 7f0957fff640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09580ffdc0 0x7f0958100fa0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0948031460 tx=0x7f09480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.946+0000 7f09557fa640 1 -- 172.21.15.7:0/2665510755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f094803f070 con 0x7f09580ffdc0 2024-09-21T05:08:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.946+0000 7f09557fa640 1 -- 172.21.15.7:0/2665510755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0948004cf0 con 0x7f09580ffdc0 2024-09-21T05:08:17.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.946+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0958101cb0 con 0x7f09580ffdc0 2024-09-21T05:08:17.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.946+0000 7f09557fa640 1 -- 172.21.15.7:0/2665510755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0948035030 con 0x7f09580ffdc0 2024-09-21T05:08:17.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.946+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09581a3e00 con 0x7f09580ffdc0 2024-09-21T05:08:17.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.947+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09581030e0 con 0x7f09580ffdc0 2024-09-21T05:08:17.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.947+0000 7f09557fa640 1 -- 172.21.15.7:0/2665510755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0948042070 con 0x7f09580ffdc0 2024-09-21T05:08:17.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.948+0000 7f09557fa640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f094007af80 0x7f094007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:17.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.948+0000 7f09557fa640 1 -- 172.21.15.7:0/2665510755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f09480bea50 con 0x7f09580ffdc0 2024-09-21T05:08:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.948+0000 7f09577fe640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f094007af80 0x7f094007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.949+0000 7f09577fe640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f094007af80 0x7f094007d440 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0958102a30 tx=0x7f094c005d90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:17.953+0000 7f09557fa640 1 -- 172.21.15.7:0/2665510755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f094808a070 con 0x7f09580ffdc0 2024-09-21T05:08:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.222+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f09581a4080 con 0x7f09580ffdc0 2024-09-21T05:08:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.222+0000 7f09557fa640 1 -- 172.21.15.7:0/2665510755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f094808ef20 con 0x7f09580ffdc0 2024-09-21T05:08:18.225 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:08:18.225 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:08:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:08:18.225 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:08:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:08:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:08:18.227 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:08:18.227 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:08:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:08:18.227 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:08:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:08:18.227 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:08:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:08:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f094007af80 msgr2=0x7f094007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f094007af80 0x7f094007d440 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0958102a30 tx=0x7f094c005d90 comp rx=0 tx=0).stop 2024-09-21T05:08:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09580ffdc0 msgr2=0x7f0958100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09580ffdc0 0x7f0958100fa0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0948031460 tx=0x7f09480048f0 comp rx=0 tx=0).stop 2024-09-21T05:08:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 shutdown_connections 2024-09-21T05:08:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f094007af80 0x7f094007d440 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09581069a0 0x7f09581014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 --2- 172.21.15.7:0/2665510755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09580ffdc0 0x7f0958100fa0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 >> 172.21.15.7:0/2665510755 conn(0x7f09580fbc90 msgr2=0x7f09580fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 shutdown_connections 2024-09-21T05:08:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:18.224+0000 7f095e449640 1 -- 172.21.15.7:0/2665510755 wait complete. 2024-09-21T05:08:18.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:18 smithi053 ceph-mon[80176]: from='client.35632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:18.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:18 smithi053 ceph-mon[80176]: from='client.35636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:18.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:18 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1533746390' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:08:18.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:18 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2665510755' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:08:18.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:18 smithi007 ceph-mon[108238]: from='client.35632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:18.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:18 smithi007 ceph-mon[108238]: from='client.35636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:18.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:18 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1533746390' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:08:18.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:18 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2665510755' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:08:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:19 smithi053 ceph-mon[80176]: pgmap v1007: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:19 smithi053 ceph-mon[80176]: from='client.35648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:19 smithi007 ceph-mon[108238]: pgmap v1007: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:19 smithi007 ceph-mon[108238]: from='client.35648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:20.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:20 smithi053 ceph-mon[80176]: pgmap v1008: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:20 smithi007 ceph-mon[108238]: pgmap v1008: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:22.876 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:22 smithi007 ceph-mon[108238]: pgmap v1009: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:22.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:22 smithi053 ceph-mon[80176]: pgmap v1009: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:24 smithi053 ceph-mon[80176]: pgmap v1010: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:24 smithi007 ceph-mon[108238]: pgmap v1010: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:26.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:26 smithi053 ceph-mon[80176]: pgmap v1011: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:26.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:26 smithi007 ceph-mon[108238]: pgmap v1011: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:28.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:28 smithi007 ceph-mon[108238]: pgmap v1012: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:28.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:28 smithi053 ceph-mon[80176]: pgmap v1012: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:29.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:08:29.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:08:30.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:30 smithi053 ceph-mon[80176]: pgmap v1013: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:30 smithi007 ceph-mon[108238]: pgmap v1013: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:32.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:32 smithi053 ceph-mon[80176]: pgmap v1014: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:32.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:32 smithi007 ceph-mon[108238]: pgmap v1014: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:34.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:34 smithi007 ceph-mon[108238]: pgmap v1015: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:34.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:34 smithi053 ceph-mon[80176]: pgmap v1015: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:36.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:36 smithi053 ceph-mon[80176]: pgmap v1016: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:36.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:36 smithi007 ceph-mon[108238]: pgmap v1016: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:38.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:38 smithi053 ceph-mon[80176]: pgmap v1017: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:38 smithi007 ceph-mon[108238]: pgmap v1017: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:40.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:40 smithi053 ceph-mon[80176]: pgmap v1018: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:40.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:40 smithi007 ceph-mon[108238]: pgmap v1018: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:42.887 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:42 smithi053 ceph-mon[80176]: pgmap v1019: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:42.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:42 smithi007 ceph-mon[108238]: pgmap v1019: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:43.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:08:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:08:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:44 smithi053 ceph-mon[80176]: pgmap v1020: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:44 smithi007 ceph-mon[108238]: pgmap v1020: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:46.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:46 smithi007 ceph-mon[108238]: pgmap v1021: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:46.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:46 smithi053 ceph-mon[80176]: pgmap v1021: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:48.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.320+0000 7f7c70137640 1 -- 172.21.15.7:0/3030603585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 msgr2=0x7f7c68107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.320+0000 7f7c70137640 1 --2- 172.21.15.7:0/3030603585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 0x7f7c68107ce0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009880 tx=0x7f7c6402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.320+0000 7f7c70137640 1 -- 172.21.15.7:0/3030603585 shutdown_connections 2024-09-21T05:08:48.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.320+0000 7f7c70137640 1 --2- 172.21.15.7:0/3030603585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c68101900 0x7f7c68101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.320+0000 7f7c70137640 1 --2- 172.21.15.7:0/3030603585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 0x7f7c68107ce0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.320+0000 7f7c70137640 1 -- 172.21.15.7:0/3030603585 >> 172.21.15.7:0/3030603585 conn(0x7f7c680fd600 msgr2=0x7f7c680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:48.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.321+0000 7f7c70137640 1 -- 172.21.15.7:0/3030603585 shutdown_connections 2024-09-21T05:08:48.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.321+0000 7f7c70137640 1 -- 172.21.15.7:0/3030603585 wait complete. 2024-09-21T05:08:48.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.321+0000 7f7c70137640 1 Processor -- start 2024-09-21T05:08:48.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.321+0000 7f7c70137640 1 -- start start 2024-09-21T05:08:48.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c70137640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c68101900 0x7f7c6819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c70137640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 0x7f7c6819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c70137640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6810edd0 con 0x7f7c68107900 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c70137640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6810ec50 con 0x7f7c68101900 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c6d6d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 0x7f7c6819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c6d6d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 0x7f7c6819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48522/0 (socket says 172.21.15.7:48522) 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c6d6d2640 1 -- 172.21.15.7:0/3722221704 learned_addr learned my addr 172.21.15.7:0/3722221704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c6ded3640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c68101900 0x7f7c6819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c6d6d2640 1 -- 172.21.15.7:0/3722221704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c68101900 msgr2=0x7f7c6819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c6d6d2640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c68101900 0x7f7c6819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c6d6d2640 1 -- 172.21.15.7:0/3722221704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c681977c0 con 0x7f7c68107900 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c6d6d2640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 0x7f7c6819db80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800e970 tx=0x7f7c5800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c5effd640 1 -- 172.21.15.7:0/3722221704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7c5800cac0 con 0x7f7c68107900 2024-09-21T05:08:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c5effd640 1 -- 172.21.15.7:0/3722221704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c580054e0 con 0x7f7c68107900 2024-09-21T05:08:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c5effd640 1 -- 172.21.15.7:0/3722221704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7c58010430 con 0x7f7c68107900 2024-09-21T05:08:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.322+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c68197ab0 con 0x7f7c68107900 2024-09-21T05:08:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.323+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c68198040 con 0x7f7c68107900 2024-09-21T05:08:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.324+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c68103040 con 0x7f7c68107900 2024-09-21T05:08:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.324+0000 7f7c5effd640 1 -- 172.21.15.7:0/3722221704 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7c58004e80 con 0x7f7c68107900 2024-09-21T05:08:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.325+0000 7f7c5effd640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c5407af30 0x7f7c5407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.325+0000 7f7c6ded3640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c5407af30 0x7f7c5407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.325+0000 7f7c5effd640 1 -- 172.21.15.7:0/3722221704 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f7c5809e760 con 0x7f7c68107900 2024-09-21T05:08:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.326+0000 7f7c6ded3640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c5407af30 0x7f7c5407d3f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7c640096c0 tx=0x7f7c64031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.330+0000 7f7c5effd640 1 -- 172.21.15.7:0/3722221704 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c58069d00 con 0x7f7c68107900 2024-09-21T05:08:48.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.539+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7c681983b0 con 0x7f7c5407af30 2024-09-21T05:08:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.541+0000 7f7c5effd640 1 -- 172.21.15.7:0/3722221704 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7c681983b0 con 0x7f7c5407af30 2024-09-21T05:08:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c5407af30 msgr2=0x7f7c5407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c5407af30 0x7f7c5407d3f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7c640096c0 tx=0x7f7c64031400 comp rx=0 tx=0).stop 2024-09-21T05:08:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 msgr2=0x7f7c6819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 0x7f7c6819db80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800e970 tx=0x7f7c5800ed70 comp rx=0 tx=0).stop 2024-09-21T05:08:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 shutdown_connections 2024-09-21T05:08:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7c5407af30 0x7f7c5407d3f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c68107900 0x7f7c6819db80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 --2- 172.21.15.7:0/3722221704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c68101900 0x7f7c6819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 >> 172.21.15.7:0/3722221704 conn(0x7f7c680fd600 msgr2=0x7f7c680fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 shutdown_connections 2024-09-21T05:08:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.543+0000 7f7c70137640 1 -- 172.21.15.7:0/3722221704 wait complete. 2024-09-21T05:08:48.556 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:08:48.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.628+0000 7f6a283f3640 1 -- 172.21.15.7:0/2654358306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 msgr2=0x7f6a20101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.628+0000 7f6a283f3640 1 --2- 172.21.15.7:0/2654358306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 0x7f6a20101d60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009880 tx=0x7f6a1402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.633 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:48 smithi007 ceph-mon[108238]: pgmap v1022: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 -- 172.21.15.7:0/2654358306 shutdown_connections 2024-09-21T05:08:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 --2- 172.21.15.7:0/2654358306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 0x7f6a20101d60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 --2- 172.21.15.7:0/2654358306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a20107900 0x7f6a20107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 -- 172.21.15.7:0/2654358306 >> 172.21.15.7:0/2654358306 conn(0x7f6a200fd600 msgr2=0x7f6a200ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 -- 172.21.15.7:0/2654358306 shutdown_connections 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 -- 172.21.15.7:0/2654358306 wait complete. 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 Processor -- start 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 -- start start 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 0x7f6a200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a20107900 0x7f6a20074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2010edd0 con 0x7f6a20101900 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.629+0000 7f6a283f3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2010ec50 con 0x7f6a20107900 2024-09-21T05:08:48.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a2618f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 0x7f6a200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a2618f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 0x7f6a200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48554/0 (socket says 172.21.15.7:48554) 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a2618f640 1 -- 172.21.15.7:0/1534450203 learned_addr learned my addr 172.21.15.7:0/1534450203 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a2598e640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a20107900 0x7f6a20074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a2618f640 1 -- 172.21.15.7:0/1534450203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a20107900 msgr2=0x7f6a20074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a2618f640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a20107900 0x7f6a20074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a2618f640 1 -- 172.21.15.7:0/1534450203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a20074f70 con 0x7f6a20101900 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a2618f640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 0x7f6a200744f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00ea00 tx=0x7f6a1c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a137fe640 1 -- 172.21.15.7:0/1534450203 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6a1c00cb80 con 0x7f6a20101900 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a20075260 con 0x7f6a20101900 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a201a3e20 con 0x7f6a20101900 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.630+0000 7f6a137fe640 1 -- 172.21.15.7:0/1534450203 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a1c00cd20 con 0x7f6a20101900 2024-09-21T05:08:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.631+0000 7f6a137fe640 1 -- 172.21.15.7:0/1534450203 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6a1c011470 con 0x7f6a20101900 2024-09-21T05:08:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.631+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a04005290 con 0x7f6a20101900 2024-09-21T05:08:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.632+0000 7f6a137fe640 1 -- 172.21.15.7:0/1534450203 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6a1c004a40 con 0x7f6a20101900 2024-09-21T05:08:48.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.633+0000 7f6a137fe640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a0c07af80 0x7f6a0c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.633+0000 7f6a137fe640 1 -- 172.21.15.7:0/1534450203 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f6a1c09e2e0 con 0x7f6a20101900 2024-09-21T05:08:48.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.633+0000 7f6a2598e640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a0c07af80 0x7f6a0c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.634+0000 7f6a2598e640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a0c07af80 0x7f6a0c07d440 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6a20075f80 tx=0x7f6a14031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.637+0000 7f6a137fe640 1 -- 172.21.15.7:0/1534450203 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6a1c016030 con 0x7f6a20101900 2024-09-21T05:08:48.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.847+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6a04000bc0 con 0x7f6a0c07af80 2024-09-21T05:08:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.848+0000 7f6a137fe640 1 -- 172.21.15.7:0/1534450203 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f6a04000bc0 con 0x7f6a0c07af80 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.850+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a0c07af80 msgr2=0x7f6a0c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.850+0000 7f6a283f3640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a0c07af80 0x7f6a0c07d440 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6a20075f80 tx=0x7f6a14031400 comp rx=0 tx=0).stop 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 msgr2=0x7f6a200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 0x7f6a200744f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00ea00 tx=0x7f6a1c00eec0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 shutdown_connections 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6a0c07af80 0x7f6a0c07d440 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a20107900 0x7f6a20074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 --2- 172.21.15.7:0/1534450203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a20101900 0x7f6a200744f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 >> 172.21.15.7:0/1534450203 conn(0x7f6a200fd600 msgr2=0x7f6a200fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 shutdown_connections 2024-09-21T05:08:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.851+0000 7f6a283f3640 1 -- 172.21.15.7:0/1534450203 wait complete. 2024-09-21T05:08:48.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.935+0000 7f14bdd77640 1 -- 172.21.15.7:0/956127810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b8102ac0 msgr2=0x7f14b8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.935+0000 7f14bdd77640 1 --2- 172.21.15.7:0/956127810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b8102ac0 0x7f14b8102ea0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009880 tx=0x7f14a802a530 comp rx=0 tx=0).stop 2024-09-21T05:08:48.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.935+0000 7f14bdd77640 1 -- 172.21.15.7:0/956127810 shutdown_connections 2024-09-21T05:08:48.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.935+0000 7f14bdd77640 1 --2- 172.21.15.7:0/956127810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b81033e0 0x7f14b810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.935+0000 7f14bdd77640 1 --2- 172.21.15.7:0/956127810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b8102ac0 0x7f14b8102ea0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.935+0000 7f14bdd77640 1 -- 172.21.15.7:0/956127810 >> 172.21.15.7:0/956127810 conn(0x7f14b80fc520 msgr2=0x7f14b80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:48.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.935+0000 7f14bdd77640 1 -- 172.21.15.7:0/956127810 shutdown_connections 2024-09-21T05:08:48.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.935+0000 7f14bdd77640 1 -- 172.21.15.7:0/956127810 wait complete. 2024-09-21T05:08:48.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14bdd77640 1 Processor -- start 2024-09-21T05:08:48.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14bdd77640 1 -- start start 2024-09-21T05:08:48.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14bdd77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b8102ac0 0x7f14b819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14bdd77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b81033e0 0x7f14b819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14bdd77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14b810ee50 con 0x7f14b81033e0 2024-09-21T05:08:48.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14bdd77640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14b810ecd0 con 0x7f14b8102ac0 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14aeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b81033e0 0x7f14b819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b8102ac0 0x7f14b819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14aeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b81033e0 0x7f14b819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48564/0 (socket says 172.21.15.7:48564) 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.936+0000 7f14aeffd640 1 -- 172.21.15.7:0/909396765 learned_addr learned my addr 172.21.15.7:0/909396765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14b77fe640 1 -- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b81033e0 msgr2=0x7f14b819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14b77fe640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b81033e0 0x7f14b819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14b77fe640 1 -- 172.21.15.7:0/909396765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b8199a80 con 0x7f14b8102ac0 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14aeffd640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b81033e0 0x7f14b819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14b77fe640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b8102ac0 0x7f14b819f8c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14a8006010 tx=0x7f14a80045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14b57fa640 1 -- 172.21.15.7:0/909396765 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14a803f070 con 0x7f14b8102ac0 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b8199d10 con 0x7f14b8102ac0 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14b57fa640 1 -- 172.21.15.7:0/909396765 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14a80035f0 con 0x7f14b8102ac0 2024-09-21T05:08:48.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14b57fa640 1 -- 172.21.15.7:0/909396765 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14a8035030 con 0x7f14b8102ac0 2024-09-21T05:08:48.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.937+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b819a240 con 0x7f14b8102ac0 2024-09-21T05:08:48.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:48 smithi053 ceph-mon[80176]: pgmap v1022: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.938+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b806aa90 con 0x7f14b8102ac0 2024-09-21T05:08:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.939+0000 7f14b57fa640 1 -- 172.21.15.7:0/909396765 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f14a8042070 con 0x7f14b8102ac0 2024-09-21T05:08:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.940+0000 7f14b57fa640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14a407af30 0x7f14a407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.940+0000 7f14b57fa640 1 -- 172.21.15.7:0/909396765 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f14a80beb00 con 0x7f14b8102ac0 2024-09-21T05:08:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.940+0000 7f14aeffd640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14a407af30 0x7f14a407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.940+0000 7f14aeffd640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14a407af30 0x7f14a407d3f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f14b80fe430 tx=0x7f14a0009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:48.945+0000 7f14b57fa640 1 -- 172.21.15.7:0/909396765 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14a808a0a0 con 0x7f14b8102ac0 2024-09-21T05:08:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.142+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f14b819afb0 con 0x7f14a407af30 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.152+0000 7f14b57fa640 1 -- 172.21.15.7:0/909396765 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f14b819afb0 con 0x7f14a407af30 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (30m) 9m ago 41m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (33m) 9m ago 41m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (33m) 9m ago 41m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (29m) 9m ago 41m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (28m) 9m ago 41m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (30m) 9m ago 41m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (24m) 9m ago 36m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (25m) 9m ago 36m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:08:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (9m) 9m ago 36m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (24m) 9m ago 36m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (33m) 9m ago 43m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (32m) 9m ago 40m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (29m) 9m ago 43m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (29m) 9m ago 40m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (31m) 9m ago 41m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (31m) 9m ago 41m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (28m) 9m ago 39m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (27m) 9m ago 39m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (27m) 9m ago 38m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (26m) 9m ago 38m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (26m) 9m ago 37m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (25m) 9m ago 37m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:08:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (31m) 9m ago 41m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:08:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14a407af30 msgr2=0x7f14a407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14a407af30 0x7f14a407d3f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f14b80fe430 tx=0x7f14a0009110 comp rx=0 tx=0).stop 2024-09-21T05:08:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b8102ac0 msgr2=0x7f14b819f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b8102ac0 0x7f14b819f8c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f14a8006010 tx=0x7f14a80045e0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 shutdown_connections 2024-09-21T05:08:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14a407af30 0x7f14a407d3f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14b81033e0 0x7f14b819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 --2- 172.21.15.7:0/909396765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14b8102ac0 0x7f14b819f8c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 >> 172.21.15.7:0/909396765 conn(0x7f14b80fc520 msgr2=0x7f14b8100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 shutdown_connections 2024-09-21T05:08:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.154+0000 7f14bdd77640 1 -- 172.21.15.7:0/909396765 wait complete. 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.237+0000 7f8259e2a640 1 -- 172.21.15.7:0/758882943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 msgr2=0x7f8254100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.237+0000 7f8259e2a640 1 --2- 172.21.15.7:0/758882943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 0x7f8254100c60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8248009880 tx=0x7f824802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.237+0000 7f8259e2a640 1 -- 172.21.15.7:0/758882943 shutdown_connections 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.237+0000 7f8259e2a640 1 --2- 172.21.15.7:0/758882943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 0x7f8254100c60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.237+0000 7f8259e2a640 1 --2- 172.21.15.7:0/758882943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254106800 0x7f8254106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.237+0000 7f8259e2a640 1 -- 172.21.15.7:0/758882943 >> 172.21.15.7:0/758882943 conn(0x7f82540fc520 msgr2=0x7f82540fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.237+0000 7f8259e2a640 1 -- 172.21.15.7:0/758882943 shutdown_connections 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.238+0000 7f8259e2a640 1 -- 172.21.15.7:0/758882943 wait complete. 2024-09-21T05:08:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.238+0000 7f8259e2a640 1 Processor -- start 2024-09-21T05:08:49.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.238+0000 7f8259e2a640 1 -- start start 2024-09-21T05:08:49.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.238+0000 7f8259e2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 0x7f825419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.238+0000 7f8259e2a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254106800 0x7f825419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f8259e2a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f825410dcb0 con 0x7f8254100800 2024-09-21T05:08:49.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f8259e2a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f825410db30 con 0x7f8254106800 2024-09-21T05:08:49.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f82537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 0x7f825419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f82537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 0x7f825419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48578/0 (socket says 172.21.15.7:48578) 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f82537fe640 1 -- 172.21.15.7:0/686658344 learned_addr learned my addr 172.21.15.7:0/686658344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f8252ffd640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254106800 0x7f825419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f82537fe640 1 -- 172.21.15.7:0/686658344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254106800 msgr2=0x7f825419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f82537fe640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254106800 0x7f825419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f82537fe640 1 -- 172.21.15.7:0/686658344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82541966c0 con 0x7f8254100800 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f8252ffd640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254106800 0x7f825419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f82537fe640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 0x7f825419c5f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f824400eb00 tx=0x7f8244009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f8250ff9640 1 -- 172.21.15.7:0/686658344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f824401c070 con 0x7f8254100800 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f8250ff9640 1 -- 172.21.15.7:0/686658344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f824400bbb0 con 0x7f8254100800 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f8250ff9640 1 -- 172.21.15.7:0/686658344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8244016600 con 0x7f8254100800 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.239+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8254196950 con 0x7f8254100800 2024-09-21T05:08:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.240+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8254196eb0 con 0x7f8254100800 2024-09-21T05:08:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.241+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8254101f40 con 0x7f8254100800 2024-09-21T05:08:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.241+0000 7f8250ff9640 1 -- 172.21.15.7:0/686658344 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f824401f070 con 0x7f8254100800 2024-09-21T05:08:49.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.242+0000 7f8250ff9640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823c07af40 0x7f823c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.242+0000 7f8250ff9640 1 -- 172.21.15.7:0/686658344 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f8244014070 con 0x7f8254100800 2024-09-21T05:08:49.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.242+0000 7f8252ffd640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823c07af40 0x7f823c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.242+0000 7f8252ffd640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823c07af40 0x7f823c07d400 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8254197f10 tx=0x7f8248031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.246+0000 7f8250ff9640 1 -- 172.21.15.7:0/686658344 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f824406afc0 con 0x7f8254100800 2024-09-21T05:08:49.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.508+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f8254197220 con 0x7f8254100800 2024-09-21T05:08:49.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.509+0000 7f8250ff9640 1 -- 172.21.15.7:0/686658344 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f824406fe70 con 0x7f8254100800 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:08:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:08:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:08:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:08:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:08:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:08:49.514 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.511+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823c07af40 msgr2=0x7f823c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.511+0000 7f8259e2a640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823c07af40 0x7f823c07d400 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8254197f10 tx=0x7f8248031400 comp rx=0 tx=0).stop 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.511+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 msgr2=0x7f825419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.511+0000 7f8259e2a640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 0x7f825419c5f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f824400eb00 tx=0x7f8244009f20 comp rx=0 tx=0).stop 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.512+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 shutdown_connections 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.512+0000 7f8259e2a640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f823c07af40 0x7f823c07d400 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.512+0000 7f8259e2a640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8254106800 0x7f825419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.512+0000 7f8259e2a640 1 --2- 172.21.15.7:0/686658344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254100800 0x7f825419c5f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.512+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 >> 172.21.15.7:0/686658344 conn(0x7f82540fc520 msgr2=0x7f82540fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.512+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 shutdown_connections 2024-09-21T05:08:49.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.512+0000 7f8259e2a640 1 -- 172.21.15.7:0/686658344 wait complete. 2024-09-21T05:08:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.596+0000 7f3a9eb78640 1 -- 172.21.15.7:0/3070644022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981067f0 msgr2=0x7f3a98106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.596+0000 7f3a9eb78640 1 --2- 172.21.15.7:0/3070644022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981067f0 0x7f3a98106bd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009880 tx=0x7f3a8c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.596+0000 7f3a9eb78640 1 -- 172.21.15.7:0/3070644022 shutdown_connections 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.596+0000 7f3a9eb78640 1 --2- 172.21.15.7:0/3070644022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a981007f0 0x7f3a98100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.596+0000 7f3a9eb78640 1 --2- 172.21.15.7:0/3070644022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981067f0 0x7f3a98106bd0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.596+0000 7f3a9eb78640 1 -- 172.21.15.7:0/3070644022 >> 172.21.15.7:0/3070644022 conn(0x7f3a980fc550 msgr2=0x7f3a980fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.596+0000 7f3a9eb78640 1 -- 172.21.15.7:0/3070644022 shutdown_connections 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.596+0000 7f3a9eb78640 1 -- 172.21.15.7:0/3070644022 wait complete. 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9eb78640 1 Processor -- start 2024-09-21T05:08:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9eb78640 1 -- start start 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9eb78640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981007f0 0x7f3a9819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9eb78640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a981067f0 0x7f3a9819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9eb78640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9810dcc0 con 0x7f3a981007f0 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9eb78640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9810db40 con 0x7f3a981067f0 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9c914640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981007f0 0x7f3a9819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9c914640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981007f0 0x7f3a9819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48584/0 (socket says 172.21.15.7:48584) 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a9c914640 1 -- 172.21.15.7:0/596794626 learned_addr learned my addr 172.21.15.7:0/596794626 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.597+0000 7f3a97fff640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a981067f0 0x7f3a9819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a9c914640 1 -- 172.21.15.7:0/596794626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a981067f0 msgr2=0x7f3a9819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a9c914640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a981067f0 0x7f3a9819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a9c914640 1 -- 172.21.15.7:0/596794626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a981977e0 con 0x7f3a981007f0 2024-09-21T05:08:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a97fff640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a981067f0 0x7f3a9819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:08:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a9c914640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981007f0 0x7f3a9819d660 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c031460 tx=0x7f3a8c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a95ffb640 1 -- 172.21.15.7:0/596794626 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3a8c03f070 con 0x7f3a981007f0 2024-09-21T05:08:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a95ffb640 1 -- 172.21.15.7:0/596794626 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a8c004cf0 con 0x7f3a981007f0 2024-09-21T05:08:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a98197a70 con 0x7f3a981007f0 2024-09-21T05:08:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a95ffb640 1 -- 172.21.15.7:0/596794626 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3a8c035030 con 0x7f3a981007f0 2024-09-21T05:08:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.598+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a98197fa0 con 0x7f3a981007f0 2024-09-21T05:08:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.599+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a98101f30 con 0x7f3a981007f0 2024-09-21T05:08:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.600+0000 7f3a95ffb640 1 -- 172.21.15.7:0/596794626 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3a8c042070 con 0x7f3a981007f0 2024-09-21T05:08:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.600+0000 7f3a95ffb640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a8807af80 0x7f3a8807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.601+0000 7f3a95ffb640 1 -- 172.21.15.7:0/596794626 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f3a8c0beae0 con 0x7f3a981007f0 2024-09-21T05:08:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.601+0000 7f3a97fff640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a8807af80 0x7f3a8807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.601+0000 7f3a97fff640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a8807af80 0x7f3a8807d440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005fd0 tx=0x7f3a84005ea0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.605+0000 7f3a95ffb640 1 -- 172.21.15.7:0/596794626 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a8c08a060 con 0x7f3a981007f0 2024-09-21T05:08:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.830+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3a98109d70 con 0x7f3a981007f0 2024-09-21T05:08:49.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:49 smithi007 ceph-mon[108238]: from='client.35656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:49.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:49 smithi007 ceph-mon[108238]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:49.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:49 smithi007 ceph-mon[108238]: from='client.45251 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:49.835 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:49 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/686658344' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:08:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.830+0000 7f3a95ffb640 1 -- 172.21.15.7:0/596794626 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f3a8c08ef10 con 0x7f3a981007f0 2024-09-21T05:08:49.835 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:08:49.835 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:08:49.835 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:08:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:08:49.837 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:08:49.838 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:08:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:08:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.832+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a8807af80 msgr2=0x7f3a8807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.832+0000 7f3a9eb78640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a8807af80 0x7f3a8807d440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005fd0 tx=0x7f3a84005ea0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.832+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981007f0 msgr2=0x7f3a9819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.832+0000 7f3a9eb78640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981007f0 0x7f3a9819d660 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c031460 tx=0x7f3a8c0048f0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.833+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 shutdown_connections 2024-09-21T05:08:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.833+0000 7f3a9eb78640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3a8807af80 0x7f3a8807d440 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.833+0000 7f3a9eb78640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a981067f0 0x7f3a9819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.833+0000 7f3a9eb78640 1 --2- 172.21.15.7:0/596794626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a981007f0 0x7f3a9819d660 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.833+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 >> 172.21.15.7:0/596794626 conn(0x7f3a980fc550 msgr2=0x7f3a98069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.833+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 shutdown_connections 2024-09-21T05:08:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.833+0000 7f3a9eb78640 1 -- 172.21.15.7:0/596794626 wait complete. 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.918+0000 7f6e3963d640 1 -- 172.21.15.7:0/1888630062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34100800 msgr2=0x7f6e34100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.918+0000 7f6e3963d640 1 --2- 172.21.15.7:0/1888630062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34100800 0x7f6e34100c60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009880 tx=0x7f6e2802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.918+0000 7f6e3963d640 1 -- 172.21.15.7:0/1888630062 shutdown_connections 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.918+0000 7f6e3963d640 1 --2- 172.21.15.7:0/1888630062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34100800 0x7f6e34100c60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.918+0000 7f6e3963d640 1 --2- 172.21.15.7:0/1888630062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e34106800 0x7f6e34106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.918+0000 7f6e3963d640 1 -- 172.21.15.7:0/1888630062 >> 172.21.15.7:0/1888630062 conn(0x7f6e340fc520 msgr2=0x7f6e340fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.918+0000 7f6e3963d640 1 -- 172.21.15.7:0/1888630062 shutdown_connections 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.918+0000 7f6e3963d640 1 -- 172.21.15.7:0/1888630062 wait complete. 2024-09-21T05:08:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.919+0000 7f6e3963d640 1 Processor -- start 2024-09-21T05:08:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.919+0000 7f6e3963d640 1 -- start start 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.919+0000 7f6e3963d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e34100800 0x7f6e3419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.919+0000 7f6e3963d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34106800 0x7f6e3419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.919+0000 7f6e3963d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3410dcb0 con 0x7f6e34106800 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.919+0000 7f6e3963d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3410db30 con 0x7f6e34100800 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34106800 0x7f6e3419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34106800 0x7f6e3419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48592/0 (socket says 172.21.15.7:48592) 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e327fc640 1 -- 172.21.15.7:0/3068132343 learned_addr learned my addr 172.21.15.7:0/3068132343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e32ffd640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e34100800 0x7f6e3419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e327fc640 1 -- 172.21.15.7:0/3068132343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e34100800 msgr2=0x7f6e3419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e327fc640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e34100800 0x7f6e3419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e327fc640 1 -- 172.21.15.7:0/3068132343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e341977a0 con 0x7f6e34106800 2024-09-21T05:08:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e327fc640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34106800 0x7f6e3419db60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6e28031460 tx=0x7f6e280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e1bfff640 1 -- 172.21.15.7:0/3068132343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6e2803f070 con 0x7f6e34106800 2024-09-21T05:08:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e1bfff640 1 -- 172.21.15.7:0/3068132343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e28004cf0 con 0x7f6e34106800 2024-09-21T05:08:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e34197a30 con 0x7f6e34106800 2024-09-21T05:08:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e1bfff640 1 -- 172.21.15.7:0/3068132343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6e28035030 con 0x7f6e34106800 2024-09-21T05:08:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.920+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e34197f60 con 0x7f6e34106800 2024-09-21T05:08:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.921+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e34101f40 con 0x7f6e34106800 2024-09-21T05:08:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.922+0000 7f6e1bfff640 1 -- 172.21.15.7:0/3068132343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6e28042070 con 0x7f6e34106800 2024-09-21T05:08:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.923+0000 7f6e1bfff640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e1c07af30 0x7f6e1c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.923+0000 7f6e1bfff640 1 -- 172.21.15.7:0/3068132343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f6e280beb20 con 0x7f6e34106800 2024-09-21T05:08:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.923+0000 7f6e32ffd640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e1c07af30 0x7f6e1c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.923+0000 7f6e32ffd640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e1c07af30 0x7f6e1c07d3f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6e240058c0 tx=0x7f6e2400a4c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:49.927+0000 7f6e1bfff640 1 -- 172.21.15.7:0/3068132343 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e2808a0c0 con 0x7f6e34106800 2024-09-21T05:08:49.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:49 smithi053 ceph-mon[80176]: from='client.35656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:49.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:49 smithi053 ceph-mon[80176]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:49.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:49 smithi053 ceph-mon[80176]: from='client.45251 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:49.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:49 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/686658344' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:08:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.137+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6e341982d0 con 0x7f6e1c07af30 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.139+0000 7f6e1bfff640 1 -- 172.21.15.7:0/3068132343 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f6e341982d0 con 0x7f6e1c07af30 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:08:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:08:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:08:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:08:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:08:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:08:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:08:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:08:50.144 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:08:50.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e1c07af30 msgr2=0x7f6e1c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:50.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e1c07af30 0x7f6e1c07d3f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6e240058c0 tx=0x7f6e2400a4c0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34106800 msgr2=0x7f6e3419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:50.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34106800 0x7f6e3419db60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6e28031460 tx=0x7f6e280048f0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 shutdown_connections 2024-09-21T05:08:50.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6e1c07af30 0x7f6e1c07d3f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e34106800 0x7f6e3419db60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 --2- 172.21.15.7:0/3068132343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e34100800 0x7f6e3419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 >> 172.21.15.7:0/3068132343 conn(0x7f6e340fc520 msgr2=0x7f6e3406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:50.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 shutdown_connections 2024-09-21T05:08:50.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.142+0000 7f6e3963d640 1 -- 172.21.15.7:0/3068132343 wait complete. 2024-09-21T05:08:50.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.226+0000 7f9ecbb35640 1 -- 172.21.15.7:0/2965653819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 msgr2=0x7f9ec41123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:50.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.226+0000 7f9ecbb35640 1 --2- 172.21.15.7:0/2965653819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 0x7f9ec41123d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009880 tx=0x7f9eb402a530 comp rx=0 tx=0).stop 2024-09-21T05:08:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.226+0000 7f9ecbb35640 1 -- 172.21.15.7:0/2965653819 shutdown_connections 2024-09-21T05:08:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.226+0000 7f9ecbb35640 1 --2- 172.21.15.7:0/2965653819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 0x7f9ec41123d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.226+0000 7f9ecbb35640 1 --2- 172.21.15.7:0/2965653819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ec4074590 0x7f9ec4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.226+0000 7f9ecbb35640 1 -- 172.21.15.7:0/2965653819 >> 172.21.15.7:0/2965653819 conn(0x7f9ec40fd600 msgr2=0x7f9ec40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.226+0000 7f9ecbb35640 1 -- 172.21.15.7:0/2965653819 shutdown_connections 2024-09-21T05:08:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.226+0000 7f9ecbb35640 1 -- 172.21.15.7:0/2965653819 wait complete. 2024-09-21T05:08:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ecbb35640 1 Processor -- start 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ecbb35640 1 -- start start 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ecbb35640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ec4074590 0x7f9ec419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ecbb35640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 0x7f9ec419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ecbb35640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec41153c0 con 0x7f9ec4074eb0 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ecbb35640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4115240 con 0x7f9ec4074590 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ec90d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 0x7f9ec419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ec90d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 0x7f9ec419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48624/0 (socket says 172.21.15.7:48624) 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.227+0000 7f9ec90d0640 1 -- 172.21.15.7:0/4108598163 learned_addr learned my addr 172.21.15.7:0/4108598163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ec98d1640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ec4074590 0x7f9ec419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ec90d0640 1 -- 172.21.15.7:0/4108598163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ec4074590 msgr2=0x7f9ec419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ec90d0640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ec4074590 0x7f9ec419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ec90d0640 1 -- 172.21.15.7:0/4108598163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec41a2890 con 0x7f9ec4074eb0 2024-09-21T05:08:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ec90d0640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 0x7f9ec419e500 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4031460 tx=0x7f9eb40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ebaffd640 1 -- 172.21.15.7:0/4108598163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9eb403f070 con 0x7f9ec4074eb0 2024-09-21T05:08:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ebaffd640 1 -- 172.21.15.7:0/4108598163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9eb4004b10 con 0x7f9ec4074eb0 2024-09-21T05:08:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec41a2b20 con 0x7f9ec4074eb0 2024-09-21T05:08:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ebaffd640 1 -- 172.21.15.7:0/4108598163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9eb4035030 con 0x7f9ec4074eb0 2024-09-21T05:08:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.228+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec41a3050 con 0x7f9ec4074eb0 2024-09-21T05:08:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.229+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec4075cd0 con 0x7f9ec4074eb0 2024-09-21T05:08:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.230+0000 7f9ebaffd640 1 -- 172.21.15.7:0/4108598163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9eb4042070 con 0x7f9ec4074eb0 2024-09-21T05:08:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.230+0000 7f9ebaffd640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9eb007af40 0x7f9eb007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:08:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.231+0000 7f9ebaffd640 1 -- 172.21.15.7:0/4108598163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f9eb40bebb0 con 0x7f9ec4074eb0 2024-09-21T05:08:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.231+0000 7f9ec98d1640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9eb007af40 0x7f9eb007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:08:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.231+0000 7f9ec98d1640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9eb007af40 0x7f9eb007d400 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0005fd0 tx=0x7f9ec0009870 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:08:50.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.235+0000 7f9ebaffd640 1 -- 172.21.15.7:0/4108598163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9eb40c5080 con 0x7f9ec4074eb0 2024-09-21T05:08:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.497+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9ec410fac0 con 0x7f9ec4074eb0 2024-09-21T05:08:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.497+0000 7f9ebaffd640 1 -- 172.21.15.7:0/4108598163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f9eb408a150 con 0x7f9ec4074eb0 2024-09-21T05:08:50.501 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:08:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:08:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.499+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9eb007af40 msgr2=0x7f9eb007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.499+0000 7f9ecbb35640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9eb007af40 0x7f9eb007d400 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0005fd0 tx=0x7f9ec0009870 comp rx=0 tx=0).stop 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.499+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 msgr2=0x7f9ec419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.499+0000 7f9ecbb35640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 0x7f9ec419e500 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4031460 tx=0x7f9eb40045e0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.499+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 shutdown_connections 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.499+0000 7f9ecbb35640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9eb007af40 0x7f9eb007d400 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.499+0000 7f9ecbb35640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ec4074eb0 0x7f9ec419e500 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.499+0000 7f9ecbb35640 1 --2- 172.21.15.7:0/4108598163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ec4074590 0x7f9ec419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.500+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 >> 172.21.15.7:0/4108598163 conn(0x7f9ec40fd600 msgr2=0x7f9ec4104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:08:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.500+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 shutdown_connections 2024-09-21T05:08:50.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:08:50.500+0000 7f9ecbb35640 1 -- 172.21.15.7:0/4108598163 wait complete. 2024-09-21T05:08:50.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:50 smithi053 ceph-mon[80176]: pgmap v1023: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:50.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:50 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/596794626' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:08:50.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:50 smithi053 ceph-mon[80176]: from='client.35676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:50.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:50 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4108598163' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:08:50.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:50 smithi007 ceph-mon[108238]: pgmap v1023: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:50.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:50 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/596794626' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:08:50.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:50 smithi007 ceph-mon[108238]: from='client.35676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:08:50.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:50 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4108598163' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:08:52.878 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:52 smithi007 ceph-mon[108238]: pgmap v1024: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:52.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:52 smithi053 ceph-mon[80176]: pgmap v1024: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:54.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:54 smithi053 ceph-mon[80176]: pgmap v1025: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:54.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:54 smithi007 ceph-mon[108238]: pgmap v1025: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:56.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:56 smithi053 ceph-mon[80176]: pgmap v1026: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:56 smithi007 ceph-mon[108238]: pgmap v1026: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:58.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:58 smithi007 ceph-mon[108238]: pgmap v1027: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:58.937 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:08:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:08:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:58 smithi053 ceph-mon[80176]: pgmap v1027: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:08:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:08:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:09:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:00 smithi053 ceph-mon[80176]: pgmap v1028: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:00 smithi007 ceph-mon[108238]: pgmap v1028: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:02.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:02 smithi053 ceph-mon[80176]: pgmap v1029: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:02 smithi007 ceph-mon[108238]: pgmap v1029: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:05 smithi007 ceph-mon[108238]: pgmap v1030: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:05 smithi053 ceph-mon[80176]: pgmap v1030: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:07.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:07 smithi053 ceph-mon[80176]: pgmap v1031: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:07 smithi007 ceph-mon[108238]: pgmap v1031: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:09 smithi053 ceph-mon[80176]: pgmap v1032: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:09 smithi007 ceph-mon[108238]: pgmap v1032: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:11 smithi007 ceph-mon[108238]: pgmap v1033: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:11.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:11 smithi053 ceph-mon[80176]: pgmap v1033: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:13.341 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:13 smithi053 ceph-mon[80176]: pgmap v1034: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:13 smithi007 ceph-mon[108238]: pgmap v1034: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:14.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:09:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:09:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:15 smithi053 ceph-mon[80176]: pgmap v1035: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:15 smithi007 ceph-mon[108238]: pgmap v1035: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:17.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:17 smithi007 ceph-mon[108238]: pgmap v1036: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:17.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:17 smithi053 ceph-mon[80176]: pgmap v1036: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:19.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:19 smithi053 ceph-mon[80176]: pgmap v1037: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:19 smithi007 ceph-mon[108238]: pgmap v1037: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/347177075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0101900 msgr2=0x7f0ed0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 --2- 172.21.15.7:0/347177075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0101900 0x7f0ed0101d60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009c20 tx=0x7f0ec002a970 comp rx=0 tx=0).stop 2024-09-21T05:09:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/347177075 shutdown_connections 2024-09-21T05:09:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 --2- 172.21.15.7:0/347177075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0101900 0x7f0ed0101d60 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 --2- 172.21.15.7:0/347177075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0107900 0x7f0ed0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/347177075 >> 172.21.15.7:0/347177075 conn(0x7f0ed00fd600 msgr2=0x7f0ed00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/347177075 shutdown_connections 2024-09-21T05:09:20.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/347177075 wait complete. 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 Processor -- start 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.598+0000 7f0ed6eaf640 1 -- start start 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ed6eaf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0101900 0x7f0ed019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ed6eaf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0107900 0x7f0ed019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ed6eaf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed010edd0 con 0x7f0ed0107900 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ed6eaf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed010ec50 con 0x7f0ed0101900 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0107900 0x7f0ed019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ecffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0107900 0x7f0ed019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34562/0 (socket says 172.21.15.7:34562) 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ecffff640 1 -- 172.21.15.7:0/124663561 learned_addr learned my addr 172.21.15.7:0/124663561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:20.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ed4c4b640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0101900 0x7f0ed019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ecffff640 1 -- 172.21.15.7:0/124663561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0101900 msgr2=0x7f0ed019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ecffff640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0101900 0x7f0ed019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.599+0000 7f0ecffff640 1 -- 172.21.15.7:0/124663561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed01977a0 con 0x7f0ed0107900 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.600+0000 7f0ecffff640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0107900 0x7f0ed019db60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0006010 tx=0x7f0ec002fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.600+0000 7f0ecdffb640 1 -- 172.21.15.7:0/124663561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ec0047070 con 0x7f0ed0107900 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.600+0000 7f0ecdffb640 1 -- 172.21.15.7:0/124663561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ec0003e30 con 0x7f0ed0107900 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.600+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed0197a30 con 0x7f0ed0107900 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.600+0000 7f0ecdffb640 1 -- 172.21.15.7:0/124663561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ec0034070 con 0x7f0ed0107900 2024-09-21T05:09:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.600+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed0197f60 con 0x7f0ed0107900 2024-09-21T05:09:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.601+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed0103040 con 0x7f0ed0107900 2024-09-21T05:09:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.602+0000 7f0ecdffb640 1 -- 172.21.15.7:0/124663561 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0ec004a070 con 0x7f0ed0107900 2024-09-21T05:09:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.603+0000 7f0ecdffb640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ebc07af90 0x7f0ebc07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.603+0000 7f0ecdffb640 1 -- 172.21.15.7:0/124663561 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0ec00bec70 con 0x7f0ed0107900 2024-09-21T05:09:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.603+0000 7f0ed4c4b640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ebc07af90 0x7f0ebc07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.603+0000 7f0ed4c4b640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ebc07af90 0x7f0ebc07d450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005fd0 tx=0x7f0ec800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:20.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.607+0000 7f0ecdffb640 1 -- 172.21.15.7:0/124663561 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ec003f070 con 0x7f0ed0107900 2024-09-21T05:09:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.815+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0ed01982d0 con 0x7f0ebc07af90 2024-09-21T05:09:20.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.816+0000 7f0ecdffb640 1 -- 172.21.15.7:0/124663561 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f0ed01982d0 con 0x7f0ebc07af90 2024-09-21T05:09:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ebc07af90 msgr2=0x7f0ebc07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ebc07af90 0x7f0ebc07d450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005fd0 tx=0x7f0ec800b390 comp rx=0 tx=0).stop 2024-09-21T05:09:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0107900 msgr2=0x7f0ed019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0107900 0x7f0ed019db60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0006010 tx=0x7f0ec002fdd0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 shutdown_connections 2024-09-21T05:09:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0ebc07af90 0x7f0ebc07d450 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed0107900 0x7f0ed019db60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 --2- 172.21.15.7:0/124663561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed0101900 0x7f0ed019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 >> 172.21.15.7:0/124663561 conn(0x7f0ed00fd600 msgr2=0x7f0ed00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 shutdown_connections 2024-09-21T05:09:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.818+0000 7f0ed6eaf640 1 -- 172.21.15.7:0/124663561 wait complete. 2024-09-21T05:09:20.830 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:09:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.904+0000 7fbed89cc640 1 -- 172.21.15.7:0/864015650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4107900 msgr2=0x7fbed4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.904+0000 7fbed89cc640 1 --2- 172.21.15.7:0/864015650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4107900 0x7fbed4107ce0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009880 tx=0x7fbec402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.904+0000 7fbed89cc640 1 -- 172.21.15.7:0/864015650 shutdown_connections 2024-09-21T05:09:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.904+0000 7fbed89cc640 1 --2- 172.21.15.7:0/864015650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbed4101900 0x7fbed4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.904+0000 7fbed89cc640 1 --2- 172.21.15.7:0/864015650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4107900 0x7fbed4107ce0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.904+0000 7fbed89cc640 1 -- 172.21.15.7:0/864015650 >> 172.21.15.7:0/864015650 conn(0x7fbed40fd600 msgr2=0x7fbed40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.904+0000 7fbed89cc640 1 -- 172.21.15.7:0/864015650 shutdown_connections 2024-09-21T05:09:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.904+0000 7fbed89cc640 1 -- 172.21.15.7:0/864015650 wait complete. 2024-09-21T05:09:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed89cc640 1 Processor -- start 2024-09-21T05:09:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed89cc640 1 -- start start 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed89cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4101900 0x7fbed419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed89cc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbed4107900 0x7fbed419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed89cc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbed40762e0 con 0x7fbed4101900 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed89cc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4076160 con 0x7fbed4107900 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4101900 0x7fbed419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4101900 0x7fbed419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34582/0 (socket says 172.21.15.7:34582) 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.905+0000 7fbed259c640 1 -- 172.21.15.7:0/2722028987 learned_addr learned my addr 172.21.15.7:0/2722028987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbed1d9b640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbed4107900 0x7fbed419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbed259c640 1 -- 172.21.15.7:0/2722028987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbed4107900 msgr2=0x7fbed419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbed259c640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbed4107900 0x7fbed419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbed259c640 1 -- 172.21.15.7:0/2722028987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed4199a20 con 0x7fbed4101900 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbed259c640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4101900 0x7fbed419f860 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbec4031460 tx=0x7fbec40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbec37fe640 1 -- 172.21.15.7:0/2722028987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbec403f070 con 0x7fbed4101900 2024-09-21T05:09:20.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbec37fe640 1 -- 172.21.15.7:0/2722028987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbec4004cf0 con 0x7fbed4101900 2024-09-21T05:09:20.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed4199cb0 con 0x7fbed4101900 2024-09-21T05:09:20.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbec37fe640 1 -- 172.21.15.7:0/2722028987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbec4035030 con 0x7fbed4101900 2024-09-21T05:09:20.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.906+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed419a1e0 con 0x7fbed4101900 2024-09-21T05:09:20.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.907+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed4103040 con 0x7fbed4101900 2024-09-21T05:09:20.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.909+0000 7fbec37fe640 1 -- 172.21.15.7:0/2722028987 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbec4042070 con 0x7fbed4101900 2024-09-21T05:09:20.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.909+0000 7fbec37fe640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbebc07af80 0x7fbebc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:20.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.909+0000 7fbec37fe640 1 -- 172.21.15.7:0/2722028987 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbec40be990 con 0x7fbed4101900 2024-09-21T05:09:20.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.909+0000 7fbed1d9b640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbebc07af80 0x7fbebc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:20.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.910+0000 7fbed1d9b640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbebc07af80 0x7fbebc07d440 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbed40ffbf0 tx=0x7fbec800a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:20.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:20.914+0000 7fbec37fe640 1 -- 172.21.15.7:0/2722028987 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbec4089fb0 con 0x7fbed4101900 2024-09-21T05:09:21.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.119+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbed419af50 con 0x7fbebc07af80 2024-09-21T05:09:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.120+0000 7fbec37fe640 1 -- 172.21.15.7:0/2722028987 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fbed419af50 con 0x7fbebc07af80 2024-09-21T05:09:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbebc07af80 msgr2=0x7fbebc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbebc07af80 0x7fbebc07d440 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbed40ffbf0 tx=0x7fbec800a250 comp rx=0 tx=0).stop 2024-09-21T05:09:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4101900 msgr2=0x7fbed419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4101900 0x7fbed419f860 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbec4031460 tx=0x7fbec40048f0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 shutdown_connections 2024-09-21T05:09:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbebc07af80 0x7fbebc07d440 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbed4107900 0x7fbed419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 --2- 172.21.15.7:0/2722028987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed4101900 0x7fbed419f860 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 >> 172.21.15.7:0/2722028987 conn(0x7fbed40fd600 msgr2=0x7fbed40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 shutdown_connections 2024-09-21T05:09:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.122+0000 7fbed89cc640 1 -- 172.21.15.7:0/2722028987 wait complete. 2024-09-21T05:09:21.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:21 smithi007 ceph-mon[108238]: pgmap v1038: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.207+0000 7f0f599f5640 1 -- 172.21.15.7:0/836840148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54106800 msgr2=0x7f0f54106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.207+0000 7f0f599f5640 1 --2- 172.21.15.7:0/836840148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54106800 0x7f0f54106be0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009880 tx=0x7f0f4402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.207+0000 7f0f599f5640 1 -- 172.21.15.7:0/836840148 shutdown_connections 2024-09-21T05:09:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.207+0000 7f0f599f5640 1 --2- 172.21.15.7:0/836840148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54100800 0x7f0f54100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.207+0000 7f0f599f5640 1 --2- 172.21.15.7:0/836840148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54106800 0x7f0f54106be0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.207+0000 7f0f599f5640 1 -- 172.21.15.7:0/836840148 >> 172.21.15.7:0/836840148 conn(0x7f0f540fc520 msgr2=0x7f0f540fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.207+0000 7f0f599f5640 1 -- 172.21.15.7:0/836840148 shutdown_connections 2024-09-21T05:09:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.207+0000 7f0f599f5640 1 -- 172.21.15.7:0/836840148 wait complete. 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.208+0000 7f0f599f5640 1 Processor -- start 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.208+0000 7f0f599f5640 1 -- start start 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.208+0000 7f0f599f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54100800 0x7f0f5419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.208+0000 7f0f599f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54106800 0x7f0f5419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.208+0000 7f0f599f5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410dcd0 con 0x7f0f54100800 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.208+0000 7f0f599f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410db50 con 0x7f0f54106800 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f52ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54100800 0x7f0f5419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f52ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54100800 0x7f0f5419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34612/0 (socket says 172.21.15.7:34612) 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f52ffd640 1 -- 172.21.15.7:0/1589173023 learned_addr learned my addr 172.21.15.7:0/1589173023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f527fc640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54106800 0x7f0f5419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f52ffd640 1 -- 172.21.15.7:0/1589173023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54106800 msgr2=0x7f0f5419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f52ffd640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54106800 0x7f0f5419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f52ffd640 1 -- 172.21.15.7:0/1589173023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541977c0 con 0x7f0f54100800 2024-09-21T05:09:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f52ffd640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54100800 0x7f0f5419d640 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0f44031460 tx=0x7f0f440045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f589f3640 1 -- 172.21.15.7:0/1589173023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f4403f070 con 0x7f0f54100800 2024-09-21T05:09:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f589f3640 1 -- 172.21.15.7:0/1589173023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f440035f0 con 0x7f0f54100800 2024-09-21T05:09:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f54197a50 con 0x7f0f54100800 2024-09-21T05:09:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f589f3640 1 -- 172.21.15.7:0/1589173023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f44035030 con 0x7f0f54100800 2024-09-21T05:09:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.209+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f54197f80 con 0x7f0f54100800 2024-09-21T05:09:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.210+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f54101f40 con 0x7f0f54100800 2024-09-21T05:09:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.211+0000 7f0f589f3640 1 -- 172.21.15.7:0/1589173023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0f44042070 con 0x7f0f54100800 2024-09-21T05:09:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.212+0000 7f0f589f3640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3c07af30 0x7f0f3c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.212+0000 7f0f589f3640 1 -- 172.21.15.7:0/1589173023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0f440bea30 con 0x7f0f54100800 2024-09-21T05:09:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.212+0000 7f0f527fc640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3c07af30 0x7f0f3c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.212+0000 7f0f527fc640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3c07af30 0x7f0f3c07d3f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0f54104f50 tx=0x7f0f4800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.216+0000 7f0f589f3640 1 -- 172.21.15.7:0/1589173023 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f44089f70 con 0x7f0f54100800 2024-09-21T05:09:21.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.412+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0f541982f0 con 0x7f0f3c07af30 2024-09-21T05:09:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.421+0000 7f0f589f3640 1 -- 172.21.15.7:0/1589173023 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f0f541982f0 con 0x7f0f3c07af30 2024-09-21T05:09:21.424 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:09:21.424 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (31m) 10m ago 42m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:09:21.424 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (34m) 10m ago 42m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:09:21.424 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (34m) 10m ago 41m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (29m) 10m ago 42m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (29m) 10m ago 41m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (30m) 10m ago 42m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (25m) 10m ago 36m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (25m) 10m ago 37m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (10m) 10m ago 36m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (25m) 10m ago 36m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (33m) 10m ago 43m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (32m) 10m ago 41m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (30m) 10m ago 43m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (29m) 10m ago 41m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (32m) 10m ago 42m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:09:21.425 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (32m) 10m ago 41m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:09:21.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (29m) 10m ago 40m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:09:21.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (28m) 10m ago 39m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:09:21.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (27m) 10m ago 39m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:09:21.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (27m) 10m ago 38m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:09:21.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (26m) 10m ago 38m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:09:21.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (26m) 10m ago 38m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:09:21.426 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (31m) 10m ago 41m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.423+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3c07af30 msgr2=0x7f0f3c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3c07af30 0x7f0f3c07d3f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0f54104f50 tx=0x7f0f4800b390 comp rx=0 tx=0).stop 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54100800 msgr2=0x7f0f5419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54100800 0x7f0f5419d640 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0f44031460 tx=0x7f0f440045e0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 shutdown_connections 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3c07af30 0x7f0f3c07d3f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f54106800 0x7f0f5419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 --2- 172.21.15.7:0/1589173023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f54100800 0x7f0f5419d640 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 >> 172.21.15.7:0/1589173023 conn(0x7f0f540fc520 msgr2=0x7f0f5406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 shutdown_connections 2024-09-21T05:09:21.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.424+0000 7f0f599f5640 1 -- 172.21.15.7:0/1589173023 wait complete. 2024-09-21T05:09:21.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:21 smithi053 ceph-mon[80176]: pgmap v1038: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:21.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.506+0000 7f772c627640 1 -- 172.21.15.7:0/4030025681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 msgr2=0x7f7724101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.506+0000 7f772c627640 1 --2- 172.21.15.7:0/4030025681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 0x7f7724101d60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7714009880 tx=0x7f771402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.507+0000 7f772c627640 1 -- 172.21.15.7:0/4030025681 shutdown_connections 2024-09-21T05:09:21.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.507+0000 7f772c627640 1 --2- 172.21.15.7:0/4030025681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 0x7f7724101d60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.507+0000 7f772c627640 1 --2- 172.21.15.7:0/4030025681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724107900 0x7f7724107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.507+0000 7f772c627640 1 -- 172.21.15.7:0/4030025681 >> 172.21.15.7:0/4030025681 conn(0x7f77240fd5e0 msgr2=0x7f77240ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:21.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.507+0000 7f772c627640 1 -- 172.21.15.7:0/4030025681 shutdown_connections 2024-09-21T05:09:21.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.507+0000 7f772c627640 1 -- 172.21.15.7:0/4030025681 wait complete. 2024-09-21T05:09:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.507+0000 7f772c627640 1 Processor -- start 2024-09-21T05:09:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.507+0000 7f772c627640 1 -- start start 2024-09-21T05:09:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772c627640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 0x7f772419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772c627640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724107900 0x7f772419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772c627640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f772410edd0 con 0x7f7724101900 2024-09-21T05:09:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772c627640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f772410ec50 con 0x7f7724107900 2024-09-21T05:09:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772a3c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 0x7f772419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772a3c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 0x7f772419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34632/0 (socket says 172.21.15.7:34632) 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772a3c3640 1 -- 172.21.15.7:0/3987932856 learned_addr learned my addr 172.21.15.7:0/3987932856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f7729bc2640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724107900 0x7f772419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772a3c3640 1 -- 172.21.15.7:0/3987932856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724107900 msgr2=0x7f772419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772a3c3640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724107900 0x7f772419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772a3c3640 1 -- 172.21.15.7:0/3987932856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77241977a0 con 0x7f7724101900 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.508+0000 7f772a3c3640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 0x7f772419d620 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f772000e970 tx=0x7f772000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.509+0000 7f771b7fe640 1 -- 172.21.15.7:0/3987932856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f772000cac0 con 0x7f7724101900 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.509+0000 7f771b7fe640 1 -- 172.21.15.7:0/3987932856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77200054e0 con 0x7f7724101900 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.509+0000 7f771b7fe640 1 -- 172.21.15.7:0/3987932856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7720010430 con 0x7f7724101900 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.509+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7724197a90 con 0x7f7724101900 2024-09-21T05:09:21.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.509+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7724197f90 con 0x7f7724101900 2024-09-21T05:09:21.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.510+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7724103040 con 0x7f7724101900 2024-09-21T05:09:21.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.510+0000 7f771b7fe640 1 -- 172.21.15.7:0/3987932856 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7720004e80 con 0x7f7724101900 2024-09-21T05:09:21.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.511+0000 7f771b7fe640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f771007af40 0x7f771007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.511+0000 7f771b7fe640 1 -- 172.21.15.7:0/3987932856 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f772009e6b0 con 0x7f7724101900 2024-09-21T05:09:21.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.511+0000 7f7729bc2640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f771007af40 0x7f771007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.512+0000 7f7729bc2640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f771007af40 0x7f771007d400 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7724198af0 tx=0x7f7714031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.516+0000 7f771b7fe640 1 -- 172.21.15.7:0/3987932856 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7720069c50 con 0x7f7724101900 2024-09-21T05:09:21.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.781+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7724109e70 con 0x7f7724101900 2024-09-21T05:09:21.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.782+0000 7f771b7fe640 1 -- 172.21.15.7:0/3987932856 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f772006eb00 con 0x7f7724101900 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:09:21.785 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:09:21.786 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:09:21.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:09:21.786 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:09:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f771007af40 msgr2=0x7f771007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f771007af40 0x7f771007d400 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7724198af0 tx=0x7f7714031400 comp rx=0 tx=0).stop 2024-09-21T05:09:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 msgr2=0x7f772419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 0x7f772419d620 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f772000e970 tx=0x7f772000ed70 comp rx=0 tx=0).stop 2024-09-21T05:09:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 shutdown_connections 2024-09-21T05:09:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f771007af40 0x7f771007d400 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724107900 0x7f772419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 --2- 172.21.15.7:0/3987932856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7724101900 0x7f772419d620 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 >> 172.21.15.7:0/3987932856 conn(0x7f77240fd5e0 msgr2=0x7f77240fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:21.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 shutdown_connections 2024-09-21T05:09:21.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.784+0000 7f772c627640 1 -- 172.21.15.7:0/3987932856 wait complete. 2024-09-21T05:09:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 -- 172.21.15.7:0/1672396755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 msgr2=0x7f897c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 --2- 172.21.15.7:0/1672396755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 0x7f897c106bd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8978009880 tx=0x7f897802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 -- 172.21.15.7:0/1672396755 shutdown_connections 2024-09-21T05:09:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 --2- 172.21.15.7:0/1672396755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f897c1007f0 0x7f897c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 --2- 172.21.15.7:0/1672396755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 0x7f897c106bd0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 -- 172.21.15.7:0/1672396755 >> 172.21.15.7:0/1672396755 conn(0x7f897c0fc550 msgr2=0x7f897c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 -- 172.21.15.7:0/1672396755 shutdown_connections 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 -- 172.21.15.7:0/1672396755 wait complete. 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 Processor -- start 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.868+0000 7f89847bc640 1 -- start start 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f89847bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f897c1007f0 0x7f897c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f89847bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 0x7f897c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f89847bc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10dcc0 con 0x7f897c1067f0 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f89847bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10db40 con 0x7f897c1007f0 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f8981d57640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 0x7f897c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f8981d57640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 0x7f897c19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34658/0 (socket says 172.21.15.7:34658) 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f8981d57640 1 -- 172.21.15.7:0/865946480 learned_addr learned my addr 172.21.15.7:0/865946480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f8982558640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f897c1007f0 0x7f897c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f8981d57640 1 -- 172.21.15.7:0/865946480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f897c1007f0 msgr2=0x7f897c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f8981d57640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f897c1007f0 0x7f897c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f8981d57640 1 -- 172.21.15.7:0/865946480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c1977b0 con 0x7f897c1067f0 2024-09-21T05:09:21.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f8981d57640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 0x7f897c19db70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f896c00d6f0 tx=0x7f896c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.869+0000 7f89737fe640 1 -- 172.21.15.7:0/865946480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f896c017070 con 0x7f897c1067f0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.870+0000 7f89737fe640 1 -- 172.21.15.7:0/865946480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f896c00de00 con 0x7f897c1067f0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.870+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c197aa0 con 0x7f897c1067f0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.870+0000 7f89737fe640 1 -- 172.21.15.7:0/865946480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f896c004bb0 con 0x7f897c1067f0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.870+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c198030 con 0x7f897c1067f0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.871+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c101f30 con 0x7f897c1067f0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.871+0000 7f89737fe640 1 -- 172.21.15.7:0/865946480 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f896c004e60 con 0x7f897c1067f0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.872+0000 7f89737fe640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f896807af30 0x7f896807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.872+0000 7f89737fe640 1 -- 172.21.15.7:0/865946480 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f896c09dcb0 con 0x7f897c1067f0 2024-09-21T05:09:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.872+0000 7f8982558640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f896807af30 0x7f896807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.872+0000 7f8982558640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f896807af30 0x7f896807d3f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f89780096c0 tx=0x7f8978031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:21.876+0000 7f89737fe640 1 -- 172.21.15.7:0/865946480 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f896c069250 con 0x7f897c1067f0 2024-09-21T05:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.105+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f897c198ce0 con 0x7f897c1067f0 2024-09-21T05:09:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.105+0000 7f89737fe640 1 -- 172.21.15.7:0/865946480 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f896c06e100 con 0x7f897c1067f0 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:09:22.109 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:09:22.110 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:09:22.111 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:09:22.112 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:09:22.112 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:09:22.112 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:09:22.112 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:09:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f896807af30 msgr2=0x7f896807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f896807af30 0x7f896807d3f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f89780096c0 tx=0x7f8978031400 comp rx=0 tx=0).stop 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 msgr2=0x7f897c19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 0x7f897c19db70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f896c00d6f0 tx=0x7f896c00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 shutdown_connections 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f896807af30 0x7f896807d3f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f897c1067f0 0x7f897c19db70 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 --2- 172.21.15.7:0/865946480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f897c1007f0 0x7f897c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 >> 172.21.15.7:0/865946480 conn(0x7f897c0fc550 msgr2=0x7f897c069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 shutdown_connections 2024-09-21T05:09:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.108+0000 7f89847bc640 1 -- 172.21.15.7:0/865946480 wait complete. 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.194+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/1610904900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c4102ac0 msgr2=0x7fc9c4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.194+0000 7fc9cbf5c640 1 --2- 172.21.15.7:0/1610904900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c4102ac0 0x7fc9c4102ea0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009880 tx=0x7fc9c002a530 comp rx=0 tx=0).stop 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.194+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/1610904900 shutdown_connections 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.194+0000 7fc9cbf5c640 1 --2- 172.21.15.7:0/1610904900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c41033e0 0x7fc9c410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.194+0000 7fc9cbf5c640 1 --2- 172.21.15.7:0/1610904900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c4102ac0 0x7fc9c4102ea0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.194+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/1610904900 >> 172.21.15.7:0/1610904900 conn(0x7fc9c40fc520 msgr2=0x7fc9c40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.194+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/1610904900 shutdown_connections 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.194+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/1610904900 wait complete. 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.195+0000 7fc9cbf5c640 1 Processor -- start 2024-09-21T05:09:22.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.195+0000 7fc9cbf5c640 1 -- start start 2024-09-21T05:09:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.195+0000 7fc9cbf5c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c4102ac0 0x7fc9c419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.195+0000 7fc9cbf5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c41033e0 0x7fc9c419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.195+0000 7fc9cbf5c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c410edd0 con 0x7fc9c41033e0 2024-09-21T05:09:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.195+0000 7fc9cbf5c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c410ec50 con 0x7fc9c4102ac0 2024-09-21T05:09:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.195+0000 7fc9c94f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c41033e0 0x7fc9c419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9c94f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c41033e0 0x7fc9c419dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34682/0 (socket says 172.21.15.7:34682) 2024-09-21T05:09:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9c94f7640 1 -- 172.21.15.7:0/3031351996 learned_addr learned my addr 172.21.15.7:0/3031351996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9c9cf8640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c4102ac0 0x7fc9c419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9c94f7640 1 -- 172.21.15.7:0/3031351996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c4102ac0 msgr2=0x7fc9c419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9c94f7640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c4102ac0 0x7fc9c419d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9c94f7640 1 -- 172.21.15.7:0/3031351996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c419e310 con 0x7fc9c41033e0 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9c94f7640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c41033e0 0x7fc9c419dc60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c8f0 tx=0x7fc9b400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9baffd640 1 -- 172.21.15.7:0/3031351996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc9b40079b0 con 0x7fc9c41033e0 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c4197a00 con 0x7fc9c41033e0 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9baffd640 1 -- 172.21.15.7:0/3031351996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc9b4007b50 con 0x7fc9c41033e0 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9baffd640 1 -- 172.21.15.7:0/3031351996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc9b4018610 con 0x7fc9c41033e0 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.196+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c4197f60 con 0x7fc9c41033e0 2024-09-21T05:09:22.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.197+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c406aa90 con 0x7fc9c41033e0 2024-09-21T05:09:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.198+0000 7fc9baffd640 1 -- 172.21.15.7:0/3031351996 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc9b4007d00 con 0x7fc9c41033e0 2024-09-21T05:09:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.199+0000 7fc9baffd640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9b007af30 0x7fc9b007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.199+0000 7fc9c9cf8640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9b007af30 0x7fc9b007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.199+0000 7fc9baffd640 1 -- 172.21.15.7:0/3031351996 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc9b409e1f0 con 0x7fc9c41033e0 2024-09-21T05:09:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.199+0000 7fc9c9cf8640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9b007af30 0x7fc9b007d3f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000c00 tx=0x7fc9c0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:22.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.203+0000 7fc9baffd640 1 -- 172.21.15.7:0/3031351996 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9b4014030 con 0x7fc9c41033e0 2024-09-21T05:09:22.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.408+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc9c4198e10 con 0x7fc9b007af30 2024-09-21T05:09:22.411 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:22 smithi007 ceph-mon[108238]: from='client.35684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:22.411 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:22 smithi007 ceph-mon[108238]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:22.411 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:22 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3987932856' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:09:22.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.409+0000 7fc9baffd640 1 -- 172.21.15.7:0/3031351996 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc9c4198e10 con 0x7fc9b007af30 2024-09-21T05:09:22.412 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:09:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:09:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:09:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:09:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:09:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:09:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:09:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:09:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:09:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:09:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:09:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:09:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:09:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:09:22.413 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9b007af30 msgr2=0x7fc9b007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9b007af30 0x7fc9b007d3f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000c00 tx=0x7fc9c0031400 comp rx=0 tx=0).stop 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c41033e0 msgr2=0x7fc9c419dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c41033e0 0x7fc9c419dc60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c8f0 tx=0x7fc9b400cdb0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 shutdown_connections 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc9b007af30 0x7fc9b007d3f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9c41033e0 0x7fc9c419dc60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 --2- 172.21.15.7:0/3031351996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9c4102ac0 0x7fc9c419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 >> 172.21.15.7:0/3031351996 conn(0x7fc9c40fc520 msgr2=0x7fc9c4108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 shutdown_connections 2024-09-21T05:09:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.412+0000 7fc9cbf5c640 1 -- 172.21.15.7:0/3031351996 wait complete. 2024-09-21T05:09:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:22 smithi053 ceph-mon[80176]: from='client.35684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:22 smithi053 ceph-mon[80176]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:22.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:22 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3987932856' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:09:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.495+0000 7f87d0c83640 1 -- 172.21.15.7:0/197748078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 msgr2=0x7f87cc106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.495+0000 7f87d0c83640 1 --2- 172.21.15.7:0/197748078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 0x7f87cc106bd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009c20 tx=0x7f87bc02a970 comp rx=0 tx=0).stop 2024-09-21T05:09:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.496+0000 7f87d0c83640 1 -- 172.21.15.7:0/197748078 shutdown_connections 2024-09-21T05:09:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.496+0000 7f87d0c83640 1 --2- 172.21.15.7:0/197748078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87cc1007f0 0x7f87cc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.496+0000 7f87d0c83640 1 --2- 172.21.15.7:0/197748078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 0x7f87cc106bd0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.496+0000 7f87d0c83640 1 -- 172.21.15.7:0/197748078 >> 172.21.15.7:0/197748078 conn(0x7f87cc0fc550 msgr2=0x7f87cc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.496+0000 7f87d0c83640 1 -- 172.21.15.7:0/197748078 shutdown_connections 2024-09-21T05:09:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.496+0000 7f87d0c83640 1 -- 172.21.15.7:0/197748078 wait complete. 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.496+0000 7f87d0c83640 1 Processor -- start 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.496+0000 7f87d0c83640 1 -- start start 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87d0c83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87cc1007f0 0x7f87cc0fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87d0c83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 0x7f87cc0fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87d0c83640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc10dcc0 con 0x7f87cc1067f0 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87d0c83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc10db40 con 0x7f87cc1007f0 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 0x7f87cc0fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 0x7f87cc0fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34706/0 (socket says 172.21.15.7:34706) 2024-09-21T05:09:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c9d9b640 1 -- 172.21.15.7:0/2937262364 learned_addr learned my addr 172.21.15.7:0/2937262364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87ca59c640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87cc1007f0 0x7f87cc0fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c9d9b640 1 -- 172.21.15.7:0/2937262364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87cc1007f0 msgr2=0x7f87cc0fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c9d9b640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87cc1007f0 0x7f87cc0fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c9d9b640 1 -- 172.21.15.7:0/2937262364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc0fd7b0 con 0x7f87cc1067f0 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c9d9b640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 0x7f87cc0fd270 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f87b400d6f0 tx=0x7f87b400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c37fe640 1 -- 172.21.15.7:0/2937262364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f87b4017070 con 0x7f87cc1067f0 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.497+0000 7f87c37fe640 1 -- 172.21.15.7:0/2937262364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87b400de00 con 0x7f87cc1067f0 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.498+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc0fdaa0 con 0x7f87cc1067f0 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.498+0000 7f87c37fe640 1 -- 172.21.15.7:0/2937262364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f87b4004bb0 con 0x7f87cc1067f0 2024-09-21T05:09:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.498+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc1a3e30 con 0x7f87cc1067f0 2024-09-21T05:09:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.499+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87cc101f30 con 0x7f87cc1067f0 2024-09-21T05:09:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.499+0000 7f87c37fe640 1 -- 172.21.15.7:0/2937262364 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f87b4009a30 con 0x7f87cc1067f0 2024-09-21T05:09:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.500+0000 7f87c37fe640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87b807af80 0x7f87b807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.500+0000 7f87ca59c640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87b807af80 0x7f87b807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.500+0000 7f87c37fe640 1 -- 172.21.15.7:0/2937262364 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f87b409dbe0 con 0x7f87cc1067f0 2024-09-21T05:09:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.501+0000 7f87ca59c640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87b807af80 0x7f87b807d440 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f87bc005fc0 tx=0x7f87bc005f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:22.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.504+0000 7f87c37fe640 1 -- 172.21.15.7:0/2937262364 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87b4069200 con 0x7f87cc1067f0 2024-09-21T05:09:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.769+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f87cc100c50 con 0x7f87cc1067f0 2024-09-21T05:09:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.770+0000 7f87c37fe640 1 -- 172.21.15.7:0/2937262364 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f87b405ee20 con 0x7f87cc1067f0 2024-09-21T05:09:22.773 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:09:22.773 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:09:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:09:22.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:09:22.773 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:09:22.773 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:09:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:09:22.774 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:09:22.775 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:09:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:09:22.775 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:09:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:09:22.775 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:09:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.772+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87b807af80 msgr2=0x7f87b807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.772+0000 7f87d0c83640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87b807af80 0x7f87b807d440 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f87bc005fc0 tx=0x7f87bc005f30 comp rx=0 tx=0).stop 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.772+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 msgr2=0x7f87cc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.772+0000 7f87d0c83640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 0x7f87cc0fd270 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f87b400d6f0 tx=0x7f87b400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.773+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 shutdown_connections 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.773+0000 7f87d0c83640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f87b807af80 0x7f87b807d440 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.773+0000 7f87d0c83640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87cc1067f0 0x7f87cc0fd270 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.773+0000 7f87d0c83640 1 --2- 172.21.15.7:0/2937262364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87cc1007f0 0x7f87cc0fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.773+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 >> 172.21.15.7:0/2937262364 conn(0x7f87cc0fc550 msgr2=0x7f87cc069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.773+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 shutdown_connections 2024-09-21T05:09:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:22.773+0000 7f87d0c83640 1 -- 172.21.15.7:0/2937262364 wait complete. 2024-09-21T05:09:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:23 smithi053 ceph-mon[80176]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:23 smithi053 ceph-mon[80176]: pgmap v1039: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/865946480' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:09:23.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2937262364' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:09:23.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:23 smithi007 ceph-mon[108238]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:23.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:23 smithi007 ceph-mon[108238]: pgmap v1039: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:23.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/865946480' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:09:23.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2937262364' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:09:24.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:24 smithi053 ceph-mon[80176]: from='client.35704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:24.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:24 smithi007 ceph-mon[108238]: from='client.35704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:25.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:25 smithi053 ceph-mon[80176]: pgmap v1040: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:25 smithi007 ceph-mon[108238]: pgmap v1040: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:27.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:27 smithi053 ceph-mon[80176]: pgmap v1041: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:27 smithi007 ceph-mon[108238]: pgmap v1041: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:29 smithi007 ceph-mon[108238]: pgmap v1042: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:09:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:29 smithi053 ceph-mon[80176]: pgmap v1042: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:09:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:31 smithi053 ceph-mon[80176]: pgmap v1043: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:31 smithi007 ceph-mon[108238]: pgmap v1043: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:33 smithi053 ceph-mon[80176]: pgmap v1044: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:33 smithi007 ceph-mon[108238]: pgmap v1044: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:35 smithi007 ceph-mon[108238]: pgmap v1045: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:35.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:35 smithi053 ceph-mon[80176]: pgmap v1045: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:37 smithi053 ceph-mon[80176]: pgmap v1046: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:37 smithi007 ceph-mon[108238]: pgmap v1046: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:39 smithi053 ceph-mon[80176]: pgmap v1047: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:39 smithi007 ceph-mon[108238]: pgmap v1047: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:41 smithi007 ceph-mon[108238]: pgmap v1048: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:41.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:41 smithi053 ceph-mon[80176]: pgmap v1048: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:43 smithi053 ceph-mon[80176]: pgmap v1049: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:43 smithi007 ceph-mon[108238]: pgmap v1049: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:09:44.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:09:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:45 smithi053 ceph-mon[80176]: pgmap v1050: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:45 smithi007 ceph-mon[108238]: pgmap v1050: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:47 smithi007 ceph-mon[108238]: pgmap v1051: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:47 smithi053 ceph-mon[80176]: pgmap v1051: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:49.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:49 smithi053 ceph-mon[80176]: pgmap v1052: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:49 smithi007 ceph-mon[108238]: pgmap v1052: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:51.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:51 smithi053 ceph-mon[80176]: pgmap v1053: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:51 smithi007 ceph-mon[108238]: pgmap v1053: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:52.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.869+0000 7fca37094640 1 -- 172.21.15.7:0/1548638618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30103430 msgr2=0x7fca3010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:52.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.869+0000 7fca37094640 1 --2- 172.21.15.7:0/1548638618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30103430 0x7fca3010b920 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fca240098b0 tx=0x7fca2402a560 comp rx=0 tx=0).stop 2024-09-21T05:09:52.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.869+0000 7fca37094640 1 -- 172.21.15.7:0/1548638618 shutdown_connections 2024-09-21T05:09:52.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.869+0000 7fca37094640 1 --2- 172.21.15.7:0/1548638618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30103430 0x7fca3010b920 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:52.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.869+0000 7fca37094640 1 --2- 172.21.15.7:0/1548638618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca30102b10 0x7fca30102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:52.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.869+0000 7fca37094640 1 -- 172.21.15.7:0/1548638618 >> 172.21.15.7:0/1548638618 conn(0x7fca300fc550 msgr2=0x7fca300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:52.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.869+0000 7fca37094640 1 -- 172.21.15.7:0/1548638618 shutdown_connections 2024-09-21T05:09:52.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.869+0000 7fca37094640 1 -- 172.21.15.7:0/1548638618 wait complete. 2024-09-21T05:09:52.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca37094640 1 Processor -- start 2024-09-21T05:09:52.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca37094640 1 -- start start 2024-09-21T05:09:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca37094640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30102b10 0x7fca3019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca37094640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca30103430 0x7fca3019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca37094640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca3010ede0 con 0x7fca30102b10 2024-09-21T05:09:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca37094640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca3010ec60 con 0x7fca30103430 2024-09-21T05:09:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca34e30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30102b10 0x7fca3019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:52.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca34e30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30102b10 0x7fca3019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60586/0 (socket says 172.21.15.7:60586) 2024-09-21T05:09:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca34e30640 1 -- 172.21.15.7:0/3496798589 learned_addr learned my addr 172.21.15.7:0/3496798589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.870+0000 7fca2ffff640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca30103430 0x7fca3019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca34e30640 1 -- 172.21.15.7:0/3496798589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca30103430 msgr2=0x7fca3019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca34e30640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca30103430 0x7fca3019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca34e30640 1 -- 172.21.15.7:0/3496798589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca301977f0 con 0x7fca30102b10 2024-09-21T05:09:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca34e30640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30102b10 0x7fca3019d670 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d6f0 tx=0x7fca2000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:52.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca2dffb640 1 -- 172.21.15.7:0/3496798589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fca20017070 con 0x7fca30102b10 2024-09-21T05:09:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca2dffb640 1 -- 172.21.15.7:0/3496798589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca2000de00 con 0x7fca30102b10 2024-09-21T05:09:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca30197ae0 con 0x7fca30102b10 2024-09-21T05:09:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca2dffb640 1 -- 172.21.15.7:0/3496798589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fca20004bb0 con 0x7fca30102b10 2024-09-21T05:09:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.871+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca30198040 con 0x7fca30102b10 2024-09-21T05:09:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.872+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3006a520 con 0x7fca30102b10 2024-09-21T05:09:52.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.873+0000 7fca2dffb640 1 -- 172.21.15.7:0/3496798589 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fca20009a30 con 0x7fca30102b10 2024-09-21T05:09:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.873+0000 7fca2dffb640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af80 0x7fca1807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.874+0000 7fca2dffb640 1 -- 172.21.15.7:0/3496798589 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fca2009dbd0 con 0x7fca30102b10 2024-09-21T05:09:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.874+0000 7fca2ffff640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af80 0x7fca1807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.874+0000 7fca2ffff640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af80 0x7fca1807d440 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fca24000c00 tx=0x7fca24031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:52.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:52.878+0000 7fca2dffb640 1 -- 172.21.15.7:0/3496798589 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fca200691f0 con 0x7fca30102b10 2024-09-21T05:09:53.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.095+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fca301982c0 con 0x7fca1807af80 2024-09-21T05:09:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.097+0000 7fca2dffb640 1 -- 172.21.15.7:0/3496798589 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fca301982c0 con 0x7fca1807af80 2024-09-21T05:09:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af80 msgr2=0x7fca1807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af80 0x7fca1807d440 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fca24000c00 tx=0x7fca24031400 comp rx=0 tx=0).stop 2024-09-21T05:09:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30102b10 msgr2=0x7fca3019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30102b10 0x7fca3019d670 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d6f0 tx=0x7fca2000dbb0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 shutdown_connections 2024-09-21T05:09:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fca1807af80 0x7fca1807d440 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca30103430 0x7fca3019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 --2- 172.21.15.7:0/3496798589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca30102b10 0x7fca3019d670 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 >> 172.21.15.7:0/3496798589 conn(0x7fca300fc550 msgr2=0x7fca30103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 shutdown_connections 2024-09-21T05:09:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.099+0000 7fca37094640 1 -- 172.21.15.7:0/3496798589 wait complete. 2024-09-21T05:09:53.112 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:09:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.187+0000 7f942d769640 1 -- 172.21.15.7:0/3520289508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428107900 msgr2=0x7f9428107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.187+0000 7f942d769640 1 --2- 172.21.15.7:0/3520289508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428107900 0x7f9428107ce0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9418009880 tx=0x7f941802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.187+0000 7f942d769640 1 -- 172.21.15.7:0/3520289508 shutdown_connections 2024-09-21T05:09:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.187+0000 7f942d769640 1 --2- 172.21.15.7:0/3520289508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9428101900 0x7f9428101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.187+0000 7f942d769640 1 --2- 172.21.15.7:0/3520289508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428107900 0x7f9428107ce0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.187+0000 7f942d769640 1 -- 172.21.15.7:0/3520289508 >> 172.21.15.7:0/3520289508 conn(0x7f94280fd600 msgr2=0x7f94280ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.187+0000 7f942d769640 1 -- 172.21.15.7:0/3520289508 shutdown_connections 2024-09-21T05:09:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.187+0000 7f942d769640 1 -- 172.21.15.7:0/3520289508 wait complete. 2024-09-21T05:09:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f942d769640 1 Processor -- start 2024-09-21T05:09:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f942d769640 1 -- start start 2024-09-21T05:09:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f942d769640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428101900 0x7f94280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f942d769640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9428107900 0x7f9428074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f942d769640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f942810edd0 con 0x7f9428101900 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f942d769640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f942810ec50 con 0x7f9428107900 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f9426ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428101900 0x7f94280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f9426ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428101900 0x7f94280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60616/0 (socket says 172.21.15.7:60616) 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.188+0000 7f9426ffd640 1 -- 172.21.15.7:0/2960312803 learned_addr learned my addr 172.21.15.7:0/2960312803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f94267fc640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9428107900 0x7f9428074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f9426ffd640 1 -- 172.21.15.7:0/2960312803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9428107900 msgr2=0x7f9428074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f9426ffd640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9428107900 0x7f9428074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f9426ffd640 1 -- 172.21.15.7:0/2960312803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9428074f70 con 0x7f9428101900 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f94267fc640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9428107900 0x7f9428074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f9426ffd640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428101900 0x7f94280744f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9418031460 tx=0x7f94180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f940ffff640 1 -- 172.21.15.7:0/2960312803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f941803f070 con 0x7f9428101900 2024-09-21T05:09:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9428075200 con 0x7f9428101900 2024-09-21T05:09:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f940ffff640 1 -- 172.21.15.7:0/2960312803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9418004cf0 con 0x7f9428101900 2024-09-21T05:09:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f940ffff640 1 -- 172.21.15.7:0/2960312803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9418035030 con 0x7f9428101900 2024-09-21T05:09:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.189+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94281a3d80 con 0x7f9428101900 2024-09-21T05:09:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.190+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9428103040 con 0x7f9428101900 2024-09-21T05:09:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.191+0000 7f940ffff640 1 -- 172.21.15.7:0/2960312803 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9418042070 con 0x7f9428101900 2024-09-21T05:09:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.191+0000 7f940ffff640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f941007af30 0x7f941007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.191+0000 7f940ffff640 1 -- 172.21.15.7:0/2960312803 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f94180beb20 con 0x7f9428101900 2024-09-21T05:09:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.192+0000 7f94267fc640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f941007af30 0x7f941007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.192+0000 7f94267fc640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f941007af30 0x7f941007d3f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9428075f80 tx=0x7f941c005d30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.197+0000 7f940ffff640 1 -- 172.21.15.7:0/2960312803 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f941808a0c0 con 0x7f9428101900 2024-09-21T05:09:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.402+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9428062630 con 0x7f941007af30 2024-09-21T05:09:53.406 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:53 smithi007 ceph-mon[108238]: pgmap v1054: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.403+0000 7f940ffff640 1 -- 172.21.15.7:0/2960312803 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f9428062630 con 0x7f941007af30 2024-09-21T05:09:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f941007af30 msgr2=0x7f941007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f941007af30 0x7f941007d3f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9428075f80 tx=0x7f941c005d30 comp rx=0 tx=0).stop 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428101900 msgr2=0x7f94280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428101900 0x7f94280744f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9418031460 tx=0x7f94180048f0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 shutdown_connections 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f941007af30 0x7f941007d3f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9428107900 0x7f9428074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 --2- 172.21.15.7:0/2960312803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9428101900 0x7f94280744f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.405+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 >> 172.21.15.7:0/2960312803 conn(0x7f94280fd600 msgr2=0x7f94280fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.406+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 shutdown_connections 2024-09-21T05:09:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.406+0000 7f942d769640 1 -- 172.21.15.7:0/2960312803 wait complete. 2024-09-21T05:09:53.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:53 smithi053 ceph-mon[80176]: pgmap v1054: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:53.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 -- 172.21.15.7:0/325522690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 msgr2=0x7f36bc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 --2- 172.21.15.7:0/325522690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 0x7f36bc107ce0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009880 tx=0x7f36ac02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 -- 172.21.15.7:0/325522690 shutdown_connections 2024-09-21T05:09:53.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 --2- 172.21.15.7:0/325522690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36bc101900 0x7f36bc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 --2- 172.21.15.7:0/325522690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 0x7f36bc107ce0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 -- 172.21.15.7:0/325522690 >> 172.21.15.7:0/325522690 conn(0x7f36bc0fd600 msgr2=0x7f36bc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 -- 172.21.15.7:0/325522690 shutdown_connections 2024-09-21T05:09:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 -- 172.21.15.7:0/325522690 wait complete. 2024-09-21T05:09:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 Processor -- start 2024-09-21T05:09:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.490+0000 7f36c279f640 1 -- start start 2024-09-21T05:09:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36c279f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36bc101900 0x7f36bc19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36c279f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 0x7f36bc19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36c279f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc10edd0 con 0x7f36bc107900 2024-09-21T05:09:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36c279f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc10ec50 con 0x7f36bc101900 2024-09-21T05:09:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 0x7f36bc19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 0x7f36bc19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36024/0 (socket says 172.21.15.7:36024) 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36bb7fe640 1 -- 172.21.15.7:0/1176892501 learned_addr learned my addr 172.21.15.7:0/1176892501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36bbfff640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36bc101900 0x7f36bc19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36bb7fe640 1 -- 172.21.15.7:0/1176892501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36bc101900 msgr2=0x7f36bc19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36bb7fe640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36bc101900 0x7f36bc19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.491+0000 7f36bb7fe640 1 -- 172.21.15.7:0/1176892501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc1977a0 con 0x7f36bc107900 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.492+0000 7f36bb7fe640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 0x7f36bc19db60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f36b000d6f0 tx=0x7f36b000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.492+0000 7f36b97fa640 1 -- 172.21.15.7:0/1176892501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f36b0017070 con 0x7f36bc107900 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.492+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc197a90 con 0x7f36bc107900 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.492+0000 7f36b97fa640 1 -- 172.21.15.7:0/1176892501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f36b000de00 con 0x7f36bc107900 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.492+0000 7f36b97fa640 1 -- 172.21.15.7:0/1176892501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f36b0004bb0 con 0x7f36bc107900 2024-09-21T05:09:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.492+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc198020 con 0x7f36bc107900 2024-09-21T05:09:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.493+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc103040 con 0x7f36bc107900 2024-09-21T05:09:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.493+0000 7f36b97fa640 1 -- 172.21.15.7:0/1176892501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f36b0009a30 con 0x7f36bc107900 2024-09-21T05:09:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.494+0000 7f36b97fa640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f36a407af30 0x7f36a407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.494+0000 7f36b97fa640 1 -- 172.21.15.7:0/1176892501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f36b009dcb0 con 0x7f36bc107900 2024-09-21T05:09:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.494+0000 7f36bbfff640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f36a407af30 0x7f36a407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.495+0000 7f36bbfff640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f36a407af30 0x7f36a407d3f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0096c0 tx=0x7f36ac031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:53.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.498+0000 7f36b97fa640 1 -- 172.21.15.7:0/1176892501 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f36b00691b0 con 0x7f36bc107900 2024-09-21T05:09:53.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.693+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f36bc1982d0 con 0x7f36a407af30 2024-09-21T05:09:53.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.704+0000 7f36b97fa640 1 -- 172.21.15.7:0/1176892501 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f36bc1982d0 con 0x7f36a407af30 2024-09-21T05:09:53.708 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:09:53.708 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (31m) 10m ago 42m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:09:53.708 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (35m) 10m ago 43m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:09:53.708 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (34m) 10m ago 42m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:09:53.708 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (30m) 10m ago 43m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:09:53.708 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (30m) 10m ago 42m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:09:53.708 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (31m) 10m ago 42m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (26m) 10m ago 37m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (26m) 10m ago 37m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (10m) 10m ago 37m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (25m) 10m ago 37m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (34m) 10m ago 44m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (33m) 10m ago 42m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (30m) 10m ago 44m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (30m) 10m ago 41m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (32m) 10m ago 42m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (32m) 10m ago 42m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (29m) 10m ago 40m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (29m) 10m ago 40m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:09:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (28m) 10m ago 39m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:09:53.710 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (27m) 10m ago 39m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:09:53.710 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (27m) 10m ago 39m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:09:53.710 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (26m) 10m ago 38m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:09:53.710 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (32m) 10m ago 42m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:09:53.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f36a407af30 msgr2=0x7f36a407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f36a407af30 0x7f36a407d3f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0096c0 tx=0x7f36ac031400 comp rx=0 tx=0).stop 2024-09-21T05:09:53.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 msgr2=0x7f36bc19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 0x7f36bc19db60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f36b000d6f0 tx=0x7f36b000dbb0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 shutdown_connections 2024-09-21T05:09:53.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f36a407af30 0x7f36a407d3f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36bc107900 0x7f36bc19db60 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 --2- 172.21.15.7:0/1176892501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36bc101900 0x7f36bc19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 >> 172.21.15.7:0/1176892501 conn(0x7f36bc0fd600 msgr2=0x7f36bc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:53.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 shutdown_connections 2024-09-21T05:09:53.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.706+0000 7f36c279f640 1 -- 172.21.15.7:0/1176892501 wait complete. 2024-09-21T05:09:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.788+0000 7fe5d98c2640 1 -- 172.21.15.7:0/132873496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d4102ab0 msgr2=0x7fe5d4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.788+0000 7fe5d98c2640 1 --2- 172.21.15.7:0/132873496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d4102ab0 0x7fe5d4102e90 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009880 tx=0x7fe5c402a530 comp rx=0 tx=0).stop 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.788+0000 7fe5d98c2640 1 -- 172.21.15.7:0/132873496 shutdown_connections 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.788+0000 7fe5d98c2640 1 --2- 172.21.15.7:0/132873496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5d41033d0 0x7fe5d410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.788+0000 7fe5d98c2640 1 --2- 172.21.15.7:0/132873496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d4102ab0 0x7fe5d4102e90 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.788+0000 7fe5d98c2640 1 -- 172.21.15.7:0/132873496 >> 172.21.15.7:0/132873496 conn(0x7fe5d40fc550 msgr2=0x7fe5d40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.789+0000 7fe5d98c2640 1 -- 172.21.15.7:0/132873496 shutdown_connections 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.789+0000 7fe5d98c2640 1 -- 172.21.15.7:0/132873496 wait complete. 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.789+0000 7fe5d98c2640 1 Processor -- start 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.789+0000 7fe5d98c2640 1 -- start start 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d98c2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5d4102ab0 0x7fe5d419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d98c2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d41033d0 0x7fe5d419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d98c2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410edc0 con 0x7fe5d41033d0 2024-09-21T05:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d98c2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410ec40 con 0x7fe5d4102ab0 2024-09-21T05:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d41033d0 0x7fe5d419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d41033d0 0x7fe5d419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36034/0 (socket says 172.21.15.7:36034) 2024-09-21T05:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d27fc640 1 -- 172.21.15.7:0/1037799847 learned_addr learned my addr 172.21.15.7:0/1037799847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d2ffd640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5d4102ab0 0x7fe5d419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d27fc640 1 -- 172.21.15.7:0/1037799847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5d4102ab0 msgr2=0x7fe5d419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d27fc640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5d4102ab0 0x7fe5d419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d27fc640 1 -- 172.21.15.7:0/1037799847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d41977b0 con 0x7fe5d41033d0 2024-09-21T05:09:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d27fc640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d41033d0 0x7fe5d419db70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c8c0 tx=0x7fe5bc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d88c0640 1 -- 172.21.15.7:0/1037799847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe5bc004020 con 0x7fe5d41033d0 2024-09-21T05:09:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d88c0640 1 -- 172.21.15.7:0/1037799847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe5bc0041c0 con 0x7fe5d41033d0 2024-09-21T05:09:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d88c0640 1 -- 172.21.15.7:0/1037799847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe5bc0047a0 con 0x7fe5d41033d0 2024-09-21T05:09:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.790+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d4197aa0 con 0x7fe5d41033d0 2024-09-21T05:09:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.791+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d4198030 con 0x7fe5d41033d0 2024-09-21T05:09:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.791+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d406a580 con 0x7fe5d41033d0 2024-09-21T05:09:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.792+0000 7fe5d88c0640 1 -- 172.21.15.7:0/1037799847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe5bc0079b0 con 0x7fe5d41033d0 2024-09-21T05:09:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.793+0000 7fe5d88c0640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe5c007af80 0x7fe5c007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.793+0000 7fe5d88c0640 1 -- 172.21.15.7:0/1037799847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fe5bc09dd80 con 0x7fe5d41033d0 2024-09-21T05:09:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.793+0000 7fe5d2ffd640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe5c007af80 0x7fe5c007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.793+0000 7fe5d2ffd640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe5c007af80 0x7fe5c007d440 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000c00 tx=0x7fe5c4031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:53.797+0000 7fe5d88c0640 1 -- 172.21.15.7:0/1037799847 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5bc014030 con 0x7fe5d41033d0 2024-09-21T05:09:54.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.058+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fe5d4105f80 con 0x7fe5d41033d0 2024-09-21T05:09:54.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.058+0000 7fe5d88c0640 1 -- 172.21.15.7:0/1037799847 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fe5bc0693a0 con 0x7fe5d41033d0 2024-09-21T05:09:54.062 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:09:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:09:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:09:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:09:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:09:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:09:54.064 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:09:54.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.060+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe5c007af80 msgr2=0x7fe5c007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.060+0000 7fe5d98c2640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe5c007af80 0x7fe5c007d440 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000c00 tx=0x7fe5c4031400 comp rx=0 tx=0).stop 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.060+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d41033d0 msgr2=0x7fe5d419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.060+0000 7fe5d98c2640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d41033d0 0x7fe5d419db70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c8c0 tx=0x7fe5bc00cd80 comp rx=0 tx=0).stop 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.060+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 shutdown_connections 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.061+0000 7fe5d98c2640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe5c007af80 0x7fe5c007d440 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.061+0000 7fe5d98c2640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d41033d0 0x7fe5d419db70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.061+0000 7fe5d98c2640 1 --2- 172.21.15.7:0/1037799847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5d4102ab0 0x7fe5d419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.061+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 >> 172.21.15.7:0/1037799847 conn(0x7fe5d40fc550 msgr2=0x7fe5d4108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.061+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 shutdown_connections 2024-09-21T05:09:54.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.061+0000 7fe5d98c2640 1 -- 172.21.15.7:0/1037799847 wait complete. 2024-09-21T05:09:54.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.146+0000 7fd10e05a640 1 -- 172.21.15.7:0/1485166093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108101910 msgr2=0x7fd108101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.146+0000 7fd10e05a640 1 --2- 172.21.15.7:0/1485166093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108101910 0x7fd108101d70 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009880 tx=0x7fd0fc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.146+0000 7fd10e05a640 1 -- 172.21.15.7:0/1485166093 shutdown_connections 2024-09-21T05:09:54.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.146+0000 7fd10e05a640 1 --2- 172.21.15.7:0/1485166093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108101910 0x7fd108101d70 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.146+0000 7fd10e05a640 1 --2- 172.21.15.7:0/1485166093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd108107910 0x7fd108107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.146+0000 7fd10e05a640 1 -- 172.21.15.7:0/1485166093 >> 172.21.15.7:0/1485166093 conn(0x7fd1080fd630 msgr2=0x7fd1080ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:54.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.146+0000 7fd10e05a640 1 -- 172.21.15.7:0/1485166093 shutdown_connections 2024-09-21T05:09:54.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.146+0000 7fd10e05a640 1 -- 172.21.15.7:0/1485166093 wait complete. 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd10e05a640 1 Processor -- start 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd10e05a640 1 -- start start 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd10e05a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd108101910 0x7fd10819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd10e05a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108107910 0x7fd10819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd10e05a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd10810ede0 con 0x7fd108107910 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd10e05a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd10810ec60 con 0x7fd108101910 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd106ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108107910 0x7fd10819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd106ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108107910 0x7fd10819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36050/0 (socket says 172.21.15.7:36050) 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd106ffd640 1 -- 172.21.15.7:0/840813929 learned_addr learned my addr 172.21.15.7:0/840813929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.147+0000 7fd1077fe640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd108101910 0x7fd10819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd106ffd640 1 -- 172.21.15.7:0/840813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd108101910 msgr2=0x7fd10819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd106ffd640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd108101910 0x7fd10819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd106ffd640 1 -- 172.21.15.7:0/840813929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd108197780 con 0x7fd108107910 2024-09-21T05:09:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd1077fe640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd108101910 0x7fd10819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:09:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd106ffd640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108107910 0x7fd10819db40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc031460 tx=0x7fd0fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd104ff9640 1 -- 172.21.15.7:0/840813929 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd0fc03f070 con 0x7fd108107910 2024-09-21T05:09:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd108197a10 con 0x7fd108107910 2024-09-21T05:09:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd104ff9640 1 -- 172.21.15.7:0/840813929 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0fc0035f0 con 0x7fd108107910 2024-09-21T05:09:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd104ff9640 1 -- 172.21.15.7:0/840813929 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd0fc035030 con 0x7fd108107910 2024-09-21T05:09:54.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.148+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd108197f40 con 0x7fd108107910 2024-09-21T05:09:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.149+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd108103050 con 0x7fd108107910 2024-09-21T05:09:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.150+0000 7fd104ff9640 1 -- 172.21.15.7:0/840813929 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd0fc042070 con 0x7fd108107910 2024-09-21T05:09:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.151+0000 7fd104ff9640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd0f007af30 0x7fd0f007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.151+0000 7fd1077fe640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd0f007af30 0x7fd0f007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.151+0000 7fd1077fe640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd0f007af30 0x7fd0f007d3f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8005fd0 tx=0x7fd0f800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:54.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.152+0000 7fd104ff9640 1 -- 172.21.15.7:0/840813929 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fd0fc0bcdf0 con 0x7fd108107910 2024-09-21T05:09:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.157+0000 7fd104ff9640 1 -- 172.21.15.7:0/840813929 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0fc0c0030 con 0x7fd108107910 2024-09-21T05:09:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.381+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fd108198cb0 con 0x7fd108107910 2024-09-21T05:09:54.385 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:54 smithi007 ceph-mon[108238]: from='client.35712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:54.385 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:54 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1037799847' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:09:54.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.381+0000 7fd104ff9640 1 -- 172.21.15.7:0/840813929 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7fd0fc089f70 con 0x7fd108107910 2024-09-21T05:09:54.386 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:09:54.386 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:09:54.386 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:09:54.386 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:09:54.386 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:09:54.386 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:09:54.386 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:09:54.387 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:09:54.388 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:09:54.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:09:54.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:09:54.389 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:09:54.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:09:54.389 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:09:54.389 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:09:54.389 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:09:54.389 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd0f007af30 msgr2=0x7fd0f007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd0f007af30 0x7fd0f007d3f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8005fd0 tx=0x7fd0f800b390 comp rx=0 tx=0).stop 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108107910 msgr2=0x7fd10819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108107910 0x7fd10819db40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc031460 tx=0x7fd0fc0045e0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 shutdown_connections 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd0f007af30 0x7fd0f007d3f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd108107910 0x7fd10819db40 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 --2- 172.21.15.7:0/840813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd108101910 0x7fd10819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.384+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 >> 172.21.15.7:0/840813929 conn(0x7fd1080fd630 msgr2=0x7fd1080fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.385+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 shutdown_connections 2024-09-21T05:09:54.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.385+0000 7fd10e05a640 1 -- 172.21.15.7:0/840813929 wait complete. 2024-09-21T05:09:54.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:54 smithi053 ceph-mon[80176]: from='client.35712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:54.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:54 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1037799847' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:09:54.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.469+0000 7fe2d5979640 1 -- 172.21.15.7:0/3872253892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 msgr2=0x7fe2d0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.469+0000 7fe2d5979640 1 --2- 172.21.15.7:0/3872253892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 0x7fe2d0101d60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009880 tx=0x7fe2b802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.469+0000 7fe2d5979640 1 -- 172.21.15.7:0/3872253892 shutdown_connections 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.469+0000 7fe2d5979640 1 --2- 172.21.15.7:0/3872253892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 0x7fe2d0101d60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.469+0000 7fe2d5979640 1 --2- 172.21.15.7:0/3872253892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d0107900 0x7fe2d0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.469+0000 7fe2d5979640 1 -- 172.21.15.7:0/3872253892 >> 172.21.15.7:0/3872253892 conn(0x7fe2d00fd600 msgr2=0x7fe2d00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.469+0000 7fe2d5979640 1 -- 172.21.15.7:0/3872253892 shutdown_connections 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.469+0000 7fe2d5979640 1 -- 172.21.15.7:0/3872253892 wait complete. 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2d5979640 1 Processor -- start 2024-09-21T05:09:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2d5979640 1 -- start start 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2d5979640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 0x7fe2d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2d5979640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d0107900 0x7fe2d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2d5979640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d010edb0 con 0x7fe2d0101900 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2d5979640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d010ec30 con 0x7fe2d0107900 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2ceffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 0x7fe2d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2ceffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 0x7fe2d00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36056/0 (socket says 172.21.15.7:36056) 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2ceffd640 1 -- 172.21.15.7:0/2902434889 learned_addr learned my addr 172.21.15.7:0/2902434889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.470+0000 7fe2c67fc640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d0107900 0x7fe2d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2ceffd640 1 -- 172.21.15.7:0/2902434889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d0107900 msgr2=0x7fe2d0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2ceffd640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d0107900 0x7fe2d0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2ceffd640 1 -- 172.21.15.7:0/2902434889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d0074f70 con 0x7fe2d0101900 2024-09-21T05:09:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2ceffd640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 0x7fe2d00744f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d8d0 tx=0x7fe2c000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2d4977640 1 -- 172.21.15.7:0/2902434889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe2c0017070 con 0x7fe2d0101900 2024-09-21T05:09:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2d4977640 1 -- 172.21.15.7:0/2902434889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe2c0010460 con 0x7fe2d0101900 2024-09-21T05:09:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d0075260 con 0x7fe2d0101900 2024-09-21T05:09:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2d4977640 1 -- 172.21.15.7:0/2902434889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe2c000f5f0 con 0x7fe2d0101900 2024-09-21T05:09:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.471+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d01a3e50 con 0x7fe2d0101900 2024-09-21T05:09:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.472+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d0103040 con 0x7fe2d0101900 2024-09-21T05:09:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.473+0000 7fe2d4977640 1 -- 172.21.15.7:0/2902434889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe2c0009a30 con 0x7fe2d0101900 2024-09-21T05:09:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.473+0000 7fe2d4977640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe2bc07af80 0x7fe2bc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.474+0000 7fe2d4977640 1 -- 172.21.15.7:0/2902434889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fe2c009ddc0 con 0x7fe2d0101900 2024-09-21T05:09:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.474+0000 7fe2c67fc640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe2bc07af80 0x7fe2bc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.474+0000 7fe2c67fc640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe2bc07af80 0x7fe2bc07d440 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0075f80 tx=0x7fe2b8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:54.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.478+0000 7fe2d4977640 1 -- 172.21.15.7:0/2902434889 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe2c00693e0 con 0x7fe2d0101900 2024-09-21T05:09:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.684+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe2d0062630 con 0x7fe2bc07af80 2024-09-21T05:09:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.685+0000 7fe2d4977640 1 -- 172.21.15.7:0/2902434889 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fe2d0062630 con 0x7fe2bc07af80 2024-09-21T05:09:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:09:54.689 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:09:54.689 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:09:54.689 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:09:54.689 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:09:54.689 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:09:54.690 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:09:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe2bc07af80 msgr2=0x7fe2bc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe2bc07af80 0x7fe2bc07d440 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0075f80 tx=0x7fe2b8031400 comp rx=0 tx=0).stop 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 msgr2=0x7fe2d00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 0x7fe2d00744f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d8d0 tx=0x7fe2c000dd90 comp rx=0 tx=0).stop 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 shutdown_connections 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe2bc07af80 0x7fe2bc07d440 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d0107900 0x7fe2d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 --2- 172.21.15.7:0/2902434889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d0101900 0x7fe2d00744f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 >> 172.21.15.7:0/2902434889 conn(0x7fe2d00fd600 msgr2=0x7fe2d00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 shutdown_connections 2024-09-21T05:09:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.688+0000 7fe2d5979640 1 -- 172.21.15.7:0/2902434889 wait complete. 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.772+0000 7f4b7e09e640 1 -- 172.21.15.7:0/1489928565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 msgr2=0x7f4b78102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.772+0000 7f4b7e09e640 1 --2- 172.21.15.7:0/1489928565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 0x7f4b78102ea0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009880 tx=0x7f4b6802a530 comp rx=0 tx=0).stop 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.772+0000 7f4b7e09e640 1 -- 172.21.15.7:0/1489928565 shutdown_connections 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.772+0000 7f4b7e09e640 1 --2- 172.21.15.7:0/1489928565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b781033e0 0x7f4b7810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.772+0000 7f4b7e09e640 1 --2- 172.21.15.7:0/1489928565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 0x7f4b78102ea0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.772+0000 7f4b7e09e640 1 -- 172.21.15.7:0/1489928565 >> 172.21.15.7:0/1489928565 conn(0x7f4b780fc520 msgr2=0x7f4b780fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.772+0000 7f4b7e09e640 1 -- 172.21.15.7:0/1489928565 shutdown_connections 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.772+0000 7f4b7e09e640 1 -- 172.21.15.7:0/1489928565 wait complete. 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b7e09e640 1 Processor -- start 2024-09-21T05:09:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b7e09e640 1 -- start start 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b7e09e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 0x7f4b7819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b7e09e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b781033e0 0x7f4b7819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b7e09e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810edd0 con 0x7f4b78102ac0 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b7e09e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810ec50 con 0x7f4b781033e0 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b777fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 0x7f4b7819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b777fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 0x7f4b7819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36068/0 (socket says 172.21.15.7:36068) 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.773+0000 7f4b777fe640 1 -- 172.21.15.7:0/2394812036 learned_addr learned my addr 172.21.15.7:0/2394812036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b76ffd640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b781033e0 0x7f4b7819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b777fe640 1 -- 172.21.15.7:0/2394812036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b781033e0 msgr2=0x7f4b7819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b777fe640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b781033e0 0x7f4b7819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b777fe640 1 -- 172.21.15.7:0/2394812036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b781977c0 con 0x7f4b78102ac0 2024-09-21T05:09:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b777fe640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 0x7f4b7819d640 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4b68031460 tx=0x7f4b680045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b74ff9640 1 -- 172.21.15.7:0/2394812036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4b6803f070 con 0x7f4b78102ac0 2024-09-21T05:09:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b74ff9640 1 -- 172.21.15.7:0/2394812036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b680035f0 con 0x7f4b78102ac0 2024-09-21T05:09:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b78197a50 con 0x7f4b78102ac0 2024-09-21T05:09:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b74ff9640 1 -- 172.21.15.7:0/2394812036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4b68035030 con 0x7f4b78102ac0 2024-09-21T05:09:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.774+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b78197f80 con 0x7f4b78102ac0 2024-09-21T05:09:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.775+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b7806aa90 con 0x7f4b78102ac0 2024-09-21T05:09:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.776+0000 7f4b74ff9640 1 -- 172.21.15.7:0/2394812036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4b68042070 con 0x7f4b78102ac0 2024-09-21T05:09:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.776+0000 7f4b74ff9640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4b6007af30 0x7f4b6007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:09:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.777+0000 7f4b74ff9640 1 -- 172.21.15.7:0/2394812036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f4b680bea70 con 0x7f4b78102ac0 2024-09-21T05:09:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.777+0000 7f4b76ffd640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4b6007af30 0x7f4b6007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:09:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.777+0000 7f4b76ffd640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4b6007af30 0x7f4b6007d3f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4b78108900 tx=0x7f4b6c00b360 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:09:54.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:54.781+0000 7f4b74ff9640 1 -- 172.21.15.7:0/2394812036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4b68089f70 con 0x7f4b78102ac0 2024-09-21T05:09:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.045+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f4b78105f90 con 0x7f4b78102ac0 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.046+0000 7f4b74ff9640 1 -- 172.21.15.7:0/2394812036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f4b6808ee20 con 0x7f4b78102ac0 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:09:55.050 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:09:55.051 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:09:55.052 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:09:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:09:55.052 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:09:55.052 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:09:55.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4b6007af30 msgr2=0x7f4b6007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:55.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4b6007af30 0x7f4b6007d3f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4b78108900 tx=0x7f4b6c00b360 comp rx=0 tx=0).stop 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 msgr2=0x7f4b7819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 0x7f4b7819d640 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4b68031460 tx=0x7f4b680045e0 comp rx=0 tx=0).stop 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 shutdown_connections 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4b6007af30 0x7f4b6007d3f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b781033e0 0x7f4b7819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 --2- 172.21.15.7:0/2394812036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78102ac0 0x7f4b7819d640 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 >> 172.21.15.7:0/2394812036 conn(0x7f4b780fc520 msgr2=0x7f4b78108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 shutdown_connections 2024-09-21T05:09:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:09:55.048+0000 7f4b7e09e640 1 -- 172.21.15.7:0/2394812036 wait complete. 2024-09-21T05:09:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:55 smithi053 ceph-mon[80176]: from='client.35716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:55 smithi053 ceph-mon[80176]: pgmap v1055: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:55 smithi053 ceph-mon[80176]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:55 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/840813929' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:09:55.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:55 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2394812036' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:09:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:55 smithi007 ceph-mon[108238]: from='client.35716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:55 smithi007 ceph-mon[108238]: pgmap v1055: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:55 smithi007 ceph-mon[108238]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:55 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/840813929' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:09:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:55 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2394812036' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:09:56.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:56 smithi007 ceph-mon[108238]: from='client.35732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:56.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:56 smithi053 ceph-mon[80176]: from='client.35732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:09:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:57 smithi053 ceph-mon[80176]: pgmap v1056: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:57 smithi007 ceph-mon[108238]: pgmap v1056: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:59 smithi007 ceph-mon[108238]: pgmap v1057: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:09:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:09:59.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:59 smithi053 ceph-mon[80176]: pgmap v1057: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:09:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:09:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:00.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:00 smithi053 ceph-mon[80176]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:10:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:00 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T05:09:59.999+0000 7f54cf66c640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:10:00.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:00 smithi007 ceph-mon[108238]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:10:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:01 smithi053 ceph-mon[80176]: pgmap v1058: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:01 smithi007 ceph-mon[108238]: pgmap v1058: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:03 smithi053 ceph-mon[80176]: pgmap v1059: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:03 smithi007 ceph-mon[108238]: pgmap v1059: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:05 smithi007 ceph-mon[108238]: pgmap v1060: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:05.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:05 smithi053 ceph-mon[80176]: pgmap v1060: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:07 smithi053 ceph-mon[80176]: pgmap v1061: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:07.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:07 smithi007 ceph-mon[108238]: pgmap v1061: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:09.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:09 smithi053 ceph-mon[80176]: pgmap v1062: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:09 smithi007 ceph-mon[108238]: pgmap v1062: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:11 smithi053 ceph-mon[80176]: pgmap v1063: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:11.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:11 smithi007 ceph-mon[108238]: pgmap v1063: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:13.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:13 smithi053 ceph-mon[80176]: pgmap v1064: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:13 smithi007 ceph-mon[108238]: pgmap v1064: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:14.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:15.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:15 smithi007 ceph-mon[108238]: pgmap v1065: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:15.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:15 smithi053 ceph-mon[80176]: pgmap v1065: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:17 smithi053 ceph-mon[80176]: pgmap v1066: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:17 smithi007 ceph-mon[108238]: pgmap v1066: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:19 smithi053 ceph-mon[80176]: pgmap v1067: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:19 smithi007 ceph-mon[108238]: pgmap v1067: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:21.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:21 smithi053 ceph-mon[80176]: pgmap v1068: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:21.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:21 smithi007 ceph-mon[108238]: pgmap v1068: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:23.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:23 smithi053 ceph-mon[80176]: pgmap v1069: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:23 smithi007 ceph-mon[108238]: pgmap v1069: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.147+0000 7fc347732640 1 -- 172.21.15.7:0/2223304752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 msgr2=0x7fc340100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.147+0000 7fc347732640 1 --2- 172.21.15.7:0/2223304752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 0x7fc340100a20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc330009c20 tx=0x7fc33002a960 comp rx=0 tx=0).stop 2024-09-21T05:10:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.147+0000 7fc347732640 1 -- 172.21.15.7:0/2223304752 shutdown_connections 2024-09-21T05:10:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.147+0000 7fc347732640 1 --2- 172.21.15.7:0/2223304752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc340100ff0 0x7fc34010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.147+0000 7fc347732640 1 --2- 172.21.15.7:0/2223304752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 0x7fc340100a20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.147+0000 7fc347732640 1 -- 172.21.15.7:0/2223304752 >> 172.21.15.7:0/2223304752 conn(0x7fc3400fc550 msgr2=0x7fc3400fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.147+0000 7fc347732640 1 -- 172.21.15.7:0/2223304752 shutdown_connections 2024-09-21T05:10:25.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.147+0000 7fc347732640 1 -- 172.21.15.7:0/2223304752 wait complete. 2024-09-21T05:10:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc347732640 1 Processor -- start 2024-09-21T05:10:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc347732640 1 -- start start 2024-09-21T05:10:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc347732640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 0x7fc34019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc347732640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc340100ff0 0x7fc34019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc347732640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc340110f80 con 0x7fc340100640 2024-09-21T05:10:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc347732640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc340110e00 con 0x7fc340100ff0 2024-09-21T05:10:25.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc3454ce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 0x7fc34019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc3454ce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 0x7fc34019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38516/0 (socket says 172.21.15.7:38516) 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.148+0000 7fc3454ce640 1 -- 172.21.15.7:0/1410637887 learned_addr learned my addr 172.21.15.7:0/1410637887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc344ccd640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc340100ff0 0x7fc34019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc3454ce640 1 -- 172.21.15.7:0/1410637887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc340100ff0 msgr2=0x7fc34019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc3454ce640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc340100ff0 0x7fc34019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc3454ce640 1 -- 172.21.15.7:0/1410637887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc340199970 con 0x7fc340100640 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc3454ce640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 0x7fc34019f7b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc33002ae80 tx=0x7fc33002fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc33e7fc640 1 -- 172.21.15.7:0/1410637887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc330034030 con 0x7fc340100640 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc33e7fc640 1 -- 172.21.15.7:0/1410637887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc330003e20 con 0x7fc340100640 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc340199c00 con 0x7fc340100640 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc33e7fc640 1 -- 172.21.15.7:0/1410637887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc3300046c0 con 0x7fc340100640 2024-09-21T05:10:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.149+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34019a190 con 0x7fc340100640 2024-09-21T05:10:25.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.150+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34006a580 con 0x7fc340100640 2024-09-21T05:10:25.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.151+0000 7fc33e7fc640 1 -- 172.21.15.7:0/1410637887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc330037020 con 0x7fc340100640 2024-09-21T05:10:25.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.152+0000 7fc33e7fc640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc32807af30 0x7fc32807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.152+0000 7fc33e7fc640 1 -- 172.21.15.7:0/1410637887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc3300bef10 con 0x7fc340100640 2024-09-21T05:10:25.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.152+0000 7fc344ccd640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc32807af30 0x7fc32807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.152+0000 7fc344ccd640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc32807af30 0x7fc32807d3f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc3400fef70 tx=0x7fc33400a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:25.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.156+0000 7fc33e7fc640 1 -- 172.21.15.7:0/1410637887 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc33008a4b0 con 0x7fc340100640 2024-09-21T05:10:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.367+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc34019aea0 con 0x7fc32807af30 2024-09-21T05:10:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.368+0000 7fc33e7fc640 1 -- 172.21.15.7:0/1410637887 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc34019aea0 con 0x7fc32807af30 2024-09-21T05:10:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc32807af30 msgr2=0x7fc32807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc32807af30 0x7fc32807d3f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc3400fef70 tx=0x7fc33400a250 comp rx=0 tx=0).stop 2024-09-21T05:10:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 msgr2=0x7fc34019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 0x7fc34019f7b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc33002ae80 tx=0x7fc33002fb90 comp rx=0 tx=0).stop 2024-09-21T05:10:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 shutdown_connections 2024-09-21T05:10:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc32807af30 0x7fc32807d3f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc340100ff0 0x7fc34019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 --2- 172.21.15.7:0/1410637887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc340100640 0x7fc34019f7b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 >> 172.21.15.7:0/1410637887 conn(0x7fc3400fc550 msgr2=0x7fc3400fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 shutdown_connections 2024-09-21T05:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.370+0000 7fc347732640 1 -- 172.21.15.7:0/1410637887 wait complete. 2024-09-21T05:10:25.382 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:10:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:25 smithi007 ceph-mon[108238]: pgmap v1070: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.456+0000 7fd1c4335640 1 -- 172.21.15.7:0/3545149657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 msgr2=0x7fd1bc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.456+0000 7fd1c4335640 1 --2- 172.21.15.7:0/3545149657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 0x7fd1bc100c60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009880 tx=0x7fd1ac02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.456+0000 7fd1c4335640 1 -- 172.21.15.7:0/3545149657 shutdown_connections 2024-09-21T05:10:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.456+0000 7fd1c4335640 1 --2- 172.21.15.7:0/3545149657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 0x7fd1bc100c60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.456+0000 7fd1c4335640 1 --2- 172.21.15.7:0/3545149657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1bc106800 0x7fd1bc106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.456+0000 7fd1c4335640 1 -- 172.21.15.7:0/3545149657 >> 172.21.15.7:0/3545149657 conn(0x7fd1bc0fc520 msgr2=0x7fd1bc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.456+0000 7fd1c4335640 1 -- 172.21.15.7:0/3545149657 shutdown_connections 2024-09-21T05:10:25.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.456+0000 7fd1c4335640 1 -- 172.21.15.7:0/3545149657 wait complete. 2024-09-21T05:10:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c4335640 1 Processor -- start 2024-09-21T05:10:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c4335640 1 -- start start 2024-09-21T05:10:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c4335640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 0x7fd1bc10b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c4335640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1bc106800 0x7fd1bc10bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c4335640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc10edd0 con 0x7fd1bc100800 2024-09-21T05:10:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c4335640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc10ec50 con 0x7fd1bc106800 2024-09-21T05:10:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c20d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 0x7fd1bc10b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c20d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 0x7fd1bc10b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38548/0 (socket says 172.21.15.7:38548) 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c20d1640 1 -- 172.21.15.7:0/962745875 learned_addr learned my addr 172.21.15.7:0/962745875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c18d0640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1bc106800 0x7fd1bc10bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c20d1640 1 -- 172.21.15.7:0/962745875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1bc106800 msgr2=0x7fd1bc10bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c20d1640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1bc106800 0x7fd1bc10bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.457+0000 7fd1c20d1640 1 -- 172.21.15.7:0/962745875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc10c210 con 0x7fd1bc100800 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.458+0000 7fd1c20d1640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 0x7fd1bc10b790 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800c8f0 tx=0x7fd1b800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.458+0000 7fd1b37fe640 1 -- 172.21.15.7:0/962745875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd1b80079b0 con 0x7fd1bc100800 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.458+0000 7fd1b37fe640 1 -- 172.21.15.7:0/962745875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd1b80054e0 con 0x7fd1bc100800 2024-09-21T05:10:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.458+0000 7fd1b37fe640 1 -- 172.21.15.7:0/962745875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd1b8004dd0 con 0x7fd1bc100800 2024-09-21T05:10:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.458+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc10c500 con 0x7fd1bc100800 2024-09-21T05:10:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.458+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc1a3e20 con 0x7fd1bc100800 2024-09-21T05:10:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.459+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc101f40 con 0x7fd1bc100800 2024-09-21T05:10:25.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.460+0000 7fd1b37fe640 1 -- 172.21.15.7:0/962745875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd1b8005020 con 0x7fd1bc100800 2024-09-21T05:10:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.460+0000 7fd1b37fe640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1a807aed0 0x7fd1a807d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.460+0000 7fd1c18d0640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1a807aed0 0x7fd1a807d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.460+0000 7fd1b37fe640 1 -- 172.21.15.7:0/962745875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fd1b809e020 con 0x7fd1bc100800 2024-09-21T05:10:25.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.461+0000 7fd1c18d0640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1a807aed0 0x7fd1a807d390 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc10d220 tx=0x7fd1ac031420 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.465+0000 7fd1b37fe640 1 -- 172.21.15.7:0/962745875 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1b8014030 con 0x7fd1bc100800 2024-09-21T05:10:25.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.670+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd1bc062630 con 0x7fd1a807aed0 2024-09-21T05:10:25.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.671+0000 7fd1b37fe640 1 -- 172.21.15.7:0/962745875 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fd1bc062630 con 0x7fd1a807aed0 2024-09-21T05:10:25.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.673+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1a807aed0 msgr2=0x7fd1a807d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.673+0000 7fd1c4335640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1a807aed0 0x7fd1a807d390 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc10d220 tx=0x7fd1ac031420 comp rx=0 tx=0).stop 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.673+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 msgr2=0x7fd1bc10b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.673+0000 7fd1c4335640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 0x7fd1bc10b790 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800c8f0 tx=0x7fd1b800cdb0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.674+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 shutdown_connections 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.674+0000 7fd1c4335640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd1a807aed0 0x7fd1a807d390 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.674+0000 7fd1c4335640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1bc106800 0x7fd1bc10bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.674+0000 7fd1c4335640 1 --2- 172.21.15.7:0/962745875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1bc100800 0x7fd1bc10b790 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.674+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 >> 172.21.15.7:0/962745875 conn(0x7fd1bc0fc520 msgr2=0x7fd1bc0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.674+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 shutdown_connections 2024-09-21T05:10:25.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.674+0000 7fd1c4335640 1 -- 172.21.15.7:0/962745875 wait complete. 2024-09-21T05:10:25.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:25 smithi053 ceph-mon[80176]: pgmap v1070: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 -- 172.21.15.7:0/302663437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a41033e0 msgr2=0x7ff4a410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 --2- 172.21.15.7:0/302663437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a41033e0 0x7ff4a410b8d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff48c009880 tx=0x7ff48c02a530 comp rx=0 tx=0).stop 2024-09-21T05:10:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 -- 172.21.15.7:0/302663437 shutdown_connections 2024-09-21T05:10:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 --2- 172.21.15.7:0/302663437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a41033e0 0x7ff4a410b8d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 --2- 172.21.15.7:0/302663437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4a4102ac0 0x7ff4a4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 -- 172.21.15.7:0/302663437 >> 172.21.15.7:0/302663437 conn(0x7ff4a40fc520 msgr2=0x7ff4a40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 -- 172.21.15.7:0/302663437 shutdown_connections 2024-09-21T05:10:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 -- 172.21.15.7:0/302663437 wait complete. 2024-09-21T05:10:25.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.758+0000 7ff4a942d640 1 Processor -- start 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a942d640 1 -- start start 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a942d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a4102ac0 0x7ff4a419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a942d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4a41033e0 0x7ff4a419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a942d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410edf0 con 0x7ff4a4102ac0 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a942d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410ec70 con 0x7ff4a41033e0 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a4102ac0 0x7ff4a419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a4102ac0 0x7ff4a419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38552/0 (socket says 172.21.15.7:38552) 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a2ffd640 1 -- 172.21.15.7:0/2786165064 learned_addr learned my addr 172.21.15.7:0/2786165064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff49a7fc640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4a41033e0 0x7ff4a419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a2ffd640 1 -- 172.21.15.7:0/2786165064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4a41033e0 msgr2=0x7ff4a419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a2ffd640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4a41033e0 0x7ff4a419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.759+0000 7ff4a2ffd640 1 -- 172.21.15.7:0/2786165064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a4199a10 con 0x7ff4a4102ac0 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.760+0000 7ff4a2ffd640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a4102ac0 0x7ff4a419f850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d6f0 tx=0x7ff49400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.760+0000 7ff4a0ff9640 1 -- 172.21.15.7:0/2786165064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff494017070 con 0x7ff4a4102ac0 2024-09-21T05:10:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.760+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a4199d00 con 0x7ff4a4102ac0 2024-09-21T05:10:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.760+0000 7ff4a0ff9640 1 -- 172.21.15.7:0/2786165064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff49400de00 con 0x7ff4a4102ac0 2024-09-21T05:10:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.760+0000 7ff4a0ff9640 1 -- 172.21.15.7:0/2786165064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff494004bb0 con 0x7ff4a4102ac0 2024-09-21T05:10:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.760+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a419a260 con 0x7ff4a4102ac0 2024-09-21T05:10:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.761+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a406aa90 con 0x7ff4a4102ac0 2024-09-21T05:10:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.762+0000 7ff4a0ff9640 1 -- 172.21.15.7:0/2786165064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff494009a30 con 0x7ff4a4102ac0 2024-09-21T05:10:25.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.762+0000 7ff4a0ff9640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff49007af80 0x7ff49007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:25.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.762+0000 7ff4a0ff9640 1 -- 172.21.15.7:0/2786165064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff49409db20 con 0x7ff4a4102ac0 2024-09-21T05:10:25.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.762+0000 7ff49a7fc640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff49007af80 0x7ff49007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:25.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.763+0000 7ff49a7fc640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff49007af80 0x7ff49007d440 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff4a419b200 tx=0x7ff48c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:25.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.767+0000 7ff4a0ff9640 1 -- 172.21.15.7:0/2786165064 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff494069140 con 0x7ff4a4102ac0 2024-09-21T05:10:25.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.964+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff4a419af40 con 0x7ff49007af80 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.974+0000 7ff4a0ff9640 1 -- 172.21.15.7:0/2786165064 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7ff4a419af40 con 0x7ff49007af80 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (32m) 11m ago 43m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (35m) 11m ago 43m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (35m) 11m ago 42m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (30m) 11m ago 43m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (30m) 11m ago 42m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (32m) 11m ago 43m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:10:25.977 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (26m) 11m ago 38m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (26m) 11m ago 38m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (11m) 11m ago 37m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (26m) 11m ago 38m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (34m) 11m ago 44m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (33m) 11m ago 42m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (31m) 11m ago 44m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (30m) 11m ago 42m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (33m) 11m ago 43m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (33m) 11m ago 42m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (30m) 11m ago 41m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (29m) 11m ago 40m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (28m) 11m ago 40m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (28m) 11m ago 39m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (27m) 11m ago 39m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:10:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (27m) 11m ago 39m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:10:25.979 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (32m) 11m ago 43m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:10:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.976+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff49007af80 msgr2=0x7ff49007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.976+0000 7ff4a942d640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff49007af80 0x7ff49007d440 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff4a419b200 tx=0x7ff48c031400 comp rx=0 tx=0).stop 2024-09-21T05:10:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.976+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a4102ac0 msgr2=0x7ff4a419f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.976+0000 7ff4a942d640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a4102ac0 0x7ff4a419f850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d6f0 tx=0x7ff49400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.976+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 shutdown_connections 2024-09-21T05:10:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.976+0000 7ff4a942d640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff49007af80 0x7ff49007d440 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.976+0000 7ff4a942d640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4a41033e0 0x7ff4a419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.977+0000 7ff4a942d640 1 --2- 172.21.15.7:0/2786165064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a4102ac0 0x7ff4a419f850 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.977+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 >> 172.21.15.7:0/2786165064 conn(0x7ff4a40fc520 msgr2=0x7ff4a4100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.977+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 shutdown_connections 2024-09-21T05:10:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:25.977+0000 7ff4a942d640 1 -- 172.21.15.7:0/2786165064 wait complete. 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.060+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1410165921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100c00 msgr2=0x7fbad4104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.060+0000 7fbad9a1a640 1 --2- 172.21.15.7:0/1410165921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100c00 0x7fbad4104c50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009880 tx=0x7fbac802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.060+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1410165921 shutdown_connections 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.060+0000 7fbad9a1a640 1 --2- 172.21.15.7:0/1410165921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100c00 0x7fbad4104c50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.060+0000 7fbad9a1a640 1 --2- 172.21.15.7:0/1410165921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbad4100250 0x7fbad4100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.060+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1410165921 >> 172.21.15.7:0/1410165921 conn(0x7fbad40fbc90 msgr2=0x7fbad40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.061+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1410165921 shutdown_connections 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.061+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1410165921 wait complete. 2024-09-21T05:10:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.061+0000 7fbad9a1a640 1 Processor -- start 2024-09-21T05:10:26.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.061+0000 7fbad9a1a640 1 -- start start 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad9a1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100250 0x7fbad40fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad9a1a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbad4100c00 0x7fbad40fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad9a1a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbad410d4d0 con 0x7fbad4100250 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad9a1a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbad410d350 con 0x7fbad4100c00 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100250 0x7fbad40fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100250 0x7fbad40fc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38576/0 (socket says 172.21.15.7:38576) 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad2ffd640 1 -- 172.21.15.7:0/1089174213 learned_addr learned my addr 172.21.15.7:0/1089174213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad27fc640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbad4100c00 0x7fbad40fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad2ffd640 1 -- 172.21.15.7:0/1089174213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbad4100c00 msgr2=0x7fbad40fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad2ffd640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbad4100c00 0x7fbad40fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad2ffd640 1 -- 172.21.15.7:0/1089174213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad40fcbc0 con 0x7fbad4100250 2024-09-21T05:10:26.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad27fc640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbad4100c00 0x7fbad40fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:10:26.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad2ffd640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100250 0x7fbad40fc140 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e970 tx=0x7fbac400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:26.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.062+0000 7fbad8a18640 1 -- 172.21.15.7:0/1089174213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbac400cac0 con 0x7fbad4100250 2024-09-21T05:10:26.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.063+0000 7fbad8a18640 1 -- 172.21.15.7:0/1089174213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbac40054e0 con 0x7fbad4100250 2024-09-21T05:10:26.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.063+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad40fceb0 con 0x7fbad4100250 2024-09-21T05:10:26.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.063+0000 7fbad8a18640 1 -- 172.21.15.7:0/1089174213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbac4010430 con 0x7fbad4100250 2024-09-21T05:10:26.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.063+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a3f10 con 0x7fbad4100250 2024-09-21T05:10:26.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.064+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad4101730 con 0x7fbad4100250 2024-09-21T05:10:26.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.064+0000 7fbad8a18640 1 -- 172.21.15.7:0/1089174213 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbac4004ee0 con 0x7fbad4100250 2024-09-21T05:10:26.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.065+0000 7fbad8a18640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbabc07af80 0x7fbabc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.065+0000 7fbad8a18640 1 -- 172.21.15.7:0/1089174213 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbac409e700 con 0x7fbad4100250 2024-09-21T05:10:26.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.065+0000 7fbad27fc640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbabc07af80 0x7fbabc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.066+0000 7fbad27fc640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbabc07af80 0x7fbabc07d440 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbad40fdbd0 tx=0x7fbac8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:26.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.070+0000 7fbad8a18640 1 -- 172.21.15.7:0/1089174213 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbac4069d20 con 0x7fbad4100250 2024-09-21T05:10:26.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.331+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fbad40fd670 con 0x7fbad4100250 2024-09-21T05:10:26.334 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:26 smithi007 ceph-mon[108238]: from='client.35740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:26.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.331+0000 7fbad8a18640 1 -- 172.21.15.7:0/1089174213 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fbac406ebd0 con 0x7fbad4100250 2024-09-21T05:10:26.334 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:10:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:10:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:10:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:10:26.336 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:10:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbabc07af80 msgr2=0x7fbabc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbabc07af80 0x7fbabc07d440 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbad40fdbd0 tx=0x7fbac8031400 comp rx=0 tx=0).stop 2024-09-21T05:10:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100250 msgr2=0x7fbad40fc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100250 0x7fbad40fc140 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e970 tx=0x7fbac400ed70 comp rx=0 tx=0).stop 2024-09-21T05:10:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 shutdown_connections 2024-09-21T05:10:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbabc07af80 0x7fbabc07d440 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbad4100c00 0x7fbad40fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 --2- 172.21.15.7:0/1089174213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbad4100250 0x7fbad40fc140 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 >> 172.21.15.7:0/1089174213 conn(0x7fbad40fbc90 msgr2=0x7fbad4101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.333+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 shutdown_connections 2024-09-21T05:10:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.334+0000 7fbad9a1a640 1 -- 172.21.15.7:0/1089174213 wait complete. 2024-09-21T05:10:26.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 -- 172.21.15.7:0/3070674826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 msgr2=0x7f6b6c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 --2- 172.21.15.7:0/3070674826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 0x7f6b6c106bd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009880 tx=0x7f6b6802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 -- 172.21.15.7:0/3070674826 shutdown_connections 2024-09-21T05:10:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 --2- 172.21.15.7:0/3070674826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6c1007f0 0x7f6b6c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 --2- 172.21.15.7:0/3070674826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 0x7f6b6c106bd0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 -- 172.21.15.7:0/3070674826 >> 172.21.15.7:0/3070674826 conn(0x7f6b6c0fc550 msgr2=0x7f6b6c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 -- 172.21.15.7:0/3070674826 shutdown_connections 2024-09-21T05:10:26.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 -- 172.21.15.7:0/3070674826 wait complete. 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 Processor -- start 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.418+0000 7f6b7329c640 1 -- start start 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b7329c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6c1007f0 0x7f6b6c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b7329c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 0x7f6b6c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b7329c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10dd20 con 0x7f6b6c1067f0 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b7329c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10dba0 con 0x7f6b6c1007f0 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b70837640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 0x7f6b6c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b70837640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 0x7f6b6c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38598/0 (socket says 172.21.15.7:38598) 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b70837640 1 -- 172.21.15.7:0/2124058096 learned_addr learned my addr 172.21.15.7:0/2124058096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:26.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b71038640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6c1007f0 0x7f6b6c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b70837640 1 -- 172.21.15.7:0/2124058096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6c1007f0 msgr2=0x7f6b6c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b70837640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6c1007f0 0x7f6b6c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b70837640 1 -- 172.21.15.7:0/2124058096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c197800 con 0x7f6b6c1067f0 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.419+0000 7f6b70837640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 0x7f6b6c19dbc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b420 tx=0x7f6b5c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.420+0000 7f6b627fc640 1 -- 172.21.15.7:0/2124058096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c011630 con 0x7f6b6c1067f0 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.420+0000 7f6b627fc640 1 -- 172.21.15.7:0/2124058096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b5c016070 con 0x7f6b6c1067f0 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.420+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c197af0 con 0x7f6b6c1067f0 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.420+0000 7f6b627fc640 1 -- 172.21.15.7:0/2124058096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c010d40 con 0x7f6b6c1067f0 2024-09-21T05:10:26.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.420+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c198080 con 0x7f6b6c1067f0 2024-09-21T05:10:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.421+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c101f30 con 0x7f6b6c1067f0 2024-09-21T05:10:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.421+0000 7f6b627fc640 1 -- 172.21.15.7:0/2124058096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f6b5c01b460 con 0x7f6b6c1067f0 2024-09-21T05:10:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.422+0000 7f6b627fc640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6b5807af30 0x7f6b5807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.422+0000 7f6b627fc640 1 -- 172.21.15.7:0/2124058096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f6b5c09d720 con 0x7f6b6c1067f0 2024-09-21T05:10:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.422+0000 7f6b71038640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6b5807af30 0x7f6b5807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.423+0000 7f6b71038640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6b5807af30 0x7f6b5807d3f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6b680096c0 tx=0x7f6b68031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.427+0000 7f6b627fc640 1 -- 172.21.15.7:0/2124058096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b5c068cc0 con 0x7f6b6c1067f0 2024-09-21T05:10:26.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.655+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6b6c109d70 con 0x7f6b6c1067f0 2024-09-21T05:10:26.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.656+0000 7f6b627fc640 1 -- 172.21.15.7:0/2124058096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f6b5c06db70 con 0x7f6b6c1067f0 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:10:26.659 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:10:26.660 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:10:26.661 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:10:26.662 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:10:26.662 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:10:26.662 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:10:26.662 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:10:26.662 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:10:26.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6b5807af30 msgr2=0x7f6b5807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6b5807af30 0x7f6b5807d3f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6b680096c0 tx=0x7f6b68031400 comp rx=0 tx=0).stop 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 msgr2=0x7f6b6c19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 0x7f6b6c19dbc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b420 tx=0x7f6b5c00b8e0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 shutdown_connections 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f6b5807af30 0x7f6b5807d3f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b6c1067f0 0x7f6b6c19dbc0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 --2- 172.21.15.7:0/2124058096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b6c1007f0 0x7f6b6c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 >> 172.21.15.7:0/2124058096 conn(0x7f6b6c0fc550 msgr2=0x7f6b6c069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 shutdown_connections 2024-09-21T05:10:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.658+0000 7f6b7329c640 1 -- 172.21.15.7:0/2124058096 wait complete. 2024-09-21T05:10:26.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:26 smithi053 ceph-mon[80176]: from='client.35740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.743+0000 7fbd1968a640 1 -- 172.21.15.7:0/2330917472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141067f0 msgr2=0x7fbd14106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.743+0000 7fbd1968a640 1 --2- 172.21.15.7:0/2330917472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141067f0 0x7fbd14106bd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009880 tx=0x7fbd0402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.743+0000 7fbd1968a640 1 -- 172.21.15.7:0/2330917472 shutdown_connections 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.743+0000 7fbd1968a640 1 --2- 172.21.15.7:0/2330917472 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd141007f0 0x7fbd14100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.743+0000 7fbd1968a640 1 --2- 172.21.15.7:0/2330917472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141067f0 0x7fbd14106bd0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.743+0000 7fbd1968a640 1 -- 172.21.15.7:0/2330917472 >> 172.21.15.7:0/2330917472 conn(0x7fbd140fc550 msgr2=0x7fbd140fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.743+0000 7fbd1968a640 1 -- 172.21.15.7:0/2330917472 shutdown_connections 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.743+0000 7fbd1968a640 1 -- 172.21.15.7:0/2330917472 wait complete. 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.744+0000 7fbd1968a640 1 Processor -- start 2024-09-21T05:10:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.744+0000 7fbd1968a640 1 -- start start 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.744+0000 7fbd1968a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141007f0 0x7fbd1419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.744+0000 7fbd1968a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd141067f0 0x7fbd1419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.744+0000 7fbd1968a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1410dcc0 con 0x7fbd141007f0 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.744+0000 7fbd1968a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1410db40 con 0x7fbd141067f0 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd12ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141007f0 0x7fbd1419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd12ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141007f0 0x7fbd1419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38614/0 (socket says 172.21.15.7:38614) 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd12ffd640 1 -- 172.21.15.7:0/3998867596 learned_addr learned my addr 172.21.15.7:0/3998867596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd127fc640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd141067f0 0x7fbd1419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd12ffd640 1 -- 172.21.15.7:0/3998867596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd141067f0 msgr2=0x7fbd1419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd12ffd640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd141067f0 0x7fbd1419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd12ffd640 1 -- 172.21.15.7:0/3998867596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd141999d0 con 0x7fbd141007f0 2024-09-21T05:10:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd12ffd640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141007f0 0x7fbd1419f810 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbd04031460 tx=0x7fbd040045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbcfbfff640 1 -- 172.21.15.7:0/3998867596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbd0403f070 con 0x7fbd141007f0 2024-09-21T05:10:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbcfbfff640 1 -- 172.21.15.7:0/3998867596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd040035f0 con 0x7fbd141007f0 2024-09-21T05:10:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbcfbfff640 1 -- 172.21.15.7:0/3998867596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbd04035030 con 0x7fbd141007f0 2024-09-21T05:10:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd14199c60 con 0x7fbd141007f0 2024-09-21T05:10:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.745+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1419a190 con 0x7fbd141007f0 2024-09-21T05:10:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.747+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd14101f30 con 0x7fbd141007f0 2024-09-21T05:10:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.747+0000 7fbcfbfff640 1 -- 172.21.15.7:0/3998867596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbd04042070 con 0x7fbd141007f0 2024-09-21T05:10:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.748+0000 7fbcfbfff640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbcfc07af80 0x7fbcfc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.748+0000 7fbd127fc640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbcfc07af80 0x7fbcfc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.748+0000 7fbcfbfff640 1 -- 172.21.15.7:0/3998867596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbd040bea10 con 0x7fbd141007f0 2024-09-21T05:10:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.748+0000 7fbd127fc640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbcfc07af80 0x7fbcfc07d440 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd08005fd0 tx=0x7fbd08005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.752+0000 7fbcfbfff640 1 -- 172.21.15.7:0/3998867596 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd04089f90 con 0x7fbd141007f0 2024-09-21T05:10:26.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.959+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbd1419b010 con 0x7fbcfc07af80 2024-09-21T05:10:26.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.960+0000 7fbcfbfff640 1 -- 172.21.15.7:0/3998867596 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fbd1419b010 con 0x7fbcfc07af80 2024-09-21T05:10:26.963 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:10:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:10:26.964 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:10:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbcfc07af80 msgr2=0x7fbcfc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbcfc07af80 0x7fbcfc07d440 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd08005fd0 tx=0x7fbd08005850 comp rx=0 tx=0).stop 2024-09-21T05:10:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141007f0 msgr2=0x7fbd1419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141007f0 0x7fbd1419f810 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbd04031460 tx=0x7fbd040045e0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 shutdown_connections 2024-09-21T05:10:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbcfc07af80 0x7fbcfc07d440 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd141067f0 0x7fbd1419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 --2- 172.21.15.7:0/3998867596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd141007f0 0x7fbd1419f810 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 >> 172.21.15.7:0/3998867596 conn(0x7fbd140fc550 msgr2=0x7fbd140fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 shutdown_connections 2024-09-21T05:10:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:26.963+0000 7fbd1968a640 1 -- 172.21.15.7:0/3998867596 wait complete. 2024-09-21T05:10:27.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 -- 172.21.15.7:0/1278204810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 msgr2=0x7ffb90100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:27.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 --2- 172.21.15.7:0/1278204810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 0x7ffb90100c60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009880 tx=0x7ffb7c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:10:27.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 -- 172.21.15.7:0/1278204810 shutdown_connections 2024-09-21T05:10:27.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 --2- 172.21.15.7:0/1278204810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 0x7ffb90100c60 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:27.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 --2- 172.21.15.7:0/1278204810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb90106800 0x7ffb90106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 -- 172.21.15.7:0/1278204810 >> 172.21.15.7:0/1278204810 conn(0x7ffb900fc520 msgr2=0x7ffb900fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 -- 172.21.15.7:0/1278204810 shutdown_connections 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 -- 172.21.15.7:0/1278204810 wait complete. 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 Processor -- start 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.047+0000 7ffb96fb9640 1 -- start start 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb96fb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 0x7ffb900fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb96fb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb90106800 0x7ffb900fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb96fb9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9010dcd0 con 0x7ffb90100800 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb96fb9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9010db50 con 0x7ffb90106800 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb94d55640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 0x7ffb900fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb94d55640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 0x7ffb900fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38636/0 (socket says 172.21.15.7:38636) 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb94d55640 1 -- 172.21.15.7:0/52560232 learned_addr learned my addr 172.21.15.7:0/52560232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb8ffff640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb90106800 0x7ffb900fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb94d55640 1 -- 172.21.15.7:0/52560232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb90106800 msgr2=0x7ffb900fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb94d55640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb90106800 0x7ffb900fd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb94d55640 1 -- 172.21.15.7:0/52560232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb900fd670 con 0x7ffb90100800 2024-09-21T05:10:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb94d55640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 0x7ffb900fcbf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400d6f0 tx=0x7ffb8400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb8dffb640 1 -- 172.21.15.7:0/52560232 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffb84017070 con 0x7ffb90100800 2024-09-21T05:10:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb900fd960 con 0x7ffb90100800 2024-09-21T05:10:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb8dffb640 1 -- 172.21.15.7:0/52560232 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb8400de00 con 0x7ffb90100800 2024-09-21T05:10:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.048+0000 7ffb8dffb640 1 -- 172.21.15.7:0/52560232 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffb84004bb0 con 0x7ffb90100800 2024-09-21T05:10:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.049+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb901a3e70 con 0x7ffb90100800 2024-09-21T05:10:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.050+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb90101f40 con 0x7ffb90100800 2024-09-21T05:10:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.050+0000 7ffb8dffb640 1 -- 172.21.15.7:0/52560232 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ffb84009a30 con 0x7ffb90100800 2024-09-21T05:10:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.051+0000 7ffb8dffb640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffb8007af80 0x7ffb8007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.051+0000 7ffb8dffb640 1 -- 172.21.15.7:0/52560232 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ffb8409dc80 con 0x7ffb90100800 2024-09-21T05:10:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.051+0000 7ffb8ffff640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffb8007af80 0x7ffb8007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.052+0000 7ffb8ffff640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffb8007af80 0x7ffb8007d440 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffb900fe680 tx=0x7ffb7c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.055+0000 7ffb8dffb640 1 -- 172.21.15.7:0/52560232 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffb84069200 con 0x7ffb90100800 2024-09-21T05:10:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.317+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7ffb900fe120 con 0x7ffb90100800 2024-09-21T05:10:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.317+0000 7ffb8dffb640 1 -- 172.21.15.7:0/52560232 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7ffb8405ee20 con 0x7ffb90100800 2024-09-21T05:10:27.320 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:10:27.320 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:10:27.320 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:10:27.321 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:10:27.322 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:10:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffb8007af80 msgr2=0x7ffb8007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffb8007af80 0x7ffb8007d440 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffb900fe680 tx=0x7ffb7c031400 comp rx=0 tx=0).stop 2024-09-21T05:10:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 msgr2=0x7ffb900fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 0x7ffb900fcbf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400d6f0 tx=0x7ffb8400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:10:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 shutdown_connections 2024-09-21T05:10:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ffb8007af80 0x7ffb8007d440 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb90106800 0x7ffb900fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 --2- 172.21.15.7:0/52560232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90100800 0x7ffb900fcbf0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 >> 172.21.15.7:0/52560232 conn(0x7ffb900fc520 msgr2=0x7ffb9006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 shutdown_connections 2024-09-21T05:10:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:27.319+0000 7ffb96fb9640 1 -- 172.21.15.7:0/52560232 wait complete. 2024-09-21T05:10:27.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:27 smithi053 ceph-mon[80176]: pgmap v1071: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:27.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:27 smithi053 ceph-mon[80176]: from='client.35744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:27.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:27 smithi053 ceph-mon[80176]: from='client.35748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:27.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:27 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1089174213' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:10:27.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:27 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2124058096' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:10:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:27 smithi007 ceph-mon[108238]: pgmap v1071: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:27 smithi007 ceph-mon[108238]: from='client.35744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:27 smithi007 ceph-mon[108238]: from='client.35748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:27 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1089174213' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:10:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:27 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2124058096' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:10:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:28 smithi053 ceph-mon[80176]: from='client.35760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/52560232' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:10:28.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:28 smithi007 ceph-mon[108238]: from='client.35760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:28.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/52560232' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:10:29.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:29 smithi053 ceph-mon[80176]: pgmap v1072: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:29.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:29 smithi007 ceph-mon[108238]: pgmap v1072: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:31.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:31 smithi053 ceph-mon[80176]: pgmap v1073: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:31.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:31 smithi007 ceph-mon[108238]: pgmap v1073: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:33 smithi007 ceph-mon[108238]: pgmap v1074: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:33.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:33 smithi053 ceph-mon[80176]: pgmap v1074: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:35.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:35 smithi007 ceph-mon[108238]: pgmap v1075: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:35.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:35 smithi053 ceph-mon[80176]: pgmap v1075: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:37 smithi053 ceph-mon[80176]: pgmap v1076: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:37 smithi007 ceph-mon[108238]: pgmap v1076: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:39.624 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:39 smithi053 ceph-mon[80176]: pgmap v1077: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:39 smithi007 ceph-mon[108238]: pgmap v1077: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:41.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:41 smithi053 ceph-mon[80176]: pgmap v1078: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:41 smithi007 ceph-mon[108238]: pgmap v1078: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:43 smithi007 ceph-mon[108238]: pgmap v1079: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:43 smithi053 ceph-mon[80176]: pgmap v1079: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:44.594 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:45.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:45 smithi007 ceph-mon[108238]: pgmap v1080: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:45.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:45 smithi053 ceph-mon[80176]: pgmap v1080: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:47.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:47 smithi053 ceph-mon[80176]: pgmap v1081: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:47 smithi007 ceph-mon[108238]: pgmap v1081: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:49 smithi053 ceph-mon[80176]: pgmap v1082: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:49.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:49 smithi007 ceph-mon[108238]: pgmap v1082: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:51.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:51 smithi053 ceph-mon[80176]: pgmap v1083: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:51 smithi007 ceph-mon[108238]: pgmap v1083: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:53.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:53 smithi053 ceph-mon[80176]: pgmap v1084: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:53 smithi007 ceph-mon[108238]: pgmap v1084: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:55.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:55 smithi007 ceph-mon[108238]: pgmap v1085: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:55.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:55 smithi053 ceph-mon[80176]: pgmap v1085: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:57.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.416+0000 7fee2adc1640 1 -- 172.21.15.7:0/221843000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24102b50 msgr2=0x7fee24102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:57.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.416+0000 7fee2adc1640 1 --2- 172.21.15.7:0/221843000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24102b50 0x7fee24102f30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fee14009880 tx=0x7fee1402a530 comp rx=0 tx=0).stop 2024-09-21T05:10:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.416+0000 7fee2adc1640 1 -- 172.21.15.7:0/221843000 shutdown_connections 2024-09-21T05:10:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.416+0000 7fee2adc1640 1 --2- 172.21.15.7:0/221843000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24103470 0x7fee2410b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.416+0000 7fee2adc1640 1 --2- 172.21.15.7:0/221843000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24102b50 0x7fee24102f30 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.416+0000 7fee2adc1640 1 -- 172.21.15.7:0/221843000 >> 172.21.15.7:0/221843000 conn(0x7fee240fc550 msgr2=0x7fee240fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.416+0000 7fee2adc1640 1 -- 172.21.15.7:0/221843000 shutdown_connections 2024-09-21T05:10:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.416+0000 7fee2adc1640 1 -- 172.21.15.7:0/221843000 wait complete. 2024-09-21T05:10:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee2adc1640 1 Processor -- start 2024-09-21T05:10:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee2adc1640 1 -- start start 2024-09-21T05:10:57.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee2adc1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b50 0x7fee2419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee2adc1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103470 0x7fee2419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee2adc1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410ede0 con 0x7fee24103470 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee2adc1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410ec60 con 0x7fee24102b50 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee23fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103470 0x7fee2419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee23fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103470 0x7fee2419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45428/0 (socket says 172.21.15.7:45428) 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee23fff640 1 -- 172.21.15.7:0/3631963638 learned_addr learned my addr 172.21.15.7:0/3631963638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.417+0000 7fee28b5d640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b50 0x7fee2419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee23fff640 1 -- 172.21.15.7:0/3631963638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b50 msgr2=0x7fee2419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee23fff640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b50 0x7fee2419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee23fff640 1 -- 172.21.15.7:0/3631963638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee24199990 con 0x7fee24103470 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee23fff640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103470 0x7fee2419fd10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fee1800d6f0 tx=0x7fee1800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee21ffb640 1 -- 172.21.15.7:0/3631963638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee18017070 con 0x7fee24103470 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee21ffb640 1 -- 172.21.15.7:0/3631963638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee1800de00 con 0x7fee24103470 2024-09-21T05:10:57.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee24199c80 con 0x7fee24103470 2024-09-21T05:10:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee21ffb640 1 -- 172.21.15.7:0/3631963638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee18004bb0 con 0x7fee24103470 2024-09-21T05:10:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.418+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2419a210 con 0x7fee24103470 2024-09-21T05:10:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.419+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2406a4e0 con 0x7fee24103470 2024-09-21T05:10:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.420+0000 7fee21ffb640 1 -- 172.21.15.7:0/3631963638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fee18009a30 con 0x7fee24103470 2024-09-21T05:10:57.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.421+0000 7fee21ffb640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fee0c07af90 0x7fee0c07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.421+0000 7fee28b5d640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fee0c07af90 0x7fee0c07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.421+0000 7fee21ffb640 1 -- 172.21.15.7:0/3631963638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fee1809db20 con 0x7fee24103470 2024-09-21T05:10:57.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.421+0000 7fee28b5d640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fee0c07af90 0x7fee0c07d450 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fee14000c00 tx=0x7fee14004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.425+0000 7fee21ffb640 1 -- 172.21.15.7:0/3631963638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee18069140 con 0x7fee24103470 2024-09-21T05:10:57.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.630+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fee2419aec0 con 0x7fee0c07af90 2024-09-21T05:10:57.634 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:57 smithi007 ceph-mon[108238]: pgmap v1086: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:57.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.632+0000 7fee21ffb640 1 -- 172.21.15.7:0/3631963638 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fee2419aec0 con 0x7fee0c07af90 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fee0c07af90 msgr2=0x7fee0c07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fee0c07af90 0x7fee0c07d450 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fee14000c00 tx=0x7fee14004f30 comp rx=0 tx=0).stop 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103470 msgr2=0x7fee2419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103470 0x7fee2419fd10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fee1800d6f0 tx=0x7fee1800dbb0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 shutdown_connections 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fee0c07af90 0x7fee0c07d450 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee24103470 0x7fee2419fd10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 --2- 172.21.15.7:0/3631963638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee24102b50 0x7fee2419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 >> 172.21.15.7:0/3631963638 conn(0x7fee240fc550 msgr2=0x7fee24100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 shutdown_connections 2024-09-21T05:10:57.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.634+0000 7fee2adc1640 1 -- 172.21.15.7:0/3631963638 wait complete. 2024-09-21T05:10:57.646 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:10:57.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:57 smithi053 ceph-mon[80176]: pgmap v1086: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.719+0000 7f316c24e640 1 -- 172.21.15.7:0/2285701019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3164102ac0 msgr2=0x7f3164102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.719+0000 7f316c24e640 1 --2- 172.21.15.7:0/2285701019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3164102ac0 0x7f3164102ea0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3160009880 tx=0x7f316002a530 comp rx=0 tx=0).stop 2024-09-21T05:10:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.719+0000 7f316c24e640 1 -- 172.21.15.7:0/2285701019 shutdown_connections 2024-09-21T05:10:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.719+0000 7f316c24e640 1 --2- 172.21.15.7:0/2285701019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31641033e0 0x7f316410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.719+0000 7f316c24e640 1 --2- 172.21.15.7:0/2285701019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3164102ac0 0x7f3164102ea0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.719+0000 7f316c24e640 1 -- 172.21.15.7:0/2285701019 >> 172.21.15.7:0/2285701019 conn(0x7f31640fc520 msgr2=0x7f31640fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.719+0000 7f316c24e640 1 -- 172.21.15.7:0/2285701019 shutdown_connections 2024-09-21T05:10:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.719+0000 7f316c24e640 1 -- 172.21.15.7:0/2285701019 wait complete. 2024-09-21T05:10:57.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f316c24e640 1 Processor -- start 2024-09-21T05:10:57.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f316c24e640 1 -- start start 2024-09-21T05:10:57.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f316c24e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164102ac0 0x7f3164100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:57.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f316c24e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31641033e0 0x7f3164100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:57.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f316c24e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f316410ee50 con 0x7f31641033e0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f316c24e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f316410ecd0 con 0x7f3164102ac0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f31697e9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31641033e0 0x7f3164100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f31697e9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31641033e0 0x7f3164100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45446/0 (socket says 172.21.15.7:45446) 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f31697e9640 1 -- 172.21.15.7:0/3679583207 learned_addr learned my addr 172.21.15.7:0/3679583207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.720+0000 7f3169fea640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164102ac0 0x7f3164100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f31697e9640 1 -- 172.21.15.7:0/3679583207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164102ac0 msgr2=0x7f3164100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f31697e9640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164102ac0 0x7f3164100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f31697e9640 1 -- 172.21.15.7:0/3679583207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31641011a0 con 0x7f31641033e0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f3169fea640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164102ac0 0x7f3164100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f31697e9640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31641033e0 0x7f3164100c60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f315400b370 tx=0x7f315400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f315affd640 1 -- 172.21.15.7:0/3679583207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3154015020 con 0x7f31641033e0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f315affd640 1 -- 172.21.15.7:0/3679583207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3154011af0 con 0x7f31641033e0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3164101490 con 0x7f31641033e0 2024-09-21T05:10:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f315affd640 1 -- 172.21.15.7:0/3679583207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3154010d60 con 0x7f31641033e0 2024-09-21T05:10:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.721+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31641a3fe0 con 0x7f31641033e0 2024-09-21T05:10:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.722+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316406aa90 con 0x7f31641033e0 2024-09-21T05:10:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.723+0000 7f315affd640 1 -- 172.21.15.7:0/3679583207 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3154018020 con 0x7f31641033e0 2024-09-21T05:10:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.723+0000 7f315affd640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f315007af40 0x7f315007d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.723+0000 7f315affd640 1 -- 172.21.15.7:0/3679583207 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f315409d9c0 con 0x7f31641033e0 2024-09-21T05:10:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.723+0000 7f3169fea640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f315007af40 0x7f315007d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:57.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.724+0000 7f3169fea640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f315007af40 0x7f315007d400 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3160000c00 tx=0x7f3160031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:57.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.728+0000 7f315affd640 1 -- 172.21.15.7:0/3679583207 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f315401f020 con 0x7f31641033e0 2024-09-21T05:10:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.932+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3164062630 con 0x7f315007af40 2024-09-21T05:10:57.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.933+0000 7f315affd640 1 -- 172.21.15.7:0/3679583207 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3164062630 con 0x7f315007af40 2024-09-21T05:10:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f315007af40 msgr2=0x7f315007d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f315007af40 0x7f315007d400 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3160000c00 tx=0x7f3160031400 comp rx=0 tx=0).stop 2024-09-21T05:10:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31641033e0 msgr2=0x7f3164100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31641033e0 0x7f3164100c60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f315400b370 tx=0x7f315400b830 comp rx=0 tx=0).stop 2024-09-21T05:10:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 shutdown_connections 2024-09-21T05:10:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f315007af40 0x7f315007d400 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31641033e0 0x7f3164100c60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 --2- 172.21.15.7:0/3679583207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164102ac0 0x7f3164100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 >> 172.21.15.7:0/3679583207 conn(0x7f31640fc520 msgr2=0x7f3164108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 shutdown_connections 2024-09-21T05:10:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:57.936+0000 7f316c24e640 1 -- 172.21.15.7:0/3679583207 wait complete. 2024-09-21T05:10:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 -- 172.21.15.7:0/324252833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 msgr2=0x7f86d0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 --2- 172.21.15.7:0/324252833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 0x7f86d0102ea0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009880 tx=0x7f86c002a530 comp rx=0 tx=0).stop 2024-09-21T05:10:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 -- 172.21.15.7:0/324252833 shutdown_connections 2024-09-21T05:10:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 --2- 172.21.15.7:0/324252833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d01033e0 0x7f86d010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 --2- 172.21.15.7:0/324252833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 0x7f86d0102ea0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 -- 172.21.15.7:0/324252833 >> 172.21.15.7:0/324252833 conn(0x7f86d00fc520 msgr2=0x7f86d00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 -- 172.21.15.7:0/324252833 shutdown_connections 2024-09-21T05:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 -- 172.21.15.7:0/324252833 wait complete. 2024-09-21T05:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.020+0000 7f86d5ec7640 1 Processor -- start 2024-09-21T05:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86d5ec7640 1 -- start start 2024-09-21T05:10:58.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86d5ec7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 0x7f86d019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86d5ec7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d01033e0 0x7f86d019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86d5ec7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010ee50 con 0x7f86d0102ac0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86d5ec7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010ecd0 con 0x7f86d01033e0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86cf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 0x7f86d019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86cf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 0x7f86d019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45464/0 (socket says 172.21.15.7:45464) 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86cf7fe640 1 -- 172.21.15.7:0/1279925983 learned_addr learned my addr 172.21.15.7:0/1279925983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86cf7fe640 1 -- 172.21.15.7:0/1279925983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d01033e0 msgr2=0x7f86d019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86cf7fe640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d01033e0 0x7f86d019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.021+0000 7f86cf7fe640 1 -- 172.21.15.7:0/1279925983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d0197860 con 0x7f86d0102ac0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.022+0000 7f86cf7fe640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 0x7f86d019d6a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f86c0031460 tx=0x7f86c00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.022+0000 7f86cd7fa640 1 -- 172.21.15.7:0/1279925983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f86c003f070 con 0x7f86d0102ac0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.022+0000 7f86cd7fa640 1 -- 172.21.15.7:0/1279925983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f86c0004cf0 con 0x7f86d0102ac0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.022+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d0197af0 con 0x7f86d0102ac0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.022+0000 7f86cd7fa640 1 -- 172.21.15.7:0/1279925983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f86c0035030 con 0x7f86d0102ac0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.022+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d0198020 con 0x7f86d0102ac0 2024-09-21T05:10:58.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.023+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d006aa90 con 0x7f86d0102ac0 2024-09-21T05:10:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.023+0000 7f86cd7fa640 1 -- 172.21.15.7:0/1279925983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f86c0042070 con 0x7f86d0102ac0 2024-09-21T05:10:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.024+0000 7f86cd7fa640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f86bc07af30 0x7f86bc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.024+0000 7f86cd7fa640 1 -- 172.21.15.7:0/1279925983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f86c00beb60 con 0x7f86d0102ac0 2024-09-21T05:10:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.024+0000 7f86c6ffd640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f86bc07af30 0x7f86bc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:58.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.025+0000 7f86c6ffd640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f86bc07af30 0x7f86bc07d3f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f86d0108900 tx=0x7f86b8006d30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:58.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.028+0000 7f86cd7fa640 1 -- 172.21.15.7:0/1279925983 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86c008a100 con 0x7f86d0102ac0 2024-09-21T05:10:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.224+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f86d0198e70 con 0x7f86bc07af30 2024-09-21T05:10:58.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.234+0000 7f86cd7fa640 1 -- 172.21.15.7:0/1279925983 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f86d0198e70 con 0x7f86bc07af30 2024-09-21T05:10:58.238 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:10:58.238 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (32m) 11m ago 43m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:10:58.238 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (36m) 11m ago 44m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:10:58.239 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (35m) 11m ago 43m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:10:58.239 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (31m) 11m ago 44m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:10:58.239 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (31m) 11m ago 43m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:10:58.239 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (32m) 11m ago 43m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:10:58.239 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (27m) 11m ago 38m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:10:58.239 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (27m) 11m ago 38m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:10:58.239 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (11m) 11m ago 38m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:10:58.239 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (26m) 11m ago 38m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:10:58.240 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (35m) 11m ago 45m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:10:58.240 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (34m) 11m ago 43m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:10:58.240 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (31m) 11m ago 45m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:10:58.240 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (31m) 11m ago 43m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:10:58.240 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (33m) 11m ago 44m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:10:58.240 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (33m) 11m ago 43m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:10:58.240 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (30m) 11m ago 41m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:10:58.240 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (30m) 11m ago 41m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:10:58.241 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (29m) 11m ago 40m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:10:58.241 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (28m) 11m ago 40m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:10:58.241 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (28m) 11m ago 40m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:10:58.241 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (27m) 11m ago 39m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:10:58.241 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (33m) 11m ago 43m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:10:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f86bc07af30 msgr2=0x7f86bc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f86bc07af30 0x7f86bc07d3f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f86d0108900 tx=0x7f86b8006d30 comp rx=0 tx=0).stop 2024-09-21T05:10:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 msgr2=0x7f86d019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 0x7f86d019d6a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f86c0031460 tx=0x7f86c00048f0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 shutdown_connections 2024-09-21T05:10:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f86bc07af30 0x7f86bc07d3f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86d01033e0 0x7f86d019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 --2- 172.21.15.7:0/1279925983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86d0102ac0 0x7f86d019d6a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 >> 172.21.15.7:0/1279925983 conn(0x7f86d00fc520 msgr2=0x7f86d010af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 shutdown_connections 2024-09-21T05:10:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.236+0000 7f86d5ec7640 1 -- 172.21.15.7:0/1279925983 wait complete. 2024-09-21T05:10:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.319+0000 7f714108d640 1 -- 172.21.15.7:0/1402543631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 msgr2=0x7f713c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.319+0000 7f714108d640 1 --2- 172.21.15.7:0/1402543631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 0x7f713c102ea0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f712c009880 tx=0x7f712c02a530 comp rx=0 tx=0).stop 2024-09-21T05:10:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.319+0000 7f714108d640 1 -- 172.21.15.7:0/1402543631 shutdown_connections 2024-09-21T05:10:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.319+0000 7f714108d640 1 --2- 172.21.15.7:0/1402543631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c1033e0 0x7f713c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.319+0000 7f714108d640 1 --2- 172.21.15.7:0/1402543631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 0x7f713c102ea0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.319+0000 7f714108d640 1 -- 172.21.15.7:0/1402543631 >> 172.21.15.7:0/1402543631 conn(0x7f713c0fc520 msgr2=0x7f713c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.319+0000 7f714108d640 1 -- 172.21.15.7:0/1402543631 shutdown_connections 2024-09-21T05:10:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.319+0000 7f714108d640 1 -- 172.21.15.7:0/1402543631 wait complete. 2024-09-21T05:10:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f714108d640 1 Processor -- start 2024-09-21T05:10:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f714108d640 1 -- start start 2024-09-21T05:10:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f714108d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 0x7f713c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f714108d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c1033e0 0x7f713c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f714108d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f713c10ee50 con 0x7f713c102ac0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f714108d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f713c10ecd0 con 0x7f713c1033e0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f713ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 0x7f713c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f713ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 0x7f713c19f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45474/0 (socket says 172.21.15.7:45474) 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.320+0000 7f713ad9d640 1 -- 172.21.15.7:0/1467023720 learned_addr learned my addr 172.21.15.7:0/1467023720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f713259c640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c1033e0 0x7f713c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f713ad9d640 1 -- 172.21.15.7:0/1467023720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c1033e0 msgr2=0x7f713c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f713ad9d640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c1033e0 0x7f713c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f713ad9d640 1 -- 172.21.15.7:0/1467023720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713c199a90 con 0x7f713c102ac0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f713ad9d640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 0x7f713c19f880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f712c031460 tx=0x7f712c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f7138d99640 1 -- 172.21.15.7:0/1467023720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f712c03f070 con 0x7f713c102ac0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f7138d99640 1 -- 172.21.15.7:0/1467023720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f712c004cf0 con 0x7f713c102ac0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713c199d20 con 0x7f713c102ac0 2024-09-21T05:10:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f7138d99640 1 -- 172.21.15.7:0/1467023720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f712c035030 con 0x7f713c102ac0 2024-09-21T05:10:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.321+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713c19a250 con 0x7f713c102ac0 2024-09-21T05:10:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.322+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713c06aa90 con 0x7f713c102ac0 2024-09-21T05:10:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.323+0000 7f7138d99640 1 -- 172.21.15.7:0/1467023720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f712c042070 con 0x7f713c102ac0 2024-09-21T05:10:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.323+0000 7f7138d99640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f712807af30 0x7f712807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.324+0000 7f7138d99640 1 -- 172.21.15.7:0/1467023720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f712c0beb50 con 0x7f713c102ac0 2024-09-21T05:10:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.324+0000 7f713259c640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f712807af30 0x7f712807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.324+0000 7f713259c640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f712807af30 0x7f712807d3f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f713c0fee80 tx=0x7f7124009060 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.328+0000 7f7138d99640 1 -- 172.21.15.7:0/1467023720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f712c08a050 con 0x7f713c102ac0 2024-09-21T05:10:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.591+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f713c19af70 con 0x7f713c102ac0 2024-09-21T05:10:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.591+0000 7f7138d99640 1 -- 172.21.15.7:0/1467023720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f712c08ef00 con 0x7f713c102ac0 2024-09-21T05:10:58.595 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:10:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:10:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:10:58.597 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:10:58.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.593+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f712807af30 msgr2=0x7f712807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.593+0000 7f714108d640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f712807af30 0x7f712807d3f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f713c0fee80 tx=0x7f7124009060 comp rx=0 tx=0).stop 2024-09-21T05:10:58.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.593+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 msgr2=0x7f713c19f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.593+0000 7f714108d640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 0x7f713c19f880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f712c031460 tx=0x7f712c0048f0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.594+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 shutdown_connections 2024-09-21T05:10:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.594+0000 7f714108d640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f712807af30 0x7f712807d3f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.594+0000 7f714108d640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f713c1033e0 0x7f713c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.594+0000 7f714108d640 1 --2- 172.21.15.7:0/1467023720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f713c102ac0 0x7f713c19f880 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.594+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 >> 172.21.15.7:0/1467023720 conn(0x7f713c0fc520 msgr2=0x7f713c100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.594+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 shutdown_connections 2024-09-21T05:10:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.594+0000 7f714108d640 1 -- 172.21.15.7:0/1467023720 wait complete. 2024-09-21T05:10:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 -- 172.21.15.7:0/2592200824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 msgr2=0x7f1a5c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 --2- 172.21.15.7:0/2592200824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 0x7f1a5c102e90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009880 tx=0x7f1a4c02a530 comp rx=0 tx=0).stop 2024-09-21T05:10:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 -- 172.21.15.7:0/2592200824 shutdown_connections 2024-09-21T05:10:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 --2- 172.21.15.7:0/2592200824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c1033d0 0x7f1a5c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 --2- 172.21.15.7:0/2592200824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 0x7f1a5c102e90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 -- 172.21.15.7:0/2592200824 >> 172.21.15.7:0/2592200824 conn(0x7f1a5c0fc550 msgr2=0x7f1a5c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 -- 172.21.15.7:0/2592200824 shutdown_connections 2024-09-21T05:10:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 -- 172.21.15.7:0/2592200824 wait complete. 2024-09-21T05:10:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 Processor -- start 2024-09-21T05:10:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.678+0000 7f1a62aab640 1 -- start start 2024-09-21T05:10:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a62aab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 0x7f1a5c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a62aab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c1033d0 0x7f1a5c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a62aab640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c10ede0 con 0x7f1a5c102ab0 2024-09-21T05:10:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a62aab640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c10ec60 con 0x7f1a5c1033d0 2024-09-21T05:10:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a60847640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 0x7f1a5c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a60847640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 0x7f1a5c19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45504/0 (socket says 172.21.15.7:45504) 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a60847640 1 -- 172.21.15.7:0/2650587768 learned_addr learned my addr 172.21.15.7:0/2650587768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a5bfff640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c1033d0 0x7f1a5c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a60847640 1 -- 172.21.15.7:0/2650587768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c1033d0 msgr2=0x7f1a5c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a60847640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c1033d0 0x7f1a5c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a60847640 1 -- 172.21.15.7:0/2650587768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c199a20 con 0x7f1a5c102ab0 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a5bfff640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c1033d0 0x7f1a5c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a60847640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 0x7f1a5c19f810 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c031460 tx=0x7f1a4c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a59ffb640 1 -- 172.21.15.7:0/2650587768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1a4c03f070 con 0x7f1a5c102ab0 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.680+0000 7f1a59ffb640 1 -- 172.21.15.7:0/2650587768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a4c004cf0 con 0x7f1a5c102ab0 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.679+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5c199cb0 con 0x7f1a5c102ab0 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.680+0000 7f1a59ffb640 1 -- 172.21.15.7:0/2650587768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1a4c035030 con 0x7f1a5c102ab0 2024-09-21T05:10:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.680+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5c19a1e0 con 0x7f1a5c102ab0 2024-09-21T05:10:58.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.681+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a5c06a580 con 0x7f1a5c102ab0 2024-09-21T05:10:58.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.681+0000 7f1a59ffb640 1 -- 172.21.15.7:0/2650587768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1a4c042070 con 0x7f1a5c102ab0 2024-09-21T05:10:58.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.682+0000 7f1a59ffb640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1a4407af30 0x7f1a4407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:58.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.682+0000 7f1a59ffb640 1 -- 172.21.15.7:0/2650587768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f1a4c0beb20 con 0x7f1a5c102ab0 2024-09-21T05:10:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.682+0000 7f1a5bfff640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1a4407af30 0x7f1a4407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.683+0000 7f1a5bfff640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1a4407af30 0x7f1a4407d3f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0feeb0 tx=0x7f1a50005dd0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.686+0000 7f1a59ffb640 1 -- 172.21.15.7:0/2650587768 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a4c08a0c0 con 0x7f1a5c102ab0 2024-09-21T05:10:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.911+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1a5c19af00 con 0x7f1a5c102ab0 2024-09-21T05:10:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.911+0000 7f1a59ffb640 1 -- 172.21.15.7:0/2650587768 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f1a4c08ef70 con 0x7f1a5c102ab0 2024-09-21T05:10:58.916 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:10:58.916 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:10:58.916 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:10:58.916 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:10:58.916 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:10:58.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:10:58.917 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:10:58.917 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:10:58.917 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:10:58.917 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:10:58.917 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:10:58.917 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:10:58.918 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:10:58.918 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:10:58.918 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:10:58.918 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:10:58.918 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:10:58.918 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:10:58.918 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:10:58.918 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:10:58.919 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:10:58.919 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:10:58.919 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:10:58.919 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:10:58.919 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:10:58.919 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:10:58.919 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:10:58.920 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:10:58.920 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:10:58.920 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:10:58.920 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:10:58.920 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:10:58.920 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:10:58.920 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:10:58.921 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:10:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:10:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.913+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1a4407af30 msgr2=0x7f1a4407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.913+0000 7f1a62aab640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1a4407af30 0x7f1a4407d3f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0feeb0 tx=0x7f1a50005dd0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.913+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 msgr2=0x7f1a5c19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.913+0000 7f1a62aab640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 0x7f1a5c19f810 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c031460 tx=0x7f1a4c0048f0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.914+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 shutdown_connections 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.914+0000 7f1a62aab640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f1a4407af30 0x7f1a4407d3f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.914+0000 7f1a62aab640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a5c1033d0 0x7f1a5c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.914+0000 7f1a62aab640 1 --2- 172.21.15.7:0/2650587768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c102ab0 0x7f1a5c19f810 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.914+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 >> 172.21.15.7:0/2650587768 conn(0x7f1a5c0fc550 msgr2=0x7f1a5c100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.914+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 shutdown_connections 2024-09-21T05:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:58.914+0000 7f1a62aab640 1 -- 172.21.15.7:0/2650587768 wait complete. 2024-09-21T05:10:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.000+0000 7f32ac8fc640 1 -- 172.21.15.7:0/77418348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 msgr2=0x7f32a8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.000+0000 7f32ac8fc640 1 --2- 172.21.15.7:0/77418348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 0x7f32a8100c60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f329c009880 tx=0x7f329c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.001+0000 7f32ac8fc640 1 -- 172.21.15.7:0/77418348 shutdown_connections 2024-09-21T05:10:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.001+0000 7f32ac8fc640 1 --2- 172.21.15.7:0/77418348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 0x7f32a8100c60 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.001+0000 7f32ac8fc640 1 --2- 172.21.15.7:0/77418348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8106800 0x7f32a8106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.001+0000 7f32ac8fc640 1 -- 172.21.15.7:0/77418348 >> 172.21.15.7:0/77418348 conn(0x7f32a80fc520 msgr2=0x7f32a80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.001+0000 7f32ac8fc640 1 -- 172.21.15.7:0/77418348 shutdown_connections 2024-09-21T05:10:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.001+0000 7f32ac8fc640 1 -- 172.21.15.7:0/77418348 wait complete. 2024-09-21T05:10:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32ac8fc640 1 Processor -- start 2024-09-21T05:10:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32ac8fc640 1 -- start start 2024-09-21T05:10:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32ac8fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 0x7f32a819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32ac8fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8106800 0x7f32a819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32ac8fc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810dcd0 con 0x7f32a8100800 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32ac8fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f32a810db50 con 0x7f32a8106800 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32a659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 0x7f32a819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32a659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 0x7f32a819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45520/0 (socket says 172.21.15.7:45520) 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32a659c640 1 -- 172.21.15.7:0/2112907673 learned_addr learned my addr 172.21.15.7:0/2112907673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32a5d9b640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8106800 0x7f32a819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32a659c640 1 -- 172.21.15.7:0/2112907673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8106800 msgr2=0x7f32a819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32a659c640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8106800 0x7f32a819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32a659c640 1 -- 172.21.15.7:0/2112907673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a81966c0 con 0x7f32a8100800 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.002+0000 7f32a5d9b640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8106800 0x7f32a819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.003+0000 7f32a659c640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 0x7f32a819c5f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f329800d6f0 tx=0x7f329800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.003+0000 7f32977fe640 1 -- 172.21.15.7:0/2112907673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3298017070 con 0x7f32a8100800 2024-09-21T05:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.003+0000 7f32977fe640 1 -- 172.21.15.7:0/2112907673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f329800de00 con 0x7f32a8100800 2024-09-21T05:10:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.003+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a81969b0 con 0x7f32a8100800 2024-09-21T05:10:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.003+0000 7f32977fe640 1 -- 172.21.15.7:0/2112907673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3298004bb0 con 0x7f32a8100800 2024-09-21T05:10:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.003+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a8196f10 con 0x7f32a8100800 2024-09-21T05:10:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.004+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a8101f40 con 0x7f32a8100800 2024-09-21T05:10:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.005+0000 7f32977fe640 1 -- 172.21.15.7:0/2112907673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3298009a30 con 0x7f32a8100800 2024-09-21T05:10:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.005+0000 7f32977fe640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f329007af90 0x7f329007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.005+0000 7f32977fe640 1 -- 172.21.15.7:0/2112907673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f329809dbd0 con 0x7f32a8100800 2024-09-21T05:10:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.005+0000 7f32a5d9b640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f329007af90 0x7f329007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.006+0000 7f32a5d9b640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f329007af90 0x7f329007d450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f329c031460 tx=0x7f329c0041a0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.009+0000 7f32977fe640 1 -- 172.21.15.7:0/2112907673 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32980691f0 con 0x7f32a8100800 2024-09-21T05:10:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.214+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f32a81971c0 con 0x7f329007af90 2024-09-21T05:10:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.216+0000 7f32977fe640 1 -- 172.21.15.7:0/2112907673 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f32a81971c0 con 0x7f329007af90 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:10:59.220 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:10:59.221 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:10:59.221 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:10:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f329007af90 msgr2=0x7f329007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f329007af90 0x7f329007d450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f329c031460 tx=0x7f329c0041a0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 msgr2=0x7f32a819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 0x7f32a819c5f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f329800d6f0 tx=0x7f329800dbb0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 shutdown_connections 2024-09-21T05:10:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f329007af90 0x7f329007d450 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32a8106800 0x7f32a819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 --2- 172.21.15.7:0/2112907673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32a8100800 0x7f32a819c5f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 >> 172.21.15.7:0/2112907673 conn(0x7f32a80fc520 msgr2=0x7f32a80fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 shutdown_connections 2024-09-21T05:10:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.218+0000 7f32ac8fc640 1 -- 172.21.15.7:0/2112907673 wait complete. 2024-09-21T05:10:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 -- 172.21.15.7:0/3586352011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100ff0 msgr2=0x7fd7b010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 --2- 172.21.15.7:0/3586352011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100ff0 0x7fd7b010dcb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009880 tx=0x7fd7a402a490 comp rx=0 tx=0).stop 2024-09-21T05:10:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 -- 172.21.15.7:0/3586352011 shutdown_connections 2024-09-21T05:10:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 --2- 172.21.15.7:0/3586352011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100ff0 0x7fd7b010dcb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 --2- 172.21.15.7:0/3586352011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7b0100640 0x7fd7b0100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 -- 172.21.15.7:0/3586352011 >> 172.21.15.7:0/3586352011 conn(0x7fd7b00fc550 msgr2=0x7fd7b00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 -- 172.21.15.7:0/3586352011 shutdown_connections 2024-09-21T05:10:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 -- 172.21.15.7:0/3586352011 wait complete. 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 Processor -- start 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.303+0000 7fd7b7c89640 1 -- start start 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b7c89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100640 0x7fd7b019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b7c89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7b0100ff0 0x7fd7b019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b7c89640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b0110ff0 con 0x7fd7b0100640 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b7c89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b0110e70 con 0x7fd7b0100ff0 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b5a25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100640 0x7fd7b019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b5a25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100640 0x7fd7b019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45534/0 (socket says 172.21.15.7:45534) 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b5a25640 1 -- 172.21.15.7:0/1094452991 learned_addr learned my addr 172.21.15.7:0/1094452991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:10:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b5a25640 1 -- 172.21.15.7:0/1094452991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7b0100ff0 msgr2=0x7fd7b019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b5a25640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7b0100ff0 0x7fd7b019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b5a25640 1 -- 172.21.15.7:0/1094452991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b0199a60 con 0x7fd7b0100640 2024-09-21T05:10:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7b5a25640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100640 0x7fd7b019f8a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000ba50 tx=0x7fd7a000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.304+0000 7fd7aeffd640 1 -- 172.21.15.7:0/1094452991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd7a0019070 con 0x7fd7b0100640 2024-09-21T05:10:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.305+0000 7fd7aeffd640 1 -- 172.21.15.7:0/1094452991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd7a000ce20 con 0x7fd7b0100640 2024-09-21T05:10:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.305+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b0199d50 con 0x7fd7b0100640 2024-09-21T05:10:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.305+0000 7fd7aeffd640 1 -- 172.21.15.7:0/1094452991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd7a00075a0 con 0x7fd7b0100640 2024-09-21T05:10:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.305+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b019a2e0 con 0x7fd7b0100640 2024-09-21T05:10:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.306+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b006a580 con 0x7fd7b0100640 2024-09-21T05:10:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.307+0000 7fd7aeffd640 1 -- 172.21.15.7:0/1094452991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd7a001c070 con 0x7fd7b0100640 2024-09-21T05:10:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.307+0000 7fd7aeffd640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd79807af80 0x7fd79807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:10:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.307+0000 7fd7aeffd640 1 -- 172.21.15.7:0/1094452991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fd7a009d900 con 0x7fd7b0100640 2024-09-21T05:10:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.307+0000 7fd7b5224640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd79807af80 0x7fd79807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:10:59.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.308+0000 7fd7b5224640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd79807af80 0x7fd79807d440 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40096c0 tx=0x7fd7a4000bc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:10:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.311+0000 7fd7aeffd640 1 -- 172.21.15.7:0/1094452991 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd7a0068e80 con 0x7fd7b0100640 2024-09-21T05:10:59.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.575+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fd7b019af90 con 0x7fd7b0100640 2024-09-21T05:10:59.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:59 smithi007 ceph-mon[108238]: pgmap v1087: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:59.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:59 smithi007 ceph-mon[108238]: from='client.35768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:59.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:59 smithi007 ceph-mon[108238]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:59.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:59 smithi007 ceph-mon[108238]: from='client.35776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:59.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:59.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:59 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1467023720' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:10:59.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:10:59 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2650587768' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:10:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.576+0000 7fd7aeffd640 1 -- 172.21.15.7:0/1094452991 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fd7a006dd30 con 0x7fd7b0100640 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:10:59.581 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:10:59.582 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:10:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd79807af80 msgr2=0x7fd79807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd79807af80 0x7fd79807d440 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40096c0 tx=0x7fd7a4000bc0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100640 msgr2=0x7fd7b019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:10:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100640 0x7fd7b019f8a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000ba50 tx=0x7fd7a000bf10 comp rx=0 tx=0).stop 2024-09-21T05:10:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 shutdown_connections 2024-09-21T05:10:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd79807af80 0x7fd79807d440 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7b0100ff0 0x7fd7b019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 --2- 172.21.15.7:0/1094452991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b0100640 0x7fd7b019f8a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:10:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 >> 172.21.15.7:0/1094452991 conn(0x7fd7b00fc550 msgr2=0x7fd7b00fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:10:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 shutdown_connections 2024-09-21T05:10:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:10:59.578+0000 7fd7b7c89640 1 -- 172.21.15.7:0/1094452991 wait complete. 2024-09-21T05:10:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:59 smithi053 ceph-mon[80176]: pgmap v1087: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:10:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:59 smithi053 ceph-mon[80176]: from='client.35768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:59 smithi053 ceph-mon[80176]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:59 smithi053 ceph-mon[80176]: from='client.35776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:10:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:10:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:59 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1467023720' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:10:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:10:59 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2650587768' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:11:00.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:00 smithi053 ceph-mon[80176]: from='client.35788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:00.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:00 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1094452991' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:11:00.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:00 smithi007 ceph-mon[108238]: from='client.35788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:00.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:00 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1094452991' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:11:01.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:01 smithi053 ceph-mon[80176]: pgmap v1088: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:01 smithi007 ceph-mon[108238]: pgmap v1088: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:03.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:03 smithi053 ceph-mon[80176]: pgmap v1089: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:03.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:03 smithi007 ceph-mon[108238]: pgmap v1089: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:05.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:05 smithi053 ceph-mon[80176]: pgmap v1090: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:05.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:05 smithi007 ceph-mon[108238]: pgmap v1090: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:07.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:07 smithi053 ceph-mon[80176]: pgmap v1091: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:07.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:07 smithi007 ceph-mon[108238]: pgmap v1091: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:09.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:09 smithi053 ceph-mon[80176]: pgmap v1092: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:09.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:09 smithi007 ceph-mon[108238]: pgmap v1092: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:11.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:11 smithi053 ceph-mon[80176]: pgmap v1093: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:11.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:11 smithi007 ceph-mon[108238]: pgmap v1093: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:13.612 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:13 smithi007 ceph-mon[108238]: pgmap v1094: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:13.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:13 smithi053 ceph-mon[80176]: pgmap v1094: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:14.594 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:11:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:11:15.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:15 smithi053 ceph-mon[80176]: pgmap v1095: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:15.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:15 smithi007 ceph-mon[108238]: pgmap v1095: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:17 smithi053 ceph-mon[80176]: pgmap v1096: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:17 smithi007 ceph-mon[108238]: pgmap v1096: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:19.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:19 smithi053 ceph-mon[80176]: pgmap v1097: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:19 smithi007 ceph-mon[108238]: pgmap v1097: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:21.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:21 smithi053 ceph-mon[80176]: pgmap v1098: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:21.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:21 smithi007 ceph-mon[108238]: pgmap v1098: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:23.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:23 smithi053 ceph-mon[80176]: pgmap v1099: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:23 smithi007 ceph-mon[108238]: pgmap v1099: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:25.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:25 smithi007 ceph-mon[108238]: pgmap v1100: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:25.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:25 smithi053 ceph-mon[80176]: pgmap v1100: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:27.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:27 smithi053 ceph-mon[80176]: pgmap v1101: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:27 smithi007 ceph-mon[108238]: pgmap v1101: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.677+0000 7fd565bd4640 1 -- 172.21.15.7:0/282629010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5601069a0 msgr2=0x7fd560106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.677+0000 7fd565bd4640 1 --2- 172.21.15.7:0/282629010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5601069a0 0x7fd560106d80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd550009880 tx=0x7fd55002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.677+0000 7fd565bd4640 1 -- 172.21.15.7:0/282629010 shutdown_connections 2024-09-21T05:11:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.677+0000 7fd565bd4640 1 --2- 172.21.15.7:0/282629010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5600ffdc0 0x7fd560100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.677+0000 7fd565bd4640 1 --2- 172.21.15.7:0/282629010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5601069a0 0x7fd560106d80 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.677+0000 7fd565bd4640 1 -- 172.21.15.7:0/282629010 >> 172.21.15.7:0/282629010 conn(0x7fd5600fbc90 msgr2=0x7fd5600fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.677+0000 7fd565bd4640 1 -- 172.21.15.7:0/282629010 shutdown_connections 2024-09-21T05:11:29.681 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:29 smithi007 ceph-mon[108238]: pgmap v1102: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:29.682 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:11:29.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.677+0000 7fd565bd4640 1 -- 172.21.15.7:0/282629010 wait complete. 2024-09-21T05:11:29.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd565bd4640 1 Processor -- start 2024-09-21T05:11:29.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd565bd4640 1 -- start start 2024-09-21T05:11:29.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd565bd4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5600ffdc0 0x7fd560100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:29.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd565bd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5601069a0 0x7fd5601014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:29.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd565bd4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd56010ef40 con 0x7fd5600ffdc0 2024-09-21T05:11:29.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd565bd4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd56010edc0 con 0x7fd5601069a0 2024-09-21T05:11:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd55f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5600ffdc0 0x7fd560100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd55f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5600ffdc0 0x7fd560100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50966/0 (socket says 172.21.15.7:50966) 2024-09-21T05:11:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.678+0000 7fd55f7fe640 1 -- 172.21.15.7:0/1399091235 learned_addr learned my addr 172.21.15.7:0/1399091235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:11:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd55effd640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5601069a0 0x7fd5601014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd55f7fe640 1 -- 172.21.15.7:0/1399091235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5601069a0 msgr2=0x7fd5601014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd55f7fe640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5601069a0 0x7fd5601014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd55f7fe640 1 -- 172.21.15.7:0/1399091235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd560101a20 con 0x7fd5600ffdc0 2024-09-21T05:11:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd55f7fe640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5600ffdc0 0x7fd560100fa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd550031460 tx=0x7fd550004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd55cff9640 1 -- 172.21.15.7:0/1399091235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd55003f070 con 0x7fd5600ffdc0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd560101cb0 con 0x7fd5600ffdc0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd55cff9640 1 -- 172.21.15.7:0/1399091235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5500035f0 con 0x7fd5600ffdc0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd55cff9640 1 -- 172.21.15.7:0/1399091235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd550035030 con 0x7fd5600ffdc0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.679+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5601a3eb0 con 0x7fd5600ffdc0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.680+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5601030e0 con 0x7fd5600ffdc0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.681+0000 7fd55cff9640 1 -- 172.21.15.7:0/1399091235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd550042070 con 0x7fd5600ffdc0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.682+0000 7fd55cff9640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd54807af30 0x7fd54807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.682+0000 7fd55cff9640 1 -- 172.21.15.7:0/1399091235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fd5500beab0 con 0x7fd5600ffdc0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.682+0000 7fd55effd640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd54807af30 0x7fd54807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.682+0000 7fd55effd640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd54807af30 0x7fd54807d3f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd560102a30 tx=0x7fd554005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:29.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.686+0000 7fd55cff9640 1 -- 172.21.15.7:0/1399091235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd550089fb0 con 0x7fd5600ffdc0 2024-09-21T05:11:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:29 smithi053 ceph-mon[80176]: pgmap v1102: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:11:29.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.892+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd560062630 con 0x7fd54807af30 2024-09-21T05:11:29.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.893+0000 7fd55cff9640 1 -- 172.21.15.7:0/1399091235 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fd560062630 con 0x7fd54807af30 2024-09-21T05:11:29.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd54807af30 msgr2=0x7fd54807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:29.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd54807af30 0x7fd54807d3f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd560102a30 tx=0x7fd554005f90 comp rx=0 tx=0).stop 2024-09-21T05:11:29.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5600ffdc0 msgr2=0x7fd560100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5600ffdc0 0x7fd560100fa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd550031460 tx=0x7fd550004530 comp rx=0 tx=0).stop 2024-09-21T05:11:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 shutdown_connections 2024-09-21T05:11:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd54807af30 0x7fd54807d3f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5601069a0 0x7fd5601014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 --2- 172.21.15.7:0/1399091235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5600ffdc0 0x7fd560100fa0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.895+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 >> 172.21.15.7:0/1399091235 conn(0x7fd5600fbc90 msgr2=0x7fd5600fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.896+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 shutdown_connections 2024-09-21T05:11:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.896+0000 7fd565bd4640 1 -- 172.21.15.7:0/1399091235 wait complete. 2024-09-21T05:11:29.907 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:11:29.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.982+0000 7fc84da51640 1 -- 172.21.15.7:0/2684174223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 msgr2=0x7fc8481004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.982+0000 7fc84da51640 1 --2- 172.21.15.7:0/2684174223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 0x7fc8481004c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc838009880 tx=0x7fc83802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.982+0000 7fc84da51640 1 -- 172.21.15.7:0/2684174223 shutdown_connections 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.982+0000 7fc84da51640 1 --2- 172.21.15.7:0/2684174223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc848100a90 0x7fc8481065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.982+0000 7fc84da51640 1 --2- 172.21.15.7:0/2684174223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 0x7fc8481004c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.982+0000 7fc84da51640 1 -- 172.21.15.7:0/2684174223 >> 172.21.15.7:0/2684174223 conn(0x7fc8480fbc90 msgr2=0x7fc8480fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.982+0000 7fc84da51640 1 -- 172.21.15.7:0/2684174223 shutdown_connections 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.982+0000 7fc84da51640 1 -- 172.21.15.7:0/2684174223 wait complete. 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc84da51640 1 Processor -- start 2024-09-21T05:11:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc84da51640 1 -- start start 2024-09-21T05:11:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc84da51640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 0x7fc84819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc84da51640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc848100a90 0x7fc84819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc84da51640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810ee30 con 0x7fc8481000e0 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc84da51640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810ecb0 con 0x7fc848100a90 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 0x7fc84819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 0x7fc84819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50970/0 (socket says 172.21.15.7:50970) 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.983+0000 7fc846ffd640 1 -- 172.21.15.7:0/662027801 learned_addr learned my addr 172.21.15.7:0/662027801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc8467fc640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc848100a90 0x7fc84819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc846ffd640 1 -- 172.21.15.7:0/662027801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc848100a90 msgr2=0x7fc84819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc846ffd640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc848100a90 0x7fc84819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc846ffd640 1 -- 172.21.15.7:0/662027801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481a0450 con 0x7fc8481000e0 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc846ffd640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 0x7fc84819f860 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc838031460 tx=0x7fc8380048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc84ca4f640 1 -- 172.21.15.7:0/662027801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc83803f070 con 0x7fc8481000e0 2024-09-21T05:11:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc84ca4f640 1 -- 172.21.15.7:0/662027801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc838004cf0 con 0x7fc8481000e0 2024-09-21T05:11:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc848199ae0 con 0x7fc8481000e0 2024-09-21T05:11:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc84ca4f640 1 -- 172.21.15.7:0/662027801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc838035030 con 0x7fc8481000e0 2024-09-21T05:11:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.984+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc848199fe0 con 0x7fc8481000e0 2024-09-21T05:11:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.985+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8481031a0 con 0x7fc8481000e0 2024-09-21T05:11:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.986+0000 7fc84ca4f640 1 -- 172.21.15.7:0/662027801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc838042070 con 0x7fc8481000e0 2024-09-21T05:11:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.986+0000 7fc84ca4f640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc83007af30 0x7fc83007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.986+0000 7fc84ca4f640 1 -- 172.21.15.7:0/662027801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc8380bea30 con 0x7fc8481000e0 2024-09-21T05:11:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.987+0000 7fc8467fc640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc83007af30 0x7fc83007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.987+0000 7fc8467fc640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc83007af30 0x7fc83007d3f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc8480fdc10 tx=0x7fc83c00a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:29.991+0000 7fc84ca4f640 1 -- 172.21.15.7:0/662027801 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc838089fd0 con 0x7fc8481000e0 2024-09-21T05:11:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.198+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc84819b060 con 0x7fc83007af30 2024-09-21T05:11:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.199+0000 7fc84ca4f640 1 -- 172.21.15.7:0/662027801 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc84819b060 con 0x7fc83007af30 2024-09-21T05:11:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc83007af30 msgr2=0x7fc83007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc83007af30 0x7fc83007d3f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc8480fdc10 tx=0x7fc83c00a2d0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 msgr2=0x7fc84819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 0x7fc84819f860 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc838031460 tx=0x7fc8380048f0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 shutdown_connections 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc83007af30 0x7fc83007d3f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc848100a90 0x7fc84819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 --2- 172.21.15.7:0/662027801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8481000e0 0x7fc84819f860 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 >> 172.21.15.7:0/662027801 conn(0x7fc8480fbc90 msgr2=0x7fc8480fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 shutdown_connections 2024-09-21T05:11:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.202+0000 7fc84da51640 1 -- 172.21.15.7:0/662027801 wait complete. 2024-09-21T05:11:30.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 -- 172.21.15.7:0/1422753414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb8102ab0 msgr2=0x7f2cb8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 --2- 172.21.15.7:0/1422753414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb8102ab0 0x7f2cb8102e90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009880 tx=0x7f2ca802a530 comp rx=0 tx=0).stop 2024-09-21T05:11:30.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 -- 172.21.15.7:0/1422753414 shutdown_connections 2024-09-21T05:11:30.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 --2- 172.21.15.7:0/1422753414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cb81033d0 0x7f2cb810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 --2- 172.21.15.7:0/1422753414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb8102ab0 0x7f2cb8102e90 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 -- 172.21.15.7:0/1422753414 >> 172.21.15.7:0/1422753414 conn(0x7f2cb80fc550 msgr2=0x7f2cb80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:30.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 -- 172.21.15.7:0/1422753414 shutdown_connections 2024-09-21T05:11:30.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 -- 172.21.15.7:0/1422753414 wait complete. 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.286+0000 7f2cbdf98640 1 Processor -- start 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cbdf98640 1 -- start start 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cbdf98640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cb8102ab0 0x7f2cb819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cbdf98640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb81033d0 0x7f2cb819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cbdf98640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb810ede0 con 0x7f2cb81033d0 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cbdf98640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb810ec60 con 0x7f2cb8102ab0 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cb6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb81033d0 0x7f2cb819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cb6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb81033d0 0x7f2cb819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50980/0 (socket says 172.21.15.7:50980) 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cb6ffd640 1 -- 172.21.15.7:0/2082067435 learned_addr learned my addr 172.21.15.7:0/2082067435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:11:30.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cb77fe640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cb8102ab0 0x7f2cb819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cb6ffd640 1 -- 172.21.15.7:0/2082067435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cb8102ab0 msgr2=0x7f2cb819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cb6ffd640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cb8102ab0 0x7f2cb819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.287+0000 7f2cb6ffd640 1 -- 172.21.15.7:0/2082067435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb81977b0 con 0x7f2cb81033d0 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.288+0000 7f2cb6ffd640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb81033d0 0x7f2cb819db70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c8f0 tx=0x7f2cac00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.288+0000 7f2cb4ff9640 1 -- 172.21.15.7:0/2082067435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2cac00ae50 con 0x7f2cb81033d0 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.288+0000 7f2cb4ff9640 1 -- 172.21.15.7:0/2082067435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2cac005020 con 0x7f2cb81033d0 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.288+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb8197aa0 con 0x7f2cb81033d0 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.288+0000 7f2cb4ff9640 1 -- 172.21.15.7:0/2082067435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2cac004d70 con 0x7f2cb81033d0 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.288+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb8198030 con 0x7f2cb81033d0 2024-09-21T05:11:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.289+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb806a580 con 0x7f2cb81033d0 2024-09-21T05:11:30.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.290+0000 7f2cb4ff9640 1 -- 172.21.15.7:0/2082067435 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2cac0051d0 con 0x7f2cb81033d0 2024-09-21T05:11:30.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.291+0000 7f2cb4ff9640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2ca007b0e0 0x7f2ca007d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.291+0000 7f2cb4ff9640 1 -- 172.21.15.7:0/2082067435 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f2cac09de10 con 0x7f2cb81033d0 2024-09-21T05:11:30.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.291+0000 7f2cb77fe640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2ca007b0e0 0x7f2ca007d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.291+0000 7f2cb77fe640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2ca007b0e0 0x7f2ca007d5a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c00 tx=0x7f2ca8031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:30.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.296+0000 7f2cb4ff9640 1 -- 172.21.15.7:0/2082067435 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2cac014030 con 0x7f2cb81033d0 2024-09-21T05:11:30.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.493+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2cb8198350 con 0x7f2ca007b0e0 2024-09-21T05:11:30.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.503+0000 7f2cb4ff9640 1 -- 172.21.15.7:0/2082067435 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f2cb8198350 con 0x7f2ca007b0e0 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (33m) 12m ago 44m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (36m) 12m ago 44m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (36m) 12m ago 43m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (31m) 12m ago 44m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (31m) 12m ago 43m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (33m) 12m ago 44m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (27m) 12m ago 39m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:11:30.506 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (27m) 12m ago 39m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (12m) 12m ago 39m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (27m) 12m ago 39m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (35m) 12m ago 45m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (34m) 12m ago 43m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (32m) 12m ago 45m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (31m) 12m ago 43m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (34m) 12m ago 44m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (34m) 12m ago 43m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (31m) 12m ago 42m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (30m) 12m ago 41m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (30m) 12m ago 41m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (29m) 12m ago 41m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (28m) 12m ago 40m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:11:30.507 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (28m) 12m ago 40m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:11:30.508 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (33m) 12m ago 44m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:11:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2ca007b0e0 msgr2=0x7f2ca007d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2ca007b0e0 0x7f2ca007d5a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c00 tx=0x7f2ca8031400 comp rx=0 tx=0).stop 2024-09-21T05:11:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb81033d0 msgr2=0x7f2cb819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb81033d0 0x7f2cb819db70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c8f0 tx=0x7f2cac00cdb0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 shutdown_connections 2024-09-21T05:11:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2ca007b0e0 0x7f2ca007d5a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cb81033d0 0x7f2cb819db70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 --2- 172.21.15.7:0/2082067435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cb8102ab0 0x7f2cb819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 >> 172.21.15.7:0/2082067435 conn(0x7f2cb80fc550 msgr2=0x7f2cb810aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 shutdown_connections 2024-09-21T05:11:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.505+0000 7f2cbdf98640 1 -- 172.21.15.7:0/2082067435 wait complete. 2024-09-21T05:11:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.589+0000 7f1404cb3640 1 -- 172.21.15.7:0/1287610174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400101900 msgr2=0x7f1400101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.589+0000 7f1404cb3640 1 --2- 172.21.15.7:0/1287610174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400101900 0x7f1400101d60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009880 tx=0x7f13f402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.589+0000 7f1404cb3640 1 -- 172.21.15.7:0/1287610174 shutdown_connections 2024-09-21T05:11:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.589+0000 7f1404cb3640 1 --2- 172.21.15.7:0/1287610174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400101900 0x7f1400101d60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.589+0000 7f1404cb3640 1 --2- 172.21.15.7:0/1287610174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1400107900 0x7f1400107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.589+0000 7f1404cb3640 1 -- 172.21.15.7:0/1287610174 >> 172.21.15.7:0/1287610174 conn(0x7f14000fd600 msgr2=0x7f14000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.589+0000 7f1404cb3640 1 -- 172.21.15.7:0/1287610174 shutdown_connections 2024-09-21T05:11:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.589+0000 7f1404cb3640 1 -- 172.21.15.7:0/1287610174 wait complete. 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f1404cb3640 1 Processor -- start 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f1404cb3640 1 -- start start 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f1404cb3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1400101900 0x7f140019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f1404cb3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400107900 0x7f140019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f1404cb3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f140010ee30 con 0x7f1400107900 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f1404cb3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f140010ecb0 con 0x7f1400101900 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f13fdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400107900 0x7f140019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f13fdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400107900 0x7f140019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50992/0 (socket says 172.21.15.7:50992) 2024-09-21T05:11:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f13fdd9b640 1 -- 172.21.15.7:0/3931655105 learned_addr learned my addr 172.21.15.7:0/3931655105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:11:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.590+0000 7f13fe59c640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1400101900 0x7f140019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.591+0000 7f13fdd9b640 1 -- 172.21.15.7:0/3931655105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1400101900 msgr2=0x7f140019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.591+0000 7f13fdd9b640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1400101900 0x7f140019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.591+0000 7f13fdd9b640 1 -- 172.21.15.7:0/3931655105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1400197840 con 0x7f1400107900 2024-09-21T05:11:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.591+0000 7f13fdd9b640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400107900 0x7f140019dc00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f13f4031460 tx=0x7f13f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.591+0000 7f13ef7fe640 1 -- 172.21.15.7:0/3931655105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13f403f070 con 0x7f1400107900 2024-09-21T05:11:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.591+0000 7f13ef7fe640 1 -- 172.21.15.7:0/3931655105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13f4004cf0 con 0x7f1400107900 2024-09-21T05:11:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.591+0000 7f13ef7fe640 1 -- 172.21.15.7:0/3931655105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13f4035030 con 0x7f1400107900 2024-09-21T05:11:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.592+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1400197ad0 con 0x7f1400107900 2024-09-21T05:11:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.592+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1400198000 con 0x7f1400107900 2024-09-21T05:11:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.593+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1400103040 con 0x7f1400107900 2024-09-21T05:11:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.594+0000 7f13ef7fe640 1 -- 172.21.15.7:0/3931655105 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f13f4042070 con 0x7f1400107900 2024-09-21T05:11:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.594+0000 7f13ef7fe640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13e807af30 0x7f13e807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.594+0000 7f13ef7fe640 1 -- 172.21.15.7:0/3931655105 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f13f40beaf0 con 0x7f1400107900 2024-09-21T05:11:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.594+0000 7f13fe59c640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13e807af30 0x7f13e807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.595+0000 7f13fe59c640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13e807af30 0x7f13e807d3f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009990 tx=0x7f13f0009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.599+0000 7f13ef7fe640 1 -- 172.21.15.7:0/3931655105 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13f408a090 con 0x7f1400107900 2024-09-21T05:11:30.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.864+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f1400198d70 con 0x7f1400107900 2024-09-21T05:11:30.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.864+0000 7f13ef7fe640 1 -- 172.21.15.7:0/3931655105 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f13f408ef40 con 0x7f1400107900 2024-09-21T05:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:11:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:11:30.868 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:11:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13e807af30 msgr2=0x7f13e807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13e807af30 0x7f13e807d3f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009990 tx=0x7f13f0009110 comp rx=0 tx=0).stop 2024-09-21T05:11:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400107900 msgr2=0x7f140019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400107900 0x7f140019dc00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f13f4031460 tx=0x7f13f40048f0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 shutdown_connections 2024-09-21T05:11:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f13e807af30 0x7f13e807d3f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1400107900 0x7f140019dc00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 --2- 172.21.15.7:0/3931655105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1400101900 0x7f140019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 >> 172.21.15.7:0/3931655105 conn(0x7f14000fd600 msgr2=0x7f14000fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 shutdown_connections 2024-09-21T05:11:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.866+0000 7f1404cb3640 1 -- 172.21.15.7:0/3931655105 wait complete. 2024-09-21T05:11:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.950+0000 7f2df14ff640 1 -- 172.21.15.7:0/917221495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec101900 msgr2=0x7f2dec101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.950+0000 7f2df14ff640 1 --2- 172.21.15.7:0/917221495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec101900 0x7f2dec101d60 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009880 tx=0x7f2de002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.950+0000 7f2df14ff640 1 -- 172.21.15.7:0/917221495 shutdown_connections 2024-09-21T05:11:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.950+0000 7f2df14ff640 1 --2- 172.21.15.7:0/917221495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec101900 0x7f2dec101d60 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.950+0000 7f2df14ff640 1 --2- 172.21.15.7:0/917221495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dec107900 0x7f2dec107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.950+0000 7f2df14ff640 1 -- 172.21.15.7:0/917221495 >> 172.21.15.7:0/917221495 conn(0x7f2dec0fd600 msgr2=0x7f2dec0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.950+0000 7f2df14ff640 1 -- 172.21.15.7:0/917221495 shutdown_connections 2024-09-21T05:11:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.950+0000 7f2df14ff640 1 -- 172.21.15.7:0/917221495 wait complete. 2024-09-21T05:11:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2df14ff640 1 Processor -- start 2024-09-21T05:11:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2df14ff640 1 -- start start 2024-09-21T05:11:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2df14ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dec101900 0x7f2dec19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2df14ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107900 0x7f2dec19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2df14ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec10edd0 con 0x7f2dec107900 2024-09-21T05:11:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2df14ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec10ec50 con 0x7f2dec101900 2024-09-21T05:11:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2dea7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107900 0x7f2dec19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2dea7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107900 0x7f2dec19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51010/0 (socket says 172.21.15.7:51010) 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.951+0000 7f2dea7fc640 1 -- 172.21.15.7:0/4266768887 learned_addr learned my addr 172.21.15.7:0/4266768887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2deaffd640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dec101900 0x7f2dec19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2dea7fc640 1 -- 172.21.15.7:0/4266768887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dec101900 msgr2=0x7f2dec19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2dea7fc640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dec101900 0x7f2dec19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2dea7fc640 1 -- 172.21.15.7:0/4266768887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec197800 con 0x7f2dec107900 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2deaffd640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dec101900 0x7f2dec19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2dea7fc640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107900 0x7f2dec19dbc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2de0031460 tx=0x7f2de0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2dd3fff640 1 -- 172.21.15.7:0/4266768887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2de003f070 con 0x7f2dec107900 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2dd3fff640 1 -- 172.21.15.7:0/4266768887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2de00035f0 con 0x7f2dec107900 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec197a90 con 0x7f2dec107900 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2dd3fff640 1 -- 172.21.15.7:0/4266768887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2de0035030 con 0x7f2dec107900 2024-09-21T05:11:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.952+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec197fc0 con 0x7f2dec107900 2024-09-21T05:11:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.953+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dec103040 con 0x7f2dec107900 2024-09-21T05:11:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.954+0000 7f2dd3fff640 1 -- 172.21.15.7:0/4266768887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2de0042070 con 0x7f2dec107900 2024-09-21T05:11:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.954+0000 7f2dd3fff640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2dd407af30 0x7f2dd407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.955+0000 7f2dd3fff640 1 -- 172.21.15.7:0/4266768887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f2de00beab0 con 0x7f2dec107900 2024-09-21T05:11:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.955+0000 7f2deaffd640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2dd407af30 0x7f2dd407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.955+0000 7f2deaffd640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2dd407af30 0x7f2dd407d3f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009c20 tx=0x7f2ddc005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:30.959+0000 7f2dd3fff640 1 -- 172.21.15.7:0/4266768887 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2de0089fb0 con 0x7f2dec107900 2024-09-21T05:11:31.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.184+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2dec198d30 con 0x7f2dec107900 2024-09-21T05:11:31.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.184+0000 7f2dd3fff640 1 -- 172.21.15.7:0/4266768887 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f2de008ee60 con 0x7f2dec107900 2024-09-21T05:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:11:31.187 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:11:31.188 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:11:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:11:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:11:31.191 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:11:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:11:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2dd407af30 msgr2=0x7f2dd407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2dd407af30 0x7f2dd407d3f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009c20 tx=0x7f2ddc005f90 comp rx=0 tx=0).stop 2024-09-21T05:11:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107900 msgr2=0x7f2dec19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107900 0x7f2dec19dbc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2de0031460 tx=0x7f2de0004530 comp rx=0 tx=0).stop 2024-09-21T05:11:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 shutdown_connections 2024-09-21T05:11:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2dd407af30 0x7f2dd407d3f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dec107900 0x7f2dec19dbc0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 --2- 172.21.15.7:0/4266768887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dec101900 0x7f2dec19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.187+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 >> 172.21.15.7:0/4266768887 conn(0x7f2dec0fd600 msgr2=0x7f2dec0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.188+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 shutdown_connections 2024-09-21T05:11:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.188+0000 7f2df14ff640 1 -- 172.21.15.7:0/4266768887 wait complete. 2024-09-21T05:11:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.273+0000 7fa36fa21640 1 -- 172.21.15.7:0/3718226491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3681033e0 msgr2=0x7fa36810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.273+0000 7fa36fa21640 1 --2- 172.21.15.7:0/3718226491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3681033e0 0x7fa36810b8d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa358009880 tx=0x7fa35802a530 comp rx=0 tx=0).stop 2024-09-21T05:11:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.273+0000 7fa36fa21640 1 -- 172.21.15.7:0/3718226491 shutdown_connections 2024-09-21T05:11:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.273+0000 7fa36fa21640 1 --2- 172.21.15.7:0/3718226491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3681033e0 0x7fa36810b8d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.273+0000 7fa36fa21640 1 --2- 172.21.15.7:0/3718226491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa368102ac0 0x7fa368102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.273+0000 7fa36fa21640 1 -- 172.21.15.7:0/3718226491 >> 172.21.15.7:0/3718226491 conn(0x7fa3680fc520 msgr2=0x7fa3680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.273+0000 7fa36fa21640 1 -- 172.21.15.7:0/3718226491 shutdown_connections 2024-09-21T05:11:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.273+0000 7fa36fa21640 1 -- 172.21.15.7:0/3718226491 wait complete. 2024-09-21T05:11:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36fa21640 1 Processor -- start 2024-09-21T05:11:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36fa21640 1 -- start start 2024-09-21T05:11:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36fa21640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa368102ac0 0x7fa36819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36fa21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3681033e0 0x7fa36819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36fa21640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa36810edf0 con 0x7fa368102ac0 2024-09-21T05:11:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36fa21640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa36810ec70 con 0x7fa3681033e0 2024-09-21T05:11:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36d7bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa368102ac0 0x7fa36819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36d7bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa368102ac0 0x7fa36819f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51026/0 (socket says 172.21.15.7:51026) 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36d7bd640 1 -- 172.21.15.7:0/3133923502 learned_addr learned my addr 172.21.15.7:0/3133923502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36cfbc640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3681033e0 0x7fa36819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36d7bd640 1 -- 172.21.15.7:0/3133923502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3681033e0 msgr2=0x7fa36819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36d7bd640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3681033e0 0x7fa36819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36d7bd640 1 -- 172.21.15.7:0/3133923502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa368199ab0 con 0x7fa368102ac0 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.274+0000 7fa36d7bd640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa368102ac0 0x7fa36819f8f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa36400e970 tx=0x7fa36400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.275+0000 7fa35e7fc640 1 -- 172.21.15.7:0/3133923502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa36400cac0 con 0x7fa368102ac0 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.275+0000 7fa35e7fc640 1 -- 172.21.15.7:0/3133923502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3640054e0 con 0x7fa368102ac0 2024-09-21T05:11:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.275+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa368199da0 con 0x7fa368102ac0 2024-09-21T05:11:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.275+0000 7fa35e7fc640 1 -- 172.21.15.7:0/3133923502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa364010430 con 0x7fa368102ac0 2024-09-21T05:11:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.275+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36819a300 con 0x7fa368102ac0 2024-09-21T05:11:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.276+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36806aa90 con 0x7fa368102ac0 2024-09-21T05:11:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.276+0000 7fa35e7fc640 1 -- 172.21.15.7:0/3133923502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa364004f00 con 0x7fa368102ac0 2024-09-21T05:11:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.277+0000 7fa35e7fc640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa35407af90 0x7fa35407d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.277+0000 7fa35e7fc640 1 -- 172.21.15.7:0/3133923502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fa36409e670 con 0x7fa368102ac0 2024-09-21T05:11:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.277+0000 7fa36cfbc640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa35407af90 0x7fa35407d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.277+0000 7fa36cfbc640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa35407af90 0x7fa35407d450 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa36819b2a0 tx=0x7fa358031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.281+0000 7fa35e7fc640 1 -- 172.21.15.7:0/3133923502 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa364069c90 con 0x7fa368102ac0 2024-09-21T05:11:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:31 smithi007 ceph-mon[108238]: pgmap v1103: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:31 smithi007 ceph-mon[108238]: from='client.35796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:31 smithi007 ceph-mon[108238]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:31 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3931655105' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:11:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:31 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4266768887' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:11:31.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.489+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa36819afe0 con 0x7fa35407af90 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.490+0000 7fa35e7fc640 1 -- 172.21.15.7:0/3133923502 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fa36819afe0 con 0x7fa35407af90 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:11:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:11:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:11:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:11:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:11:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:11:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:11:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:11:31.494 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:11:31.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa35407af90 msgr2=0x7fa35407d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa35407af90 0x7fa35407d450 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa36819b2a0 tx=0x7fa358031400 comp rx=0 tx=0).stop 2024-09-21T05:11:31.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa368102ac0 msgr2=0x7fa36819f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa368102ac0 0x7fa36819f8f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa36400e970 tx=0x7fa36400ed70 comp rx=0 tx=0).stop 2024-09-21T05:11:31.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 shutdown_connections 2024-09-21T05:11:31.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa35407af90 0x7fa35407d450 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3681033e0 0x7fa36819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 --2- 172.21.15.7:0/3133923502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa368102ac0 0x7fa36819f8f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 >> 172.21.15.7:0/3133923502 conn(0x7fa3680fc520 msgr2=0x7fa368100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:31.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 shutdown_connections 2024-09-21T05:11:31.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.492+0000 7fa36fa21640 1 -- 172.21.15.7:0/3133923502 wait complete. 2024-09-21T05:11:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.576+0000 7f59757c4640 1 -- 172.21.15.7:0/3580167832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 msgr2=0x7f5970074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.576+0000 7f59757c4640 1 --2- 172.21.15.7:0/3580167832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 0x7f5970074a00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5960009880 tx=0x7f596002a540 comp rx=0 tx=0).stop 2024-09-21T05:11:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.576+0000 7f59757c4640 1 -- 172.21.15.7:0/3580167832 shutdown_connections 2024-09-21T05:11:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.576+0000 7f59757c4640 1 --2- 172.21.15.7:0/3580167832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5970074f40 0x7f597010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.576+0000 7f59757c4640 1 --2- 172.21.15.7:0/3580167832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 0x7f5970074a00 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.576+0000 7f59757c4640 1 -- 172.21.15.7:0/3580167832 >> 172.21.15.7:0/3580167832 conn(0x7f59700fd600 msgr2=0x7f59700ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.576+0000 7f59757c4640 1 -- 172.21.15.7:0/3580167832 shutdown_connections 2024-09-21T05:11:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.576+0000 7f59757c4640 1 -- 172.21.15.7:0/3580167832 wait complete. 2024-09-21T05:11:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.577+0000 7f59757c4640 1 Processor -- start 2024-09-21T05:11:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.577+0000 7f59757c4640 1 -- start start 2024-09-21T05:11:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.577+0000 7f59757c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 0x7f597019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.577+0000 7f59757c4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5970074f40 0x7f597019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.577+0000 7f59757c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5970110fd0 con 0x7f5970074620 2024-09-21T05:11:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.577+0000 7f59757c4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5970110e50 con 0x7f5970074f40 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.577+0000 7f596effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 0x7f597019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 0x7f597019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51046/0 (socket says 172.21.15.7:51046) 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596effd640 1 -- 172.21.15.7:0/4151833908 learned_addr learned my addr 172.21.15.7:0/4151833908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f59667fc640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5970074f40 0x7f597019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596effd640 1 -- 172.21.15.7:0/4151833908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5970074f40 msgr2=0x7f597019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596effd640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5970074f40 0x7f597019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596effd640 1 -- 172.21.15.7:0/4151833908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59701a0410 con 0x7f5970074620 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f59667fc640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5970074f40 0x7f597019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596effd640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 0x7f597019f820 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5960031460 tx=0x7f59600048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596cff9640 1 -- 172.21.15.7:0/4151833908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f596003f070 con 0x7f5970074620 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596cff9640 1 -- 172.21.15.7:0/4151833908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59600044d0 con 0x7f5970074620 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5970199aa0 con 0x7f5970074620 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f596cff9640 1 -- 172.21.15.7:0/4151833908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5960035030 con 0x7f5970074620 2024-09-21T05:11:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.578+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5970199fa0 con 0x7f5970074620 2024-09-21T05:11:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.579+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5970075cd0 con 0x7f5970074620 2024-09-21T05:11:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.580+0000 7f596cff9640 1 -- 172.21.15.7:0/4151833908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5960042070 con 0x7f5970074620 2024-09-21T05:11:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.581+0000 7f596cff9640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f595c07af40 0x7f595c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:11:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.581+0000 7f59667fc640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f595c07af40 0x7f595c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:11:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.581+0000 7f596cff9640 1 -- 172.21.15.7:0/4151833908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f59600bebd0 con 0x7f5970074620 2024-09-21T05:11:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.581+0000 7f59667fc640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f595c07af40 0x7f595c07d400 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f597010ac30 tx=0x7f595800a5c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:11:31.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.585+0000 7f596cff9640 1 -- 172.21.15.7:0/4151833908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5960004cf0 con 0x7f5970074620 2024-09-21T05:11:31.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:31 smithi053 ceph-mon[80176]: pgmap v1103: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:31.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:31 smithi053 ceph-mon[80176]: from='client.35796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:31.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:31 smithi053 ceph-mon[80176]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:31.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:31 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3931655105' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:11:31.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:31 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4266768887' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.850+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f597019af10 con 0x7f5970074620 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.850+0000 7f596cff9640 1 -- 172.21.15.7:0/4151833908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f596008a170 con 0x7f5970074620 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:11:31.853 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:11:31.854 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:11:31.855 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:11:31.855 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:11:31.855 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:11:31.855 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:11:31.855 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:11:31.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f595c07af40 msgr2=0x7f595c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f595c07af40 0x7f595c07d400 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f597010ac30 tx=0x7f595800a5c0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 msgr2=0x7f597019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 0x7f597019f820 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5960031460 tx=0x7f59600048f0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 shutdown_connections 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f595c07af40 0x7f595c07d400 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5970074f40 0x7f597019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 --2- 172.21.15.7:0/4151833908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5970074620 0x7f597019f820 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 >> 172.21.15.7:0/4151833908 conn(0x7f59700fd600 msgr2=0x7f597010b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 shutdown_connections 2024-09-21T05:11:31.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:11:31.853+0000 7f59757c4640 1 -- 172.21.15.7:0/4151833908 wait complete. 2024-09-21T05:11:32.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:32 smithi053 ceph-mon[80176]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:32.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:32 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4151833908' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:11:32.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:32 smithi007 ceph-mon[108238]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:32.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:32 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4151833908' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:11:33.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:33 smithi053 ceph-mon[80176]: from='client.35816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:33.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:33 smithi053 ceph-mon[80176]: pgmap v1104: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:33 smithi007 ceph-mon[108238]: from='client.35816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:11:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:33 smithi007 ceph-mon[108238]: pgmap v1104: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:35.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:35 smithi007 ceph-mon[108238]: pgmap v1105: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:35.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:35 smithi053 ceph-mon[80176]: pgmap v1105: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:37.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:37 smithi053 ceph-mon[80176]: pgmap v1106: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:37 smithi007 ceph-mon[108238]: pgmap v1106: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:39.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:39 smithi053 ceph-mon[80176]: pgmap v1107: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:39 smithi007 ceph-mon[108238]: pgmap v1107: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:41.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:41 smithi053 ceph-mon[80176]: pgmap v1108: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:41 smithi007 ceph-mon[108238]: pgmap v1108: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:43 smithi053 ceph-mon[80176]: pgmap v1109: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:43 smithi007 ceph-mon[108238]: pgmap v1109: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:44.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:11:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:11:45.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:45 smithi053 ceph-mon[80176]: pgmap v1110: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:45.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:45 smithi007 ceph-mon[108238]: pgmap v1110: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:47.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:47 smithi053 ceph-mon[80176]: pgmap v1111: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:47 smithi007 ceph-mon[108238]: pgmap v1111: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:49 smithi007 ceph-mon[108238]: pgmap v1112: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:49 smithi053 ceph-mon[80176]: pgmap v1112: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:51.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:51 smithi053 ceph-mon[80176]: pgmap v1113: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:51 smithi007 ceph-mon[108238]: pgmap v1113: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:53.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:53 smithi053 ceph-mon[80176]: pgmap v1114: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:53 smithi007 ceph-mon[108238]: pgmap v1114: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:55.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:55 smithi007 ceph-mon[108238]: pgmap v1115: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:55.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:55 smithi053 ceph-mon[80176]: pgmap v1115: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:57.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:57 smithi053 ceph-mon[80176]: pgmap v1116: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:57.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:57 smithi007 ceph-mon[108238]: pgmap v1116: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:59.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:59 smithi053 ceph-mon[80176]: pgmap v1117: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:11:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:11:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:59 smithi007 ceph-mon[108238]: pgmap v1117: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:11:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:11:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:12:01.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:01 smithi053 ceph-mon[80176]: pgmap v1118: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:01 smithi007 ceph-mon[108238]: pgmap v1118: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:01.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.954+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3220956613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100c00 msgr2=0x7f559c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:01.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.954+0000 7f55a3dd4640 1 --2- 172.21.15.7:0/3220956613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100c00 0x7f559c104c50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f558c009880 tx=0x7f558c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:01.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.954+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3220956613 shutdown_connections 2024-09-21T05:12:01.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.954+0000 7f55a3dd4640 1 --2- 172.21.15.7:0/3220956613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100c00 0x7f559c104c50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:01.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.954+0000 7f55a3dd4640 1 --2- 172.21.15.7:0/3220956613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f559c100250 0x7f559c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:01.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.954+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3220956613 >> 172.21.15.7:0/3220956613 conn(0x7f559c0fbc90 msgr2=0x7f559c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:01.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.954+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3220956613 shutdown_connections 2024-09-21T05:12:01.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.954+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3220956613 wait complete. 2024-09-21T05:12:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a3dd4640 1 Processor -- start 2024-09-21T05:12:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a3dd4640 1 -- start start 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a3dd4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100250 0x7f559c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a3dd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f559c100c00 0x7f559c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a3dd4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f559c10dd30 con 0x7f559c100250 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a3dd4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f559c10dbb0 con 0x7f559c100c00 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a1b70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100250 0x7f559c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a1b70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100250 0x7f559c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51686/0 (socket says 172.21.15.7:51686) 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a1b70640 1 -- 172.21.15.7:0/3835480380 learned_addr learned my addr 172.21.15.7:0/3835480380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.955+0000 7f55a136f640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f559c100c00 0x7f559c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f55a1b70640 1 -- 172.21.15.7:0/3835480380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f559c100c00 msgr2=0x7f559c198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f55a1b70640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f559c100c00 0x7f559c198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f55a1b70640 1 -- 172.21.15.7:0/3835480380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c199380 con 0x7f559c100250 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f55a1b70640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100250 0x7f559c198840 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f559800d8d0 tx=0x7f559800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f5592ffd640 1 -- 172.21.15.7:0/3835480380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5598017070 con 0x7f559c100250 2024-09-21T05:12:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f5592ffd640 1 -- 172.21.15.7:0/3835480380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5598010460 con 0x7f559c100250 2024-09-21T05:12:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c199670 con 0x7f559c100250 2024-09-21T05:12:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f5592ffd640 1 -- 172.21.15.7:0/3835480380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f559800f5f0 con 0x7f559c100250 2024-09-21T05:12:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.956+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1a1c70 con 0x7f559c100250 2024-09-21T05:12:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.957+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c101730 con 0x7f559c100250 2024-09-21T05:12:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.958+0000 7f5592ffd640 1 -- 172.21.15.7:0/3835480380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5598009a30 con 0x7f559c100250 2024-09-21T05:12:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.959+0000 7f5592ffd640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f558807af80 0x7f558807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.959+0000 7f55a136f640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f558807af80 0x7f558807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.959+0000 7f5592ffd640 1 -- 172.21.15.7:0/3835480380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f559809ddc0 con 0x7f559c100250 2024-09-21T05:12:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.959+0000 7f55a136f640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f558807af80 0x7f558807d440 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f559c10ba40 tx=0x7f558c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:01.963+0000 7f5592ffd640 1 -- 172.21.15.7:0/3835480380 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f559c199670 con 0x7f559c100250 2024-09-21T05:12:02.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.169+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f559c062630 con 0x7f558807af80 2024-09-21T05:12:02.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.171+0000 7f5592ffd640 1 -- 172.21.15.7:0/3835480380 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f559c062630 con 0x7f558807af80 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f558807af80 msgr2=0x7f558807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f558807af80 0x7f558807d440 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f559c10ba40 tx=0x7f558c031400 comp rx=0 tx=0).stop 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100250 msgr2=0x7f559c198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100250 0x7f559c198840 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f559800d8d0 tx=0x7f559800dd90 comp rx=0 tx=0).stop 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 shutdown_connections 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f558807af80 0x7f558807d440 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f559c100c00 0x7f559c198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 --2- 172.21.15.7:0/3835480380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f559c100250 0x7f559c198840 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 >> 172.21.15.7:0/3835480380 conn(0x7f559c0fbc90 msgr2=0x7f559c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 shutdown_connections 2024-09-21T05:12:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.173+0000 7f55a3dd4640 1 -- 172.21.15.7:0/3835480380 wait complete. 2024-09-21T05:12:02.186 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.259+0000 7f7807328640 1 -- 172.21.15.7:0/687064198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074eb0 msgr2=0x7f78001123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.259+0000 7f7807328640 1 --2- 172.21.15.7:0/687064198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074eb0 0x7f78001123d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009880 tx=0x7f77f002a530 comp rx=0 tx=0).stop 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.259+0000 7f7807328640 1 -- 172.21.15.7:0/687064198 shutdown_connections 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.259+0000 7f7807328640 1 --2- 172.21.15.7:0/687064198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074eb0 0x7f78001123d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.259+0000 7f7807328640 1 --2- 172.21.15.7:0/687064198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7800074590 0x7f7800074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.259+0000 7f7807328640 1 -- 172.21.15.7:0/687064198 >> 172.21.15.7:0/687064198 conn(0x7f78000fd600 msgr2=0x7f78000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.260+0000 7f7807328640 1 -- 172.21.15.7:0/687064198 shutdown_connections 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.260+0000 7f7807328640 1 -- 172.21.15.7:0/687064198 wait complete. 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.260+0000 7f7807328640 1 Processor -- start 2024-09-21T05:12:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.260+0000 7f7807328640 1 -- start start 2024-09-21T05:12:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.260+0000 7f7807328640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074590 0x7f780019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.260+0000 7f7807328640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7800074eb0 0x7f780019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.260+0000 7f7807328640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78001153e0 con 0x7f7800074590 2024-09-21T05:12:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.260+0000 7f7807328640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7800115260 con 0x7f7800074eb0 2024-09-21T05:12:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78050c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074590 0x7f780019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78050c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074590 0x7f780019dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51716/0 (socket says 172.21.15.7:51716) 2024-09-21T05:12:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78050c4640 1 -- 172.21.15.7:0/1976620179 learned_addr learned my addr 172.21.15.7:0/1976620179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78048c3640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7800074eb0 0x7f780019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78050c4640 1 -- 172.21.15.7:0/1976620179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7800074eb0 msgr2=0x7f780019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78050c4640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7800074eb0 0x7f780019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78050c4640 1 -- 172.21.15.7:0/1976620179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78001a28b0 con 0x7f7800074590 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78048c3640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7800074eb0 0x7f780019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f78050c4640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074590 0x7f780019dfe0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c8f0 tx=0x7f77fc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f77f67fc640 1 -- 172.21.15.7:0/1976620179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f77fc0079b0 con 0x7f7800074590 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f77f67fc640 1 -- 172.21.15.7:0/1976620179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77fc0054e0 con 0x7f7800074590 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78001a2ba0 con 0x7f7800074590 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f77f67fc640 1 -- 172.21.15.7:0/1976620179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f77fc004dd0 con 0x7f7800074590 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.261+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78001a3130 con 0x7f7800074590 2024-09-21T05:12:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.262+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7800075cd0 con 0x7f7800074590 2024-09-21T05:12:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.263+0000 7f77f67fc640 1 -- 172.21.15.7:0/1976620179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f77fc005020 con 0x7f7800074590 2024-09-21T05:12:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.264+0000 7f77f67fc640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f77ec07aed0 0x7f77ec07d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.264+0000 7f77f67fc640 1 -- 172.21.15.7:0/1976620179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f77fc09e040 con 0x7f7800074590 2024-09-21T05:12:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.264+0000 7f78048c3640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f77ec07aed0 0x7f77ec07d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.264+0000 7f78048c3640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f77ec07aed0 0x7f77ec07d390 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f78001047f0 tx=0x7f77f0009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.268+0000 7f77f67fc640 1 -- 172.21.15.7:0/1976620179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77fc014030 con 0x7f7800074590 2024-09-21T05:12:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.474+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f78001a3440 con 0x7f77ec07aed0 2024-09-21T05:12:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.475+0000 7f77f67fc640 1 -- 172.21.15.7:0/1976620179 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f78001a3440 con 0x7f77ec07aed0 2024-09-21T05:12:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f77ec07aed0 msgr2=0x7f77ec07d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f77ec07aed0 0x7f77ec07d390 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f78001047f0 tx=0x7f77f0009560 comp rx=0 tx=0).stop 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074590 msgr2=0x7f780019dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074590 0x7f780019dfe0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c8f0 tx=0x7f77fc00cdb0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 shutdown_connections 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f77ec07aed0 0x7f77ec07d390 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7800074eb0 0x7f780019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 --2- 172.21.15.7:0/1976620179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7800074590 0x7f780019dfe0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 >> 172.21.15.7:0/1976620179 conn(0x7f78000fd600 msgr2=0x7f7800104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 shutdown_connections 2024-09-21T05:12:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.478+0000 7f7807328640 1 -- 172.21.15.7:0/1976620179 wait complete. 2024-09-21T05:12:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.562+0000 7f836e9f5640 1 -- 172.21.15.7:0/21981665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 msgr2=0x7f8368107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.562+0000 7f836e9f5640 1 --2- 172.21.15.7:0/21981665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 0x7f8368107ce0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8358009880 tx=0x7f835802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.562+0000 7f836e9f5640 1 -- 172.21.15.7:0/21981665 shutdown_connections 2024-09-21T05:12:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.562+0000 7f836e9f5640 1 --2- 172.21.15.7:0/21981665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8368101900 0x7f8368101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.562+0000 7f836e9f5640 1 --2- 172.21.15.7:0/21981665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 0x7f8368107ce0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.562+0000 7f836e9f5640 1 -- 172.21.15.7:0/21981665 >> 172.21.15.7:0/21981665 conn(0x7f83680fd600 msgr2=0x7f83680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.562+0000 7f836e9f5640 1 -- 172.21.15.7:0/21981665 shutdown_connections 2024-09-21T05:12:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.562+0000 7f836e9f5640 1 -- 172.21.15.7:0/21981665 wait complete. 2024-09-21T05:12:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.563+0000 7f836e9f5640 1 Processor -- start 2024-09-21T05:12:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.563+0000 7f836e9f5640 1 -- start start 2024-09-21T05:12:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.563+0000 7f836e9f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8368101900 0x7f836819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.563+0000 7f836e9f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 0x7f836819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.563+0000 7f836e9f5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f836810edd0 con 0x7f8368107900 2024-09-21T05:12:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.563+0000 7f836e9f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f836810ec50 con 0x7f8368101900 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.563+0000 7f83677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 0x7f836819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.563+0000 7f83677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 0x7f836819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51736/0 (socket says 172.21.15.7:51736) 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f83677fe640 1 -- 172.21.15.7:0/3191662737 learned_addr learned my addr 172.21.15.7:0/3191662737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f8367fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8368101900 0x7f836819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f83677fe640 1 -- 172.21.15.7:0/3191662737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8368101900 msgr2=0x7f836819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f83677fe640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8368101900 0x7f836819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f83677fe640 1 -- 172.21.15.7:0/3191662737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681977a0 con 0x7f8368107900 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f83677fe640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 0x7f836819db60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f835c00d6f0 tx=0x7f835c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f83657fa640 1 -- 172.21.15.7:0/3191662737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f835c017070 con 0x7f8368107900 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f83657fa640 1 -- 172.21.15.7:0/3191662737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f835c00de00 con 0x7f8368107900 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8368197a90 con 0x7f8368107900 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f83657fa640 1 -- 172.21.15.7:0/3191662737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f835c004bb0 con 0x7f8368107900 2024-09-21T05:12:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.564+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8368198020 con 0x7f8368107900 2024-09-21T05:12:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.565+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8368103040 con 0x7f8368107900 2024-09-21T05:12:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.566+0000 7f83657fa640 1 -- 172.21.15.7:0/3191662737 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f835c004e60 con 0x7f8368107900 2024-09-21T05:12:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.566+0000 7f83657fa640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f835007af30 0x7f835007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.566+0000 7f83657fa640 1 -- 172.21.15.7:0/3191662737 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f835c09dc00 con 0x7f8368107900 2024-09-21T05:12:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.566+0000 7f8367fff640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f835007af30 0x7f835007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.567+0000 7f8367fff640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f835007af30 0x7f835007d3f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f83580096c0 tx=0x7f8358031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.571+0000 7f83657fa640 1 -- 172.21.15.7:0/3191662737 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f835c0a3020 con 0x7f8368107900 2024-09-21T05:12:02.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.767+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f83681982d0 con 0x7f835007af30 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.777+0000 7f83657fa640 1 -- 172.21.15.7:0/3191662737 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f83681982d0 con 0x7f835007af30 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (34m) 12m ago 44m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (37m) 12m ago 45m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (36m) 12m ago 44m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (32m) 12m ago 45m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (32m) 12m ago 44m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (33m) 12m ago 44m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:12:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (28m) 12m ago 39m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (28m) 12m ago 39m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (12m) 12m ago 39m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (27m) 12m ago 39m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (36m) 12m ago 46m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (35m) 12m ago 44m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (32m) 12m ago 46m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (32m) 12m ago 44m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (34m) 12m ago 45m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (34m) 12m ago 44m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (31m) 12m ago 43m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (31m) 12m ago 42m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (30m) 12m ago 42m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (29m) 12m ago 41m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:12:02.782 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (29m) 12m ago 41m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:12:02.783 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (28m) 12m ago 40m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:12:02.783 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (34m) 12m ago 44m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:12:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f835007af30 msgr2=0x7f835007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f835007af30 0x7f835007d3f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f83580096c0 tx=0x7f8358031400 comp rx=0 tx=0).stop 2024-09-21T05:12:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 msgr2=0x7f836819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 0x7f836819db60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f835c00d6f0 tx=0x7f835c00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 shutdown_connections 2024-09-21T05:12:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f835007af30 0x7f835007d3f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8368107900 0x7f836819db60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 --2- 172.21.15.7:0/3191662737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8368101900 0x7f836819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 >> 172.21.15.7:0/3191662737 conn(0x7f83680fd600 msgr2=0x7f83680fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 shutdown_connections 2024-09-21T05:12:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.779+0000 7f836e9f5640 1 -- 172.21.15.7:0/3191662737 wait complete. 2024-09-21T05:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 -- 172.21.15.7:0/2382090773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 msgr2=0x7fc5f0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 --2- 172.21.15.7:0/2382090773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 0x7fc5f0100c50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009880 tx=0x7fc5e002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 -- 172.21.15.7:0/2382090773 shutdown_connections 2024-09-21T05:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 --2- 172.21.15.7:0/2382090773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 0x7fc5f0100c50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 --2- 172.21.15.7:0/2382090773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f01067f0 0x7fc5f0106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 -- 172.21.15.7:0/2382090773 >> 172.21.15.7:0/2382090773 conn(0x7fc5f00fc550 msgr2=0x7fc5f00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 -- 172.21.15.7:0/2382090773 shutdown_connections 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 -- 172.21.15.7:0/2382090773 wait complete. 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 Processor -- start 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.863+0000 7fc5f75c7640 1 -- start start 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f75c7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 0x7fc5f00fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f75c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f01067f0 0x7fc5f00fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f75c7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010dcc0 con 0x7fc5f01007f0 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f75c7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f010db40 con 0x7fc5f01067f0 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f5363640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 0x7fc5f00fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f5363640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 0x7fc5f00fccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51758/0 (socket says 172.21.15.7:51758) 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f5363640 1 -- 172.21.15.7:0/1066974727 learned_addr learned my addr 172.21.15.7:0/1066974727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f4b62640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f01067f0 0x7fc5f00fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f5363640 1 -- 172.21.15.7:0/1066974727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f01067f0 msgr2=0x7fc5f00fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f5363640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f01067f0 0x7fc5f00fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f5363640 1 -- 172.21.15.7:0/1066974727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f00fd750 con 0x7fc5f01007f0 2024-09-21T05:12:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f5363640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 0x7fc5f00fccd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00c8f0 tx=0x7fc5ec00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5e67fc640 1 -- 172.21.15.7:0/1066974727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5ec0079b0 con 0x7fc5f01007f0 2024-09-21T05:12:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5e67fc640 1 -- 172.21.15.7:0/1066974727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5ec0054e0 con 0x7fc5f01007f0 2024-09-21T05:12:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.864+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f00fda40 con 0x7fc5f01007f0 2024-09-21T05:12:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.865+0000 7fc5e67fc640 1 -- 172.21.15.7:0/1066974727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5ec004dd0 con 0x7fc5f01007f0 2024-09-21T05:12:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.865+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f01a3e60 con 0x7fc5f01007f0 2024-09-21T05:12:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.866+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0101f30 con 0x7fc5f01007f0 2024-09-21T05:12:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.866+0000 7fc5e67fc640 1 -- 172.21.15.7:0/1066974727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc5ec005020 con 0x7fc5f01007f0 2024-09-21T05:12:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.867+0000 7fc5e67fc640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc5dc07aed0 0x7fc5dc07d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.867+0000 7fc5e67fc640 1 -- 172.21.15.7:0/1066974727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc5ec09e040 con 0x7fc5f01007f0 2024-09-21T05:12:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.867+0000 7fc5f4b62640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc5dc07aed0 0x7fc5dc07d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.867+0000 7fc5f4b62640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc5dc07aed0 0x7fc5dc07d390 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0031460 tx=0x7fc5e0009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:02.871+0000 7fc5e67fc640 1 -- 172.21.15.7:0/1066974727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc5ec014030 con 0x7fc5f01007f0 2024-09-21T05:12:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.136+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc5f0109d70 con 0x7fc5f01007f0 2024-09-21T05:12:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.137+0000 7fc5e67fc640 1 -- 172.21.15.7:0/1066974727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fc5ec0695e0 con 0x7fc5f01007f0 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:12:03.141 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:12:03.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:12:03.142 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:12:03.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc5dc07aed0 msgr2=0x7fc5dc07d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc5dc07aed0 0x7fc5dc07d390 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0031460 tx=0x7fc5e0009560 comp rx=0 tx=0).stop 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 msgr2=0x7fc5f00fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 0x7fc5f00fccd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00c8f0 tx=0x7fc5ec00cdb0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 shutdown_connections 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc5dc07aed0 0x7fc5dc07d390 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f01067f0 0x7fc5f00fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 --2- 172.21.15.7:0/1066974727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f01007f0 0x7fc5f00fccd0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 >> 172.21.15.7:0/1066974727 conn(0x7fc5f00fc550 msgr2=0x7fc5f0069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 shutdown_connections 2024-09-21T05:12:03.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.140+0000 7fc5f75c7640 1 -- 172.21.15.7:0/1066974727 wait complete. 2024-09-21T05:12:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.226+0000 7f147a16f640 1 -- 172.21.15.7:0/101558586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 msgr2=0x7f1474107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.226+0000 7f147a16f640 1 --2- 172.21.15.7:0/101558586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 0x7f1474107ce0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1464009880 tx=0x7f146402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.226+0000 7f147a16f640 1 -- 172.21.15.7:0/101558586 shutdown_connections 2024-09-21T05:12:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.226+0000 7f147a16f640 1 --2- 172.21.15.7:0/101558586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1474101900 0x7f1474101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.226+0000 7f147a16f640 1 --2- 172.21.15.7:0/101558586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 0x7f1474107ce0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.226+0000 7f147a16f640 1 -- 172.21.15.7:0/101558586 >> 172.21.15.7:0/101558586 conn(0x7f14740fd600 msgr2=0x7f14740ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.226+0000 7f147a16f640 1 -- 172.21.15.7:0/101558586 shutdown_connections 2024-09-21T05:12:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.226+0000 7f147a16f640 1 -- 172.21.15.7:0/101558586 wait complete. 2024-09-21T05:12:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f147a16f640 1 Processor -- start 2024-09-21T05:12:03.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f147a16f640 1 -- start start 2024-09-21T05:12:03.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f147a16f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1474101900 0x7f147419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f147a16f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 0x7f147419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f147a16f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f147410edd0 con 0x7f1474107900 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f147a16f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f147410ec50 con 0x7f1474101900 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f1472ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 0x7f147419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f1472ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 0x7f147419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51790/0 (socket says 172.21.15.7:51790) 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f1472ffd640 1 -- 172.21.15.7:0/1111811306 learned_addr learned my addr 172.21.15.7:0/1111811306 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f14737fe640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1474101900 0x7f147419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f1472ffd640 1 -- 172.21.15.7:0/1111811306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1474101900 msgr2=0x7f147419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f1472ffd640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1474101900 0x7f147419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.227+0000 7f1472ffd640 1 -- 172.21.15.7:0/1111811306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14741977a0 con 0x7f1474107900 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.228+0000 7f1472ffd640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 0x7f147419db60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f145c00d6f0 tx=0x7f145c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.228+0000 7f1470ff9640 1 -- 172.21.15.7:0/1111811306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f145c017070 con 0x7f1474107900 2024-09-21T05:12:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.228+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1474197a90 con 0x7f1474107900 2024-09-21T05:12:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.228+0000 7f1470ff9640 1 -- 172.21.15.7:0/1111811306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f145c00de00 con 0x7f1474107900 2024-09-21T05:12:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.228+0000 7f1470ff9640 1 -- 172.21.15.7:0/1111811306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f145c004bb0 con 0x7f1474107900 2024-09-21T05:12:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.228+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1474198020 con 0x7f1474107900 2024-09-21T05:12:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.229+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1474103040 con 0x7f1474107900 2024-09-21T05:12:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.230+0000 7f1470ff9640 1 -- 172.21.15.7:0/1111811306 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f145c004e60 con 0x7f1474107900 2024-09-21T05:12:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.230+0000 7f1470ff9640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f146007af30 0x7f146007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.230+0000 7f1470ff9640 1 -- 172.21.15.7:0/1111811306 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f145c09dd50 con 0x7f1474107900 2024-09-21T05:12:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.230+0000 7f14737fe640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f146007af30 0x7f146007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.231+0000 7f14737fe640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f146007af30 0x7f146007d3f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1464031460 tx=0x7f1464009560 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.235+0000 7f1470ff9640 1 -- 172.21.15.7:0/1111811306 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f145c069250 con 0x7f1474107900 2024-09-21T05:12:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:03 smithi007 ceph-mon[108238]: pgmap v1119: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:03 smithi007 ceph-mon[108238]: from='client.35824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:03 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1066974727' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:12:03.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.461+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1474109e70 con 0x7f1474107900 2024-09-21T05:12:03.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.462+0000 7f1470ff9640 1 -- 172.21.15.7:0/1111811306 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f145c06e100 con 0x7f1474107900 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:12:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:12:03.467 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:12:03.468 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:12:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:12:03.469 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:12:03.469 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:12:03.469 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:12:03.469 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:12:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:12:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f146007af30 msgr2=0x7f146007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f146007af30 0x7f146007d3f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1464031460 tx=0x7f1464009560 comp rx=0 tx=0).stop 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 msgr2=0x7f147419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 0x7f147419db60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f145c00d6f0 tx=0x7f145c00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 shutdown_connections 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f146007af30 0x7f146007d3f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1474107900 0x7f147419db60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 --2- 172.21.15.7:0/1111811306 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1474101900 0x7f147419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 >> 172.21.15.7:0/1111811306 conn(0x7f14740fd600 msgr2=0x7f14740fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 shutdown_connections 2024-09-21T05:12:03.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.464+0000 7f147a16f640 1 -- 172.21.15.7:0/1111811306 wait complete. 2024-09-21T05:12:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.549+0000 7f66c243c640 1 -- 172.21.15.7:0/54435860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 msgr2=0x7f66bc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.549+0000 7f66c243c640 1 --2- 172.21.15.7:0/54435860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 0x7f66bc107ce0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009880 tx=0x7f66ac02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.550+0000 7f66c243c640 1 -- 172.21.15.7:0/54435860 shutdown_connections 2024-09-21T05:12:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.550+0000 7f66c243c640 1 --2- 172.21.15.7:0/54435860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66bc101900 0x7f66bc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.550+0000 7f66c243c640 1 --2- 172.21.15.7:0/54435860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 0x7f66bc107ce0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.550+0000 7f66c243c640 1 -- 172.21.15.7:0/54435860 >> 172.21.15.7:0/54435860 conn(0x7f66bc0fd600 msgr2=0x7f66bc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.550+0000 7f66c243c640 1 -- 172.21.15.7:0/54435860 shutdown_connections 2024-09-21T05:12:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.550+0000 7f66c243c640 1 -- 172.21.15.7:0/54435860 wait complete. 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.550+0000 7f66c243c640 1 Processor -- start 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.550+0000 7f66c243c640 1 -- start start 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66c243c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66bc101900 0x7f66bc19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66c243c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 0x7f66bc19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66c243c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc10edd0 con 0x7f66bc107900 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66c243c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc10ec50 con 0x7f66bc101900 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 0x7f66bc19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 0x7f66bc19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38260/0 (socket says 172.21.15.7:38260) 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66bb7fe640 1 -- 172.21.15.7:0/3819639842 learned_addr learned my addr 172.21.15.7:0/3819639842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66bb7fe640 1 -- 172.21.15.7:0/3819639842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66bc101900 msgr2=0x7f66bc19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66bb7fe640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66bc101900 0x7f66bc19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66bb7fe640 1 -- 172.21.15.7:0/3819639842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc197840 con 0x7f66bc107900 2024-09-21T05:12:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66bb7fe640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 0x7f66bc19dc00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c8f0 tx=0x7f66b000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66b97fa640 1 -- 172.21.15.7:0/3819639842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66b00079b0 con 0x7f66bc107900 2024-09-21T05:12:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66b97fa640 1 -- 172.21.15.7:0/3819639842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66b00054e0 con 0x7f66bc107900 2024-09-21T05:12:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc197b30 con 0x7f66bc107900 2024-09-21T05:12:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.551+0000 7f66b97fa640 1 -- 172.21.15.7:0/3819639842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66b0004dd0 con 0x7f66bc107900 2024-09-21T05:12:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.552+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc1980c0 con 0x7f66bc107900 2024-09-21T05:12:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.552+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc103040 con 0x7f66bc107900 2024-09-21T05:12:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.553+0000 7f66b97fa640 1 -- 172.21.15.7:0/3819639842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f66b0005020 con 0x7f66bc107900 2024-09-21T05:12:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.554+0000 7f66b97fa640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66a407aed0 0x7f66a407d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.554+0000 7f66b97fa640 1 -- 172.21.15.7:0/3819639842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f66b009e0e0 con 0x7f66bc107900 2024-09-21T05:12:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.554+0000 7f66bbfff640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66a407aed0 0x7f66a407d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.554+0000 7f66bbfff640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66a407aed0 0x7f66a407d390 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0096c0 tx=0x7f66ac000bc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.558+0000 7f66b97fa640 1 -- 172.21.15.7:0/3819639842 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66b0014030 con 0x7f66bc107900 2024-09-21T05:12:03.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:03 smithi053 ceph-mon[80176]: pgmap v1119: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:03.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:03 smithi053 ceph-mon[80176]: from='client.35824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:03.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:03 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1066974727' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.764+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f66bc1983d0 con 0x7f66a407aed0 2024-09-21T05:12:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.766+0000 7f66b97fa640 1 -- 172.21.15.7:0/3819639842 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f66bc1983d0 con 0x7f66a407aed0 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:12:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:12:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:12:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:12:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:12:03.771 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:12:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.768+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66a407aed0 msgr2=0x7f66a407d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.768+0000 7f66c243c640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66a407aed0 0x7f66a407d390 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0096c0 tx=0x7f66ac000bc0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.768+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 msgr2=0x7f66bc19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.768+0000 7f66c243c640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 0x7f66bc19dc00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c8f0 tx=0x7f66b000cdb0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.769+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 shutdown_connections 2024-09-21T05:12:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.769+0000 7f66c243c640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f66a407aed0 0x7f66a407d390 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.769+0000 7f66c243c640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66bc107900 0x7f66bc19dc00 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.769+0000 7f66c243c640 1 --2- 172.21.15.7:0/3819639842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66bc101900 0x7f66bc19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.769+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 >> 172.21.15.7:0/3819639842 conn(0x7f66bc0fd600 msgr2=0x7f66bc0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.769+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 shutdown_connections 2024-09-21T05:12:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.769+0000 7f66c243c640 1 -- 172.21.15.7:0/3819639842 wait complete. 2024-09-21T05:12:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.853+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/489005485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 msgr2=0x7fd2ac102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.853+0000 7fd2b3a6a640 1 --2- 172.21.15.7:0/489005485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 0x7fd2ac102ea0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009880 tx=0x7fd2a802a530 comp rx=0 tx=0).stop 2024-09-21T05:12:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.853+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/489005485 shutdown_connections 2024-09-21T05:12:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.853+0000 7fd2b3a6a640 1 --2- 172.21.15.7:0/489005485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac1033e0 0x7fd2ac10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.853+0000 7fd2b3a6a640 1 --2- 172.21.15.7:0/489005485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 0x7fd2ac102ea0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.853+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/489005485 >> 172.21.15.7:0/489005485 conn(0x7fd2ac0fc520 msgr2=0x7fd2ac0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.853+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/489005485 shutdown_connections 2024-09-21T05:12:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.853+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/489005485 wait complete. 2024-09-21T05:12:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b3a6a640 1 Processor -- start 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b3a6a640 1 -- start start 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b3a6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 0x7fd2ac19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b3a6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac1033e0 0x7fd2ac19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b3a6a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac10ee50 con 0x7fd2ac102ac0 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b3a6a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac10ecd0 con 0x7fd2ac1033e0 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b1806640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 0x7fd2ac19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b1806640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 0x7fd2ac19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38272/0 (socket says 172.21.15.7:38272) 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.854+0000 7fd2b1806640 1 -- 172.21.15.7:0/532824183 learned_addr learned my addr 172.21.15.7:0/532824183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2b1005640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac1033e0 0x7fd2ac19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2b1806640 1 -- 172.21.15.7:0/532824183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac1033e0 msgr2=0x7fd2ac19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2b1806640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac1033e0 0x7fd2ac19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2b1806640 1 -- 172.21.15.7:0/532824183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac197860 con 0x7fd2ac102ac0 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2b1005640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac1033e0 0x7fd2ac19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2b1806640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 0x7fd2ac19d6a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8031460 tx=0x7fd2a80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2a2ffd640 1 -- 172.21.15.7:0/532824183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd2a803f070 con 0x7fd2ac102ac0 2024-09-21T05:12:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2a2ffd640 1 -- 172.21.15.7:0/532824183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd2a8004cf0 con 0x7fd2ac102ac0 2024-09-21T05:12:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac197af0 con 0x7fd2ac102ac0 2024-09-21T05:12:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2a2ffd640 1 -- 172.21.15.7:0/532824183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd2a8035030 con 0x7fd2ac102ac0 2024-09-21T05:12:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.855+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac198020 con 0x7fd2ac102ac0 2024-09-21T05:12:03.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.856+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ac06aa90 con 0x7fd2ac102ac0 2024-09-21T05:12:03.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.857+0000 7fd2a2ffd640 1 -- 172.21.15.7:0/532824183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd2a8042070 con 0x7fd2ac102ac0 2024-09-21T05:12:03.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.858+0000 7fd2a2ffd640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd29807af30 0x7fd29807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:03.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.858+0000 7fd2a2ffd640 1 -- 172.21.15.7:0/532824183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fd2a80beb60 con 0x7fd2ac102ac0 2024-09-21T05:12:03.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.858+0000 7fd2b1005640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd29807af30 0x7fd29807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:03.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.858+0000 7fd2b1005640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd29807af30 0x7fd29807d3f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac108900 tx=0x7fd29c008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:03.862+0000 7fd2a2ffd640 1 -- 172.21.15.7:0/532824183 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2a808a100 con 0x7fd2ac102ac0 2024-09-21T05:12:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.123+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fd2ac198e60 con 0x7fd2ac102ac0 2024-09-21T05:12:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.123+0000 7fd2a2ffd640 1 -- 172.21.15.7:0/532824183 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fd2a808efb0 con 0x7fd2ac102ac0 2024-09-21T05:12:04.127 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:12:04.127 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:12:04.127 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:12:04.128 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:12:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:12:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:12:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:12:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:12:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:12:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:12:04.129 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:12:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.125+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd29807af30 msgr2=0x7fd29807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.125+0000 7fd2b3a6a640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd29807af30 0x7fd29807d3f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac108900 tx=0x7fd29c008040 comp rx=0 tx=0).stop 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.125+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 msgr2=0x7fd2ac19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.125+0000 7fd2b3a6a640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 0x7fd2ac19d6a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8031460 tx=0x7fd2a80048f0 comp rx=0 tx=0).stop 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.126+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 shutdown_connections 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.126+0000 7fd2b3a6a640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd29807af30 0x7fd29807d3f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.126+0000 7fd2b3a6a640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2ac1033e0 0x7fd2ac19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.126+0000 7fd2b3a6a640 1 --2- 172.21.15.7:0/532824183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2ac102ac0 0x7fd2ac19d6a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.126+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 >> 172.21.15.7:0/532824183 conn(0x7fd2ac0fc520 msgr2=0x7fd2ac10af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.126+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 shutdown_connections 2024-09-21T05:12:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:04.126+0000 7fd2b3a6a640 1 -- 172.21.15.7:0/532824183 wait complete. 2024-09-21T05:12:04.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:04 smithi053 ceph-mon[80176]: from='client.35828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:04.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:04 smithi053 ceph-mon[80176]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:04.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:04 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1111811306' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:12:04.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:04 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/532824183' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:12:04.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:04 smithi007 ceph-mon[108238]: from='client.35828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:04.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:04 smithi007 ceph-mon[108238]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:04.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:04 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1111811306' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:12:04.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:04 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/532824183' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:12:05.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:05 smithi053 ceph-mon[80176]: pgmap v1120: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:05.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:05 smithi053 ceph-mon[80176]: from='client.35844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:05.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:05 smithi007 ceph-mon[108238]: pgmap v1120: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:05.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:05 smithi007 ceph-mon[108238]: from='client.35844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:06.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:06 smithi053 ceph-mon[80176]: pgmap v1121: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:06.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:06 smithi007 ceph-mon[108238]: pgmap v1121: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:08.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:08 smithi053 ceph-mon[80176]: pgmap v1122: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:08 smithi007 ceph-mon[108238]: pgmap v1122: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:10.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:10 smithi007 ceph-mon[108238]: pgmap v1123: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:10.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:10 smithi053 ceph-mon[80176]: pgmap v1123: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:12.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:12 smithi053 ceph-mon[80176]: pgmap v1124: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:12.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:12 smithi007 ceph-mon[108238]: pgmap v1124: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:13.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:13 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:12:13.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:13 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:12:14.843 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:14 smithi053 ceph-mon[80176]: pgmap v1125: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:14 smithi007 ceph-mon[108238]: pgmap v1125: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:16.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:16 smithi007 ceph-mon[108238]: pgmap v1126: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:16.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:16 smithi053 ceph-mon[80176]: pgmap v1126: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:18.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:18 smithi053 ceph-mon[80176]: pgmap v1127: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:18.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:18 smithi007 ceph-mon[108238]: pgmap v1127: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:20.945 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:20 smithi053 ceph-mon[80176]: pgmap v1128: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:20.946 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:20 smithi007 ceph-mon[108238]: pgmap v1128: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:22.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:22 smithi007 ceph-mon[108238]: pgmap v1129: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:22.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:22 smithi053 ceph-mon[80176]: pgmap v1129: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:25 smithi053 ceph-mon[80176]: pgmap v1130: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:25 smithi007 ceph-mon[108238]: pgmap v1130: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:27.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:27 smithi053 ceph-mon[80176]: pgmap v1131: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:27.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:27 smithi007 ceph-mon[108238]: pgmap v1131: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:29 smithi007 ceph-mon[108238]: pgmap v1132: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:29.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:12:29.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:29 smithi053 ceph-mon[80176]: pgmap v1132: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:29.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:12:31.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:31 smithi053 ceph-mon[80176]: pgmap v1133: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:31.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:31 smithi007 ceph-mon[108238]: pgmap v1133: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:33 smithi053 ceph-mon[80176]: pgmap v1134: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:33 smithi007 ceph-mon[108238]: pgmap v1134: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:34.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.225+0000 7fe0137a0640 1 -- 172.21.15.7:0/2222822888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c101900 msgr2=0x7fe00c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.225+0000 7fe0137a0640 1 --2- 172.21.15.7:0/2222822888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c101900 0x7fe00c101d60 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009880 tx=0x7fdffc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.225+0000 7fe0137a0640 1 -- 172.21.15.7:0/2222822888 shutdown_connections 2024-09-21T05:12:34.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.225+0000 7fe0137a0640 1 --2- 172.21.15.7:0/2222822888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c101900 0x7fe00c101d60 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.225+0000 7fe0137a0640 1 --2- 172.21.15.7:0/2222822888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe00c107900 0x7fe00c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.225+0000 7fe0137a0640 1 -- 172.21.15.7:0/2222822888 >> 172.21.15.7:0/2222822888 conn(0x7fe00c0fd600 msgr2=0x7fe00c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.225+0000 7fe0137a0640 1 -- 172.21.15.7:0/2222822888 shutdown_connections 2024-09-21T05:12:34.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.225+0000 7fe0137a0640 1 -- 172.21.15.7:0/2222822888 wait complete. 2024-09-21T05:12:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe0137a0640 1 Processor -- start 2024-09-21T05:12:34.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe0137a0640 1 -- start start 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe0137a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe00c101900 0x7fe00c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe0137a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c107900 0x7fe00c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe0137a0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c10edd0 con 0x7fe00c107900 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe0137a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c10ec50 con 0x7fe00c101900 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe010d3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c107900 0x7fe00c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe010d3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c107900 0x7fe00c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59096/0 (socket says 172.21.15.7:59096) 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.226+0000 7fe010d3b640 1 -- 172.21.15.7:0/447298334 learned_addr learned my addr 172.21.15.7:0/447298334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe01153c640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe00c101900 0x7fe00c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe010d3b640 1 -- 172.21.15.7:0/447298334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe00c101900 msgr2=0x7fe00c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe010d3b640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe00c101900 0x7fe00c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe010d3b640 1 -- 172.21.15.7:0/447298334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c074f70 con 0x7fe00c107900 2024-09-21T05:12:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe010d3b640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c107900 0x7fe00c074a30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fdffc031460 tx=0x7fdffc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe0027fc640 1 -- 172.21.15.7:0/447298334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdffc03f070 con 0x7fe00c107900 2024-09-21T05:12:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe0027fc640 1 -- 172.21.15.7:0/447298334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdffc0035f0 con 0x7fe00c107900 2024-09-21T05:12:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c075200 con 0x7fe00c107900 2024-09-21T05:12:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe0027fc640 1 -- 172.21.15.7:0/447298334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdffc035030 con 0x7fe00c107900 2024-09-21T05:12:34.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.227+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c1a3d60 con 0x7fe00c107900 2024-09-21T05:12:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.228+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00c103040 con 0x7fe00c107900 2024-09-21T05:12:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.229+0000 7fe0027fc640 1 -- 172.21.15.7:0/447298334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fdffc042070 con 0x7fe00c107900 2024-09-21T05:12:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.229+0000 7fe0027fc640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.230+0000 7fe0027fc640 1 -- 172.21.15.7:0/447298334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fdffc0be940 con 0x7fe00c107900 2024-09-21T05:12:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.230+0000 7fe01153c640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.230+0000 7fe01153c640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe008009b60 tx=0x7fe008005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:34.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.234+0000 7fe0027fc640 1 -- 172.21.15.7:0/447298334 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdffc089f60 con 0x7fe00c107900 2024-09-21T05:12:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.440+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe00c062630 con 0x7fdff807af80 2024-09-21T05:12:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.441+0000 7fe0027fc640 1 -- 172.21.15.7:0/447298334 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fe00c062630 con 0x7fdff807af80 2024-09-21T05:12:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 msgr2=0x7fdff807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe008009b60 tx=0x7fe008005f90 comp rx=0 tx=0).stop 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c107900 msgr2=0x7fe00c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c107900 0x7fe00c074a30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fdffc031460 tx=0x7fdffc004530 comp rx=0 tx=0).stop 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 shutdown_connections 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c107900 0x7fe00c074a30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 --2- 172.21.15.7:0/447298334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe00c101900 0x7fe00c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.443+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 >> 172.21.15.7:0/447298334 conn(0x7fe00c0fd600 msgr2=0x7fe00c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.444+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 shutdown_connections 2024-09-21T05:12:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.444+0000 7fe0137a0640 1 -- 172.21.15.7:0/447298334 wait complete. 2024-09-21T05:12:34.455 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:12:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.529+0000 7f0551694640 1 -- 172.21.15.7:0/3888788278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 msgr2=0x7f054c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.529+0000 7f0551694640 1 --2- 172.21.15.7:0/3888788278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 0x7f054c107ce0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f053c009880 tx=0x7f053c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.529+0000 7f0551694640 1 -- 172.21.15.7:0/3888788278 shutdown_connections 2024-09-21T05:12:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.529+0000 7f0551694640 1 --2- 172.21.15.7:0/3888788278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f054c101900 0x7f054c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.529+0000 7f0551694640 1 --2- 172.21.15.7:0/3888788278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 0x7f054c107ce0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.529+0000 7f0551694640 1 -- 172.21.15.7:0/3888788278 >> 172.21.15.7:0/3888788278 conn(0x7f054c0fd600 msgr2=0x7f054c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.529+0000 7f0551694640 1 -- 172.21.15.7:0/3888788278 shutdown_connections 2024-09-21T05:12:34.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.529+0000 7f0551694640 1 -- 172.21.15.7:0/3888788278 wait complete. 2024-09-21T05:12:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f0551694640 1 Processor -- start 2024-09-21T05:12:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f0551694640 1 -- start start 2024-09-21T05:12:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f0551694640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f054c101900 0x7f054c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f0551694640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 0x7f054c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f0551694640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f054c10edd0 con 0x7f054c107900 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f0551694640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f054c10ec50 con 0x7f054c101900 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f05427fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 0x7f054c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f05427fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 0x7f054c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59122/0 (socket says 172.21.15.7:59122) 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.530+0000 7f05427fc640 1 -- 172.21.15.7:0/3702341608 learned_addr learned my addr 172.21.15.7:0/3702341608 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f054affd640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f054c101900 0x7f054c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f054affd640 1 -- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 msgr2=0x7f054c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f054affd640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 0x7f054c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f054affd640 1 -- 172.21.15.7:0/3702341608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c1977c0 con 0x7f054c101900 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f05427fc640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 0x7f054c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f054affd640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f054c101900 0x7f054c19d640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f053c006010 tx=0x7f053c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f0548ff9640 1 -- 172.21.15.7:0/3702341608 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f053c03f070 con 0x7f054c101900 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c197a50 con 0x7f054c101900 2024-09-21T05:12:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.531+0000 7f0548ff9640 1 -- 172.21.15.7:0/3702341608 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f053c0035f0 con 0x7f054c101900 2024-09-21T05:12:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.532+0000 7f0548ff9640 1 -- 172.21.15.7:0/3702341608 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f053c035030 con 0x7f054c101900 2024-09-21T05:12:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.532+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c197f80 con 0x7f054c101900 2024-09-21T05:12:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.533+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c103040 con 0x7f054c101900 2024-09-21T05:12:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.533+0000 7f0548ff9640 1 -- 172.21.15.7:0/3702341608 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f053c042070 con 0x7f054c101900 2024-09-21T05:12:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.534+0000 7f0548ff9640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f053807af30 0x7f053807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.534+0000 7f0548ff9640 1 -- 172.21.15.7:0/3702341608 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f053c0bea30 con 0x7f054c101900 2024-09-21T05:12:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.534+0000 7f05427fc640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f053807af30 0x7f053807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.534+0000 7f05427fc640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f053807af30 0x7f053807d3f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f054c0ffbf0 tx=0x7f053400b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.539+0000 7f0548ff9640 1 -- 172.21.15.7:0/3702341608 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f053c0c0e20 con 0x7f054c101900 2024-09-21T05:12:34.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.746+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f054c198230 con 0x7f053807af30 2024-09-21T05:12:34.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.748+0000 7f0548ff9640 1 -- 172.21.15.7:0/3702341608 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f054c198230 con 0x7f053807af30 2024-09-21T05:12:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f053807af30 msgr2=0x7f053807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f053807af30 0x7f053807d3f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f054c0ffbf0 tx=0x7f053400b410 comp rx=0 tx=0).stop 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f054c101900 msgr2=0x7f054c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f054c101900 0x7f054c19d640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f053c006010 tx=0x7f053c0045e0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 shutdown_connections 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f053807af30 0x7f053807d3f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f054c107900 0x7f054c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 --2- 172.21.15.7:0/3702341608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f054c101900 0x7f054c19d640 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 >> 172.21.15.7:0/3702341608 conn(0x7f054c0fd600 msgr2=0x7f054c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 shutdown_connections 2024-09-21T05:12:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.750+0000 7f0551694640 1 -- 172.21.15.7:0/3702341608 wait complete. 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.835+0000 7fbeaaec6640 1 -- 172.21.15.7:0/1632411452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4100800 msgr2=0x7fbea4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.835+0000 7fbeaaec6640 1 --2- 172.21.15.7:0/1632411452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4100800 0x7fbea4100c60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009c20 tx=0x7fbe9402a970 comp rx=0 tx=0).stop 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.835+0000 7fbeaaec6640 1 -- 172.21.15.7:0/1632411452 shutdown_connections 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.835+0000 7fbeaaec6640 1 --2- 172.21.15.7:0/1632411452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4100800 0x7fbea4100c60 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.835+0000 7fbeaaec6640 1 --2- 172.21.15.7:0/1632411452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbea4106800 0x7fbea4106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.835+0000 7fbeaaec6640 1 -- 172.21.15.7:0/1632411452 >> 172.21.15.7:0/1632411452 conn(0x7fbea40fc520 msgr2=0x7fbea40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.835+0000 7fbeaaec6640 1 -- 172.21.15.7:0/1632411452 shutdown_connections 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.835+0000 7fbeaaec6640 1 -- 172.21.15.7:0/1632411452 wait complete. 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.836+0000 7fbeaaec6640 1 Processor -- start 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.836+0000 7fbeaaec6640 1 -- start start 2024-09-21T05:12:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.836+0000 7fbeaaec6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbea4100800 0x7fbea40fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.836+0000 7fbeaaec6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4106800 0x7fbea40fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.836+0000 7fbeaaec6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbea410dcb0 con 0x7fbea4106800 2024-09-21T05:12:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.836+0000 7fbeaaec6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbea410db30 con 0x7fbea4100800 2024-09-21T05:12:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.836+0000 7fbea3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4106800 0x7fbea40fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.836+0000 7fbea3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4106800 0x7fbea40fd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59128/0 (socket says 172.21.15.7:59128) 2024-09-21T05:12:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.837+0000 7fbea3fff640 1 -- 172.21.15.7:0/3740032333 learned_addr learned my addr 172.21.15.7:0/3740032333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.837+0000 7fbea8c62640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbea4100800 0x7fbea40fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.837+0000 7fbea3fff640 1 -- 172.21.15.7:0/3740032333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbea4100800 msgr2=0x7fbea40fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.837+0000 7fbea3fff640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbea4100800 0x7fbea40fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.837+0000 7fbea3fff640 1 -- 172.21.15.7:0/3740032333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea40fd690 con 0x7fbea4106800 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.837+0000 7fbea3fff640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4106800 0x7fbea40fd150 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbe9402ae90 tx=0x7fbe9402f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.838+0000 7fbea1ffb640 1 -- 172.21.15.7:0/3740032333 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbe94047070 con 0x7fbea4106800 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.838+0000 7fbea1ffb640 1 -- 172.21.15.7:0/3740032333 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbe9402f7e0 con 0x7fbea4106800 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.838+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea40fd920 con 0x7fbea4106800 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.838+0000 7fbea1ffb640 1 -- 172.21.15.7:0/3740032333 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbe94034070 con 0x7fbea4106800 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.838+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea41a3db0 con 0x7fbea4106800 2024-09-21T05:12:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.839+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea4101f40 con 0x7fbea4106800 2024-09-21T05:12:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.840+0000 7fbea1ffb640 1 -- 172.21.15.7:0/3740032333 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbe9404a070 con 0x7fbea4106800 2024-09-21T05:12:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.840+0000 7fbea1ffb640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbe9007af80 0x7fbe9007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.841+0000 7fbea8c62640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbe9007af80 0x7fbe9007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.841+0000 7fbea1ffb640 1 -- 172.21.15.7:0/3740032333 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbe940bec60 con 0x7fbea4106800 2024-09-21T05:12:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.841+0000 7fbea8c62640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbe9007af80 0x7fbe9007d440 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00bbe0 tx=0x7fbe8c00b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:34.845+0000 7fbea1ffb640 1 -- 172.21.15.7:0/3740032333 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe9403f070 con 0x7fbea4106800 2024-09-21T05:12:35.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.057+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbea4062630 con 0x7fbe9007af80 2024-09-21T05:12:35.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.067+0000 7fbea1ffb640 1 -- 172.21.15.7:0/3740032333 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fbea4062630 con 0x7fbe9007af80 2024-09-21T05:12:35.070 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:12:35.070 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (34m) 13m ago 45m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:12:35.070 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (37m) 13m ago 45m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:12:35.070 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (37m) 13m ago 44m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:12:35.070 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (32m) 13m ago 45m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:12:35.070 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (32m) 13m ago 44m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (34m) 13m ago 45m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (28m) 13m ago 40m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (29m) 13m ago 40m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (13m) 13m ago 40m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (28m) 13m ago 40m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (36m) 13m ago 46m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (35m) 13m ago 44m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (33m) 13m ago 47m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (33m) 13m ago 44m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (35m) 13m ago 45m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (35m) 13m ago 44m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (32m) 13m ago 43m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (31m) 13m ago 43m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:12:35.071 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (31m) 13m ago 42m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:12:35.072 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (30m) 13m ago 42m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:12:35.072 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (29m) 13m ago 41m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:12:35.072 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (29m) 13m ago 41m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:12:35.072 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (34m) 13m ago 45m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:12:35.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.069+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbe9007af80 msgr2=0x7fbe9007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbe9007af80 0x7fbe9007d440 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00bbe0 tx=0x7fbe8c00b390 comp rx=0 tx=0).stop 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4106800 msgr2=0x7fbea40fd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4106800 0x7fbea40fd150 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbe9402ae90 tx=0x7fbe9402f5f0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 shutdown_connections 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbe9007af80 0x7fbe9007d440 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea4106800 0x7fbea40fd150 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 --2- 172.21.15.7:0/3740032333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbea4100800 0x7fbea40fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 >> 172.21.15.7:0/3740032333 conn(0x7fbea40fc520 msgr2=0x7fbea406a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 shutdown_connections 2024-09-21T05:12:35.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.070+0000 7fbeaaec6640 1 -- 172.21.15.7:0/3740032333 wait complete. 2024-09-21T05:12:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.153+0000 7fd6604dd640 1 -- 172.21.15.7:0/2728943677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658107900 msgr2=0x7fd658107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.153+0000 7fd6604dd640 1 --2- 172.21.15.7:0/2728943677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658107900 0x7fd658107ce0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fd654009880 tx=0x7fd65402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.153+0000 7fd6604dd640 1 -- 172.21.15.7:0/2728943677 shutdown_connections 2024-09-21T05:12:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.153+0000 7fd6604dd640 1 --2- 172.21.15.7:0/2728943677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd658101900 0x7fd658101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.153+0000 7fd6604dd640 1 --2- 172.21.15.7:0/2728943677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658107900 0x7fd658107ce0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.153+0000 7fd6604dd640 1 -- 172.21.15.7:0/2728943677 >> 172.21.15.7:0/2728943677 conn(0x7fd6580fd600 msgr2=0x7fd6580ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.153+0000 7fd6604dd640 1 -- 172.21.15.7:0/2728943677 shutdown_connections 2024-09-21T05:12:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.153+0000 7fd6604dd640 1 -- 172.21.15.7:0/2728943677 wait complete. 2024-09-21T05:12:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd6604dd640 1 Processor -- start 2024-09-21T05:12:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd6604dd640 1 -- start start 2024-09-21T05:12:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd6604dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658101900 0x7fd65819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd6604dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd658107900 0x7fd65819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd6604dd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd65810edb0 con 0x7fd658101900 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd6604dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd65810ec30 con 0x7fd658107900 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd65e279640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658101900 0x7fd65819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd65e279640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658101900 0x7fd65819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59146/0 (socket says 172.21.15.7:59146) 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd65e279640 1 -- 172.21.15.7:0/1670853419 learned_addr learned my addr 172.21.15.7:0/1670853419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.154+0000 7fd65da78640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd658107900 0x7fd65819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd65e279640 1 -- 172.21.15.7:0/1670853419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd658107900 msgr2=0x7fd65819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd65e279640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd658107900 0x7fd65819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd65e279640 1 -- 172.21.15.7:0/1670853419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6581977c0 con 0x7fd658101900 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd65e279640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658101900 0x7fd65819d640 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd654031460 tx=0x7fd6540048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd64f7fe640 1 -- 172.21.15.7:0/1670853419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd65403f070 con 0x7fd658101900 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd64f7fe640 1 -- 172.21.15.7:0/1670853419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd654004cf0 con 0x7fd658101900 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd658197a50 con 0x7fd658101900 2024-09-21T05:12:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd64f7fe640 1 -- 172.21.15.7:0/1670853419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd654035030 con 0x7fd658101900 2024-09-21T05:12:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.155+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd658197f80 con 0x7fd658101900 2024-09-21T05:12:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.156+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd658103040 con 0x7fd658101900 2024-09-21T05:12:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.157+0000 7fd64f7fe640 1 -- 172.21.15.7:0/1670853419 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fd654042070 con 0x7fd658101900 2024-09-21T05:12:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.157+0000 7fd64f7fe640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd64407af80 0x7fd64407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.157+0000 7fd64f7fe640 1 -- 172.21.15.7:0/1670853419 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fd6540be970 con 0x7fd658101900 2024-09-21T05:12:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.157+0000 7fd65da78640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd64407af80 0x7fd64407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.158+0000 7fd65da78640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd64407af80 0x7fd64407d440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd658100e00 tx=0x7fd64800a2d0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:35.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.162+0000 7fd64f7fe640 1 -- 172.21.15.7:0/1670853419 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6540c0e20 con 0x7fd658101900 2024-09-21T05:12:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.428+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fd658198cf0 con 0x7fd658101900 2024-09-21T05:12:35.431 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:35 smithi007 ceph-mon[108238]: pgmap v1135: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.429+0000 7fd64f7fe640 1 -- 172.21.15.7:0/1670853419 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fd654089f90 con 0x7fd658101900 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:12:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd64407af80 msgr2=0x7fd64407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd64407af80 0x7fd64407d440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd658100e00 tx=0x7fd64800a2d0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658101900 msgr2=0x7fd65819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658101900 0x7fd65819d640 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd654031460 tx=0x7fd6540048f0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 shutdown_connections 2024-09-21T05:12:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fd64407af80 0x7fd64407d440 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd658107900 0x7fd65819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 --2- 172.21.15.7:0/1670853419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd658101900 0x7fd65819d640 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 >> 172.21.15.7:0/1670853419 conn(0x7fd6580fd600 msgr2=0x7fd6580ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 shutdown_connections 2024-09-21T05:12:35.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.431+0000 7fd6604dd640 1 -- 172.21.15.7:0/1670853419 wait complete. 2024-09-21T05:12:35.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:35 smithi053 ceph-mon[80176]: pgmap v1135: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:35.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.515+0000 7fe6a7083640 1 -- 172.21.15.7:0/409767085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 msgr2=0x7fe6a0106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.515+0000 7fe6a7083640 1 --2- 172.21.15.7:0/409767085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 0x7fe6a0106be0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe694009880 tx=0x7fe69402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.515+0000 7fe6a7083640 1 -- 172.21.15.7:0/409767085 shutdown_connections 2024-09-21T05:12:35.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.515+0000 7fe6a7083640 1 --2- 172.21.15.7:0/409767085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0100800 0x7fe6a0100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.515+0000 7fe6a7083640 1 --2- 172.21.15.7:0/409767085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 0x7fe6a0106be0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.515+0000 7fe6a7083640 1 -- 172.21.15.7:0/409767085 >> 172.21.15.7:0/409767085 conn(0x7fe6a00fc520 msgr2=0x7fe6a00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.515+0000 7fe6a7083640 1 -- 172.21.15.7:0/409767085 shutdown_connections 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.515+0000 7fe6a7083640 1 -- 172.21.15.7:0/409767085 wait complete. 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.516+0000 7fe6a7083640 1 Processor -- start 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.516+0000 7fe6a7083640 1 -- start start 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.516+0000 7fe6a7083640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0100800 0x7fe6a00fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.516+0000 7fe6a7083640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 0x7fe6a00fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.516+0000 7fe6a7083640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a010dcd0 con 0x7fe6a0106800 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.516+0000 7fe6a7083640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a010db50 con 0x7fe6a0100800 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.516+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 0x7fe6a00fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.516+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 0x7fe6a00fd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59164/0 (socket says 172.21.15.7:59164) 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe69ffff640 1 -- 172.21.15.7:0/3142789454 learned_addr learned my addr 172.21.15.7:0/3142789454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe6a4e1f640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0100800 0x7fe6a00fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe69ffff640 1 -- 172.21.15.7:0/3142789454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0100800 msgr2=0x7fe6a00fcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe69ffff640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0100800 0x7fe6a00fcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe69ffff640 1 -- 172.21.15.7:0/3142789454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a00fd6f0 con 0x7fe6a0106800 2024-09-21T05:12:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe69ffff640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 0x7fe6a00fd1b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00e970 tx=0x7fe68c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe69dffb640 1 -- 172.21.15.7:0/3142789454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe68c00cac0 con 0x7fe6a0106800 2024-09-21T05:12:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe69dffb640 1 -- 172.21.15.7:0/3142789454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe68c0054e0 con 0x7fe6a0106800 2024-09-21T05:12:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a00fd9e0 con 0x7fe6a0106800 2024-09-21T05:12:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe69dffb640 1 -- 172.21.15.7:0/3142789454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe68c010430 con 0x7fe6a0106800 2024-09-21T05:12:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.517+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01a3e20 con 0x7fe6a0106800 2024-09-21T05:12:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.518+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a0101f40 con 0x7fe6a0106800 2024-09-21T05:12:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.519+0000 7fe69dffb640 1 -- 172.21.15.7:0/3142789454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe68c004e80 con 0x7fe6a0106800 2024-09-21T05:12:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.519+0000 7fe69dffb640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe69007af30 0x7fe69007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.520+0000 7fe69dffb640 1 -- 172.21.15.7:0/3142789454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fe68c09e6b0 con 0x7fe6a0106800 2024-09-21T05:12:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.520+0000 7fe6a4e1f640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe69007af30 0x7fe69007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.520+0000 7fe6a4e1f640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe69007af30 0x7fe69007d3f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe6940096c0 tx=0x7fe694031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:35.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.524+0000 7fe69dffb640 1 -- 172.21.15.7:0/3142789454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe68c069c50 con 0x7fe6a0106800 2024-09-21T05:12:35.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.753+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe6a0109d80 con 0x7fe6a0106800 2024-09-21T05:12:35.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.753+0000 7fe69dffb640 1 -- 172.21.15.7:0/3142789454 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7fe68c06eb00 con 0x7fe6a0106800 2024-09-21T05:12:35.756 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:12:35.756 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:12:35.757 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:12:35.758 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:12:35.759 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:12:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:12:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe69007af30 msgr2=0x7fe69007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe69007af30 0x7fe69007d3f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe6940096c0 tx=0x7fe694031400 comp rx=0 tx=0).stop 2024-09-21T05:12:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 msgr2=0x7fe6a00fd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 0x7fe6a00fd1b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00e970 tx=0x7fe68c00ed70 comp rx=0 tx=0).stop 2024-09-21T05:12:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 shutdown_connections 2024-09-21T05:12:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe69007af30 0x7fe69007d3f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a0106800 0x7fe6a00fd1b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 --2- 172.21.15.7:0/3142789454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0100800 0x7fe6a00fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 >> 172.21.15.7:0/3142789454 conn(0x7fe6a00fc520 msgr2=0x7fe6a006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 shutdown_connections 2024-09-21T05:12:35.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.756+0000 7fe6a7083640 1 -- 172.21.15.7:0/3142789454 wait complete. 2024-09-21T05:12:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.840+0000 7fac9290e640 1 -- 172.21.15.7:0/920054826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c1033e0 msgr2=0x7fac8c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.840+0000 7fac9290e640 1 --2- 172.21.15.7:0/920054826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c1033e0 0x7fac8c10b8d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fac80009880 tx=0x7fac8002a530 comp rx=0 tx=0).stop 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.840+0000 7fac9290e640 1 -- 172.21.15.7:0/920054826 shutdown_connections 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.840+0000 7fac9290e640 1 --2- 172.21.15.7:0/920054826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c1033e0 0x7fac8c10b8d0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.840+0000 7fac9290e640 1 --2- 172.21.15.7:0/920054826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac8c102ac0 0x7fac8c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.840+0000 7fac9290e640 1 -- 172.21.15.7:0/920054826 >> 172.21.15.7:0/920054826 conn(0x7fac8c0fc520 msgr2=0x7fac8c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.840+0000 7fac9290e640 1 -- 172.21.15.7:0/920054826 shutdown_connections 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.840+0000 7fac9290e640 1 -- 172.21.15.7:0/920054826 wait complete. 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac9290e640 1 Processor -- start 2024-09-21T05:12:35.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac9290e640 1 -- start start 2024-09-21T05:12:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac9290e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c102ac0 0x7fac8c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac9290e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac8c1033e0 0x7fac8c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac9290e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c10edd0 con 0x7fac8c102ac0 2024-09-21T05:12:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac9290e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c10ec50 con 0x7fac8c1033e0 2024-09-21T05:12:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac8bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c102ac0 0x7fac8c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac8bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c102ac0 0x7fac8c19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59176/0 (socket says 172.21.15.7:59176) 2024-09-21T05:12:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.841+0000 7fac8bfff640 1 -- 172.21.15.7:0/937113204 learned_addr learned my addr 172.21.15.7:0/937113204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac8b7fe640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac8c1033e0 0x7fac8c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac8bfff640 1 -- 172.21.15.7:0/937113204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac8c1033e0 msgr2=0x7fac8c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac8bfff640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac8c1033e0 0x7fac8c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac8bfff640 1 -- 172.21.15.7:0/937113204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c1977e0 con 0x7fac8c102ac0 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac8bfff640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c102ac0 0x7fac8c19d660 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00d8d0 tx=0x7fac7c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac897fa640 1 -- 172.21.15.7:0/937113204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac7c017070 con 0x7fac8c102ac0 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac897fa640 1 -- 172.21.15.7:0/937113204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac7c010460 con 0x7fac8c102ac0 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c197ad0 con 0x7fac8c102ac0 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac897fa640 1 -- 172.21.15.7:0/937113204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac7c00f5f0 con 0x7fac8c102ac0 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.842+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c198030 con 0x7fac8c102ac0 2024-09-21T05:12:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.843+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8c06aa90 con 0x7fac8c102ac0 2024-09-21T05:12:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.844+0000 7fac897fa640 1 -- 172.21.15.7:0/937113204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fac7c009a30 con 0x7fac8c102ac0 2024-09-21T05:12:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.844+0000 7fac897fa640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac7407af40 0x7fac7407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.845+0000 7fac897fa640 1 -- 172.21.15.7:0/937113204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fac7c09de50 con 0x7fac8c102ac0 2024-09-21T05:12:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.845+0000 7fac8b7fe640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac7407af40 0x7fac7407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:35.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.845+0000 7fac8b7fe640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac7407af40 0x7fac7407d400 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fac8c198b30 tx=0x7fac80031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:35.849+0000 7fac897fa640 1 -- 172.21.15.7:0/937113204 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fac7c0693f0 con 0x7fac8c102ac0 2024-09-21T05:12:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.052+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fac8c1982e0 con 0x7fac7407af40 2024-09-21T05:12:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.054+0000 7fac897fa640 1 -- 172.21.15.7:0/937113204 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fac8c1982e0 con 0x7fac7407af40 2024-09-21T05:12:36.056 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:12:36.056 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:12:36.057 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:12:36.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac7407af40 msgr2=0x7fac7407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:36.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac7407af40 0x7fac7407d400 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fac8c198b30 tx=0x7fac80031400 comp rx=0 tx=0).stop 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c102ac0 msgr2=0x7fac8c19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c102ac0 0x7fac8c19d660 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00d8d0 tx=0x7fac7c00dd90 comp rx=0 tx=0).stop 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 shutdown_connections 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fac7407af40 0x7fac7407d400 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac8c1033e0 0x7fac8c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 --2- 172.21.15.7:0/937113204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac8c102ac0 0x7fac8c19d660 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 >> 172.21.15.7:0/937113204 conn(0x7fac8c0fc520 msgr2=0x7fac8c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 shutdown_connections 2024-09-21T05:12:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.056+0000 7fac9290e640 1 -- 172.21.15.7:0/937113204 wait complete. 2024-09-21T05:12:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.143+0000 7f7ed75a8640 1 -- 172.21.15.7:0/3973126161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 msgr2=0x7f7ed0074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.143+0000 7f7ed75a8640 1 --2- 172.21.15.7:0/3973126161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 0x7f7ed0074970 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009880 tx=0x7f7ecc02a530 comp rx=0 tx=0).stop 2024-09-21T05:12:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.143+0000 7f7ed75a8640 1 -- 172.21.15.7:0/3973126161 shutdown_connections 2024-09-21T05:12:36.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.143+0000 7f7ed75a8640 1 --2- 172.21.15.7:0/3973126161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0074eb0 0x7f7ed01123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.143+0000 7f7ed75a8640 1 --2- 172.21.15.7:0/3973126161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 0x7f7ed0074970 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.143+0000 7f7ed75a8640 1 -- 172.21.15.7:0/3973126161 >> 172.21.15.7:0/3973126161 conn(0x7f7ed00fd600 msgr2=0x7f7ed00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.143+0000 7f7ed75a8640 1 -- 172.21.15.7:0/3973126161 shutdown_connections 2024-09-21T05:12:36.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.144+0000 7f7ed75a8640 1 -- 172.21.15.7:0/3973126161 wait complete. 2024-09-21T05:12:36.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.144+0000 7f7ed75a8640 1 Processor -- start 2024-09-21T05:12:36.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.144+0000 7f7ed75a8640 1 -- start start 2024-09-21T05:12:36.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.144+0000 7f7ed75a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 0x7f7ed019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:36.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.144+0000 7f7ed75a8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0074eb0 0x7f7ed019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.144+0000 7f7ed75a8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01153f0 con 0x7f7ed0074590 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.144+0000 7f7ed75a8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0115270 con 0x7f7ed0074eb0 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.144+0000 7f7ed5344640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 0x7f7ed019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed5344640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 0x7f7ed019e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59198/0 (socket says 172.21.15.7:59198) 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed5344640 1 -- 172.21.15.7:0/2107753421 learned_addr learned my addr 172.21.15.7:0/2107753421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed4b43640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0074eb0 0x7f7ed019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed5344640 1 -- 172.21.15.7:0/2107753421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0074eb0 msgr2=0x7f7ed019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed5344640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0074eb0 0x7f7ed019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed5344640 1 -- 172.21.15.7:0/2107753421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed01a2900 con 0x7f7ed0074590 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed4b43640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0074eb0 0x7f7ed019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed5344640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 0x7f7ed019e030 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc031460 tx=0x7f7ecc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ec67fc640 1 -- 172.21.15.7:0/2107753421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ecc03f070 con 0x7f7ed0074590 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ec67fc640 1 -- 172.21.15.7:0/2107753421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ecc004cf0 con 0x7f7ed0074590 2024-09-21T05:12:36.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed01a2b90 con 0x7f7ed0074590 2024-09-21T05:12:36.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ec67fc640 1 -- 172.21.15.7:0/2107753421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ecc035030 con 0x7f7ed0074590 2024-09-21T05:12:36.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.145+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01a30c0 con 0x7f7ed0074590 2024-09-21T05:12:36.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.146+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed0075cd0 con 0x7f7ed0074590 2024-09-21T05:12:36.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.147+0000 7f7ec67fc640 1 -- 172.21.15.7:0/2107753421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7ecc042070 con 0x7f7ed0074590 2024-09-21T05:12:36.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.148+0000 7f7ec67fc640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ebc07af80 0x7f7ebc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:12:36.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.148+0000 7f7ec67fc640 1 -- 172.21.15.7:0/2107753421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f7ecc0beae0 con 0x7f7ed0074590 2024-09-21T05:12:36.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.148+0000 7f7ed4b43640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ebc07af80 0x7f7ebc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:12:36.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.148+0000 7f7ed4b43640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ebc07af80 0x7f7ebc07d440 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec0005ea0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:12:36.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.152+0000 7f7ec67fc640 1 -- 172.21.15.7:0/2107753421 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ecc08a060 con 0x7f7ed0074590 2024-09-21T05:12:36.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.413+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f7ed010fac0 con 0x7f7ed0074590 2024-09-21T05:12:36.416 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:36 smithi007 ceph-mon[108238]: from='client.35852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:36.416 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:36 smithi007 ceph-mon[108238]: from='client.45383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:36.416 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:36 smithi007 ceph-mon[108238]: from='client.35860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:36.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:36 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1670853419' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:12:36.417 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:36 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3142789454' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:12:36.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.413+0000 7f7ec67fc640 1 -- 172.21.15.7:0/2107753421 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f7ecc08ef10 con 0x7f7ed0074590 2024-09-21T05:12:36.417 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:12:36.417 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:12:36.417 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:12:36.417 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:12:36.417 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:12:36.417 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:12:36.418 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:12:36.419 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:12:36.419 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:12:36.419 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:12:36.419 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:12:36.419 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:12:36.419 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ebc07af80 msgr2=0x7f7ebc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ebc07af80 0x7f7ebc07d440 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec0005ea0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 msgr2=0x7f7ed019e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 0x7f7ed019e030 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc031460 tx=0x7f7ecc0048f0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 shutdown_connections 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7ebc07af80 0x7f7ebc07d440 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ed0074eb0 0x7f7ed019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 --2- 172.21.15.7:0/2107753421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ed0074590 0x7f7ed019e030 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 >> 172.21.15.7:0/2107753421 conn(0x7f7ed00fd600 msgr2=0x7f7ed0104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 shutdown_connections 2024-09-21T05:12:36.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:12:36.416+0000 7f7ed75a8640 1 -- 172.21.15.7:0/2107753421 wait complete. 2024-09-21T05:12:36.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:36 smithi053 ceph-mon[80176]: from='client.35852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:36.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:36 smithi053 ceph-mon[80176]: from='client.45383 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:36 smithi053 ceph-mon[80176]: from='client.35860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:36 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1670853419' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:12:36.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:36 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3142789454' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:12:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:37 smithi053 ceph-mon[80176]: pgmap v1136: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:37.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:37 smithi053 ceph-mon[80176]: from='client.35872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:37.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:37 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2107753421' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:12:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:37 smithi007 ceph-mon[108238]: pgmap v1136: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:37 smithi007 ceph-mon[108238]: from='client.35872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:12:37.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:37 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2107753421' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:12:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:39 smithi053 ceph-mon[80176]: pgmap v1137: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:39.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:39 smithi007 ceph-mon[108238]: pgmap v1137: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:41.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:41 smithi007 ceph-mon[108238]: pgmap v1138: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:41.437 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:41 smithi053 ceph-mon[80176]: pgmap v1138: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:43.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:43 smithi053 ceph-mon[80176]: pgmap v1139: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:43.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:43 smithi007 ceph-mon[108238]: pgmap v1139: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:44.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:12:44.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:12:45.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:45 smithi053 ceph-mon[80176]: pgmap v1140: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:45.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:45 smithi007 ceph-mon[108238]: pgmap v1140: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:47.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:47 smithi007 ceph-mon[108238]: pgmap v1141: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:47.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:47 smithi053 ceph-mon[80176]: pgmap v1141: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:49.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:49 smithi053 ceph-mon[80176]: pgmap v1142: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:49.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:49 smithi007 ceph-mon[108238]: pgmap v1142: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:51.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:51 smithi053 ceph-mon[80176]: pgmap v1143: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:51.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:51 smithi007 ceph-mon[108238]: pgmap v1143: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:53.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:53 smithi007 ceph-mon[108238]: pgmap v1144: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:53.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:53 smithi053 ceph-mon[80176]: pgmap v1144: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:55.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:55 smithi053 ceph-mon[80176]: pgmap v1145: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:55.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:55 smithi007 ceph-mon[108238]: pgmap v1145: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:57.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:57 smithi053 ceph-mon[80176]: pgmap v1146: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:57.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:57 smithi007 ceph-mon[108238]: pgmap v1146: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:59 smithi007 ceph-mon[108238]: pgmap v1147: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:59.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:12:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:12:59.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:59 smithi053 ceph-mon[80176]: pgmap v1147: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:12:59.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:12:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:13:01.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:01 smithi053 ceph-mon[80176]: pgmap v1148: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:01 smithi007 ceph-mon[108238]: pgmap v1148: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:03.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:03 smithi053 ceph-mon[80176]: pgmap v1149: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:03.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:03 smithi007 ceph-mon[108238]: pgmap v1149: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:05.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:05 smithi007 ceph-mon[108238]: pgmap v1150: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:05.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:05 smithi053 ceph-mon[80176]: pgmap v1150: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:06.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.513+0000 7f1645bed640 1 -- 172.21.15.7:0/2980109370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401069a0 msgr2=0x7f1640106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:06.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.513+0000 7f1645bed640 1 --2- 172.21.15.7:0/2980109370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401069a0 0x7f1640106d80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1630009880 tx=0x7f163002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.513+0000 7f1645bed640 1 -- 172.21.15.7:0/2980109370 shutdown_connections 2024-09-21T05:13:06.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.513+0000 7f1645bed640 1 --2- 172.21.15.7:0/2980109370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16400ffdc0 0x7f1640100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.513+0000 7f1645bed640 1 --2- 172.21.15.7:0/2980109370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16401069a0 0x7f1640106d80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.513+0000 7f1645bed640 1 -- 172.21.15.7:0/2980109370 >> 172.21.15.7:0/2980109370 conn(0x7f16400fbc90 msgr2=0x7f16400fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.513+0000 7f1645bed640 1 -- 172.21.15.7:0/2980109370 shutdown_connections 2024-09-21T05:13:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.513+0000 7f1645bed640 1 -- 172.21.15.7:0/2980109370 wait complete. 2024-09-21T05:13:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.514+0000 7f1645bed640 1 Processor -- start 2024-09-21T05:13:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.514+0000 7f1645bed640 1 -- start start 2024-09-21T05:13:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.514+0000 7f1645bed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16400ffdc0 0x7f164019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.514+0000 7f1645bed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16401069a0 0x7f164019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.514+0000 7f1645bed640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f164010ef40 con 0x7f16400ffdc0 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.514+0000 7f1645bed640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f164010edc0 con 0x7f16401069a0 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16400ffdc0 0x7f164019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16400ffdc0 0x7f164019f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43558/0 (socket says 172.21.15.7:43558) 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163f7fe640 1 -- 172.21.15.7:0/3108933963 learned_addr learned my addr 172.21.15.7:0/3108933963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163effd640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16401069a0 0x7f164019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163f7fe640 1 -- 172.21.15.7:0/3108933963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16401069a0 msgr2=0x7f164019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163f7fe640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16401069a0 0x7f164019fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163f7fe640 1 -- 172.21.15.7:0/3108933963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1640199b00 con 0x7f16400ffdc0 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163f7fe640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16400ffdc0 0x7f164019f940 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1630031460 tx=0x7f1630004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163cff9640 1 -- 172.21.15.7:0/3108933963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f163003f070 con 0x7f16400ffdc0 2024-09-21T05:13:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f163cff9640 1 -- 172.21.15.7:0/3108933963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f16300035f0 con 0x7f16400ffdc0 2024-09-21T05:13:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.515+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1640199d90 con 0x7f16400ffdc0 2024-09-21T05:13:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.516+0000 7f163cff9640 1 -- 172.21.15.7:0/3108933963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1630035030 con 0x7f16400ffdc0 2024-09-21T05:13:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.516+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164019a2c0 con 0x7f16400ffdc0 2024-09-21T05:13:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.517+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16401030e0 con 0x7f16400ffdc0 2024-09-21T05:13:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.517+0000 7f163cff9640 1 -- 172.21.15.7:0/3108933963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1630042070 con 0x7f16400ffdc0 2024-09-21T05:13:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.518+0000 7f163cff9640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f162807af30 0x7f162807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.518+0000 7f163cff9640 1 -- 172.21.15.7:0/3108933963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f16300be9d0 con 0x7f16400ffdc0 2024-09-21T05:13:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.518+0000 7f163effd640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f162807af30 0x7f162807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.518+0000 7f163effd640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f162807af30 0x7f162807d3f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f16401022b0 tx=0x7f1634005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.522+0000 7f163cff9640 1 -- 172.21.15.7:0/3108933963 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1630089f70 con 0x7f16400ffdc0 2024-09-21T05:13:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.727+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f164019aff0 con 0x7f162807af30 2024-09-21T05:13:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.728+0000 7f163cff9640 1 -- 172.21.15.7:0/3108933963 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f164019aff0 con 0x7f162807af30 2024-09-21T05:13:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.730+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f162807af30 msgr2=0x7f162807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.730+0000 7f1645bed640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f162807af30 0x7f162807d3f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f16401022b0 tx=0x7f1634005f90 comp rx=0 tx=0).stop 2024-09-21T05:13:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.730+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16400ffdc0 msgr2=0x7f164019f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.730+0000 7f1645bed640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16400ffdc0 0x7f164019f940 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1630031460 tx=0x7f1630004530 comp rx=0 tx=0).stop 2024-09-21T05:13:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.730+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 shutdown_connections 2024-09-21T05:13:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.730+0000 7f1645bed640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f162807af30 0x7f162807d3f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.731+0000 7f1645bed640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16401069a0 0x7f164019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.731+0000 7f1645bed640 1 --2- 172.21.15.7:0/3108933963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16400ffdc0 0x7f164019f940 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.731+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 >> 172.21.15.7:0/3108933963 conn(0x7f16400fbc90 msgr2=0x7f1640100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.731+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 shutdown_connections 2024-09-21T05:13:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.731+0000 7f1645bed640 1 -- 172.21.15.7:0/3108933963 wait complete. 2024-09-21T05:13:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:13:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.816+0000 7ff97d357640 1 -- 172.21.15.7:0/1248610837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 msgr2=0x7ff978102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.816+0000 7ff97d357640 1 --2- 172.21.15.7:0/1248610837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 0x7ff978102ea0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff968009880 tx=0x7ff96802a530 comp rx=0 tx=0).stop 2024-09-21T05:13:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.816+0000 7ff97d357640 1 -- 172.21.15.7:0/1248610837 shutdown_connections 2024-09-21T05:13:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.816+0000 7ff97d357640 1 --2- 172.21.15.7:0/1248610837 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9781033e0 0x7ff97810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.816+0000 7ff97d357640 1 --2- 172.21.15.7:0/1248610837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 0x7ff978102ea0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.816+0000 7ff97d357640 1 -- 172.21.15.7:0/1248610837 >> 172.21.15.7:0/1248610837 conn(0x7ff9780fc520 msgr2=0x7ff9780fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.816+0000 7ff97d357640 1 -- 172.21.15.7:0/1248610837 shutdown_connections 2024-09-21T05:13:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.817+0000 7ff97d357640 1 -- 172.21.15.7:0/1248610837 wait complete. 2024-09-21T05:13:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.817+0000 7ff97d357640 1 Processor -- start 2024-09-21T05:13:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff97d357640 1 -- start start 2024-09-21T05:13:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff97d357640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 0x7ff97819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff97d357640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9781033e0 0x7ff97819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff97d357640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff97810edd0 con 0x7ff978102ac0 2024-09-21T05:13:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff97d357640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff97810ec50 con 0x7ff9781033e0 2024-09-21T05:13:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff976ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 0x7ff97819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff976ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 0x7ff97819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43574/0 (socket says 172.21.15.7:43574) 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff976ffd640 1 -- 172.21.15.7:0/449083079 learned_addr learned my addr 172.21.15.7:0/449083079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff9767fc640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9781033e0 0x7ff97819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff976ffd640 1 -- 172.21.15.7:0/449083079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9781033e0 msgr2=0x7ff97819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff976ffd640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9781033e0 0x7ff97819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.818+0000 7ff976ffd640 1 -- 172.21.15.7:0/449083079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9781978e0 con 0x7ff978102ac0 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.819+0000 7ff976ffd640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 0x7ff97819d720 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff968031460 tx=0x7ff968004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.819+0000 7ff95ffff640 1 -- 172.21.15.7:0/449083079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff96803f070 con 0x7ff978102ac0 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.819+0000 7ff95ffff640 1 -- 172.21.15.7:0/449083079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9680035f0 con 0x7ff978102ac0 2024-09-21T05:13:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.819+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff978197b70 con 0x7ff978102ac0 2024-09-21T05:13:06.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.819+0000 7ff95ffff640 1 -- 172.21.15.7:0/449083079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff968035030 con 0x7ff978102ac0 2024-09-21T05:13:06.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.819+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9781980a0 con 0x7ff978102ac0 2024-09-21T05:13:06.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.820+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97806aa90 con 0x7ff978102ac0 2024-09-21T05:13:06.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.822+0000 7ff95ffff640 1 -- 172.21.15.7:0/449083079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff968042070 con 0x7ff978102ac0 2024-09-21T05:13:06.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.822+0000 7ff95ffff640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:06.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.822+0000 7ff95ffff640 1 -- 172.21.15.7:0/449083079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff9680be960 con 0x7ff978102ac0 2024-09-21T05:13:06.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.822+0000 7ff9767fc640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:06.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.823+0000 7ff9767fc640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff9780feec0 tx=0x7ff96c005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:06.827+0000 7ff95ffff640 1 -- 172.21.15.7:0/449083079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff968089f00 con 0x7ff978102ac0 2024-09-21T05:13:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.035+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff978198e10 con 0x7ff96007af30 2024-09-21T05:13:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.037+0000 7ff95ffff640 1 -- 172.21.15.7:0/449083079 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ff978198e10 con 0x7ff96007af30 2024-09-21T05:13:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 msgr2=0x7ff96007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff9780feec0 tx=0x7ff96c005f90 comp rx=0 tx=0).stop 2024-09-21T05:13:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 msgr2=0x7ff97819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 0x7ff97819d720 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff968031460 tx=0x7ff968004530 comp rx=0 tx=0).stop 2024-09-21T05:13:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 shutdown_connections 2024-09-21T05:13:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff96007af30 0x7ff96007d3f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9781033e0 0x7ff97819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 --2- 172.21.15.7:0/449083079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff978102ac0 0x7ff97819d720 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 >> 172.21.15.7:0/449083079 conn(0x7ff9780fc520 msgr2=0x7ff978108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:07.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 shutdown_connections 2024-09-21T05:13:07.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.039+0000 7ff97d357640 1 -- 172.21.15.7:0/449083079 wait complete. 2024-09-21T05:13:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.123+0000 7ff2443c9640 1 -- 172.21.15.7:0/2771702656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 msgr2=0x7ff23c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.123+0000 7ff2443c9640 1 --2- 172.21.15.7:0/2771702656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 0x7ff23c101d60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff230009880 tx=0x7ff23002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.123+0000 7ff2443c9640 1 -- 172.21.15.7:0/2771702656 shutdown_connections 2024-09-21T05:13:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.123+0000 7ff2443c9640 1 --2- 172.21.15.7:0/2771702656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 0x7ff23c101d60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.123+0000 7ff2443c9640 1 --2- 172.21.15.7:0/2771702656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff23c107900 0x7ff23c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.123+0000 7ff2443c9640 1 -- 172.21.15.7:0/2771702656 >> 172.21.15.7:0/2771702656 conn(0x7ff23c0fd600 msgr2=0x7ff23c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.123+0000 7ff2443c9640 1 -- 172.21.15.7:0/2771702656 shutdown_connections 2024-09-21T05:13:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.123+0000 7ff2443c9640 1 -- 172.21.15.7:0/2771702656 wait complete. 2024-09-21T05:13:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff2443c9640 1 Processor -- start 2024-09-21T05:13:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff2443c9640 1 -- start start 2024-09-21T05:13:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff2443c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 0x7ff23c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff2443c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff23c107900 0x7ff23c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff2443c9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c10ee30 con 0x7ff23c101900 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff2443c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c10ecb0 con 0x7ff23c107900 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff242165640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 0x7ff23c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff242165640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 0x7ff23c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43594/0 (socket says 172.21.15.7:43594) 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.124+0000 7ff242165640 1 -- 172.21.15.7:0/506207719 learned_addr learned my addr 172.21.15.7:0/506207719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff241964640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff23c107900 0x7ff23c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff242165640 1 -- 172.21.15.7:0/506207719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff23c107900 msgr2=0x7ff23c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff242165640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff23c107900 0x7ff23c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff242165640 1 -- 172.21.15.7:0/506207719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c074f70 con 0x7ff23c101900 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff242165640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 0x7ff23c0744f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00ba50 tx=0x7ff22c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff23b7fe640 1 -- 172.21.15.7:0/506207719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff22c019070 con 0x7ff23c101900 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff23b7fe640 1 -- 172.21.15.7:0/506207719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff22c00ce20 con 0x7ff23c101900 2024-09-21T05:13:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c075260 con 0x7ff23c101900 2024-09-21T05:13:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff23b7fe640 1 -- 172.21.15.7:0/506207719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff22c0075a0 con 0x7ff23c101900 2024-09-21T05:13:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.125+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c1a3ec0 con 0x7ff23c101900 2024-09-21T05:13:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.126+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c103040 con 0x7ff23c101900 2024-09-21T05:13:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.127+0000 7ff23b7fe640 1 -- 172.21.15.7:0/506207719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff22c01c070 con 0x7ff23c101900 2024-09-21T05:13:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.128+0000 7ff23b7fe640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff22407af80 0x7ff22407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.128+0000 7ff23b7fe640 1 -- 172.21.15.7:0/506207719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff22c09d900 con 0x7ff23c101900 2024-09-21T05:13:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.128+0000 7ff241964640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff22407af80 0x7ff22407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.128+0000 7ff241964640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff22407af80 0x7ff22407d440 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff23c075f80 tx=0x7ff230031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:07.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.132+0000 7ff23b7fe640 1 -- 172.21.15.7:0/506207719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff22c068ec0 con 0x7ff23c101900 2024-09-21T05:13:07.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.329+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff23c062630 con 0x7ff22407af80 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.339+0000 7ff23b7fe640 1 -- 172.21.15.7:0/506207719 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7ff23c062630 con 0x7ff22407af80 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (35m) 13m ago 46m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (38m) 13m ago 46m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (37m) 13m ago 45m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (33m) 13m ago 46m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (33m) 13m ago 45m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (34m) 13m ago 45m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (29m) 13m ago 40m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (29m) 13m ago 40m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (13m) 13m ago 40m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:13:07.343 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (29m) 13m ago 40m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (37m) 13m ago 47m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (36m) 13m ago 45m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (33m) 13m ago 47m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (33m) 13m ago 45m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (36m) 13m ago 46m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (35m) 13m ago 45m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (32m) 13m ago 44m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (32m) 13m ago 43m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (31m) 13m ago 43m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (31m) 13m ago 42m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (30m) 13m ago 42m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (29m) 13m ago 41m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:13:07.344 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (35m) 13m ago 45m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:13:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff22407af80 msgr2=0x7ff22407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff22407af80 0x7ff22407d440 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff23c075f80 tx=0x7ff230031400 comp rx=0 tx=0).stop 2024-09-21T05:13:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 msgr2=0x7ff23c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 0x7ff23c0744f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00ba50 tx=0x7ff22c00bf10 comp rx=0 tx=0).stop 2024-09-21T05:13:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 shutdown_connections 2024-09-21T05:13:07.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff22407af80 0x7ff22407d440 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff23c107900 0x7ff23c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 --2- 172.21.15.7:0/506207719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff23c101900 0x7ff23c0744f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 >> 172.21.15.7:0/506207719 conn(0x7ff23c0fd600 msgr2=0x7ff23c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:07.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.341+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 shutdown_connections 2024-09-21T05:13:07.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.342+0000 7ff2443c9640 1 -- 172.21.15.7:0/506207719 wait complete. 2024-09-21T05:13:07.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.424+0000 7f984511d640 1 -- 172.21.15.7:0/239276782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840101900 msgr2=0x7f9840101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.424+0000 7f984511d640 1 --2- 172.21.15.7:0/239276782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840101900 0x7f9840101d60 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9834009880 tx=0x7f983402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.424+0000 7f984511d640 1 -- 172.21.15.7:0/239276782 shutdown_connections 2024-09-21T05:13:07.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.424+0000 7f984511d640 1 --2- 172.21.15.7:0/239276782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840101900 0x7f9840101d60 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.424+0000 7f984511d640 1 --2- 172.21.15.7:0/239276782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9840107900 0x7f9840107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.424+0000 7f984511d640 1 -- 172.21.15.7:0/239276782 >> 172.21.15.7:0/239276782 conn(0x7f98400fd600 msgr2=0x7f98400ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:07.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:07 smithi007 ceph-mon[108238]: pgmap v1151: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:07.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.424+0000 7f984511d640 1 -- 172.21.15.7:0/239276782 shutdown_connections 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.424+0000 7f984511d640 1 -- 172.21.15.7:0/239276782 wait complete. 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.425+0000 7f984511d640 1 Processor -- start 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.425+0000 7f984511d640 1 -- start start 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f984511d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9840101900 0x7f98400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f984511d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840107900 0x7f9840074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f984511d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f984010ee30 con 0x7f9840107900 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f984511d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f984010ecb0 con 0x7f9840101900 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840107900 0x7f9840074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840107900 0x7f9840074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43608/0 (socket says 172.21.15.7:43608) 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983e59c640 1 -- 172.21.15.7:0/2960147765 learned_addr learned my addr 172.21.15.7:0/2960147765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983ed9d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9840101900 0x7f98400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983e59c640 1 -- 172.21.15.7:0/2960147765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9840101900 msgr2=0x7f98400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983e59c640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9840101900 0x7f98400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983e59c640 1 -- 172.21.15.7:0/2960147765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840074f70 con 0x7f9840107900 2024-09-21T05:13:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983ed9d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9840101900 0x7f98400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:13:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.426+0000 7f983e59c640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840107900 0x7f9840074a30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9834031460 tx=0x7f98340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.427+0000 7f9827fff640 1 -- 172.21.15.7:0/2960147765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f983403f070 con 0x7f9840107900 2024-09-21T05:13:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.427+0000 7f9827fff640 1 -- 172.21.15.7:0/2960147765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98340044d0 con 0x7f9840107900 2024-09-21T05:13:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.427+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9840075200 con 0x7f9840107900 2024-09-21T05:13:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.427+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401a3e00 con 0x7f9840107900 2024-09-21T05:13:07.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.427+0000 7f9827fff640 1 -- 172.21.15.7:0/2960147765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9834035030 con 0x7f9840107900 2024-09-21T05:13:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.428+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c005290 con 0x7f9840107900 2024-09-21T05:13:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.429+0000 7f9827fff640 1 -- 172.21.15.7:0/2960147765 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9834042070 con 0x7f9840107900 2024-09-21T05:13:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.429+0000 7f9827fff640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f982807af40 0x7f982807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.429+0000 7f9827fff640 1 -- 172.21.15.7:0/2960147765 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f98340beb00 con 0x7f9840107900 2024-09-21T05:13:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.429+0000 7f983ed9d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f982807af40 0x7f982807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.430+0000 7f983ed9d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f982807af40 0x7f982807d400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9830009930 tx=0x7f9830008040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:07.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.434+0000 7f9827fff640 1 -- 172.21.15.7:0/2960147765 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9834004cf0 con 0x7f9840107900 2024-09-21T05:13:07.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:07 smithi053 ceph-mon[80176]: pgmap v1151: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:07.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.701+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f982c005d90 con 0x7f9840107900 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.702+0000 7f9827fff640 1 -- 172.21.15.7:0/2960147765 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f983408a0a0 con 0x7f9840107900 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:13:07.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:13:07.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:13:07.707 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:13:07.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:13:07.707 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:13:07.707 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:13:07.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:13:07.707 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:13:07.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f982807af40 msgr2=0x7f982807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f982807af40 0x7f982807d400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9830009930 tx=0x7f9830008040 comp rx=0 tx=0).stop 2024-09-21T05:13:07.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840107900 msgr2=0x7f9840074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840107900 0x7f9840074a30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9834031460 tx=0x7f98340048f0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 shutdown_connections 2024-09-21T05:13:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f982807af40 0x7f982807d400 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9840107900 0x7f9840074a30 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 --2- 172.21.15.7:0/2960147765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9840101900 0x7f98400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.704+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 >> 172.21.15.7:0/2960147765 conn(0x7f98400fd600 msgr2=0x7f98400fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.705+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 shutdown_connections 2024-09-21T05:13:07.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.705+0000 7f984511d640 1 -- 172.21.15.7:0/2960147765 wait complete. 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.788+0000 7f138f34f640 1 -- 172.21.15.7:0/1680731363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13881033e0 msgr2=0x7f138810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.788+0000 7f138f34f640 1 --2- 172.21.15.7:0/1680731363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13881033e0 0x7f138810b8d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1378009880 tx=0x7f137802a530 comp rx=0 tx=0).stop 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.788+0000 7f138f34f640 1 -- 172.21.15.7:0/1680731363 shutdown_connections 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.788+0000 7f138f34f640 1 --2- 172.21.15.7:0/1680731363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13881033e0 0x7f138810b8d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.788+0000 7f138f34f640 1 --2- 172.21.15.7:0/1680731363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1388102ac0 0x7f1388102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.788+0000 7f138f34f640 1 -- 172.21.15.7:0/1680731363 >> 172.21.15.7:0/1680731363 conn(0x7f13880fc520 msgr2=0x7f13880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.789+0000 7f138f34f640 1 -- 172.21.15.7:0/1680731363 shutdown_connections 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.789+0000 7f138f34f640 1 -- 172.21.15.7:0/1680731363 wait complete. 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.789+0000 7f138f34f640 1 Processor -- start 2024-09-21T05:13:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.789+0000 7f138f34f640 1 -- start start 2024-09-21T05:13:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.789+0000 7f138f34f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388102ac0 0x7f138819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.789+0000 7f138f34f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13881033e0 0x7f138819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.789+0000 7f138f34f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f138810edf0 con 0x7f1388102ac0 2024-09-21T05:13:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.789+0000 7f138f34f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f138810ec70 con 0x7f13881033e0 2024-09-21T05:13:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138d0eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388102ac0 0x7f138819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138d0eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388102ac0 0x7f138819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43616/0 (socket says 172.21.15.7:43616) 2024-09-21T05:13:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138d0eb640 1 -- 172.21.15.7:0/1565155909 learned_addr learned my addr 172.21.15.7:0/1565155909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:07.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138c8ea640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13881033e0 0x7f138819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138d0eb640 1 -- 172.21.15.7:0/1565155909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13881033e0 msgr2=0x7f138819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138d0eb640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13881033e0 0x7f138819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138d0eb640 1 -- 172.21.15.7:0/1565155909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1388197800 con 0x7f1388102ac0 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138c8ea640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13881033e0 0x7f138819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138d0eb640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388102ac0 0x7f138819d680 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f138400c8f0 tx=0x7f138400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f137e7fc640 1 -- 172.21.15.7:0/1565155909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13840079b0 con 0x7f1388102ac0 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f137e7fc640 1 -- 172.21.15.7:0/1565155909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13840054e0 con 0x7f1388102ac0 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1388197af0 con 0x7f1388102ac0 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f137e7fc640 1 -- 172.21.15.7:0/1565155909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1384004dd0 con 0x7f1388102ac0 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.790+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1388198080 con 0x7f1388102ac0 2024-09-21T05:13:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.791+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138806aa90 con 0x7f1388102ac0 2024-09-21T05:13:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.792+0000 7f137e7fc640 1 -- 172.21.15.7:0/1565155909 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1384005020 con 0x7f1388102ac0 2024-09-21T05:13:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.793+0000 7f137e7fc640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137407aed0 0x7f137407d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.793+0000 7f137e7fc640 1 -- 172.21.15.7:0/1565155909 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f138409e0d0 con 0x7f1388102ac0 2024-09-21T05:13:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.793+0000 7f138c8ea640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137407aed0 0x7f137407d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.793+0000 7f138c8ea640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137407aed0 0x7f137407d390 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1378031460 tx=0x7f13780095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:07.797+0000 7f137e7fc640 1 -- 172.21.15.7:0/1565155909 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1384014030 con 0x7f1388102ac0 2024-09-21T05:13:08.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.030+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1388105f60 con 0x7f1388102ac0 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.031+0000 7f137e7fc640 1 -- 172.21.15.7:0/1565155909 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7f13840695d0 con 0x7f1388102ac0 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:13:08.035 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:13:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:13:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:13:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:13:08.038 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:13:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:13:08.038 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:13:08.038 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:13:08.038 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:13:08.038 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137407aed0 msgr2=0x7f137407d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137407aed0 0x7f137407d390 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1378031460 tx=0x7f13780095f0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388102ac0 msgr2=0x7f138819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388102ac0 0x7f138819d680 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f138400c8f0 tx=0x7f138400cdb0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 shutdown_connections 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f137407aed0 0x7f137407d390 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13881033e0 0x7f138819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 --2- 172.21.15.7:0/1565155909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1388102ac0 0x7f138819d680 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 >> 172.21.15.7:0/1565155909 conn(0x7f13880fc520 msgr2=0x7f1388108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:08.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 shutdown_connections 2024-09-21T05:13:08.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.033+0000 7f138f34f640 1 -- 172.21.15.7:0/1565155909 wait complete. 2024-09-21T05:13:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.119+0000 7f8544dbb640 1 -- 172.21.15.7:0/3347811577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8540102ac0 msgr2=0x7f8540102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.119+0000 7f8544dbb640 1 --2- 172.21.15.7:0/3347811577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8540102ac0 0x7f8540102ea0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8530009880 tx=0x7f853002a530 comp rx=0 tx=0).stop 2024-09-21T05:13:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.119+0000 7f8544dbb640 1 -- 172.21.15.7:0/3347811577 shutdown_connections 2024-09-21T05:13:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.119+0000 7f8544dbb640 1 --2- 172.21.15.7:0/3347811577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85401033e0 0x7f854010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.119+0000 7f8544dbb640 1 --2- 172.21.15.7:0/3347811577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8540102ac0 0x7f8540102ea0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.119+0000 7f8544dbb640 1 -- 172.21.15.7:0/3347811577 >> 172.21.15.7:0/3347811577 conn(0x7f85400fc520 msgr2=0x7f85400fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.119+0000 7f8544dbb640 1 -- 172.21.15.7:0/3347811577 shutdown_connections 2024-09-21T05:13:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.119+0000 7f8544dbb640 1 -- 172.21.15.7:0/3347811577 wait complete. 2024-09-21T05:13:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8544dbb640 1 Processor -- start 2024-09-21T05:13:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8544dbb640 1 -- start start 2024-09-21T05:13:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8544dbb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8540102ac0 0x7f854019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8544dbb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85401033e0 0x7f854019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8544dbb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f854010edf0 con 0x7f85401033e0 2024-09-21T05:13:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8544dbb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f854010ec70 con 0x7f8540102ac0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8537fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85401033e0 0x7f854019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8537fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85401033e0 0x7f854019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43640/0 (socket says 172.21.15.7:43640) 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f8537fff640 1 -- 172.21.15.7:0/2143424991 learned_addr learned my addr 172.21.15.7:0/2143424991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.120+0000 7f853e59c640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8540102ac0 0x7f854019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f8537fff640 1 -- 172.21.15.7:0/2143424991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8540102ac0 msgr2=0x7f854019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f8537fff640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8540102ac0 0x7f854019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f8537fff640 1 -- 172.21.15.7:0/2143424991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85401977c0 con 0x7f85401033e0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f8537fff640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85401033e0 0x7f854019db80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f852800d6d0 tx=0x7f852800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f85377fe640 1 -- 172.21.15.7:0/2143424991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8528017070 con 0x7f85401033e0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f85377fe640 1 -- 172.21.15.7:0/2143424991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85280054e0 con 0x7f85401033e0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8540197ab0 con 0x7f85401033e0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f85377fe640 1 -- 172.21.15.7:0/2143424991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8528004ae0 con 0x7f85401033e0 2024-09-21T05:13:08.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.121+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8540198040 con 0x7f85401033e0 2024-09-21T05:13:08.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.122+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854006aa90 con 0x7f85401033e0 2024-09-21T05:13:08.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.123+0000 7f85377fe640 1 -- 172.21.15.7:0/2143424991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8528004d90 con 0x7f85401033e0 2024-09-21T05:13:08.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.124+0000 7f85377fe640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f852c07af80 0x7f852c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:08.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.124+0000 7f853e59c640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f852c07af80 0x7f852c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:08.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.124+0000 7f85377fe640 1 -- 172.21.15.7:0/2143424991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f852809dbb0 con 0x7f85401033e0 2024-09-21T05:13:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.124+0000 7f853e59c640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f852c07af80 0x7f852c07d440 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f85300096c0 tx=0x7f8530031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:08.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.128+0000 7f85377fe640 1 -- 172.21.15.7:0/2143424991 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f85280a4080 con 0x7f85401033e0 2024-09-21T05:13:08.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.337+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f85401982f0 con 0x7f852c07af80 2024-09-21T05:13:08.341 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:08 smithi007 ceph-mon[108238]: from='client.35880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:08.341 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:08 smithi007 ceph-mon[108238]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:08.341 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:08 smithi007 ceph-mon[108238]: from='client.35888 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:08.341 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:08 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2960147765' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:13:08.341 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:08 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1565155909' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.338+0000 7f85377fe640 1 -- 172.21.15.7:0/2143424991 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f85401982f0 con 0x7f852c07af80 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:13:08.342 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:13:08.343 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:13:08.343 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:13:08.343 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:13:08.343 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:13:08.343 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:13:08.343 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:13:08.343 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:13:08.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f852c07af80 msgr2=0x7f852c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f852c07af80 0x7f852c07d440 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f85300096c0 tx=0x7f8530031400 comp rx=0 tx=0).stop 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85401033e0 msgr2=0x7f854019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85401033e0 0x7f854019db80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f852800d6d0 tx=0x7f852800db90 comp rx=0 tx=0).stop 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 shutdown_connections 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f852c07af80 0x7f852c07d440 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85401033e0 0x7f854019db80 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 --2- 172.21.15.7:0/2143424991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8540102ac0 0x7f854019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 >> 172.21.15.7:0/2143424991 conn(0x7f85400fc520 msgr2=0x7f854010af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 shutdown_connections 2024-09-21T05:13:08.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.341+0000 7f8544dbb640 1 -- 172.21.15.7:0/2143424991 wait complete. 2024-09-21T05:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/3027074527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 msgr2=0x7fbc0410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 --2- 172.21.15.7:0/3027074527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 0x7fbc0410b8d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009880 tx=0x7fbbf402a530 comp rx=0 tx=0).stop 2024-09-21T05:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/3027074527 shutdown_connections 2024-09-21T05:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 --2- 172.21.15.7:0/3027074527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 0x7fbc0410b8d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 --2- 172.21.15.7:0/3027074527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc04102ac0 0x7fbc04102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/3027074527 >> 172.21.15.7:0/3027074527 conn(0x7fbc040fc520 msgr2=0x7fbc040fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/3027074527 shutdown_connections 2024-09-21T05:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/3027074527 wait complete. 2024-09-21T05:13:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 Processor -- start 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.425+0000 7fbc0b6a6640 1 -- start start 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc0b6a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc04102ac0 0x7fbc0419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc0b6a6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 0x7fbc0419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc0b6a6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0410edd0 con 0x7fbc041033e0 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc0b6a6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0410ec50 con 0x7fbc04102ac0 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc08c41640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 0x7fbc0419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc08c41640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 0x7fbc0419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43662/0 (socket says 172.21.15.7:43662) 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc08c41640 1 -- 172.21.15.7:0/509748694 learned_addr learned my addr 172.21.15.7:0/509748694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc09442640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc04102ac0 0x7fbc0419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc08c41640 1 -- 172.21.15.7:0/509748694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc04102ac0 msgr2=0x7fbc0419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc08c41640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc04102ac0 0x7fbc0419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc08c41640 1 -- 172.21.15.7:0/509748694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc041977c0 con 0x7fbc041033e0 2024-09-21T05:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbc08c41640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 0x7fbc0419db80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4031460 tx=0x7fbbf4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.426+0000 7fbbfa7fc640 1 -- 172.21.15.7:0/509748694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbbf403f070 con 0x7fbc041033e0 2024-09-21T05:13:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.427+0000 7fbbfa7fc640 1 -- 172.21.15.7:0/509748694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbbf4004a90 con 0x7fbc041033e0 2024-09-21T05:13:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.427+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc04197a50 con 0x7fbc041033e0 2024-09-21T05:13:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.427+0000 7fbbfa7fc640 1 -- 172.21.15.7:0/509748694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbbf4035030 con 0x7fbc041033e0 2024-09-21T05:13:08.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.427+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc04197f80 con 0x7fbc041033e0 2024-09-21T05:13:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.428+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0406aa90 con 0x7fbc041033e0 2024-09-21T05:13:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.428+0000 7fbbfa7fc640 1 -- 172.21.15.7:0/509748694 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbbf4042070 con 0x7fbc041033e0 2024-09-21T05:13:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.429+0000 7fbbfa7fc640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbbf007af90 0x7fbbf007d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.429+0000 7fbbfa7fc640 1 -- 172.21.15.7:0/509748694 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbbf40bea30 con 0x7fbc041033e0 2024-09-21T05:13:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.429+0000 7fbc09442640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbbf007af90 0x7fbbf007d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.429+0000 7fbc09442640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbbf007af90 0x7fbbf007d450 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000bbe0 tx=0x7fbc0000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.433+0000 7fbbfa7fc640 1 -- 172.21.15.7:0/509748694 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbbf408a050 con 0x7fbc041033e0 2024-09-21T05:13:08.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:08 smithi053 ceph-mon[80176]: from='client.35880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:08.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:08 smithi053 ceph-mon[80176]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:08 smithi053 ceph-mon[80176]: from='client.35888 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:08 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2960147765' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:13:08.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:08 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1565155909' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:13:08.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.693+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fbc04198cf0 con 0x7fbc041033e0 2024-09-21T05:13:08.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.693+0000 7fbbfa7fc640 1 -- 172.21.15.7:0/509748694 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fbbf408ef00 con 0x7fbc041033e0 2024-09-21T05:13:08.697 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:13:08.698 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:13:08.699 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:13:08.700 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:13:08.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.695+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbbf007af90 msgr2=0x7fbbf007d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.695+0000 7fbc0b6a6640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbbf007af90 0x7fbbf007d450 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000bbe0 tx=0x7fbc0000b390 comp rx=0 tx=0).stop 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.695+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 msgr2=0x7fbc0419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.695+0000 7fbc0b6a6640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 0x7fbc0419db80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4031460 tx=0x7fbbf4004530 comp rx=0 tx=0).stop 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.696+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 shutdown_connections 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.696+0000 7fbc0b6a6640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbbf007af90 0x7fbbf007d450 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.696+0000 7fbc0b6a6640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc041033e0 0x7fbc0419db80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.696+0000 7fbc0b6a6640 1 --2- 172.21.15.7:0/509748694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc04102ac0 0x7fbc0419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.696+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 >> 172.21.15.7:0/509748694 conn(0x7fbc040fc520 msgr2=0x7fbc04108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.696+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 shutdown_connections 2024-09-21T05:13:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:08.696+0000 7fbc0b6a6640 1 -- 172.21.15.7:0/509748694 wait complete. 2024-09-21T05:13:09.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:09 smithi053 ceph-mon[80176]: pgmap v1152: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:09 smithi053 ceph-mon[80176]: from='client.35900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:09.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:09 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/509748694' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:13:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:09 smithi007 ceph-mon[108238]: pgmap v1152: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:09 smithi007 ceph-mon[108238]: from='client.35900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:09 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/509748694' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:13:11.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:11 smithi007 ceph-mon[108238]: pgmap v1153: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:11 smithi053 ceph-mon[80176]: pgmap v1153: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:13.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:13 smithi053 ceph-mon[80176]: pgmap v1154: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:13 smithi007 ceph-mon[108238]: pgmap v1154: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:13:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:13:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:15 smithi053 ceph-mon[80176]: pgmap v1155: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:15.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:15 smithi007 ceph-mon[108238]: pgmap v1155: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:17.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:17 smithi053 ceph-mon[80176]: pgmap v1156: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:17 smithi007 ceph-mon[108238]: pgmap v1156: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:19.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:19 smithi053 ceph-mon[80176]: pgmap v1157: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:19 smithi007 ceph-mon[108238]: pgmap v1157: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:21.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:21 smithi053 ceph-mon[80176]: pgmap v1158: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:21.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:21 smithi007 ceph-mon[108238]: pgmap v1158: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:23.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:23 smithi053 ceph-mon[80176]: pgmap v1159: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:23 smithi007 ceph-mon[108238]: pgmap v1159: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:25.496 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:25 smithi007 ceph-mon[108238]: pgmap v1160: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:25.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:25 smithi053 ceph-mon[80176]: pgmap v1160: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:27.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:27 smithi053 ceph-mon[80176]: pgmap v1161: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:27 smithi007 ceph-mon[108238]: pgmap v1161: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:29 smithi053 ceph-mon[80176]: pgmap v1162: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:13:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:29 smithi007 ceph-mon[108238]: pgmap v1162: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:13:31.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:31 smithi053 ceph-mon[80176]: pgmap v1163: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:31.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:31 smithi007 ceph-mon[108238]: pgmap v1163: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:33.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:33 smithi053 ceph-mon[80176]: pgmap v1164: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:33 smithi007 ceph-mon[108238]: pgmap v1164: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:35.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:35 smithi007 ceph-mon[108238]: pgmap v1165: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:35.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:35 smithi053 ceph-mon[80176]: pgmap v1165: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:37 smithi053 ceph-mon[80176]: pgmap v1166: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:37 smithi007 ceph-mon[108238]: pgmap v1166: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.795+0000 7ff8e4a83640 1 -- 172.21.15.7:0/3655191406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e0102ac0 msgr2=0x7ff8e0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.795+0000 7ff8e4a83640 1 --2- 172.21.15.7:0/3655191406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e0102ac0 0x7ff8e0102ea0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009880 tx=0x7ff8d002a530 comp rx=0 tx=0).stop 2024-09-21T05:13:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.795+0000 7ff8e4a83640 1 -- 172.21.15.7:0/3655191406 shutdown_connections 2024-09-21T05:13:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.795+0000 7ff8e4a83640 1 --2- 172.21.15.7:0/3655191406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e01033e0 0x7ff8e010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.795+0000 7ff8e4a83640 1 --2- 172.21.15.7:0/3655191406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e0102ac0 0x7ff8e0102ea0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.795+0000 7ff8e4a83640 1 -- 172.21.15.7:0/3655191406 >> 172.21.15.7:0/3655191406 conn(0x7ff8e00fc520 msgr2=0x7ff8e00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.795+0000 7ff8e4a83640 1 -- 172.21.15.7:0/3655191406 shutdown_connections 2024-09-21T05:13:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.795+0000 7ff8e4a83640 1 -- 172.21.15.7:0/3655191406 wait complete. 2024-09-21T05:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8e4a83640 1 Processor -- start 2024-09-21T05:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8e4a83640 1 -- start start 2024-09-21T05:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8e4a83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e0102ac0 0x7ff8e019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8e4a83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e01033e0 0x7ff8e019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8e4a83640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e010edf0 con 0x7ff8e01033e0 2024-09-21T05:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8e4a83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e010ec70 con 0x7ff8e0102ac0 2024-09-21T05:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8ddd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e01033e0 0x7ff8e019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8ddd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e01033e0 0x7ff8e019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38548/0 (socket says 172.21.15.7:38548) 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.796+0000 7ff8ddd9b640 1 -- 172.21.15.7:0/2035562482 learned_addr learned my addr 172.21.15.7:0/2035562482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8de59c640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e0102ac0 0x7ff8e019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8ddd9b640 1 -- 172.21.15.7:0/2035562482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e0102ac0 msgr2=0x7ff8e019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8ddd9b640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e0102ac0 0x7ff8e019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8ddd9b640 1 -- 172.21.15.7:0/2035562482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e01977c0 con 0x7ff8e01033e0 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8ddd9b640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e01033e0 0x7ff8e019db80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400d6d0 tx=0x7ff8d400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8cf7fe640 1 -- 172.21.15.7:0/2035562482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff8d4017070 con 0x7ff8e01033e0 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8cf7fe640 1 -- 172.21.15.7:0/2035562482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8d40054e0 con 0x7ff8e01033e0 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8cf7fe640 1 -- 172.21.15.7:0/2035562482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff8d4004ae0 con 0x7ff8e01033e0 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e0197ab0 con 0x7ff8e01033e0 2024-09-21T05:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.797+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e0198040 con 0x7ff8e01033e0 2024-09-21T05:13:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.798+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e006aa90 con 0x7ff8e01033e0 2024-09-21T05:13:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.799+0000 7ff8cf7fe640 1 -- 172.21.15.7:0/2035562482 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff8d4004d90 con 0x7ff8e01033e0 2024-09-21T05:13:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.800+0000 7ff8cf7fe640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8c807af90 0x7ff8c807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.800+0000 7ff8cf7fe640 1 -- 172.21.15.7:0/2035562482 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff8d409db00 con 0x7ff8e01033e0 2024-09-21T05:13:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.800+0000 7ff8de59c640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8c807af90 0x7ff8c807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.800+0000 7ff8de59c640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8c807af90 0x7ff8c807d450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00096c0 tx=0x7ff8d0031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:38.804+0000 7ff8cf7fe640 1 -- 172.21.15.7:0/2035562482 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8d4069120 con 0x7ff8e01033e0 2024-09-21T05:13:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.011+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff8e01982f0 con 0x7ff8c807af90 2024-09-21T05:13:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.012+0000 7ff8cf7fe640 1 -- 172.21.15.7:0/2035562482 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ff8e01982f0 con 0x7ff8c807af90 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8c807af90 msgr2=0x7ff8c807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8c807af90 0x7ff8c807d450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00096c0 tx=0x7ff8d0031400 comp rx=0 tx=0).stop 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e01033e0 msgr2=0x7ff8e019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e01033e0 0x7ff8e019db80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400d6d0 tx=0x7ff8d400db90 comp rx=0 tx=0).stop 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 shutdown_connections 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff8c807af90 0x7ff8c807d450 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e01033e0 0x7ff8e019db80 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 --2- 172.21.15.7:0/2035562482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e0102ac0 0x7ff8e019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 >> 172.21.15.7:0/2035562482 conn(0x7ff8e00fc520 msgr2=0x7ff8e010af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 shutdown_connections 2024-09-21T05:13:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.015+0000 7ff8e4a83640 1 -- 172.21.15.7:0/2035562482 wait complete. 2024-09-21T05:13:39.027 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:13:39.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 -- 172.21.15.7:0/3073234342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1067f0 msgr2=0x7f00ac106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 --2- 172.21.15.7:0/3073234342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1067f0 0x7f00ac106bd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009880 tx=0x7f00a802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 -- 172.21.15.7:0/3073234342 shutdown_connections 2024-09-21T05:13:39.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 --2- 172.21.15.7:0/3073234342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ac1007f0 0x7f00ac100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 --2- 172.21.15.7:0/3073234342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1067f0 0x7f00ac106bd0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 -- 172.21.15.7:0/3073234342 >> 172.21.15.7:0/3073234342 conn(0x7f00ac0fc550 msgr2=0x7f00ac0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:39.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 -- 172.21.15.7:0/3073234342 shutdown_connections 2024-09-21T05:13:39.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 -- 172.21.15.7:0/3073234342 wait complete. 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 Processor -- start 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.101+0000 7f00b3a80640 1 -- start start 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b3a80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1007f0 0x7f00ac107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b3a80640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ac1067f0 0x7f00ac107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b3a80640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac10dca0 con 0x7f00ac1007f0 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b3a80640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac10db20 con 0x7f00ac1067f0 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b181c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1007f0 0x7f00ac107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b181c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1007f0 0x7f00ac107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38568/0 (socket says 172.21.15.7:38568) 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b181c640 1 -- 172.21.15.7:0/2315128640 learned_addr learned my addr 172.21.15.7:0/2315128640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b101b640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ac1067f0 0x7f00ac107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b181c640 1 -- 172.21.15.7:0/2315128640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ac1067f0 msgr2=0x7f00ac107c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b181c640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ac1067f0 0x7f00ac107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b181c640 1 -- 172.21.15.7:0/2315128640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac1081a0 con 0x7f00ac1007f0 2024-09-21T05:13:39.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b181c640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1007f0 0x7f00ac107720 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f00a8031460 tx=0x7f00a8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:39.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f009effd640 1 -- 172.21.15.7:0/2315128640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00a803f070 con 0x7f00ac1007f0 2024-09-21T05:13:39.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f009effd640 1 -- 172.21.15.7:0/2315128640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00a80035f0 con 0x7f00ac1007f0 2024-09-21T05:13:39.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.102+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac108430 con 0x7f00ac1007f0 2024-09-21T05:13:39.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.103+0000 7f009effd640 1 -- 172.21.15.7:0/2315128640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00a8035030 con 0x7f00ac1007f0 2024-09-21T05:13:39.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.103+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1a3d10 con 0x7f00ac1007f0 2024-09-21T05:13:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.104+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ac101f30 con 0x7f00ac1007f0 2024-09-21T05:13:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.104+0000 7f009effd640 1 -- 172.21.15.7:0/2315128640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f00a8042070 con 0x7f00ac1007f0 2024-09-21T05:13:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.105+0000 7f009effd640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f009807af80 0x7f009807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.105+0000 7f009effd640 1 -- 172.21.15.7:0/2315128640 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f00a80be940 con 0x7f00ac1007f0 2024-09-21T05:13:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.105+0000 7f00b101b640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f009807af80 0x7f009807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.105+0000 7f00b101b640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f009807af80 0x7f009807d440 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00ac1091b0 tx=0x7f00a0005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:39.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.109+0000 7f009effd640 1 -- 172.21.15.7:0/2315128640 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f00a8089f60 con 0x7f00ac1007f0 2024-09-21T05:13:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.314+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f00ac06a520 con 0x7f009807af80 2024-09-21T05:13:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.316+0000 7f009effd640 1 -- 172.21.15.7:0/2315128640 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f00ac06a520 con 0x7f009807af80 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f009807af80 msgr2=0x7f009807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f009807af80 0x7f009807d440 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00ac1091b0 tx=0x7f00a0005f90 comp rx=0 tx=0).stop 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1007f0 msgr2=0x7f00ac107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1007f0 0x7f00ac107720 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f00a8031460 tx=0x7f00a8004530 comp rx=0 tx=0).stop 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 shutdown_connections 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f009807af80 0x7f009807d440 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ac1067f0 0x7f00ac107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 --2- 172.21.15.7:0/2315128640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00ac1007f0 0x7f00ac107720 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 >> 172.21.15.7:0/2315128640 conn(0x7f00ac0fc550 msgr2=0x7f00ac069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:39.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 shutdown_connections 2024-09-21T05:13:39.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.318+0000 7f00b3a80640 1 -- 172.21.15.7:0/2315128640 wait complete. 2024-09-21T05:13:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.402+0000 7f8e05212640 1 -- 172.21.15.7:0/3329622068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 msgr2=0x7f8e00107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.402+0000 7f8e05212640 1 --2- 172.21.15.7:0/3329622068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 0x7f8e00107ce0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009880 tx=0x7f8df002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:39 smithi007 ceph-mon[108238]: pgmap v1167: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:39.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.402+0000 7f8e05212640 1 -- 172.21.15.7:0/3329622068 shutdown_connections 2024-09-21T05:13:39.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.402+0000 7f8e05212640 1 --2- 172.21.15.7:0/3329622068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101900 0x7f8e00101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.402+0000 7f8e05212640 1 --2- 172.21.15.7:0/3329622068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 0x7f8e00107ce0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.402+0000 7f8e05212640 1 -- 172.21.15.7:0/3329622068 >> 172.21.15.7:0/3329622068 conn(0x7f8e000fd600 msgr2=0x7f8e000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:39.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.402+0000 7f8e05212640 1 -- 172.21.15.7:0/3329622068 shutdown_connections 2024-09-21T05:13:39.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.402+0000 7f8e05212640 1 -- 172.21.15.7:0/3329622068 wait complete. 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8e05212640 1 Processor -- start 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8e05212640 1 -- start start 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8e05212640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101900 0x7f8e0019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8e05212640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 0x7f8e0019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8e05212640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0010edb0 con 0x7f8e00107900 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8e05212640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0010ec30 con 0x7f8e00101900 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8df7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 0x7f8e0019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8df7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 0x7f8e0019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38590/0 (socket says 172.21.15.7:38590) 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8df7fff640 1 -- 172.21.15.7:0/835688281 learned_addr learned my addr 172.21.15.7:0/835688281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8dfed9d640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101900 0x7f8e0019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8df7fff640 1 -- 172.21.15.7:0/835688281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101900 msgr2=0x7f8e0019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8df7fff640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101900 0x7f8e0019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.403+0000 7f8df7fff640 1 -- 172.21.15.7:0/835688281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e001977c0 con 0x7f8e00107900 2024-09-21T05:13:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.404+0000 7f8df7fff640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 0x7f8e0019db80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8de800c8a0 tx=0x7f8de800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.404+0000 7f8dfcd99640 1 -- 172.21.15.7:0/835688281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8de80079b0 con 0x7f8e00107900 2024-09-21T05:13:39.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.404+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e00197ab0 con 0x7f8e00107900 2024-09-21T05:13:39.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.404+0000 7f8dfcd99640 1 -- 172.21.15.7:0/835688281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8de8007b50 con 0x7f8e00107900 2024-09-21T05:13:39.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.404+0000 7f8dfcd99640 1 -- 172.21.15.7:0/835688281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8de8018700 con 0x7f8e00107900 2024-09-21T05:13:39.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.404+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e00198040 con 0x7f8e00107900 2024-09-21T05:13:39.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.405+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e00103040 con 0x7f8e00107900 2024-09-21T05:13:39.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.406+0000 7f8dfcd99640 1 -- 172.21.15.7:0/835688281 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8de8007d00 con 0x7f8e00107900 2024-09-21T05:13:39.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.406+0000 7f8dfcd99640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8dec07af30 0x7f8dec07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.406+0000 7f8dfcd99640 1 -- 172.21.15.7:0/835688281 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f8de809e380 con 0x7f8e00107900 2024-09-21T05:13:39.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.406+0000 7f8dfed9d640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8dec07af30 0x7f8dec07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.407+0000 7f8dfed9d640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8dec07af30 0x7f8dec07d3f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8df0031460 tx=0x7f8df0005bd0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.411+0000 7f8dfcd99640 1 -- 172.21.15.7:0/835688281 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8de8014030 con 0x7f8e00107900 2024-09-21T05:13:39.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.606+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8e00198350 con 0x7f8dec07af30 2024-09-21T05:13:39.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.616+0000 7f8dfcd99640 1 -- 172.21.15.7:0/835688281 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f8e00198350 con 0x7f8dec07af30 2024-09-21T05:13:39.618 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:13:39.618 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (35m) 14m ago 46m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (38m) 14m ago 46m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (38m) 14m ago 45m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (33m) 14m ago 46m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (33m) 14m ago 45m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (35m) 14m ago 46m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (29m) 14m ago 41m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (30m) 14m ago 41m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (14m) 14m ago 41m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (29m) 14m ago 41m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (37m) 14m ago 48m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (36m) 14m ago 45m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (34m) 14m ago 48m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:13:39.619 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (34m) 14m ago 45m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (36m) 14m ago 46m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (36m) 14m ago 45m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (33m) 14m ago 44m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (32m) 14m ago 44m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (32m) 14m ago 43m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (31m) 14m ago 43m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (30m) 14m ago 42m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (30m) 14m ago 42m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:13:39.620 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (36m) 14m ago 46m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8dec07af30 msgr2=0x7f8dec07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8dec07af30 0x7f8dec07d3f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8df0031460 tx=0x7f8df0005bd0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 msgr2=0x7f8e0019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 0x7f8e0019db80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8de800c8a0 tx=0x7f8de800cd60 comp rx=0 tx=0).stop 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 shutdown_connections 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f8dec07af30 0x7f8dec07d3f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e00107900 0x7f8e0019db80 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 --2- 172.21.15.7:0/835688281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e00101900 0x7f8e0019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 >> 172.21.15.7:0/835688281 conn(0x7f8e000fd600 msgr2=0x7f8e000ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 shutdown_connections 2024-09-21T05:13:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.618+0000 7f8e05212640 1 -- 172.21.15.7:0/835688281 wait complete. 2024-09-21T05:13:39.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:39 smithi053 ceph-mon[80176]: pgmap v1167: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 -- 172.21.15.7:0/540069221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 msgr2=0x7fcacc106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 --2- 172.21.15.7:0/540069221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 0x7fcacc106bd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009880 tx=0x7fcabc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 -- 172.21.15.7:0/540069221 shutdown_connections 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 --2- 172.21.15.7:0/540069221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1007f0 0x7fcacc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 --2- 172.21.15.7:0/540069221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 0x7fcacc106bd0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 -- 172.21.15.7:0/540069221 >> 172.21.15.7:0/540069221 conn(0x7fcacc0fc550 msgr2=0x7fcacc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 -- 172.21.15.7:0/540069221 shutdown_connections 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 -- 172.21.15.7:0/540069221 wait complete. 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 Processor -- start 2024-09-21T05:13:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.701+0000 7fcad2130640 1 -- start start 2024-09-21T05:13:39.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcad2130640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1007f0 0x7fcacc1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcad2130640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 0x7fcacc198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcad2130640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc10dca0 con 0x7fcacc1067f0 2024-09-21T05:13:39.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcad2130640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc10db20 con 0x7fcacc1007f0 2024-09-21T05:13:39.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcac2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 0x7fcacc198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcac2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 0x7fcacc198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38606/0 (socket says 172.21.15.7:38606) 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1007f0 0x7fcacc1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcac2ffd640 1 -- 172.21.15.7:0/2593502418 learned_addr learned my addr 172.21.15.7:0/2593502418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcac2ffd640 1 -- 172.21.15.7:0/2593502418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1007f0 msgr2=0x7fcacc1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcac2ffd640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1007f0 0x7fcacc1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.702+0000 7fcac2ffd640 1 -- 172.21.15.7:0/2593502418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc199410 con 0x7fcacc1067f0 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.703+0000 7fcac2ffd640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 0x7fcacc198e10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcab400d8d0 tx=0x7fcab400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.703+0000 7fcac97fa640 1 -- 172.21.15.7:0/2593502418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcab4017030 con 0x7fcacc1067f0 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.703+0000 7fcac97fa640 1 -- 172.21.15.7:0/2593502418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcab4004020 con 0x7fcacc1067f0 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.703+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc199700 con 0x7fcacc1067f0 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.703+0000 7fcac97fa640 1 -- 172.21.15.7:0/2593502418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcab40047a0 con 0x7fcacc1067f0 2024-09-21T05:13:39.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.703+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc1a1d00 con 0x7fcacc1067f0 2024-09-21T05:13:39.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.704+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcacc101f30 con 0x7fcacc1067f0 2024-09-21T05:13:39.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.705+0000 7fcac97fa640 1 -- 172.21.15.7:0/2593502418 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fcab40041d0 con 0x7fcacc1067f0 2024-09-21T05:13:39.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.706+0000 7fcac97fa640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcab807af90 0x7fcab807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:39.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.706+0000 7fcacb7fe640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcab807af90 0x7fcab807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:39.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.706+0000 7fcac97fa640 1 -- 172.21.15.7:0/2593502418 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fcab4018070 con 0x7fcacc1067f0 2024-09-21T05:13:39.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.706+0000 7fcacb7fe640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcab807af90 0x7fcab807d450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0096c0 tx=0x7fcabc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.710+0000 7fcac97fa640 1 -- 172.21.15.7:0/2593502418 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcacc199700 con 0x7fcacc1067f0 2024-09-21T05:13:39.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.975+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fcacc06a2c0 con 0x7fcacc1067f0 2024-09-21T05:13:39.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.975+0000 7fcac97fa640 1 -- 172.21.15.7:0/2593502418 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fcacc06a2c0 con 0x7fcacc1067f0 2024-09-21T05:13:39.978 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:13:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:13:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:13:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:13:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:13:39.979 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcab807af90 msgr2=0x7fcab807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcab807af90 0x7fcab807d450 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0096c0 tx=0x7fcabc031400 comp rx=0 tx=0).stop 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 msgr2=0x7fcacc198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 0x7fcacc198e10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcab400d8d0 tx=0x7fcab400dd90 comp rx=0 tx=0).stop 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 shutdown_connections 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fcab807af90 0x7fcab807d450 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcacc1067f0 0x7fcacc198e10 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 --2- 172.21.15.7:0/2593502418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcacc1007f0 0x7fcacc1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 >> 172.21.15.7:0/2593502418 conn(0x7fcacc0fc550 msgr2=0x7fcacc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 shutdown_connections 2024-09-21T05:13:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:39.978+0000 7fcad2130640 1 -- 172.21.15.7:0/2593502418 wait complete. 2024-09-21T05:13:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.062+0000 7eff29f44640 1 -- 172.21.15.7:0/3086740884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24101900 msgr2=0x7eff24101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.062+0000 7eff29f44640 1 --2- 172.21.15.7:0/3086740884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24101900 0x7eff24101d60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7eff18009880 tx=0x7eff1802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.062+0000 7eff29f44640 1 -- 172.21.15.7:0/3086740884 shutdown_connections 2024-09-21T05:13:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.062+0000 7eff29f44640 1 --2- 172.21.15.7:0/3086740884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24101900 0x7eff24101d60 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.062+0000 7eff29f44640 1 --2- 172.21.15.7:0/3086740884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24107900 0x7eff24107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.062+0000 7eff29f44640 1 -- 172.21.15.7:0/3086740884 >> 172.21.15.7:0/3086740884 conn(0x7eff240fd600 msgr2=0x7eff240ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.062+0000 7eff29f44640 1 -- 172.21.15.7:0/3086740884 shutdown_connections 2024-09-21T05:13:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.062+0000 7eff29f44640 1 -- 172.21.15.7:0/3086740884 wait complete. 2024-09-21T05:13:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff29f44640 1 Processor -- start 2024-09-21T05:13:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff29f44640 1 -- start start 2024-09-21T05:13:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff29f44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101900 0x7eff240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff29f44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24107900 0x7eff24074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff29f44640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff2410edb0 con 0x7eff24107900 2024-09-21T05:13:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff29f44640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff2410ec30 con 0x7eff24101900 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff22ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24107900 0x7eff24074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff22ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24107900 0x7eff24074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38622/0 (socket says 172.21.15.7:38622) 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.063+0000 7eff22ffd640 1 -- 172.21.15.7:0/4028127731 learned_addr learned my addr 172.21.15.7:0/4028127731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff237fe640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101900 0x7eff240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff22ffd640 1 -- 172.21.15.7:0/4028127731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101900 msgr2=0x7eff240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff22ffd640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101900 0x7eff240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff22ffd640 1 -- 172.21.15.7:0/4028127731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff24074f70 con 0x7eff24107900 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff22ffd640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24107900 0x7eff24074a30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7eff18031460 tx=0x7eff180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff20ff9640 1 -- 172.21.15.7:0/4028127731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7eff1803f070 con 0x7eff24107900 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff24075200 con 0x7eff24107900 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff20ff9640 1 -- 172.21.15.7:0/4028127731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7eff18004cf0 con 0x7eff24107900 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff20ff9640 1 -- 172.21.15.7:0/4028127731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7eff18035030 con 0x7eff24107900 2024-09-21T05:13:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.064+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff241a3dc0 con 0x7eff24107900 2024-09-21T05:13:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.065+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff24103040 con 0x7eff24107900 2024-09-21T05:13:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.066+0000 7eff20ff9640 1 -- 172.21.15.7:0/4028127731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7eff18042070 con 0x7eff24107900 2024-09-21T05:13:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.066+0000 7eff20ff9640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff0c07af80 0x7eff0c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.066+0000 7eff20ff9640 1 -- 172.21.15.7:0/4028127731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7eff180bea40 con 0x7eff24107900 2024-09-21T05:13:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.066+0000 7eff237fe640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff0c07af80 0x7eff0c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.067+0000 7eff237fe640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff0c07af80 0x7eff0c07d440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7eff140058c0 tx=0x7eff1400a4c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.071+0000 7eff20ff9640 1 -- 172.21.15.7:0/4028127731 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff1808a060 con 0x7eff24107900 2024-09-21T05:13:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.297+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7eff24075a20 con 0x7eff24107900 2024-09-21T05:13:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.298+0000 7eff20ff9640 1 -- 172.21.15.7:0/4028127731 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7eff1808ef10 con 0x7eff24107900 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:13:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:13:40.302 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:13:40.303 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:13:40.304 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:13:40.304 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:13:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:13:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.299+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff0c07af80 msgr2=0x7eff0c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.299+0000 7eff29f44640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff0c07af80 0x7eff0c07d440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7eff140058c0 tx=0x7eff1400a4c0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24107900 msgr2=0x7eff24074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24107900 0x7eff24074a30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7eff18031460 tx=0x7eff180048f0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 shutdown_connections 2024-09-21T05:13:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7eff0c07af80 0x7eff0c07d440 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24107900 0x7eff24074a30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 --2- 172.21.15.7:0/4028127731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff24101900 0x7eff240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 >> 172.21.15.7:0/4028127731 conn(0x7eff240fd600 msgr2=0x7eff240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 shutdown_connections 2024-09-21T05:13:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.300+0000 7eff29f44640 1 -- 172.21.15.7:0/4028127731 wait complete. 2024-09-21T05:13:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 -- 172.21.15.7:0/2379372613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102b70 msgr2=0x7f452c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 --2- 172.21.15.7:0/2379372613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102b70 0x7f452c102f50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4528009880 tx=0x7f452802a530 comp rx=0 tx=0).stop 2024-09-21T05:13:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 -- 172.21.15.7:0/2379372613 shutdown_connections 2024-09-21T05:13:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 --2- 172.21.15.7:0/2379372613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f452c103490 0x7f452c10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 --2- 172.21.15.7:0/2379372613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c102b70 0x7f452c102f50 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 -- 172.21.15.7:0/2379372613 >> 172.21.15.7:0/2379372613 conn(0x7f452c0fc550 msgr2=0x7f452c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 -- 172.21.15.7:0/2379372613 shutdown_connections 2024-09-21T05:13:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 -- 172.21.15.7:0/2379372613 wait complete. 2024-09-21T05:13:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.386+0000 7f453419b640 1 Processor -- start 2024-09-21T05:13:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f453419b640 1 -- start start 2024-09-21T05:13:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f453419b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f452c102b70 0x7f452c1007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f453419b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c103490 0x7f452c100d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f453419b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f452c10ede0 con 0x7f452c103490 2024-09-21T05:13:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f453419b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f452c10ec60 con 0x7f452c102b70 2024-09-21T05:13:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531736640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c103490 0x7f452c100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531736640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c103490 0x7f452c100d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38652/0 (socket says 172.21.15.7:38652) 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531736640 1 -- 172.21.15.7:0/3010543420 learned_addr learned my addr 172.21.15.7:0/3010543420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531f37640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f452c102b70 0x7f452c1007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531736640 1 -- 172.21.15.7:0/3010543420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f452c102b70 msgr2=0x7f452c1007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531736640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f452c102b70 0x7f452c1007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531736640 1 -- 172.21.15.7:0/3010543420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c101250 con 0x7f452c103490 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531f37640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f452c102b70 0x7f452c1007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.387+0000 7f4531736640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c103490 0x7f452c100d10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f451c00d6f0 tx=0x7f451c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.388+0000 7f4522ffd640 1 -- 172.21.15.7:0/3010543420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f451c017070 con 0x7f452c103490 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.388+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c101540 con 0x7f452c103490 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.388+0000 7f4522ffd640 1 -- 172.21.15.7:0/3010543420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f451c00de00 con 0x7f452c103490 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.388+0000 7f4522ffd640 1 -- 172.21.15.7:0/3010543420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f451c004bb0 con 0x7f452c103490 2024-09-21T05:13:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.388+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a3e30 con 0x7f452c103490 2024-09-21T05:13:40.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.389+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c06a4c0 con 0x7f452c103490 2024-09-21T05:13:40.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.389+0000 7f4522ffd640 1 -- 172.21.15.7:0/3010543420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f451c004e60 con 0x7f452c103490 2024-09-21T05:13:40.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.390+0000 7f4522ffd640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f451807af30 0x7f451807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.390+0000 7f4522ffd640 1 -- 172.21.15.7:0/3010543420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f451c09dcf0 con 0x7f452c103490 2024-09-21T05:13:40.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.390+0000 7f4531f37640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f451807af30 0x7f451807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.391+0000 7f4531f37640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f451807af30 0x7f451807d3f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4528000c00 tx=0x7f4528004f30 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.395+0000 7f4522ffd640 1 -- 172.21.15.7:0/3010543420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f451c069230 con 0x7f452c103490 2024-09-21T05:13:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.599+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f452c062630 con 0x7f451807af30 2024-09-21T05:13:40.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:40 smithi007 ceph-mon[108238]: from='client.35908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:40.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:40 smithi007 ceph-mon[108238]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:40.602 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:40 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2593502418' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:13:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.600+0000 7f4522ffd640 1 -- 172.21.15.7:0/3010543420 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f452c062630 con 0x7f451807af30 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:13:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:13:40.604 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:13:40.604 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:13:40.604 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f451807af30 msgr2=0x7f451807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f451807af30 0x7f451807d3f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4528000c00 tx=0x7f4528004f30 comp rx=0 tx=0).stop 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c103490 msgr2=0x7f452c100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c103490 0x7f452c100d10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f451c00d6f0 tx=0x7f451c00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 shutdown_connections 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f451807af30 0x7f451807d3f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f452c103490 0x7f452c100d10 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 --2- 172.21.15.7:0/3010543420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f452c102b70 0x7f452c1007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 >> 172.21.15.7:0/3010543420 conn(0x7f452c0fc550 msgr2=0x7f452c108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 shutdown_connections 2024-09-21T05:13:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.603+0000 7f453419b640 1 -- 172.21.15.7:0/3010543420 wait complete. 2024-09-21T05:13:40.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 -- 172.21.15.7:0/101545826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301007f0 msgr2=0x7fe830100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 --2- 172.21.15.7:0/101545826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301007f0 0x7fe830100c50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe820009880 tx=0x7fe82002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 -- 172.21.15.7:0/101545826 shutdown_connections 2024-09-21T05:13:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 --2- 172.21.15.7:0/101545826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301007f0 0x7fe830100c50 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 --2- 172.21.15.7:0/101545826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8301067f0 0x7fe830106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 -- 172.21.15.7:0/101545826 >> 172.21.15.7:0/101545826 conn(0x7fe8300fc550 msgr2=0x7fe8300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 -- 172.21.15.7:0/101545826 shutdown_connections 2024-09-21T05:13:40.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 -- 172.21.15.7:0/101545826 wait complete. 2024-09-21T05:13:40.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.687+0000 7fe83807c640 1 Processor -- start 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe83807c640 1 -- start start 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe83807c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8301007f0 0x7fe8301987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe83807c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301067f0 0x7fe830198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe83807c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010dcc0 con 0x7fe8301067f0 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe83807c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010db40 con 0x7fe8301007f0 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe835617640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301067f0 0x7fe830198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe835617640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301067f0 0x7fe830198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38666/0 (socket says 172.21.15.7:38666) 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe835617640 1 -- 172.21.15.7:0/3756854268 learned_addr learned my addr 172.21.15.7:0/3756854268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe835e18640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8301007f0 0x7fe8301987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe835617640 1 -- 172.21.15.7:0/3756854268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8301007f0 msgr2=0x7fe8301987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe835617640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8301007f0 0x7fe8301987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe835617640 1 -- 172.21.15.7:0/3756854268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe830199330 con 0x7fe8301067f0 2024-09-21T05:13:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.688+0000 7fe835617640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301067f0 0x7fe830198d30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe820031460 tx=0x7fe8200045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.689+0000 7fe826ffd640 1 -- 172.21.15.7:0/3756854268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe82003f070 con 0x7fe8301067f0 2024-09-21T05:13:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.689+0000 7fe826ffd640 1 -- 172.21.15.7:0/3756854268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe8200035f0 con 0x7fe8301067f0 2024-09-21T05:13:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.689+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8301995c0 con 0x7fe8301067f0 2024-09-21T05:13:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.689+0000 7fe826ffd640 1 -- 172.21.15.7:0/3756854268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe820035030 con 0x7fe8301067f0 2024-09-21T05:13:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.689+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8301a1b70 con 0x7fe8301067f0 2024-09-21T05:13:40.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:40 smithi053 ceph-mon[80176]: from='client.35908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:40.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:40 smithi053 ceph-mon[80176]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:40.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:40 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2593502418' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:13:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.690+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe830101f30 con 0x7fe8301067f0 2024-09-21T05:13:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.690+0000 7fe826ffd640 1 -- 172.21.15.7:0/3756854268 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe820042070 con 0x7fe8301067f0 2024-09-21T05:13:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.691+0000 7fe826ffd640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81c07af30 0x7fe81c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:13:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.691+0000 7fe826ffd640 1 -- 172.21.15.7:0/3756854268 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fe8200beae0 con 0x7fe8301067f0 2024-09-21T05:13:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.691+0000 7fe835e18640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81c07af30 0x7fe81c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:13:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.692+0000 7fe835e18640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81c07af30 0x7fe81c07d3f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe82c005fd0 tx=0x7fe82c005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:13:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.695+0000 7fe826ffd640 1 -- 172.21.15.7:0/3756854268 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe82008a020 con 0x7fe8301067f0 2024-09-21T05:13:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.961+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe830100c50 con 0x7fe8301067f0 2024-09-21T05:13:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.962+0000 7fe826ffd640 1 -- 172.21.15.7:0/3756854268 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fe830100c50 con 0x7fe8301067f0 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:13:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:13:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:13:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81c07af30 msgr2=0x7fe81c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81c07af30 0x7fe81c07d3f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe82c005fd0 tx=0x7fe82c005850 comp rx=0 tx=0).stop 2024-09-21T05:13:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301067f0 msgr2=0x7fe830198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:13:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301067f0 0x7fe830198d30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe820031460 tx=0x7fe8200045e0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 shutdown_connections 2024-09-21T05:13:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe81c07af30 0x7fe81c07d3f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8301067f0 0x7fe830198d30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 --2- 172.21.15.7:0/3756854268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8301007f0 0x7fe8301987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:13:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 >> 172.21.15.7:0/3756854268 conn(0x7fe8300fc550 msgr2=0x7fe8300fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:13:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 shutdown_connections 2024-09-21T05:13:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:13:40.964+0000 7fe83807c640 1 -- 172.21.15.7:0/3756854268 wait complete. 2024-09-21T05:13:41.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:41 smithi053 ceph-mon[80176]: pgmap v1168: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:41.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:41 smithi053 ceph-mon[80176]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:41 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4028127731' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:13:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:41 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3756854268' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:13:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:41 smithi007 ceph-mon[108238]: pgmap v1168: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:41 smithi007 ceph-mon[108238]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:41 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4028127731' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:13:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:41 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3756854268' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:13:42.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:42 smithi053 ceph-mon[80176]: from='client.35928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:42.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:42 smithi007 ceph-mon[108238]: from='client.35928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:13:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:43 smithi053 ceph-mon[80176]: pgmap v1169: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:43 smithi007 ceph-mon[108238]: pgmap v1169: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:44.593 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:13:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:13:45.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:45 smithi007 ceph-mon[108238]: pgmap v1170: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:45.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:45 smithi053 ceph-mon[80176]: pgmap v1170: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:47.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:47 smithi053 ceph-mon[80176]: pgmap v1171: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:47.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:47 smithi007 ceph-mon[108238]: pgmap v1171: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:49 smithi053 ceph-mon[80176]: pgmap v1172: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:49.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:49 smithi007 ceph-mon[108238]: pgmap v1172: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:51.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:51 smithi053 ceph-mon[80176]: pgmap v1173: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:51 smithi007 ceph-mon[108238]: pgmap v1173: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:53.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:53 smithi053 ceph-mon[80176]: pgmap v1174: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:53.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:53 smithi007 ceph-mon[108238]: pgmap v1174: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:55.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:55 smithi007 ceph-mon[108238]: pgmap v1175: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:55.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:55 smithi053 ceph-mon[80176]: pgmap v1175: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:57.571 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:57 smithi053 ceph-mon[80176]: pgmap v1176: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:57.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:57 smithi007 ceph-mon[108238]: pgmap v1176: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:59 smithi053 ceph-mon[80176]: pgmap v1177: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:59.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:13:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:13:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:59 smithi007 ceph-mon[108238]: pgmap v1177: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:13:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:13:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:14:01.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:01 smithi007 ceph-mon[108238]: pgmap v1178: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:01.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:01 smithi053 ceph-mon[80176]: pgmap v1178: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:03.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:03 smithi053 ceph-mon[80176]: pgmap v1179: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:03.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:03 smithi007 ceph-mon[108238]: pgmap v1179: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:05.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:05 smithi053 ceph-mon[80176]: pgmap v1180: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:05.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:05 smithi007 ceph-mon[108238]: pgmap v1180: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:07.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:07 smithi053 ceph-mon[80176]: pgmap v1181: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:07.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:07 smithi007 ceph-mon[108238]: pgmap v1181: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:09.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:09 smithi053 ceph-mon[80176]: pgmap v1182: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:09.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:09 smithi007 ceph-mon[108238]: pgmap v1182: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:11.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.059+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3813621501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 msgr2=0x7fbda4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.059+0000 7fbdaaad9640 1 --2- 172.21.15.7:0/3813621501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 0x7fbda4102ea0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009880 tx=0x7fbda002a530 comp rx=0 tx=0).stop 2024-09-21T05:14:11.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.059+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3813621501 shutdown_connections 2024-09-21T05:14:11.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.059+0000 7fbdaaad9640 1 --2- 172.21.15.7:0/3813621501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbda41033e0 0x7fbda410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.059+0000 7fbdaaad9640 1 --2- 172.21.15.7:0/3813621501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 0x7fbda4102ea0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.059+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3813621501 >> 172.21.15.7:0/3813621501 conn(0x7fbda40fc520 msgr2=0x7fbda40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:11.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.059+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3813621501 shutdown_connections 2024-09-21T05:14:11.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.059+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3813621501 wait complete. 2024-09-21T05:14:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.060+0000 7fbdaaad9640 1 Processor -- start 2024-09-21T05:14:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.060+0000 7fbdaaad9640 1 -- start start 2024-09-21T05:14:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbdaaad9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 0x7fbda419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbdaaad9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbda41033e0 0x7fbda419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbdaaad9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbda410edd0 con 0x7fbda4102ac0 2024-09-21T05:14:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbdaaad9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbda410ec50 con 0x7fbda41033e0 2024-09-21T05:14:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbda8875640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 0x7fbda419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbda8875640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 0x7fbda419d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49558/0 (socket says 172.21.15.7:49558) 2024-09-21T05:14:11.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbda8875640 1 -- 172.21.15.7:0/3680097757 learned_addr learned my addr 172.21.15.7:0/3680097757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:11.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbd9bfff640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbda41033e0 0x7fbda419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbda8875640 1 -- 172.21.15.7:0/3680097757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbda41033e0 msgr2=0x7fbda419dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbda8875640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbda41033e0 0x7fbda419dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbda8875640 1 -- 172.21.15.7:0/3680097757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda419e310 con 0x7fbda4102ac0 2024-09-21T05:14:11.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.061+0000 7fbda8875640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 0x7fbda419d720 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbda0031460 tx=0x7fbda00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.062+0000 7fbd99ffb640 1 -- 172.21.15.7:0/3680097757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbda003f070 con 0x7fbda4102ac0 2024-09-21T05:14:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.062+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda41979a0 con 0x7fbda4102ac0 2024-09-21T05:14:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.062+0000 7fbd99ffb640 1 -- 172.21.15.7:0/3680097757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbda0004cf0 con 0x7fbda4102ac0 2024-09-21T05:14:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.062+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda4197ea0 con 0x7fbda4102ac0 2024-09-21T05:14:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.062+0000 7fbd99ffb640 1 -- 172.21.15.7:0/3680097757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbda0049060 con 0x7fbda4102ac0 2024-09-21T05:14:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.064+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda406aa90 con 0x7fbda4102ac0 2024-09-21T05:14:11.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.064+0000 7fbd99ffb640 1 -- 172.21.15.7:0/3680097757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbda0042070 con 0x7fbda4102ac0 2024-09-21T05:14:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.065+0000 7fbd99ffb640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd8c07af30 0x7fbd8c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.065+0000 7fbd99ffb640 1 -- 172.21.15.7:0/3680097757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbda00be870 con 0x7fbda4102ac0 2024-09-21T05:14:11.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.065+0000 7fbd9bfff640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd8c07af30 0x7fbd8c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.065+0000 7fbd9bfff640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd8c07af30 0x7fbd8c07d3f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbda40feec0 tx=0x7fbd9400a250 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.069+0000 7fbd99ffb640 1 -- 172.21.15.7:0/3680097757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbda0089e10 con 0x7fbda4102ac0 2024-09-21T05:14:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.277+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbda4198e10 con 0x7fbd8c07af30 2024-09-21T05:14:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.278+0000 7fbd99ffb640 1 -- 172.21.15.7:0/3680097757 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fbda4198e10 con 0x7fbd8c07af30 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd8c07af30 msgr2=0x7fbd8c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd8c07af30 0x7fbd8c07d3f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbda40feec0 tx=0x7fbd9400a250 comp rx=0 tx=0).stop 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 msgr2=0x7fbda419d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 0x7fbda419d720 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbda0031460 tx=0x7fbda00048f0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 shutdown_connections 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbd8c07af30 0x7fbd8c07d3f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbda41033e0 0x7fbda419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 --2- 172.21.15.7:0/3680097757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda4102ac0 0x7fbda419d720 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 >> 172.21.15.7:0/3680097757 conn(0x7fbda40fc520 msgr2=0x7fbda4108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:11.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 shutdown_connections 2024-09-21T05:14:11.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.280+0000 7fbdaaad9640 1 -- 172.21.15.7:0/3680097757 wait complete. 2024-09-21T05:14:11.294 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:14:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.373+0000 7f2653b37640 1 -- 172.21.15.7:0/2624220140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 msgr2=0x7f264c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.373+0000 7f2653b37640 1 --2- 172.21.15.7:0/2624220140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 0x7f264c106bd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2648009880 tx=0x7f264802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.373+0000 7f2653b37640 1 -- 172.21.15.7:0/2624220140 shutdown_connections 2024-09-21T05:14:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.373+0000 7f2653b37640 1 --2- 172.21.15.7:0/2624220140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264c1007f0 0x7f264c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.373+0000 7f2653b37640 1 --2- 172.21.15.7:0/2624220140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 0x7f264c106bd0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.373+0000 7f2653b37640 1 -- 172.21.15.7:0/2624220140 >> 172.21.15.7:0/2624220140 conn(0x7f264c0fc550 msgr2=0x7f264c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:11.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.373+0000 7f2653b37640 1 -- 172.21.15.7:0/2624220140 shutdown_connections 2024-09-21T05:14:11.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.373+0000 7f2653b37640 1 -- 172.21.15.7:0/2624220140 wait complete. 2024-09-21T05:14:11.378 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:11 smithi007 ceph-mon[108238]: pgmap v1183: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:11.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.374+0000 7f2653b37640 1 Processor -- start 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.374+0000 7f2653b37640 1 -- start start 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.374+0000 7f2653b37640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264c1007f0 0x7f264c0fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.374+0000 7f2653b37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 0x7f264c0fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.374+0000 7f2653b37640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f264c10dcc0 con 0x7f264c1067f0 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.374+0000 7f2653b37640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f264c10db40 con 0x7f264c1007f0 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f26510d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 0x7f264c0fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f26510d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 0x7f264c0fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49586/0 (socket says 172.21.15.7:49586) 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f26510d2640 1 -- 172.21.15.7:0/4046255531 learned_addr learned my addr 172.21.15.7:0/4046255531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f26518d3640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264c1007f0 0x7f264c0fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f26510d2640 1 -- 172.21.15.7:0/4046255531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264c1007f0 msgr2=0x7f264c0fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f26510d2640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264c1007f0 0x7f264c0fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f26510d2640 1 -- 172.21.15.7:0/4046255531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0fd7b0 con 0x7f264c1067f0 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f26510d2640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 0x7f264c0fd270 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f263c00c8f0 tx=0x7f263c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f2642ffd640 1 -- 172.21.15.7:0/4046255531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f263c0079b0 con 0x7f264c1067f0 2024-09-21T05:14:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f2642ffd640 1 -- 172.21.15.7:0/4046255531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f263c0054e0 con 0x7f264c1067f0 2024-09-21T05:14:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c0fdaa0 con 0x7f264c1067f0 2024-09-21T05:14:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.375+0000 7f2642ffd640 1 -- 172.21.15.7:0/4046255531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f263c004dd0 con 0x7f264c1067f0 2024-09-21T05:14:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.376+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c1a3e30 con 0x7f264c1067f0 2024-09-21T05:14:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.377+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c101f30 con 0x7f264c1067f0 2024-09-21T05:14:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.377+0000 7f2642ffd640 1 -- 172.21.15.7:0/4046255531 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f263c005020 con 0x7f264c1067f0 2024-09-21T05:14:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.378+0000 7f2642ffd640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f263807aed0 0x7f263807d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.378+0000 7f26518d3640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f263807aed0 0x7f263807d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.378+0000 7f2642ffd640 1 -- 172.21.15.7:0/4046255531 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f263c09df90 con 0x7f264c1067f0 2024-09-21T05:14:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.378+0000 7f26518d3640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f263807aed0 0x7f263807d390 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2648000c00 tx=0x7f2648031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.383+0000 7f2642ffd640 1 -- 172.21.15.7:0/4046255531 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f263c014030 con 0x7f264c1067f0 2024-09-21T05:14:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.586+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f264c062630 con 0x7f263807aed0 2024-09-21T05:14:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.587+0000 7f2642ffd640 1 -- 172.21.15.7:0/4046255531 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f264c062630 con 0x7f263807aed0 2024-09-21T05:14:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f263807aed0 msgr2=0x7f263807d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f263807aed0 0x7f263807d390 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2648000c00 tx=0x7f2648031400 comp rx=0 tx=0).stop 2024-09-21T05:14:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 msgr2=0x7f264c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 0x7f264c0fd270 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f263c00c8f0 tx=0x7f263c00cdb0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 shutdown_connections 2024-09-21T05:14:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f263807aed0 0x7f263807d390 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c1067f0 0x7f264c0fd270 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 --2- 172.21.15.7:0/4046255531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264c1007f0 0x7f264c0fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 >> 172.21.15.7:0/4046255531 conn(0x7f264c0fc550 msgr2=0x7f264c069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 shutdown_connections 2024-09-21T05:14:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.589+0000 7f2653b37640 1 -- 172.21.15.7:0/4046255531 wait complete. 2024-09-21T05:14:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.673+0000 7fc38a5de640 1 -- 172.21.15.7:0/3415795092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841007f0 msgr2=0x7fc384100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.673+0000 7fc38a5de640 1 --2- 172.21.15.7:0/3415795092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841007f0 0x7fc384100c50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc378009880 tx=0x7fc37802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.673+0000 7fc38a5de640 1 -- 172.21.15.7:0/3415795092 shutdown_connections 2024-09-21T05:14:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.673+0000 7fc38a5de640 1 --2- 172.21.15.7:0/3415795092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841007f0 0x7fc384100c50 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.673+0000 7fc38a5de640 1 --2- 172.21.15.7:0/3415795092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3841067f0 0x7fc384106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.673+0000 7fc38a5de640 1 -- 172.21.15.7:0/3415795092 >> 172.21.15.7:0/3415795092 conn(0x7fc3840fc550 msgr2=0x7fc3840fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.673+0000 7fc38a5de640 1 -- 172.21.15.7:0/3415795092 shutdown_connections 2024-09-21T05:14:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.673+0000 7fc38a5de640 1 -- 172.21.15.7:0/3415795092 wait complete. 2024-09-21T05:14:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.674+0000 7fc38a5de640 1 Processor -- start 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.674+0000 7fc38a5de640 1 -- start start 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.674+0000 7fc38a5de640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3841007f0 0x7fc3840fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.674+0000 7fc38a5de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841067f0 0x7fc3840fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.674+0000 7fc38a5de640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc38410dcc0 con 0x7fc3841067f0 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.674+0000 7fc38a5de640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc38410db40 con 0x7fc3841007f0 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841067f0 0x7fc3840fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841067f0 0x7fc3840fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49612/0 (socket says 172.21.15.7:49612) 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3837fe640 1 -- 172.21.15.7:0/2046955669 learned_addr learned my addr 172.21.15.7:0/2046955669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc383fff640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3841007f0 0x7fc3840fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3837fe640 1 -- 172.21.15.7:0/2046955669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3841007f0 msgr2=0x7fc3840fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3837fe640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3841007f0 0x7fc3840fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3837fe640 1 -- 172.21.15.7:0/2046955669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3840fd750 con 0x7fc3841067f0 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3837fe640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841067f0 0x7fc3840fd210 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc378031460 tx=0x7fc3780045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3817fa640 1 -- 172.21.15.7:0/2046955669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc37803f070 con 0x7fc3841067f0 2024-09-21T05:14:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc3817fa640 1 -- 172.21.15.7:0/2046955669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc3780035f0 con 0x7fc3841067f0 2024-09-21T05:14:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.675+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3840fd9e0 con 0x7fc3841067f0 2024-09-21T05:14:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.676+0000 7fc3817fa640 1 -- 172.21.15.7:0/2046955669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc378035030 con 0x7fc3841067f0 2024-09-21T05:14:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.676+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3841a3da0 con 0x7fc3841067f0 2024-09-21T05:14:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.677+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc384101f30 con 0x7fc3841067f0 2024-09-21T05:14:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.677+0000 7fc3817fa640 1 -- 172.21.15.7:0/2046955669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc378042070 con 0x7fc3841067f0 2024-09-21T05:14:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.678+0000 7fc3817fa640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc36c07af30 0x7fc36c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.678+0000 7fc3817fa640 1 -- 172.21.15.7:0/2046955669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc3780be9e0 con 0x7fc3841067f0 2024-09-21T05:14:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.678+0000 7fc383fff640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc36c07af30 0x7fc36c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.678+0000 7fc383fff640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc36c07af30 0x7fc36c07d3f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc374005fd0 tx=0x7fc37400b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.682+0000 7fc3817fa640 1 -- 172.21.15.7:0/2046955669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc378089f20 con 0x7fc3841067f0 2024-09-21T05:14:11.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:11 smithi053 ceph-mon[80176]: pgmap v1183: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.877+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc384062630 con 0x7fc36c07af30 2024-09-21T05:14:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.887+0000 7fc3817fa640 1 -- 172.21.15.7:0/2046955669 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fc384062630 con 0x7fc36c07af30 2024-09-21T05:14:11.891 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:14:11.891 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (36m) 14m ago 47m 19.7M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:14:11.891 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (39m) 14m ago 47m 10.2M - 19.1.1-373-g4e45e6b4 164465d2334a 5113758f706d 2024-09-21T05:14:11.891 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (38m) 14m ago 46m 10.0M - 19.1.1-373-g4e45e6b4 164465d2334a 83893d0b7d1c 2024-09-21T05:14:11.891 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (34m) 14m ago 47m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (34m) 14m ago 46m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (35m) 14m ago 46m 78.8M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (30m) 14m ago 41m 20.8M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (30m) 14m ago 41m 20.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (14m) 14m ago 41m 15.5M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (30m) 14m ago 41m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (38m) 14m ago 48m 534M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (37m) 14m ago 46m 463M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (34m) 14m ago 48m 71.0M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (34m) 14m ago 46m 51.4M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (37m) 14m ago 47m 12.1M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (37m) 14m ago 46m 10.6M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (33m) 14m ago 45m 94.3M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (33m) 14m ago 44m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:14:11.892 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (32m) 14m ago 44m 91.0M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:14:11.893 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (32m) 14m ago 43m 100M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:14:11.893 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (31m) 14m ago 43m 89.3M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:14:11.893 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (30m) 14m ago 42m 87.1M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:14:11.893 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (36m) 14m ago 46m 69.4M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:14:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc36c07af30 msgr2=0x7fc36c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc36c07af30 0x7fc36c07d3f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc374005fd0 tx=0x7fc37400b390 comp rx=0 tx=0).stop 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841067f0 msgr2=0x7fc3840fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841067f0 0x7fc3840fd210 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc378031460 tx=0x7fc3780045e0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 shutdown_connections 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc36c07af30 0x7fc36c07d3f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3841067f0 0x7fc3840fd210 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 --2- 172.21.15.7:0/2046955669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3841007f0 0x7fc3840fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 >> 172.21.15.7:0/2046955669 conn(0x7fc3840fc550 msgr2=0x7fc384069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 shutdown_connections 2024-09-21T05:14:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.890+0000 7fc38a5de640 1 -- 172.21.15.7:0/2046955669 wait complete. 2024-09-21T05:14:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.973+0000 7f8302e1d640 1 -- 172.21.15.7:0/3521425705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc107900 msgr2=0x7f82fc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.973+0000 7f8302e1d640 1 --2- 172.21.15.7:0/3521425705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc107900 0x7f82fc107ce0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009880 tx=0x7f82f002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.973+0000 7f8302e1d640 1 -- 172.21.15.7:0/3521425705 shutdown_connections 2024-09-21T05:14:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.973+0000 7f8302e1d640 1 --2- 172.21.15.7:0/3521425705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82fc101900 0x7f82fc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.973+0000 7f8302e1d640 1 --2- 172.21.15.7:0/3521425705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc107900 0x7f82fc107ce0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.973+0000 7f8302e1d640 1 -- 172.21.15.7:0/3521425705 >> 172.21.15.7:0/3521425705 conn(0x7f82fc0fd600 msgr2=0x7f82fc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.973+0000 7f8302e1d640 1 -- 172.21.15.7:0/3521425705 shutdown_connections 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.973+0000 7f8302e1d640 1 -- 172.21.15.7:0/3521425705 wait complete. 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8302e1d640 1 Processor -- start 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8302e1d640 1 -- start start 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8302e1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc101900 0x7f82fc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8302e1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82fc107900 0x7f82fc19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8302e1d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc10edb0 con 0x7f82fc101900 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8302e1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc10ec30 con 0x7f82fc107900 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8300bb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc101900 0x7f82fc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8300bb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc101900 0x7f82fc19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49632/0 (socket says 172.21.15.7:49632) 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f8300bb9640 1 -- 172.21.15.7:0/312632962 learned_addr learned my addr 172.21.15.7:0/312632962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.974+0000 7f82fbfff640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82fc107900 0x7f82fc19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f8300bb9640 1 -- 172.21.15.7:0/312632962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82fc107900 msgr2=0x7f82fc19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f8300bb9640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82fc107900 0x7f82fc19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f8300bb9640 1 -- 172.21.15.7:0/312632962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc1977c0 con 0x7f82fc101900 2024-09-21T05:14:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f8300bb9640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc101900 0x7f82fc19d640 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f82f0031460 tx=0x7f82f00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f82f9ffb640 1 -- 172.21.15.7:0/312632962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f82f003f070 con 0x7f82fc101900 2024-09-21T05:14:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f82f9ffb640 1 -- 172.21.15.7:0/312632962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f82f00035f0 con 0x7f82fc101900 2024-09-21T05:14:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc197a50 con 0x7f82fc101900 2024-09-21T05:14:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f82f9ffb640 1 -- 172.21.15.7:0/312632962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f82f0035030 con 0x7f82fc101900 2024-09-21T05:14:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.975+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc197f80 con 0x7f82fc101900 2024-09-21T05:14:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.976+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82fc103040 con 0x7f82fc101900 2024-09-21T05:14:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.977+0000 7f82f9ffb640 1 -- 172.21.15.7:0/312632962 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f82f0042070 con 0x7f82fc101900 2024-09-21T05:14:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.977+0000 7f82f9ffb640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f82ec07af80 0x7f82ec07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.978+0000 7f82f9ffb640 1 -- 172.21.15.7:0/312632962 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f82f00be8c0 con 0x7f82fc101900 2024-09-21T05:14:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.978+0000 7f82fbfff640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f82ec07af80 0x7f82ec07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.978+0000 7f82fbfff640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f82ec07af80 0x7f82ec07d440 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f82fc100e00 tx=0x7f82e800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:11.982+0000 7f82f9ffb640 1 -- 172.21.15.7:0/312632962 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82f0089ee0 con 0x7f82fc101900 2024-09-21T05:14:12.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.247+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f82fc104b10 con 0x7f82fc101900 2024-09-21T05:14:12.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.247+0000 7f82f9ffb640 1 -- 172.21.15.7:0/312632962 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f82f008ed90 con 0x7f82fc101900 2024-09-21T05:14:12.251 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:14:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:14:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:14:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:14:12.252 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f82ec07af80 msgr2=0x7f82ec07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f82ec07af80 0x7f82ec07d440 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f82fc100e00 tx=0x7f82e800b390 comp rx=0 tx=0).stop 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc101900 msgr2=0x7f82fc19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc101900 0x7f82fc19d640 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f82f0031460 tx=0x7f82f00045e0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 shutdown_connections 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f82ec07af80 0x7f82ec07d440 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82fc107900 0x7f82fc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 --2- 172.21.15.7:0/312632962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82fc101900 0x7f82fc19d640 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 >> 172.21.15.7:0/312632962 conn(0x7f82fc0fd600 msgr2=0x7f82fc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 shutdown_connections 2024-09-21T05:14:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.250+0000 7f8302e1d640 1 -- 172.21.15.7:0/312632962 wait complete. 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.334+0000 7fe016696640 1 -- 172.21.15.7:0/367542100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010101010 msgr2=0x7fe010101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.334+0000 7fe016696640 1 --2- 172.21.15.7:0/367542100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010101010 0x7fe010101450 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe004009880 tx=0x7fe00402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.334+0000 7fe016696640 1 -- 172.21.15.7:0/367542100 shutdown_connections 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.334+0000 7fe016696640 1 --2- 172.21.15.7:0/367542100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010101010 0x7fe010101450 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.334+0000 7fe016696640 1 --2- 172.21.15.7:0/367542100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe010105860 0x7fe010105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.334+0000 7fe016696640 1 -- 172.21.15.7:0/367542100 >> 172.21.15.7:0/367542100 conn(0x7fe0100fbc90 msgr2=0x7fe0100fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.335+0000 7fe016696640 1 -- 172.21.15.7:0/367542100 shutdown_connections 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.335+0000 7fe016696640 1 -- 172.21.15.7:0/367542100 wait complete. 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.335+0000 7fe016696640 1 Processor -- start 2024-09-21T05:14:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.335+0000 7fe016696640 1 -- start start 2024-09-21T05:14:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe016696640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe010101010 0x7fe01019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe016696640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010105860 0x7fe01019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe016696640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe01010de20 con 0x7fe010105860 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe016696640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe01010dca0 con 0x7fe010101010 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010105860 0x7fe01019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010105860 0x7fe01019dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49664/0 (socket says 172.21.15.7:49664) 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00f7fe640 1 -- 172.21.15.7:0/1243566086 learned_addr learned my addr 172.21.15.7:0/1243566086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00ffff640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe010101010 0x7fe01019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00f7fe640 1 -- 172.21.15.7:0/1243566086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe010101010 msgr2=0x7fe01019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00f7fe640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe010101010 0x7fe01019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00f7fe640 1 -- 172.21.15.7:0/1243566086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0101978e0 con 0x7fe010105860 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00f7fe640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010105860 0x7fe01019dc60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe004031460 tx=0x7fe0040045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00d7fa640 1 -- 172.21.15.7:0/1243566086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe00403f070 con 0x7fe010105860 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00d7fa640 1 -- 172.21.15.7:0/1243566086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe0040035f0 con 0x7fe010105860 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe010197b70 con 0x7fe010105860 2024-09-21T05:14:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.336+0000 7fe00d7fa640 1 -- 172.21.15.7:0/1243566086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe004035030 con 0x7fe010105860 2024-09-21T05:14:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.337+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0101980a0 con 0x7fe010105860 2024-09-21T05:14:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.337+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe010102020 con 0x7fe010105860 2024-09-21T05:14:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.338+0000 7fe00d7fa640 1 -- 172.21.15.7:0/1243566086 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe004042070 con 0x7fe010105860 2024-09-21T05:14:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.339+0000 7fe00d7fa640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.339+0000 7fe00d7fa640 1 -- 172.21.15.7:0/1243566086 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fe0040bea80 con 0x7fe010105860 2024-09-21T05:14:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.339+0000 7fe00ffff640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.339+0000 7fe00ffff640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe000005fd0 tx=0x7fe000005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.343+0000 7fe00d7fa640 1 -- 172.21.15.7:0/1243566086 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe00408a000 con 0x7fe010105860 2024-09-21T05:14:12.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.567+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe010109d90 con 0x7fe010105860 2024-09-21T05:14:12.571 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:12 smithi007 ceph-mon[108238]: from='client.35936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:12.571 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:12 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/312632962' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:12.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.567+0000 7fe00d7fa640 1 -- 172.21.15.7:0/1243566086 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+1994 (secure 0 0 0) 0x7fe00408eeb0 con 0x7fe010105860 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:e30 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T04:59:15:494001+0000 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:14:12.572 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T04:43:25.693123+0000 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-09-21T05:14:12.573 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{-1:34354} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:14:12.574 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{-1:34358} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:14:12.575 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:14:12.575 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{-1:44909} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:14:12.575 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:14:12.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.569+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 msgr2=0x7fdff807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.569+0000 7fe016696640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe000005fd0 tx=0x7fe000005850 comp rx=0 tx=0).stop 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.569+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010105860 msgr2=0x7fe01019dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.569+0000 7fe016696640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010105860 0x7fe01019dc60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe004031460 tx=0x7fe0040045e0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.570+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 shutdown_connections 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.570+0000 7fe016696640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fdff807af80 0x7fdff807d440 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.570+0000 7fe016696640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe010105860 0x7fe01019dc60 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.570+0000 7fe016696640 1 --2- 172.21.15.7:0/1243566086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe010101010 0x7fe01019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.570+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 >> 172.21.15.7:0/1243566086 conn(0x7fe0100fbc90 msgr2=0x7fe0101001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.570+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 shutdown_connections 2024-09-21T05:14:12.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.570+0000 7fe016696640 1 -- 172.21.15.7:0/1243566086 wait complete. 2024-09-21T05:14:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.654+0000 7f2bd4add640 1 -- 172.21.15.7:0/4189134369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd01033d0 msgr2=0x7f2bd010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.654+0000 7f2bd4add640 1 --2- 172.21.15.7:0/4189134369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd01033d0 0x7f2bd010b8c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009880 tx=0x7f2bb802a530 comp rx=0 tx=0).stop 2024-09-21T05:14:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.655+0000 7f2bd4add640 1 -- 172.21.15.7:0/4189134369 shutdown_connections 2024-09-21T05:14:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.655+0000 7f2bd4add640 1 --2- 172.21.15.7:0/4189134369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd01033d0 0x7f2bd010b8c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.655+0000 7f2bd4add640 1 --2- 172.21.15.7:0/4189134369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd0102ab0 0x7f2bd0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.655+0000 7f2bd4add640 1 -- 172.21.15.7:0/4189134369 >> 172.21.15.7:0/4189134369 conn(0x7f2bd00fc550 msgr2=0x7f2bd00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.655+0000 7f2bd4add640 1 -- 172.21.15.7:0/4189134369 shutdown_connections 2024-09-21T05:14:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.655+0000 7f2bd4add640 1 -- 172.21.15.7:0/4189134369 wait complete. 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bd4add640 1 Processor -- start 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bd4add640 1 -- start start 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bd4add640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd0102ab0 0x7f2bd0100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bd4add640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd01033d0 0x7f2bd0100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bd4add640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd010ede0 con 0x7f2bd0102ab0 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bd4add640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd010ec60 con 0x7f2bd01033d0 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bce59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd0102ab0 0x7f2bd0100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bce59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd0102ab0 0x7f2bd0100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49692/0 (socket says 172.21.15.7:49692) 2024-09-21T05:14:12.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bce59c640 1 -- 172.21.15.7:0/2828869395 learned_addr learned my addr 172.21.15.7:0/2828869395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bce59c640 1 -- 172.21.15.7:0/2828869395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd01033d0 msgr2=0x7f2bd0100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bcdd9b640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd01033d0 0x7f2bd0100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bce59c640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd01033d0 0x7f2bd0100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.656+0000 7f2bce59c640 1 -- 172.21.15.7:0/2828869395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd0101190 con 0x7f2bd0102ab0 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.657+0000 7f2bce59c640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd0102ab0 0x7f2bd0100710 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800cc60 tx=0x7f2bc800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.657+0000 7f2bc77fe640 1 -- 172.21.15.7:0/2828869395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bc800ad30 con 0x7f2bd0102ab0 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.657+0000 7f2bc77fe640 1 -- 172.21.15.7:0/2828869395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bc8005020 con 0x7f2bd0102ab0 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.657+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd0101480 con 0x7f2bd0102ab0 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.657+0000 7f2bc77fe640 1 -- 172.21.15.7:0/2828869395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bc800f640 con 0x7f2bd0102ab0 2024-09-21T05:14:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.657+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01a3e90 con 0x7f2bd0102ab0 2024-09-21T05:14:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.658+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd006a580 con 0x7f2bd0102ab0 2024-09-21T05:14:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.659+0000 7f2bc77fe640 1 -- 172.21.15.7:0/2828869395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2bc80051d0 con 0x7f2bd0102ab0 2024-09-21T05:14:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.659+0000 7f2bc77fe640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bbc07af40 0x7f2bbc07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.659+0000 7f2bc77fe640 1 -- 172.21.15.7:0/2828869395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f2bc809e360 con 0x7f2bd0102ab0 2024-09-21T05:14:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.659+0000 7f2bcdd9b640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bbc07af40 0x7f2bbc07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.660+0000 7f2bcdd9b640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bbc07af40 0x7f2bbc07d400 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80096c0 tx=0x7f2bb8000bc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:12.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.664+0000 7f2bc77fe640 1 -- 172.21.15.7:0/2828869395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2bc8014030 con 0x7f2bd0102ab0 2024-09-21T05:14:12.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:12 smithi053 ceph-mon[80176]: from='client.35936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:12.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:12 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/312632962' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:12.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.868+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2bd0105fc0 con 0x7f2bbc07af40 2024-09-21T05:14:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.869+0000 7f2bc77fe640 1 -- 172.21.15.7:0/2828869395 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f2bb8004340 con 0x7f2bbc07af40 2024-09-21T05:14:12.873 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:14:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00", 2024-09-21T05:14:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-09-21T05:14:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-21T05:14:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-09-21T05:14:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-09-21T05:14:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds", 2024-09-21T05:14:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-09-21T05:14:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-21T05:14:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-09-21T05:14:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-21T05:14:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "16/23 daemons upgraded", 2024-09-21T05:14:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-09-21T05:14:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:14:12.874 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:14:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.871+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bbc07af40 msgr2=0x7f2bbc07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.871+0000 7f2bd4add640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bbc07af40 0x7f2bbc07d400 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80096c0 tx=0x7f2bb8000bc0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.871+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd0102ab0 msgr2=0x7f2bd0100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.871+0000 7f2bd4add640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd0102ab0 0x7f2bd0100710 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800cc60 tx=0x7f2bc800a620 comp rx=0 tx=0).stop 2024-09-21T05:14:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.872+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 shutdown_connections 2024-09-21T05:14:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.872+0000 7f2bd4add640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2bbc07af40 0x7f2bbc07d400 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.872+0000 7f2bd4add640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd01033d0 0x7f2bd0100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.872+0000 7f2bd4add640 1 --2- 172.21.15.7:0/2828869395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd0102ab0 0x7f2bd0100710 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.872+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 >> 172.21.15.7:0/2828869395 conn(0x7f2bd00fc550 msgr2=0x7f2bd0108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.872+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 shutdown_connections 2024-09-21T05:14:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.872+0000 7f2bd4add640 1 -- 172.21.15.7:0/2828869395 wait complete. 2024-09-21T05:14:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.955+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/1299059023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981011d0 msgr2=0x7f2a981056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.955+0000 7f2a9f4c6640 1 --2- 172.21.15.7:0/1299059023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981011d0 0x7f2a981056a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009880 tx=0x7f2a8c02a530 comp rx=0 tx=0).stop 2024-09-21T05:14:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.955+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/1299059023 shutdown_connections 2024-09-21T05:14:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.955+0000 7f2a9f4c6640 1 --2- 172.21.15.7:0/1299059023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981011d0 0x7f2a981056a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.955+0000 7f2a9f4c6640 1 --2- 172.21.15.7:0/1299059023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a981008b0 0x7f2a98100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.955+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/1299059023 >> 172.21.15.7:0/1299059023 conn(0x7f2a980fc520 msgr2=0x7f2a980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:12.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.955+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/1299059023 shutdown_connections 2024-09-21T05:14:12.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.956+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/1299059023 wait complete. 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.956+0000 7f2a9f4c6640 1 Processor -- start 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.956+0000 7f2a9f4c6640 1 -- start start 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.956+0000 7f2a9f4c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981008b0 0x7f2a9819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.956+0000 7f2a9f4c6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a981011d0 0x7f2a9819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.956+0000 7f2a9f4c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98115450 con 0x7f2a981008b0 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.956+0000 7f2a9f4c6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981152d0 con 0x7f2a981011d0 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9d262640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981008b0 0x7f2a9819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9d262640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981008b0 0x7f2a9819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49712/0 (socket says 172.21.15.7:49712) 2024-09-21T05:14:12.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9d262640 1 -- 172.21.15.7:0/2770683938 learned_addr learned my addr 172.21.15.7:0/2770683938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9ca61640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a981011d0 0x7f2a9819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9d262640 1 -- 172.21.15.7:0/2770683938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a981011d0 msgr2=0x7f2a9819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9d262640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a981011d0 0x7f2a9819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9d262640 1 -- 172.21.15.7:0/2770683938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a981a28e0 con 0x7f2a981008b0 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9d262640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981008b0 0x7f2a9819e010 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400b420 tx=0x7f2a9400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a8a7fc640 1 -- 172.21.15.7:0/2770683938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a94011630 con 0x7f2a981008b0 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a8a7fc640 1 -- 172.21.15.7:0/2770683938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a94016070 con 0x7f2a981008b0 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a981a2bd0 con 0x7f2a981008b0 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a8a7fc640 1 -- 172.21.15.7:0/2770683938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a94010d40 con 0x7f2a981008b0 2024-09-21T05:14:12.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.957+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a981a3160 con 0x7f2a981008b0 2024-09-21T05:14:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.958+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9806aa90 con 0x7f2a981008b0 2024-09-21T05:14:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.959+0000 7f2a8a7fc640 1 -- 172.21.15.7:0/2770683938 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f2a9401b460 con 0x7f2a981008b0 2024-09-21T05:14:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.960+0000 7f2a8a7fc640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a8407af30 0x7f2a8407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.960+0000 7f2a8a7fc640 1 -- 172.21.15.7:0/2770683938 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f2a9409d7c0 con 0x7f2a981008b0 2024-09-21T05:14:12.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.960+0000 7f2a9ca61640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a8407af30 0x7f2a8407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:12.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.960+0000 7f2a9ca61640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a8407af30 0x7f2a8407d3f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2a981124f0 tx=0x7f2a8c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:12.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:12.964+0000 7f2a8a7fc640 1 -- 172.21.15.7:0/2770683938 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2a94068cc0 con 0x7f2a981008b0 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.227+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f2a9819f370 con 0x7f2a981008b0 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.227+0000 7f2a8a7fc640 1 -- 172.21.15.7:0/2770683938 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f2a9406db70 con 0x7f2a981008b0 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: host smithi053 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi053/config 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:Traceback (most recent call last): 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-21T05:14:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: return _run_code(code, main_globals, None, 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: exec(code, run_globals) 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5628, in 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 5616, in main 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: File "/tmp/tmprqjgn1a7.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 -e NODE_NAME=smithi053 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/run/ceph:z -v /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490:/var/log/ceph:z -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpbq4xkfqt:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 inventory --format=json-pretty --filter-for-batch 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:14:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:14:13.233 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-21T05:14:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-21T05:14:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a8407af30 msgr2=0x7f2a8407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a8407af30 0x7f2a8407d3f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2a981124f0 tx=0x7f2a8c031400 comp rx=0 tx=0).stop 2024-09-21T05:14:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981008b0 msgr2=0x7f2a9819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981008b0 0x7f2a9819e010 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400b420 tx=0x7f2a9400b8e0 comp rx=0 tx=0).stop 2024-09-21T05:14:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 shutdown_connections 2024-09-21T05:14:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f2a8407af30 0x7f2a8407d3f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a981011d0 0x7f2a9819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 --2- 172.21.15.7:0/2770683938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981008b0 0x7f2a9819e010 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 >> 172.21.15.7:0/2770683938 conn(0x7f2a980fc520 msgr2=0x7f2a98112bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 shutdown_connections 2024-09-21T05:14:13.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:13.229+0000 7f2a9f4c6640 1 -- 172.21.15.7:0/2770683938 wait complete. 2024-09-21T05:14:13.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:13 smithi053 ceph-mon[80176]: pgmap v1184: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:13 smithi053 ceph-mon[80176]: from='client.35940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:13 smithi053 ceph-mon[80176]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:13 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1243566086' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:14:13.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:13 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2770683938' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:14:13.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:13 smithi007 ceph-mon[108238]: pgmap v1184: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:13.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:13 smithi007 ceph-mon[108238]: from='client.35940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:13.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:13 smithi007 ceph-mon[108238]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:13.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:13 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1243566086' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:14:13.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:13 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2770683938' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:14:14.594 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:14 smithi053 ceph-mon[80176]: from='client.35956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:14.594 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:14:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:14 smithi007 ceph-mon[108238]: from='client.35956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:14:15.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:15 smithi007 ceph-mon[108238]: pgmap v1185: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:15.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:15 smithi053 ceph-mon[80176]: pgmap v1185: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.eyygpg"} : dispatch 2024-09-21T05:14:16.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.eyygpg"}]': finished 2024-09-21T05:14:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.oxcnqk"} : dispatch 2024-09-21T05:14:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.oxcnqk"}]': finished 2024-09-21T05:14:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi053.jvuwme"} : dispatch 2024-09-21T05:14:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi053.jvuwme"}]': finished 2024-09-21T05:14:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi053.wdnfat"} : dispatch 2024-09-21T05:14:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi053.wdnfat"}]': finished 2024-09-21T05:14:16.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:16 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} : dispatch 2024-09-21T05:14:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.eyygpg"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.eyygpg"}]': finished 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.oxcnqk"} : dispatch 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.oxcnqk"}]': finished 2024-09-21T05:14:16.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi053.jvuwme"} : dispatch 2024-09-21T05:14:16.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi053.jvuwme"}]': finished 2024-09-21T05:14:16.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi053.wdnfat"} : dispatch 2024-09-21T05:14:16.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi053.wdnfat"}]': finished 2024-09-21T05:14:16.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:16 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} : dispatch 2024-09-21T05:14:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: pgmap v1186: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: pgmap v1187: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: pgmap v1188: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all mds 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: Upgrade: Setting filesystem cephfs Joinable 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: Standby daemon mds.cephfs.smithi007.oxcnqk assigned to filesystem cephfs as rank 0 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: Standby daemon mds.cephfs.smithi007.eyygpg assigned to filesystem cephfs as rank 1 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:replay,1=cephfs.smithi007.eyygpg=up:replay} 2 up:standby 2024-09-21T05:14:17.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:17 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"} : dispatch 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: pgmap v1186: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: pgmap v1187: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: pgmap v1188: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all mds 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: Upgrade: Setting filesystem cephfs Joinable 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: Standby daemon mds.cephfs.smithi007.oxcnqk assigned to filesystem cephfs as rank 0 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: Standby daemon mds.cephfs.smithi007.eyygpg assigned to filesystem cephfs as rank 1 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T05:14:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-21T05:14:17.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:replay,1=cephfs.smithi007.eyygpg=up:replay} 2 up:standby 2024-09-21T05:14:17.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:17 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"} : dispatch 2024-09-21T05:14:18.443 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-09-21T05:14:18.443 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-09-21T05:14:18.443 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:resolve 2024-09-21T05:14:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:replay,1=cephfs.smithi007.eyygpg=up:resolve} 2 up:standby 2024-09-21T05:14:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:18.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:18 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:resolve 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:replay,1=cephfs.smithi007.eyygpg=up:resolve} 2 up:standby 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:18.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:18 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:19 smithi007 ceph-mon[108238]: pgmap v1189: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 507 KiB/s rd, 0 op/s 2024-09-21T05:14:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:19 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all rgw 2024-09-21T05:14:19.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:19 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all rbd-mirror 2024-09-21T05:14:19.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:19 smithi053 ceph-mon[80176]: pgmap v1189: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 507 KiB/s rd, 0 op/s 2024-09-21T05:14:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:19 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all rgw 2024-09-21T05:14:19.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:19 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all rbd-mirror 2024-09-21T05:14:20.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:20 smithi053 ceph-mon[80176]: Upgrade: Updating ceph-exporter.smithi007 (1/2) 2024-09-21T05:14:20.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:20 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:20.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:20 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T05:14:20.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:20 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:20.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:20 smithi053 ceph-mon[80176]: Deploying daemon ceph-exporter.smithi007 on smithi007 2024-09-21T05:14:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:20 smithi007 ceph-mon[108238]: Upgrade: Updating ceph-exporter.smithi007 (1/2) 2024-09-21T05:14:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:20 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:20 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi007", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T05:14:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:20 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:20 smithi007 ceph-mon[108238]: Deploying daemon ceph-exporter.smithi007 on smithi007 2024-09-21T05:14:21.432 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:21 smithi007 ceph-mon[108238]: pgmap v1190: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 4 op/s 2024-09-21T05:14:21.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:21 smithi053 ceph-mon[80176]: pgmap v1190: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 4 op/s 2024-09-21T05:14:23.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:23 smithi053 ceph-mon[80176]: pgmap v1191: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 6 op/s 2024-09-21T05:14:23.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:23 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:23.606 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:23 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:23 smithi007 ceph-mon[108238]: pgmap v1191: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 6 op/s 2024-09-21T05:14:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:23 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:23 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:25.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:25 smithi053 ceph-mon[80176]: pgmap v1192: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 8 op/s 2024-09-21T05:14:25.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:25.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T05:14:25.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:25 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:25.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:25 smithi007 ceph-mon[108238]: pgmap v1192: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 8 op/s 2024-09-21T05:14:25.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:25.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi053", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-21T05:14:25.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:25 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:26.619 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:26 smithi053 ceph-mon[80176]: Upgrade: Updating ceph-exporter.smithi053 (2/2) 2024-09-21T05:14:26.619 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:26 smithi053 ceph-mon[80176]: Deploying daemon ceph-exporter.smithi053 on smithi053 2024-09-21T05:14:26.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:26 smithi007 ceph-mon[108238]: Upgrade: Updating ceph-exporter.smithi053 (2/2) 2024-09-21T05:14:26.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:26 smithi007 ceph-mon[108238]: Deploying daemon ceph-exporter.smithi053 on smithi053 2024-09-21T05:14:27.676 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:27 smithi053 ceph-mon[80176]: pgmap v1193: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-09-21T05:14:27.677 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:27 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:resolve 2024-09-21T05:14:27.677 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:27 smithi053 ceph-mon[80176]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:resolve,1=cephfs.smithi007.eyygpg=up:resolve} 2 up:standby 2024-09-21T05:14:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:27 smithi007 ceph-mon[108238]: pgmap v1193: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-09-21T05:14:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:27 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:resolve 2024-09-21T05:14:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:27 smithi007 ceph-mon[108238]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:resolve,1=cephfs.smithi007.eyygpg=up:resolve} 2 up:standby 2024-09-21T05:14:28.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:28 smithi007 ceph-mon[108238]: reconnect by client.14614 192.168.0.1:0/1658045521 after 0.000999985 2024-09-21T05:14:28.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:28 smithi007 ceph-mon[108238]: reconnect by client.24375 192.168.0.1:0/3970860440 after 0.000999985 2024-09-21T05:14:28.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:28 smithi007 ceph-mon[108238]: reconnect by client.24375 192.168.0.1:0/3970860440 after 0.000999985 2024-09-21T05:14:28.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:28 smithi007 ceph-mon[108238]: reconnect by client.14614 192.168.0.1:0/1658045521 after 0.000999985 2024-09-21T05:14:28.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:28 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:reconnect 2024-09-21T05:14:28.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:28 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:reconnect 2024-09-21T05:14:28.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:28 smithi007 ceph-mon[108238]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:reconnect,1=cephfs.smithi007.eyygpg=up:reconnect} 2 up:standby 2024-09-21T05:14:28.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:28 smithi053 ceph-mon[80176]: reconnect by client.14614 192.168.0.1:0/1658045521 after 0.000999985 2024-09-21T05:14:28.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:28 smithi053 ceph-mon[80176]: reconnect by client.24375 192.168.0.1:0/3970860440 after 0.000999985 2024-09-21T05:14:28.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:28 smithi053 ceph-mon[80176]: reconnect by client.24375 192.168.0.1:0/3970860440 after 0.000999985 2024-09-21T05:14:28.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:28 smithi053 ceph-mon[80176]: reconnect by client.14614 192.168.0.1:0/1658045521 after 0.000999985 2024-09-21T05:14:28.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:28 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:reconnect 2024-09-21T05:14:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:28 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:reconnect 2024-09-21T05:14:28.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:28 smithi053 ceph-mon[80176]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:reconnect,1=cephfs.smithi007.eyygpg=up:reconnect} 2 up:standby 2024-09-21T05:14:29.427 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: pgmap v1194: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 7 op/s 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:rejoin 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:rejoin 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:rejoin,1=cephfs.smithi007.eyygpg=up:rejoin} 2 up:standby 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: daemon mds.cephfs.smithi007.eyygpg is now active in filesystem cephfs as rank 1 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: daemon mds.cephfs.smithi007.oxcnqk is now active in filesystem cephfs as rank 0 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.428 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.429 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.429 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.429 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T05:14:29.429 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:active 2024-09-21T05:14:29.429 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:active 2024-09-21T05:14:29.429 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:29 smithi053 ceph-mon[80176]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi007.eyygpg=up:active} 2 up:standby-replay 2024-09-21T05:14:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: pgmap v1194: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 7 op/s 2024-09-21T05:14:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:rejoin 2024-09-21T05:14:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:rejoin 2024-09-21T05:14:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: fsmap cephfs:2/2 {0=cephfs.smithi007.oxcnqk=up:rejoin,1=cephfs.smithi007.eyygpg=up:rejoin} 2 up:standby 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: daemon mds.cephfs.smithi007.eyygpg is now active in filesystem cephfs as rank 1 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: daemon mds.cephfs.smithi007.oxcnqk is now active in filesystem cephfs as rank 0 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T05:14:29.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] up:active 2024-09-21T05:14:29.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: mds.? [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] up:active 2024-09-21T05:14:29.696 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:29 smithi007 ceph-mon[108238]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi007.eyygpg=up:active} 2 up:standby-replay 2024-09-21T05:14:31.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:31 smithi053 ceph-mon[80176]: pgmap v1195: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 85 B/s wr, 11 op/s 2024-09-21T05:14:31.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:31 smithi007 ceph-mon[108238]: pgmap v1195: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 85 B/s wr, 11 op/s 2024-09-21T05:14:33.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:33 smithi053 ceph-mon[80176]: pgmap v1196: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T05:14:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:33 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:33 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:33 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:33.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:33 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:33 smithi007 ceph-mon[108238]: pgmap v1196: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T05:14:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:33 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:33 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:33 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:33.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:33 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:35.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:35 smithi053 ceph-mon[80176]: pgmap v1197: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-09-21T05:14:35.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:35 smithi007 ceph-mon[108238]: pgmap v1197: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-09-21T05:14:37.604 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:37 smithi007 ceph-mon[108238]: pgmap v1198: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-21T05:14:37.604 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:37 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.605 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:37 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.605 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:37 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.605 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:37 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.605 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:37 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.605 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:37 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:37 smithi053 ceph-mon[80176]: pgmap v1198: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-21T05:14:37.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:37 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:37 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:37 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:37 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:37 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:37.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:37 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:39.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:39 smithi053 ceph-mon[80176]: pgmap v1199: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-09-21T05:14:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:39 smithi007 ceph-mon[108238]: pgmap v1199: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.7 KiB/s wr, 18 op/s 2024-09-21T05:14:41.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: pgmap v1200: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: pgmap v1201: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 5.1 KiB/s wr, 17 op/s 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:41 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} : dispatch 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: pgmap v1200: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: pgmap v1201: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 5.1 KiB/s wr, 17 op/s 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:41.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:41 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} : dispatch 2024-09-21T05:14:42.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: Upgrade: Setting filesystem cephfs Joinable 2024-09-21T05:14:42.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-21T05:14:42.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-21T05:14:42.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi007.eyygpg=up:active} 2 up:standby-replay 2024-09-21T05:14:42.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all ceph-exporter 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi007"} : dispatch 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi007"}]': finished 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi053"} : dispatch 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi053"}]': finished 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all iscsi 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all nfs 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: Upgrade: Setting container_image for all nvmeof 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: Upgrade: Finalizing container_image settings 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mgr"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.695 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash"} : dispatch 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd"} : dispatch 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds"} : dispatch 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rgw"} : dispatch 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"} : dispatch 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"} : dispatch 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.iscsi"} : dispatch 2024-09-21T05:14:42.696 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nfs"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:14:42.697 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.698 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:42.698 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:42.698 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:14:42.698 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.698 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: Upgrade: Setting filesystem cephfs Joinable 2024-09-21T05:14:42.698 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-21T05:14:42.698 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-21T05:14:42.698 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: fsmap cephfs:2 {0=cephfs.smithi007.oxcnqk=up:active,1=cephfs.smithi007.eyygpg=up:active} 2 up:standby-replay 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all ceph-exporter 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi007"} : dispatch 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi007"}]': finished 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi053"} : dispatch 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi053"}]': finished 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all iscsi 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.699 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all nfs 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: Upgrade: Setting container_image for all nvmeof 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: Upgrade: Finalizing container_image settings 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mgr"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.700 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash"} : dispatch 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "osd"} : dispatch 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mds"} : dispatch 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rgw"} : dispatch 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"} : dispatch 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"} : dispatch 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-21T05:14:42.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.iscsi"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nfs"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"} : dispatch 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-21T05:14:42.702 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:42.703 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:42.703 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:14:42.703 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:42.703 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:14:42.703 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:14:42.703 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:14:42.703 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:43.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.327+0000 7fa57f6a3640 1 -- 172.21.15.7:0/671628665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781067f0 msgr2=0x7fa578106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.327+0000 7fa57f6a3640 1 --2- 172.21.15.7:0/671628665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781067f0 0x7fa578106bd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa574009880 tx=0x7fa57402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:14:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.327+0000 7fa57f6a3640 1 -- 172.21.15.7:0/671628665 shutdown_connections 2024-09-21T05:14:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.327+0000 7fa57f6a3640 1 --2- 172.21.15.7:0/671628665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5781007f0 0x7fa578100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.327+0000 7fa57f6a3640 1 --2- 172.21.15.7:0/671628665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781067f0 0x7fa578106bd0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.327+0000 7fa57f6a3640 1 -- 172.21.15.7:0/671628665 >> 172.21.15.7:0/671628665 conn(0x7fa5780fc550 msgr2=0x7fa5780fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.328+0000 7fa57f6a3640 1 -- 172.21.15.7:0/671628665 shutdown_connections 2024-09-21T05:14:43.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.328+0000 7fa57f6a3640 1 -- 172.21.15.7:0/671628665 wait complete. 2024-09-21T05:14:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.328+0000 7fa57f6a3640 1 Processor -- start 2024-09-21T05:14:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.328+0000 7fa57f6a3640 1 -- start start 2024-09-21T05:14:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57f6a3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781007f0 0x7fa57819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57f6a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5781067f0 0x7fa57819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57f6a3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa57810dcc0 con 0x7fa5781007f0 2024-09-21T05:14:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57f6a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa57810db40 con 0x7fa5781067f0 2024-09-21T05:14:43.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57d43f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781007f0 0x7fa57819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57d43f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781007f0 0x7fa57819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50486/0 (socket says 172.21.15.7:50486) 2024-09-21T05:14:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57d43f640 1 -- 172.21.15.7:0/2270190255 learned_addr learned my addr 172.21.15.7:0/2270190255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57cc3e640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5781067f0 0x7fa57819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57d43f640 1 -- 172.21.15.7:0/2270190255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5781067f0 msgr2=0x7fa57819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57d43f640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5781067f0 0x7fa57819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.329+0000 7fa57d43f640 1 -- 172.21.15.7:0/2270190255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5781977e0 con 0x7fa5781007f0 2024-09-21T05:14:43.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.330+0000 7fa57d43f640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781007f0 0x7fa57819d660 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa574031460 tx=0x7fa5740045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.331+0000 7fa56e7fc640 1 -- 172.21.15.7:0/2270190255 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa57403f070 con 0x7fa5781007f0 2024-09-21T05:14:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.331+0000 7fa56e7fc640 1 -- 172.21.15.7:0/2270190255 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa5740035f0 con 0x7fa5781007f0 2024-09-21T05:14:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.331+0000 7fa56e7fc640 1 -- 172.21.15.7:0/2270190255 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa574035030 con 0x7fa5781007f0 2024-09-21T05:14:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.331+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa578197a70 con 0x7fa5781007f0 2024-09-21T05:14:43.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.331+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa578197fa0 con 0x7fa5781007f0 2024-09-21T05:14:43.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.333+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa578101f30 con 0x7fa5781007f0 2024-09-21T05:14:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.333+0000 7fa56e7fc640 1 -- 172.21.15.7:0/2270190255 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fa574042070 con 0x7fa5781007f0 2024-09-21T05:14:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.334+0000 7fa56e7fc640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa56407af80 0x7fa56407d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.334+0000 7fa56e7fc640 1 -- 172.21.15.7:0/2270190255 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fa5740be8c0 con 0x7fa5781007f0 2024-09-21T05:14:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.334+0000 7fa57cc3e640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa56407af80 0x7fa56407d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.335+0000 7fa57cc3e640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa56407af80 0x7fa56407d440 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa578102380 tx=0x7fa56800b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:43.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.340+0000 7fa56e7fc640 1 -- 172.21.15.7:0/2270190255 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa574089ee0 con 0x7fa5781007f0 2024-09-21T05:14:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.553+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa578198310 con 0x7fa56407af80 2024-09-21T05:14:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.554+0000 7fa56e7fc640 1 -- 172.21.15.7:0/2270190255 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+175 (secure 0 0 0) 0x7fa578198310 con 0x7fa56407af80 2024-09-21T05:14:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.556+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa56407af80 msgr2=0x7fa56407d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.556+0000 7fa57f6a3640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa56407af80 0x7fa56407d440 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa578102380 tx=0x7fa56800b390 comp rx=0 tx=0).stop 2024-09-21T05:14:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.556+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781007f0 msgr2=0x7fa57819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.556+0000 7fa57f6a3640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781007f0 0x7fa57819d660 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa574031460 tx=0x7fa5740045e0 comp rx=0 tx=0).stop 2024-09-21T05:14:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.556+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 shutdown_connections 2024-09-21T05:14:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.557+0000 7fa57f6a3640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fa56407af80 0x7fa56407d440 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.557+0000 7fa57f6a3640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5781067f0 0x7fa57819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.557+0000 7fa57f6a3640 1 --2- 172.21.15.7:0/2270190255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5781007f0 0x7fa57819d660 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.557+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 >> 172.21.15.7:0/2270190255 conn(0x7fa5780fc550 msgr2=0x7fa578069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.557+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 shutdown_connections 2024-09-21T05:14:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:43.557+0000 7fa57f6a3640 1 -- 172.21.15.7:0/2270190255 wait complete. 2024-09-21T05:14:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:43 smithi053 ceph-mon[80176]: Upgrade: Complete! 2024-09-21T05:14:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:43 smithi053 ceph-mon[80176]: pgmap v1202: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 5.0 KiB/s wr, 13 op/s 2024-09-21T05:14:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:43 smithi007 ceph-mon[108238]: Upgrade: Complete! 2024-09-21T05:14:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:43 smithi007 ceph-mon[108238]: pgmap v1202: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 5.0 KiB/s wr, 13 op/s 2024-09-21T05:14:44.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch ps' 2024-09-21T05:14:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:14:44.770 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:44.770 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:14:44.770 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:44 smithi007 ceph-mon[108238]: from='client.35968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:44.770 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:44.770 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:44 smithi007 ceph-mon[108238]: pgmap v1203: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 6 op/s 2024-09-21T05:14:44.879 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:44.879 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:14:44.879 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:44 smithi053 ceph-mon[80176]: from='client.35968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:44.879 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:14:44.880 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:44 smithi053 ceph-mon[80176]: pgmap v1203: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 6 op/s 2024-09-21T05:14:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.964+0000 7f4a37ede640 1 -- 172.21.15.7:0/60154867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301067f0 msgr2=0x7f4a30106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.964+0000 7f4a37ede640 1 --2- 172.21.15.7:0/60154867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301067f0 0x7f4a30106bd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4a280097e0 tx=0x7f4a2802a3f0 comp rx=0 tx=0).stop 2024-09-21T05:14:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.964+0000 7f4a37ede640 1 -- 172.21.15.7:0/60154867 shutdown_connections 2024-09-21T05:14:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.964+0000 7f4a37ede640 1 --2- 172.21.15.7:0/60154867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301007f0 0x7f4a30100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.964+0000 7f4a37ede640 1 --2- 172.21.15.7:0/60154867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301067f0 0x7f4a30106bd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.964+0000 7f4a37ede640 1 -- 172.21.15.7:0/60154867 >> 172.21.15.7:0/60154867 conn(0x7f4a300fc550 msgr2=0x7f4a300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.964+0000 7f4a37ede640 1 -- 172.21.15.7:0/60154867 shutdown_connections 2024-09-21T05:14:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.964+0000 7f4a37ede640 1 -- 172.21.15.7:0/60154867 wait complete. 2024-09-21T05:14:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a37ede640 1 Processor -- start 2024-09-21T05:14:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a37ede640 1 -- start start 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a37ede640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301007f0 0x7f4a3019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a37ede640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301067f0 0x7f4a3019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a37ede640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3010dca0 con 0x7f4a301007f0 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a37ede640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3010db20 con 0x7f4a301067f0 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a35c7a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301007f0 0x7f4a3019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a35c7a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301007f0 0x7f4a3019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44992/0 (socket says 172.21.15.7:44992) 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a35c7a640 1 -- 172.21.15.7:0/2324994353 learned_addr learned my addr 172.21.15.7:0/2324994353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a35479640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301067f0 0x7f4a3019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a35c7a640 1 -- 172.21.15.7:0/2324994353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301067f0 msgr2=0x7f4a3019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a35c7a640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301067f0 0x7f4a3019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a35c7a640 1 -- 172.21.15.7:0/2324994353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a30197750 con 0x7f4a301007f0 2024-09-21T05:14:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.965+0000 7f4a35479640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301067f0 0x7f4a3019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:14:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.966+0000 7f4a35c7a640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301007f0 0x7f4a3019d680 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4a2802a910 tx=0x7f4a28004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.966+0000 7f4a2effd640 1 -- 172.21.15.7:0/2324994353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a28040070 con 0x7f4a301007f0 2024-09-21T05:14:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.966+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a301979e0 con 0x7f4a301007f0 2024-09-21T05:14:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.966+0000 7f4a2effd640 1 -- 172.21.15.7:0/2324994353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a28031400 con 0x7f4a301007f0 2024-09-21T05:14:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.966+0000 7f4a2effd640 1 -- 172.21.15.7:0/2324994353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a28003bf0 con 0x7f4a301007f0 2024-09-21T05:14:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.966+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a30197ee0 con 0x7f4a301007f0 2024-09-21T05:14:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.967+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a30101f30 con 0x7f4a301007f0 2024-09-21T05:14:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.967+0000 7f4a2effd640 1 -- 172.21.15.7:0/2324994353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f4a28039040 con 0x7f4a301007f0 2024-09-21T05:14:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.968+0000 7f4a2effd640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a1807af30 0x7f4a1807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.968+0000 7f4a2effd640 1 -- 172.21.15.7:0/2324994353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f4a280bf360 con 0x7f4a301007f0 2024-09-21T05:14:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.968+0000 7f4a35479640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a1807af30 0x7f4a1807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:44.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.969+0000 7f4a35479640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a1807af30 0x7f4a1807d3f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4a240096c0 tx=0x7f4a24009260 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:44.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:44.972+0000 7f4a2effd640 1 -- 172.21.15.7:0/2324994353 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4a2808a980 con 0x7f4a301007f0 2024-09-21T05:14:45.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.167+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4a30198190 con 0x7f4a1807af30 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.177+0000 7f4a2effd640 1 -- 172.21.15.7:0/2324994353 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f4a30198190 con 0x7f4a1807af30 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (36m) 8s ago 47m 20.1M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (22s) 8s ago 47m 8246k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 75aeb253c7d6 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (17s) 13s ago 47m 7696k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 405bcf2dcae3 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (34m) 8s ago 47m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (34m) 13s ago 47m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (36m) 8s ago 47m 77.2M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (30m) 8s ago 42m 25.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (31m) 8s ago 42m 90.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (15m) 13s ago 42m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (30m) 13s ago 42m 27.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:14:45.180 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (38m) 8s ago 49m 537M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (38m) 13s ago 46m 464M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (35m) 8s ago 49m 84.9M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (35m) 13s ago 46m 57.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (37m) 8s ago 47m 11.4M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (37m) 13s ago 46m 11.3M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (34m) 8s ago 45m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (33m) 8s ago 45m 108M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (33m) 8s ago 44m 100M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (32m) 13s ago 44m 110M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (32m) 13s ago 43m 93.9M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (31m) 13s ago 43m 94.7M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:14:45.181 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (37m) 8s ago 47m 71.7M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a1807af30 msgr2=0x7f4a1807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a1807af30 0x7f4a1807d3f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4a240096c0 tx=0x7f4a24009260 comp rx=0 tx=0).stop 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301007f0 msgr2=0x7f4a3019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301007f0 0x7f4a3019d680 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4a2802a910 tx=0x7f4a28004270 comp rx=0 tx=0).stop 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 shutdown_connections 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f4a1807af30 0x7f4a1807d3f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a301067f0 0x7f4a3019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 --2- 172.21.15.7:0/2324994353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a301007f0 0x7f4a3019d680 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 >> 172.21.15.7:0/2324994353 conn(0x7f4a300fc550 msgr2=0x7f4a30069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:45.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 shutdown_connections 2024-09-21T05:14:45.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:45.179+0000 7f4a37ede640 1 -- 172.21.15.7:0/2324994353 wait complete. 2024-09-21T05:14:45.845 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:45 smithi007 ceph-mon[108238]: from='client.35972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:45.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch upgrade status' 2024-09-21T05:14:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:14:46.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:45 smithi053 ceph-mon[80176]: from='client.35972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:46.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.572+0000 7f24f5df8640 1 -- 172.21.15.7:0/2012004959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01007f0 msgr2=0x7f24f0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:46.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.572+0000 7f24f5df8640 1 --2- 172.21.15.7:0/2012004959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01007f0 0x7f24f0100c50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009880 tx=0x7f24e402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:14:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.572+0000 7f24f5df8640 1 -- 172.21.15.7:0/2012004959 shutdown_connections 2024-09-21T05:14:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.572+0000 7f24f5df8640 1 --2- 172.21.15.7:0/2012004959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01007f0 0x7f24f0100c50 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.572+0000 7f24f5df8640 1 --2- 172.21.15.7:0/2012004959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24f01067f0 0x7f24f0106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.572+0000 7f24f5df8640 1 -- 172.21.15.7:0/2012004959 >> 172.21.15.7:0/2012004959 conn(0x7f24f00fc550 msgr2=0x7f24f00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.572+0000 7f24f5df8640 1 -- 172.21.15.7:0/2012004959 shutdown_connections 2024-09-21T05:14:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.572+0000 7f24f5df8640 1 -- 172.21.15.7:0/2012004959 wait complete. 2024-09-21T05:14:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24f5df8640 1 Processor -- start 2024-09-21T05:14:46.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24f5df8640 1 -- start start 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24f5df8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24f01007f0 0x7f24f00fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24f5df8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01067f0 0x7f24f00fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24f5df8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010dcc0 con 0x7f24f01067f0 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24f5df8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010db40 con 0x7f24f01007f0 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24eeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01067f0 0x7f24f00fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24f01007f0 0x7f24f00fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24eeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01067f0 0x7f24f00fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45008/0 (socket says 172.21.15.7:45008) 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24eeffd640 1 -- 172.21.15.7:0/78133307 learned_addr learned my addr 172.21.15.7:0/78133307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:46.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.573+0000 7f24ef7fe640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24f01007f0 0x7f24f00fccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.7:56296/0 (socket says 172.21.15.7:56296) 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24eeffd640 1 -- 172.21.15.7:0/78133307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24f01007f0 msgr2=0x7f24f00fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24eeffd640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24f01007f0 0x7f24f00fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24eeffd640 1 -- 172.21.15.7:0/78133307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f00fd750 con 0x7f24f01067f0 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24ef7fe640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24f01007f0 0x7f24f00fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24eeffd640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01067f0 0x7f24f00fd210 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f24e4031460 tx=0x7f24e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24ecff9640 1 -- 172.21.15.7:0/78133307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24e403f070 con 0x7f24f01067f0 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24ecff9640 1 -- 172.21.15.7:0/78133307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f24e40035f0 con 0x7f24f01067f0 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f00fd9e0 con 0x7f24f01067f0 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24ecff9640 1 -- 172.21.15.7:0/78133307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24e4035030 con 0x7f24f01067f0 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.574+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01a3da0 con 0x7f24f01067f0 2024-09-21T05:14:46.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.575+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f0101f30 con 0x7f24f01067f0 2024-09-21T05:14:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.576+0000 7f24ecff9640 1 -- 172.21.15.7:0/78133307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f24e4042070 con 0x7f24f01067f0 2024-09-21T05:14:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.576+0000 7f24ecff9640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24d807af30 0x7f24d807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.577+0000 7f24ecff9640 1 -- 172.21.15.7:0/78133307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f24e40bea70 con 0x7f24f01067f0 2024-09-21T05:14:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.577+0000 7f24ef7fe640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24d807af30 0x7f24d807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:46.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.577+0000 7f24ef7fe640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24d807af30 0x7f24d807d3f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24e000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:46.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.581+0000 7f24ecff9640 1 -- 172.21.15.7:0/78133307 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24e4089f70 con 0x7f24f01067f0 2024-09-21T05:14:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.788+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f24f0062630 con 0x7f24d807af30 2024-09-21T05:14:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.789+0000 7f24ecff9640 1 -- 172.21.15.7:0/78133307 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+175 (secure 0 0 0) 0x7f24f0062630 con 0x7f24d807af30 2024-09-21T05:14:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:14:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": null, 2024-09-21T05:14:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": false, 2024-09-21T05:14:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "", 2024-09-21T05:14:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-09-21T05:14:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": null, 2024-09-21T05:14:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-09-21T05:14:46.793 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-09-21T05:14:46.793 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24d807af30 msgr2=0x7f24d807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24d807af30 0x7f24d807d3f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24e000b390 comp rx=0 tx=0).stop 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01067f0 msgr2=0x7f24f00fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01067f0 0x7f24f00fd210 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f24e4031460 tx=0x7f24e40045e0 comp rx=0 tx=0).stop 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 shutdown_connections 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f24d807af30 0x7f24d807d3f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24f01067f0 0x7f24f00fd210 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 --2- 172.21.15.7:0/78133307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24f01007f0 0x7f24f00fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 >> 172.21.15.7:0/78133307 conn(0x7f24f00fc550 msgr2=0x7f24f0069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:46.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 shutdown_connections 2024-09-21T05:14:46.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:46.792+0000 7f24f5df8640 1 -- 172.21.15.7:0/78133307 wait complete. 2024-09-21T05:14:46.928 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:46 smithi007 ceph-mon[108238]: pgmap v1204: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.7 MiB/s rd, 4 op/s 2024-09-21T05:14:47.192 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:46 smithi053 ceph-mon[80176]: pgmap v1204: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.7 MiB/s rd, 4 op/s 2024-09-21T05:14:47.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph health detail' 2024-09-21T05:14:47.770 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:14:47.997 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:47 smithi007 ceph-mon[108238]: from='client.35976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.172+0000 7f7d872c7640 1 -- 172.21.15.7:0/191917856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80103440 msgr2=0x7f7d8010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.172+0000 7f7d872c7640 1 --2- 172.21.15.7:0/191917856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80103440 0x7f7d8010b930 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7d700098b0 tx=0x7f7d7002a560 comp rx=0 tx=0).stop 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.172+0000 7f7d872c7640 1 -- 172.21.15.7:0/191917856 shutdown_connections 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.172+0000 7f7d872c7640 1 --2- 172.21.15.7:0/191917856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80103440 0x7f7d8010b930 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.172+0000 7f7d872c7640 1 --2- 172.21.15.7:0/191917856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d80102b20 0x7f7d80102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.172+0000 7f7d872c7640 1 -- 172.21.15.7:0/191917856 >> 172.21.15.7:0/191917856 conn(0x7f7d800fc520 msgr2=0x7f7d800fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.172+0000 7f7d872c7640 1 -- 172.21.15.7:0/191917856 shutdown_connections 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.172+0000 7f7d872c7640 1 -- 172.21.15.7:0/191917856 wait complete. 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d872c7640 1 Processor -- start 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d872c7640 1 -- start start 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d872c7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80102b20 0x7f7d80100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:48.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d872c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d80103440 0x7f7d80100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d872c7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8010edf0 con 0x7f7d80102b20 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d872c7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8010ec70 con 0x7f7d80103440 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d85063640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80102b20 0x7f7d80100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d85063640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80102b20 0x7f7d80100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45022/0 (socket says 172.21.15.7:45022) 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d85063640 1 -- 172.21.15.7:0/1898865304 learned_addr learned my addr 172.21.15.7:0/1898865304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.173+0000 7f7d84862640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d80103440 0x7f7d80100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d85063640 1 -- 172.21.15.7:0/1898865304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d80103440 msgr2=0x7f7d80100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d85063640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d80103440 0x7f7d80100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d85063640 1 -- 172.21.15.7:0/1898865304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d80101200 con 0x7f7d80102b20 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d85063640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80102b20 0x7f7d80100780 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00d6f0 tx=0x7f7d7c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d767fc640 1 -- 172.21.15.7:0/1898865304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7d7c017070 con 0x7f7d80102b20 2024-09-21T05:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d767fc640 1 -- 172.21.15.7:0/1898865304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7d7c00de00 con 0x7f7d80102b20 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d801014f0 con 0x7f7d80102b20 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d767fc640 1 -- 172.21.15.7:0/1898865304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7d7c004bb0 con 0x7f7d80102b20 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.174+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d801a3f20 con 0x7f7d80102b20 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.175+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8006aa30 con 0x7f7d80102b20 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.176+0000 7f7d767fc640 1 -- 172.21.15.7:0/1898865304 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f7d7c004e60 con 0x7f7d80102b20 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.176+0000 7f7d767fc640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7d6c07af30 0x7f7d6c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.177+0000 7f7d767fc640 1 -- 172.21.15.7:0/1898865304 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f7d7c09dbe0 con 0x7f7d80102b20 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.177+0000 7f7d84862640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7d6c07af30 0x7f7d6c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.177+0000 7f7d84862640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7d6c07af30 0x7f7d6c07d3f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7d70000c00 tx=0x7f7d70031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.181+0000 7f7d767fc640 1 -- 172.21.15.7:0/1898865304 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7d7c069180 con 0x7f7d80102b20 2024-09-21T05:14:48.194 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:47 smithi053 ceph-mon[80176]: from='client.35976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:14:48.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.442+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f7d80102f00 con 0x7f7d80102b20 2024-09-21T05:14:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.442+0000 7f7d767fc640 1 -- 172.21.15.7:0/1898865304 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7f7d7c06e030 con 0x7f7d80102b20 2024-09-21T05:14:48.445 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T05:14:48.445 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:14:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:14:48.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7d6c07af30 msgr2=0x7f7d6c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:48.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7d6c07af30 0x7f7d6c07d3f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7d70000c00 tx=0x7f7d70031400 comp rx=0 tx=0).stop 2024-09-21T05:14:48.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80102b20 msgr2=0x7f7d80100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:48.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80102b20 0x7f7d80100780 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00d6f0 tx=0x7f7d7c00dbb0 comp rx=0 tx=0).stop 2024-09-21T05:14:48.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 shutdown_connections 2024-09-21T05:14:48.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f7d6c07af30 0x7f7d6c07d3f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:48.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d80103440 0x7f7d80100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:48.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 --2- 172.21.15.7:0/1898865304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d80102b20 0x7f7d80100780 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:48.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.444+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 >> 172.21.15.7:0/1898865304 conn(0x7f7d800fc520 msgr2=0x7f7d80103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:48.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.445+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 shutdown_connections 2024-09-21T05:14:48.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:48.445+0000 7f7d872c7640 1 -- 172.21.15.7:0/1898865304 wait complete. 2024-09-21T05:14:48.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:48 smithi053 ceph-mon[80176]: pgmap v1205: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s rd, 4 op/s 2024-09-21T05:14:48.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:48 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1898865304' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:14:49.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:48 smithi007 ceph-mon[108238]: pgmap v1205: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s rd, 4 op/s 2024-09-21T05:14:49.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:48 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1898865304' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-21T05:14:49.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions' 2024-09-21T05:14:49.548 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:14:50.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.055+0000 7fc27f30c640 1 -- 172.21.15.7:0/2880886323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074590 msgr2=0x7fc278074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:50.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.055+0000 7fc27f30c640 1 --2- 172.21.15.7:0/2880886323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074590 0x7fc278074970 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009880 tx=0x7fc26c02a530 comp rx=0 tx=0).stop 2024-09-21T05:14:50.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.055+0000 7fc27f30c640 1 -- 172.21.15.7:0/2880886323 shutdown_connections 2024-09-21T05:14:50.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.055+0000 7fc27f30c640 1 --2- 172.21.15.7:0/2880886323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc278074eb0 0x7fc2781123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:50.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.055+0000 7fc27f30c640 1 --2- 172.21.15.7:0/2880886323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074590 0x7fc278074970 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:50.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.055+0000 7fc27f30c640 1 -- 172.21.15.7:0/2880886323 >> 172.21.15.7:0/2880886323 conn(0x7fc2780fd600 msgr2=0x7fc2780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:50.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.056+0000 7fc27f30c640 1 -- 172.21.15.7:0/2880886323 shutdown_connections 2024-09-21T05:14:50.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.056+0000 7fc27f30c640 1 -- 172.21.15.7:0/2880886323 wait complete. 2024-09-21T05:14:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.056+0000 7fc27f30c640 1 Processor -- start 2024-09-21T05:14:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.056+0000 7fc27f30c640 1 -- start start 2024-09-21T05:14:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27f30c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc278074590 0x7fc27819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27f30c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074eb0 0x7fc27819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27f30c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc2781153d0 con 0x7fc278074eb0 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27f30c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc278115250 con 0x7fc278074590 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27c8a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074eb0 0x7fc27819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27c8a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074eb0 0x7fc27819e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45028/0 (socket says 172.21.15.7:45028) 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27c8a7640 1 -- 172.21.15.7:0/2069314669 learned_addr learned my addr 172.21.15.7:0/2069314669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27d0a8640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc278074590 0x7fc27819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27c8a7640 1 -- 172.21.15.7:0/2069314669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc278074590 msgr2=0x7fc27819dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27c8a7640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc278074590 0x7fc27819dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27c8a7640 1 -- 172.21.15.7:0/2069314669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2781a28c0 con 0x7fc278074eb0 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27d0a8640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc278074590 0x7fc27819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27c8a7640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074eb0 0x7fc27819e530 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc27400cc60 tx=0x7fc27400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc26a7fc640 1 -- 172.21.15.7:0/2069314669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc27400ad30 con 0x7fc278074eb0 2024-09-21T05:14:50.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2781a2bb0 con 0x7fc278074eb0 2024-09-21T05:14:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc26a7fc640 1 -- 172.21.15.7:0/2069314669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc274005020 con 0x7fc278074eb0 2024-09-21T05:14:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.057+0000 7fc26a7fc640 1 -- 172.21.15.7:0/2069314669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc27400f660 con 0x7fc278074eb0 2024-09-21T05:14:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.058+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2781a3140 con 0x7fc278074eb0 2024-09-21T05:14:50.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.059+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc278075cd0 con 0x7fc278074eb0 2024-09-21T05:14:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.059+0000 7fc26a7fc640 1 -- 172.21.15.7:0/2069314669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc2740051d0 con 0x7fc278074eb0 2024-09-21T05:14:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.060+0000 7fc26a7fc640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc26407af40 0x7fc26407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:14:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.060+0000 7fc26a7fc640 1 -- 172.21.15.7:0/2069314669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc27409e710 con 0x7fc278074eb0 2024-09-21T05:14:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.060+0000 7fc27d0a8640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc26407af40 0x7fc26407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:14:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.060+0000 7fc27d0a8640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc26407af40 0x7fc26407d400 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc26c031460 tx=0x7fc26c009580 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:14:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.064+0000 7fc26a7fc640 1 -- 172.21.15.7:0/2069314669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc274014030 con 0x7fc278074eb0 2024-09-21T05:14:50.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.326+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc27819f350 con 0x7fc278074eb0 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.326+0000 7fc26a7fc640 1 -- 172.21.15.7:0/2069314669 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fc274069c10 con 0x7fc278074eb0 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:14:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:14:50.330 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:14:50.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.328+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc26407af40 msgr2=0x7fc26407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:50.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.328+0000 7fc27f30c640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc26407af40 0x7fc26407d400 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc26c031460 tx=0x7fc26c009580 comp rx=0 tx=0).stop 2024-09-21T05:14:50.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.328+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074eb0 msgr2=0x7fc27819e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:14:50.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.328+0000 7fc27f30c640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074eb0 0x7fc27819e530 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc27400cc60 tx=0x7fc27400a620 comp rx=0 tx=0).stop 2024-09-21T05:14:50.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.329+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 shutdown_connections 2024-09-21T05:14:50.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.329+0000 7fc27f30c640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc26407af40 0x7fc26407d400 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:50.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.329+0000 7fc27f30c640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc278074eb0 0x7fc27819e530 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:50.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.329+0000 7fc27f30c640 1 --2- 172.21.15.7:0/2069314669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc278074590 0x7fc27819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:14:50.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.329+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 >> 172.21.15.7:0/2069314669 conn(0x7fc2780fd600 msgr2=0x7fc278104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:14:50.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.329+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 shutdown_connections 2024-09-21T05:14:50.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:14:50.329+0000 7fc27f30c640 1 -- 172.21.15.7:0/2069314669 wait complete. 2024-09-21T05:14:50.653 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:50 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2069314669' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:50.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:50 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2069314669' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:14:50.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-09-21T05:14:51.256 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:14:51.632 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:51 smithi007 ceph-mon[108238]: pgmap v1206: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 389 KiB/s rd, 4 op/s 2024-09-21T05:14:51.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:51 smithi053 ceph-mon[80176]: pgmap v1206: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 389 KiB/s rd, 4 op/s 2024-09-21T05:14:51.749 INFO:teuthology.orchestra.run.smithi007.stdout:wait for servicemap items w/ changing names to refresh 2024-09-21T05:14:52.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'sleep 60' 2024-09-21T05:14:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:14:53.666 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:53 smithi007 ceph-mon[108238]: pgmap v1207: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.2 KiB/s rd, 3 op/s 2024-09-21T05:14:53.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:53 smithi053 ceph-mon[80176]: pgmap v1207: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.2 KiB/s rd, 3 op/s 2024-09-21T05:14:55.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:55 smithi007 ceph-mon[108238]: pgmap v1208: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:14:55.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:55 smithi053 ceph-mon[80176]: pgmap v1208: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:14:57.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:57 smithi053 ceph-mon[80176]: pgmap v1209: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:14:57.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:57 smithi007 ceph-mon[108238]: pgmap v1209: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:14:59.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:59 smithi053 ceph-mon[80176]: pgmap v1210: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:14:59.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:14:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:14:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:59 smithi007 ceph-mon[108238]: pgmap v1210: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:14:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:14:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:15:01.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:01 smithi053 ceph-mon[80176]: pgmap v1211: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:01.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:01 smithi007 ceph-mon[108238]: pgmap v1211: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:03.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:03 smithi053 ceph-mon[80176]: pgmap v1212: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:03.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:03 smithi007 ceph-mon[108238]: pgmap v1212: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:05.683 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:05 smithi007 ceph-mon[108238]: pgmap v1213: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:05.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:05 smithi053 ceph-mon[80176]: pgmap v1213: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:07.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:07 smithi053 ceph-mon[80176]: pgmap v1214: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:07.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:07 smithi007 ceph-mon[108238]: pgmap v1214: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:09.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:09 smithi053 ceph-mon[80176]: pgmap v1215: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:09.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:09 smithi007 ceph-mon[108238]: pgmap v1215: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:11.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:11 smithi053 ceph-mon[80176]: pgmap v1216: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:11.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:11 smithi007 ceph-mon[108238]: pgmap v1216: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:13.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:13 smithi053 ceph-mon[80176]: pgmap v1217: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:13.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:13 smithi007 ceph-mon[108238]: pgmap v1217: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:14.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:15:14.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:15:15.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:15 smithi007 ceph-mon[108238]: pgmap v1218: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:15.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:15 smithi053 ceph-mon[80176]: pgmap v1218: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:16 smithi053 ceph-mon[80176]: pgmap v1219: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:16.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:16 smithi007 ceph-mon[108238]: pgmap v1219: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:19.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:19 smithi053 ceph-mon[80176]: pgmap v1220: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:19 smithi007 ceph-mon[108238]: pgmap v1220: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:21.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:21 smithi053 ceph-mon[80176]: pgmap v1221: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:21.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:21 smithi007 ceph-mon[108238]: pgmap v1221: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:23.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:23 smithi053 ceph-mon[80176]: pgmap v1222: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:23 smithi007 ceph-mon[108238]: pgmap v1222: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:25.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:25 smithi007 ceph-mon[108238]: pgmap v1223: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:25.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:25 smithi053 ceph-mon[80176]: pgmap v1223: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:27.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:27 smithi053 ceph-mon[80176]: pgmap v1224: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:27.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:27 smithi007 ceph-mon[108238]: pgmap v1224: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:29 smithi053 ceph-mon[80176]: pgmap v1225: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:15:29.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:29 smithi007 ceph-mon[108238]: pgmap v1225: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:15:31.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:31 smithi053 ceph-mon[80176]: pgmap v1226: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:31.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:31 smithi007 ceph-mon[108238]: pgmap v1226: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:33.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:33 smithi053 ceph-mon[80176]: pgmap v1227: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:33.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:33 smithi007 ceph-mon[108238]: pgmap v1227: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:35.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:35 smithi007 ceph-mon[108238]: pgmap v1228: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:35.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:35 smithi053 ceph-mon[80176]: pgmap v1228: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:37.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:37 smithi053 ceph-mon[80176]: pgmap v1229: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:37 smithi007 ceph-mon[108238]: pgmap v1229: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:38.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:38 smithi053 ceph-mon[80176]: pgmap v1230: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:38.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:38 smithi007 ceph-mon[108238]: pgmap v1230: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:41.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:41 smithi053 ceph-mon[80176]: pgmap v1231: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:41.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:41 smithi007 ceph-mon[108238]: pgmap v1231: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:42.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:15:42.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:15:42.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:15:42.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:15:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:15:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:15:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:15:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:15:43.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:43 smithi053 ceph-mon[80176]: pgmap v1232: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:43.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:43 smithi007 ceph-mon[108238]: pgmap v1232: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:44.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:15:44.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:15:45.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:45 smithi007 ceph-mon[108238]: pgmap v1233: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:45.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:45 smithi053 ceph-mon[80176]: pgmap v1233: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:47.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:47 smithi053 ceph-mon[80176]: pgmap v1234: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:47.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:47 smithi007 ceph-mon[108238]: pgmap v1234: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:49 smithi053 ceph-mon[80176]: pgmap v1235: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:49.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:49 smithi007 ceph-mon[108238]: pgmap v1235: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:51.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:51 smithi053 ceph-mon[80176]: pgmap v1236: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:51 smithi007 ceph-mon[108238]: pgmap v1236: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:53.667 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:53 smithi007 ceph-mon[108238]: pgmap v1237: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:53.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:53 smithi053 ceph-mon[80176]: pgmap v1237: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:53.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph orch ps' 2024-09-21T05:15:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 -- 172.21.15.7:0/883248207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 msgr2=0x7f3514106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 --2- 172.21.15.7:0/883248207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 0x7f3514106bd0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3504009880 tx=0x7f350402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 -- 172.21.15.7:0/883248207 shutdown_connections 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 --2- 172.21.15.7:0/883248207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35141007f0 0x7f3514100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 --2- 172.21.15.7:0/883248207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 0x7f3514106bd0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 -- 172.21.15.7:0/883248207 >> 172.21.15.7:0/883248207 conn(0x7f35140fc550 msgr2=0x7f35140fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 -- 172.21.15.7:0/883248207 shutdown_connections 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 -- 172.21.15.7:0/883248207 wait complete. 2024-09-21T05:15:54.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.445+0000 7f3519911640 1 Processor -- start 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f3519911640 1 -- start start 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f3519911640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35141007f0 0x7f3514198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f3519911640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 0x7f3514198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f3519911640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f351410dcc0 con 0x7f35141067f0 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f3519911640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f351410db40 con 0x7f35141007f0 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f35127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 0x7f3514198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f35127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 0x7f3514198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56614/0 (socket says 172.21.15.7:56614) 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f35127fc640 1 -- 172.21.15.7:0/3210033805 learned_addr learned my addr 172.21.15.7:0/3210033805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f3512ffd640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35141007f0 0x7f3514198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f35127fc640 1 -- 172.21.15.7:0/3210033805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35141007f0 msgr2=0x7f3514198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f35127fc640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35141007f0 0x7f3514198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f35127fc640 1 -- 172.21.15.7:0/3210033805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3514199350 con 0x7f35141067f0 2024-09-21T05:15:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.446+0000 7f35127fc640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 0x7f3514198d50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f350800ea00 tx=0x7f350800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.447+0000 7f351890f640 1 -- 172.21.15.7:0/3210033805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f350800cb80 con 0x7f35141067f0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.447+0000 7f351890f640 1 -- 172.21.15.7:0/3210033805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f350800cd20 con 0x7f35141067f0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.447+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3514199640 con 0x7f35141067f0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.447+0000 7f351890f640 1 -- 172.21.15.7:0/3210033805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3508011470 con 0x7f35141067f0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.447+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35141a1c40 con 0x7f35141067f0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.448+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3514101f30 con 0x7f35141067f0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.448+0000 7f351890f640 1 -- 172.21.15.7:0/3210033805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3508011750 con 0x7f35141067f0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.449+0000 7f351890f640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34fc07af30 0x7f34fc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.449+0000 7f351890f640 1 -- 172.21.15.7:0/3210033805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f350809e490 con 0x7f35141067f0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.449+0000 7f3512ffd640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34fc07af30 0x7f34fc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:54.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.449+0000 7f3512ffd640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34fc07af30 0x7f34fc07d3f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f35040096c0 tx=0x7f3504031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:15:54.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.453+0000 7f351890f640 1 -- 172.21.15.7:0/3210033805 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3514199640 con 0x7f35141067f0 2024-09-21T05:15:54.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.653+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 --> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3514062630 con 0x7f34fc07af30 2024-09-21T05:15:54.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.663+0000 7f351890f640 1 -- 172.21.15.7:0/3210033805 <== mgr.14772 v2:172.21.15.7:6800/4204093365 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f3514062630 con 0x7f34fc07af30 2024-09-21T05:15:54.665 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:15:54.665 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (37m) 78s ago 48m 20.1M - 0.27.0 11f11916f8cd 50032b8dace1 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi007 smithi007 running (92s) 78s ago 49m 8246k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 75aeb253c7d6 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-exporter.smithi053 smithi053 running (86s) 82s ago 48m 7696k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 405bcf2dcae3 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (36m) 78s ago 49m 6895k - 19.3.0-5103-gf36283e7 48ed0a2cf89d 96209ebb7c71 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi053 smithi053 running (36m) 82s ago 48m 6899k - 19.3.0-5103-gf36283e7 48ed0a2cf89d b1ee6c33fdc7 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (37m) 78s ago 48m 77.2M - 10.4.8 5aad1d7cf004 4ecacd68d50b 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.eyygpg smithi007 running (32m) 78s ago 43m 25.9M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 431c4b53206f 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.oxcnqk smithi007 running (32m) 78s ago 43m 90.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d ed087fe6d90b 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.jvuwme smithi053 running (16m) 82s ago 43m 23.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d a9e2fb9b5968 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi053.wdnfat smithi053 running (31m) 82s ago 43m 27.0M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 340c95848a00 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.huohfl smithi007 *:8443,9283,8765 running (40m) 78s ago 50m 537M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 7636d6651a1e 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi053.lnadao smithi053 *:8443,9283,8765 running (39m) 82s ago 48m 464M - 19.3.0-5103-gf36283e7 48ed0a2cf89d 8f5d82fb9766 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (36m) 78s ago 50m 84.9M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1ac9cb9a20ec 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi053 smithi053 running (36m) 82s ago 47m 57.5M 2048M 19.3.0-5103-gf36283e7 48ed0a2cf89d fdf38e86da7b 2024-09-21T05:15:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (38m) 78s ago 48m 11.4M - 1.7.0 72c9c2088986 49ebcf43f4ae 2024-09-21T05:15:54.667 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi053 smithi053 *:9100 running (38m) 82s ago 48m 11.3M - 1.7.0 72c9c2088986 25086059c197 2024-09-21T05:15:54.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (35m) 78s ago 46m 98.5M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 9ec59c74f4bd 2024-09-21T05:15:54.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (35m) 78s ago 46m 108M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d 55f4289c1275 2024-09-21T05:15:54.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (34m) 78s ago 45m 100M 1217M 19.3.0-5103-gf36283e7 48ed0a2cf89d fb138708649c 2024-09-21T05:15:54.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi053 running (33m) 82s ago 45m 110M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 1a3233caf51a 2024-09-21T05:15:54.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi053 running (33m) 82s ago 45m 93.9M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d 632b5d2947ba 2024-09-21T05:15:54.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi053 running (32m) 82s ago 44m 94.7M 2241M 19.3.0-5103-gf36283e7 48ed0a2cf89d d8e874d488eb 2024-09-21T05:15:54.667 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (38m) 78s ago 48m 71.7M - 2.51.0 1d3b7f56885b a45fae9b37b7 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34fc07af30 msgr2=0x7f34fc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34fc07af30 0x7f34fc07d3f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f35040096c0 tx=0x7f3504031400 comp rx=0 tx=0).stop 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 msgr2=0x7f3514198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 0x7f3514198d50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f350800ea00 tx=0x7f350800eec0 comp rx=0 tx=0).stop 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 shutdown_connections 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34fc07af30 0x7f34fc07d3f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35141067f0 0x7f3514198d50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 --2- 172.21.15.7:0/3210033805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35141007f0 0x7f3514198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 >> 172.21.15.7:0/3210033805 conn(0x7f35140fc550 msgr2=0x7f35140fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 shutdown_connections 2024-09-21T05:15:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:54.665+0000 7f3519911640 1 -- 172.21.15.7:0/3210033805 wait complete. 2024-09-21T05:15:55.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions' 2024-09-21T05:15:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:15:55.675 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:55 smithi007 ceph-mon[108238]: pgmap v1238: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:55.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:55 smithi053 ceph-mon[80176]: pgmap v1238: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:15:56.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.178+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1268678983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 msgr2=0x7f0d0010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.178+0000 7f0d04f2f640 1 --2- 172.21.15.7:0/1268678983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 0x7f0d0010b980 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009880 tx=0x7f0ce802a530 comp rx=0 tx=0).stop 2024-09-21T05:15:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.178+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1268678983 shutdown_connections 2024-09-21T05:15:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.178+0000 7f0d04f2f640 1 --2- 172.21.15.7:0/1268678983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 0x7f0d0010b980 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.178+0000 7f0d04f2f640 1 --2- 172.21.15.7:0/1268678983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d00102b70 0x7f0d00102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.178+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1268678983 >> 172.21.15.7:0/1268678983 conn(0x7f0d000fc550 msgr2=0x7f0d000fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:15:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.178+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1268678983 shutdown_connections 2024-09-21T05:15:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.178+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1268678983 wait complete. 2024-09-21T05:15:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0d04f2f640 1 Processor -- start 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0d04f2f640 1 -- start start 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0d04f2f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d00102b70 0x7f0d0019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0d04f2f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 0x7f0d0019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0d04f2f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010ede0 con 0x7f0d00103490 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0d04f2f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010ec60 con 0x7f0d00102b70 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 0x7f0d0019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0cf7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 0x7f0d0019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56626/0 (socket says 172.21.15.7:56626) 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0cf7fff640 1 -- 172.21.15.7:0/1868860380 learned_addr learned my addr 172.21.15.7:0/1868860380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0cfe59c640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d00102b70 0x7f0d0019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0cf7fff640 1 -- 172.21.15.7:0/1868860380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d00102b70 msgr2=0x7f0d0019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0cf7fff640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d00102b70 0x7f0d0019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.179+0000 7f0cf7fff640 1 -- 172.21.15.7:0/1868860380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d001977d0 con 0x7f0d00103490 2024-09-21T05:15:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.180+0000 7f0cf7fff640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 0x7f0d0019db90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8004f90 tx=0x7f0ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:15:56.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.180+0000 7f0cf77fe640 1 -- 172.21.15.7:0/1868860380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ce803f070 con 0x7f0d00103490 2024-09-21T05:15:56.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.180+0000 7f0cf77fe640 1 -- 172.21.15.7:0/1868860380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ce8004ac0 con 0x7f0d00103490 2024-09-21T05:15:56.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.180+0000 7f0cf77fe640 1 -- 172.21.15.7:0/1868860380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ce8035030 con 0x7f0d00103490 2024-09-21T05:15:56.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.180+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d00197a60 con 0x7f0d00103490 2024-09-21T05:15:56.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.180+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d00197f90 con 0x7f0d00103490 2024-09-21T05:15:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.181+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d0006a4c0 con 0x7f0d00103490 2024-09-21T05:15:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.182+0000 7f0cf77fe640 1 -- 172.21.15.7:0/1868860380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0ce8042070 con 0x7f0d00103490 2024-09-21T05:15:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.182+0000 7f0cf77fe640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cec07af30 0x7f0cec07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.182+0000 7f0cf77fe640 1 -- 172.21.15.7:0/1868860380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0ce80be9a0 con 0x7f0d00103490 2024-09-21T05:15:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.182+0000 7f0cfe59c640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cec07af30 0x7f0cec07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.183+0000 7f0cfe59c640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cec07af30 0x7f0cec07d3f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009b60 tx=0x7f0cf0005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:15:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.187+0000 7f0cf77fe640 1 -- 172.21.15.7:0/1868860380 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ce8089f40 con 0x7f0d00103490 2024-09-21T05:15:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.455+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f0d00106000 con 0x7f0d00103490 2024-09-21T05:15:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.455+0000 7f0cf77fe640 1 -- 172.21.15.7:0/1868860380 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f0ce808edf0 con 0x7f0d00103490 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 2 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 6 2024-09-21T05:15:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:15:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-09-21T05:15:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 4 2024-09-21T05:15:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-21T05:15:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-09-21T05:15:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)": 14 2024-09-21T05:15:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-21T05:15:56.459 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-21T05:15:56.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cec07af30 msgr2=0x7f0cec07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:56.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cec07af30 0x7f0cec07d3f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009b60 tx=0x7f0cf0005f90 comp rx=0 tx=0).stop 2024-09-21T05:15:56.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 msgr2=0x7f0d0019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:56.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 0x7f0d0019db90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8004f90 tx=0x7f0ce80046c0 comp rx=0 tx=0).stop 2024-09-21T05:15:56.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 shutdown_connections 2024-09-21T05:15:56.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0cec07af30 0x7f0cec07d3f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:56.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d00103490 0x7f0d0019db90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:56.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 --2- 172.21.15.7:0/1868860380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d00102b70 0x7f0d0019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:56.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 >> 172.21.15.7:0/1868860380 conn(0x7f0d000fc550 msgr2=0x7f0d00103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:15:56.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.457+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 shutdown_connections 2024-09-21T05:15:56.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:56.458+0000 7f0d04f2f640 1 -- 172.21.15.7:0/1868860380 wait complete. 2024-09-21T05:15:56.690 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:56 smithi007 ceph-mon[108238]: from='client.35988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:15:56.690 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:56 smithi007 ceph-mon[108238]: pgmap v1239: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:56.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:56 smithi053 ceph-mon[80176]: from='client.35988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:15:56.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:56 smithi053 ceph-mon[80176]: pgmap v1239: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:15:57.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-09-21T05:15:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:15:57.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:57 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1868860380' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:15:57.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:57 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1868860380' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:15:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.885+0000 7f8972b27640 1 -- 172.21.15.7:0/2540484417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 msgr2=0x7f896c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.885+0000 7f8972b27640 1 --2- 172.21.15.7:0/2540484417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 0x7f896c100c60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8960009880 tx=0x7f896002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:15:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 -- 172.21.15.7:0/2540484417 shutdown_connections 2024-09-21T05:15:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 --2- 172.21.15.7:0/2540484417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 0x7f896c100c60 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 --2- 172.21.15.7:0/2540484417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f896c106800 0x7f896c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 -- 172.21.15.7:0/2540484417 >> 172.21.15.7:0/2540484417 conn(0x7f896c0fc520 msgr2=0x7f896c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:15:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 -- 172.21.15.7:0/2540484417 shutdown_connections 2024-09-21T05:15:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 -- 172.21.15.7:0/2540484417 wait complete. 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 Processor -- start 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 -- start start 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 0x7f896c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f896c106800 0x7f896c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f896c10dcd0 con 0x7f896c100800 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.886+0000 7f8972b27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f896c10db50 con 0x7f896c106800 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f89708c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 0x7f896c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f89708c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 0x7f896c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56646/0 (socket says 172.21.15.7:56646) 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f89708c3640 1 -- 172.21.15.7:0/321067909 learned_addr learned my addr 172.21.15.7:0/321067909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f896bfff640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f896c106800 0x7f896c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f89708c3640 1 -- 172.21.15.7:0/321067909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f896c106800 msgr2=0x7f896c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f89708c3640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f896c106800 0x7f896c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f89708c3640 1 -- 172.21.15.7:0/321067909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c199350 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f89708c3640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 0x7f896c1987c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f895c00c8f0 tx=0x7f895c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f8969ffb640 1 -- 172.21.15.7:0/321067909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f895c0079b0 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c1a1830 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f8969ffb640 1 -- 172.21.15.7:0/321067909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f895c0054e0 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.887+0000 7f8969ffb640 1 -- 172.21.15.7:0/321067909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f895c004dd0 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.888+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c1a1c30 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.889+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896c101f40 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.889+0000 7f8969ffb640 1 -- 172.21.15.7:0/321067909 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f895c005020 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.890+0000 7f8969ffb640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f895407af30 0x7f895407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.890+0000 7f8969ffb640 1 -- 172.21.15.7:0/321067909 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f895c09deb0 con 0x7f896c100800 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.890+0000 7f896bfff640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f895407af30 0x7f895407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.890+0000 7f896bfff640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f895407af30 0x7f895407d3f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f896c06ad40 tx=0x7f8960031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:15:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:57.894+0000 7f8969ffb640 1 -- 172.21.15.7:0/321067909 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f895c014030 con 0x7f896c100800 2024-09-21T05:15:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.155+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f896c1087e0 con 0x7f896c100800 2024-09-21T05:15:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.155+0000 7f8969ffb640 1 -- 172.21.15.7:0/321067909 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f895c069470 con 0x7f896c100800 2024-09-21T05:15:58.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f895407af30 msgr2=0x7f895407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:58.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f895407af30 0x7f895407d3f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f896c06ad40 tx=0x7f8960031400 comp rx=0 tx=0).stop 2024-09-21T05:15:58.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 msgr2=0x7f896c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 0x7f896c1987c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f895c00c8f0 tx=0x7f895c00cdb0 comp rx=0 tx=0).stop 2024-09-21T05:15:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 shutdown_connections 2024-09-21T05:15:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f895407af30 0x7f895407d3f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f896c106800 0x7f896c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 --2- 172.21.15.7:0/321067909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f896c100800 0x7f896c1987c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 >> 172.21.15.7:0/321067909 conn(0x7f896c0fc520 msgr2=0x7f896c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:15:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 shutdown_connections 2024-09-21T05:15:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:58.158+0000 7f8972b27640 1 -- 172.21.15.7:0/321067909 wait complete. 2024-09-21T05:15:58.169 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-09-21T05:15:58.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:58 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/321067909' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:15:58.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:58 smithi053 ceph-mon[80176]: pgmap v1240: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:58 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/321067909' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:15:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:58 smithi007 ceph-mon[108238]: pgmap v1240: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:15:58.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-09-21T05:15:59.132 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:15:59.545 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:15:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:15:59.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.624+0000 7ff95c088640 1 -- 172.21.15.7:0/590832158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9541033e0 msgr2=0x7ff95410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:59.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.624+0000 7ff95c088640 1 --2- 172.21.15.7:0/590832158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9541033e0 0x7ff95410b8d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff948009880 tx=0x7ff94802a530 comp rx=0 tx=0).stop 2024-09-21T05:15:59.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.624+0000 7ff95c088640 1 -- 172.21.15.7:0/590832158 shutdown_connections 2024-09-21T05:15:59.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.624+0000 7ff95c088640 1 --2- 172.21.15.7:0/590832158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9541033e0 0x7ff95410b8d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:59.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.624+0000 7ff95c088640 1 --2- 172.21.15.7:0/590832158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff954102ac0 0x7ff954102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:59.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.624+0000 7ff95c088640 1 -- 172.21.15.7:0/590832158 >> 172.21.15.7:0/590832158 conn(0x7ff9540fc520 msgr2=0x7ff9540fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.624+0000 7ff95c088640 1 -- 172.21.15.7:0/590832158 shutdown_connections 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.624+0000 7ff95c088640 1 -- 172.21.15.7:0/590832158 wait complete. 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff95c088640 1 Processor -- start 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff95c088640 1 -- start start 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff95c088640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff954102ac0 0x7ff954100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff95c088640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9541033e0 0x7ff954100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff95c088640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff95410edf0 con 0x7ff954102ac0 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff95c088640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff95410ec70 con 0x7ff9541033e0 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff959e24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff954102ac0 0x7ff954100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff959e24640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff954102ac0 0x7ff954100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56674/0 (socket says 172.21.15.7:56674) 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff959e24640 1 -- 172.21.15.7:0/1235114152 learned_addr learned my addr 172.21.15.7:0/1235114152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:15:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.625+0000 7ff959623640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9541033e0 0x7ff954100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff959e24640 1 -- 172.21.15.7:0/1235114152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9541033e0 msgr2=0x7ff954100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff959e24640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9541033e0 0x7ff954100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff959e24640 1 -- 172.21.15.7:0/1235114152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9541011a0 con 0x7ff954102ac0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff959623640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9541033e0 0x7ff954100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff959e24640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff954102ac0 0x7ff954100720 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff95000d6f0 tx=0x7ff95000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff946ffd640 1 -- 172.21.15.7:0/1235114152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff950017070 con 0x7ff954102ac0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff954101490 con 0x7ff954102ac0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff946ffd640 1 -- 172.21.15.7:0/1235114152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff95000de00 con 0x7ff954102ac0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff946ffd640 1 -- 172.21.15.7:0/1235114152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff950004bb0 con 0x7ff954102ac0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.626+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541a3e80 con 0x7ff954102ac0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.627+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95406aa90 con 0x7ff954102ac0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.628+0000 7ff946ffd640 1 -- 172.21.15.7:0/1235114152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7ff950004e60 con 0x7ff954102ac0 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.628+0000 7ff946ffd640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff94007af30 0x7ff94007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:15:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.629+0000 7ff946ffd640 1 -- 172.21.15.7:0/1235114152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7ff95009dd50 con 0x7ff954102ac0 2024-09-21T05:15:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.629+0000 7ff959623640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff94007af30 0x7ff94007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:15:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.629+0000 7ff959623640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff94007af30 0x7ff94007d3f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff9541021b0 tx=0x7ff948031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:15:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.633+0000 7ff946ffd640 1 -- 172.21.15.7:0/1235114152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff950069250 con 0x7ff954102ac0 2024-09-21T05:15:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.895+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ff954101c80 con 0x7ff954102ac0 2024-09-21T05:15:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.896+0000 7ff946ffd640 1 -- 172.21.15.7:0/1235114152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7ff954101c80 con 0x7ff954102ac0 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff94007af30 msgr2=0x7ff94007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff94007af30 0x7ff94007d3f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff9541021b0 tx=0x7ff948031400 comp rx=0 tx=0).stop 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff954102ac0 msgr2=0x7ff954100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff954102ac0 0x7ff954100720 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff95000d6f0 tx=0x7ff95000dbb0 comp rx=0 tx=0).stop 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 shutdown_connections 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7ff94007af30 0x7ff94007d3f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9541033e0 0x7ff954100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 --2- 172.21.15.7:0/1235114152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff954102ac0 0x7ff954100720 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 >> 172.21.15.7:0/1235114152 conn(0x7ff9540fc520 msgr2=0x7ff954108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 shutdown_connections 2024-09-21T05:15:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:15:59.898+0000 7ff95c088640 1 -- 172.21.15.7:0/1235114152 wait complete. 2024-09-21T05:15:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.3.0-5103-gf36283e7 (f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2) squid (dev)" 2024-09-21T05:15:59.914 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:15:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:16:00.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:00 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1235114152' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:16:00.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:00 smithi007 ceph-mon[108238]: pgmap v1241: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:00.523 DEBUG:teuthology.parallel:result is None 2024-09-21T05:16:00.523 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T05:16:00.540 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T05:16:00.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T05:16:00.805 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:00 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1235114152' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-21T05:16:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:00 smithi053 ceph-mon[80176]: pgmap v1241: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.241+0000 7fc96cc4d640 1 -- 172.21.15.7:0/3293499585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968100800 msgr2=0x7fc968100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.241+0000 7fc96cc4d640 1 --2- 172.21.15.7:0/3293499585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968100800 0x7fc968100c60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009880 tx=0x7fc95c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.242+0000 7fc96cc4d640 1 -- 172.21.15.7:0/3293499585 shutdown_connections 2024-09-21T05:16:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.242+0000 7fc96cc4d640 1 --2- 172.21.15.7:0/3293499585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968100800 0x7fc968100c60 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.242+0000 7fc96cc4d640 1 --2- 172.21.15.7:0/3293499585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc968106800 0x7fc968106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.242+0000 7fc96cc4d640 1 -- 172.21.15.7:0/3293499585 >> 172.21.15.7:0/3293499585 conn(0x7fc9680fc520 msgr2=0x7fc9680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.242+0000 7fc96cc4d640 1 -- 172.21.15.7:0/3293499585 shutdown_connections 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.242+0000 7fc96cc4d640 1 -- 172.21.15.7:0/3293499585 wait complete. 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.242+0000 7fc96cc4d640 1 Processor -- start 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc96cc4d640 1 -- start start 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc96cc4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc968100800 0x7fc96819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc96cc4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968106800 0x7fc96819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc96cc4d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc96810dcd0 con 0x7fc968106800 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc96cc4d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc96810db50 con 0x7fc968100800 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc965d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968106800 0x7fc96819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc965d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968106800 0x7fc96819fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56686/0 (socket says 172.21.15.7:56686) 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc965d9b640 1 -- 172.21.15.7:0/60753703 learned_addr learned my addr 172.21.15.7:0/60753703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc96659c640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc968100800 0x7fc96819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc965d9b640 1 -- 172.21.15.7:0/60753703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc968100800 msgr2=0x7fc96819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc965d9b640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc968100800 0x7fc96819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.243+0000 7fc965d9b640 1 -- 172.21.15.7:0/60753703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9681999e0 con 0x7fc968106800 2024-09-21T05:16:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.244+0000 7fc965d9b640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968106800 0x7fc96819fd60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc95c031460 tx=0x7fc95c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.244+0000 7fc9577fe640 1 -- 172.21.15.7:0/60753703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc95c03f070 con 0x7fc968106800 2024-09-21T05:16:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.244+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc968199c70 con 0x7fc968106800 2024-09-21T05:16:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.244+0000 7fc9577fe640 1 -- 172.21.15.7:0/60753703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc95c0035f0 con 0x7fc968106800 2024-09-21T05:16:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.244+0000 7fc9577fe640 1 -- 172.21.15.7:0/60753703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc95c035030 con 0x7fc968106800 2024-09-21T05:16:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.244+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96819a1a0 con 0x7fc968106800 2024-09-21T05:16:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.245+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc968101f40 con 0x7fc968106800 2024-09-21T05:16:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.246+0000 7fc9577fe640 1 -- 172.21.15.7:0/60753703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc95c042070 con 0x7fc968106800 2024-09-21T05:16:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.246+0000 7fc9577fe640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc95007af80 0x7fc95007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.246+0000 7fc9577fe640 1 -- 172.21.15.7:0/60753703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc95c0be890 con 0x7fc968106800 2024-09-21T05:16:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.246+0000 7fc96659c640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc95007af80 0x7fc95007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.247+0000 7fc96659c640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc95007af80 0x7fc95007d440 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc96819ab20 tx=0x7fc958005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:01.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.250+0000 7fc9577fe640 1 -- 172.21.15.7:0/60753703 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc95c089eb0 con 0x7fc968106800 2024-09-21T05:16:01.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.476+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc96819af10 con 0x7fc968106800 2024-09-21T05:16:01.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.477+0000 7fc9577fe640 1 -- 172.21.15.7:0/60753703 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) ==== 76+0+2044 (secure 0 0 0) 0x7fc95c08ed60 con 0x7fc968106800 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:e38 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-21T05:14:41:335964+0000 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 38 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-21T04:32:16.358433+0000 2024-09-21T05:16:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-21T05:14:40.432139+0000 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 75 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-09-21T05:16:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=34354,1=34358} 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data enabled 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 34354 members: 34358,34354 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.oxcnqk{0:34354} state up:active seq 467 join_fscid=1 addr [v2:172.21.15.7:6826/3186196668,v1:172.21.15.7:6827/3186196668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:16:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.wdnfat{0:44327} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.53:6824/1618778609,v1:172.21.15.53:6825/1618778609] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:16:01.483 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.eyygpg{1:34358} state up:active seq 463 join_fscid=1 addr [v2:172.21.15.7:6828/2665314222,v1:172.21.15.7:6829/2665314222] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:16:01.483 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi053.jvuwme{1:44909} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.53:6826/3204998384,v1:172.21.15.53:6827/3204998384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:16:01.483 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:01.483 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:01.483 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc95007af80 msgr2=0x7fc95007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc95007af80 0x7fc95007d440 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc96819ab20 tx=0x7fc958005f90 comp rx=0 tx=0).stop 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968106800 msgr2=0x7fc96819fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968106800 0x7fc96819fd60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc95c031460 tx=0x7fc95c004530 comp rx=0 tx=0).stop 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 shutdown_connections 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc95007af80 0x7fc95007d440 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968106800 0x7fc96819fd60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 --2- 172.21.15.7:0/60753703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc968100800 0x7fc96819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 >> 172.21.15.7:0/60753703 conn(0x7fc9680fc520 msgr2=0x7fc9680fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 shutdown_connections 2024-09-21T05:16:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:01.479+0000 7fc96cc4d640 1 -- 172.21.15.7:0/60753703 wait complete. 2024-09-21T05:16:01.858 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:01 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/60753703' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:16:01.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:01 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/60753703' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:16:02.206 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-09-21T05:16:02.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:16:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:02.870 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:02 smithi007 ceph-mon[108238]: pgmap v1242: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:02.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:02 smithi053 ceph-mon[80176]: pgmap v1242: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/3474887835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 msgr2=0x7f0f44106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 --2- 172.21.15.7:0/3474887835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 0x7f0f44106bd0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009880 tx=0x7f0f4002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/3474887835 shutdown_connections 2024-09-21T05:16:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 --2- 172.21.15.7:0/3474887835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f441007f0 0x7f0f44100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 --2- 172.21.15.7:0/3474887835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 0x7f0f44106bd0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/3474887835 >> 172.21.15.7:0/3474887835 conn(0x7f0f440fc550 msgr2=0x7f0f440fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/3474887835 shutdown_connections 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/3474887835 wait complete. 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 Processor -- start 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.951+0000 7f0f4b3a3640 1 -- start start 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4b3a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f441007f0 0x7f0f4419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4b3a3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 0x7f0f4419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4b3a3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4410dcc0 con 0x7f0f441067f0 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4b3a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4410db40 con 0x7f0f441007f0 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4893e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 0x7f0f4419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4893e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 0x7f0f4419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56716/0 (socket says 172.21.15.7:56716) 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4893e640 1 -- 172.21.15.7:0/4058659266 learned_addr learned my addr 172.21.15.7:0/4058659266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4913f640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f441007f0 0x7f0f4419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4893e640 1 -- 172.21.15.7:0/4058659266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f441007f0 msgr2=0x7f0f4419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:02.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4893e640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f441007f0 0x7f0f4419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4893e640 1 -- 172.21.15.7:0/4058659266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f441977b0 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4893e640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 0x7f0f4419db70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400d6f0 tx=0x7f0f3400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f3a7fc640 1 -- 172.21.15.7:0/4058659266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f34017070 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f44197aa0 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f3a7fc640 1 -- 172.21.15.7:0/4058659266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f3400de00 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f3a7fc640 1 -- 172.21.15.7:0/4058659266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f34004bb0 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.952+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f44198030 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.953+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f44101f30 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.954+0000 7f0f3a7fc640 1 -- 172.21.15.7:0/4058659266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0f34004e60 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.955+0000 7f0f3a7fc640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3007af30 0x7f0f3007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.955+0000 7f0f3a7fc640 1 -- 172.21.15.7:0/4058659266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0f3409dca0 con 0x7f0f441067f0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.955+0000 7f0f4913f640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3007af30 0x7f0f3007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.955+0000 7f0f4913f640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3007af30 0x7f0f3007d3f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0f400096c0 tx=0x7f0f40031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:02.959+0000 7f0f3a7fc640 1 -- 172.21.15.7:0/4058659266 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f340691a0 con 0x7f0f441067f0 2024-09-21T05:16:03.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.188+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f44109d70 con 0x7f0f441067f0 2024-09-21T05:16:03.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.188+0000 7f0f3a7fc640 1 -- 172.21.15.7:0/4058659266 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5352 (secure 0 0 0) 0x7f0f3406e050 con 0x7f0f441067f0 2024-09-21T05:16:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:03.191 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"btime":"2024-09-21T05:14:41:335964+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T05:14:40.432139+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34354,"mds_1":34358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34354":{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":0,"incarnation":32,"state":"up:active","state_seq":467,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34358":{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":1,"incarnation":32,"state":"up:active","state_seq":463,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44327":{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44909":{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34354,"qdb_cluster":[34358,34354]},"id":1}]} 2024-09-21T05:16:03.191 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2024-09-21T05:16:03.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3007af30 msgr2=0x7f0f3007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:03.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3007af30 0x7f0f3007d3f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0f400096c0 tx=0x7f0f40031400 comp rx=0 tx=0).stop 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 msgr2=0x7f0f4419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 0x7f0f4419db70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400d6f0 tx=0x7f0f3400dbb0 comp rx=0 tx=0).stop 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 shutdown_connections 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0f3007af30 0x7f0f3007d3f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f441067f0 0x7f0f4419db70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 --2- 172.21.15.7:0/4058659266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f441007f0 0x7f0f4419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 >> 172.21.15.7:0/4058659266 conn(0x7f0f440fc550 msgr2=0x7f0f44069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 shutdown_connections 2024-09-21T05:16:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:03.191+0000 7f0f4b3a3640 1 -- 172.21.15.7:0/4058659266 wait complete. 2024-09-21T05:16:03.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:03 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/4058659266' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:16:03.815 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 19, 'max_mds': 2, 'flags': 50} 2024-09-21T05:16:03.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 20 2024-09-21T05:16:03.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:03 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/4058659266' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:16:04.076 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.567+0000 7f13a4b54640 1 -- 172.21.15.7:0/4017813194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0101900 msgr2=0x7f13a0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.567+0000 7f13a4b54640 1 --2- 172.21.15.7:0/4017813194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0101900 0x7f13a0101d60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1394009c20 tx=0x7f139402a970 comp rx=0 tx=0).stop 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.567+0000 7f13a4b54640 1 -- 172.21.15.7:0/4017813194 shutdown_connections 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.567+0000 7f13a4b54640 1 --2- 172.21.15.7:0/4017813194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0101900 0x7f13a0101d60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.567+0000 7f13a4b54640 1 --2- 172.21.15.7:0/4017813194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0107900 0x7f13a0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.567+0000 7f13a4b54640 1 -- 172.21.15.7:0/4017813194 >> 172.21.15.7:0/4017813194 conn(0x7f13a00fd600 msgr2=0x7f13a00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.567+0000 7f13a4b54640 1 -- 172.21.15.7:0/4017813194 shutdown_connections 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.567+0000 7f13a4b54640 1 -- 172.21.15.7:0/4017813194 wait complete. 2024-09-21T05:16:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.568+0000 7f13a4b54640 1 Processor -- start 2024-09-21T05:16:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.568+0000 7f13a4b54640 1 -- start start 2024-09-21T05:16:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.568+0000 7f13a4b54640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0101900 0x7f13a019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.568+0000 7f13a4b54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0107900 0x7f13a019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.568+0000 7f13a4b54640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010edd0 con 0x7f13a0107900 2024-09-21T05:16:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.568+0000 7f13a4b54640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010ec50 con 0x7f13a0101900 2024-09-21T05:16:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f139dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0107900 0x7f13a019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f139dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0107900 0x7f13a019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33308/0 (socket says 172.21.15.7:33308) 2024-09-21T05:16:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f139dd9b640 1 -- 172.21.15.7:0/1369548483 learned_addr learned my addr 172.21.15.7:0/1369548483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f139e59c640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0101900 0x7f13a019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f139dd9b640 1 -- 172.21.15.7:0/1369548483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0101900 msgr2=0x7f13a019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f139dd9b640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0101900 0x7f13a019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f139dd9b640 1 -- 172.21.15.7:0/1369548483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01977a0 con 0x7f13a0107900 2024-09-21T05:16:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f139dd9b640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0107900 0x7f13a019db60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f139402ae90 tx=0x7f139402f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f138f7fe640 1 -- 172.21.15.7:0/1369548483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1394047070 con 0x7f13a0107900 2024-09-21T05:16:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f138f7fe640 1 -- 172.21.15.7:0/1369548483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f139402f7e0 con 0x7f13a0107900 2024-09-21T05:16:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a0197a30 con 0x7f13a0107900 2024-09-21T05:16:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.569+0000 7f138f7fe640 1 -- 172.21.15.7:0/1369548483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1394034070 con 0x7f13a0107900 2024-09-21T05:16:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.570+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a0197f60 con 0x7f13a0107900 2024-09-21T05:16:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.570+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a0103040 con 0x7f13a0107900 2024-09-21T05:16:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.571+0000 7f138f7fe640 1 -- 172.21.15.7:0/1369548483 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f139404a070 con 0x7f13a0107900 2024-09-21T05:16:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.572+0000 7f138f7fe640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f138807af80 0x7f138807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.572+0000 7f138f7fe640 1 -- 172.21.15.7:0/1369548483 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f13940bec60 con 0x7f13a0107900 2024-09-21T05:16:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.572+0000 7f139e59c640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f138807af80 0x7f138807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.572+0000 7f139e59c640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f138807af80 0x7f138807d440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1390005fd0 tx=0x7f139000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.576+0000 7f138f7fe640 1 -- 172.21.15.7:0/1369548483 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f139403f070 con 0x7f13a0107900 2024-09-21T05:16:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.803+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) -- 0x7f13a0198cd0 con 0x7f13a0107900 2024-09-21T05:16:04.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:04 smithi007 ceph-mon[108238]: pgmap v1243: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.803+0000 7f138f7fe640 1 -- 172.21.15.7:0/1369548483 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v38) ==== 107+0+3639 (secure 0 0 0) 0x7f139408a280 con 0x7f13a0107900 2024-09-21T05:16:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:04.806 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"btime":"2024-09-21T04:43:25:588155+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:24.590189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14578,"mds_1":24327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14578":{"gid":14578,"name":"cephfs.smithi007.eyygpg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.7:6829/406067991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":406067991},{"type":"v1","addr":"172.21.15.7:6829","nonce":406067991}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24327":{"gid":24327,"name":"cephfs.smithi053.wdnfat","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6825/2105303008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2105303008},{"type":"v1","addr":"172.21.15.53:6825","nonce":2105303008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14578,"qdb_cluster":[14578,24327]},"id":1}]} 2024-09-21T05:16:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f138807af80 msgr2=0x7f138807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f138807af80 0x7f138807d440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1390005fd0 tx=0x7f139000b390 comp rx=0 tx=0).stop 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0107900 msgr2=0x7f13a019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0107900 0x7f13a019db60 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f139402ae90 tx=0x7f139402f5f0 comp rx=0 tx=0).stop 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 shutdown_connections 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f138807af80 0x7f138807d440 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a0107900 0x7f13a019db60 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 --2- 172.21.15.7:0/1369548483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13a0101900 0x7f13a019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 >> 172.21.15.7:0/1369548483 conn(0x7f13a00fd600 msgr2=0x7f13a00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 shutdown_connections 2024-09-21T05:16:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:04.806+0000 7f13a4b54640 1 -- 172.21.15.7:0/1369548483 wait complete. 2024-09-21T05:16:04.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:04 smithi053 ceph-mon[80176]: pgmap v1243: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:05.462 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-09-21T05:16:05.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 21 2024-09-21T05:16:05.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:05 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1369548483' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 20, "format": "json"} : dispatch 2024-09-21T05:16:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:05.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:05 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1369548483' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 20, "format": "json"} : dispatch 2024-09-21T05:16:06.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.183+0000 7f3515f01640 1 -- 172.21.15.7:0/1634422416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510106800 msgr2=0x7f3510106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:06.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.183+0000 7f3515f01640 1 --2- 172.21.15.7:0/1634422416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510106800 0x7f3510106be0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3500009880 tx=0x7f350002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.183+0000 7f3515f01640 1 -- 172.21.15.7:0/1634422416 shutdown_connections 2024-09-21T05:16:06.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.183+0000 7f3515f01640 1 --2- 172.21.15.7:0/1634422416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3510100800 0x7f3510100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.183+0000 7f3515f01640 1 --2- 172.21.15.7:0/1634422416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510106800 0x7f3510106be0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.183+0000 7f3515f01640 1 -- 172.21.15.7:0/1634422416 >> 172.21.15.7:0/1634422416 conn(0x7f35100fc520 msgr2=0x7f35100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.184+0000 7f3515f01640 1 -- 172.21.15.7:0/1634422416 shutdown_connections 2024-09-21T05:16:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.184+0000 7f3515f01640 1 -- 172.21.15.7:0/1634422416 wait complete. 2024-09-21T05:16:06.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.184+0000 7f3515f01640 1 Processor -- start 2024-09-21T05:16:06.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f3515f01640 1 -- start start 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f3515f01640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510100800 0x7f351019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f3515f01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3510106800 0x7f351019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f3515f01640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f351010dcb0 con 0x7f3510100800 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f3515f01640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f351010db30 con 0x7f3510106800 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510100800 0x7f351019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510100800 0x7f351019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33322/0 (socket says 172.21.15.7:33322) 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350f7fe640 1 -- 172.21.15.7:0/3625835900 learned_addr learned my addr 172.21.15.7:0/3625835900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350effd640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3510106800 0x7f351019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350f7fe640 1 -- 172.21.15.7:0/3625835900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3510106800 msgr2=0x7f351019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350f7fe640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3510106800 0x7f351019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350f7fe640 1 -- 172.21.15.7:0/3625835900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3510197740 con 0x7f3510100800 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350effd640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3510106800 0x7f351019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.185+0000 7f350f7fe640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510100800 0x7f351019d670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3500031460 tx=0x7f35000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.186+0000 7f350cff9640 1 -- 172.21.15.7:0/3625835900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f350003f070 con 0x7f3510100800 2024-09-21T05:16:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.186+0000 7f350cff9640 1 -- 172.21.15.7:0/3625835900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f35000044d0 con 0x7f3510100800 2024-09-21T05:16:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.186+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35101979d0 con 0x7f3510100800 2024-09-21T05:16:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.186+0000 7f350cff9640 1 -- 172.21.15.7:0/3625835900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3500035030 con 0x7f3510100800 2024-09-21T05:16:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.186+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3510197f00 con 0x7f3510100800 2024-09-21T05:16:06.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.187+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3510101f40 con 0x7f3510100800 2024-09-21T05:16:06.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.188+0000 7f350cff9640 1 -- 172.21.15.7:0/3625835900 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3500042070 con 0x7f3510100800 2024-09-21T05:16:06.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.188+0000 7f350cff9640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f807af40 0x7f34f807d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:06.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.188+0000 7f350cff9640 1 -- 172.21.15.7:0/3625835900 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f35000bebc0 con 0x7f3510100800 2024-09-21T05:16:06.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.188+0000 7f350effd640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f807af40 0x7f34f807d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:06.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.189+0000 7f350effd640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f807af40 0x7f34f807d400 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3510198f90 tx=0x7f350400a5c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:06.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.193+0000 7f350cff9640 1 -- 172.21.15.7:0/3625835900 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3500004cf0 con 0x7f3510100800 2024-09-21T05:16:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.418+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) -- 0x7f35101981b0 con 0x7f3510100800 2024-09-21T05:16:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.418+0000 7f350cff9640 1 -- 172.21.15.7:0/3625835900 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v38) ==== 107+0+3600 (secure 0 0 0) 0x7f350008a160 con 0x7f3510100800 2024-09-21T05:16:06.421 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:06.421 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"btime":"2024-09-21T04:43:25:693128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34314,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/269195221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":269195221},{"type":"v1","addr":"172.21.15.7:6827","nonce":269195221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44275,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/606146784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":606146784},{"type":"v1","addr":"172.21.15.53:6827","nonce":606146784}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 21 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.420+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f807af40 msgr2=0x7f34f807d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.420+0000 7f3515f01640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f807af40 0x7f34f807d400 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3510198f90 tx=0x7f350400a5c0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.420+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510100800 msgr2=0x7f351019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.420+0000 7f3515f01640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510100800 0x7f351019d670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3500031460 tx=0x7f35000048f0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.421+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 shutdown_connections 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.421+0000 7f3515f01640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f34f807af40 0x7f34f807d400 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.421+0000 7f3515f01640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3510106800 0x7f351019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.421+0000 7f3515f01640 1 --2- 172.21.15.7:0/3625835900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3510100800 0x7f351019d670 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.421+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 >> 172.21.15.7:0/3625835900 conn(0x7f35100fc520 msgr2=0x7f351006a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.421+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 shutdown_connections 2024-09-21T05:16:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:06.421+0000 7f3515f01640 1 -- 172.21.15.7:0/3625835900 wait complete. 2024-09-21T05:16:06.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:06 smithi007 ceph-mon[108238]: pgmap v1244: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:06.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:06 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3625835900' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 21, "format": "json"} : dispatch 2024-09-21T05:16:06.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:06 smithi053 ceph-mon[80176]: pgmap v1244: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:06.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:06 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3625835900' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 21, "format": "json"} : dispatch 2024-09-21T05:16:07.118 DEBUG:tasks.fs:max_mds not reduced in epoch 21 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:07.119 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-09-21T05:16:07.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 22 2024-09-21T05:16:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:07.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 -- 172.21.15.7:0/3164016495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074620 msgr2=0x7f8770074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:07.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 --2- 172.21.15.7:0/3164016495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074620 0x7f8770074a00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f876c009880 tx=0x7f876c02a530 comp rx=0 tx=0).stop 2024-09-21T05:16:07.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 -- 172.21.15.7:0/3164016495 shutdown_connections 2024-09-21T05:16:07.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 --2- 172.21.15.7:0/3164016495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8770074f40 0x7f877010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 --2- 172.21.15.7:0/3164016495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074620 0x7f8770074a00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 -- 172.21.15.7:0/3164016495 >> 172.21.15.7:0/3164016495 conn(0x7f87700fd600 msgr2=0x7f87700ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 -- 172.21.15.7:0/3164016495 shutdown_connections 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 -- 172.21.15.7:0/3164016495 wait complete. 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.866+0000 7f8777294640 1 Processor -- start 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f8777294640 1 -- start start 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f8777294640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8770074620 0x7f877019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f8777294640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074f40 0x7f877019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f8777294640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8770110fd0 con 0x7f8770074f40 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f8777294640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8770110e50 con 0x7f8770074620 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f877482f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074f40 0x7f877019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f877482f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074f40 0x7f877019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33348/0 (socket says 172.21.15.7:33348) 2024-09-21T05:16:07.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f877482f640 1 -- 172.21.15.7:0/3486792100 learned_addr learned my addr 172.21.15.7:0/3486792100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f8775030640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8770074620 0x7f877019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f877482f640 1 -- 172.21.15.7:0/3486792100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8770074620 msgr2=0x7f877019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f877482f640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8770074620 0x7f877019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f877482f640 1 -- 172.21.15.7:0/3486792100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87701999e0 con 0x7f8770074f40 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.867+0000 7f877482f640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074f40 0x7f877019fd60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f876000d8d0 tx=0x7f876000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.868+0000 7f87667fc640 1 -- 172.21.15.7:0/3486792100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8760017070 con 0x7f8770074f40 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.868+0000 7f87667fc640 1 -- 172.21.15.7:0/3486792100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8760010460 con 0x7f8770074f40 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.868+0000 7f87667fc640 1 -- 172.21.15.7:0/3486792100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f876000f5f0 con 0x7f8770074f40 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.868+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8770199cd0 con 0x7f8770074f40 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.868+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877019a260 con 0x7f8770074f40 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.869+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8770075cd0 con 0x7f8770074f40 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.869+0000 7f87667fc640 1 -- 172.21.15.7:0/3486792100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f8760009a30 con 0x7f8770074f40 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.870+0000 7f87667fc640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f875c07af80 0x7f875c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:07.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.870+0000 7f87667fc640 1 -- 172.21.15.7:0/3486792100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f876009de50 con 0x7f8770074f40 2024-09-21T05:16:07.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.870+0000 7f8775030640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f875c07af80 0x7f875c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:07.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.871+0000 7f8775030640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f875c07af80 0x7f875c07d440 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f876c000c00 tx=0x7f876c031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:07.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:07.874+0000 7f87667fc640 1 -- 172.21.15.7:0/3486792100 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8760069470 con 0x7f8770074f40 2024-09-21T05:16:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.103+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) -- 0x7f877019af10 con 0x7f8770074f40 2024-09-21T05:16:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.104+0000 7f87667fc640 1 -- 172.21.15.7:0/3486792100 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v38) ==== 107+0+5288 (secure 0 0 0) 0x7f876006e320 con 0x7f8770074f40 2024-09-21T05:16:08.107 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:08.107 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"btime":"2024-09-21T04:43:26:694777+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34314,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/269195221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":269195221},{"type":"v1","addr":"172.21.15.7:6827","nonce":269195221}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34322,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/840879872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":840879872},{"type":"v1","addr":"172.21.15.7:6829","nonce":840879872}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":44275,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/606146784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":606146784},{"type":"v1","addr":"172.21.15.53:6827","nonce":606146784}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44283,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1786424362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1786424362},{"type":"v1","addr":"172.21.15.53:6825","nonce":1786424362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-09-21T05:16:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.106+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f875c07af80 msgr2=0x7f875c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f875c07af80 0x7f875c07d440 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f876c000c00 tx=0x7f876c031400 comp rx=0 tx=0).stop 2024-09-21T05:16:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074f40 msgr2=0x7f877019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074f40 0x7f877019fd60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f876000d8d0 tx=0x7f876000dd90 comp rx=0 tx=0).stop 2024-09-21T05:16:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 shutdown_connections 2024-09-21T05:16:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f875c07af80 0x7f875c07d440 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8770074f40 0x7f877019fd60 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 --2- 172.21.15.7:0/3486792100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8770074620 0x7f877019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 >> 172.21.15.7:0/3486792100 conn(0x7f87700fd600 msgr2=0x7f877010b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 shutdown_connections 2024-09-21T05:16:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:08.107+0000 7f8777294640 1 -- 172.21.15.7:0/3486792100 wait complete. 2024-09-21T05:16:08.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:08 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3486792100' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 22, "format": "json"} : dispatch 2024-09-21T05:16:08.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:08 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3486792100' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 22, "format": "json"} : dispatch 2024-09-21T05:16:08.772 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:08.772 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-09-21T05:16:08.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 23 2024-09-21T05:16:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:09.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:09 smithi053 ceph-mon[80176]: pgmap v1245: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:09.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:09 smithi007 ceph-mon[108238]: pgmap v1245: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.549+0000 7f6307583640 1 -- 172.21.15.7:0/3993207044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300102b20 msgr2=0x7f6300102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.549+0000 7f6307583640 1 --2- 172.21.15.7:0/3993207044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300102b20 0x7f6300102f00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0098b0 tx=0x7f62fc02a560 comp rx=0 tx=0).stop 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.549+0000 7f6307583640 1 -- 172.21.15.7:0/3993207044 shutdown_connections 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.549+0000 7f6307583640 1 --2- 172.21.15.7:0/3993207044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6300103440 0x7f630010b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.549+0000 7f6307583640 1 --2- 172.21.15.7:0/3993207044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300102b20 0x7f6300102f00 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.549+0000 7f6307583640 1 -- 172.21.15.7:0/3993207044 >> 172.21.15.7:0/3993207044 conn(0x7f63000fc520 msgr2=0x7f63000fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.549+0000 7f6307583640 1 -- 172.21.15.7:0/3993207044 shutdown_connections 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.549+0000 7f6307583640 1 -- 172.21.15.7:0/3993207044 wait complete. 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6307583640 1 Processor -- start 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6307583640 1 -- start start 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6307583640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6300102b20 0x7f630019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6307583640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300103440 0x7f630019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6307583640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f630010ee50 con 0x7f6300103440 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6307583640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f630010ecd0 con 0x7f6300102b20 2024-09-21T05:16:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6304b1e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300103440 0x7f630019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6304b1e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300103440 0x7f630019dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33364/0 (socket says 172.21.15.7:33364) 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.550+0000 7f6304b1e640 1 -- 172.21.15.7:0/1635174973 learned_addr learned my addr 172.21.15.7:0/1635174973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f630531f640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6300102b20 0x7f630019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f6304b1e640 1 -- 172.21.15.7:0/1635174973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6300102b20 msgr2=0x7f630019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f6304b1e640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6300102b20 0x7f630019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f6304b1e640 1 -- 172.21.15.7:0/1635174973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6300197860 con 0x7f6300103440 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f6304b1e640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300103440 0x7f630019dbe0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f62f000b420 tx=0x7f62f000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f62f67fc640 1 -- 172.21.15.7:0/1635174973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62f0011630 con 0x7f6300103440 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f62f67fc640 1 -- 172.21.15.7:0/1635174973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f62f0016070 con 0x7f6300103440 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6300197b50 con 0x7f6300103440 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f62f67fc640 1 -- 172.21.15.7:0/1635174973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62f0010d40 con 0x7f6300103440 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.551+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001980e0 con 0x7f6300103440 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.552+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630006aa30 con 0x7f6300103440 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.553+0000 7f62f67fc640 1 -- 172.21.15.7:0/1635174973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f62f001b460 con 0x7f6300103440 2024-09-21T05:16:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.554+0000 7f62f67fc640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f62ec07af80 0x7f62ec07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.554+0000 7f630531f640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f62ec07af80 0x7f62ec07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.554+0000 7f62f67fc640 1 -- 172.21.15.7:0/1635174973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f62f009d750 con 0x7f6300103440 2024-09-21T05:16:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.554+0000 7f630531f640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f62ec07af80 0x7f62ec07d440 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009230 tx=0x7f62fc031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.558+0000 7f62f67fc640 1 -- 172.21.15.7:0/1635174973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62f0021250 con 0x7f6300103440 2024-09-21T05:16:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.786+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) -- 0x7f6300105fe0 con 0x7f6300103440 2024-09-21T05:16:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.787+0000 7f62f67fc640 1 -- 172.21.15.7:0/1635174973 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v38) ==== 107+0+4447 (secure 0 0 0) 0x7f62f0068d70 con 0x7f6300103440 2024-09-21T05:16:09.790 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:09.790 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"btime":"2024-09-21T04:43:29:471342+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34322,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/840879872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":840879872},{"type":"v1","addr":"172.21.15.7:6829","nonce":840879872}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":44275,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/606146784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":606146784},{"type":"v1","addr":"172.21.15.53:6827","nonce":606146784}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44283,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1786424362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1786424362},{"type":"v1","addr":"172.21.15.53:6825","nonce":1786424362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-09-21T05:16:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.789+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f62ec07af80 msgr2=0x7f62ec07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.789+0000 7f6307583640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f62ec07af80 0x7f62ec07d440 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009230 tx=0x7f62fc031400 comp rx=0 tx=0).stop 2024-09-21T05:16:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.789+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300103440 msgr2=0x7f630019dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.789+0000 7f6307583640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300103440 0x7f630019dbe0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f62f000b420 tx=0x7f62f000b8e0 comp rx=0 tx=0).stop 2024-09-21T05:16:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.790+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 shutdown_connections 2024-09-21T05:16:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.790+0000 7f6307583640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f62ec07af80 0x7f62ec07d440 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.790+0000 7f6307583640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300103440 0x7f630019dbe0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.790+0000 7f6307583640 1 --2- 172.21.15.7:0/1635174973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6300102b20 0x7f630019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.790+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 >> 172.21.15.7:0/1635174973 conn(0x7f63000fc520 msgr2=0x7f6300108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.790+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 shutdown_connections 2024-09-21T05:16:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:09.790+0000 7f6307583640 1 -- 172.21.15.7:0/1635174973 wait complete. 2024-09-21T05:16:10.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:10 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1635174973' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 23, "format": "json"} : dispatch 2024-09-21T05:16:10.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:10 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1635174973' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 23, "format": "json"} : dispatch 2024-09-21T05:16:10.510 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:10.511 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-09-21T05:16:10.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 24 2024-09-21T05:16:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 -- 172.21.15.7:0/4114663194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c107900 msgr2=0x7f936c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 --2- 172.21.15.7:0/4114663194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c107900 0x7f936c107ce0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f935c009880 tx=0x7f935c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 -- 172.21.15.7:0/4114663194 shutdown_connections 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 --2- 172.21.15.7:0/4114663194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c101900 0x7f936c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 --2- 172.21.15.7:0/4114663194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c107900 0x7f936c107ce0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 -- 172.21.15.7:0/4114663194 >> 172.21.15.7:0/4114663194 conn(0x7f936c0fd600 msgr2=0x7f936c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 -- 172.21.15.7:0/4114663194 shutdown_connections 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 -- 172.21.15.7:0/4114663194 wait complete. 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.238+0000 7f9370df7640 1 Processor -- start 2024-09-21T05:16:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f9370df7640 1 -- start start 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f9370df7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c101900 0x7f936c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f9370df7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c107900 0x7f936c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f9370df7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f936c10edb0 con 0x7f936c101900 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f9370df7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f936c10ec30 con 0x7f936c107900 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f936a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c101900 0x7f936c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f936a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c101900 0x7f936c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33368/0 (socket says 172.21.15.7:33368) 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f936a59c640 1 -- 172.21.15.7:0/3474292574 learned_addr learned my addr 172.21.15.7:0/3474292574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f9361d9b640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c107900 0x7f936c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f936a59c640 1 -- 172.21.15.7:0/3474292574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c107900 msgr2=0x7f936c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f936a59c640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c107900 0x7f936c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f936a59c640 1 -- 172.21.15.7:0/3474292574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c074f70 con 0x7f936c101900 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.239+0000 7f9361d9b640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c107900 0x7f936c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.240+0000 7f936a59c640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c101900 0x7f936c0744f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f935c031460 tx=0x7f935c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.240+0000 7f9363fff640 1 -- 172.21.15.7:0/3474292574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f935c03f070 con 0x7f936c101900 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.240+0000 7f9363fff640 1 -- 172.21.15.7:0/3474292574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f935c004cf0 con 0x7f936c101900 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.240+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c075200 con 0x7f936c101900 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.240+0000 7f9363fff640 1 -- 172.21.15.7:0/3474292574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f935c035030 con 0x7f936c101900 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.240+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c1a3d50 con 0x7f936c101900 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.241+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936c103040 con 0x7f936c101900 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.242+0000 7f9363fff640 1 -- 172.21.15.7:0/3474292574 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f935c042070 con 0x7f936c101900 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.242+0000 7f9363fff640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f935807af80 0x7f935807d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.242+0000 7f9363fff640 1 -- 172.21.15.7:0/3474292574 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f935c0bea90 con 0x7f936c101900 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.242+0000 7f9361d9b640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f935807af80 0x7f935807d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.243+0000 7f9361d9b640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f935807af80 0x7f935807d440 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f936c075f80 tx=0x7f9354005d90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.247+0000 7f9363fff640 1 -- 172.21.15.7:0/3474292574 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f935c08a0b0 con 0x7f936c101900 2024-09-21T05:16:11.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:11 smithi053 ceph-mon[80176]: pgmap v1246: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:11.443 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:11 smithi007 ceph-mon[108238]: pgmap v1246: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.472+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) -- 0x7f936c104b10 con 0x7f936c101900 2024-09-21T05:16:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.472+0000 7f9363fff640 1 -- 172.21.15.7:0/3474292574 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v38) ==== 107+0+5291 (secure 0 0 0) 0x7f935c08ef60 con 0x7f936c101900 2024-09-21T05:16:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:11.475 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"btime":"2024-09-21T04:43:35:494166+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34322,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/840879872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":840879872},{"type":"v1","addr":"172.21.15.7:6829","nonce":840879872}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44275,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/606146784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":606146784},{"type":"v1","addr":"172.21.15.53:6827","nonce":606146784}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44283,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1786424362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1786424362},{"type":"v1","addr":"172.21.15.53:6825","nonce":1786424362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-21T05:16:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.474+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f935807af80 msgr2=0x7f935807d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.474+0000 7f9370df7640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f935807af80 0x7f935807d440 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f936c075f80 tx=0x7f9354005d90 comp rx=0 tx=0).stop 2024-09-21T05:16:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.474+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c101900 msgr2=0x7f936c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.474+0000 7f9370df7640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c101900 0x7f936c0744f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f935c031460 tx=0x7f935c0048f0 comp rx=0 tx=0).stop 2024-09-21T05:16:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.474+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 shutdown_connections 2024-09-21T05:16:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.474+0000 7f9370df7640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f935807af80 0x7f935807d440 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.474+0000 7f9370df7640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f936c107900 0x7f936c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.474+0000 7f9370df7640 1 --2- 172.21.15.7:0/3474292574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f936c101900 0x7f936c0744f0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.475+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 >> 172.21.15.7:0/3474292574 conn(0x7f936c0fd600 msgr2=0x7f936c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.475+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 shutdown_connections 2024-09-21T05:16:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:11.475+0000 7f9370df7640 1 -- 172.21.15.7:0/3474292574 wait complete. 2024-09-21T05:16:12.183 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:12.183 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-09-21T05:16:12.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 25 2024-09-21T05:16:12.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:12 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3474292574' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 24, "format": "json"} : dispatch 2024-09-21T05:16:12.443 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:12 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3474292574' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 24, "format": "json"} : dispatch 2024-09-21T05:16:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 -- 172.21.15.7:0/3590248068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 msgr2=0x7fea1c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 --2- 172.21.15.7:0/3590248068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 0x7fea1c106be0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fea18009880 tx=0x7fea1802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 -- 172.21.15.7:0/3590248068 shutdown_connections 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 --2- 172.21.15.7:0/3590248068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1c100800 0x7fea1c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 --2- 172.21.15.7:0/3590248068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 0x7fea1c106be0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 -- 172.21.15.7:0/3590248068 >> 172.21.15.7:0/3590248068 conn(0x7fea1c0fc520 msgr2=0x7fea1c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 -- 172.21.15.7:0/3590248068 shutdown_connections 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 -- 172.21.15.7:0/3590248068 wait complete. 2024-09-21T05:16:12.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 Processor -- start 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.958+0000 7fea238ce640 1 -- start start 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea238ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1c100800 0x7fea1c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea238ce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 0x7fea1c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea238ce640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c10dcd0 con 0x7fea1c106800 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea238ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c10db50 con 0x7fea1c100800 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea20e69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 0x7fea1c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea20e69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 0x7fea1c19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33382/0 (socket says 172.21.15.7:33382) 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea20e69640 1 -- 172.21.15.7:0/2607981791 learned_addr learned my addr 172.21.15.7:0/2607981791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea2166a640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1c100800 0x7fea1c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea20e69640 1 -- 172.21.15.7:0/2607981791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1c100800 msgr2=0x7fea1c19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea20e69640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1c100800 0x7fea1c19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea20e69640 1 -- 172.21.15.7:0/2607981791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c1977e0 con 0x7fea1c106800 2024-09-21T05:16:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea20e69640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 0x7fea1c19dba0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c8f0 tx=0x7fea0c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea127fc640 1 -- 172.21.15.7:0/2607981791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fea0c0079b0 con 0x7fea1c106800 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea127fc640 1 -- 172.21.15.7:0/2607981791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea0c0054e0 con 0x7fea1c106800 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1c197ad0 con 0x7fea1c106800 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea127fc640 1 -- 172.21.15.7:0/2607981791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fea0c004dd0 con 0x7fea1c106800 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.959+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea1c198060 con 0x7fea1c106800 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.960+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1c101f40 con 0x7fea1c106800 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.961+0000 7fea127fc640 1 -- 172.21.15.7:0/2607981791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fea0c005020 con 0x7fea1c106800 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.961+0000 7fea127fc640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea0807af30 0x7fea0807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.962+0000 7fea127fc640 1 -- 172.21.15.7:0/2607981791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fea0c09df00 con 0x7fea1c106800 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.962+0000 7fea2166a640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea0807af30 0x7fea0807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.962+0000 7fea2166a640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea0807af30 0x7fea0807d3f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fea18031460 tx=0x7fea18005bd0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:12.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:12.966+0000 7fea127fc640 1 -- 172.21.15.7:0/2607981791 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea0c014030 con 0x7fea1c106800 2024-09-21T05:16:13.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.192+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) -- 0x7fea1c109d80 con 0x7fea1c106800 2024-09-21T05:16:13.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.192+0000 7fea127fc640 1 -- 172.21.15.7:0/2607981791 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v38) ==== 107+0+4450 (secure 0 0 0) 0x7fea0c069520 con 0x7fea1c106800 2024-09-21T05:16:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:13.195 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"btime":"2024-09-21T04:43:47:192651+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44275,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/606146784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":606146784},{"type":"v1","addr":"172.21.15.53:6827","nonce":606146784}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44283,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1786424362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1786424362},{"type":"v1","addr":"172.21.15.53:6825","nonce":1786424362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:13.195 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-09-21T05:16:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea0807af30 msgr2=0x7fea0807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea0807af30 0x7fea0807d3f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fea18031460 tx=0x7fea18005bd0 comp rx=0 tx=0).stop 2024-09-21T05:16:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 msgr2=0x7fea1c19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 0x7fea1c19dba0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c8f0 tx=0x7fea0c00cdb0 comp rx=0 tx=0).stop 2024-09-21T05:16:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 shutdown_connections 2024-09-21T05:16:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fea0807af30 0x7fea0807d3f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea1c106800 0x7fea1c19dba0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 --2- 172.21.15.7:0/2607981791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea1c100800 0x7fea1c19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.194+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 >> 172.21.15.7:0/2607981791 conn(0x7fea1c0fc520 msgr2=0x7fea1c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.195+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 shutdown_connections 2024-09-21T05:16:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:13.195+0000 7fea238ce640 1 -- 172.21.15.7:0/2607981791 wait complete. 2024-09-21T05:16:13.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:13 smithi053 ceph-mon[80176]: pgmap v1247: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:13.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:13 smithi007 ceph-mon[108238]: pgmap v1247: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:13.845 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:13.845 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-09-21T05:16:13.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 26 2024-09-21T05:16:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:14 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2607981791' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 25, "format": "json"} : dispatch 2024-09-21T05:16:14.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:16:14.443 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:14 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2607981791' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 25, "format": "json"} : dispatch 2024-09-21T05:16:14.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:16:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 -- 172.21.15.7:0/4010807068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d101033d0 msgr2=0x7f3d1010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 --2- 172.21.15.7:0/4010807068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d101033d0 0x7f3d1010b8c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009880 tx=0x7f3d0002a530 comp rx=0 tx=0).stop 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 -- 172.21.15.7:0/4010807068 shutdown_connections 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 --2- 172.21.15.7:0/4010807068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d101033d0 0x7f3d1010b8c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 --2- 172.21.15.7:0/4010807068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d10102ab0 0x7f3d10102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 -- 172.21.15.7:0/4010807068 >> 172.21.15.7:0/4010807068 conn(0x7f3d100fc550 msgr2=0x7f3d100fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 -- 172.21.15.7:0/4010807068 shutdown_connections 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 -- 172.21.15.7:0/4010807068 wait complete. 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.575+0000 7f3d176a3640 1 Processor -- start 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d176a3640 1 -- start start 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d176a3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d10102ab0 0x7f3d10100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d176a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d101033d0 0x7f3d10100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d176a3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010ee40 con 0x7f3d10102ab0 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d176a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010ecc0 con 0x7f3d101033d0 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d1543f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d10102ab0 0x7f3d10100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d1543f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d10102ab0 0x7f3d10100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48432/0 (socket says 172.21.15.7:48432) 2024-09-21T05:16:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d1543f640 1 -- 172.21.15.7:0/3412392940 learned_addr learned my addr 172.21.15.7:0/3412392940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d14c3e640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d101033d0 0x7f3d10100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d1543f640 1 -- 172.21.15.7:0/3412392940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d101033d0 msgr2=0x7f3d10100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d1543f640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d101033d0 0x7f3d10100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d1543f640 1 -- 172.21.15.7:0/3412392940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d10101190 con 0x7f3d10102ab0 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d14c3e640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d101033d0 0x7f3d10100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.576+0000 7f3d1543f640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d10102ab0 0x7f3d10100710 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00b420 tx=0x7f3d0c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.577+0000 7f3d067fc640 1 -- 172.21.15.7:0/3412392940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3d0c011630 con 0x7f3d10102ab0 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.577+0000 7f3d067fc640 1 -- 172.21.15.7:0/3412392940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3d0c016070 con 0x7f3d10102ab0 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.577+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d10101480 con 0x7f3d10102ab0 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.577+0000 7f3d067fc640 1 -- 172.21.15.7:0/3412392940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3d0c010d40 con 0x7f3d10102ab0 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.577+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101a3eb0 con 0x7f3d10102ab0 2024-09-21T05:16:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.578+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d1006a580 con 0x7f3d10102ab0 2024-09-21T05:16:14.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.578+0000 7f3d067fc640 1 -- 172.21.15.7:0/3412392940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3d0c01b460 con 0x7f3d10102ab0 2024-09-21T05:16:14.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.579+0000 7f3d067fc640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cfc07af80 0x7f3cfc07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:14.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.579+0000 7f3d067fc640 1 -- 172.21.15.7:0/3412392940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f3d0c09d700 con 0x7f3d10102ab0 2024-09-21T05:16:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.579+0000 7f3d14c3e640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cfc07af80 0x7f3cfc07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.580+0000 7f3d14c3e640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cfc07af80 0x7f3cfc07d440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3d101021a0 tx=0x7f3d00031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.584+0000 7f3d067fc640 1 -- 172.21.15.7:0/3412392940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3d0c021250 con 0x7f3d10102ab0 2024-09-21T05:16:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.811+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) -- 0x7f3d10105f50 con 0x7f3d10102ab0 2024-09-21T05:16:14.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.811+0000 7f3d067fc640 1 -- 172.21.15.7:0/3412392940 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v38) ==== 107+0+5294 (secure 0 0 0) 0x7f3d0c068d20 con 0x7f3d10102ab0 2024-09-21T05:16:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:14.814 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-21T04:43:51:891103+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44275,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/606146784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":606146784},{"type":"v1","addr":"172.21.15.53:6827","nonce":606146784}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44283,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1786424362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1786424362},{"type":"v1","addr":"172.21.15.53:6825","nonce":1786424362}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:14.815 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-21T05:16:14.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cfc07af80 msgr2=0x7f3cfc07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cfc07af80 0x7f3cfc07d440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3d101021a0 tx=0x7f3d00031400 comp rx=0 tx=0).stop 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d10102ab0 msgr2=0x7f3d10100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d10102ab0 0x7f3d10100710 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00b420 tx=0x7f3d0c00b8e0 comp rx=0 tx=0).stop 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 shutdown_connections 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3cfc07af80 0x7f3cfc07d440 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d101033d0 0x7f3d10100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 --2- 172.21.15.7:0/3412392940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d10102ab0 0x7f3d10100710 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 >> 172.21.15.7:0/3412392940 conn(0x7f3d100fc550 msgr2=0x7f3d10108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 shutdown_connections 2024-09-21T05:16:14.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:14.814+0000 7f3d176a3640 1 -- 172.21.15.7:0/3412392940 wait complete. 2024-09-21T05:16:15.398 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:15 smithi007 ceph-mon[108238]: pgmap v1248: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:15.399 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:15 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3412392940' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 26, "format": "json"} : dispatch 2024-09-21T05:16:15.426 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:15.426 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-09-21T05:16:15.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 27 2024-09-21T05:16:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:15 smithi053 ceph-mon[80176]: pgmap v1248: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:15.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:15 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3412392940' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 26, "format": "json"} : dispatch 2024-09-21T05:16:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.142+0000 7f58766d5640 1 -- 172.21.15.7:0/2224732283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58701033d0 msgr2=0x7f587010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.142+0000 7f58766d5640 1 --2- 172.21.15.7:0/2224732283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58701033d0 0x7f587010b8c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5858009880 tx=0x7f585802a530 comp rx=0 tx=0).stop 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.142+0000 7f58766d5640 1 -- 172.21.15.7:0/2224732283 shutdown_connections 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.142+0000 7f58766d5640 1 --2- 172.21.15.7:0/2224732283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58701033d0 0x7f587010b8c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.142+0000 7f58766d5640 1 --2- 172.21.15.7:0/2224732283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5870102ab0 0x7f5870102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.142+0000 7f58766d5640 1 -- 172.21.15.7:0/2224732283 >> 172.21.15.7:0/2224732283 conn(0x7f58700fc550 msgr2=0x7f58700fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.142+0000 7f58766d5640 1 -- 172.21.15.7:0/2224732283 shutdown_connections 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.142+0000 7f58766d5640 1 -- 172.21.15.7:0/2224732283 wait complete. 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f58766d5640 1 Processor -- start 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f58766d5640 1 -- start start 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f58766d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5870102ab0 0x7f5870100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f58766d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58701033d0 0x7f5870100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f58766d5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f587010ede0 con 0x7f5870102ab0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f58766d5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f587010ec60 con 0x7f58701033d0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f586ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5870102ab0 0x7f5870100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f586ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5870102ab0 0x7f5870100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48454/0 (socket says 172.21.15.7:48454) 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f586ffff640 1 -- 172.21.15.7:0/1430669897 learned_addr learned my addr 172.21.15.7:0/1430669897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f5867fff640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58701033d0 0x7f5870100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f586ffff640 1 -- 172.21.15.7:0/1430669897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58701033d0 msgr2=0x7f5870100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f586ffff640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58701033d0 0x7f5870100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.143+0000 7f586ffff640 1 -- 172.21.15.7:0/1430669897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5870101190 con 0x7f5870102ab0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.144+0000 7f586ffff640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5870102ab0 0x7f5870100710 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f586000d6f0 tx=0x7f586000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.144+0000 7f586dffb640 1 -- 172.21.15.7:0/1430669897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5860017070 con 0x7f5870102ab0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.144+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5870101480 con 0x7f5870102ab0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.144+0000 7f586dffb640 1 -- 172.21.15.7:0/1430669897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f586000de00 con 0x7f5870102ab0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.144+0000 7f586dffb640 1 -- 172.21.15.7:0/1430669897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5860004bb0 con 0x7f5870102ab0 2024-09-21T05:16:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.144+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701a3e90 con 0x7f5870102ab0 2024-09-21T05:16:16.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.145+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587006a580 con 0x7f5870102ab0 2024-09-21T05:16:16.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.145+0000 7f586dffb640 1 -- 172.21.15.7:0/1430669897 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5860004e60 con 0x7f5870102ab0 2024-09-21T05:16:16.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.146+0000 7f586dffb640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f585c07af40 0x7f585c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:16.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.146+0000 7f5867fff640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f585c07af40 0x7f585c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:16.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.146+0000 7f586dffb640 1 -- 172.21.15.7:0/1430669897 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f586009dbe0 con 0x7f5870102ab0 2024-09-21T05:16:16.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.147+0000 7f5867fff640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f585c07af40 0x7f585c07d400 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f58701021a0 tx=0x7f5858031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:16.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.151+0000 7f586dffb640 1 -- 172.21.15.7:0/1430669897 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5860069180 con 0x7f5870102ab0 2024-09-21T05:16:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.377+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) -- 0x7f5870102e90 con 0x7f5870102ab0 2024-09-21T05:16:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.377+0000 7f586dffb640 1 -- 172.21.15.7:0/1430669897 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v38) ==== 107+0+4447 (secure 0 0 0) 0x7f586006e030 con 0x7f5870102ab0 2024-09-21T05:16:16.380 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:16.380 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"btime":"2024-09-21T04:44:03:751154+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44275,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/606146784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":606146784},{"type":"v1","addr":"172.21.15.53:6827","nonce":606146784}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:16.381 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-09-21T05:16:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.379+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f585c07af40 msgr2=0x7f585c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.379+0000 7f58766d5640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f585c07af40 0x7f585c07d400 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f58701021a0 tx=0x7f5858031400 comp rx=0 tx=0).stop 2024-09-21T05:16:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.379+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5870102ab0 msgr2=0x7f5870100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.379+0000 7f58766d5640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5870102ab0 0x7f5870100710 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f586000d6f0 tx=0x7f586000dbb0 comp rx=0 tx=0).stop 2024-09-21T05:16:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.380+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 shutdown_connections 2024-09-21T05:16:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.380+0000 7f58766d5640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f585c07af40 0x7f585c07d400 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.380+0000 7f58766d5640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58701033d0 0x7f5870100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.380+0000 7f58766d5640 1 --2- 172.21.15.7:0/1430669897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5870102ab0 0x7f5870100710 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.380+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 >> 172.21.15.7:0/1430669897 conn(0x7f58700fc550 msgr2=0x7f5870108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.380+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 shutdown_connections 2024-09-21T05:16:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:16.380+0000 7f58766d5640 1 -- 172.21.15.7:0/1430669897 wait complete. 2024-09-21T05:16:16.681 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:16 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1430669897' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 27, "format": "json"} : dispatch 2024-09-21T05:16:16.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:16 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1430669897' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 27, "format": "json"} : dispatch 2024-09-21T05:16:17.030 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:17.030 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-09-21T05:16:17.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 28 2024-09-21T05:16:17.292 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:17.677 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:17 smithi007 ceph-mon[108238]: pgmap v1249: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:17 smithi053 ceph-mon[80176]: pgmap v1249: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 -- 172.21.15.7:0/822107901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8101900 msgr2=0x7f49f8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 --2- 172.21.15.7:0/822107901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8101900 0x7f49f8101d60 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009880 tx=0x7f49ec02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 -- 172.21.15.7:0/822107901 shutdown_connections 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 --2- 172.21.15.7:0/822107901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8101900 0x7f49f8101d60 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 --2- 172.21.15.7:0/822107901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8107900 0x7f49f8107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 -- 172.21.15.7:0/822107901 >> 172.21.15.7:0/822107901 conn(0x7f49f80fd600 msgr2=0x7f49f80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 -- 172.21.15.7:0/822107901 shutdown_connections 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 -- 172.21.15.7:0/822107901 wait complete. 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.757+0000 7f49fddf6640 1 Processor -- start 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49fddf6640 1 -- start start 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49fddf6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8101900 0x7f49f819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49fddf6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8107900 0x7f49f819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49fddf6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49f810edd0 con 0x7f49f8107900 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49fddf6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49f810ec50 con 0x7f49f8101900 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49f6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8107900 0x7f49f819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49f6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8107900 0x7f49f819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48482/0 (socket says 172.21.15.7:48482) 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49f6ffd640 1 -- 172.21.15.7:0/3561778063 learned_addr learned my addr 172.21.15.7:0/3561778063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49f77fe640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8101900 0x7f49f819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49f6ffd640 1 -- 172.21.15.7:0/3561778063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8101900 msgr2=0x7f49f819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49f6ffd640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8101900 0x7f49f819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49f6ffd640 1 -- 172.21.15.7:0/3561778063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81999d0 con 0x7f49f8107900 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.758+0000 7f49f6ffd640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8107900 0x7f49f819fd50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f49ec031460 tx=0x7f49ec0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.759+0000 7f49f4ff9640 1 -- 172.21.15.7:0/3561778063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49ec03f070 con 0x7f49f8107900 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.759+0000 7f49f4ff9640 1 -- 172.21.15.7:0/3561778063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49ec004cf0 con 0x7f49f8107900 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.759+0000 7f49f4ff9640 1 -- 172.21.15.7:0/3561778063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49ec035030 con 0x7f49f8107900 2024-09-21T05:16:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.759+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f8199c60 con 0x7f49f8107900 2024-09-21T05:16:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.759+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f819a190 con 0x7f49f8107900 2024-09-21T05:16:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.760+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f8103040 con 0x7f49f8107900 2024-09-21T05:16:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.760+0000 7f49f4ff9640 1 -- 172.21.15.7:0/3561778063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f49ec042070 con 0x7f49f8107900 2024-09-21T05:16:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.761+0000 7f49f4ff9640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49e007af80 0x7f49e007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.761+0000 7f49f77fe640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49e007af80 0x7f49e007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.761+0000 7f49f4ff9640 1 -- 172.21.15.7:0/3561778063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f49ec0bea40 con 0x7f49f8107900 2024-09-21T05:16:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.762+0000 7f49f77fe640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49e007af80 0x7f49e007d440 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005fd0 tx=0x7f49e8005e20 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.766+0000 7f49f4ff9640 1 -- 172.21.15.7:0/3561778063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49ec08a060 con 0x7f49f8107900 2024-09-21T05:16:17.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.991+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) -- 0x7f49f819afd0 con 0x7f49f8107900 2024-09-21T05:16:17.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.992+0000 7f49f4ff9640 1 -- 172.21.15.7:0/3561778063 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v38) ==== 107+0+5294 (secure 0 0 0) 0x7f49ec08ef10 con 0x7f49f8107900 2024-09-21T05:16:17.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:17.995 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"btime":"2024-09-21T04:44:07:773367+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44275,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/606146784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":606146784},{"type":"v1","addr":"172.21.15.53:6827","nonce":606146784}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-21T05:16:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49e007af80 msgr2=0x7f49e007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49e007af80 0x7f49e007d440 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005fd0 tx=0x7f49e8005e20 comp rx=0 tx=0).stop 2024-09-21T05:16:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8107900 msgr2=0x7f49f819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8107900 0x7f49f819fd50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f49ec031460 tx=0x7f49ec0048f0 comp rx=0 tx=0).stop 2024-09-21T05:16:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 shutdown_connections 2024-09-21T05:16:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f49e007af80 0x7f49e007d440 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f8107900 0x7f49f819fd50 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 --2- 172.21.15.7:0/3561778063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f8101900 0x7f49f819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 >> 172.21.15.7:0/3561778063 conn(0x7f49f80fd600 msgr2=0x7f49f8074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 shutdown_connections 2024-09-21T05:16:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:17.995+0000 7f49fddf6640 1 -- 172.21.15.7:0/3561778063 wait complete. 2024-09-21T05:16:18.646 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:18.646 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-09-21T05:16:18.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 29 2024-09-21T05:16:18.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:18 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3561778063' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 28, "format": "json"} : dispatch 2024-09-21T05:16:18.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:18 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3561778063' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 28, "format": "json"} : dispatch 2024-09-21T05:16:18.909 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2012455113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 msgr2=0x7f0ac4102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 --2- 172.21.15.7:0/2012455113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 0x7f0ac4102f50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009880 tx=0x7f0ab402a530 comp rx=0 tx=0).stop 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2012455113 shutdown_connections 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 --2- 172.21.15.7:0/2012455113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4103490 0x7f0ac410b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 --2- 172.21.15.7:0/2012455113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 0x7f0ac4102f50 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2012455113 >> 172.21.15.7:0/2012455113 conn(0x7f0ac40fc550 msgr2=0x7f0ac40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2012455113 shutdown_connections 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2012455113 wait complete. 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 Processor -- start 2024-09-21T05:16:19.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.333+0000 7f0aca2d9640 1 -- start start 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0aca2d9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 0x7f0ac419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0aca2d9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4103490 0x7f0ac419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0aca2d9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410edc0 con 0x7f0ac4102b70 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0aca2d9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410ec40 con 0x7f0ac4103490 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 0x7f0ac419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 0x7f0ac419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48512/0 (socket says 172.21.15.7:48512) 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac3fff640 1 -- 172.21.15.7:0/2385109156 learned_addr learned my addr 172.21.15.7:0/2385109156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac37fe640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4103490 0x7f0ac419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac3fff640 1 -- 172.21.15.7:0/2385109156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4103490 msgr2=0x7f0ac419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac3fff640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4103490 0x7f0ac419e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac3fff640 1 -- 172.21.15.7:0/2385109156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac419eb60 con 0x7f0ac4102b70 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac3fff640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 0x7f0ac419df70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4004f90 tx=0x7f0ab40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.334+0000 7f0ac17fa640 1 -- 172.21.15.7:0/2385109156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ab403f070 con 0x7f0ac4102b70 2024-09-21T05:16:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.335+0000 7f0ac17fa640 1 -- 172.21.15.7:0/2385109156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ab4004cf0 con 0x7f0ac4102b70 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.335+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac41a28a0 con 0x7f0ac4102b70 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.335+0000 7f0ac17fa640 1 -- 172.21.15.7:0/2385109156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ab4035030 con 0x7f0ac4102b70 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.335+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41a2dd0 con 0x7f0ac4102b70 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.336+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac406a4c0 con 0x7f0ac4102b70 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.336+0000 7f0ac17fa640 1 -- 172.21.15.7:0/2385109156 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f0ab4042070 con 0x7f0ac4102b70 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.337+0000 7f0ac17fa640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0aac07af90 0x7f0aac07d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.337+0000 7f0ac37fe640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0aac07af90 0x7f0aac07d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.337+0000 7f0ac17fa640 1 -- 172.21.15.7:0/2385109156 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f0ab40beac0 con 0x7f0ac4102b70 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.337+0000 7f0ac37fe640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0aac07af90 0x7f0aac07d450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4102760 tx=0x7f0ab8005cc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.341+0000 7f0ac17fa640 1 -- 172.21.15.7:0/2385109156 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ab408a0e0 con 0x7f0ac4102b70 2024-09-21T05:16:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.567+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) -- 0x7f0ac4106b00 con 0x7f0ac4102b70 2024-09-21T05:16:19.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.568+0000 7f0ac17fa640 1 -- 172.21.15.7:0/2385109156 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v38) ==== 107+0+4450 (secure 0 0 0) 0x7f0ab408ef90 con 0x7f0ac4102b70 2024-09-21T05:16:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:19.572 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"btime":"2024-09-21T04:59:11:187816+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0aac07af90 msgr2=0x7f0aac07d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0aac07af90 0x7f0aac07d450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4102760 tx=0x7f0ab8005cc0 comp rx=0 tx=0).stop 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 msgr2=0x7f0ac419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 0x7f0ac419df70 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4004f90 tx=0x7f0ab40048f0 comp rx=0 tx=0).stop 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 shutdown_connections 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f0aac07af90 0x7f0aac07d450 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ac4103490 0x7f0ac419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 --2- 172.21.15.7:0/2385109156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac4102b70 0x7f0ac419df70 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 >> 172.21.15.7:0/2385109156 conn(0x7f0ac40fc550 msgr2=0x7f0ac4100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:19.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 shutdown_connections 2024-09-21T05:16:19.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:19.570+0000 7f0aca2d9640 1 -- 172.21.15.7:0/2385109156 wait complete. 2024-09-21T05:16:19.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:19 smithi053 ceph-mon[80176]: pgmap v1250: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:19.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:19 smithi007 ceph-mon[108238]: pgmap v1250: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:20.495 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:20.495 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-09-21T05:16:20.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 30 2024-09-21T05:16:20.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:20 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2385109156' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 29, "format": "json"} : dispatch 2024-09-21T05:16:20.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:20 smithi053 ceph-mon[80176]: pgmap v1251: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:20.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:20 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2385109156' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 29, "format": "json"} : dispatch 2024-09-21T05:16:20.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:20 smithi007 ceph-mon[108238]: pgmap v1251: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:21.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.199+0000 7fe9dde99640 1 -- 172.21.15.7:0/1217869544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 msgr2=0x7fe9d8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.199+0000 7fe9dde99640 1 --2- 172.21.15.7:0/1217869544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 0x7fe9d8106be0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009880 tx=0x7fe9c802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.199+0000 7fe9dde99640 1 -- 172.21.15.7:0/1217869544 shutdown_connections 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.199+0000 7fe9dde99640 1 --2- 172.21.15.7:0/1217869544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9d8100800 0x7fe9d8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.199+0000 7fe9dde99640 1 --2- 172.21.15.7:0/1217869544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 0x7fe9d8106be0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.199+0000 7fe9dde99640 1 -- 172.21.15.7:0/1217869544 >> 172.21.15.7:0/1217869544 conn(0x7fe9d80fc520 msgr2=0x7fe9d80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.199+0000 7fe9dde99640 1 -- 172.21.15.7:0/1217869544 shutdown_connections 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.200+0000 7fe9dde99640 1 -- 172.21.15.7:0/1217869544 wait complete. 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.200+0000 7fe9dde99640 1 Processor -- start 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.200+0000 7fe9dde99640 1 -- start start 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9dde99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9d8100800 0x7fe9d8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9dde99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 0x7fe9d8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9dde99640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d810dd30 con 0x7fe9d8106800 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9dde99640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d810dbb0 con 0x7fe9d8100800 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 0x7fe9d8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:21.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 0x7fe9d8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48518/0 (socket says 172.21.15.7:48518) 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d6ffd640 1 -- 172.21.15.7:0/3048919545 learned_addr learned my addr 172.21.15.7:0/3048919545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d77fe640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9d8100800 0x7fe9d8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d6ffd640 1 -- 172.21.15.7:0/3048919545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9d8100800 msgr2=0x7fe9d8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d6ffd640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9d8100800 0x7fe9d8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d6ffd640 1 -- 172.21.15.7:0/3048919545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d81993b0 con 0x7fe9d8106800 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d77fe640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9d8100800 0x7fe9d8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.201+0000 7fe9d6ffd640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 0x7fe9d8198db0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00b420 tx=0x7fe9cc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.202+0000 7fe9d4ff9640 1 -- 172.21.15.7:0/3048919545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe9cc011630 con 0x7fe9d8106800 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.202+0000 7fe9d4ff9640 1 -- 172.21.15.7:0/3048919545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9cc016070 con 0x7fe9d8106800 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.202+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d81996a0 con 0x7fe9d8106800 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.202+0000 7fe9d4ff9640 1 -- 172.21.15.7:0/3048919545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe9cc010d40 con 0x7fe9d8106800 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.202+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d81a1ca0 con 0x7fe9d8106800 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.203+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d8101f40 con 0x7fe9d8106800 2024-09-21T05:16:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.203+0000 7fe9d4ff9640 1 -- 172.21.15.7:0/3048919545 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe9cc01b460 con 0x7fe9d8106800 2024-09-21T05:16:21.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.204+0000 7fe9d4ff9640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe9c007af30 0x7fe9c007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:21.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.204+0000 7fe9d4ff9640 1 -- 172.21.15.7:0/3048919545 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fe9cc09d7a0 con 0x7fe9d8106800 2024-09-21T05:16:21.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.204+0000 7fe9d77fe640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe9c007af30 0x7fe9c007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:21.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.204+0000 7fe9d77fe640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe9c007af30 0x7fe9c007d3f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006010 tx=0x7fe9c80095f0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:21.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.208+0000 7fe9d4ff9640 1 -- 172.21.15.7:0/3048919545 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9d81996a0 con 0x7fe9d8106800 2024-09-21T05:16:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.437+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) -- 0x7fe9d8108d70 con 0x7fe9d8106800 2024-09-21T05:16:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.438+0000 7fe9d4ff9640 1 -- 172.21.15.7:0/3048919545 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v38) ==== 107+0+5297 (secure 0 0 0) 0x7fe9cc021250 con 0x7fe9d8106800 2024-09-21T05:16:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:21.442 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"btime":"2024-09-21T04:59:15:494001+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T04:43:25.693123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:21.442 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-21T05:16:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe9c007af30 msgr2=0x7fe9c007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe9c007af30 0x7fe9c007d3f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006010 tx=0x7fe9c80095f0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 msgr2=0x7fe9d8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 0x7fe9d8198db0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00b420 tx=0x7fe9cc00b8e0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 shutdown_connections 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe9c007af30 0x7fe9c007d3f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d8106800 0x7fe9d8198db0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 --2- 172.21.15.7:0/3048919545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9d8100800 0x7fe9d8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 >> 172.21.15.7:0/3048919545 conn(0x7fe9d80fc520 msgr2=0x7fe9d80fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 shutdown_connections 2024-09-21T05:16:21.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:21.440+0000 7fe9dde99640 1 -- 172.21.15.7:0/3048919545 wait complete. 2024-09-21T05:16:21.454 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:21 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3048919545' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 30, "format": "json"} : dispatch 2024-09-21T05:16:21.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:21 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3048919545' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 30, "format": "json"} : dispatch 2024-09-21T05:16:22.154 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-21T05:16:22.155 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-09-21T05:16:22.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 31 2024-09-21T05:16:22.420 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:22.710 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:22 smithi007 ceph-mon[108238]: pgmap v1252: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:22.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.883+0000 7fc04a765640 1 -- 172.21.15.7:0/1850833878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441067f0 msgr2=0x7fc044106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:22.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.883+0000 7fc04a765640 1 --2- 172.21.15.7:0/1850833878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441067f0 0x7fc044106bd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc034009880 tx=0x7fc03402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:22.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.883+0000 7fc04a765640 1 -- 172.21.15.7:0/1850833878 shutdown_connections 2024-09-21T05:16:22.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.883+0000 7fc04a765640 1 --2- 172.21.15.7:0/1850833878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0441007f0 0x7fc044100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.883+0000 7fc04a765640 1 --2- 172.21.15.7:0/1850833878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441067f0 0x7fc044106bd0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.883+0000 7fc04a765640 1 -- 172.21.15.7:0/1850833878 >> 172.21.15.7:0/1850833878 conn(0x7fc0440fc550 msgr2=0x7fc0440fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.883+0000 7fc04a765640 1 -- 172.21.15.7:0/1850833878 shutdown_connections 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.883+0000 7fc04a765640 1 -- 172.21.15.7:0/1850833878 wait complete. 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc04a765640 1 Processor -- start 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc04a765640 1 -- start start 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc04a765640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441007f0 0x7fc044198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc04a765640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0441067f0 0x7fc044198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc04a765640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc04410dd20 con 0x7fc0441007f0 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc04a765640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc04410dba0 con 0x7fc0441067f0 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc043fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441007f0 0x7fc044198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc043fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441007f0 0x7fc044198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48532/0 (socket says 172.21.15.7:48532) 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc043fff640 1 -- 172.21.15.7:0/54497490 learned_addr learned my addr 172.21.15.7:0/54497490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:22.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc043fff640 1 -- 172.21.15.7:0/54497490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0441067f0 msgr2=0x7fc044198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc043fff640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0441067f0 0x7fc044198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc043fff640 1 -- 172.21.15.7:0/54497490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0441994b0 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.884+0000 7fc043fff640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441007f0 0x7fc044198970 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc034031460 tx=0x7fc0340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.885+0000 7fc041ffb640 1 -- 172.21.15.7:0/54497490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc03403f070 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.885+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc044199740 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.885+0000 7fc041ffb640 1 -- 172.21.15.7:0/54497490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc034004cf0 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.885+0000 7fc041ffb640 1 -- 172.21.15.7:0/54497490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc034035030 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.885+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441a1cf0 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.886+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc044101f30 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.886+0000 7fc041ffb640 1 -- 172.21.15.7:0/54497490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fc034042070 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.887+0000 7fc041ffb640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc03007af80 0x7fc03007d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.887+0000 7fc041ffb640 1 -- 172.21.15.7:0/54497490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fc0340bea40 con 0x7fc0441007f0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.887+0000 7fc03bfff640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc03007af80 0x7fc03007d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.887+0000 7fc03bfff640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc03007af80 0x7fc03007d440 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc04406a7f0 tx=0x7fc02c006c90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:22.891+0000 7fc041ffb640 1 -- 172.21.15.7:0/54497490 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc03408a060 con 0x7fc0441007f0 2024-09-21T05:16:22.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:22 smithi053 ceph-mon[80176]: pgmap v1252: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:23.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.119+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) -- 0x7fc04406a290 con 0x7fc0441007f0 2024-09-21T05:16:23.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.120+0000 7fc041ffb640 1 -- 172.21.15.7:0/54497490 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v38) ==== 107+0+5296 (secure 0 0 0) 0x7fc03408ef10 con 0x7fc0441007f0 2024-09-21T05:16:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:23.123 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":31,"btime":"2024-09-21T05:14:16:744917+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T05:14:15.749024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:23.124 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 31 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc03007af80 msgr2=0x7fc03007d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc03007af80 0x7fc03007d440 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc04406a7f0 tx=0x7fc02c006c90 comp rx=0 tx=0).stop 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441007f0 msgr2=0x7fc044198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441007f0 0x7fc044198970 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc034031460 tx=0x7fc0340048f0 comp rx=0 tx=0).stop 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 shutdown_connections 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fc03007af80 0x7fc03007d440 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0441067f0 0x7fc044198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 --2- 172.21.15.7:0/54497490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0441007f0 0x7fc044198970 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 >> 172.21.15.7:0/54497490 conn(0x7fc0440fc550 msgr2=0x7fc0440fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 shutdown_connections 2024-09-21T05:16:23.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:23.122+0000 7fc04a765640 1 -- 172.21.15.7:0/54497490 wait complete. 2024-09-21T05:16:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:23 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/54497490' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 31, "format": "json"} : dispatch 2024-09-21T05:16:23.781 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-09-21T05:16:23.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 32 2024-09-21T05:16:23.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:23 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/54497490' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 31, "format": "json"} : dispatch 2024-09-21T05:16:24.045 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.642+0000 7fbb86878640 1 -- 172.21.15.7:0/1278949918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80103440 msgr2=0x7fbb8010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.642+0000 7fbb86878640 1 --2- 172.21.15.7:0/1278949918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80103440 0x7fbb8010b930 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbb740098b0 tx=0x7fbb7402a560 comp rx=0 tx=0).stop 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.642+0000 7fbb86878640 1 -- 172.21.15.7:0/1278949918 shutdown_connections 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.642+0000 7fbb86878640 1 --2- 172.21.15.7:0/1278949918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80103440 0x7fbb8010b930 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.642+0000 7fbb86878640 1 --2- 172.21.15.7:0/1278949918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb80102b20 0x7fbb80102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.642+0000 7fbb86878640 1 -- 172.21.15.7:0/1278949918 >> 172.21.15.7:0/1278949918 conn(0x7fbb800fc520 msgr2=0x7fbb800fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.642+0000 7fbb86878640 1 -- 172.21.15.7:0/1278949918 shutdown_connections 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.642+0000 7fbb86878640 1 -- 172.21.15.7:0/1278949918 wait complete. 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.643+0000 7fbb86878640 1 Processor -- start 2024-09-21T05:16:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.643+0000 7fbb86878640 1 -- start start 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.643+0000 7fbb86878640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80102b20 0x7fbb8019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.643+0000 7fbb86878640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb80103440 0x7fbb8019dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.643+0000 7fbb86878640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8010edf0 con 0x7fbb80102b20 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.643+0000 7fbb86878640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8010ec70 con 0x7fbb80103440 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80102b20 0x7fbb8019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80102b20 0x7fbb8019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49954/0 (socket says 172.21.15.7:49954) 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7ffff640 1 -- 172.21.15.7:0/3280542179 learned_addr learned my addr 172.21.15.7:0/3280542179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7f7fe640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb80103440 0x7fbb8019dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7ffff640 1 -- 172.21.15.7:0/3280542179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb80103440 msgr2=0x7fbb8019dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7ffff640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb80103440 0x7fbb8019dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7ffff640 1 -- 172.21.15.7:0/3280542179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb801978a0 con 0x7fbb80102b20 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7ffff640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80102b20 0x7fbb8019d6e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000c8f0 tx=0x7fbb7000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7d7fa640 1 -- 172.21.15.7:0/3280542179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb700079b0 con 0x7fbb80102b20 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7d7fa640 1 -- 172.21.15.7:0/3280542179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb700054e0 con 0x7fbb80102b20 2024-09-21T05:16:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb80197b90 con 0x7fbb80102b20 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb7d7fa640 1 -- 172.21.15.7:0/3280542179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb70004dd0 con 0x7fbb80102b20 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.644+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb801980f0 con 0x7fbb80102b20 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.646+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8006aa30 con 0x7fbb80102b20 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.646+0000 7fbb7d7fa640 1 -- 172.21.15.7:0/3280542179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fbb70005020 con 0x7fbb80102b20 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.647+0000 7fbb7d7fa640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb6807af30 0x7fbb6807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.647+0000 7fbb7f7fe640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb6807af30 0x7fbb6807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.647+0000 7fbb7d7fa640 1 -- 172.21.15.7:0/3280542179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fbb7009def0 con 0x7fbb80102b20 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.647+0000 7fbb7f7fe640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb6807af30 0x7fbb6807d3f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbb74000c00 tx=0x7fbb74031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.652+0000 7fbb7d7fa640 1 -- 172.21.15.7:0/3280542179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbb70014030 con 0x7fbb80102b20 2024-09-21T05:16:24.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:24 smithi007 ceph-mon[108238]: pgmap v1253: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.880+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) -- 0x7fbb80102f00 con 0x7fbb80102b20 2024-09-21T05:16:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.880+0000 7fbb7d7fa640 1 -- 172.21.15.7:0/3280542179 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v38) ==== 107+0+5319 (secure 0 0 0) 0x7fbb70069470 con 0x7fbb80102b20 2024-09-21T05:16:24.885 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:24.885 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"btime":"2024-09-21T05:14:16:749579+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T05:14:16.749564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34354,"mds_1":34358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34354":{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34358":{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":1,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:24.885 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-09-21T05:16:24.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb6807af30 msgr2=0x7fbb6807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb6807af30 0x7fbb6807d3f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbb74000c00 tx=0x7fbb74031400 comp rx=0 tx=0).stop 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80102b20 msgr2=0x7fbb8019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80102b20 0x7fbb8019d6e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000c8f0 tx=0x7fbb7000cdb0 comp rx=0 tx=0).stop 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 shutdown_connections 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fbb6807af30 0x7fbb6807d3f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb80103440 0x7fbb8019dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 --2- 172.21.15.7:0/3280542179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb80102b20 0x7fbb8019d6e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 >> 172.21.15.7:0/3280542179 conn(0x7fbb800fc520 msgr2=0x7fbb80103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 shutdown_connections 2024-09-21T05:16:24.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:24.883+0000 7fbb86878640 1 -- 172.21.15.7:0/3280542179 wait complete. 2024-09-21T05:16:24.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:24 smithi053 ceph-mon[80176]: pgmap v1253: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:25.586 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-09-21T05:16:25.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 33 2024-09-21T05:16:25.693 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:25 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/3280542179' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 32, "format": "json"} : dispatch 2024-09-21T05:16:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:25.868 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:25 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/3280542179' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 32, "format": "json"} : dispatch 2024-09-21T05:16:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.336+0000 7f14dc9ee640 1 -- 172.21.15.7:0/4191725088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 msgr2=0x7f14d8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.336+0000 7f14dc9ee640 1 --2- 172.21.15.7:0/4191725088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 0x7f14d8100c50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009880 tx=0x7f14c002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.336+0000 7f14dc9ee640 1 -- 172.21.15.7:0/4191725088 shutdown_connections 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.336+0000 7f14dc9ee640 1 --2- 172.21.15.7:0/4191725088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 0x7f14d8100c50 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.336+0000 7f14dc9ee640 1 --2- 172.21.15.7:0/4191725088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d81067f0 0x7f14d8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.336+0000 7f14dc9ee640 1 -- 172.21.15.7:0/4191725088 >> 172.21.15.7:0/4191725088 conn(0x7f14d80fc550 msgr2=0x7f14d80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.336+0000 7f14dc9ee640 1 -- 172.21.15.7:0/4191725088 shutdown_connections 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.336+0000 7f14dc9ee640 1 -- 172.21.15.7:0/4191725088 wait complete. 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14dc9ee640 1 Processor -- start 2024-09-21T05:16:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14dc9ee640 1 -- start start 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14dc9ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 0x7f14d81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14dc9ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d81067f0 0x7f14d8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14dc9ee640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14d810dcc0 con 0x7f14d81007f0 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14dc9ee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14d810db40 con 0x7f14d81067f0 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 0x7f14d81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 0x7f14d81987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49970/0 (socket says 172.21.15.7:49970) 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14d659c640 1 -- 172.21.15.7:0/772203908 learned_addr learned my addr 172.21.15.7:0/772203908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.337+0000 7f14d5d9b640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d81067f0 0x7f14d8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14d659c640 1 -- 172.21.15.7:0/772203908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d81067f0 msgr2=0x7f14d8198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14d659c640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d81067f0 0x7f14d8198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14d659c640 1 -- 172.21.15.7:0/772203908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d8199330 con 0x7f14d81007f0 2024-09-21T05:16:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14d659c640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 0x7f14d81987f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f14c800eb10 tx=0x7f14c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14cf7fe640 1 -- 172.21.15.7:0/772203908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14c800cce0 con 0x7f14d81007f0 2024-09-21T05:16:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d8199620 con 0x7f14d81007f0 2024-09-21T05:16:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14cf7fe640 1 -- 172.21.15.7:0/772203908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14c8005020 con 0x7f14d81007f0 2024-09-21T05:16:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d81a1c20 con 0x7f14d81007f0 2024-09-21T05:16:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.338+0000 7f14cf7fe640 1 -- 172.21.15.7:0/772203908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14c8010660 con 0x7f14d81007f0 2024-09-21T05:16:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.339+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b8005290 con 0x7f14d81007f0 2024-09-21T05:16:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.340+0000 7f14cf7fe640 1 -- 172.21.15.7:0/772203908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f14c8004a40 con 0x7f14d81007f0 2024-09-21T05:16:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.340+0000 7f14cf7fe640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14c407af40 0x7f14c407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.341+0000 7f14cf7fe640 1 -- 172.21.15.7:0/772203908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f14c809e830 con 0x7f14d81007f0 2024-09-21T05:16:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.341+0000 7f14d5d9b640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14c407af40 0x7f14c407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.341+0000 7f14d5d9b640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14c407af40 0x7f14c407d400 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f14d806a7f0 tx=0x7f14c00095c0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.345+0000 7f14cf7fe640 1 -- 172.21.15.7:0/772203908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14d8199620 con 0x7f14d81007f0 2024-09-21T05:16:26.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.587+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) -- 0x7f14b8005580 con 0x7f14d81007f0 2024-09-21T05:16:26.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.587+0000 7f14cf7fe640 1 -- 172.21.15.7:0/772203908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v38) ==== 107+0+5321 (secure 0 0 0) 0x7f14c8069dd0 con 0x7f14d81007f0 2024-09-21T05:16:26.591 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:26.591 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"btime":"2024-09-21T05:14:17:750537+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T05:14:16.761398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34354,"mds_1":34358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34354":{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34358":{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":1,"incarnation":32,"state":"up:resolve","state_seq":458,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:26.591 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14c407af40 msgr2=0x7f14c407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14c407af40 0x7f14c407d400 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f14d806a7f0 tx=0x7f14c00095c0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 msgr2=0x7f14d81987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 0x7f14d81987f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f14c800eb10 tx=0x7f14c800c2d0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 shutdown_connections 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f14c407af40 0x7f14c407d400 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14d81067f0 0x7f14d8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 --2- 172.21.15.7:0/772203908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d81007f0 0x7f14d81987f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 >> 172.21.15.7:0/772203908 conn(0x7f14d80fc550 msgr2=0x7f14d80fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 shutdown_connections 2024-09-21T05:16:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:26.590+0000 7f14dc9ee640 1 -- 172.21.15.7:0/772203908 wait complete. 2024-09-21T05:16:26.892 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:26 smithi007 ceph-mon[108238]: pgmap v1254: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:26.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:26 smithi053 ceph-mon[80176]: pgmap v1254: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:27.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 34 2024-09-21T05:16:27.473 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:27.712 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:27 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/772203908' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 33, "format": "json"} : dispatch 2024-09-21T05:16:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.899+0000 7fe650387640 1 -- 172.21.15.7:0/3253265742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe648102ac0 msgr2=0x7fe648102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.899+0000 7fe650387640 1 --2- 172.21.15.7:0/3253265742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe648102ac0 0x7fe648102ea0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe644009880 tx=0x7fe64402a530 comp rx=0 tx=0).stop 2024-09-21T05:16:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.899+0000 7fe650387640 1 -- 172.21.15.7:0/3253265742 shutdown_connections 2024-09-21T05:16:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.899+0000 7fe650387640 1 --2- 172.21.15.7:0/3253265742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6481033e0 0x7fe64810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.899+0000 7fe650387640 1 --2- 172.21.15.7:0/3253265742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe648102ac0 0x7fe648102ea0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.899+0000 7fe650387640 1 -- 172.21.15.7:0/3253265742 >> 172.21.15.7:0/3253265742 conn(0x7fe6480fc520 msgr2=0x7fe6480fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.899+0000 7fe650387640 1 -- 172.21.15.7:0/3253265742 shutdown_connections 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.899+0000 7fe650387640 1 -- 172.21.15.7:0/3253265742 wait complete. 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe650387640 1 Processor -- start 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe650387640 1 -- start start 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe650387640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe648102ac0 0x7fe64819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe650387640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6481033e0 0x7fe64819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe650387640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe64810edf0 con 0x7fe6481033e0 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe650387640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe64810ec70 con 0x7fe648102ac0 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe64d922640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6481033e0 0x7fe64819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe64d922640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6481033e0 0x7fe64819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49980/0 (socket says 172.21.15.7:49980) 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe64d922640 1 -- 172.21.15.7:0/2303286276 learned_addr learned my addr 172.21.15.7:0/2303286276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.900+0000 7fe64e123640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe648102ac0 0x7fe64819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe64d922640 1 -- 172.21.15.7:0/2303286276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe648102ac0 msgr2=0x7fe64819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe64d922640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe648102ac0 0x7fe64819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe64d922640 1 -- 172.21.15.7:0/2303286276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6481977c0 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe64e123640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe648102ac0 0x7fe64819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe64d922640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6481033e0 0x7fe64819db80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00d6d0 tx=0x7fe63c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe63b7fe640 1 -- 172.21.15.7:0/2303286276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe63c017070 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe63b7fe640 1 -- 172.21.15.7:0/2303286276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe63c0054e0 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe648197ab0 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe63b7fe640 1 -- 172.21.15.7:0/2303286276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe63c004ae0 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.901+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe648198040 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.902+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64806aa90 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.903+0000 7fe63b7fe640 1 -- 172.21.15.7:0/2303286276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fe63c004d90 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.903+0000 7fe63b7fe640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe63407af90 0x7fe63407d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.903+0000 7fe63b7fe640 1 -- 172.21.15.7:0/2303286276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fe63c09dc50 con 0x7fe6481033e0 2024-09-21T05:16:27.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.903+0000 7fe64e123640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe63407af90 0x7fe63407d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.904+0000 7fe64e123640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe63407af90 0x7fe63407d450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe644000c00 tx=0x7fe644031400 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:27.908+0000 7fe63b7fe640 1 -- 172.21.15.7:0/2303286276 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe63c0691d0 con 0x7fe6481033e0 2024-09-21T05:16:27.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:27 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/772203908' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 33, "format": "json"} : dispatch 2024-09-21T05:16:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.136+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) -- 0x7fe6481982f0 con 0x7fe6481033e0 2024-09-21T05:16:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.137+0000 7fe63b7fe640 1 -- 172.21.15.7:0/2303286276 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v38) ==== 107+0+5324 (secure 0 0 0) 0x7fe63c06e080 con 0x7fe6481033e0 2024-09-21T05:16:28.140 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:28.141 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"btime":"2024-09-21T05:14:26:353714+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T05:14:25.399429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34354,"mds_1":34358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34354":{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":0,"incarnation":32,"state":"up:resolve","state_seq":464,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34358":{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":1,"incarnation":32,"state":"up:resolve","state_seq":458,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe63407af90 msgr2=0x7fe63407d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe63407af90 0x7fe63407d450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe644000c00 tx=0x7fe644031400 comp rx=0 tx=0).stop 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6481033e0 msgr2=0x7fe64819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6481033e0 0x7fe64819db80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00d6d0 tx=0x7fe63c00db90 comp rx=0 tx=0).stop 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 shutdown_connections 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fe63407af90 0x7fe63407d450 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6481033e0 0x7fe64819db80 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 --2- 172.21.15.7:0/2303286276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe648102ac0 0x7fe64819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 >> 172.21.15.7:0/2303286276 conn(0x7fe6480fc520 msgr2=0x7fe64810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 shutdown_connections 2024-09-21T05:16:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:28.139+0000 7fe650387640 1 -- 172.21.15.7:0/2303286276 wait complete. 2024-09-21T05:16:28.782 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:28 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2303286276' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 34, "format": "json"} : dispatch 2024-09-21T05:16:28.782 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:28 smithi007 ceph-mon[108238]: pgmap v1255: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:28.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 35 2024-09-21T05:16:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:28 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2303286276' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 34, "format": "json"} : dispatch 2024-09-21T05:16:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:28 smithi053 ceph-mon[80176]: pgmap v1255: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:29.071 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:29.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 -- 172.21.15.7:0/3102841027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541067f0 msgr2=0x7f1154106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:29.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 --2- 172.21.15.7:0/3102841027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541067f0 0x7f1154106bd0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1148009880 tx=0x7f114802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:29.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 -- 172.21.15.7:0/3102841027 shutdown_connections 2024-09-21T05:16:29.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 --2- 172.21.15.7:0/3102841027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11541007f0 0x7f1154100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 --2- 172.21.15.7:0/3102841027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541067f0 0x7f1154106bd0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 -- 172.21.15.7:0/3102841027 >> 172.21.15.7:0/3102841027 conn(0x7f11540fc550 msgr2=0x7f11540fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 -- 172.21.15.7:0/3102841027 shutdown_connections 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 -- 172.21.15.7:0/3102841027 wait complete. 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 Processor -- start 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.584+0000 7f115bf74640 1 -- start start 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f115bf74640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541007f0 0x7f11540fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f115bf74640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11541067f0 0x7f11540fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f115bf74640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f115410dd20 con 0x7f11541007f0 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f115bf74640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f115410dba0 con 0x7f11541067f0 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1159d10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541007f0 0x7f11540fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1159d10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541007f0 0x7f11540fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49994/0 (socket says 172.21.15.7:49994) 2024-09-21T05:16:29.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1159d10640 1 -- 172.21.15.7:0/198469176 learned_addr learned my addr 172.21.15.7:0/198469176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f115950f640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11541067f0 0x7f11540fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1159d10640 1 -- 172.21.15.7:0/198469176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11541067f0 msgr2=0x7f11540fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1159d10640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11541067f0 0x7f11540fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1159d10640 1 -- 172.21.15.7:0/198469176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11540fd7c0 con 0x7f11541007f0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1159d10640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541007f0 0x7f11540fcd40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1148031460 tx=0x7f11480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1146ffd640 1 -- 172.21.15.7:0/198469176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f114803f070 con 0x7f11541007f0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1146ffd640 1 -- 172.21.15.7:0/198469176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1148004cf0 con 0x7f11541007f0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11540fda50 con 0x7f11541007f0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f1146ffd640 1 -- 172.21.15.7:0/198469176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1148035030 con 0x7f11541007f0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.585+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a3e50 con 0x7f11541007f0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.587+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1154101f30 con 0x7f11541007f0 2024-09-21T05:16:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.587+0000 7f1146ffd640 1 -- 172.21.15.7:0/198469176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f1148042070 con 0x7f11541007f0 2024-09-21T05:16:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.587+0000 7f1146ffd640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f114007af30 0x7f114007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.587+0000 7f1146ffd640 1 -- 172.21.15.7:0/198469176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f11480beac0 con 0x7f11541007f0 2024-09-21T05:16:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.587+0000 7f115950f640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f114007af30 0x7f114007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.587+0000 7f115950f640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f114007af30 0x7f114007d3f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f11540fe7d0 tx=0x7f1150009110 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.592+0000 7f1146ffd640 1 -- 172.21.15.7:0/198469176 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f114808a060 con 0x7f11541007f0 2024-09-21T05:16:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.818+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) -- 0x7f1154109d70 con 0x7f11541007f0 2024-09-21T05:16:29.821 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:16:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.819+0000 7f1146ffd640 1 -- 172.21.15.7:0/198469176 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v38) ==== 107+0+5328 (secure 0 0 0) 0x7f114808ef10 con 0x7f11541007f0 2024-09-21T05:16:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:29.822 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"btime":"2024-09-21T05:14:27:358451+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T05:14:26.412749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34354,"mds_1":34358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34354":{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":465,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34358":{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":1,"incarnation":32,"state":"up:reconnect","state_seq":461,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f114007af30 msgr2=0x7f114007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f114007af30 0x7f114007d3f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f11540fe7d0 tx=0x7f1150009110 comp rx=0 tx=0).stop 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541007f0 msgr2=0x7f11540fcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541007f0 0x7f11540fcd40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f1148031460 tx=0x7f11480048f0 comp rx=0 tx=0).stop 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 shutdown_connections 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f114007af30 0x7f114007d3f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11541067f0 0x7f11540fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 --2- 172.21.15.7:0/198469176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11541007f0 0x7f11540fcd40 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 >> 172.21.15.7:0/198469176 conn(0x7f11540fc550 msgr2=0x7f1154069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:29.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.821+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 shutdown_connections 2024-09-21T05:16:29.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:29.822+0000 7f115bf74640 1 -- 172.21.15.7:0/198469176 wait complete. 2024-09-21T05:16:29.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:16:30.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 36 2024-09-21T05:16:30.756 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:30.877 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:30 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/198469176' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 35, "format": "json"} : dispatch 2024-09-21T05:16:30.878 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:30 smithi007 ceph-mon[108238]: pgmap v1256: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:30.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:30 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/198469176' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 35, "format": "json"} : dispatch 2024-09-21T05:16:30.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:30 smithi053 ceph-mon[80176]: pgmap v1256: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.216+0000 7f9a483b4640 1 -- 172.21.15.7:0/2293232423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 msgr2=0x7f9a4010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.216+0000 7f9a483b4640 1 --2- 172.21.15.7:0/2293232423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 0x7f9a4010b8d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009880 tx=0x7f9a3002a530 comp rx=0 tx=0).stop 2024-09-21T05:16:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.216+0000 7f9a483b4640 1 -- 172.21.15.7:0/2293232423 shutdown_connections 2024-09-21T05:16:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.216+0000 7f9a483b4640 1 --2- 172.21.15.7:0/2293232423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 0x7f9a4010b8d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.216+0000 7f9a483b4640 1 --2- 172.21.15.7:0/2293232423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a40102ac0 0x7f9a40102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.216+0000 7f9a483b4640 1 -- 172.21.15.7:0/2293232423 >> 172.21.15.7:0/2293232423 conn(0x7f9a400fc520 msgr2=0x7f9a400fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.216+0000 7f9a483b4640 1 -- 172.21.15.7:0/2293232423 shutdown_connections 2024-09-21T05:16:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.216+0000 7f9a483b4640 1 -- 172.21.15.7:0/2293232423 wait complete. 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a483b4640 1 Processor -- start 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a483b4640 1 -- start start 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a483b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a40102ac0 0x7f9a40100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a483b4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 0x7f9a40100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a483b4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4010edf0 con 0x7f9a401033e0 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a483b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4010ec70 con 0x7f9a40102ac0 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a4594f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 0x7f9a40100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a4594f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 0x7f9a40100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50004/0 (socket says 172.21.15.7:50004) 2024-09-21T05:16:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.217+0000 7f9a4594f640 1 -- 172.21.15.7:0/1619667637 learned_addr learned my addr 172.21.15.7:0/1619667637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a46150640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a40102ac0 0x7f9a40100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a4594f640 1 -- 172.21.15.7:0/1619667637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a40102ac0 msgr2=0x7f9a40100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a4594f640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a40102ac0 0x7f9a40100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a4594f640 1 -- 172.21.15.7:0/1619667637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a401011a0 con 0x7f9a401033e0 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a4594f640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 0x7f9a40100c60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f9a30031460 tx=0x7f9a30004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a377fe640 1 -- 172.21.15.7:0/1619667637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9a3003f070 con 0x7f9a401033e0 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a377fe640 1 -- 172.21.15.7:0/1619667637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9a300035f0 con 0x7f9a401033e0 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a40101430 con 0x7f9a401033e0 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a377fe640 1 -- 172.21.15.7:0/1619667637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9a30035030 con 0x7f9a401033e0 2024-09-21T05:16:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.218+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a401a3de0 con 0x7f9a401033e0 2024-09-21T05:16:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.219+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4006aa90 con 0x7f9a401033e0 2024-09-21T05:16:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.220+0000 7f9a377fe640 1 -- 172.21.15.7:0/1619667637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f9a30042070 con 0x7f9a401033e0 2024-09-21T05:16:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.220+0000 7f9a377fe640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9a2c07af80 0x7f9a2c07d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.220+0000 7f9a377fe640 1 -- 172.21.15.7:0/1619667637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f9a300be940 con 0x7f9a401033e0 2024-09-21T05:16:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.220+0000 7f9a46150640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9a2c07af80 0x7f9a2c07d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:31.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.221+0000 7f9a46150640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9a2c07af80 0x7f9a2c07d440 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005fd0 tx=0x7f9a3c005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.225+0000 7f9a377fe640 1 -- 172.21.15.7:0/1619667637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a30089f60 con 0x7f9a401033e0 2024-09-21T05:16:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.448+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) -- 0x7f9a40101c50 con 0x7f9a401033e0 2024-09-21T05:16:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.449+0000 7f9a377fe640 1 -- 172.21.15.7:0/1619667637 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v38) ==== 107+0+5322 (secure 0 0 0) 0x7f9a3008ee10 con 0x7f9a401033e0 2024-09-21T05:16:31.452 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:31.452 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"btime":"2024-09-21T05:14:28:363776+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T05:14:27.366765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34354,"mds_1":34358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34354":{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":466,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34358":{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":1,"incarnation":32,"state":"up:rejoin","state_seq":462,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-21T05:16:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-21T05:16:31.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9a2c07af80 msgr2=0x7f9a2c07d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:31.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9a2c07af80 0x7f9a2c07d440 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005fd0 tx=0x7f9a3c005850 comp rx=0 tx=0).stop 2024-09-21T05:16:31.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 msgr2=0x7f9a40100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:31.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 0x7f9a40100c60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f9a30031460 tx=0x7f9a30004530 comp rx=0 tx=0).stop 2024-09-21T05:16:31.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 shutdown_connections 2024-09-21T05:16:31.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f9a2c07af80 0x7f9a2c07d440 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:31.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a401033e0 0x7f9a40100c60 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:31.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 --2- 172.21.15.7:0/1619667637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a40102ac0 0x7f9a40100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.451+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 >> 172.21.15.7:0/1619667637 conn(0x7f9a400fc520 msgr2=0x7f9a40108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.452+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 shutdown_connections 2024-09-21T05:16:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:31.452+0000 7f9a483b4640 1 -- 172.21.15.7:0/1619667637 wait complete. 2024-09-21T05:16:31.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:31 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1619667637' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 36, "format": "json"} : dispatch 2024-09-21T05:16:31.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:31 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1619667637' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 36, "format": "json"} : dispatch 2024-09-21T05:16:32.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 37 2024-09-21T05:16:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:32.827 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:32 smithi007 ceph-mon[108238]: pgmap v1257: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.908+0000 7f5571d3b640 1 -- 172.21.15.7:0/1416259912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 msgr2=0x7f556c10b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.908+0000 7f5571d3b640 1 --2- 172.21.15.7:0/1416259912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 0x7f556c10b930 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f55540098b0 tx=0x7f555402a560 comp rx=0 tx=0).stop 2024-09-21T05:16:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.908+0000 7f5571d3b640 1 -- 172.21.15.7:0/1416259912 shutdown_connections 2024-09-21T05:16:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.908+0000 7f5571d3b640 1 --2- 172.21.15.7:0/1416259912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 0x7f556c10b930 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.908+0000 7f5571d3b640 1 --2- 172.21.15.7:0/1416259912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f556c102b20 0x7f556c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.908+0000 7f5571d3b640 1 -- 172.21.15.7:0/1416259912 >> 172.21.15.7:0/1416259912 conn(0x7f556c0fc520 msgr2=0x7f556c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.908+0000 7f5571d3b640 1 -- 172.21.15.7:0/1416259912 shutdown_connections 2024-09-21T05:16:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.908+0000 7f5571d3b640 1 -- 172.21.15.7:0/1416259912 wait complete. 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5571d3b640 1 Processor -- start 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5571d3b640 1 -- start start 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5571d3b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f556c102b20 0x7f556c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5571d3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 0x7f556c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5571d3b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f556c10edd0 con 0x7f556c103440 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5571d3b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f556c10ec50 con 0x7f556c102b20 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5563fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 0x7f556c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5563fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 0x7f556c19dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50030/0 (socket says 172.21.15.7:50030) 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.909+0000 7f5563fff640 1 -- 172.21.15.7:0/779674195 learned_addr learned my addr 172.21.15.7:0/779674195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f556b7fe640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f556c102b20 0x7f556c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f5563fff640 1 -- 172.21.15.7:0/779674195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f556c102b20 msgr2=0x7f556c19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f5563fff640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f556c102b20 0x7f556c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f5563fff640 1 -- 172.21.15.7:0/779674195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c197820 con 0x7f556c103440 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f5563fff640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 0x7f556c19dbe0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5554009230 tx=0x7f55540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f55697fa640 1 -- 172.21.15.7:0/779674195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f555403f070 con 0x7f556c103440 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f55697fa640 1 -- 172.21.15.7:0/779674195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5554004ac0 con 0x7f556c103440 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f55697fa640 1 -- 172.21.15.7:0/779674195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5554035030 con 0x7f556c103440 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c197ab0 con 0x7f556c103440 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.910+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c197fe0 con 0x7f556c103440 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.911+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556c06aa30 con 0x7f556c103440 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.912+0000 7f55697fa640 1 -- 172.21.15.7:0/779674195 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f5554042070 con 0x7f556c103440 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.912+0000 7f55697fa640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f555807af90 0x7f555807d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.912+0000 7f55697fa640 1 -- 172.21.15.7:0/779674195 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f55540beb30 con 0x7f556c103440 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.912+0000 7f556b7fe640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f555807af90 0x7f555807d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.913+0000 7f556b7fe640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f555807af90 0x7f555807d450 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f555c00ba40 tx=0x7f555c005cc0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:32.917+0000 7f55697fa640 1 -- 172.21.15.7:0/779674195 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f555408a150 con 0x7f556c103440 2024-09-21T05:16:32.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:32 smithi053 ceph-mon[80176]: pgmap v1257: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:33.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.142+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) -- 0x7f556c198d80 con 0x7f556c103440 2024-09-21T05:16:33.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.143+0000 7f55697fa640 1 -- 172.21.15.7:0/779674195 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v38) ==== 107+0+5352 (secure 0 0 0) 0x7f556c198d80 con 0x7f556c103440 2024-09-21T05:16:33.145 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-21T05:16:33.146 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"btime":"2024-09-21T05:14:29:121160+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:32:16.358433+0000","modified":"2024-09-21T05:14:29.121156+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34354,"mds_1":34358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34354":{"gid":34354,"name":"cephfs.smithi007.oxcnqk","rank":0,"incarnation":32,"state":"up:active","state_seq":467,"addr":"172.21.15.7:6827/3186196668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3186196668},{"type":"v1","addr":"172.21.15.7:6827","nonce":3186196668}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34358":{"gid":34358,"name":"cephfs.smithi007.eyygpg","rank":1,"incarnation":32,"state":"up:active","state_seq":463,"addr":"172.21.15.7:6829/2665314222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2665314222},{"type":"v1","addr":"172.21.15.7:6829","nonce":2665314222}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44327":{"gid":44327,"name":"cephfs.smithi053.wdnfat","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6825/1618778609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1618778609},{"type":"v1","addr":"172.21.15.53:6825","nonce":1618778609}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44909":{"gid":44909,"name":"cephfs.smithi053.jvuwme","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6827/3204998384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":3204998384},{"type":"v1","addr":"172.21.15.53:6827","nonce":3204998384}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34354,"qdb_cluster":[34354,34358]},"id":1}]} 2024-09-21T05:16:33.146 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f555807af90 msgr2=0x7f555807d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f555807af90 0x7f555807d450 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f555c00ba40 tx=0x7f555c005cc0 comp rx=0 tx=0).stop 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 msgr2=0x7f556c19dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 0x7f556c19dbe0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5554009230 tx=0x7f55540046c0 comp rx=0 tx=0).stop 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 shutdown_connections 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f555807af90 0x7f555807d450 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f556c103440 0x7f556c19dbe0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 --2- 172.21.15.7:0/779674195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f556c102b20 0x7f556c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 >> 172.21.15.7:0/779674195 conn(0x7f556c0fc520 msgr2=0x7f556c103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 shutdown_connections 2024-09-21T05:16:33.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:33.145+0000 7f5571d3b640 1 -- 172.21.15.7:0/779674195 wait complete. 2024-09-21T05:16:33.756 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:33 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/779674195' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 37, "format": "json"} : dispatch 2024-09-21T05:16:33.787 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-21T05:16:33.797 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-21T05:16:33.797 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T05:16:33.797 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:16:33.815 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T05:16:33.816 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:16:33.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T05:16:33.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:33 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/779674195' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 37, "format": "json"} : dispatch 2024-09-21T05:16:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:34.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:34 smithi007 ceph-mon[108238]: pgmap v1258: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 -- 172.21.15.7:0/3218814061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4106800 msgr2=0x7fb8b4106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 --2- 172.21.15.7:0/3218814061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4106800 0x7fb8b4106be0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009880 tx=0x7fb8a402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 -- 172.21.15.7:0/3218814061 shutdown_connections 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 --2- 172.21.15.7:0/3218814061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b4100800 0x7fb8b4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 --2- 172.21.15.7:0/3218814061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4106800 0x7fb8b4106be0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 -- 172.21.15.7:0/3218814061 >> 172.21.15.7:0/3218814061 conn(0x7fb8b40fc520 msgr2=0x7fb8b40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 -- 172.21.15.7:0/3218814061 shutdown_connections 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 -- 172.21.15.7:0/3218814061 wait complete. 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.667+0000 7fb8b8dac640 1 Processor -- start 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b8dac640 1 -- start start 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b8dac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4100800 0x7fb8b40fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b8dac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b4106800 0x7fb8b40fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b8dac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410dcd0 con 0x7fb8b4100800 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b8dac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410db50 con 0x7fb8b4106800 2024-09-21T05:16:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4100800 0x7fb8b40fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4100800 0x7fb8b40fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42752/0 (socket says 172.21.15.7:42752) 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b259c640 1 -- 172.21.15.7:0/1914152235 learned_addr learned my addr 172.21.15.7:0/1914152235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b259c640 1 -- 172.21.15.7:0/1914152235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b4106800 msgr2=0x7fb8b40fd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b259c640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b4106800 0x7fb8b40fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b259c640 1 -- 172.21.15.7:0/1914152235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b40fd680 con 0x7fb8b4100800 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8b259c640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4100800 0x7fb8b40fcc00 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4031460 tx=0x7fb8a4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.668+0000 7fb8ab7fe640 1 -- 172.21.15.7:0/1914152235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb8a403f070 con 0x7fb8b4100800 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.669+0000 7fb8ab7fe640 1 -- 172.21.15.7:0/1914152235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb8a40035f0 con 0x7fb8b4100800 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.669+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b40fd910 con 0x7fb8b4100800 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.669+0000 7fb8ab7fe640 1 -- 172.21.15.7:0/1914152235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb8a4035030 con 0x7fb8b4100800 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.669+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41a3da0 con 0x7fb8b4100800 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.670+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b4101f40 con 0x7fb8b4100800 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.670+0000 7fb8ab7fe640 1 -- 172.21.15.7:0/1914152235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7fb8a4042070 con 0x7fb8b4100800 2024-09-21T05:16:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.671+0000 7fb8ab7fe640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8a007af30 0x7fb8a007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.671+0000 7fb8ab7fe640 1 -- 172.21.15.7:0/1914152235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7fb8a40bea10 con 0x7fb8b4100800 2024-09-21T05:16:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.671+0000 7fb8abfff640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8a007af30 0x7fb8a007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.671+0000 7fb8abfff640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8a007af30 0x7fb8a007d3f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0058c0 tx=0x7fb89c005850 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.675+0000 7fb8ab7fe640 1 -- 172.21.15.7:0/1914152235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8a4089fb0 con 0x7fb8b4100800 2024-09-21T05:16:34.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.875+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb8b4100c60 con 0x7fb8b4100800 2024-09-21T05:16:34.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.876+0000 7fb8ab7fe640 1 -- 172.21.15.7:0/1914152235 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 33 entries v80) ==== 81+0+1934 (secure 0 0 0) 0x7fb8a408ee60 con 0x7fb8b4100800 2024-09-21T05:16:34.878 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6828/406067991 2024-09-22T04:43:25.692735+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3995670699 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6829/406067991 2024-09-22T04:43:25.692735+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3901749065 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4286237328 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2645587315 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:6824/2105303008 2024-09-22T04:43:25.692747+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2211484432 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2202116322 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2645587315 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6826/2872814664 2024-09-22T04:32:29.881122+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:6828/2371363507 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3340019345 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6827/2872814664 2024-09-22T04:32:29.881122+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:0/2979088329 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:34.879 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1384976015 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:0/3422800458 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/4030185662 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2677828643 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2851424446 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:6825/2105303008 2024-09-22T04:43:25.692747+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/4030185662 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/554975665 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:6829/2371363507 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3808261568 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3808261568 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:0/1152658732 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3995670699 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4161500397 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1594571982 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2178763757 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:34.881 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3304037039 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:34.881 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:0/2843202265 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:listed 33 entries 2024-09-21T05:16:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8a007af30 msgr2=0x7fb8a007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8a007af30 0x7fb8a007d3f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0058c0 tx=0x7fb89c005850 comp rx=0 tx=0).stop 2024-09-21T05:16:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4100800 msgr2=0x7fb8b40fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4100800 0x7fb8b40fcc00 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4031460 tx=0x7fb8a4004530 comp rx=0 tx=0).stop 2024-09-21T05:16:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 shutdown_connections 2024-09-21T05:16:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7fb8a007af30 0x7fb8a007d3f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8b4106800 0x7fb8b40fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 --2- 172.21.15.7:0/1914152235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b4100800 0x7fb8b40fcc00 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 >> 172.21.15.7:0/1914152235 conn(0x7fb8b40fc520 msgr2=0x7fb8b406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 shutdown_connections 2024-09-21T05:16:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:34.878+0000 7fb8b8dac640 1 -- 172.21.15.7:0/1914152235 wait complete. 2024-09-21T05:16:34.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:34 smithi053 ceph-mon[80176]: pgmap v1258: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:35.479 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-21T05:16:35.479 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:16:35.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 70154924-77d1-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T05:16:35.800 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/mon.smithi007/config 2024-09-21T05:16:35.921 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:35 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/1914152235' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T05:16:35.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:35 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/1914152235' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T05:16:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.259+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2543984602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681008b0 msgr2=0x7f3b68100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.259+0000 7f3b6fa2f640 1 --2- 172.21.15.7:0/2543984602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681008b0 0x7f3b68100c90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009c20 tx=0x7f3b6402aa00 comp rx=0 tx=0).stop 2024-09-21T05:16:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.259+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2543984602 shutdown_connections 2024-09-21T05:16:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.259+0000 7f3b6fa2f640 1 --2- 172.21.15.7:0/2543984602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b681011d0 0x7f3b681056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.259+0000 7f3b6fa2f640 1 --2- 172.21.15.7:0/2543984602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681008b0 0x7f3b68100c90 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.259+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2543984602 >> 172.21.15.7:0/2543984602 conn(0x7f3b680fc520 msgr2=0x7f3b680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.259+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2543984602 shutdown_connections 2024-09-21T05:16:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.259+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2543984602 wait complete. 2024-09-21T05:16:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6fa2f640 1 Processor -- start 2024-09-21T05:16:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6fa2f640 1 -- start start 2024-09-21T05:16:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6fa2f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b681008b0 0x7f3b6819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6fa2f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681011d0 0x7f3b6819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6fa2f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681153f0 con 0x7f3b681011d0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6fa2f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68115270 con 0x7f3b681008b0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6cfca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681011d0 0x7f3b6819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6cfca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681011d0 0x7f3b6819e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42778/0 (socket says 172.21.15.7:42778) 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6cfca640 1 -- 172.21.15.7:0/2882358409 learned_addr learned my addr 172.21.15.7:0/2882358409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.260+0000 7f3b6d7cb640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b681008b0 0x7f3b6819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b6cfca640 1 -- 172.21.15.7:0/2882358409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b681008b0 msgr2=0x7f3b6819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b6cfca640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b681008b0 0x7f3b6819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b6cfca640 1 -- 172.21.15.7:0/2882358409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b681a2880 con 0x7f3b681011d0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b6cfca640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681011d0 0x7f3b6819e4f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800d6d0 tx=0x7f3b5800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b5e7fc640 1 -- 172.21.15.7:0/2882358409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3b58017070 con 0x7f3b681011d0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b5e7fc640 1 -- 172.21.15.7:0/2882358409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b580054e0 con 0x7f3b681011d0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b681a2b70 con 0x7f3b681011d0 2024-09-21T05:16:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b5e7fc640 1 -- 172.21.15.7:0/2882358409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3b58004ae0 con 0x7f3b681011d0 2024-09-21T05:16:36.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.261+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b681a3100 con 0x7f3b681011d0 2024-09-21T05:16:36.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.262+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6806aa90 con 0x7f3b681011d0 2024-09-21T05:16:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.263+0000 7f3b5e7fc640 1 -- 172.21.15.7:0/2882358409 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) ==== 103318+0+0 (secure 0 0 0) 0x7f3b58004de0 con 0x7f3b681011d0 2024-09-21T05:16:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.264+0000 7f3b5e7fc640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b5407af40 0x7f3b5407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:16:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.264+0000 7f3b5e7fc640 1 -- 172.21.15.7:0/2882358409 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6242+0+0 (secure 0 0 0) 0x7f3b5809d980 con 0x7f3b681011d0 2024-09-21T05:16:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.264+0000 7f3b6d7cb640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b5407af40 0x7f3b5407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:16:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.264+0000 7f3b6d7cb640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b5407af40 0x7f3b5407d400 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009430 tx=0x7f3b6403a040 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:16:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.268+0000 7f3b5e7fc640 1 -- 172.21.15.7:0/2882358409 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b58068f00 con 0x7f3b681011d0 2024-09-21T05:16:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.468+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b6819f310 con 0x7f3b681011d0 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.468+0000 7f3b5e7fc640 1 -- 172.21.15.7:0/2882358409 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 33 entries v80) ==== 81+0+1934 (secure 0 0 0) 0x7f3b5806ddb0 con 0x7f3b681011d0 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6828/406067991 2024-09-22T04:43:25.692735+0000 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3995670699 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6829/406067991 2024-09-22T04:43:25.692735+0000 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3901749065 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4286237328 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2645587315 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:6824/2105303008 2024-09-22T04:43:25.692747+0000 2024-09-21T05:16:36.471 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2211484432 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2202116322 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2645587315 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6826/2872814664 2024-09-22T04:32:29.881122+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:6828/2371363507 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3340019345 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6827/2872814664 2024-09-22T04:32:29.881122+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:0/2979088329 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1384976015 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:0/3422800458 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/4030185662 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2677828643 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2851424446 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:6825/2105303008 2024-09-22T04:43:25.692747+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/4030185662 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:36.472 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/554975665 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:6829/2371363507 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3808261568 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3808261568 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:0/1152658732 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3995670699 2024-09-22T04:27:31.004325+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4161500397 2024-09-22T04:26:33.198427+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1594571982 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2178763757 2024-09-22T04:34:23.270980+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3304037039 2024-09-22T04:25:58.338381+0000 2024-09-21T05:16:36.473 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.53:0/2843202265 2024-09-22T04:35:57.694668+0000 2024-09-21T05:16:36.474 INFO:teuthology.orchestra.run.smithi007.stderr:listed 33 entries 2024-09-21T05:16:36.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b5407af40 msgr2=0x7f3b5407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:36.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b5407af40 0x7f3b5407d400 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009430 tx=0x7f3b6403a040 comp rx=0 tx=0).stop 2024-09-21T05:16:36.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681011d0 msgr2=0x7f3b6819e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:16:36.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681011d0 0x7f3b6819e4f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800d6d0 tx=0x7f3b5800db90 comp rx=0 tx=0).stop 2024-09-21T05:16:36.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 shutdown_connections 2024-09-21T05:16:36.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:6800/4204093365,v1:172.21.15.7:6801/4204093365] conn(0x7f3b5407af40 0x7f3b5407d400 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:36.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b681011d0 0x7f3b6819e4f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:36.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 --2- 172.21.15.7:0/2882358409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b681008b0 0x7f3b6819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:16:36.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 >> 172.21.15.7:0/2882358409 conn(0x7f3b680fc520 msgr2=0x7f3b68112ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:16:36.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 shutdown_connections 2024-09-21T05:16:36.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-21T05:16:36.470+0000 7f3b6fa2f640 1 -- 172.21.15.7:0/2882358409 wait complete. 2024-09-21T05:16:36.882 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:36 smithi007 ceph-mon[108238]: pgmap v1259: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:36.882 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:36 smithi007 ceph-mon[108238]: from='client.? 172.21.15.7:0/2882358409' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T05:16:36.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:36 smithi053 ceph-mon[80176]: pgmap v1259: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:36.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:36 smithi053 ceph-mon[80176]: from='client.? 172.21.15.7:0/2882358409' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T05:16:37.249 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-09-21T05:16:37.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:16:37.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-21T05:16:37.278 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:16:39.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:39 smithi053 ceph-mon[80176]: pgmap v1260: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:39.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:39 smithi007 ceph-mon[108238]: pgmap v1260: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:41.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:41 smithi053 ceph-mon[80176]: pgmap v1261: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:41.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:41 smithi007 ceph-mon[108238]: pgmap v1261: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:42.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:42 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:16:42.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:42 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:16:43.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:43 smithi053 ceph-mon[80176]: pgmap v1262: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:43.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:16:43.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:16:43.694 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:16:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:43 smithi007 ceph-mon[108238]: pgmap v1262: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:43.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:16:43.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:16:43.695 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:16:44.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:16:44.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:16:45.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:45 smithi053 ceph-mon[80176]: pgmap v1263: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:45.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:45 smithi007 ceph-mon[108238]: pgmap v1263: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:46.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:46 smithi007 ceph-mon[108238]: pgmap v1264: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:46.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:46 smithi053 ceph-mon[80176]: pgmap v1264: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:49 smithi053 ceph-mon[80176]: pgmap v1265: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:49.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:49 smithi007 ceph-mon[108238]: pgmap v1265: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:51.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:51 smithi053 ceph-mon[80176]: pgmap v1266: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:51.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:51 smithi007 ceph-mon[108238]: pgmap v1266: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:53.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:53 smithi053 ceph-mon[80176]: pgmap v1267: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:53.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:53 smithi007 ceph-mon[108238]: pgmap v1267: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:55.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:55 smithi007 ceph-mon[108238]: pgmap v1268: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:55.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:55 smithi053 ceph-mon[80176]: pgmap v1268: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:16:57.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:57 smithi053 ceph-mon[80176]: pgmap v1269: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:57.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:57 smithi007 ceph-mon[108238]: pgmap v1269: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:16:58.932 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:58 smithi007 ceph-mon[108238]: pgmap v1270: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:58 smithi053 ceph-mon[80176]: pgmap v1270: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:16:59.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:16:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:16:59.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:16:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:17:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:00 smithi053 ceph-mon[80176]: pgmap v1271: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:00 smithi007 ceph-mon[108238]: pgmap v1271: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:03.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:03 smithi053 ceph-mon[80176]: pgmap v1272: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:03.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:03 smithi007 ceph-mon[108238]: pgmap v1272: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:05.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:05 smithi053 ceph-mon[80176]: pgmap v1273: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:05.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:05 smithi007 ceph-mon[108238]: pgmap v1273: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:07.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:07 smithi053 ceph-mon[80176]: pgmap v1274: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:07.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:07 smithi007 ceph-mon[108238]: pgmap v1274: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:09.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:09 smithi053 ceph-mon[80176]: pgmap v1275: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:09.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:09 smithi007 ceph-mon[108238]: pgmap v1275: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:11.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:11 smithi053 ceph-mon[80176]: pgmap v1276: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:11.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:11 smithi007 ceph-mon[108238]: pgmap v1276: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:13.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:13 smithi053 ceph-mon[80176]: pgmap v1277: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:13.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:13 smithi007 ceph-mon[108238]: pgmap v1277: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:14.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:17:14.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:14 smithi053 ceph-mon[80176]: pgmap v1278: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:17:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:14 smithi007 ceph-mon[108238]: pgmap v1278: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:17.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:17 smithi053 ceph-mon[80176]: pgmap v1279: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:17.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:17 smithi007 ceph-mon[108238]: pgmap v1279: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:19.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:19 smithi053 ceph-mon[80176]: pgmap v1280: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:19.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:19 smithi007 ceph-mon[108238]: pgmap v1280: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:21.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:21 smithi053 ceph-mon[80176]: pgmap v1281: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:21.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:21 smithi007 ceph-mon[108238]: pgmap v1281: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:23.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:23 smithi053 ceph-mon[80176]: pgmap v1282: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:23.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:23 smithi007 ceph-mon[108238]: pgmap v1282: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:25.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:25 smithi053 ceph-mon[80176]: pgmap v1283: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:25.444 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:25 smithi007 ceph-mon[108238]: pgmap v1283: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:27.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:27 smithi053 ceph-mon[80176]: pgmap v1284: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:27.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:27 smithi007 ceph-mon[108238]: pgmap v1284: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:29 smithi053 ceph-mon[80176]: pgmap v1285: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:29.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:17:29.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:29 smithi007 ceph-mon[108238]: pgmap v1285: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:29.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:17:31.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:31 smithi053 ceph-mon[80176]: pgmap v1286: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:31.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:31 smithi007 ceph-mon[108238]: pgmap v1286: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:33.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:33 smithi007 ceph-mon[108238]: pgmap v1287: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:33.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:33 smithi053 ceph-mon[80176]: pgmap v1287: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:35.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:35 smithi007 ceph-mon[108238]: pgmap v1288: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:35.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:35 smithi053 ceph-mon[80176]: pgmap v1288: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:37.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:37 smithi053 ceph-mon[80176]: pgmap v1289: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:37.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:37 smithi007 ceph-mon[108238]: pgmap v1289: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:39.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:39 smithi053 ceph-mon[80176]: pgmap v1290: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:39.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:39 smithi007 ceph-mon[108238]: pgmap v1290: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:41.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:41 smithi053 ceph-mon[80176]: pgmap v1291: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:41.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:41 smithi007 ceph-mon[108238]: pgmap v1291: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:43.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:43 smithi007 ceph-mon[108238]: pgmap v1292: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:43.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:17:43.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:17:43.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:17:43.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:17:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:43 smithi053 ceph-mon[80176]: pgmap v1292: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:17:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:17:43.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:17:43.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:17:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:17:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:17:45.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:45 smithi053 ceph-mon[80176]: pgmap v1293: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:45.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:45 smithi007 ceph-mon[108238]: pgmap v1293: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:46.852 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:46 smithi053 ceph-mon[80176]: pgmap v1294: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:46.904 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:46 smithi007 ceph-mon[108238]: pgmap v1294: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:49.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:49 smithi053 ceph-mon[80176]: pgmap v1295: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:49.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:49 smithi007 ceph-mon[108238]: pgmap v1295: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:51.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:51 smithi007 ceph-mon[108238]: pgmap v1296: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:51.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:51 smithi053 ceph-mon[80176]: pgmap v1296: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:53.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:53 smithi053 ceph-mon[80176]: pgmap v1297: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:53.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:53 smithi007 ceph-mon[108238]: pgmap v1297: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:54.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:54 smithi053 ceph-mon[80176]: pgmap v1298: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:54.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:54 smithi007 ceph-mon[108238]: pgmap v1298: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:17:57.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:57 smithi007 ceph-mon[108238]: pgmap v1299: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:57.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:57 smithi053 ceph-mon[80176]: pgmap v1299: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:17:59.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:59 smithi053 ceph-mon[80176]: pgmap v1300: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:59.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:17:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:17:59.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:59 smithi007 ceph-mon[108238]: pgmap v1300: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:17:59.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:17:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:18:01.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:01 smithi007 ceph-mon[108238]: pgmap v1301: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:01.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:01 smithi053 ceph-mon[80176]: pgmap v1301: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:02.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:02 smithi053 ceph-mon[80176]: pgmap v1302: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:02.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:02 smithi007 ceph-mon[108238]: pgmap v1302: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:05.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:05 smithi053 ceph-mon[80176]: pgmap v1303: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:05.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:05 smithi007 ceph-mon[108238]: pgmap v1303: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:07.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:07 smithi053 ceph-mon[80176]: pgmap v1304: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:07.693 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:07 smithi007 ceph-mon[108238]: pgmap v1304: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:09.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:09 smithi053 ceph-mon[80176]: pgmap v1305: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:09.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:09 smithi007 ceph-mon[108238]: pgmap v1305: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:10.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:10 smithi007 ceph-mon[108238]: pgmap v1306: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:10.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:10 smithi053 ceph-mon[80176]: pgmap v1306: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:13.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:13 smithi007 ceph-mon[108238]: pgmap v1307: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:13.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:13 smithi053 ceph-mon[80176]: pgmap v1307: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:14.843 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:14 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:18:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:14 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:18:15.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:15 smithi053 ceph-mon[80176]: pgmap v1308: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:15.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:15 smithi007 ceph-mon[108238]: pgmap v1308: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:16.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:16 smithi053 ceph-mon[80176]: pgmap v1309: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:16.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:16 smithi007 ceph-mon[108238]: pgmap v1309: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:19.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:19 smithi007 ceph-mon[108238]: pgmap v1310: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:19.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:19 smithi053 ceph-mon[80176]: pgmap v1310: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:21.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:21 smithi053 ceph-mon[80176]: pgmap v1311: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:21.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:21 smithi007 ceph-mon[108238]: pgmap v1311: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:22.875 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:22 smithi007 ceph-mon[108238]: pgmap v1312: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:22.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:22 smithi053 ceph-mon[80176]: pgmap v1312: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:25.897 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:25 smithi007 ceph-mon[108238]: pgmap v1313: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:25.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:25 smithi053 ceph-mon[80176]: pgmap v1313: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:27.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:27 smithi053 ceph-mon[80176]: pgmap v1314: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:27.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:27 smithi007 ceph-mon[108238]: pgmap v1314: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:28.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:28 smithi007 ceph-mon[108238]: pgmap v1315: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:28 smithi053 ceph-mon[80176]: pgmap v1315: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:29.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:18:29.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:18:30.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:30 smithi053 ceph-mon[80176]: pgmap v1316: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:30.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:30 smithi007 ceph-mon[108238]: pgmap v1316: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:33.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:33 smithi053 ceph-mon[80176]: pgmap v1317: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:33.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:33 smithi007 ceph-mon[108238]: pgmap v1317: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:34.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:34 smithi007 ceph-mon[108238]: pgmap v1318: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:34.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:34 smithi053 ceph-mon[80176]: pgmap v1318: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:37.775 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:37 smithi053 ceph-mon[80176]: pgmap v1319: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:37.864 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:37 smithi007 ceph-mon[108238]: pgmap v1319: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:38.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:38 smithi053 ceph-mon[80176]: pgmap v1320: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:38.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:38 smithi007 ceph-mon[108238]: pgmap v1320: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:41.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:41 smithi053 ceph-mon[80176]: pgmap v1321: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:41.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:41 smithi007 ceph-mon[108238]: pgmap v1321: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:42.867 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:42 smithi053 ceph-mon[80176]: pgmap v1322: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:42.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:42 smithi007 ceph-mon[108238]: pgmap v1322: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:43.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:18:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:18:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:18:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:18:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:18:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:18:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:44 smithi053 ceph-mon[80176]: pgmap v1323: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:18:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:18:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:18:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:18:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:44 smithi007 ceph-mon[108238]: pgmap v1323: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:46.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:46 smithi007 ceph-mon[108238]: pgmap v1324: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:46.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:46 smithi053 ceph-mon[80176]: pgmap v1324: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:48.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:48 smithi053 ceph-mon[80176]: pgmap v1325: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:48.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:48 smithi007 ceph-mon[108238]: pgmap v1325: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:50.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:50 smithi053 ceph-mon[80176]: pgmap v1326: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:50.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:50 smithi007 ceph-mon[108238]: pgmap v1326: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:52.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:52 smithi007 ceph-mon[108238]: pgmap v1327: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:52.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:52 smithi053 ceph-mon[80176]: pgmap v1327: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:54.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:54 smithi053 ceph-mon[80176]: pgmap v1328: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:54.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:54 smithi007 ceph-mon[108238]: pgmap v1328: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:18:56.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:56 smithi053 ceph-mon[80176]: pgmap v1329: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:56.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:56 smithi007 ceph-mon[108238]: pgmap v1329: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:18:58.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:58 smithi007 ceph-mon[108238]: pgmap v1330: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:58 smithi053 ceph-mon[80176]: pgmap v1330: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:18:59.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:18:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:18:59.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:18:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:19:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:00 smithi053 ceph-mon[80176]: pgmap v1331: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:00 smithi007 ceph-mon[108238]: pgmap v1331: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:02.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:02 smithi053 ceph-mon[80176]: pgmap v1332: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:02 smithi007 ceph-mon[108238]: pgmap v1332: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:04.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:04 smithi007 ceph-mon[108238]: pgmap v1333: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:04.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:04 smithi053 ceph-mon[80176]: pgmap v1333: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:06.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:06 smithi053 ceph-mon[80176]: pgmap v1334: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:06.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:06 smithi007 ceph-mon[108238]: pgmap v1334: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:08.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:08 smithi053 ceph-mon[80176]: pgmap v1335: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:08 smithi007 ceph-mon[108238]: pgmap v1335: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:10.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:10 smithi007 ceph-mon[108238]: pgmap v1336: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:10.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:10 smithi053 ceph-mon[80176]: pgmap v1336: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:12.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:12 smithi053 ceph-mon[80176]: pgmap v1337: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:12.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:12 smithi007 ceph-mon[108238]: pgmap v1337: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:13.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:13 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:19:13.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:13 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:19:14.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:14 smithi053 ceph-mon[80176]: pgmap v1338: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:14 smithi007 ceph-mon[108238]: pgmap v1338: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:16.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:16 smithi007 ceph-mon[108238]: pgmap v1339: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:16.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:16 smithi053 ceph-mon[80176]: pgmap v1339: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:18.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:18 smithi053 ceph-mon[80176]: pgmap v1340: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:18.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:18 smithi007 ceph-mon[108238]: pgmap v1340: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:20.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:20 smithi053 ceph-mon[80176]: pgmap v1341: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:20.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:20 smithi007 ceph-mon[108238]: pgmap v1341: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:22.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:22 smithi007 ceph-mon[108238]: pgmap v1342: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:22.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:22 smithi053 ceph-mon[80176]: pgmap v1342: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:24.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:24 smithi053 ceph-mon[80176]: pgmap v1343: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:24.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:24 smithi007 ceph-mon[108238]: pgmap v1343: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:26.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:26 smithi053 ceph-mon[80176]: pgmap v1344: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:26.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:26 smithi007 ceph-mon[108238]: pgmap v1344: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:28.832 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:28 smithi007 ceph-mon[108238]: pgmap v1345: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:28 smithi053 ceph-mon[80176]: pgmap v1345: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:29.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:29 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:19:29.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:29 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:19:30.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:30 smithi053 ceph-mon[80176]: pgmap v1346: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:30.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:30 smithi007 ceph-mon[108238]: pgmap v1346: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:32.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:32 smithi053 ceph-mon[80176]: pgmap v1347: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:32.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:32 smithi007 ceph-mon[108238]: pgmap v1347: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:34.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:34 smithi053 ceph-mon[80176]: pgmap v1348: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:34.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:34 smithi007 ceph-mon[108238]: pgmap v1348: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:36.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:36 smithi053 ceph-mon[80176]: pgmap v1349: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:36.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:36 smithi007 ceph-mon[108238]: pgmap v1349: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:38.881 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:38 smithi053 ceph-mon[80176]: pgmap v1350: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:38.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:38 smithi007 ceph-mon[108238]: pgmap v1350: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:40.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:40 smithi007 ceph-mon[108238]: pgmap v1351: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:40.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:40 smithi053 ceph-mon[80176]: pgmap v1351: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:42.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:42 smithi053 ceph-mon[80176]: pgmap v1352: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:42.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:42 smithi007 ceph-mon[108238]: pgmap v1352: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:43.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:19:43.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:19:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:19:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:19:44.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:19:44.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:19:44.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:19:44.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:44 smithi053 ceph-mon[80176]: pgmap v1353: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:44.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:19:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:19:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:19:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:44 smithi007 ceph-mon[108238]: pgmap v1353: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:46.928 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:46 smithi007 ceph-mon[108238]: pgmap v1354: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:46.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:46 smithi053 ceph-mon[80176]: pgmap v1354: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:48.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:48 smithi053 ceph-mon[80176]: pgmap v1355: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:48.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:48 smithi007 ceph-mon[108238]: pgmap v1355: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:50.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:50 smithi053 ceph-mon[80176]: pgmap v1356: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:50.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:50 smithi007 ceph-mon[108238]: pgmap v1356: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:52.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:52 smithi007 ceph-mon[108238]: pgmap v1357: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:52.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:52 smithi053 ceph-mon[80176]: pgmap v1357: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:54.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:54 smithi007 ceph-mon[108238]: pgmap v1358: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:54.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:54 smithi053 ceph-mon[80176]: pgmap v1358: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:19:56.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:56 smithi053 ceph-mon[80176]: pgmap v1359: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:56.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:56 smithi007 ceph-mon[108238]: pgmap v1359: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:19:58.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:58 smithi007 ceph-mon[108238]: pgmap v1360: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:58 smithi053 ceph-mon[80176]: pgmap v1360: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:19:59.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:19:59 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:19:59.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:19:59 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:20:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:00 smithi053 ceph-mon[80176]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T05:20:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:00 smithi053 ceph-mon[80176]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:20:00.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:00 smithi053 ceph-mon[80176]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:20:00.943 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:00 smithi053 ceph-mon[80176]: pgmap v1361: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:00 smithi007 ceph-mon[108238]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-21T05:20:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:00 smithi007 ceph-mon[108238]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-21T05:20:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:00 smithi007 ceph-mon[108238]: fs cephfs has deprecated feature inline_data enabled. 2024-09-21T05:20:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:00 smithi007 ceph-mon[108238]: pgmap v1361: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:02.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:02 smithi053 ceph-mon[80176]: pgmap v1362: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:02.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:02 smithi007 ceph-mon[108238]: pgmap v1362: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:04.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:04 smithi053 ceph-mon[80176]: pgmap v1363: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:04.932 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:04 smithi007 ceph-mon[108238]: pgmap v1363: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:06.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:06 smithi053 ceph-mon[80176]: pgmap v1364: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:06.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:06 smithi007 ceph-mon[108238]: pgmap v1364: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:08.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:08 smithi053 ceph-mon[80176]: pgmap v1365: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:08.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:08 smithi007 ceph-mon[108238]: pgmap v1365: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:10.936 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:10 smithi007 ceph-mon[108238]: pgmap v1366: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:10.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:10 smithi053 ceph-mon[80176]: pgmap v1366: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:12.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:12 smithi053 ceph-mon[80176]: pgmap v1367: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:12.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:12 smithi007 ceph-mon[108238]: pgmap v1367: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:13.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:13 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:20:13.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:13 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:20:14.843 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:14 smithi053 ceph-mon[80176]: pgmap v1368: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:14 smithi007 ceph-mon[108238]: pgmap v1368: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:16.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:16 smithi007 ceph-mon[108238]: pgmap v1369: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:16.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:16 smithi053 ceph-mon[80176]: pgmap v1369: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:18.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:18 smithi053 ceph-mon[80176]: pgmap v1370: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:18.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:18 smithi007 ceph-mon[108238]: pgmap v1370: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:20.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:20 smithi053 ceph-mon[80176]: pgmap v1371: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:20.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:20 smithi007 ceph-mon[108238]: pgmap v1371: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:22.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:22 smithi007 ceph-mon[108238]: pgmap v1372: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:22.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:22 smithi053 ceph-mon[80176]: pgmap v1372: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:24.879 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:24 smithi007 ceph-mon[108238]: pgmap v1373: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:24.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:24 smithi053 ceph-mon[80176]: pgmap v1373: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:26.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:26 smithi053 ceph-mon[80176]: pgmap v1374: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:26.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:26 smithi007 ceph-mon[108238]: pgmap v1374: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:28.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:28 smithi007 ceph-mon[108238]: pgmap v1375: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:28.933 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:20:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:28 smithi053 ceph-mon[80176]: pgmap v1375: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:20:30.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:30 smithi053 ceph-mon[80176]: pgmap v1376: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:30.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:30 smithi007 ceph-mon[108238]: pgmap v1376: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:32.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:32 smithi053 ceph-mon[80176]: pgmap v1377: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:32.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:32 smithi007 ceph-mon[108238]: pgmap v1377: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:35.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:35 smithi053 ceph-mon[80176]: pgmap v1378: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:35.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:35 smithi007 ceph-mon[108238]: pgmap v1378: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:36.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:36 smithi053 ceph-mon[80176]: pgmap v1379: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:36.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:36 smithi007 ceph-mon[108238]: pgmap v1379: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:38.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:38 smithi053 ceph-mon[80176]: pgmap v1380: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:38.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:38 smithi007 ceph-mon[108238]: pgmap v1380: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:40.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:40 smithi007 ceph-mon[108238]: pgmap v1381: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:40.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:40 smithi053 ceph-mon[80176]: pgmap v1381: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:42.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:42 smithi053 ceph-mon[80176]: pgmap v1382: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:42.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:42 smithi007 ceph-mon[108238]: pgmap v1382: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:43.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:43 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:20:43.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:43 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:20:44.843 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:20:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:44 smithi053 ceph-mon[80176]: pgmap v1383: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:20:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:20:44.844 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:44 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:20:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:20:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:44 smithi007 ceph-mon[108238]: pgmap v1383: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:20:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:20:44.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:44 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' 2024-09-21T05:20:46.928 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:46 smithi007 ceph-mon[108238]: pgmap v1384: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:46.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:46 smithi053 ceph-mon[80176]: pgmap v1384: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:48.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:48 smithi053 ceph-mon[80176]: pgmap v1385: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:48.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:48 smithi007 ceph-mon[108238]: pgmap v1385: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:50.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:50 smithi007 ceph-mon[108238]: pgmap v1386: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:50.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:50 smithi053 ceph-mon[80176]: pgmap v1386: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:52.928 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:52 smithi007 ceph-mon[108238]: pgmap v1387: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:52.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:52 smithi053 ceph-mon[80176]: pgmap v1387: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:54.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:54 smithi053 ceph-mon[80176]: pgmap v1388: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:54.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:54 smithi007 ceph-mon[108238]: pgmap v1388: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:20:56.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:56 smithi053 ceph-mon[80176]: pgmap v1389: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:56.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:56 smithi007 ceph-mon[108238]: pgmap v1389: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:20:58.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:58 smithi007 ceph-mon[108238]: pgmap v1390: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:58.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:20:58 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:20:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:58 smithi053 ceph-mon[80176]: pgmap v1390: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:20:58.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:20:58 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:21:00.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:00 smithi053 ceph-mon[80176]: pgmap v1391: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:00.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:00 smithi007 ceph-mon[108238]: pgmap v1391: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:02.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:02 smithi053 ceph-mon[80176]: pgmap v1392: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:02.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:02 smithi007 ceph-mon[108238]: pgmap v1392: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:04.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:04 smithi007 ceph-mon[108238]: pgmap v1393: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:04.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:04 smithi053 ceph-mon[80176]: pgmap v1393: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:06.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:06 smithi053 ceph-mon[80176]: pgmap v1394: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:06.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:06 smithi007 ceph-mon[108238]: pgmap v1394: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:08.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:08 smithi053 ceph-mon[80176]: pgmap v1395: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:21:08.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:08 smithi007 ceph-mon[108238]: pgmap v1395: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:21:10.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:10 smithi007 ceph-mon[108238]: pgmap v1396: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:10.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:10 smithi053 ceph-mon[80176]: pgmap v1396: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:12.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:12 smithi053 ceph-mon[80176]: pgmap v1397: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:12.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:12 smithi007 ceph-mon[108238]: pgmap v1397: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:13.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:13 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:21:13.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:13 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:21:14.843 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:14 smithi053 ceph-mon[80176]: pgmap v1398: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:14.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:14 smithi007 ceph-mon[108238]: pgmap v1398: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:16.935 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:16 smithi007 ceph-mon[108238]: pgmap v1399: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:16.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:16 smithi053 ceph-mon[80176]: pgmap v1399: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:18.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:18 smithi053 ceph-mon[80176]: pgmap v1400: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:21:18.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:18 smithi007 ceph-mon[108238]: pgmap v1400: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:21:20.927 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:20 smithi007 ceph-mon[108238]: pgmap v1401: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:20.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:20 smithi053 ceph-mon[80176]: pgmap v1401: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:22.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:22 smithi007 ceph-mon[108238]: pgmap v1402: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:22.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:22 smithi053 ceph-mon[80176]: pgmap v1402: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:24.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:24 smithi053 ceph-mon[80176]: pgmap v1403: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:24.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:24 smithi007 ceph-mon[108238]: pgmap v1403: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:26.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:26 smithi053 ceph-mon[80176]: pgmap v1404: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:26.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:26 smithi007 ceph-mon[108238]: pgmap v1404: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:28.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:28 smithi007 ceph-mon[108238]: pgmap v1405: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:21:28.941 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:28 smithi007 ceph-mon[108238]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:21:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:28 smithi053 ceph-mon[80176]: pgmap v1405: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-21T05:21:28.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:28 smithi053 ceph-mon[80176]: from='mgr.14772 172.21.15.7:0/1895919845' entity='mgr.smithi007.huohfl' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T05:21:30.906 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:30 smithi053 ceph-mon[80176]: pgmap v1406: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:30.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:30 smithi007 ceph-mon[108238]: pgmap v1406: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:32.942 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:32 smithi053 ceph-mon[80176]: pgmap v1407: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:32.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:32 smithi007 ceph-mon[108238]: pgmap v1407: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:35.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:35 smithi007 ceph-mon[108238]: pgmap v1408: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:35.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:35 smithi053 ceph-mon[80176]: pgmap v1408: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-21T05:21:36.348 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-09-21T05:21:36.349 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-21T05:21:36.351 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T05:21:36.364 INFO:tasks.cephadm:Teardown begin 2024-09-21T05:21:36.364 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/cephadm.py", line 2374, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-21T05:21:36.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T05:21:36.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T05:21:36.431 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T05:21:36.431 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T05:21:36.454 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T05:21:36.489 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T05:21:36.489 INFO:tasks.cephadm.mon.smithi007:Stopping mon.smithi007... 2024-09-21T05:21:36.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007 2024-09-21T05:21:36.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:36 smithi053 ceph-mon[80176]: pgmap v1409: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:36.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:36 smithi007 ceph-mon[108238]: pgmap v1409: 65 pgs: 65 active+clean; 275 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T05:21:36.816 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:36 smithi007 systemd[1]: Stopping Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T05:21:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:36 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T05:21:36.813+0000 7f54d2672640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi007 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:21:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:36 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007[108234]: 2024-09-21T05:21:36.813+0000 7f54d2672640 -1 mon.smithi007@0(leader) e2 *** Got Signal Terminated *** 2024-09-21T05:21:37.194 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:36 smithi007 podman[159748]: 2024-09-21 05:21:36.946660825 +0000 UTC m=+0.257121169 container died 1ac9cb9a20ec6c2bdf3e219bf56b8702e6cc90b48c9226b34a3286780fb8b239 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T05:21:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:37 smithi007 podman[159748]: 2024-09-21 05:21:37.421962919 +0000 UTC m=+0.732423262 container cleanup 1ac9cb9a20ec6c2bdf3e219bf56b8702e6cc90b48c9226b34a3286780fb8b239 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T05:21:37.694 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:37 smithi007 bash[159748]: ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007 2024-09-21T05:21:38.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service' 2024-09-21T05:21:38.035 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:37 smithi007 podman[159762]: 2024-09-21 05:21:37.782091247 +0000 UTC m=+0.834208825 container remove 1ac9cb9a20ec6c2bdf3e219bf56b8702e6cc90b48c9226b34a3286780fb8b239 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi007, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T05:21:38.035 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:38 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service: Deactivated successfully. 2024-09-21T05:21:38.035 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:38 smithi007 systemd[1]: Stopped Ceph mon.smithi007 for 70154924-77d1-11ef-baf5-efdc52797490. 2024-09-21T05:21:38.035 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Sep 21 05:21:38 smithi007 systemd[1]: ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi007.service: Consumed 27.420s CPU time. 2024-09-21T05:21:38.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T05:21:38.060 INFO:tasks.cephadm.mon.smithi007:Stopped mon.smithi007 2024-09-21T05:21:38.060 INFO:tasks.cephadm.mon.smithi053:Stopping mon.smithi053... 2024-09-21T05:21:38.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi053 2024-09-21T05:21:38.388 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:38 smithi053 systemd[1]: Stopping Ceph mon.smithi053 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T05:21:38.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:38 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053[80172]: 2024-09-21T05:21:38.384+0000 7fb4ec5bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi053 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:21:38.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:38 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053[80172]: 2024-09-21T05:21:38.384+0000 7fb4ec5bc640 -1 mon.smithi053@1(peon) e2 *** Got Signal Terminated *** 2024-09-21T05:21:38.692 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:38 smithi053 podman[114671]: 2024-09-21 05:21:38.630985313 +0000 UTC m=+0.359732271 container died fdf38e86da7b2e75065ab2df90bfc93419d828de2611519ecb98ddc48b067ff3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T05:21:39.442 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:39 smithi053 podman[114671]: 2024-09-21 05:21:39.202774529 +0000 UTC m=+0.931521484 container cleanup fdf38e86da7b2e75065ab2df90bfc93419d828de2611519ecb98ddc48b067ff3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T05:21:39.443 INFO:journalctl@ceph.mon.smithi053.smithi053.stdout:Sep 21 05:21:39 smithi053 bash[114671]: ceph-70154924-77d1-11ef-baf5-efdc52797490-mon-smithi053 2024-09-21T05:21:39.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@mon.smithi053.service' 2024-09-21T05:21:39.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T05:21:39.706 INFO:tasks.cephadm.mon.smithi053:Stopped mon.smithi053 2024-09-21T05:21:39.706 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T05:21:39.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.0 2024-09-21T05:21:40.046 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:39 smithi007 systemd[1]: Stopping Ceph osd.0 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T05:21:40.443 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:40 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[116742]: 2024-09-21T05:21:40.043+0000 7f7d489ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:21:40.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:40 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[116742]: 2024-09-21T05:21:40.043+0000 7f7d489ad640 -1 osd.0 80 *** Got signal Terminated *** 2024-09-21T05:21:40.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:40 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0[116742]: 2024-09-21T05:21:40.043+0000 7f7d489ad640 -1 osd.0 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T05:21:45.944 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:45 smithi007 podman[159881]: 2024-09-21 05:21:45.527142894 +0000 UTC m=+5.618434735 container died 9ec59c74f4bd9763e1616c0aa7ff5093c4feff2af2a651601b057ba7baaab39c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T05:21:46.306 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:46 smithi007 podman[159881]: 2024-09-21 05:21:46.052161077 +0000 UTC m=+6.143452913 container cleanup 9ec59c74f4bd9763e1616c0aa7ff5093c4feff2af2a651601b057ba7baaab39c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T05:21:46.306 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:46 smithi007 bash[159881]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0 2024-09-21T05:21:46.694 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:46 smithi007 podman[159897]: 2024-09-21 05:21:46.304135303 +0000 UTC m=+0.774319329 container remove 9ec59c74f4bd9763e1616c0aa7ff5093c4feff2af2a651601b057ba7baaab39c (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T05:21:47.027 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:46 smithi007 podman[159986]: 2024-09-21 05:21:46.652024109 +0000 UTC m=+0.017135988 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T05:21:47.027 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:46 smithi007 podman[159986]: 2024-09-21 05:21:46.771976072 +0000 UTC m=+0.137087939 container create d10c0c3f37ef552d0d971932cb849317192c81e77f9b4a04c1244f8cbf55051a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T05:21:47.027 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:47 smithi007 podman[159986]: 2024-09-21 05:21:47.025223965 +0000 UTC m=+0.390335846 container init d10c0c3f37ef552d0d971932cb849317192c81e77f9b4a04c1244f8cbf55051a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T05:21:47.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:47 smithi007 podman[159986]: 2024-09-21 05:21:47.029025257 +0000 UTC m=+0.394137124 container start d10c0c3f37ef552d0d971932cb849317192c81e77f9b4a04c1244f8cbf55051a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T05:21:47.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:47 smithi007 podman[159986]: 2024-09-21 05:21:47.133575378 +0000 UTC m=+0.498687257 container attach d10c0c3f37ef552d0d971932cb849317192c81e77f9b4a04c1244f8cbf55051a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T05:21:47.444 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 21 05:21:47 smithi007 podman[159986]: 2024-09-21 05:21:47.222971365 +0000 UTC m=+0.588083239 container died d10c0c3f37ef552d0d971932cb849317192c81e77f9b4a04c1244f8cbf55051a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T05:21:47.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T05:21:48.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T05:21:48.041 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T05:21:48.041 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T05:21:48.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.1 2024-09-21T05:21:48.369 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:48 smithi007 systemd[1]: Stopping Ceph osd.1 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T05:21:48.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:48 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[122504]: 2024-09-21T05:21:48.365+0000 7f7800f2f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:21:48.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:48 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[122504]: 2024-09-21T05:21:48.365+0000 7f7800f2f640 -1 osd.1 80 *** Got signal Terminated *** 2024-09-21T05:21:48.694 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:48 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1[122504]: 2024-09-21T05:21:48.365+0000 7f7800f2f640 -1 osd.1 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T05:21:54.161 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:53 smithi007 podman[160087]: 2024-09-21 05:21:53.796546583 +0000 UTC m=+5.562392493 container died 55f4289c127550ca7ca03d3ea36cdd3f9ac7f89254ac6e1d93ba1c4976182d48 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T05:21:54.428 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:54 smithi007 podman[160087]: 2024-09-21 05:21:54.241737032 +0000 UTC m=+6.007582951 container cleanup 55f4289c127550ca7ca03d3ea36cdd3f9ac7f89254ac6e1d93ba1c4976182d48 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T05:21:54.428 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:54 smithi007 bash[160087]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1 2024-09-21T05:21:54.693 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:54 smithi007 podman[160106]: 2024-09-21 05:21:54.426364108 +0000 UTC m=+0.619155424 container remove 55f4289c127550ca7ca03d3ea36cdd3f9ac7f89254ac6e1d93ba1c4976182d48 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T05:21:55.194 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:54 smithi007 podman[160195]: 2024-09-21 05:21:54.877509579 +0000 UTC m=+0.112422981 container create 532ae4a5351ecc81e60745c3f64516986d5059666e3e7459f7b13c030a323fc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T05:21:55.194 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:54 smithi007 podman[160195]: 2024-09-21 05:21:54.781787657 +0000 UTC m=+0.016701061 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T05:21:55.194 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:55 smithi007 podman[160195]: 2024-09-21 05:21:55.105154832 +0000 UTC m=+0.340068238 container init 532ae4a5351ecc81e60745c3f64516986d5059666e3e7459f7b13c030a323fc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T05:21:55.194 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:55 smithi007 podman[160195]: 2024-09-21 05:21:55.108817691 +0000 UTC m=+0.343731102 container start 532ae4a5351ecc81e60745c3f64516986d5059666e3e7459f7b13c030a323fc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T05:21:55.692 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:55 smithi007 podman[160195]: 2024-09-21 05:21:55.210832582 +0000 UTC m=+0.445746000 container attach 532ae4a5351ecc81e60745c3f64516986d5059666e3e7459f7b13c030a323fc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T05:21:55.692 INFO:journalctl@ceph.osd.1.smithi007.stdout:Sep 21 05:21:55 smithi007 podman[160195]: 2024-09-21 05:21:55.316245109 +0000 UTC m=+0.551158511 container died 532ae4a5351ecc81e60745c3f64516986d5059666e3e7459f7b13c030a323fc7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T05:21:55.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T05:21:56.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T05:21:56.031 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T05:21:56.031 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T05:21:56.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.2 2024-09-21T05:21:56.293 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:21:56 smithi007 systemd[1]: Stopping Ceph osd.2 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T05:21:56.546 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:21:56 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[127827]: 2024-09-21T05:21:56.404+0000 7ff702978640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:21:56.546 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:21:56 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[127827]: 2024-09-21T05:21:56.404+0000 7ff702978640 -1 osd.2 80 *** Got signal Terminated *** 2024-09-21T05:21:56.547 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:21:56 smithi007 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2[127827]: 2024-09-21T05:21:56.404+0000 7ff702978640 -1 osd.2 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T05:22:01.940 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:01 smithi007 podman[160301]: 2024-09-21 05:22:01.744456024 +0000 UTC m=+5.473110002 container died fb138708649cf2ea9b2a382e0eab986e3cbd9f5880c4ce543f29d3571274b789 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T05:22:02.193 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:02 smithi007 podman[160301]: 2024-09-21 05:22:02.105041206 +0000 UTC m=+5.833695180 container cleanup fb138708649cf2ea9b2a382e0eab986e3cbd9f5880c4ce543f29d3571274b789 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T05:22:02.194 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:02 smithi007 bash[160301]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2 2024-09-21T05:22:02.694 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:02 smithi007 podman[160327]: 2024-09-21 05:22:02.415107258 +0000 UTC m=+0.662854203 container remove fb138708649cf2ea9b2a382e0eab986e3cbd9f5880c4ce543f29d3571274b789 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T05:22:03.012 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:02 smithi007 podman[160419]: 2024-09-21 05:22:02.647355572 +0000 UTC m=+0.016768688 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T05:22:03.012 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:02 smithi007 podman[160419]: 2024-09-21 05:22:02.782810091 +0000 UTC m=+0.152223198 container create 62079dd82da397677a7c32be2ce58897c6084528c52621cb49e561b9d0d2117a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T05:22:03.012 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:03 smithi007 podman[160419]: 2024-09-21 05:22:03.010512089 +0000 UTC m=+0.379925196 container init 62079dd82da397677a7c32be2ce58897c6084528c52621cb49e561b9d0d2117a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T05:22:03.444 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:03 smithi007 podman[160419]: 2024-09-21 05:22:03.014061581 +0000 UTC m=+0.383474694 container start 62079dd82da397677a7c32be2ce58897c6084528c52621cb49e561b9d0d2117a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T05:22:03.444 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:03 smithi007 podman[160419]: 2024-09-21 05:22:03.102032546 +0000 UTC m=+0.471445665 container attach 62079dd82da397677a7c32be2ce58897c6084528c52621cb49e561b9d0d2117a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T05:22:03.444 INFO:journalctl@ceph.osd.2.smithi007.stdout:Sep 21 05:22:03 smithi007 podman[160419]: 2024-09-21 05:22:03.216153025 +0000 UTC m=+0.585566132 container died 62079dd82da397677a7c32be2ce58897c6084528c52621cb49e561b9d0d2117a (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T05:22:03.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T05:22:03.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T05:22:03.916 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T05:22:03.917 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T05:22:03.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.3 2024-09-21T05:22:04.242 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:03 smithi053 systemd[1]: Stopping Ceph osd.3 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T05:22:04.692 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:04 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T05:22:04.240+0000 7f533140c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:22:04.692 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:04 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T05:22:04.240+0000 7f533140c640 -1 osd.3 80 *** Got signal Terminated *** 2024-09-21T05:22:04.692 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:04 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T05:22:04.240+0000 7f533140c640 -1 osd.3 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T05:22:07.442 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:07 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T05:22:07.156+0000 7f532da21640 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:41.237653+0000 front 2024-09-21T05:21:41.237517+0000 (oldest deadline 2024-09-21T05:22:07.137202+0000) 2024-09-21T05:22:07.442 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:06 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:06.996+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:07.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:06 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:06.958+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:08.192 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:08 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T05:22:08.174+0000 7f532da21640 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:41.237653+0000 front 2024-09-21T05:21:41.237517+0000 (oldest deadline 2024-09-21T05:22:07.137202+0000) 2024-09-21T05:22:08.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:08 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:08.033+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:08.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:07 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:07.934+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:09.442 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:09 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3[88571]: 2024-09-21T05:22:09.199+0000 7f532da21640 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:41.237653+0000 front 2024-09-21T05:21:41.237517+0000 (oldest deadline 2024-09-21T05:22:07.137202+0000) 2024-09-21T05:22:09.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:08 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:08.962+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:09.443 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:09 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:09.022+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:10.035 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:09 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:09.978+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:10.035 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:09 smithi053 podman[114809]: 2024-09-21 05:22:09.763255821 +0000 UTC m=+5.644655413 container died 1a3233caf51a33faa85403126b8d03f758a6782aa5f4d5aa4c8014df2cfd2485 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:22:10.036 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:09 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:09.919+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:10.427 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:10 smithi053 podman[114809]: 2024-09-21 05:22:10.13224344 +0000 UTC m=+6.013643017 container cleanup 1a3233caf51a33faa85403126b8d03f758a6782aa5f4d5aa4c8014df2cfd2485 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T05:22:10.427 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:10 smithi053 bash[114809]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3 2024-09-21T05:22:10.692 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:10 smithi053 podman[114824]: 2024-09-21 05:22:10.425749377 +0000 UTC m=+0.659954158 container remove 1a3233caf51a33faa85403126b8d03f758a6782aa5f4d5aa4c8014df2cfd2485 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:22:11.023 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:10 smithi053 podman[114912]: 2024-09-21 05:22:10.667390946 +0000 UTC m=+0.016489354 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T05:22:11.023 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:10 smithi053 podman[114912]: 2024-09-21 05:22:10.801980608 +0000 UTC m=+0.151079007 container create 17c73d49e789c63630ac8fad934884641d5540292364dcb81b600e544af0ac90 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:22:11.023 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:11 smithi053 podman[114912]: 2024-09-21 05:22:11.021313129 +0000 UTC m=+0.370411538 container init 17c73d49e789c63630ac8fad934884641d5540292364dcb81b600e544af0ac90 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-21T05:22:11.023 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:10 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:10.882+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:11.023 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:10 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:10.937+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:11.308 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:11 smithi053 podman[114912]: 2024-09-21 05:22:11.025038114 +0000 UTC m=+0.374136518 container start 17c73d49e789c63630ac8fad934884641d5540292364dcb81b600e544af0ac90 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T05:22:11.308 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:11 smithi053 podman[114912]: 2024-09-21 05:22:11.10454361 +0000 UTC m=+0.453642036 container attach 17c73d49e789c63630ac8fad934884641d5540292364dcb81b600e544af0ac90 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:22:11.308 INFO:journalctl@ceph.osd.3.smithi053.stdout:Sep 21 05:22:11 smithi053 podman[114912]: 2024-09-21 05:22:11.220191182 +0000 UTC m=+0.569289596 container died 17c73d49e789c63630ac8fad934884641d5540292364dcb81b600e544af0ac90 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T05:22:11.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T05:22:11.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T05:22:11.945 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T05:22:11.945 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T05:22:11.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.4 2024-09-21T05:22:12.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:11 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:11.865+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:12.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:12 smithi053 systemd[1]: Stopping Ceph osd.4 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T05:22:12.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:11 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:11.982+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:12.692 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:12 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:12.310+0000 7ffa7c336640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:22:12.692 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:12 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:12.310+0000 7ffa7c336640 -1 osd.4 80 *** Got signal Terminated *** 2024-09-21T05:22:12.692 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:12 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:12.310+0000 7ffa7c336640 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T05:22:13.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:12 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:12.878+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:13.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:12 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:12.949+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:14.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:13 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:13.895+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:14.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:13 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:13.895+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:47.961043+0000 front 2024-09-21T05:21:47.961126+0000 (oldest deadline 2024-09-21T05:22:13.861080+0000) 2024-09-21T05:22:14.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:13 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:13.904+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:14.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:13 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:13.904+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:15.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:14 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:14.938+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:15.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:14 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:14.938+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:47.961043+0000 front 2024-09-21T05:21:47.961126+0000 (oldest deadline 2024-09-21T05:22:13.861080+0000) 2024-09-21T05:22:15.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:14 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:14.883+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:15.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:14 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:14.883+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:16.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:15 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:15.928+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:16.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:15 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:15.928+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:47.961043+0000 front 2024-09-21T05:21:47.961126+0000 (oldest deadline 2024-09-21T05:22:13.861080+0000) 2024-09-21T05:22:16.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:15 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:15.921+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:16.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:15 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:15.921+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:17.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:16.940+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:43.960659+0000 front 2024-09-21T05:21:43.960609+0000 (oldest deadline 2024-09-21T05:22:06.860290+0000) 2024-09-21T05:22:17.192 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4[93771]: 2024-09-21T05:22:16.940+0000 7ffa7894b640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:47.961043+0000 front 2024-09-21T05:21:47.961126+0000 (oldest deadline 2024-09-21T05:22:13.861080+0000) 2024-09-21T05:22:17.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:16.909+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:17.193 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:16 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:16.909+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:17.998 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:17 smithi053 podman[115023]: 2024-09-21 05:22:17.715828209 +0000 UTC m=+5.518411423 container died 632b5d2947ba1362876d08888b629ec9c9358a4f4bdcfde86669c36b496e4ed7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T05:22:17.999 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:17 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:17.863+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:17.999 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:17 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:17.863+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:18.440 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:18 smithi053 podman[115023]: 2024-09-21 05:22:18.212227083 +0000 UTC m=+6.014810295 container cleanup 632b5d2947ba1362876d08888b629ec9c9358a4f4bdcfde86669c36b496e4ed7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:22:18.440 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:18 smithi053 bash[115023]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4 2024-09-21T05:22:18.692 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:18 smithi053 podman[115039]: 2024-09-21 05:22:18.438369017 +0000 UTC m=+0.720229788 container remove 632b5d2947ba1362876d08888b629ec9c9358a4f4bdcfde86669c36b496e4ed7 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T05:22:19.035 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:18 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:18.860+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:19.035 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:18 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:18.860+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:19.035 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:18 smithi053 podman[115130]: 2024-09-21 05:22:18.66818817 +0000 UTC m=+0.016242028 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T05:22:19.036 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:18 smithi053 podman[115130]: 2024-09-21 05:22:18.797651838 +0000 UTC m=+0.145705693 container create 8e7193a454d0e6d81b119bfd1e0597112c55947832d23cc9886121bdbbb92e73 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T05:22:19.036 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:19 smithi053 podman[115130]: 2024-09-21 05:22:19.033856455 +0000 UTC m=+0.381910312 container init 8e7193a454d0e6d81b119bfd1e0597112c55947832d23cc9886121bdbbb92e73 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T05:22:19.442 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:19 smithi053 podman[115130]: 2024-09-21 05:22:19.037429467 +0000 UTC m=+0.385483314 container start 8e7193a454d0e6d81b119bfd1e0597112c55947832d23cc9886121bdbbb92e73 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-21T05:22:19.442 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:19 smithi053 podman[115130]: 2024-09-21 05:22:19.116982595 +0000 UTC m=+0.465036443 container attach 8e7193a454d0e6d81b119bfd1e0597112c55947832d23cc9886121bdbbb92e73 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T05:22:19.443 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 21 05:22:19 smithi053 podman[115130]: 2024-09-21 05:22:19.230014263 +0000 UTC m=+0.578068117 container died 8e7193a454d0e6d81b119bfd1e0597112c55947832d23cc9886121bdbbb92e73 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T05:22:19.942 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:19 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:19.854+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:19.943 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:19 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:19.854+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:19.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T05:22:20.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T05:22:20.029 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T05:22:20.030 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T05:22:20.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.5 2024-09-21T05:22:20.327 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:20 smithi053 systemd[1]: Stopping Ceph osd.5 for 70154924-77d1-11ef-baf5-efdc52797490... 2024-09-21T05:22:20.692 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:20 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:20.323+0000 7f7919db2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:22:20.692 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:20 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:20.323+0000 7f7919db2640 -1 osd.5 80 *** Got signal Terminated *** 2024-09-21T05:22:20.692 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:20 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:20.323+0000 7f7919db2640 -1 osd.5 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T05:22:21.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:20 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:20.884+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:21.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:20 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:20.884+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:22.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:21 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:21.894+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:22.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:21 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:21.894+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:23.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:22 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:22.861+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:23.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:22 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:22.861+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:24.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:23 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:23.823+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:24.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:23 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:23.823+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:25.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:24 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:24.793+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-21T05:21:42.818373+0000 front 2024-09-21T05:21:42.818346+0000 (oldest deadline 2024-09-21T05:22:06.218658+0000) 2024-09-21T05:22:25.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:24 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:24.793+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-09-21T05:21:48.519257+0000 front 2024-09-21T05:21:48.519419+0000 (oldest deadline 2024-09-21T05:22:13.819283+0000) 2024-09-21T05:22:25.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:24 smithi053 ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5[98958]: 2024-09-21T05:22:24.793+0000 7f79123bf640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.7:6822 osd.2 since back 2024-09-21T05:21:59.720169+0000 front 2024-09-21T05:21:59.720199+0000 (oldest deadline 2024-09-21T05:22:24.419970+0000) 2024-09-21T05:22:25.944 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:25 smithi053 podman[115233]: 2024-09-21 05:22:25.691444121 +0000 UTC m=+5.473072701 container died d8e874d488eb9be29e0641ad862e68030549a56efaf4329cd1b5d323e13ca5a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T05:22:26.243 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:26 smithi053 podman[115233]: 2024-09-21 05:22:26.0408727 +0000 UTC m=+5.822501278 container cleanup d8e874d488eb9be29e0641ad862e68030549a56efaf4329cd1b5d323e13ca5a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T05:22:26.244 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:26 smithi053 bash[115233]: ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5 2024-09-21T05:22:26.636 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:26 smithi053 podman[115248]: 2024-09-21 05:22:26.242104419 +0000 UTC m=+0.542563095 container remove d8e874d488eb9be29e0641ad862e68030549a56efaf4329cd1b5d323e13ca5a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T05:22:26.888 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:26 smithi053 podman[115336]: 2024-09-21 05:22:26.634562877 +0000 UTC m=+0.086474243 container create 31771eb098b6780f9930110985c0f997777e310a7754691d63680c403fbb955b (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:22:26.888 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:26 smithi053 podman[115336]: 2024-09-21 05:22:26.564139201 +0000 UTC m=+0.016050542 image pull 48ed0a2cf89d6d0371a2c6185a5869a747265f6abd5ac2a74f00cc32d5628721 quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00 2024-09-21T05:22:26.888 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:26 smithi053 podman[115336]: 2024-09-21 05:22:26.887039171 +0000 UTC m=+0.338950511 container init 31771eb098b6780f9930110985c0f997777e310a7754691d63680c403fbb955b (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T05:22:27.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:26 smithi053 podman[115336]: 2024-09-21 05:22:26.89050823 +0000 UTC m=+0.342419558 container start 31771eb098b6780f9930110985c0f997777e310a7754691d63680c403fbb955b (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T05:22:27.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:26 smithi053 podman[115336]: 2024-09-21 05:22:26.961983159 +0000 UTC m=+0.413894499 container attach 31771eb098b6780f9930110985c0f997777e310a7754691d63680c403fbb955b (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-20-2310-f36283e) 2024-09-21T05:22:27.192 INFO:journalctl@ceph.osd.5.smithi053.stdout:Sep 21 05:22:27 smithi053 podman[115336]: 2024-09-21 05:22:27.082434915 +0000 UTC m=+0.534346240 container died 31771eb098b6780f9930110985c0f997777e310a7754691d63680c403fbb955b (image=quay.ceph.io/ceph-ci/ceph@sha256:2195e5f178be0a9390728e2a14088236d8fe6f82635f5c84418fe37a04457e00, name=ceph-70154924-77d1-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-guits-main-2024-09-20-2310-f36283e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T05:22:27.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70154924-77d1-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T05:22:27.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T05:22:27.743 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T05:22:27.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 70154924-77d1-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T05:22:27.949 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T05:22:32.517 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[75710]: fuse finished with error 0 and tester_r 0 2024-09-21T05:22:50.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 70154924-77d1-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T05:22:51.109 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T05:23:04.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T05:23:05.032 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T05:23:05.060 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T05:23:05.062 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701/remote/smithi007/crash 2024-09-21T05:23:05.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash -- . 2024-09-21T05:23:05.099 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T05:23:05.099 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-09-21T05:23:05.101 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701/remote/smithi053/crash 2024-09-21T05:23:05.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash -- . 2024-09-21T05:23:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/70154924-77d1-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T05:23:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-09-21T05:23:05.127 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T05:23:05.127 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T05:23:05.179 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-21T04:50:00.000193+0000 mon.smithi007 (mon.0) 570 : cluster [ERR] fs cephfs is offline because no MDS is active for it. 2024-09-21T05:23:05.179 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-21T05:23:05.180 INFO:tasks.cephadm:Compressing logs... 2024-09-21T05:23:05.180 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T05:23:05.222 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T05:23:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T05:23:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T05:23:05.247 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mon.smithi007.log 2024-09-21T05:23:05.247 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T05:23:05.247 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mon.smithi007.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi007.oxcnqk.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi007.oxcnqk.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.2.log: 86.4% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi007.eyygpg.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi053.jvuwme.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:: No such file or directory 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mgr.smithi053.lnadao.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi053.jvuwme.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi053.wdnfat.log 2024-09-21T05:23:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mgr.smithi053.lnadao.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T05:23:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi053.wdnfat.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi053.log 2024-09-21T05:23:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T05:23:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi053.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T05:23:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mon.smithi053.log 2024-09-21T05:23:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T05:23:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mon.smithi053.log: 94.5% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi053.log.gz 2024-09-21T05:23:05.255 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.log: 91.3% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T05:23:05.256 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi007.log 2024-09-21T05:23:05.256 INFO:teuthology.orchestra.run.smithi053.stderr: 91.4% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T05:23:05.257 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi007.eyygpg.log: 89.5% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T05:23:05.257 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mgr.smithi007.huohfl.log 2024-09-21T05:23:05.263 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi007.log: 94.4% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-client.ceph-exporter.smithi007.log.gz 2024-09-21T05:23:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-21T05:23:05.268 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T05:23:05.268 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.log 2024-09-21T05:23:05.268 INFO:teuthology.orchestra.run.smithi053.stderr: 85.0% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.cephadm.log.gz/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.5.log: 2024-09-21T05:23:05.269 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T05:23:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mgr.smithi007.huohfl.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T05:23:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.log: 89.6% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T05:23:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-09-21T05:23:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.3.log: 91.3% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T05:23:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.log: 91.3% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T05:23:05.289 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T05:23:05.304 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.1.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T05:23:05.310 INFO:teuthology.orchestra.run.smithi053.stderr: 90.0% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mgr.smithi053.lnadao.log.gz 2024-09-21T05:23:05.348 INFO:teuthology.orchestra.run.smithi053.stderr: 94.6% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi053.jvuwme.log.gz 2024-09-21T05:23:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: 91.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T05:23:05.882 INFO:teuthology.orchestra.run.smithi053.stderr: 92.5% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mon.smithi053.log.gz 2024-09-21T05:23:05.916 INFO:teuthology.orchestra.run.smithi007.stderr: 89.9% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mgr.smithi007.huohfl.log.gz 2024-09-21T05:23:07.080 INFO:teuthology.orchestra.run.smithi007.stderr: 89.8% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mon.smithi007.log.gz 2024-09-21T05:23:10.342 INFO:teuthology.orchestra.run.smithi007.stderr: 93.6% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T05:23:11.415 INFO:teuthology.orchestra.run.smithi007.stderr: 93.7% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T05:23:11.596 INFO:teuthology.orchestra.run.smithi053.stderr: 93.7% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T05:23:11.815 INFO:teuthology.orchestra.run.smithi053.stderr: 93.7% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T05:23:11.948 INFO:teuthology.orchestra.run.smithi053.stderr: 94.9% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi053.wdnfat.log.gz 2024-09-21T05:23:12.177 INFO:teuthology.orchestra.run.smithi007.stderr: 93.7% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T05:23:12.682 INFO:teuthology.orchestra.run.smithi053.stderr: 93.8% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T05:23:19.965 INFO:teuthology.orchestra.run.smithi053.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-09-21T05:23:19.966 INFO:teuthology.orchestra.run.smithi053.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-09-21T05:23:19.967 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-21T05:23:19.967 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m14.730s 2024-09-21T05:23:19.967 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m41.292s 2024-09-21T05:23:19.967 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m1.166s 2024-09-21T05:23:21.020 INFO:teuthology.orchestra.run.smithi007.stderr: 94.9% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi007.oxcnqk.log.gz 2024-09-21T05:23:22.658 INFO:teuthology.orchestra.run.smithi007.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-21T05:24:49.721 INFO:teuthology.orchestra.run.smithi007.stderr: 92.8% -- replaced with /var/log/ceph/70154924-77d1-11ef-baf5-efdc52797490/ceph-mds.cephfs.smithi007.eyygpg.log.gz 2024-09-21T05:24:49.723 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-21T05:24:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:real 1m44.487s 2024-09-21T05:24:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:user 2m33.543s 2024-09-21T05:24:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m4.494s 2024-09-21T05:24:49.723 INFO:tasks.cephadm:Archiving logs... 2024-09-21T05:24:49.723 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701/remote/smithi007/log 2024-09-21T05:24:49.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T05:25:09.520 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701/remote/smithi053/log 2024-09-21T05:25:09.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T05:25:14.471 INFO:tasks.cephadm:Removing cluster... 2024-09-21T05:25:14.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 70154924-77d1-11ef-baf5-efdc52797490 --force 2024-09-21T05:25:14.683 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T05:25:15.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 70154924-77d1-11ef-baf5-efdc52797490 --force 2024-09-21T05:25:15.324 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 70154924-77d1-11ef-baf5-efdc52797490 2024-09-21T05:25:15.620 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T05:25:15.621 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T05:25:15.638 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T05:25:15.654 INFO:tasks.cephadm:Teardown complete 2024-09-21T05:25:15.654 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T05:25:15.665 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-21T05:25:15.666 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T05:25:15.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T05:25:15.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T05:25:15.763 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T05:25:15.784 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T05:25:15.785 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:25:15.787 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:25:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:25:15.801 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:25:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:25:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-21T05:25:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 8 377 198 -442us[ -475us] +/- 66ms 2024-09-21T05:25:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 +1286us[+1286us] +/- 65ms 2024-09-21T05:25:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -743us[ -743us] +/- 69ms 2024-09-21T05:25:15.805 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:25:15.806 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:25:15.806 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-21T05:25:15.806 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 7 377 70 -462us[ -529us] +/- 36ms 2024-09-21T05:25:15.806 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 +1311us[+1311us] +/- 64ms 2024-09-21T05:25:15.806 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 7 -709us[ -709us] +/- 69ms 2024-09-21T05:25:15.806 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:25:15.806 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T05:25:15.862 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T05:25:15.862 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T05:25:15.968 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T05:25:16.054 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T05:25:16.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:25:16.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:25:16.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:25:16.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:25:16.189 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T05:25:16.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T05:25:16.257 INFO:teuthology.task.internal:Duration was 3990.258376 seconds 2024-09-21T05:25:16.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T05:25:16.309 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T05:25:16.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T05:25:16.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T05:25:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:25:16.353 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:25:16.741 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T05:25:16.741 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-21T05:25:16.742 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T05:25:16.805 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-09-21T05:25:16.805 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T05:25:16.841 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T05:25:16.841 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T05:25:16.848 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T05:25:16.909 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T05:25:16.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T05:25:16.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T05:25:18.150 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T05:25:18.162 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T05:25:18.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T05:25:18.187 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T05:25:18.218 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T05:25:18.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T05:25:18.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T05:25:18.319 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-21T05:25:18.325 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-09-21T05:25:18.381 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T05:25:18.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:25:18.410 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T05:25:18.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:25:18.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T05:25:18.467 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T05:25:18.468 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701/remote/smithi007 2024-09-21T05:25:18.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T05:25:18.526 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-21_03:44:43-orch:cephadm-wip-guits-main-2024-09-20-2310-distro-default-smithi/7913701/remote/smithi053 2024-09-21T05:25:18.526 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T05:25:18.569 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T05:25:18.569 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T05:25:18.571 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T05:25:18.621 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T05:25:18.642 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T05:25:18.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T05:25:18.794 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T05:25:18.795 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T05:25:18.798 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T05:25:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: 262169 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 21 05:25 /home/ubuntu/cephtest 2024-09-21T05:25:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: 549196 4 d--------- 2 ubuntu ubuntu 4096 Sep 21 04:32 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:25:18.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:25:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-09-21T05:25:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-09-21T05:25:18.813 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi007 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-09-21T05:25:18.813 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T05:25:18.877 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T05:25:18.996 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-21T05:25:18.999 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3990.258375644684 failure_reason: reached maximum tries (50) after waiting for 300 seconds flavor: default owner: scheduled_gabrioux@teuthology status: fail success: false 2024-09-21T05:25:18.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T05:25:19.080 INFO:teuthology.run:FAIL